Reducing Latency Between Driving Behaviour and Risk Assessment: A Kafka and Spark Streaming Architecture for Usage-Based Insurance

Citation: Mohammed Ataullah Baig Mirza, "Reducing Latency Between Driving Behaviour and Risk Assessment: A Kafka and Spark Streaming Architecture for Usage-Based Insurance", Universal Library of Innovative Research and Studies, Volume 03, Issue 04.

Copyright: This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

In usage-based insurance programmes, the value of telematics depends on how quickly recorded driving behaviour is transformed into validated risk assessment. The continuous arrival of telematics events conflicts with batch processing because waiting for the next processing cycle separates the time of the trip from the time when its characteristics become available for analytics. This study proposes treating latency as a property of the entire processing pipeline, including event transmission, ingestion, queue waiting, transformation, data quality validation, and risk score generation. On this basis, the LRV-UBI model is developed, integrating data streaming, operational reliability, and the validity of behavioural features. Kafka and Spark Streaming constitute the technological core of the model; however, the reduction in processing time depends on the coordinated configuration of partitioning, consumer performance, computational parallelism, and state recovery mechanisms. Particular importance is assigned to handling late-arriving, duplicate, and invalid events that may alter previously computed features. Risk assessment is released only after verifying data completeness and consistency; therefore, validation time is included in end-to-end latency. A measurable indicator is defined for each processing stage, making it possible to localize the source of delay and distinguish computational acceleration from the actual availability of results. The LRV-UBI model is intended for designing verifiable streaming architectures for insurance telematics, in which processing speed is evaluated together with recovery resilience and the suitability of data for risk models. The model provides a foundation for future architectural comparisons using standardized indicators without directly transferring incomparable latency and throughput values from other application domains into the insurance context.

Keywords: Insurance Telematics, Driving Behaviour, Insurance Risk Assessment, Stream Processing, Latency Reduction, Data Quality, Processing Reliability.

https://doi.org/10.70315/uloap.ulirs.2026.0304002