How Jumio built a real-time feature store on AWS

Jumio, an identity verification provider, needed a real-time feature store to address challenges with data duplication, feature engineering, consistency, manual deployment, and latency. Their machine learning models required real-time features for fraud detection, which demanded immediate access to features, including upstream model outputs, with latency under 100 milliseconds.

To build their solution, Jumio used AWS services such as Amazon SageMaker Feature Store, Amazon Managed Service for Apache Flink, and Amazon Kinesis Data Streams. They adopted a streaming-first architecture with two feature stores: a real-time feature store built on Amazon SageMaker Feature Store and an offline feature store built on Amazon S3 with data stored in Apache Iceberg format.

Data flowed into the system through Amazon Kinesis Data Streams, where Apache Flink applications picked up incoming events and processed them to generate features. These features were then written directly to the real-time feature store. For offline features, the same events flowed through Amazon Data Firehose into Amazon S3, where they were processed further and stored in Iceberg tables.

This dual architecture provided the flexibility to handle both real-time and batch processing workloads while ensuring low latency for critical fraud detection use cases.