From a 15-Second Walk to a Digital Memory: Building a Canine Cognitive Twin

This submission introduces "Canine Cognitive Twin," a digital twin for a dog that creates a living, digital record of the dog's life based on a 15-second walk captured with a phone. The project utilizes several technologies, including Google's Gemini for video analysis, Snowflake for data processing and anomaly detection, ElevenLabs for text-to-speech narration, and Solana for immutable record-keeping.

The pipeline begins with an Expo mobile app capturing a 15-second video and audio of the dog's walk. This data is then sent to a Cloudflare Worker, which stores the media on R2 storage and generates metadata using Gemini 3 Pro. The analysis extracts behavior tags, gait asymmetry, emotion, and narration, which are then converted into speech by ElevenLabs, giving the dog's digital twin a voice.

Snowflake's Dynamic Tables build rolling summaries and baselines, while Cortex detects potential gait drift from historical data. Solana records each chapter of the dog's memoir on its own keys, providing an immutable digital record. The dashboard built with Streamlit displays the memoir and health radar, allowing owners to track their dog's progress over time.

The project intentionally includes synthetic data for demonstration purposes, clearly marking the distinction between synthetic and live data. The architecture is designed to separate blockchain operations from the Gemini and text-to-speech pipeline, ensuring smooth operation even under potential API failures. The goal is to create a comprehensive and user-friendly experience for dog owners, providing them with insights into their pet's daily life and health.