A serverless, data-driven Git metrics dashboard using Amazon Quick Sight

The AWS AI-Driven Development Lifecycle (AI-DLC) emphasizes the importance of measurable data when incorporating AI coding tools. To support this, a serverless and data-driven Git metrics dashboard has been developed using Amazon QuickSight. This solution automates Git metrics collection from GitHub and GitLab, delivering live analytics through an interactive dashboard.

At its core, the solution utilizes an automated, event-driven pipeline that gathers repository metrics on a scheduled basis. It then processes the data through a serverless orchestration workflow and stores the results in Amazon S3 for visualization with QuickSight dashboards. Notably, the system is designed to scale effortlessly and keep costs low, abstracting the need for infrastructure management.

The workflow begins with Amazon EventBridge Scheduler, which triggers the workflow according to the user-defined interval. Once started, the AWS Step Functions state machine orchestrates the collection process. It first checks for any changes since the last collection using a dedicated change detector. If no changes are found, it skips the processing phase altogether.

For organizations managing over 20 repositories, the solution automatically segments the workload into parallel chunks, processing each concurrently with AWS Lambda invocations. This chunking approach ensures efficient use of resources, especially when dealing with a large number of repositories. In contrast, smaller teams benefit from a single direct invocation that collects all repositories.

Once the data collection is complete, metrics are securely stored in Amazon S3, leveraging versioning and server-side encryption for data protection. The output comprises structured JSON files containing comprehensive repository details and a flattened CSV file optimized for analytics. With access to all historical metrics, teams can effectively measure development activity and track improvements over time.