🩺 Building RadScan AI: Autonomous Multimodal Radiology Triage on GCP Cloud Run & Vertex AI

Radiology professionals experience significant burnout due to the laborious process of examining volumetric MRI scans. Each scan contains multiple high-resolution digital imaging and communications in medicine (DICOM) slices, requiring radiologists to manually inspect each plane, verify ligament continuity throughout the image stack, and dictate clinical reports.

This time-consuming workflow can take up to 10-15 minutes per study. To alleviate this bottleneck, developers have created RadScan AI, an autonomous radiology triage system that leverages Google Cloud Platform's resources to automate key aspects of the radiology workflow.

RadScan AI employs two serverless microservices on Google Cloud Platform to handle various tasks. The client-side interface, built with Next.js 14, React 18, and Tailwind CSS, provides users with an interactive DICOM viewer, a multi-planar slice slider, Grad-CAM heatmap opacity controls, and an interactive 6-step guided tour. The backend infrastructure consists of a FastAPI microservice deployed on GCP Cloud Run using NVIDIA L4 GPU acceleration, enabling sub-second inference and scaling down to zero instances when idle, thus minimizing costs.

The core of RadScan AI's functionality lies in its 2.5D volumetric neural engine, which processes 24 parallel depth slices across Sagittal, Coronal, and Axial planes simultaneously. The engine utilizes a depth-wise bidirectional Gated Recurrent Unit (BiGRU) temporal slice attention mechanism to track ligament continuity across consecutive depth slices.

This involves calculating forward hidden states, backward hidden states, and combined depth features. Additionally, Grad-CAM visual explainability heatmaps are generated to pinpoint lesion coordinates within the DICOM images, providing radiologists with valuable contextual information to support their decision-making process.

To integrate machine learning capabilities, RadScan AI employs Vertex AI Gemini 3.5 Flash as its LLM engine. This powerful language model is accessed via the google-cloud-aiplatform and google-genai SDKs in native JSON mode, facilitating seamless communication between the backend microservice and the LLM. The system architecture effectively decouples the frontend and backend components, promoting modularity and maintainability while ensuring a seamless user experience.

By combining cutting-edge computer vision techniques with structured LLM agents, RadScan AI demonstrates the potential for production-grade medical automation, paving the way for more efficient and sustainable radiology workflows.