Toyota shows off SoC that stores model weights and its OS on integrated memory

Japanese automobile manufacturer Toyota is venturing into the AI processor market. Toyota is a major stakeholder in AISIN, a company that produces automotive components. Both firms recognize that modern cars heavily rely on technologies like image recognition, and that robotics and "physical AI" will require parts that intersect with cars.

AISIN believes that current chips are not suitable for these roles, as they require external memory for AI models, resulting in significant data traffic between the processor and the model. This data traffic hinders the effectiveness of image recognition and other on-device AI applications. DRAM is currently prohibitively expensive.

To address this, AISIN developed a system-on-chip (SoC) built on a 16nm FinFET process. The SoC features an Arm Cortex A53 dual-core CPU and an Arm MaliG31 GPU. Crucially, it incorporates STT-MRAM, which stores both AI model weights and the operating system. AISIN and its partners at Tohoku University claim that this integrated memory approach enables the chip to transition from power-on to kernel startup in just 65.7 milliseconds, which is 38 times faster than utilizing external memory.

To make full use of the limited on-chip memory, the researchers created a hybrid multiplication architecture combining 8-bit and 16-bit multiplication. This design reduces the amount of weight data for the AI model by 53 percent, from 17.7MB to 9.36MB, while preserving 16-bit precision. AISIN is a well-established supplier of automotive parts and currently provides components to Toyota and other automakers, but has not disclosed whether or when they plan to incorporate this chip into production.

Meanwhile, companies like Nvidia and Qualcomm are eager to sell their silicon to car manufacturers, positioning themselves as sources of technology and AI expertise that existing automotive supply chain participants cannot match.