Why aren't my two Cortex-A9 cores cache coherent?

One fascinating aspect of exploring new devkits is the opportunity to write code that runs on unfamiliar hardware. To this end, I chose the Terasic DE0-Nano-SOC, which I had owned for about seven years but had not used extensively. This board, released around 2015, features an Altera Cyclone-V SoC that combines a standard Arm SoC with an Altera FPGA.

In particular, I wanted to work with the two Arm Cortex-A9 processors that the SoC contains. The board includes a Boot ROM that can load a preloader from specific sectors on an SD Card. This preloader runs in On-Chip RAM (OCRAM), initializes external DDR3 SDRAM, and then loads and executes U-Boot from the SD Card.

For my experiments, I downloaded a disk image from Terasic's website and wrote it to a spare SD Card. Upon booting, I discovered that the default configuration booted into an older Linux kernel with the root partition stored on the SD Card. I removed the kernel, causing the board to stop at the U-Boot prompt.

Using a spare USB-to-Serial adapter, I connected my PC to the micro-USB port on the board and interacted with it via minicom on /dev/tty.usbserial-*. By setting the start address to 0x0010_0000, I avoided interfering with the Boot ROM and any RAM used by U-Boot. This configuration was adapted from a previous project I had done on a similar board, with the base address adjusted accordingly.

To transfer the code to the board, I needed a file format compatible with U-Boot. While I initially considered using ELF files, I ultimately opted for Motorola S-Record files, which U-Boot can handle. Using LLVM's binutils and the cargo-binutils plugin, I generated the necessary hex file and loaded it into U-Boot through the serial terminal.