What Is a Shadow Stack? How Can CPUs Stop Return-Address Hijacking?

A shadow stack is a secondary record of return addresses that a processor can use to verify the integrity of the stack during function calls. In traditional x86-64 architectures, the return address is stored on the normal stack, which is writable and therefore vulnerable to attacks that could corrupt its contents. This vulnerability enables return-address hijacking, where an attacker overwrites the return address to redirect control flow to arbitrary locations.

When a function is called on x86-64, the CALL instruction pushes the address of the instruction following the call onto the normal stack, and then transfers execution to the called function. The stack pointer (RSP) is decremented to allocate space for the return address. Inside the called function, the stack frame expands downward, allocating space for local variables and other state.

The return address is placed above this stack frame during the call. When the function executes the RET instruction, the processor reads the value at the top of the stack (the return address) into the instruction pointer (RIP), increments the stack pointer (RSP), and continues execution at the retrieved address.

The problem arises because the return address is stored in writable memory, which can be corrupted by memory corruption bugs such as buffer overflows. If the write extends far enough up the stack, it can overwrite the return address, causing the RET instruction to jump to an attacker-chosen location rather than the intended one. This allows an attacker to execute arbitrary code or manipulate the program's behavior for malicious purposes.

NX (no-execute) and DEP (data execution prevention) are security mechanisms that mark data regions as non-executable, preventing the execution of instructions from those regions. While these measures protect against executing attacker-placed shellcode on the stack, they do not address the issue of return-address hijacking. This is because the attack does not rely on executing arbitrary data on the stack; instead, it exploits the fact that the attacker can redirect control flow to existing executable code in the program, such as the program's own code, shared libraries, or the C runtime.

The shadow stack addresses this vulnerability by maintaining a second, protected copy of return addresses that is separate from the normal stack. When a function is called, the return address is pushed onto both the normal stack and the shadow stack. The shadow stack is protected by hardware mechanisms that prevent ordinary memory writes from modifying it.

During function return, the system checks whether the return address on the normal stack matches the corresponding address on the shadow stack. If they match, the return proceeds as usual. If they do not match, it indicates that the normal stack has been tampered with, and the return is blocked, preventing the execution of malicious code.

The security of the shadow stack relies on its protection by hardware mechanisms that treat it differently from the normal stack. These mechanisms reject unauthorized writes to the shadow stack, ensuring that it remains intact and serves as a reliable source of truth for return addresses. Software implementations of shadow stacks aim to achieve similar protections, but hardware-assisted implementations offer stronger guarantees because the protection is enforced at the processor level, regardless of the attacker's knowledge of the shadow stack's location.