Steering vectors align LLMs with human values

A recent study reveals that linear directions derived from large language model activations align closely with human values. These steering vectors retain the full structure of the value space, not just superficial behavioral adjustments. Prior safety measures, such as RLHF, DPO, COLD-Steer, and BiPO, relied on heavyweight fine-tuning or behavior-centric methods, which focused on minimizing harms rather than exploring the underlying value topology.

By contrast, distribution-driven steering restores the expected human-value topology, showing a Spearman correlation of up to 0.51. This metric compares activation projections onto linear directions against a theoretical circumplex, with a peak correlation exceeding 0.5, indicating meaningful structure. Among evaluated metrics, distribution-driven methods alone demonstrate robust geometric alignment, whereas behavior-centric approaches show no significant correlation, despite similar benchmark performance.

The findings suggest that geometric fidelity improves with model scale but diminishes after instruction-tuning. Larger models provide more detailed activation manifolds that better capture the circumplex, yet fine-tuning reshapes these manifolds, weakening the connection between steering directions and human values. The study highlights the need for instruction-tuning regimes that preserve or enhance latent alignment.

While geometry correlates with human-consistent transfer across values, it does not guarantee the absence of edge-case harms, necessitating downstream validation. If validated, safety pipelines could incorporate lightweight inference-time modules that extract and apply distribution-driven steering vectors instead of full model fine-tuning.

Applying an additional geometry metric (Spearman correlation against the Schwartz framework) to existing safety benchmarks will determine if a deployment genuinely respects the intended value structure.