Redis Stopped Writing at 3 AM: maxmemory + the Eviction Policy Nobody Set

On a cold night at 3 AM, a Redis cache suddenly stopped accepting writes. The system did not crash but rather refused to accept any new data. While reads still functioned, writes failed with the message: (error) OOM command not allowed when used memory maxmemory. This caused confusion among developers, who often jump to restarting the server or dumping the entire database as solutions.

The issue arose because the Redis cache had a maximum memory limit set and the eviction policy was not configured. With no eviction policy in place, the cache stopped writing new data once it reached its memory limit. This situation is often overlooked and can lead to OOM (Out of Memory) errors.

To identify the problem, the first step is to check the memory snapshot of Redis using the command redis-cli INFO memory. Four key lines of information are crucial:

1. used_memory_human: This line shows the current memory usage of Redis and how close it is to the set ceiling.

2. maxmemory_human: This line reveals whether a maximum memory limit is set or if it is unlimited (0), which can result in a system OOM kill.

3. maxmemory_policy: This line indicates the eviction policy set for the cache. If the policy is set to "noeviction," Redis will refuse writes instead of evicting any keys when memory is full.

4. mem_fragmentation_ratio: If this ratio is significantly greater than 1.x, it suggests that the process is holding more Resident Set Size (RSS) memory than its logical memory, which could be due to swap or fragmentation, rather than a cache-size problem.

One common mistake is setting maxmemory_policy to volatile-lru but not having any keys with Time To Live (TTL). In such cases, volatile-lru behaves similarly to noeviction, leading to OOM errors even though a safe policy is set. This issue remains hidden unless the actual key space is examined.

Another frequent offender is large data structures like hashes or lists with thousands of elements that were not properly paginated. Client output buffers can also accumulate, resulting in huge output buffers. When deleting large keys, it is important to use the UNLINK command instead of DEL to avoid blocking the server during the deletion process.

To prevent such scenarios, it is essential to determine the appropriate eviction policy based on the specific use case. For a pure cache, allkeys-lru is recommended. For a session store where keys already carry TTLs, volatile-ttl should be used, evicting the soonest-expiring keys first. If data protection is a priority, noeviction can be chosen, but it is crucial to monitor memory usage closely with alerts set at 80% of the maxmemory limit.

A significant step to prevent this issue is to set an alert threshold at 80% of the maxmemory and ensure that every write path includes a TTL by default (for example, using SET key value EX 3600 instead of a bare SET command). By adopting these practices, Redis memory problems will be transformed from alarming 3 AM surprises into routine Monday tuning tasks.