{
  "id": 7209812,
  "title": "Agent-Cache: Multi-Tier LLM Caching for Valkey and Redis",
  "url": "https://urgent.news/2026/09/14/agent-cache-multi-tier-llm-caching-for-valkey-and-redis",
  "topic": "ai",
  "section": "AI",
  "published": "2026-09-14T00:11:50.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/mech_app_ai/agent-cache-multi-tier-llm-caching-for-valkey-and-redis-10kf"
  },
  "original_language": "en",
  "account": "LLM prompting incurs repeated token and clock cycle costs as the same prompt is sent to the model multiple times, identical tool calls fetch the same data, and session state must be reconstructed entirely for each request. Agent-cache addresses this by implementing a three-tier caching architecture powered by either Valkey or Redis, consolidating LLM responses, tool outputs, and session snapshots behind a unified connection.\n\nThe initial release, v0.1.0, introduced support for Valkey 7 and Redis 6.2, followed by v0.2.0 which added cluster mode the following day. The library offers framework adapters for LangChain, LangGraph, and Vercel AI SDK, as well as OpenTelemetry and Prometheus instrumentation at the cache layer.\n\nThere are three caching tiers:\n1. LLM Response Cache (Tier 1) uses an exact-match cache keyed by prompt text and model parameters. Calling gpt-4o with the same input twice results in the second call retrieving the response from Valkey in under 1ms without hitting the API, saving a significant number of tokens for agents that loop or retry similar prompts.\n2. Tool Output Cache (Tier 2) stores function call results keyed by tool name and arguments. If get_weather(\"Sofia\") is executed twice with identical parameters, the cached result is returned instantly, preventing reinvocation of tools during backtracking or multi-step reasoning.\n3. Session State Cache (Tier 3) preserves agent checkpoints, user intent, and execution state with per-field TTL. LangGraph checkpoints are stored in this tier, along with any custom state required by an orchestrator for mid-flow resumption. Each tier employs a distinct TTL strategy, with LLM responses potentially cached for hours if the model and prompt remain stable, tool outputs expiring more rapidly when underlying data changes frequently, and session state persisting only as long as the user session is active.\n\nThe cache key design for LLM responses combines a prompt hash, model name, temperature, and top-p. Changing any parameter invalidates the cache, meaning the agent will hit the API again if modified. Tool output keys include the function name and a hash of the arguments object, working well for deterministic tools (e.g., database lookups, stable API calls), but breaking down when tools modify external state. The invalidation strategy is manual, requiring explicit calls to cache.invalidate(pattern) with Redis glob patterns. The client defaults to graceful degradation if Valkey or Redis becomes unavailable, with options to fail fast, skip the cache, or use a local fallback for single-request loops.\n\nFailure modes include Valkey or Redis unavailability, addressed through three strategies: fast failure (throwing an error), graceful degradation (skipping the cache), or local fallback (using an in-memory LRU cache for the current request). Observability via OpenTelemetry spans and Prometheus metrics tracks cache performance, enabling proactive checks on Redis availability and invalidation logic if hit rates or latency degrade unexpectedly.\n\nThe library supports integration with LangChain, LangGraph, and Vercel AI SDK through framework adapters, each handling serialization differently (JSON for LangChain, MessagePack for LangGraph). For deployment, Agent-cache assumes the presence of an existing Valkey or Redis instance, handling connection pooling and cluster mode support in v0.2.0. High-availability setups can leverage Redis Sentinel or Valkey's built-in replication, with the client automatically failing over to a replica if the primary goes down.",
  "summary": "Agent loops burn tokens and clock cycles on repeated work. The same LLM prompt fires twice, the same tool call fetches identical data, and session state gets reconstructed from scratch on every request. Agent-cache solves this with a three-tier caching architecture backed by Valkey or Redis, putting LLM responses, tool results, and session snapshots behind a single connection. The project shipped…",
  "key_points": [],
  "editors_take": null,
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}