Nam's Notes

EN | DE
CS stuff 01.08.2026

Redis in LLM Systems: Necessary Infrastructure or Premature Complexity?

Redis shows up in a lot of LLM architecture diagrams: RAG systems, AI agents, chat platforms, evaluation pipelines, and model-serving stacks. That visibility can create a misleading impression: if serious LLM systems use Redis, maybe every production LLM system needs Redis.

I do not think that is the right conclusion. Redis and Redis-compatible systems are excellent when many application processes need to read and atomically modify small, short-lived values with low latency. They are much less convincing as a default answer to every persistence, retrieval, or inference-memory problem in an LLM stack.

The practical question is not whether an LLM application needs Redis. The better question is:

Which state must be shared across application replicas, accessed with very low latency, modified atomically, and automatically expired?

If that list is empty, a network cache is probably premature complexity. If the list includes global rate limits, admission control, idempotency, cancellation flags, exact computation caches, or short-lived routing state, then Redis-like infrastructure can earn its place.

My current default for self-hosted LLM and RAG systems is simple: start without a network cache, keep PostgreSQL and pgvector authoritative, and add Valkey when shared ephemeral state becomes a measured need.

This article separates the different meanings of "cache" in LLM systems, explains where Redis-like systems help, where they are a poor fit, and compares Redis, Valkey, Garnet, Memcached, Apache Kvrocks, and Dragonfly.

The architectural question

A typical self-hosted RAG and chatbot platform may contain:

Client applications
        |
        v
Application/API servers
        |
        +-- PostgreSQL + pgvector
        |       Durable application state
        |       Conversations and messages
        |       Documents and chunks
        |       Embeddings and vector retrieval
        |       Tenants, permissions and billing
        |
        +-- Object storage
        |       Uploaded and generated files
        |
        +-- LLM inference engine
        |       llama.cpp, vLLM or another runtime
        |
        +-- Optional Redis-compatible service
                Cache
                Rate limits
                Admission control
                Idempotency
                Shared ephemeral state

The word optional is important. PostgreSQL can already store sessions, counters, job records, cached values, and locks. An application process can also maintain an in-process least-recently-used cache. Redis becomes valuable when these alternatives no longer satisfy a measured latency, concurrency, expiration, or coordination requirement.

The right question is therefore not:

Does an LLM application need Redis?

It is:

Which state must be shared across application replicas, accessed with very low latency, modified atomically, and automatically expired?

When no important state satisfies those conditions, Redis may add more operational complexity than value.

"LLM cache" refers to several different systems

The term LLM cache is overloaded. At least seven technically different caching layers may exist in one application.

In-process application cache

Each API process stores frequently used values in its own memory:

API process 1 -> local cache A
API process 2 -> local cache B
API process 3 -> local cache C

This is the lowest-latency approach because it has no network round trip. It is appropriate for:

Its main limitation is inconsistency. Every replica has an independent cache, and invalidation must be propagated explicitly.

A useful progression is therefore:

One application process
    -> local cache

Several replicas with tolerable duplication
    -> local caches plus invalidation

Several replicas requiring a common state
    -> Redis or Valkey

Exact application cache

An exact cache maps a deterministic key to a previously computed value:

embedding:
  embedding-model-version:
  preprocessing-version:
  sha256(normalized-text)
      -> embedding vector

Other examples include:

parsed-document:{parser-version}:{file-hash}
permissions:{user-id}:{permission-version}
retrieval:{tenant}:{knowledge-version}:{query-hash}
response:{tenant}:{model}:{prompt-version}:{request-hash}

Exact caches are comparatively safe because equivalent keys represent deliberately equivalent computations. Their principal correctness problem is incomplete key construction.

Semantic response cache

A semantic cache attempts to reuse a response when a new request is similar enough to an earlier request.

New question
      |
      v
Generate query embedding
      |
      v
Find similar cached queries
      |
      +-- similarity above threshold -> return cached answer
      +-- otherwise                  -> invoke the LLM

Research systems have demonstrated that semantic caches can reduce repeated LLM work by retrieving responses associated with semantically similar queries. However, semantic equality is not logical equality: superficially similar questions can require different answers because of user identity, time, permissions, conversation history, or subtle negation [1].

A static similarity threshold also does not provide a general correctness guarantee. The vCache work identifies unexpected error rates and suboptimal hit rates as limitations of fixed semantic thresholds, motivating an approach with explicit error guarantees [2].

Semantic caching should consequently be treated as an approximate decision system - not merely a faster hash table.

Retrieval-result cache

This layer caches the documents or chunk identifiers returned by RAG retrieval:

tenant
+ query embedding or hash
+ access-control version
+ knowledge-base version
+ retrieval configuration
      -> ranked chunk identifiers

It can eliminate repeated pgvector and reranking work, but invalidation is difficult. A cached result may become stale when:

Retrieval-result caching is therefore most effective for relatively static corpora with repeated queries.

RAG intermediate-state cache

Research such as RAGCache caches model-side intermediate representations of retrieved knowledge rather than only caching final responses. Its evaluation reported lower time to first token and higher throughput for its tested workloads by placing intermediate states across GPU and host-memory layers [3].

This is not an ordinary Redis cache. It interacts with the inference runtime, model architecture, GPU memory, and attention implementation.

Inference prefix and KV cache

During autoregressive inference, the model stores attention keys and values for previously processed tokens. Efficient KV-cache management is central to high-throughput inference; PagedAttention, for example, was designed to reduce fragmentation and redundant allocation while allowing greater batching [4].

This cache belongs close to the inference engine:

GPU memory
    |
    v
optional host-memory offload
    |
    v
inference-runtime-specific storage

A conventional Redis or Valkey server should generally not be used as the token-by-token KV cache. The data is large, latency-sensitive, model-specific, and accessed directly by the inference runtime. Redis may store routing metadata or references to cached prefixes, but it is not a substitute for llama.cpp's or vLLM's internal KV-cache manager.

Operational and coordination state

This is where Redis-compatible systems most consistently provide value:

These values are normally small, frequently updated, shared between replicas, and naturally associated with a time to live.

Where Redis-like systems shine in LLM applications

Global rate limiting

A multi-replica service cannot enforce a global limit using only local process memory.

Suppose four API replicas each permit 100 requests per minute. Without shared state, a user may effectively send 400 requests per minute by distributing requests across the replicas.

A shared atomic counter avoids this:

rate:request:{tenant}:{minute} -> count
TTL: approximately one minute

LLM platforms should normally limit more than request count:

Requests per minute
Input tokens per minute
Output tokens per minute
Concurrent generations
Concurrent ingestion jobs
Embedding operations
Tool executions

Because Redis and Valkey provide atomic operations and key expiration, these controls can be implemented without creating and deleting durable database rows for every short-lived time window. Valkey provides rich in-memory data structures, expiration, persistence options, replication, and cluster operation [5].

Admission control

Inference capacity should be allocated before a request reaches the model server.

Incoming generation request
        |
        v
Check tenant quota
        |
        v
Reserve active-generation slot
        |
        +-- capacity available -> model server
        +-- no capacity        -> queue or reject

A shared counter or lease can prevent multiple API replicas from admitting more work than the GPU fleet can process.

For LLM applications, this use case is often more valuable than response caching. An effective admission-control layer protects:

Idempotency

Network clients, browsers, reverse proxies, and task workers may retry a request. A conditional write such as:

idempotency:{tenant}:{request-id}

can ensure that only one replica begins an expensive document ingestion or model generation.

The durable result should still be stored in PostgreSQL. The Redis-compatible key primarily prevents duplicate work during a limited time window.

Embedding cache

Embedding generation is deterministic when the following remain fixed:

A robust key is therefore:

embedding:
  model-id:
  model-revision:
  preprocessing-version:
  sha256(normalized-input)

This cache can reduce work during repeated ingestion, duplicate uploads, recurring questions, and re-indexing. It is typically safer than semantic response caching because the key represents exact computational identity.

Permissions and configuration cache

RAG requests may require several joins to determine:

These results can be cached using explicit versioning:

permissions:{tenant}:{user}:{acl-version}
configuration:{tenant}:{configuration-version}

Versioned keys are usually safer than attempting to find and delete every affected cache entry after a configuration change.

Conversation routing and cancellation

A streaming chatbot may need to know:

These mappings are short-lived and must be visible to multiple processes. Redis-compatible stores are a natural fit.

Pub/Sub and Streams

Redis Pub/Sub broadcasts transient events but uses at-most-once delivery: a disconnected subscriber misses messages. It is suitable for presence, best-effort invalidation, and live notifications, but not for durable work that must eventually execute [6].

Redis Streams and Valkey Streams are append-only log-like structures with consumer-group functionality. Redis documents consumer-group acknowledgement and redelivery as supporting at-least-once processing semantics [7].

Streams can support a moderate ingestion pipeline. A dedicated broker such as NATS JetStream or RabbitMQ may still be preferable when queueing is a central system responsibility involving long retention, extensive routing, complex retries, or independent operational scaling.

Where Redis does not belong

The authoritative conversation history

Chat messages, citations, tool results, and audit records should normally reside in PostgreSQL.

Redis-compatible replication is commonly asynchronous. Redis Cluster explicitly does not guarantee strong consistency and can lose acknowledged writes in particular failure or partition scenarios. The WAIT command improves practical safety but does not turn Redis into a strongly consistent replicated state machine [8].

A Redis-compatible system may hold the current streaming state, but the completed conversation should be committed to durable storage.

Primary vector storage when pgvector already exists

Redis 8 and Valkey Search can perform vector and hybrid search. Redis integrates vector fields with text, numeric, geospatial, and tag filtering; Valkey Search supports HNSW and exact vector search as well as text and structured filtering [9, 10].

That does not mean an existing pgvector deployment should duplicate every embedding into Redis or Valkey.

Duplicating the vector corpus introduces:

A second vector index is justified only when it satisfies a measured workload that pgvector cannot meet economically - for example, an independent high-QPS semantic cache with a very short retention period.

Large documents and binary objects

Large PDFs, images, audio, serialized prompts, and model outputs consume expensive memory, increase network transfer, enlarge replication buffers, and create latency risks.

Store the authoritative object in S3-compatible storage and cache only:

The inference engine's active KV cache

The model's attention KV cache is not an ordinary application object cache. It is shaped by model layers, heads, token positions, data types, batching, and the inference runtime.

Its performance depends on high-bandwidth local memory and runtime-aware allocation. Research on PagedAttention and RAG-specific KV caching demonstrates that this is a specialized inference-memory problem [3, 4].

Safety-critical distributed locks

A single-node conditional Redis write can provide a useful lease for non-critical deduplication. Strong distributed mutual exclusion is more difficult.

Redis's own distributed-lock documentation explains that simply failing over to an asynchronous replica can violate mutual exclusion because the lock write may not have replicated before promotion [11].

For financially or legally critical workflows, prefer:

Workloads with a poor hit rate

A cache is beneficial only when enough expensive work is avoided.

Let:

Caching is economically rational when approximately:

\[ hC_m > C_l + fC_e + C_o \]

For exact caches, \(f\) should approach zero if keys are correctly constructed. For semantic caches, \(f\) may dominate the calculation because a wrong response can be more costly than recomputing a correct one.

Semantic Caching: Useful but Dangerous

Semantic response caching is attractive because LLM inference is expensive and natural-language questions are rarely byte-identical.

A production semantic-cache key must nevertheless incorporate more than a query embedding:

Tenant identity
User or permission scope
Conversation-state class
Model and revision
System-prompt version
Tool configuration
Knowledge-base version
Retrieval configuration
Temporal validity
Generation parameters
Language and locale

Consider:

"Can I cancel it?"

The answer depends entirely on what "it" refers to, who is asking, when the request was made, and the applicable policy. A high embedding similarity to a previous question is insufficient.

Semantic caching is most defensible for:

It is least defensible for:

A scientific evaluation should measure cache-hit precision, not merely hit rate:

\[ \text{hit precision} = \frac{\text{valid reused answers}} {\text{all reused answers}} \]

A system with a 70% hit rate but 90% precision returns an invalid cached response for 7% of all queries. Depending on the domain, that can be unacceptable.

Persistence models

Redis and Valkey are memory-oriented but can persist data.

No persistence

For a completely disposable cache:

PostgreSQL remains authoritative
Cache can be reconstructed
Cache restarts empty

This normally gives the simplest failure model.

A cache restart causes:

For exact embedding and response caches, disabling persistence is frequently reasonable.

RDB snapshots

RDB creates point-in-time snapshots. Redis describes RDB as compact and suitable for backups and disaster recovery, but data written after the last snapshot may be lost [12].

RDB is appropriate when:

Snapshot generation can create CPU, memory, and storage pressure, particularly for large, actively modified datasets.

Append-only file

AOF records write operations and replays them during recovery. Redis supports different synchronization policies, including every write and approximately every second. Synchronizing every write improves durability but adds storage latency; a one-second policy trades a small loss window for substantially lower overhead [12].

AOF is suitable for semidurable state such as:

It remains inappropriate as the sole protection for irreplaceable business data.

Replication

Redis and Valkey use asynchronous replication in their conventional configurations. Replication improves availability and read capacity but does not eliminate the write-loss window during failover [13].

This leads to a useful rule:

Correctness must survive cache loss
    -> authoritative state belongs elsewhere

Cache loss merely increases latency
    -> Redis or Valkey is appropriate

Performance and bottlenecks

Network latency

For a very small key lookup, the server-side operation may be cheaper than the network round trip.

Pipelining improves throughput by sending multiple commands without waiting for each individual response. Both Redis and Valkey document pipelining as a mechanism for amortizing request/response latency [14].

Avoid patterns such as:

GET key 1
wait
GET key 2
wait
GET key 3
wait

Prefer a multi-key operation or pipeline where correctness permits.

Serialization

JSON encoding, compression, decompression, and client-library object conversion can exceed the time required for the key lookup itself.

For hot paths, measure:

Server-only benchmarks can conceal these costs. The Memcached project similarly recommends observing latency from the client because that captures the complete round trip [15].

Big keys

One 100 MB value is not operationally equivalent to 100,000 one-kilobyte values.

Large keys increase:

Incremental scanning is preferred over commands that return entire large collections. Redis documents SCAN as a production-friendly incremental alternative to operations that can block while returning large keyspaces or collections [16].

Hot keys

A single globally popular key belongs to one shard. Adding shards does not divide the processing of that key automatically.

Possible mitigations include:

Valkey supports client-side caching, allowing application servers to keep selected values locally while receiving invalidation information [17].

Eviction

When memory reaches its configured limit, cache entries must be removed.

Potential policies include LRU-like and frequency-oriented strategies. Valkey's LRU implementation is approximate rather than a mathematically exact global LRU [18].

For LLM workloads, eviction cost should reflect more than recent access. An embedding that costs milliseconds to regenerate and an LLM response that costs seconds to regenerate have very different values.

A simple value-aware score could be:

\[ V_i = \frac{ P_i \cdot C_i }{ S_i } \]

where:

Standard eviction policies do not know the true values of these variables, so separating cache classes may improve predictability.

Persistence and fork-related latency

Persistence introduces disk I/O, log rewriting, snapshots, and memory-copy effects. Redis's latency documentation identifies operating-system, persistence, fork, and command-behaviour factors as possible sources of latency spikes [19].

Do not enable persistence by habit on a disposable cache.

Memory overhead

The required memory exceeds the sum of application payload sizes because the server also maintains:

Capacity tests should use realistic key lengths, value-size distributions, TTLs, and churn rather than idealized fixed-size values.

Comparing Redis and its alternatives

No implementation is universally superior. The correct choice depends on API requirements, licensing, working-set size, persistence, CPU topology, and operational maturity.

Redis Open Source

Redis 8 and later are available under a tri-license that includes the OSI-approved AGPLv3 as well as RSALv2 and SSPLv1. Redis 8 also incorporates search and additional data structures into Redis Open Source [20].

Strengths

Weaknesses

Best fit

Organizations that require Redis-specific integrated functionality, tooling, or commercial support and accept the licensing model.

Valkey

Valkey is a Linux Foundation-backed, BSD-licensed in-memory data-structure store. It supports strings, hashes, lists, sets, sorted sets, streams, scripting, persistence, replication, Sentinel-style availability, and cluster operation [5].

Valkey Search is also BSD-licensed and supports vector, text, numeric, and tag search through a module [10].

Strengths

Weaknesses

Best fit

The strongest default for teams seeking a genuinely open-source, self-hosted Redis-compatible cache and coordination layer.

Garnet

Garnet is an MIT-licensed RESP-compatible cache-store developed by Microsoft Research. It supports Redis clients, common data structures, persistence-related capabilities, tiered storage, replication, sharding, and key migration. Its cluster design documentation notes that the cluster is passive and expects a user-provided control plane for some management actions [21].

Strengths

Weaknesses

Best fit

Teams willing to validate compatibility in exchange for multi-core throughput, tiered storage, or close alignment with the Microsoft/.NET ecosystem.

Memcached

Memcached is a free and open-source distributed in-memory object cache intended for small pieces of arbitrary data. It deliberately provides a simpler model than Redis-like data-structure servers [22].

Strengths

Weaknesses

Best fit

A pure, disposable exact-value cache where lists, streams, sets, scripting, persistence, and advanced coordination are unnecessary.

Apache Kvrocks

Apache Kvrocks is an Apache project implementing a Redis-compatible distributed key-value database on RocksDB. Its design places data primarily on persistent storage rather than requiring the complete dataset to reside in memory. It supports a broad range of Redis-style commands, replication, and cluster-oriented operation [23].

Strengths

Weaknesses

Best fit

Large Redis-compatible datasets whose capacity economics are more important than obtaining the lowest possible all-RAM latency.

Dragonfly

Dragonfly is a Redis-compatible, thread-per-core datastore designed to exploit large multi-core machines. It is released under Business Source License 1.1 rather than an OSI-approved open-source licence. Its documentation permits many self-hosted production uses but restricts certain directly competing service offerings [24].

Strengths

Weaknesses

Best fit

Organizations that accept BSL licensing and have measured a strong advantage from vertically scaling a Redis-compatible workload on large multi-core machines.

Decision matrix

Requirement Preferred starting point
No shared state; one application instance In-process cache
Pure disposable object cache Memcached or Valkey
Shared TTL cache, counters, and rate limits Valkey
Existing organization standardized on Redis Redis Open Source
Redis-compatible cache with permissive licence Valkey
Multi-core or tiered-storage experimentation Garnet
Very large disk-backed RESP dataset Apache Kvrocks
Single large multi-core node; BSL acceptable Dragonfly
Durable relational application state PostgreSQL
Existing RAG vector retrieval PostgreSQL + pgvector
Active model attention KV cache Inference runtime
Large source documents Object storage
Complex durable messaging Dedicated message broker

For most new self-hosted LLM platforms using PostgreSQL and pgvector:

Start without a network cache, add Valkey when shared ephemeral state becomes necessary, and retain PostgreSQL as the system of record.

Recommended architecture for PostgreSQL, pgvector, and llama.cpp

                            +-----------------------+
Users ---> API gateway ---->| Stateless API servers |
                            +-----------+-----------+
                                        |
             +--------------------------+-------------------------+
             |                          |                         |
             v                          v                         v
    +----------------+        +----------------+       +----------------+
    | PostgreSQL     |        | Valkey         |       | Object storage |
    | + pgvector     |        | optional       |       |                |
    |                |        |                |       | PDFs, images,  |
    | durable state  |        | rate limits    |       | source files   |
    | embeddings     |        | admission      |       |                |
    | retrieval      |        | exact caches   |       +----------------+
    | conversations  |        | idempotency    |
    +----------------+        +----------------+
                                        |
                                        v
                                +----------------+
                                | llama.cpp pool |
                                | inference KV   |
                                | and batching   |
                                +----------------+

Suggested ownership boundaries

PostgreSQL and pgvector

Valkey

llama.cpp

Object storage

Recommended Valkey deployment strategy

Stage 1: No Valkey

Use this when:

Use a bounded in-process cache and measure actual bottlenecks.

Stage 2: One Valkey Service

Add Valkey when:

For disposable cache data:

Persistence: disabled
Memory limit: explicit
Eviction: enabled
Failure behaviour: cache miss

Stage 3: Separate Cache and Control State

At larger scale, separate two failure and eviction domains:

valkey-cache
    Embeddings
    Retrieval results
    Parsed-document results
    Disposable response cache
    Eviction enabled
    Persistence disabled

valkey-control
    Rate limits
    Admission leases
    Idempotency
    Cancellation state
    Routing information
    Restricted eviction
    Optional AOF

This prevents a large response cache from evicting concurrency controls or idempotency records.

Stage 4: Replication or Cluster

Use replication for availability and read scaling. Use sharding when memory or command throughput exceeds a single primary.

Do not introduce a cluster merely because the application is called "distributed." Cluster operation changes:

Scale only after measuring a single instance under realistic traffic.

Designing safe cache keys

A cache key is an encoded declaration of equivalence.

For an exact final-answer cache:

answer:
  tenant-id:
  permission-version:
  conversation-class:
  model-id:
  model-revision:
  system-prompt-version:
  knowledge-base-version:
  retrieval-version:
  tool-configuration-version:
  generation-parameter-hash:
  normalized-request-hash

For an embedding cache:

embedding:
  model-id:
  model-revision:
  preprocessing-version:
  normalized-text-hash

For retrieval:

retrieval:
  tenant-id:
  permission-version:
  knowledge-base-version:
  embedding-model-version:
  retrieval-configuration-version:
  normalized-query-hash

Never omit tenant identity from cached private data. Cross-tenant cache reuse can become a direct confidentiality vulnerability.

Scientific evaluation methodology

A cache should be evaluated as a system intervention rather than through isolated operations-per-second benchmarks.

Experimental conditions

Compare at least:

  1. No cache
  2. Local exact cache
  3. Distributed exact cache
  4. Semantic cache at several thresholds
  5. Distributed cache with cold restart
  6. Cache during invalidation and document updates
  7. Cache during replica failure

Use a replay of representative production queries where possible. Split tuning and evaluation data chronologically to avoid selecting thresholds that overfit previously observed requests.

Application-level metrics

Measure:

Infrastructure metrics

Measure:

Correctness metrics

For semantic caches:

A cache that reduces latency while increasing wrong answers is not an optimization. It is a change to model behaviour.

Business and operational considerations

Technical founders should resist adding Redis because it appears in successful companies' architecture diagrams. Those diagrams reflect scale that has already materialized.

An additional distributed datastore creates:

The proper adoption sequence is:

Observe repeated expensive work
        |
        v
Quantify its cost and frequency
        |
        v
Test an in-process cache
        |
        v
Determine whether replicas need shared state
        |
        v
Add Valkey or another distributed cache
        |
        v
Verify business and technical benefit

For an early-stage product, improving customer acquisition, product usefulness, reliability, and answer quality will usually matter more than removing a few milliseconds from database access.

Final recommendation

For a self-hosted multi-tenant RAG and chatbot platform already using PostgreSQL with pgvector:

  1. Keep PostgreSQL as the authoritative system.
  2. Keep embeddings and durable vector retrieval in pgvector.
  3. Keep documents in object storage.
  4. Keep active attention KV caches inside llama.cpp or the inference engine.
  5. Start with an in-process exact cache where practical.
  6. Add Valkey when multiple replicas require shared TTL state, atomic counters, admission control, or valuable exact caches.
  7. Do not begin with semantic response caching unless the workload is repetitive and low-risk.
  8. Evaluate semantic-cache correctness, not merely hit rate.
  9. Use no persistence for disposable cache data.
  10. Do not rely on asynchronous Redis-compatible replication for irreplaceable state.
  11. Use a dedicated broker when durable messaging becomes a major subsystem.
  12. Benchmark with real prompt sizes, tenants, invalidations, and concurrency - not only synthetic GET/SET workloads.

The resulting default is:

PostgreSQL + pgvector
    -> durable business and retrieval state

Valkey
    -> optional shared ephemeral state

llama.cpp
    -> model execution and KV-cache management

Object storage
    -> large immutable content

Conclusion

Redis is not an LLM requirement. It is a specialized answer to a recurring distributed-systems problem: many application processes need to read and atomically modify small, short-lived values with low latency.

In LLM systems, it is particularly effective for admission control, rate limiting, idempotency, exact computation caches, and transient routing state. It is ineffective or dangerous when used indiscriminately as a primary database, document store, model KV-cache system, or approximate semantic oracle.

Among open-source Redis-compatible choices, Valkey is the most broadly defensible default because it combines permissive licensing, community governance, familiar data structures, and strong compatibility. Redis remains attractive when its integrated ecosystem or commercial support is valuable. Garnet deserves evaluation for multi-core and tiered-storage workloads, Memcached remains excellent for simple disposable caching, and Apache Kvrocks is useful when disk-backed capacity matters more than pure in-memory latency. Dragonfly may offer compelling vertical scaling but should be described as source-available rather than strictly open source.

The scientifically and commercially sound position is therefore:

Add a Redis-compatible system only when a defined workload justifies it, assign it reconstructable state, and verify its value through application-level measurements.

References

  1. GPTCache: An Open-Source Semantic Cache for LLM Applications.
    https://arxiv.org/abs/2411.05276

  2. vCache: Verified Semantic Caching for Large Language Models.
    https://arxiv.org/html/2502.03771

  3. RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation.
    https://arxiv.org/abs/2404.12457

  4. Kwon, W. et al. Efficient Memory Management for Large Language Model Serving with PagedAttention.
    https://arxiv.org/abs/2309.06180

  5. Valkey documentation: Introduction.
    https://valkey.io/topics/introduction/

  6. Redis documentation: Pub/Sub.
    https://redis.io/docs/latest/develop/pubsub/

  7. Redis documentation: Streaming use cases.
    https://redis.io/docs/latest/develop/use-cases/streaming/

  8. Redis documentation: Scale with Redis Cluster.
    https://redis.io/docs/latest/operate/oss_and_stack/management/scaling/

  9. Redis documentation: Vector search.
    https://redis.io/docs/latest/develop/ai/search-and-query/vectors/

  10. Valkey documentation: Valkey Search.
    https://valkey.io/topics/search/

  11. Redis documentation: Distributed locks.
    https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/

  12. Redis documentation: Persistence.
    https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/

  13. Redis documentation: Replication.
    https://redis.io/docs/latest/operate/oss_and_stack/management/replication/

  14. Redis documentation: Pipelining.
    https://redis.io/docs/latest/develop/using-commands/pipelining/

  15. Memcached: How long does it take for real?
    https://memcached.org/blog/how-long-for-real/

  16. Redis documentation: SCAN.
    https://redis.io/docs/latest/commands/scan/

  17. Valkey documentation: Client-side caching.
    https://valkey.io/topics/client-side-caching/

  18. Valkey documentation: LRU cache.
    https://valkey.io/topics/lru-cache/

  19. Redis documentation: Latency optimization.
    https://redis.io/docs/latest/operate/oss_and_stack/management/optimization/latency/

  20. Redis licensing.
    https://redis.io/legal/licenses/

  21. Microsoft Garnet repository and documentation.
    https://github.com/microsoft/garnet

  22. Memcached project.
    https://memcached.org/

  23. Apache Kvrocks project.
    https://kvrocks.apache.org/

  24. Dragonfly licensing FAQ.
    https://www.dragonflydb.io/docs/about/faq

~/tools