One of the first optimisations that we as software engineer learn is caching. The idea is simple, a primary storage call is usually expensive both in terms of latency and cost, therefore we store it in memory in the form of key-value pair.

This idea works really well when it is being implemented in small-scale systems. User Profile loads faster and product page loads with much less pressure on the primary storage. We are actually removing much of the expensive computation that we used to do earlier.

But if we talk about real-life, large-scale systems, things become interesting. The moment we are introducing cache we are now storing the the same data (though in different formats), we are storing data in two different places. And whenever we have same data stored in different places, we are bound to choose between availability and consistency. These two data sources can diverge. The data in cache can become stale. A popular key can expire during peak traffic. Failed invalidation can show users wrong prices, permissions or inventory.

Thus, caching might seem like a performance optimization, but it also forces us to think deeply about reliability and consistency.

In this article, we are going to discuss the practical problems that appear when caching is introduced in production-scale systems: cache invalidation, cache stampedes, and stale data. We will also discuss common strategies used to handle them and the trade offs behind each one.

Why Caching Exists?

Suppose you have a website where users can buy products. You announced a sale on your website and made offers on famous products. All your customers are now simultaneously going to the same webpage that enlists the product which in turn is hitting the same api and you return the same response repeatedly until you notice that the primary storage cannot handle that much load and now users start receiving 5XX errors on their devices.

Caching was introduced in distributed systems for this very problem. To keep primary storage healthy by storing data in the form of key-value pair in memory, Now, each time the user hits the same product page we don't make calls to the primary storage, instead we serve it using cache.

Some pros of caching are:

• Lower latency
• Lower primary storage load
• Better throughput
• Protection during traffic spikes
• Reduced cost for expensive computation or external API calls

Cons:

• Consistency is an issue since the data is now stored in two different places.

Caching Patterns

The way we are updating and reading the data from cache largely determines how the system behaves. Here are some of the famous strategies that people use for reading and writing data into cache.

Cache Aside

This strategy is what most of us would have used when implementing a cache for the first time. The principle is simple. The application checks cache for data. If there is a cache miss, we read from primary storage and write the data to cache.

Pros:

• Application has the flexibility of what gets cached and when
• If system goes down, we can still get the data using primary storage.
• Simple to implement.

Cons:

• Initial read latency is high. Also initially multiple writes to cache can happen if the system is large scale.
• Data can become stale if updates in primary storage are not made in cache simultaneously.
• Consistency can be an issue since primary storage and cache are totally decoupled.

Write Through

In this strategy we write data into cache and primary storage together. The write operation is considered complete only when we get acknowledgements from both primary storage and cache.

Pros:

• Data consistency is guaranteed.
• Reads will be faster since if there is a cache miss, we won't need to check for its presence in primary storage.

Cons:

• Write operation's performance is affected since we now have to wait for acknowledgements from both the storage.
• For write-heavy application, many writes to the cache can be unnecessary.

Write behind

Similar to write through cache but here instead of writing data to primary storage and cache together, first we write data in cache and then a background process asynchronously writes it in the primary storage.

Pros:

• Write latency is very low and throughput is high.
• Writes can be batched and consolidated, thus reducing the I/O strain on primary storage.
• Cache always contains fresh data.

Cons:

• If the cache node crashes all the unsaved data will be lost.
• Other services accessing data directly from primary storage can get stale data for some period of time.
• Implementation can become a bit complex.

Read Through

This is somewhat similar to cache aside. But when it comes to cache miss instead of the application updating the cache, the cache layer itself reads the data from primary storage and writes it to the cache as well as returns the data to the application.

Pros:

• Introduces abstraction at cache layer.
• The data is automatically fetched into the cache without the client worrying about it.

Cons:

• First read will have a somewhat higher latency.
• Complexity can increase a lot, since cache now will be much highly coupled to the primary storage.

Cache Invalidation

Caching becomes difficult when the original data changes.

Suppose we cache a product page response where the price of the product is 90000. Now imagine the price changes in the database from 90000 to 85000. The database has the latest value, but the cache still contains the old value. If the application keeps reading from the cache, users will continue seeing the old price.

This is the core problem of cache invalidation.

Cache invalidation is the process of deciding when cached data is no longer safe to use. It sounds simple, but in real systems it becomes one of the hardest parts of caching because data can be updated from many places, cached in many layers, and read by many services.

The important thing to remember is that a cache is not the source of truth. The primary storage usually is.

TTL Based Expiry

The simplest strategy is to set an expiry or TTL (Time to Live), on each cache entry. This works well when some staleness of data is acceptable, such as blog view, recommendations, number of likes, etc.

However, in this strategy one thing is clear: staleness of data is expected. Therefore, though this strategy is simple, it should only be implemented when the system can safely tolerate stale data for a known time.

Delete Cache On Write

In this strategy, we delete the cache whenever the underlying data is updated. The next fetch after data updates will miss cache, fetch the latest value from primary storage and store it in cache. Deleting the cache keeps the database as the source of truth. The cache is rebuilt only when needed.

However, this approach is not perfect. In case of data updates if cache deletion is failed for some reason, the application can still continue to serve stale data. That's why using TTL based expiry as a safety net is important.

Update Cache On Write

Instead of deleting the cache we can think of updating the cache.

However, this comes with its own set of risks. Risks of race conditions appears if two different updates happen at almost the same time. Or let's say some key stores all the price, description and stock of an item. All the apis which are updating these attributes should know about the cache and should be updating the item's info in the cache.

Versioned Cache Keys

This type of invalidation is done when the expectation is that the whole structure of the value is changed.

Let's suppose that a value contains the name and description of a product and now we now want to add a rating to it. This changes the structure of the value and serialization and deserialization of new data may also cause issues. One solution of this would be to introduce a v1, v2, v3.... in front of the older key name. This will automatically stop fetching the old key.

Versioned cache keys are also useful when deleting cache entries is difficult, when cached objects are expensive to coordinate, or when multiple deployments may use different cache formats.

Event Based Invalidation

In larger systems, one data change may affect many caches. For example, a product update may affect homepage response, search response, product response, etc.

The service updating the product may not know every cache that depends on it. In this case, event based invalidation works better. The service updating the product will now just have to announce the update through an event. Each consuming service decides what to invalidate.

But this adds operational complexity. Events can be delayed, duplicated, or missed. Consumers can fail. Invalidation logic can be incomplete. So, event based invalidation should be combined with retries, idempotency, monitoring, and TTL fallback.

Cache Stampede

A cache stampede happens when many requests try to rebuild the same cache entry at the same time.

This usually happens when a popular cache key expires suddenly. Until the key exists in the cache, requests are served quickly. But once it expires, every request misses the cache and goes to the database or backend service.

For example, imagine we cache the homepage response of a high throughput system. For 10 minutes, everything works well. Requests are served from the cache and the database stays protected.

But when the key expires, thousands of users may request the homepage at the same time. Since the cache value is missing, all of those requests try to fetch the same data from the database.

Instead of one expensive database query, the system suddenly runs hundreds or thousands of identical queries. This can overload the database, increase latency, and sometimes bring down the entire service.

This is the core problem of a cache stampede. A few ways to handle this are given below.

Cache Locking

One possible solution would be to allow only one request to rebuild the cache while others wait for the cache to be rebuilt.

Once the cache key expires, the first request puts a lock in place and fetches the latest data while other requests either wait, retry or receive a fallback response.

This needs to be implemented carefully as the request might fail inside the lock and hold the lock forever. The lock should itself have an expiry. The system should also take decisions on what should be done by other requests while the lock is there.

For high traffic systems, the lock can add to increased latency further if many requests are waiting simultaneously. Thus cache locking should include lock expiry, timeout handling and a fallback path.

Stale While Revalidate

Another practical strategy would be to serve stale data while refreshing the cache in the background. This works well when stale data is acceptable for a short period of time.

The trade off is freshness. Clients may see old data briefly while cache is being refreshed. This strategy is useful when availability and latency are more important than immediate freshness.

Randomised TTLs

There is a possibility that cache stampede may happen when a lot of keys have the same expiry. For example a search page may include 10 products. All of them were cached in the first request at the same time. Thus all of them have the same expiry as well.

A simple fix is to add randomness to the TTL.

TTL = 10 minutes + randomness to the TTL.

This makes the cache key expire at different times thus spreading the database load instead of creating one large spike.

Pre Warming

One other strategy can be to refresh known important cache keys before they expire. For example, let's say for a website its homepage is the most visited page. Here we can try refreshing the cache keys a few seconds to a few minutes before its expiry.

This strategy is useful for predictable, high traffic data such as homepages, configuration, product catalogues, leaderboards, and popular search results.

The downside is that pre warming can waste resources. Some keys may be refreshed even if nobody requests them. It also requires the system to know which keys are important.Therefore, pre warming works best for a small set of critical or frequently accessed keys.

Request Coalescing

Request coalescing means combining multiple identical requests into one backend call. If many requests ask for the same missing cache key, the system does not fetch data from primary storage for all of them. Instead, only for one request it performs the fetch, and the others share the result.

This is similar to cache locking, but it is often implemented inside the application or service layer.

The challenge is managing waiting requests, timeouts, and failures. If the shared backend call fails, the system must decide whether all waiting requests should fail, retry, or receive stale data.

Stale Data

Seeing stale data means a user is seeing an older version of data instead of the latest value from the source of truth.

At first, this sounds like a bug. If the database has new data, why should the application show old data? But in real systems, stale data is not always bad. Sometimes it is completely acceptable. Sometimes it is even a good trade off because it allows the system to respond faster, reduce database load, and stay available during traffic spikes.

The important question is not how do we avoid stale data completely but how stale can data safely be.

The answer depends totally on the use case.

A person looking at the number of views of a video on YouTube finds it to be 2.3k when originally it has reached 2.4k. Nothing critical breaks here. Or recommendations for similar items can also be somewhat stale and it will still work. The same is true for analytics dashboards, feed ranking, trending products, and search suggestions. These features usually care more about performance and availability than perfect freshness.

However, some data cannot be stale. Bank balance should always be accurate. Access permissions must be fresh because stale permissions can become a security issue. Medical or security-related data can create serious risk if users see outdated information.

This is where caching decisions become more than technical decisions. They become product and risk decisions.

The same caching strategy should not be used everywhere. A recommendation list can probably be cached for a few minutes. A dashboard may tolerate even more delay. But permissions, payments, and inventory need much stricter freshness guarantees.

A practical way to think about stale data is to divide it into risk levels.

Low risk stale data includes things like blog views, recommendations, search suggestions, and analytics summaries. These can usually use TTL based caching or stale while revalidate.

High risk stale data includes balances, payments, inventory, access control, and security sensitive information. These should either avoid caching, use very short TTLs, or use strong invalidation whenever the source data changes.

Good caching design is not about making everything fresh all the time. That is often expensive and unnecessary. Good caching design is about knowing which data can be stale, for how long, and what damage stale data can cause.

Conclusion

Caching is one of the most useful techniques for improving performance, but it is not free. Every cache introduces another copy of data, and every extra copy creates questions around freshness, consistency, and failure handling.

In small systems, caching may look like a simple Redis lookup before hitting the database. In real systems, the difficult parts are different: knowing when to invalidate data, preventing cache stampedes during traffic spikes, and deciding how much stale data the product can safely tolerate.

A good caching strategy is not about caching everything. It is about caching the right data, with the right expiry, the right invalidation strategy, and the right fallback when things go wrong.

Before adding a cache, we should ask a few practical questions:

What is the source of truth? How stale can this data safely be? What happens when the cache expires during peak traffic? What happens if cache invalidation fails? Can the system still work if the cache is unavailable?

Caching can make a system faster, cheaper, and more scalable. But when used carelessly, it can also make bugs harder to detect and data harder to trust.

The goal is not just to make reads faster. The goal is to make the system faster while still keeping it correct enough for the use case.