Server-side caching

Technique · Chapter 10

The idea in brief

Caching = copying data that may be needed again into fast storage so later access is quicker. It improves both performance and scalability.
Server-side caches avoid repeatedly making expensive retrievals from the origin store (e.g. a relational database).
Place the cache as close to the application as possible to minimize latency. Use fast storage (e.g. an in-memory database). The more data and users, the bigger the payoff.
(HTTP caching and CDNs are covered in Improving web application performance.)

Caching in distributed applications

Two strategies, usable together in one app:

Private cache

Held on the machine running the application instance (often in-memory, extremely fast; can spill to local file system if it exceeds available memory).
Each instance has its own cache → the same query can yield different results across instances (a consistency risk).

Shared cache

Lives in a separate location, reached via a cache service; all instances use it.
Removes the divergent-view problem and improves scalability (a cluster can back the cache; the service locates data across the cluster).
Slower than a private cache due to network latency — but worth it when data consistency matters more.

Priming the cache

Pre-populate the cache at application startup with data needed at startup or widely used enough to be worth having immediately.
Improves performance from the very first requests.

Keeping cached data correct

“There are only two hard things in Computer Science: cache invalidation and naming things.” — Phil Karlton

Invalidation

Cached data goes stale when the source changes after caching. Cache invalidation = replacing or removing cached items so stale data doesn’t persist; also needed when the cache fills up.

Expiration

Configure data to expire after a set time.
Some systems support a system-wide expiration policy plus per-item policies.
Expiration is typically an absolute value (e.g. 1 day).

Eviction policies (when the cache is full)

Policy	Discards…	Assumption
LRU (Least Recently Used)	least-recently-used items first	recently used → likely used again soon
MRU (Most Recently Used)	most-recently-used items first	recently used → won’t be needed again
FIFO (First-In, First-Out)	oldest inserted item	ignores last-use time
LIFO (Last-In, First-Out)	newest inserted item	ignores last-use time
Explicit eviction	specifically chosen items	e.g. after underlying data is deleted/updated

Cache usage patterns

Two broad ways an app works with a cache:

Cache-aside

The application maintains the cache and reads/writes the database itself; the cache doesn’t touch the DB.
On read: check cache first; hit → return from cache (bypass system-of-record); miss → read DB, store in cache, return.
On write: the app must invalidate/refresh cached data to keep it consistent with the system-of-record.

Read-through / write-through / write-behind

Here the cache is treated as the system-of-record and handles DB I/O itself:

Read-through: on a miss, the cache loads from the DB, stores it, and returns it.
Write-through: the cache writes to the DB and updates the cache; the thread waits for the DB write.
Write-behind: like write-through but the DB write is queued — the thread proceeds sooner, at the cost of a brief window where cache and DB are inconsistent.

Improving web application performance — HTTP caching and CDNs.
Taking a systematic approach to performance improvement
Cross-cutting concerns — caching as a cross-cutting concern.
Database performance

Citations

Software Architect’s Handbook (Packt, 2018), Ch.10 “Server-side caching”, pp. 755-767.