application-caching-strategy
verified8d406d00-b9f4-400a-8f2f-09ea382e6f78
Decide what to cache in an application, where (in-process, Redis, CDN), and how to invalidate safely. Use when repeated expensive work dominates runtime.
Metadata
Skill file
# Application Caching Strategy
Use when repeated expensive work dominates runtime — decide *what* to cache, *where* to cache it, and *how to invalidate* it safely. Caching is a tradeoff, not a default.
## The Caching Decision Table
Ask these five questions before caching anything:
| Question | Cache if... | Don't cache if... |
|---|---|---|
| Is it **expensive** to compute? | DB query >10ms, external API, CPU-heavy | Simple attribute access, in-memory lookup |
| Is it **frequently read**? | Read many times per compute | Read once (cache adds overhead) |
| Is it **staleness-tolerant**? | A few seconds/minutes of staleness is OK | Must be exactly current (bank balance) |
| Is it **shared** or **per-user**? | Shared data (config, catalog) | Per-user data (needs user-scoped keys) |
| Is it **deterministic**? | Same input → same output | Random, time-dependent, or side-effecting |
**Rule of thumb**: Cache only when the answer to "expensive?" AND "frequent?" is yes, and "staleness-tolerant?" is yes.
## Cache Levels
| Level | Tool | Best for | Example |
|---|---|---|---|
| In-process | `functools.lru_cache` | Single-process, per-call memoization | Parsing a repeated config string |
| Local process | `cachetools.TTLCache` | Per-process with expiry | In-memory session cache |
| Distributed | Redis / Memcached | Multiple processes/servers | Shared product catalog |
| HTTP edge | CDN (CloudFront, Cloudflare) | Static/rarely-changing responses | Images, JS bundles, cached API responses |
### In-Process (functools.lru_cache)
```python
from functools import lru_cache
@lru_cache(maxsize=256)
def get_product_price(sku: str) -> float:
# Only call the DB once per unique sku per process lifetime
return db.query_price(sku)
```
**Warning**: `lru_cache` has NO expiry. If the underlying data changes, the cache stays stale until process restart. Use `cachetools.TTLCache` for anything that changes.
### Redis (distributed, with TTL)
```python
import redis
import json
r = redis.Redis(host="localhost", port=6379, decode_responses=True)
def get_cached_product(sku: str):
cached = r.get(f"product:{sku}")
if cached:
return json.loads(cached)
product = db.query_product(sku)
r.setex(f"product:{sku}", ttl_seconds=300, value=json.dumps(product)) # 5-min TTL
return product
```
## Key Design (namespace + version)
```
{namespace}:{entity}:{id}:{version}
product:sku-123:v3 # version bump invalidates everything
user:42:profile
config:feature-flags:v2
```
**Version in the key** is the bluntest, most reliable invalidation: change the version → all old keys become unreachable garbage (cleaned up by TTL).
## Invalidation Strategies
| Strategy | How | Pros | Cons |
|---|---|---|---|
| **TTL** | Set an expiry (`setex`) | Simple, self-healing, no coordination | Stale until expiry; guessing TTL |
| **Explicit invalidation** | Delete key on write (`r.delete(key)`) | Fresh immediately | Must catch *every* write path |
| **Cache-aside** | Read: check cache → miss → compute → store. Write: update DB → delete cache | Standard, well-understood | Race conditions (see pitfalls) |
| **Write-through** | Write to cache and DB together | Cache always fresh | Slower writes; write amplification |
| **Versioned keys** | `key:vN`, bump N on change | No deletion races | Old keys linger until TTL |
### Cache-aside with explicit invalidation (the workhorse)
```python
def update_product(sku: str, data: dict):
db.update_product(sku, data) # 1. Write to DB (source of truth)
r.delete(f"product:{sku}") # 2. Invalidate the cache
# Next read will miss, recompute, and repopulate
```
## Guardrails
- **Never** cache mutable shared state (a mutable object shared across threads) — one consumer mutates it and corrupts everyone.
- **Never** cache without an expiry — a cache that never expires is a permanent stale-read bug waiting to happen.
- **Never** cache security-sensitive data (passwords, tokens, PII without encryption) — caches are not access-controlled.
- **Always** treat the DB as the source of truth — the cache is disposable. If in doubt, evict.
## Pitfalls
- **Caching mutable/shared state**: Returning the *same* dict/list object from `lru_cache` means callers can mutate it. Return copies or immutable structures:
```python
@lru_cache(maxsize=128)
def get_config():
return tuple(sorted(config_items)) # immutable, safe to share
```
- **Stale reads after writes**: You write to the DB but forget to invalidate the cache — readers get old data. Every write path must invalidate.
- **A cache that never expires**: `lru_cache` has no TTL; a misconfigured Redis key with no `setex` lives forever. Always set an expiry.
- **Cache stampede**: A hot key expires, and 1000 concurrent requests all miss and hammer the DB. Use locking or jittered TTLs:
```python
import random
ttl = base_ttl + random.randint(0, 30) # jitter prevents synchronized expiry
```
- **Caching the wrong thing**: Caching a cheap in-memory lookup adds overhead without benefit. Profile first (see `python-profiling`).
## Verify / Checklist
- [ ] Cache decision table applied — only expensive + frequent + staleness-tolerant data is cached
- [ ] Correct cache level chosen (in-process vs Redis vs CDN) for the process topology
- [ ] Every cache key has a namespace + (where needed) a version
- [ ] Every cached value has an expiry (TTL) — no infinite caches
- [ ] Every write path invalidates the corresponding cache entries
- [ ] No mutable shared state returned from cache (copies or immutable structures)
- [ ] No security-sensitive data cached without encryption
- [ ] Cache hit-rate measured before/after (e.g., Redis `INFO stats` → `keyspace_hits`/`misses`)
- [ ] Stampede protection in place for hot keys (locking or jittered TTL)Attached files
No attached files.