Introduction

Async retrieval gets data without blocking UI or request threads, but performance still degrades if the same remote call is repeated excessively. Caching solves that, yet naive cache code can introduce stale data and race conditions. A strong approach combines request deduplication, time-based cache policies, and explicit invalidation.

Build a Request-Deduplicating Async Cache

A common problem is a traffic burst where multiple callers request the same key at the same time. Without deduplication, each caller triggers a separate network call. You can avoid this by storing in-flight promises per key.

1type CacheEntry<T> = {
2  value: T;
3  expiresAt: number;
4};
5
6class AsyncCache<T> {
7  private values = new Map<string, CacheEntry<T>>();
8  private inflight = new Map<string, Promise<T>>();
9
10  constructor(private ttlMs: number) {}
11
12  async get(
13    key: string,
14    loader: () => Promise<T>,
15    now: () => number = () => Date.now()
16  ): Promise<T> {
17    const current = this.values.get(key);
18    if (current && current.expiresAt > now()) {
19      return current.value;
20    }
21
22    const pending = this.inflight.get(key);
23    if (pending) {
24      return pending;
25    }
26
27    const promise = loader()
28      .then((result) => {
29        this.values.set(key, { value: result, expiresAt: now() + this.ttlMs });
30        return result;
31      })
32      .finally(() => {
33        this.inflight.delete(key);
34      });
35
36    this.inflight.set(key, promise);
37    return promise;
38  }
39
40  invalidate(key: string): void {
41    this.values.delete(key);
42  }
43}

This structure avoids duplicate remote calls and keeps latency stable under concurrency spikes.

Add Stale-While-Revalidate for Better UX

Strict TTL caches can create sudden latency spikes when entries expire. A stale-while-revalidate policy improves responsiveness by returning stale data quickly while refreshing in background.

1type SwrEntry<T> = {
2  value: T;
3  softExpireAt: number;
4  hardExpireAt: number;
5};
6
7class SwrCache<T> {
8  private data = new Map<string, SwrEntry<T>>();
9  private refreshes = new Map<string, Promise<void>>();
10
11  constructor(private softTtlMs: number, private hardTtlMs: number) {}
12
13  async get(key: string, loader: () => Promise<T>): Promise<T> {
14    const now = Date.now();
15    const hit = this.data.get(key);
16
17    if (!hit) {
18      const value = await loader();
19      this.data.set(key, this.makeEntry(value, now));
20      return value;
21    }
22
23    if (now <= hit.softExpireAt) {
24      return hit.value;
25    }
26
27    if (now <= hit.hardExpireAt) {
28      this.triggerRefresh(key, loader);
29      return hit.value;
30    }
31
32    const value = await loader();
33    this.data.set(key, this.makeEntry(value, now));
34    return value;
35  }
36
37  private makeEntry(value: T, now: number): SwrEntry<T> {
38    return {
39      value,
40      softExpireAt: now + this.softTtlMs,
41      hardExpireAt: now + this.hardTtlMs,
42    };
43  }
44
45  private triggerRefresh(key: string, loader: () => Promise<T>): void {
46    if (this.refreshes.has(key)) return;
47    const p = loader()
48      .then((value) => {
49        this.data.set(key, this.makeEntry(value, Date.now()));
50      })
51      .finally(() => {
52        this.refreshes.delete(key);
53      });
54    this.refreshes.set(key, p);
55  }
56}

This pattern is especially useful for profile pages, dashboard cards, and catalog screens where slightly old data is acceptable for short intervals.

Layer the Cache Near Data Boundaries

Place cache logic in repository or data access modules, not random call sites. Central placement makes invalidation and metrics easier to manage.

For server applications, keep per-process in-memory cache for hot keys and add distributed cache for multi-instance consistency. For frontend applications, use memory cache for current session and persistent browser storage for limited offline durability.

Metrics to track:

• cache hit ratio
• p95 loader latency
• number of in-flight deduplicated requests
• stale serve count and refresh failures

Without telemetry, cache bugs hide behind temporary performance gains.

Handle Errors and Expiration Intentionally

Do not cache failures by default unless your service has explicit negative caching rules. If upstream is unstable, short-lived failure caching may protect infrastructure, but tune it carefully to avoid masking recovery.

When invalidation events exist, such as update mutations or webhooks, clear targeted keys immediately instead of waiting for TTL expiry. Time-based expiration is a fallback, not your only consistency mechanism.

Common Pitfalls

• Caching at every call site instead of one centralized data access layer.
• Ignoring in-flight deduplication, causing thundering herd behavior.
• Using one TTL for all data classes even when freshness requirements differ.
• Caching error responses unintentionally and serving repeated failures.
• Forgetting metrics, which makes tuning cache effectiveness guesswork.

Summary

• Combine async retrieval with key-based cache and in-flight request deduplication.
• Use stale-while-revalidate when fast response is more important than absolute freshness.
• Keep cache logic near data boundaries for maintainable invalidation.
• Track hit ratio and refresh outcomes to tune policies with evidence.
• Treat TTL as one consistency tool, alongside event-driven invalidation.