Caching Strategies

Overview

Caching is one of the most effective techniques for improving application performance and reducing load on backend systems. This guide covers various caching strategies, implementation patterns, and best practices for our platform.

Related Architecture Content

Performance Optimization

System-wide performance optimization strategies and techniques.

performance

Caching Layers

1. Browser Cache

Location: Client-side (browser)
Scope: Individual user
TTL: Hours to days

# HTTP Headers for browser caching
Cache-Control: public, max-age=3600
ETag: "abc123"
Last-Modified: Wed, 21 Oct 2024 07:28:00 GMT

Use Cases:

• Static assets (CSS, JS, images)
• API responses that rarely change
• User-specific data with predictable update patterns

2. CDN Cache

Location: Edge servers
Scope: Global
TTL: Hours to days

# CloudFront distribution configuration
Distribution:
  DefaultCacheBehavior:
    CachePolicyId: "CachingOptimized"
    TTL:
      DefaultTTL: 86400  # 24 hours
      MaxTTL: 31536000   # 1 year
    CacheKeyPolicy:
      QueryStringsConfig:
        QueryStringBehavior: "whitelist"
        QueryStrings: ["version", "locale"]

Use Cases:

• Static content delivery
• API responses for public data
• Geographically distributed content

3. Application Cache

Location: Application memory
Scope: Single application instance
TTL: Minutes to hours

// In-memory caching with Node.js
const NodeCache = require('node-cache');
const cache = new NodeCache({ stdTTL: 600 }); // 10 minutes

function getCachedData(key) {
  let value = cache.get(key);
  if (value === undefined) {
    // Cache miss - fetch from database
    value = fetchFromDatabase(key);
    cache.set(key, value);
  }
  return value;
}

Use Cases:

• Frequently accessed data
• Computed results
• Session data

4. Distributed Cache

Location: Shared cache cluster
Scope: Multiple application instances
TTL: Minutes to hours

// Redis distributed caching
const redis = require('redis');
const client = redis.createClient({
  host: 'redis-cluster.internal',
  port: 6379
});

async function getCachedData(key) {
  try {
    const cached = await client.get(key);
    if (cached) {
      return JSON.parse(cached);
    }
    
    // Cache miss - fetch and store
    const data = await fetchFromDatabase(key);
    await client.setex(key, 600, JSON.stringify(data)); // 10 minutes TTL
    return data;
  } catch (error) {
    // Fallback to database on cache failure
    return await fetchFromDatabase(key);
  }
}

Use Cases:

• Shared data across services
• Session storage
• Rate limiting counters

5. Database Cache

Location: Database layer
Scope: Database queries
TTL: Automatic based on data changes

-- Query result caching
SELECT SQL_CACHE user_id, username, email 
FROM users 
WHERE status = 'active'
AND last_login > DATE_SUB(NOW(), INTERVAL 30 DAY);

Use Cases:

• Expensive query results
• Aggregated data
• Frequently accessed tables

Caching Patterns

1. Cache-Aside (Lazy Loading)

async function getUserById(userId) {
  const cacheKey = `user:${userId}`;
  
  // Try cache first
  let user = await cache.get(cacheKey);
  if (user) {
    return user;
  }
  
  // Cache miss - load from database
  user = await database.users.findById(userId);
  if (user) {
    await cache.set(cacheKey, user, 3600); // 1 hour TTL
  }
  
  return user;
}

Pros: Simple, cache only what's needed
Cons: Cache miss penalty, potential stale data

2. Write-Through

async function updateUser(userId, userData) {
  const cacheKey = `user:${userId}`;
  
  // Update database first
  const user = await database.users.update(userId, userData);
  
  // Update cache
  await cache.set(cacheKey, user, 3600);
  
  return user;
}

Pros: Cache always consistent with database
Cons: Write latency, unnecessary caching

3. Write-Behind (Write-Back)

async function updateUser(userId, userData) {
  const cacheKey = `user:${userId}`;
  
  // Update cache immediately
  const user = { ...existingUser, ...userData };
  await cache.set(cacheKey, user, 3600);
  
  // Queue database update
  await writeQueue.add('updateUser', { userId, userData });
  
  return user;
}

Pros: Fast writes, reduced database load
Cons: Risk of data loss, complex consistency

4. Refresh-Ahead

async function getUserById(userId) {
  const cacheKey = `user:${userId}`;
  const user = await cache.get(cacheKey);
  
  if (user) {
    // Check if cache is about to expire
    const ttl = await cache.ttl(cacheKey);
    if (ttl < 300) { // Less than 5 minutes
      // Refresh cache asynchronously
      refreshCache(cacheKey, userId);
    }
    return user;
  }
  
  // Cache miss - synchronous load
  return await loadAndCacheUser(userId);
}

Pros: Reduced cache miss latency
Cons: Increased complexity, resource usage

Cache Invalidation Strategies

1. TTL-Based Expiration

// Set TTL based on data characteristics
const ttlStrategies = {
  userProfile: 3600,      // 1 hour - changes occasionally
  userPreferences: 86400, // 24 hours - changes rarely
  sessionData: 1800,      // 30 minutes - security sensitive
  staticContent: 604800   // 1 week - rarely changes
};

await cache.set(key, data, ttlStrategies[dataType]);

2. Event-Driven Invalidation

// Invalidate cache on data changes
eventBus.on('user.updated', async (event) => {
  const cacheKeys = [
    `user:${event.userId}`,
    `user:profile:${event.userId}`,
    `user:permissions:${event.userId}`
  ];
  
  await cache.del(cacheKeys);
});

3. Tag-Based Invalidation

// Tag-based cache management
await cache.set('user:123', userData, {
  ttl: 3600,
  tags: ['user', 'user:123', 'department:engineering']
});

// Invalidate all engineering department caches
await cache.invalidateTag('department:engineering');

Performance Optimization

Cache Hit Rate Optimization

// Monitor cache performance
const cacheStats = {
  hits: 0,
  misses: 0,
  
  recordHit() { this.hits++; },
  recordMiss() { this.misses++; },
  
  getHitRate() {
    const total = this.hits + this.misses;
    return total > 0 ? (this.hits / total) * 100 : 0;
  }
};

async function getCachedData(key) {
  const data = await cache.get(key);
  if (data) {
    cacheStats.recordHit();
    return data;
  }
  
  cacheStats.recordMiss();
  // Load from source...
}

Memory Management

// LRU cache with size limits
const LRU = require('lru-cache');

const cache = new LRU({
  max: 10000,        // Maximum items
  maxAge: 1000 * 60 * 60, // 1 hour
  updateAgeOnGet: true,
  stale: true        // Return stale data while refreshing
});

Cache Warming

// Pre-populate cache with frequently accessed data
async function warmCache() {
  const popularUsers = await database.users.findPopular(100);
  
  for (const user of popularUsers) {
    const cacheKey = `user:${user.id}`;
    await cache.set(cacheKey, user, 3600);
  }
  
  console.log(`Warmed cache with ${popularUsers.length} users`);
}

// Run cache warming on application startup
warmCache();

Monitoring & Alerting

Key Metrics

# Prometheus metrics for cache monitoring
cache_hits_total:
  type: counter
  labels: [cache_name, key_pattern]

cache_misses_total:
  type: counter
  labels: [cache_name, key_pattern]

cache_operation_duration_seconds:
  type: histogram
  labels: [cache_name, operation]

cache_memory_usage_bytes:
  type: gauge
  labels: [cache_name]

cache_evictions_total:
  type: counter
  labels: [cache_name, reason]

Alerting Rules

groups:
  - name: cache-alerts
    rules:
      - alert: LowCacheHitRate
        expr: |
          (
            rate(cache_hits_total[5m]) / 
            (rate(cache_hits_total[5m]) + rate(cache_misses_total[5m]))
          ) < 0.8
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Cache hit rate below 80%"
          
      - alert: HighCacheMemoryUsage
        expr: cache_memory_usage_bytes / cache_memory_limit_bytes > 0.9
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "Cache memory usage above 90%"

Best Practices

Do's

• Choose appropriate TTL values based on data update frequency
• Monitor cache hit rates and adjust strategies accordingly
• Use cache tags for efficient bulk invalidation
• Implement circuit breakers for cache failures
• Warm critical caches during application startup
• Use compression for large cached objects
• Set memory limits to prevent out-of-memory errors

Don'ts

• Don't cache everything - focus on frequently accessed data
• Don't ignore cache failures - always have fallback strategies
• Don't use overly long TTLs for frequently changing data
• Don't cache sensitive data without proper security measures
• Don't forget about cache stampede protection
• Don't neglect cache eviction policies - use LRU when appropriate

Cache Stampede Prevention

// Prevent cache stampede with locks
const lockManager = require('redis-lock');

async function getCachedDataSafe(key) {
  let data = await cache.get(key);
  if (data) return data;
  
  // Use distributed lock to prevent multiple fetches
  const lock = await lockManager.acquire(`lock:${key}`, 10000); // 10s timeout
  
  try {
    // Check cache again after acquiring lock
    data = await cache.get(key);
    if (data) return data;
    
    // Fetch and cache data
    data = await fetchFromDatabase(key);
    await cache.set(key, data, 600);
    
    return data;
  } finally {
    await lock.release();
  }
}

Related Topics

• Database Performance Optimization (Coming Soon)
• CDN Configuration (Coming Soon)
• Load Balancing Strategies (Coming Soon)
• Monitoring & Observability (Coming Soon)

---

Last Updated: 2024-01-15
Maintainer: Performance Engineering Team
Review Schedule: Quarterly