Back-of-Envelope Numbers
Latency Reference
| Operation | Latency |
|---|---|
| L1 cache | ~1 ns |
| L2 cache | ~4 ns |
| RAM access | ~100 ns |
| Redis get | ~0.1–1 ms |
| SSD random read | ~0.1 ms |
| Same-DC network round trip | ~0.5 ms |
| Cross-region network | ~50–150 ms |
| HDD seek | ~10 ms |
Throughput Reference
| System | Throughput |
|---|---|
| PostgreSQL (simple indexed reads) | ~10K–50K QPS |
| PostgreSQL (complex queries / writes) | ~1K–5K QPS |
| Redis | ~100K–500K ops/sec |
| Kafka | millions of msgs/sec |
| Typical web server | ~10K–100K RPS |
| 1 Gbps NIC | ~125 MB/s |
Sizing Rules of Thumb
- Good Redis cache hit rate: >90% (>95% for hot data)
- Read:write ratio in most apps: ~80:20
- Data doubles every 18 months (Moore’s law for storage)
- 1 million users ≈ ~100 RPS at peak (assume 10x daily peak)
Availability (Nines)
| SLA | Downtime/year |
|---|---|
| 99.9% | ~8.7 hours |
| 99.99% | ~52 minutes |
| 99.999% | ~5 minutes |
CAP Theorem
Distributed systems can guarantee only 2 of 3. Partition tolerance is non-negotiable, so the real tradeoff is:
- CP (Consistency + Partition Tolerance): refuse requests during partitions. Use for: finance, inventory, distributed locks (Zookeeper, etcd, PostgreSQL).
- AP (Availability + Partition Tolerance): serve stale data during partitions. Use for: social feeds, DNS, shopping carts (Cassandra, DynamoDB).
Consistency Models
| Model | Guarantee | Cost | Use When |
|---|---|---|---|
| Strong | Every read sees latest write | High latency | Payments, locks |
| Causal | Cause precedes effect | Medium | Collaborative tools |
| Read-your-writes | You see your own writes | Low | User profiles |
| Eventual | Converges over time | Lowest | Feeds, DNS, analytics |
Scaling
Vertical: More CPU/RAM on one machine. Simple, but has a ceiling.
Horizontal: More machines behind a load balancer. Complex, but unbounded.
When to Scale What
| Bottleneck | Solution |
|---|---|
| CPU-bound app | Horizontal scale (add servers) |
| DB reads | Read replicas + caching |
| DB writes | Sharding or write-optimised DB (Cassandra) |
| Single point of failure | Redundancy + health checks |
| Traffic spikes | Message queue (decouple producers/consumers) |
| Global latency | CDN for static; regional replicas for dynamic |
Caching
Rule: Cache reads, not writes. Cache results that are expensive to compute and tolerate slight staleness.
Patterns
| Pattern | How | Use When |
|---|---|---|
| Cache-aside | App checks cache → DB on miss, writes back | General purpose |
| Write-through | Write to cache + DB simultaneously | Strong consistency needed |
| Write-behind | Write to cache, async flush to DB | Write-heavy, tolerate some loss |
| Read-through | Cache loads from DB automatically on miss | Simpler app code |
Invalidation Strategies
- TTL: Simple, may serve stale. Pick TTL based on acceptable staleness.
- Explicit invalidation: Accurate, but easy to miss edge cases.
- Event-driven: Publish cache-busting events on write (reliable, complex).
CDN
Cache static assets (images, JS, CSS) geographically close to users. For dynamic content, use CDN with short TTLs or edge caching with surrogate keys.
Databases
SQL vs NoSQL
| SQL (PostgreSQL) | NoSQL (Cassandra / DynamoDB) | |
|---|---|---|
| Schema | Fixed | Flexible |
| Transactions | ACID | Eventually consistent |
| Scaling writes | Hard (sharding required) | Built-in |
| Joins | Native | Avoid |
| Use when | Relationships, transactions | High write throughput, flexible schema |
Indexing Tips
- Index columns used in
WHERE,JOIN,ORDER BY - Covering index: includes all columns for a query — avoids table lookup entirely
- Indexes slow writes; only add what you need
- Partial indexes for filtered queries (e.g.,
WHERE status = 'active')
Sharding
Split data across DB instances by a shard key.
| Strategy | Pro | Con |
|---|---|---|
| Hash-based | Even distribution | Hard to rebalance |
| Range-based | Easy to add shards | Hotspots on skewed data |
| Geographic | Low latency per region | Uneven load |
Watch for: cross-shard queries (expensive), hotspot shards, complex application logic.
Resilience Patterns
Message Queues
When to add: any time a slow consumer would block a fast producer, or you need retry semantics.
- Point-to-point: one message → one consumer (SQS, RabbitMQ)
- Pub/Sub: one message → many subscribers (Kafka, SNS, Redis Streams)
- Always: implement a Dead Letter Queue for messages that fail repeatedly
Circuit Breaker
Stops calling a failing downstream service after N failures, returns fast errors until it recovers. Prevents cascade failures.
CLOSED → (failures > threshold) → OPEN → (timeout) → HALF-OPEN → (success) → CLOSED
Retry + Backoff
Use exponential backoff with jitter for retries. Avoid thundering herd after outages.
Idempotency
Make operations safe to retry. Techniques:
- Idempotency key in request header → server deduplicates via Redis/DB unique index
- Database
ON CONFLICT DO NOTHING/ unique constraints - HTTP
PUT/DELETEare naturally idempotent;POSTis not
Outbox Pattern
Guarantee event publishing when you update a DB and publish a message:
- Write business data + event to same DB transaction
- Background worker reads unpublished events and publishes to queue
- Mark as published. Consumers must be idempotent (at-least-once delivery).
Use CDC (Debezium + Kafka) instead of polling for efficiency at scale.
Common Interview Scenarios
URL Shortener
- Hash long URL → short 6-8 char ID (base62)
- Store mapping in DB; cache hot entries in Redis
- Redirect: 301 (client caches) vs 302 (track each click)
- Scale reads: Redis + CDN; scale writes: shard by hash
Twitter-style Feed
- Fan-out on write: push to followers’ timeline cache on post. Fast reads, expensive for celebrities.
- Fan-out on read: pull from followees on load. Cheap writes, slow reads.
- Hybrid: fan-out on write for normal users, fan-out on read for celebrities.
Rate Limiting
- Token bucket: allow bursts up to bucket size, refill at fixed rate
- Leaky bucket: smooth output rate regardless of input bursts
- Fixed window: simple but boundary spikes
- Sliding window: accurate, more complex
- Store counters in Redis with TTL keyed by
user_id:window
Monitoring
Collect 3 pillars: logs, metrics, traces (e.g., OpenTelemetry → Prometheus + Grafana + Jaeger). Define SLIs and SLOs before incidents happen.
Quick Reference: Add This When…
| Problem | Add |
|---|---|
| Slow reads | Cache (Redis) + read replicas |
| DB write bottleneck | Message queue + async processing |
| Single server failure risk | Load balancer + multiple instances |
| Cross-service consistency | Outbox pattern |
| Retry storm risk | Circuit breaker + exponential backoff |
| Duplicate writes on retry | Idempotency keys |
| Global latency | CDN + regional replicas |
| Uneven DB load | Sharding |
| Real-time event fan-out | Pub/Sub (Kafka / SNS) |