Database sharding splits a database into smaller parts (shards) distributed across multiple servers to scale writes and reads.
How It Works
Instead of one large database, data is partitioned across multiple database instances based on a shard key.
Sharding Strategies
Hash-Based Sharding
- Apply hash function to shard key (e.g., user_id)
- Distribute evenly across shards
- Pro: Uniform distribution
- Con: Hard to rebalance when adding shards
Range-Based Sharding
- Partition by ranges (e.g., A-M, N-Z)
- Pro: Easy to add shards
- Con: Risk of hotspots if data is unevenly distributed
Geographic Sharding
- Partition by location (e.g., US-East, EU-West)
- Pro: Reduced latency for regional users
- Con: Uneven load if regions have different usage
Directory-Based Sharding
- Lookup table maps keys to shards
- Pro: Flexible reassignment
- Con: Lookup adds overhead
Benefits
- Horizontal scaling: Add shards to handle more data and traffic
- Improved performance: Smaller datasets are faster to query
- Fault isolation: One shard failure doesn’t affect others
Challenges
- Cross-shard queries: Joins across shards are expensive or impossible
- Rebalancing: Moving data between shards is complex
- Hotspots: Popular data can overwhelm a single shard
- Complexity: Application logic must be shard-aware
Related Concepts
- Normalisation (vertical partitioning by columns)
- Horizontal partitioning (sharding is horizontal partitioning across servers)
- Vertical vs Horizontal Partitioning