sharding-strategy — independently scanned and version-tracked by SaferSkills.
SaferSkills independently audited sharding-strategy (Agent Skill) and scored it 100/100 (green). The audit ran 55 deterministic rules across Security, Supply Chain, Maintenance, Transparency, and Community; it found 0 high-severity and 0 lower-severity findings. The full rule-by-rule trace and per-finding evidence are below. Free, methodology-open.
Findings & checks · 0 flagged
Every scanned point with the score it earned and what moved between them.
First recorded scan — no prior version to compare against.
The primary manifest — the file an agent reads to learn what this artifact does.
Sharding (horizontal partitioning) is the discipline of dividing a database's data across multiple nodes so each node holds a subset — a shard. The unit of judgment is the shard key: the column or columns the system uses to route each row to a specific shard. Three foundational schemes: range partitioning (contiguous shard-key value ranges per shard — strong for range queries BETWEEN x AND y; weak for hotspots at range boundaries and for resharding-by-split), hash partitioning (hash(shard_key) % N — strong for even balance and no range hotspots; weak for range queries which become scatter-gather, and for resharding which rehashes nearly all data), consistent hashing (hash ring with each key routed to the nearest clockwise shard — adding shards moves only 1/N of data; virtual nodes place each physical shard at many ring positions to reduce imbalance), and directory / lookup partitioning (explicit map per key or key-range — strong for arbitrary placement; weak because the directory itself becomes a bottleneck).
Replaces "scale vertically forever" with horizontal capacity for write throughput, storage, and geographic placement. Solves the problem that when write throughput exceeds the primary's capacity, when storage approaches the node's limit, or when geographic placement is regulatory or latency-driven, the simpler single-node tools — replication (scales reads), caching (reduces load), denormalization (eliminates joins), vertical scaling (adds capacity) — are no longer sufficient. Sharding is the scaling tool of last resort — reached for only when the simpler tools are exhausted, because it adds operational complexity proportional to the gain. The shard key is the most consequential design decision: a well-chosen key gives nearly linear scaling; a poorly-chosen key pays operational complexity without capacity gain — hotspots concentrate on one shard, common queries scatter-gather across all shards, transactions require two-phase commit and become slow and failure-prone. The schema must be designed with sharding in mind from the start, or significant refactoring is required when sharding is later introduced.
Distinct from replication-patterns, which owns copying the same data across nodes for fault tolerance and read scaling; this skill owns dividing different data across nodes for write throughput and storage capacity. The two compose in production because each shard is usually replicated, but they answer different questions. It is also distinct from cap-theorem-tradeoffs, indexing-strategy, entity-relationship-modeling, query-optimization, and transaction-isolation; those skills own the theory frame, within-shard retrieval, schema design, single-query tuning, and single-system transactions respectively.
The discipline of dividing a database's data across multiple nodes through horizontal partitioning. Covers the three foundational partitioning schemes (range, hash, directory), consistent hashing as the refinement that solves the resharding problem, the shard-key choice as the most consequential design decision, the cross-shard query and transaction trade-offs, the catalog of failure modes (hot shard, skewed distribution, range-boundary overload), the relationship to replication (sharding divides; replication copies; they compose), and the rule that sharding is one of the last optimizations to reach for after replication, caching, and denormalization.
Sharding is the scaling tool of last resort. Replication scales reads; caching reduces load; denormalization eliminates joins; vertical scaling adds capacity. When write throughput, storage capacity, or geographic data placement exceeds what those tools provide, sharding becomes the answer.
The shard key is the most consequential design decision. It determines which queries are fast (single-shard) and which are slow (scatter-gather), which operations are atomic (single-shard) and which require distributed commit (cross-shard), which growth patterns hotspot and which balance. A team that chooses the shard key well gains nearly linear scaling; a team that chooses it poorly pays operational complexity without capacity gain.
The schema must be designed with sharding in mind from the start, or significant refactoring is required when sharding is later introduced. Queries must filter on the shard key; related data must be co-located on the same shard; cross-shard operations must be rare or accepted as slow. Sharding is a schema architecture, not just an operational technique.
| Scheme | How it routes | Strong for | Weak for |
|---|---|---|---|
| Range | Contiguous key ranges per shard | Range queries (BETWEEN x AND y) | Hotspots at range boundaries; resharding by split |
| Hash | Hash(key) % N | Even balance; no range hotspots | Range queries become scatter-gather; resharding rehashes |
| Consistent hashing | Hash ring; key → nearest clockwise virtual node | Adding shards moves only about 1/N of data; virtual nodes smooth imbalance | Range queries still scatter-gather; virtual-node maps must be maintained |
| Directory | Explicit map per key | Flexibility; arbitrary routing | The directory itself becomes a bottleneck |
Hash with consistent hashing is the default for most large-scale systems; range partitioning is used for time-series and naturally-ordered data; directory is rare but useful for arbitrary placement.
A good shard key:
Common keys:
tenant_id. Most queries are tenant-scoped; tenants are independent.user_id. Most queries are user-scoped.time_bucket(timestamp). Recent shards hot; older shards cold (often acceptable for time-series).region. Latency benefit; regulatory benefit.Bad keys:
created_at (range hotspot at latest range).status (low cardinality; one value dominates).| Operation | Single-shard | Cross-shard |
|---|---|---|
| Lookup by shard key | Fast | n/a (must include shard key) |
| Lookup not using shard key | Single-shard if data co-located | Scatter to every shard |
| JOIN | Fast within shard | Slow or unavailable |
| Aggregation | Fast | Scatter-gather; partial-aggregate-then-combine |
| Transaction | ACID via single-shard primary | Two-phase commit or distributed consensus; slow and failure-prone |
Schema design under sharding co-locates related data (store user's orders on user's shard) to make JOINs and transactions single-shard.
| Workload property | Tool |
|---|---|
| Read load too high | Replication (read replicas) |
| Cache hit rate possible | Caching layer |
| Joins / aggregations slow | Denormalization, materialized views |
| Single-node CPU/memory exceeded | Vertical scaling |
| Storage approaching node limit | Sharding (or larger disks first) |
| Write throughput exceeds primary | Sharding |
| Geographic latency required | Sharding by region (with replication within region) |
| Multi-tenant isolation required | Sharding by tenant |
Sharding is the answer when write throughput or storage exceeds single-node capacity. Before that, simpler tools suffice.
After applying this skill, verify:
| Instead of this skill | Use | Why |
|---|---|---|
| Copying the same data to multiple nodes | replication-patterns | replication copies; sharding partitions |
| The CAP / PACELC theoretical frame | cap-theorem-tradeoffs | CAP names the trade-off |
| Tuning a slow query | query-optimization | query-optimization is single-query |
| Designing indexes within a shard | indexing-strategy | indexing is within-node |
| Designing schema | entity-relationship-modeling | entity-relationship-modeling is the schema design; this is the partitioning of the schema |
| Single-node transactional behavior | transaction-isolation | ACID is the single-system frame |
<!-- skill-graph-context:start (generated — do not edit by hand) -->
Classification
data-engineeringtrueengineering/dataWhen to use
how should we shard, what's the right shard key, hot shard, cross-shard transaction, consistent hashingNot for
Related skills
entity-relationship-modeling, replication-patternsreplication-patterns, cap-theorem-tradeoffs, indexing-strategy, entity-relationship-modeling, connection-poolingConcept
Keywords
sharding, partitioning, horizontal partitioning, shard key, hash partitioning, range partitioning, consistent hashing, hot shard, resharding, cross-shard query<!-- skill-graph-context:end -->
~30 seconds. Free. No account. Every finding cites a rule and a line of evidence.