مبانیِ طراحیِ سیستمSystem Design Fundamentals

Let me start with the truth that sinks most people in a System Design interview: there is no right answer; there are only trade-offs. The interviewer doesn't want to see that you memorized Netflix's architecture. They want to see that when you say "I'll add a read replica," you understand what you gained (read scale) and what you gave up (replication lag and weaker consistency). A senior is someone who backs every decision with a "because…" and a number.

This chapter isn't a list of buzzwords to memorize. It teaches a way of thinking: start from requirements, size the load with back-of-envelope estimation, then build the system layer by layer — from load balancer to cache to database — always aware of the trade-offs. The same path you must walk in a 45-minute interview.

1) Method: requirements first, architecture second

The biggest junior mistake in a system design interview: the interviewer says "design a link shortener" and the candidate immediately starts drawing Kafka and Cassandra. A senior asks questions first.

Requirements come in two kinds, and this split is the backbone of the whole interview:

Functional requirements: what the system does. Verbs. "User submits a long URL, gets a short one." "Opening the short link redirects to the original." These shape your API.

Non-functional requirements: how the system must behave. Adjectives. Scale, latency, availability, consistency, durability, security. These shape your architecture — and 90% of the interview score lives here.

Four numbers you must always ask for:

• DAU / MAU (daily/monthly active users) — the basis of every estimate.
• read:write ratio — a 100:1 read-heavy system (like a link shortener) is designed completely differently from a write-heavy one.
• data volume and lifetime — retained for how many years? This determines storage.
• SLA for latency and availability — "under 200ms at p99" and "three nines or four."

2) Back-of-envelope estimation: with paper and brain, not a calculator

Back-of-the-envelope estimation means sizing the system with a few rough multiplications and divisions: how many requests per second (QPS)? how many terabytes of storage? how many gigabits of bandwidth? The goal isn't decimal precision; it's knowing whether you're dealing with "one server" or "a thousand servers" — two entirely different architectures. Like a chef who says "about 40 kilos of meat" without a scale and knows a home fridge won't cut it: it's enough to know which order of magnitude you're in.

Numbers to memorize

Seconds in a day: 86,400 ≈ 10⁵. This is the only number you need for QPS. (Average QPS = daily requests ÷ 10⁵.)

Powers of 2 for memory: 2¹⁰ ≈ thousand (KB), 2²⁰ ≈ million (MB), 2³⁰ ≈ billion (GB), 2⁴⁰ ≈ trillion (TB).

Latency numbers every engineer should know (per Jeff Dean; rounded but still valid for order-of-magnitude reasoning):

Full worked example: a link shortener

Say the interviewer gives you: 100 million new links per day, read-to-write ratio 100 to 1, retain data for 5 years.

Write QPS: 100M ÷ 10⁵ = 1,000 writes/sec (average). Peak is usually 2–3x average → ~3,000 writes/sec at peak.

Read QPS: 100x = 100,000 reads/sec (average), peak ~300,000/sec.

Storage: assume 500 bytes per record (short link, long link, metadata). Per day: 100M × 500B = 50GB/day. Over 5 years: 50GB × 365 × 5 ≈ 91TB. A single node won't do it; you need sharding.

Read bandwidth: 300,000 reads/sec × 500B ≈ 150MB/s. Entirely manageable with a cache.

3) Scaling: vertical vs horizontal

When load grows, you have two ways to get bigger.

Vertical scaling (scale up): a bigger machine. More CPU, more RAM, faster NVMe — a sharper knife and stronger stove for the same cook.

Horizontal scaling (scale out): more machines. Ten ordinary servers instead of one giant — ten cooks instead of one.

Statelessness: the precondition for scaling out

A service is stateless if it keeps no user-specific state in its own memory between requests — every request is self-contained.

Why is it vital? Because when scaling out, the load balancer may send a user's next request to a different instance. If the session lived in instance memory, a dead instance logs the user out — and a new instance is useless because it doesn't have the data.

// ❌ Stateful: session in this instance's memory; breaks scale-out
@RestController
public class CartController {
    private final Map<String, Cart> carts = new ConcurrentHashMap<>(); // in this pod's RAM

    @PostMapping("/cart/add")
    public void add(@RequestHeader("X-User") String user, @RequestBody Item item) {
        carts.computeIfAbsent(user, k -> new Cart()).add(item);
    }
}

// ✅ Stateless: push state to a shared store (Redis/DB)
@RestController
public class CartController {
    private final CartRepository repo; // Redis or DB, shared across all instances

    @PostMapping("/cart/add")
    public void add(@RequestHeader("X-User") String user, @RequestBody Item item) {
        Cart cart = repo.findByUser(user).orElseGet(Cart::new);
        cart.add(item);
        repo.save(cart);
    }
}

4) Load balancing: the traffic distributor

Once you have multiple instances, you need something to spread traffic across them and steer traffic away from sick ones. That's the load balancer.

Let's look at a typical topology.

Diagram: Traffic flow from client through the load balancer to healthy instances — جریان ترافیک از کلاینت به instanceهای سالم پشت load balancer.

flowchart LR
  Client[Clients] --> LB[Load Balancer]
  LB --> A[Instance A - healthy]
  LB --> B[Instance B - healthy]
  LB -. skipped .-> C[Instance C - unhealthy]
  A --> Cache[(Redis)]
  B --> Cache
  A --> DB[(Database)]
  B --> DB

Layer 4 vs Layer 7

This split comes from the OSI model and gets asked a lot in interviews.

L4 (transport): the load balancer only sees IP and port; it forwards the TCP/UDP packet to a backend without opening its contents. Fast, low overhead, but "blind" — it can't decide based on the URL path or headers.

L7 (application): the load balancer understands HTTP content. It can route by path (/api/* to one service, /img/* to another), terminate TLS, add headers, set cookie-based sticky sessions, and retry. More powerful but slightly slower (it must parse the request).

In the Java/Spring world an L7 like Nginx or a cloud ALB usually sits in front of everything, and for intra-system microservice routing there's an API Gateway (like Spring Cloud Gateway) that is itself an L7 router; in Kubernetes, Service and Ingress play this role.

Distribution algorithms

• Round Robin: one by one to each backend in turn. Simple and fair if all instances and all requests are equal-weight.
• Weighted Round Robin: give the stronger machine a bigger share. Useful for a heterogeneous fleet.
• Least Connections: send the next request to the instance with the fewest open connections. Great when request durations vary (some 10ms, some 10s).
• Least Response Time: combines least connections with lowest measured latency.
• IP Hash / Consistent Hash: a hash of the client's IP or key decides the backend; the same user always hits the same backend (without keeping state in the LB). Useful for cache locality.

If requests are short and uniform, Round Robin is enough. If some requests are heavy (an upload, a big report), Least Connections distributes more fairly, because Round Robin might pile several heavy requests onto one instance. If each instance has a local cache and you want a high hit rate, choose Consistent Hashing so a key always lands on the same instance.

Health checks: the heart of availability

A load balancer is only useful if it knows which backend is healthy. Two kinds of health check:

• Active (proactive): the LB sends a light request to /health every few seconds; if it doesn't get 200 or it times out, it pulls the instance out of rotation.
• Passive (reactive): the LB counts real errors from live traffic; if an instance starts returning 500s, it's removed from rotation.

# Spring Boot Actuator: health endpoints for the health check
management:
  endpoint:
    health:
      probes:
        enabled: true      # separate liveness and readiness
      show-details: never  # don't leak details externally
  endpoints:
    web:
      exposure:
        include: health,info

# Kubernetes: uses those same probes
livenessProbe:      # if dead → restart
  httpGet: { path: /actuator/health/liveness, port: 8080 }
  initialDelaySeconds: 20
  periodSeconds: 10
readinessProbe:     # if not ready → remove from Service (no traffic)
  httpGet: { path: /actuator/health/readiness, port: 8080 }
  periodSeconds: 5

5) Cache layers

The cache is the cheapest, most effective scaling lever. The golden rule: the cheapest work is work you don't do at all. Every request that hits the cache is one that didn't hit the database. I'll keep this short because there's a dedicated Caching chapter; here we just place it in the big-picture architecture.

A cache can sit at any layer, and the closer to the user, the faster:

Diagram: Cache layers from the browser down to the database — لایه‌های cache از مرورگر تا دیتابیس.

flowchart LR
  Browser[Browser cache] --> CDN[CDN edge cache]
  CDN --> Gateway[Gateway / API]
  Gateway --> AppCache[App cache - Redis]
  AppCache --> DB[(Database + buffer cache)]

• Client cache (browser): via Cache-Control and ETag headers. The fastest, since it never leaves the device.
• CDN: cache at the network edge, near the geographic user (next section).
• App cache (Redis/Memcached): hot application data; query results, sessions, counters.
• DB cache: the database's own buffer pool keeps recent data in RAM.

The two main patterns

Cache-aside (lazy loading): the most common. The app asks the cache first; on a miss, it reads the DB, populates the cache, and returns.

public Product getProduct(Long id) {
    String key = "product:" + id;
    Product cached = redis.opsForValue().get(key);
    if (cached != null) return cached;            // hit
    Product p = repository.findById(id).orElseThrow();  // miss → DB
    redis.opsForValue().set(key, p, Duration.ofMinutes(10)); // with TTL
    return p;
}

Write-through: every write goes to both cache and DB at once; the cache is always fresh but writes are slower. (Write-behind and read-through patterns also exist — see the Caching chapter.)

6) CDN: bringing content closer to the user

A CDN (content delivery network) is a network of cache servers spread across geographic points worldwide (PoPs or edges). When a user in Tehran wants an image, instead of traveling to a datacenter in the US (150ms round-trip), they get it from the nearest edge (maybe a few ms) — like a retail chain that, instead of one central warehouse, keeps a small local warehouse of best-sellers in every city.

A CDN is mainly for static content: images, video, CSS, JS, downloads. But modern CDNs (Cloudflare, Fastly, CloudFront) can also cache or accelerate dynamic content.

7) Database scaling

The database is almost always the first thing to break under pressure, because it's stateful and you can't simply clone it like a stateless service. Three levers, in increasing order of complexity:

Lever 1: Read replicas

If your system is read-heavy (most are), build one primary for writes and several replicas for reads. Writes go to the primary and propagate via replication to the replicas; reads are load-balanced across the replicas.

Diagram: Writes go to primary; reads fan out to replicas — نوشتن روی primary، خواندن پخش‌شده روی replicaها.

flowchart LR
  App[App] -- writes --> P[(Primary)]
  App -- reads --> R1[(Replica 1)]
  App -- reads --> R2[(Replica 2)]
  P -- async replication --> R1
  P -- async replication --> R2

// Spring: route reads/writes with @Transactional(readOnly = true)
@Service
public class OrderService {

    @Transactional                         // → primary
    public Order place(OrderRequest req) { ... }

    @Transactional(readOnly = true)        // → replica (with DataSource routing)
    public Order get(Long id) { ... }
}

Lever 2: Partitioning (splitting within one database)

Partitioning means splitting a large table into smaller pieces that the database manages separately — but all in the same instance. Useful for managing huge tables (e.g., partition by month, so deleting old data is just dropping a partition).

The syntax differs between databases:

-- PostgreSQL: declarative partitioning (range by date)
CREATE TABLE events (
    id      bigint,
    ts      timestamptz NOT NULL,
    payload jsonb
) PARTITION BY RANGE (ts);

CREATE TABLE events_2026_07 PARTITION OF events
    FOR VALUES FROM ('2026-07-01') TO ('2026-08-01');

-- For uniform distribution by key: hash partitioning
CREATE TABLE accounts (id bigint, name text) PARTITION BY HASH (id);
CREATE TABLE accounts_p0 PARTITION OF accounts FOR VALUES WITH (MODULUS 4, REMAINDER 0);

-- Oracle: its own native partitioning (interval by date, auto-creates partitions)
CREATE TABLE events (
    id      NUMBER,
    ts      TIMESTAMP NOT NULL,
    payload CLOB
)
PARTITION BY RANGE (ts)
INTERVAL (NUMTOYMINTERVAL(1, 'MONTH'))
( PARTITION p0 VALUES LESS THAN (TO_DATE('2026-07-01','YYYY-MM-DD')) );

-- Uniform distribution by key: hash partitioning
CREATE TABLE accounts (id NUMBER, name VARCHAR2(100))
PARTITION BY HASH (id) PARTITIONS 4;

Lever 3: Sharding (splitting across multiple databases)

When the data outgrows a single machine (like that 91TB link shortener), you must spread it across multiple independent databases — that's sharding. Each shard holds a subset and is fully independent.

The key question: which shard does a record go to? Decided by the shard key and the distribution function:

• Range-based: by range (ids 1 to a million in shard 1). Simple but prone to hotspots (if new ids all cluster in the last shard).
• Hash-based: shard = hash(key) % N. Uniform distribution, but it has one big problem, coming up next.

Consistent hashing

Consistent hashing imagines the hash space as a ring (0 to, say, 2³²−1). Both nodes and keys are hashed onto this ring. Each key belongs to the first node clockwise after it.

The benefit: adding or removing a node moves only K/N of the keys (K total, N nodes), not all of them.

Diagram: Keys map to the next node clockwise on the hash ring — نگاشتِ keyها به nodeی که ساعتگرد بعدش می‌آید روی حلقه.

flowchart LR
  subgraph Ring[Hash Ring 0..2^32]
    N1[Node A @ 30]
    N2[Node B @ 120]
    N3[Node C @ 220]
  end
  K1[key1 -> 45] --> N2
  K2[key2 -> 130] --> N3
  K3[key3 -> 250] --> N1

Virtual nodes (vnodes): if each node has just one point on the ring, distribution becomes uneven (one gets a big arc, another a small one). Fix: spread each physical node across several hundred virtual points on the ring. Now load is more uniform, and when a node dies, its load spreads across all the others rather than just one neighbor. This is the standard technique used by Dynamo, Cassandra, and ScyllaDB.

8) CAP and PACELC: the iron law of distributed systems

When data is replicated across multiple nodes (needed for availability), you hit a fundamental law.

The CAP theorem

In a distributed system, among three properties you can only guarantee two at once:

• C — Consistency: every read sees the latest write (all nodes present one view).
• A — Availability: every request gets a (non-error) response, even if not the latest.
• P — Partition tolerance: the system works despite network communication breaking between nodes.

PACELC: the honest continuation of CAP

CAP has a big weakness: it only talks about during a partition, which is rare. But every millisecond the network is healthy there's still a trade-off CAP is silent about: latency vs consistency. That's where PACELC (Daniel Abadi, 2012) comes in:

If there is a Partition (P), choose between Availability and Consistency; Else (E), choose between Latency and Consistency.

The logic: strong consistency requires a write acknowledged by multiple nodes (quorum/sync), which adds latency; wanting low latency means accepting weaker consistency. This trade-off is always present, not just during a disaster.

Consistency models (a spectrum, not a binary)

Consistency is not an on/off switch; it's a spectrum from strong to weak:

• Strong / Linearizable: every read sees the latest acknowledged write, as if one copy. Expensive (needs coordination) but simple for the programmer.
• Sequential / Causal: cause-and-effect order is preserved (if A caused B, everyone sees A before B), but unrelated events may be ordered differently.
• Read-your-writes: you at least see your own writes (session consistency).
• Eventual: if writes stop, eventually all replicas converge. Fast and available, but you may briefly read stale data.

9) Rate limiting and idempotency

Rate limiting: the system's guard

Rate limiting means capping how many requests a client can make in a time window. Without it, a misbehaving client (or an attack, or a retry bug) can swallow all your capacity.

Common algorithms:

Note that fixed window leaks at the window edge: with "100 per minute," a client can send 100 in the last second of one minute and 100 more in the first second of the next — 200 in two seconds. Sliding window solves this but costs more memory; for most uses, token bucket is a good balance.

In Spring, Spring Cloud Gateway has a built-in RequestRateLimiter with a Redis implementation using the token bucket algorithm:

spring:
  cloud:
    gateway:
      routes:
        - id: orders
          uri: lb://order-service
          predicates:
            - Path=/api/orders/**
          filters:
            - name: RequestRateLimiter
              args:
                # bucket refill rate: the most permissive sustained rate (tokens/sec)
                redis-rate-limiter.replenishRate: 10
                # bucket capacity: the maximum instantaneous burst
                redis-rate-limiter.burstCapacity: 20
                # cost per request (default 1)
                redis-rate-limiter.requestedTokens: 1
                key-resolver: "#{@userKeyResolver}"

// Define the key per user (not per whole system); returns 429 when exhausted
@Bean
KeyResolver userKeyResolver() {
    return exchange -> Mono.justOrEmpty(
        exchange.getRequest().getHeaders().getFirst("X-User-Id"))
        .defaultIfEmpty("anonymous");
}

Idempotency: guaranteeing "once, not more"

An operation is idempotent if running it multiple times with the same input has the same effect as running it once. GET, PUT, and DELETE are inherently idempotent; POST usually is not.

Why is it vital? Because the network is unreliable. The client sends a payment request, the server charges the card and sends a response, but the response is lost in transit. The client times out and retries. Without idempotency, the customer pays twice. Like an elevator button: no matter how many times you press for floor 5, the elevator comes once — an idempotent operation must be the same: harmless to retry.

The standard solution: an idempotency key. The client sends a unique key (UUID) with the request; the server stores the result of the first execution and returns the same result to later requests with the same key without re-executing.

Diagram: Duplicate key returns the stored result instead of re-executing — کلیدِ تکراری نتیجهٔ ذخیره‌شده را برمی‌گرداند نه اجرای دوباره.

sequenceDiagram
  participant C as Client
  participant S as Server
  participant DB as Idempotency Store
  C->>S: POST /payments (Idempotency-Key: k123)
  S->>DB: exists(k123)?
  DB-->>S: no
  S->>S: charge card (once)
  S->>DB: save(k123, result)
  S-->>C: 200 OK (result)
  Note over C,S: timeout → client retries
  C->>S: POST /payments (Idempotency-Key: k123)
  S->>DB: exists(k123)?
  DB-->>S: yes → stored result
  S-->>C: 200 OK (same result, no re-charge)

@PostMapping("/payments")
public ResponseEntity<PaymentResult> pay(
        @RequestHeader("Idempotency-Key") String key,
        @RequestBody PaymentRequest req) {

    // If we've seen it before, return the same result (no double charge)
    Optional<PaymentResult> prior = idempotencyStore.find(key);
    if (prior.isPresent()) {
        return ResponseEntity.ok(prior.get());
    }
    // Atomic: reserve only if the key is new (guards the race between two concurrent retries)
    if (!idempotencyStore.tryReserve(key)) {
        return ResponseEntity.status(HttpStatus.CONFLICT).build(); // in progress
    }
    PaymentResult result = paymentService.charge(req); // once
    idempotencyStore.complete(key, result);
    return ResponseEntity.ok(result);
}

-- Atomic key reservation — PostgreSQL
INSERT INTO idempotency (key, status) VALUES ('k123', 'IN_PROGRESS')
ON CONFLICT (key) DO NOTHING;   -- if 0 rows affected, it's a duplicate

-- Oracle equivalent (no ON CONFLICT; use MERGE or exception handling)
MERGE INTO idempotency d
USING (SELECT 'k123' AS key FROM dual) s
ON (d.key = s.key)
WHEN NOT MATCHED THEN INSERT (key, status) VALUES (s.key, 'IN_PROGRESS');

10) Availability math: nines, redundancy, failover

Availability is what fraction of the time the system is up. We measure it in "nines," and each extra nine cuts downtime tenfold — and multiplies cost.

Combining availability: serial vs parallel

If a request passes through several components in series, availabilities multiply and come out lower than any single part. Gateway, service, and DB each at 99.9%: 0.999³ ≈ 99.7% — a chain is weaker than even its weakest link.

Make a component parallel (redundant) and it only fails when all copies fail, so the failure probability multiplies (gets smaller). Two parallel 99% instances: both fail = 0.01 × 0.01 = 0.0001 → 99.99%. Two mediocre instances built an excellent system — the whole philosophy of redundancy.

Redundancy and failover

• Active-Passive: one active node works, a standby waits. If the active fails, failover to the standby. Simple, but the idle standby costs money and failover takes a few seconds.
• Active-Active: all are active and share the load. More efficient with instant failover, but needs state coordination (and the same consistency issues).

Diagram: Active-passive failover lifecycle when the primary fails — چرخهٔ عمرِ failover در حالتِ active-passive.

stateDiagram-v2
  [*] --> ActiveHealthy
  ActiveHealthy --> Degraded: health check fails
  Degraded --> Failover: threshold exceeded
  Failover --> StandbyPromoted: promote standby
  StandbyPromoted --> ActiveHealthy: old node recovers as standby

11) Capstone: a full architecture

Now let's put all the pieces together — the standard topology of a large-scale web system, the one you should be able to draw from memory with justification:

Diagram: Reference architecture of a scalable web system — معماریِ مرجعِ یک سیستمِ وب مقیاس‌پذیر.

flowchart TD
  User[Users] --> CDN[CDN - static assets]
  User --> LB[L7 Load Balancer]
  LB --> GW[API Gateway - auth, rate limit]
  GW --> S1[Service instances - stateless]
  S1 --> Cache[(Redis - cache + sessions)]
  S1 --> P[(DB Primary - writes)]
  S1 --> R[(DB Replicas - reads)]
  P -- replication --> R
  S1 -- events --> Q[(Message Queue)]
  Q --> W[Async Workers]

Trace a request through it: static content comes from the CDN and never reaches the system; the rest hits the L7 load balancer; the API Gateway centralizes auth and rate limiting; stateless services scale horizontally with sessions in Redis; the cache serves most reads and only misses hit the DB; primary/replica splits reads from writes (and sharding at larger scale); heavy work goes async to a queue; and every layer is redundant across AZs.