A simple solution is to use modulo hashing:

server = hash(key) % 4

This works until your application grows and you add a fifth server.

Now your formula becomes:

server = hash(key) % 5

The problem? Most keys may now map to different servers. Your cache loses much of its usefulness, requests fall through to the database, and database traffic can spike unexpectedly.

This is where consistent hashing comes in.

Consistent hashing is a technique used in distributed systems to minimize how many keys need to move when servers are added or removed.

In this guide, we'll explore why modulo hashing breaks during scaling, how a hash ring works, why virtual nodes improve load distribution, and how to implement consistent hashing in Python.

1. Why Does Hash Modulo N Break?

The modulo approach assigns a key to a server using the remainder of its hash divided by the number of servers.

server_index = hash(key) % number_of_servers

For example:

• Four servers produce indices from 0 to 3.
• Five servers produce indices from 0 to 4.

When the number of servers changes, the remainder changes for many keys.

Let's demonstrate the problem using a simplified example:

keys = range(100_000)

moved = sum(
    1 for key in keys
    if key % 4 != key % 5
)

print(f"Keys moved: {moved / len(keys):.1%}")

Output:

Keys moved: 80.0%

This example uses the keys themselves as hash values to make the calculation easy to understand. With well-distributed hash values, adding one server to a modulo-based arrangement of N servers typically remaps approximately N/(N+1) of the keys.

That means adding a fifth server to four existing servers can remap around 80% of the keys.

For a distributed cache, this can cause a large increase in cache misses. For a sharded database, it can mean substantial data redistribution.

The goal: Add or remove servers while moving as few keys as possible.

2. What Is Consistent Hashing?

Consistent hashing places both servers and keys on the same logical hash ring.

Imagine a circle representing the hash space. Every server and every key is assigned a position on that circle using a hash function.

To determine which server owns a key:

1. Hash the server identifiers and place them on the ring.
2. Hash the key to find its position.
3. Move clockwise from the key's position.
4. Assign the key to the first server encountered.
5. If the search passes the end of the ring, wrap around to the beginning.

This creates a stable mapping between keys and servers.

What happens when a server is added?

Suppose a new server is inserted into the ring.

It takes responsibility for keys in the portion of the ring between its position and the preceding server's position. Most other keys remain assigned to their existing servers.

When a server is removed, its keys move to the next server clockwise.

This is the main advantage of consistent hashing: membership changes generally affect only a portion of the key space instead of remapping almost everything.

For a ring with N evenly distributed servers, adding one server is expected to move approximately 1/(N+1) of the keys under the usual uniform-distribution assumptions.

3. Why Do We Need Virtual Nodes?

A basic hash ring places each physical server at a single position. This can create uneven partitions.

For example, one server might own a large section of the ring, while another owns a much smaller section. Even if both servers have identical capacity, their key distributions could differ significantly.

Virtual nodes help reduce this imbalance.

Instead of assigning one position to each physical server, assign multiple positions to every server.

For example:

cache-1#0
cache-1#1
cache-1#2
...
cache-1#99

Each identifier represents a virtual node belonging to the same physical server.

With 100 virtual nodes per server, the ring has 100 positions per physical server. These additional positions tend to produce a more balanced distribution of keys.

Virtual nodes also provide flexibility:

• More virtual nodes generally improve distribution.
• Servers with higher capacity can receive more virtual nodes.
• Removing a physical server removes all of its virtual nodes.
• The ring becomes larger, increasing memory use and membership-update work.

Virtual nodes improve balance statistically; they do not guarantee equal traffic or eliminate hot keys.

4. Implement Consistent Hashing in Python

Let's build a simple consistent hash ring using Python's hashlib and bisect modules.

We'll use MD5 to generate stable hash positions. MD5 is used here for distribution, not security.

import bisect import hashlib

class ConsistentHashRing: def init(self, nodes=None, vnodes=100): if vnodes < 1: raise ValueError("vnodes must be at least 1")

    self.vnodes = vnodes
    self.ring = []
    self.owner = {}

    for node in nodes or []:
        self.add_node(node)

def _hash(self, value: str) -> int:
    digest = hashlib.md5(value.encode()).digest()
    return int.from_bytes(digest, "big")

def add_node(self, node: str):
    for i in range(self.vnodes):
        pos = self._hash(f"{node}#{i}")

        if pos in self.owner:
            raise ValueError(
                "Hash collision detected; use a collision-handling strategy"
            )

        bisect.insort(self.ring, pos)
        self.owner[pos] = node

def remove_node(self, node: str):
    for i in range(self.vnodes):
        pos = self._hash(f"{node}#{i}")

        if self.owner.get(pos) != node:
            raise ValueError(f"Node not found on ring: {node}")

        self.ring.remove(pos)
        del self.owner[pos]

def get_node(self, key: str) -> str:
    if not self.ring:
        raise ValueError("No nodes in the ring")

    pos = self._hash(key)

    # Find the first ring position at or after the key.
    # Wrap around to the first position if necessary.
    idx = bisect.bisect_left(self.ring, pos) % len(self.ring)

    return self.owner[self.ring[idx]]

Understanding the implementation

_hash()

Converts an identifier into a stable integer position using the full MD5 digest.

add_node()

Creates virtual nodes, calculates their positions, and inserts those positions into the sorted ring.

remove_node()

Removes the virtual nodes associated with a physical server.

get_node()

Finds the first server position at or after the key's hash position. If the key lies beyond the last position, the modulo operation wraps the search to the beginning.

What is the time complexity?

Let V represent the total number of virtual nodes across all physical servers.

• Hashing a key: O(1) for a fixed-size digest.
• Looking up a key: O(log V) using binary search.
• Inserting a virtual node into a Python list: O(V) in the worst case because elements may need to shift.
• Removing a virtual node from a Python list: O(V) in the worst case.

The lookup is efficient, but Python's list-based implementation is not optimized for frequent server membership changes.

5. Test the Ring: Add a Server

Now let's create four cache servers, assign keys to them, and observe what happens when a fifth server joins.

ring = ConsistentHashRing(
    ["cache-1", "cache-2", "cache-3", "cache-4"],
    vnodes=100
)

keys = [f"user:{i}" for i in range(10_000)]

before = {
    key: ring.get_node(key)
    for key in keys
}

ring.add_node("cache-5")

after = {
    key: ring.get_node(key)
    for key in keys
}

moved = sum(
    1 for key in keys
    if before[key] != after[key]
)

print(f"Keys moved: {moved / len(keys):.1%}")

The program calculates the actual movement percentage for this particular implementation and key set. Run it to obtain the result rather than assuming a fixed percentage.

With four original servers and a fifth server added, the expected movement is approximately 20% under the usual uniform-distribution assumptions. The exact result varies with virtual-node placement and the tested keys.

Unlike modulo hashing, consistent hashing moves keys to the new server without randomly redistributing all the remaining keys among the original servers.

You can also test removal:

ring.remove_node("cache-5")

restored = {
    key: ring.get_node(key)
    for key in keys
}

print("Mapping restored:", before == restored)

Expected output:

Mapping restored: True

This holds for this example because the original servers and their ring positions have not changed. In a real distributed system, moving data between servers is a separate operation from changing the ring's logical mapping.

6. Where Is Consistent Hashing Used?

Distributed caches

Consistent hashing helps cache clients map keys to cache servers while minimizing remapping when the server pool changes.

Distributed databases

Distributed databases can use consistent-hashing-style partitioning to distribute keys across nodes. Dynamo is a well-known example of a distributed key-value store that uses consistent hashing concepts.

Read the Dynamo paper summary on ScaleDojo to learn more about the design.

Content delivery networks

A CDN can use hashing-based routing to distribute objects across cache nodes and reduce unnecessary movement when infrastructure changes.

Distributed storage systems

Consistent hashing can help determine which nodes should be responsible for particular keys or objects.

For fault tolerance, systems may replicate data to multiple distinct nodes. Replication is a separate design decision; assigning a key to one node does not automatically make its data resilient to failures.

7. Trade-Offs You Should Know

Consistent hashing reduces unnecessary movement, but it does not eliminate every distributed-systems challenge.

Data still needs to move. When a new server takes ownership of keys, the corresponding data may need to be copied or reconstructed.

Virtual nodes consume resources. More virtual nodes increase ring size and membership-update costs.

Hot keys remain a problem. A key receiving disproportionately high traffic can overload its owner even when keys are distributed evenly.

Load balancing is not guaranteed. Virtual nodes improve statistical distribution, but actual traffic and key sizes can still vary.

Alternatives exist. Rendezvous hashing and jump consistent hashing offer other approaches to distributing keys, each with different properties and trade-offs.

8. Consistent Hashing Interview Tips

If an interviewer asks you to design a distributed cache or sharded storage system, explain the concept in this order:

1. Explain why hash(key) % N causes extensive remapping when N changes.
2. Draw a hash ring and explain clockwise ownership.
3. Demonstrate how adding or removing a server affects only part of the ring.
4. Introduce virtual nodes to improve distribution.
5. Discuss replication, hot keys, data migration, and operational trade-offs.

A strong system design answer explains not just how consistent hashing works, but also why it is useful and where it falls short.

9. Learn More About Consistent Hashing

Want to explore the algorithm beyond the theory?

• Consistent Hashing Wiki on ScaleDojo — review the core concept.
• Consistent Hashing in the HLD Course — explore its role in high-level system design.
• Scale a Cache with Consistent Hashing — practise implementing the algorithm.

Conclusion

Consistent hashing is a practical technique for distributing keys across changing server pools without remapping most of the key space every time infrastructure changes.

By combining a hash ring with virtual nodes, you can improve key distribution and make scaling more manageable.

The important takeaway is simple: consistent hashing minimizes unnecessary key movement, while virtual nodes help improve balance. Together, they provide a useful foundation for designing scalable distributed caches, storage systems, and other distributed applications.

You can wait for production to answer that question at 3 a.m. Or you can test it yourself in a controlled environment before your users notice a problem.

That second approach is called chaos engineering.

This guide explains chaos engineering in simple terms and walks you through a five-step method for running your first experiment safely.

What Is Chaos Engineering?

Chaos engineering is the practice of deliberately introducing controlled failures into a system to understand how it behaves and discover weaknesses before real incidents occur.

Netflix helped popularize this approach with Chaos Monkey, a tool designed to terminate instances so engineers could build systems that tolerate unexpected failures.

You don't need a large infrastructure or a complex tool to get started. A small, carefully planned experiment involving a single service is a good first step.

The key word is controlled. Chaos engineering isn't about randomly breaking things. It's about testing a specific assumption, observing the result, and improving the system.

The 5-Step Method for Chaos Engineering

1. Define Your Steady State

Before introducing a failure, define what a healthy system looks like.

For example:

• 95th-percentile latency stays below 300 ms.
• Error rate remains below 1%.
• Orders continue to complete at the expected rate.

These measurements give you a baseline. Without them, you won't know whether your experiment has affected the system.

2. Write a Hypothesis

A hypothesis describes what you expect to happen when something fails.

For example:

"If one cache node becomes unavailable, the application will continue serving requests, with an error rate below 1% and latency below 300 ms."

This turns a failure scenario into a measurable experiment.

3. Limit the Blast Radius

Start small and reduce the risk of affecting users.

• Use a test or staging environment.
• Test one service or instance at a time.
• Define a clear stop condition, such as aborting if the error rate exceeds 5%.
• Prepare a kill switch to stop the experiment.
• Inform the team before running the test.

Never introduce failures into production without appropriate authorization, monitoring, and rollback procedures.

4. Inject the Failure

Now introduce one controlled failure.

For example, stop one test instance, add latency to a dependency, or simulate a network timeout.

Watch your dashboards while the experiment runs. Pay attention to latency, error rates, resource usage, and whether the system recovers automatically.

5. Observe, Fix, and Repeat

Compare the actual result with your hypothesis.

Did the system behave as expected? Did requests fail? Did latency increase? Did alerts trigger?

If you discover a weakness, fix it and rerun the same experiment. The goal is not simply to find a problem but to verify that the improvement works.

Failure Types to Try First

Here are five useful scenarios for beginners:

• Kill an instance: Check whether traffic moves to healthy instances.
• Add latency to a dependency: Find out whether requests time out or wait too long.
• Make a dependency unavailable: Verify that the application has a fallback.
• Increase CPU usage or fill a test disk: Check resource limits, monitoring, and alerts.
• Introduce network delays or packet loss: See whether retries make the situation worse.

Start with one scenario, measure the results, and increase complexity only when your system and safety procedures are ready.

A Worked Example: Testing a Checkout Service

Imagine an e-commerce application with a checkout service that depends on two other services:

• Payment service: Processes payments and is essential to completing an order.
• Recommendation service: Displays suggested products and is optional to checkout.

The steady state

Your checkout service normally succeeds 99.9% of the time, with 95th-percentile latency below 400 ms.

The hypothesis

"If the recommendation service becomes slow, checkout will continue working because recommendations are optional."

The experiment

In a test environment, introduce a two-second delay to the recommendation service.

The result

Checkout becomes slow or times out because the application waits for recommendations before continuing.

The experiment has exposed a design weakness: an optional feature is affecting a critical user journey.

The fix: Use a timeout and fallback

The checkout service should not wait indefinitely for recommendations. Instead, it should use a strict time limit and continue without personalized recommendations when the dependency is slow or unavailable.

The improved logic is simple:

1. Send a request to the recommendation service.
2. Wait for a maximum of 300 milliseconds.
3. If recommendations arrive on time, display them.
4. If the request times out or fails, show default recommendations or skip them.
5. Continue the checkout process without blocking the purchase.\

This is an example of graceful degradation: an optional feature can fail without taking down the core service.

You can strengthen the design further with a circuit breaker. When the recommendation service repeatedly fails, the circuit breaker temporarily stops sending requests to it, preventing additional load and giving the service time to recover.

Finally, rerun the same experiment. If checkout remains responsive when recommendations are delayed, you have evidence that your fix works.

How to Keep Chaos Engineering Safe

Chaos engineering is useful only when experiments are designed carefully.

• Start outside production: Use a test environment before considering production experiments.
• Establish observability: Monitor latency, errors, traffic, resource usage, and service health.
• Define abort conditions: Decide in advance when to stop the experiment.
• Test one failure at a time: This makes it easier to identify the root cause.
• Practice game days: Schedule sessions where the team tests failure scenarios and rehearses incident response.
• Document every experiment: Record the hypothesis, setup, results, findings, and fixes.

A successful experiment doesn't necessarily mean nothing failed. It means you learned something useful without causing unacceptable harm.

Common Chaos Engineering Mistakes

Starting Without a Hypothesis

If you don't know what you're testing, the results will be difficult to interpret.

Making the Blast Radius Too Large

Testing several services simultaneously can make failures harder to diagnose. Start with one service and expand gradually.

Ignoring the Findings

Discovering a weakness is only the beginning. Assign the fix, implement it, and verify the result with another experiment.

Running Experiments Only Once

Applications change as new features, dependencies, and infrastructure are introduced. Regular experiments help verify that resilience hasn't regressed.

Why Chaos Engineering Matters in System Design Interviews

Interviewers often ask questions such as, "How would you verify that your system is resilient to failures?"

Chaos engineering gives you a practical way to answer.

Explain that you would:

1. Define measurable steady-state conditions.
2. Form a hypothesis about a potential failure.
3. Limit the blast radius and establish abort conditions.
4. Inject one controlled failure.
5. Observe the results, fix weaknesses, and repeat the experiment.

You can also discuss timeouts, retries, circuit breakers, redundancy, and graceful degradation.

This demonstrates that you understand not only how to design a system but also how to validate its behavior when dependencies fail.

Learn More

Want to explore resilience and system design further?

• Chaos Engineering — ScaleDojo HLD Course
• Circuit Breakers, Retries, and Bulkheads — ScaleDojo
• Circuit Breaker and Bulkhead Patterns
• ScaleDojo — explore system-design learning and practice resources.

Final Thoughts

Chaos engineering helps you discover what happens when systems encounter unexpected failures, rather than assuming everything will always work as planned.

Start with a clear hypothesis, introduce one controlled failure, observe the results, and fix what you discover. Repeat the process as your system evolves.

The goal isn't to break your system. It's to make sure your system can handle being broken.

A rate limiter helps prevent this by controlling how frequently a client can access a service.

Rate limiting is commonly used to protect APIs, control resource consumption, reduce abuse, and maintain service availability. It is also a popular System Design interview topic because it combines algorithms, distributed systems, performance, and trade-offs.

In this article, we'll explore common rate-limiting algorithms, design a scalable architecture, implement a basic token bucket, and discuss the decisions that matter in a real interview.

1. Define the Requirements

Before choosing an algorithm, clarify what the rate limiter needs to accomplish.

Functional requirements

• Limit requests by user, API key, or IP address.
• Support configurable limits for different clients or endpoints.
• Allow requests within the configured limit.
• Reject excess requests with an appropriate response.
• Optionally provide information about remaining capacity and retry timing.

Non-functional requirements

• Low latency, because rate limiting happens on the request path.
• Scalability across multiple application servers.
• Reliable enforcement under concurrent requests.
• High availability and predictable failure behavior.

For example, imagine an API that allows each API key to make 100 requests per minute. Once the limit is reached, additional requests may receive an HTTP 429 Too Many Requests response.

The exact policy depends on the use case. A public search API, a payment endpoint, and an internal service may require different limits.

2. Understand the Main Rate-Limiting Algorithms

Different algorithms provide different levels of accuracy, complexity, and burst tolerance.

Fixed Window Counter

The fixed window algorithm counts requests within a predefined interval.

For example, a client may be allowed 100 requests between 10:00 and 10:01. The counter resets at the start of the next minute.

Advantages

• Simple to implement.
• Low memory usage.
• Easy to operate.

Limitation

A client could send 100 requests just before the window ends and another 100 immediately afterward. This creates a burst of 200 requests in a very short period.

Fixed windows are useful when simplicity matters and boundary bursts are acceptable.

Sliding Window Log

This algorithm records request timestamps and removes entries outside the rolling time window.

For a limit of 100 requests per minute, the system counts requests made during the previous 60 seconds before deciding whether to accept another request.

Advantages

• Accurate rolling-window enforcement.
• Avoids the fixed-window boundary problem.

Limitations

• Stores individual request timestamps.
• Can consume significant memory at high traffic volumes.

Use this approach when precise enforcement is important and the storage overhead is acceptable.

Token Bucket

The token bucket algorithm maintains a bucket with a maximum capacity. Tokens are replenished at a configured rate, and each request consumes one token.

For example, a bucket might hold 100 tokens and replenish at 10 tokens per second. A client can make a burst of requests while tokens are available, but its sustained request rate is controlled by the replenishment rate.

Advantages

• Supports controlled bursts.
• Controls the long-term request rate.
• Works well for many API rate-limiting use cases.

Limitations

• Requires maintaining bucket state.
• Distributed implementations need safe, atomic state updates.

Sliding Window Counter

This algorithm combines the previous window's count with the current window's count, weighting the previous count according to the portion of the time window that overlaps.

It reduces memory usage compared with storing every request timestamp, but produces an approximation of the rolling request count.

It can be a useful compromise between fixed-window simplicity and sliding-log precision.

3. Design the High-Level Architecture

A distributed rate limiter often sits at an API gateway or within request-processing middleware.

The request flow looks like this:

Client → API Gateway → Rate Limiter ↔ Redis → Application Services → Database / External Services

Here's how the components work:

1. Client: Sends a request through a web app, mobile app, or API client.
2. API Gateway: Handles request routing and identifies the applicable rate-limit policy.
3. Rate Limiter: Checks whether the client has exceeded its allowed request limit.
4. Redis: Stores shared counters or token-bucket state so multiple application instances can enforce a common limit.
5. Application Services: Process requests that pass the rate-limit check.
6. Database and External Services: Support the application after the request is allowed.

If the client exceeds the configured limit, the rate limiter can return an HTTP 429 Too Many Requests response instead of forwarding the request.

For example, a policy might allow 100 requests per minute per API key. The actual limit and algorithm depend on the application's requirements.

This architecture helps illustrate the main components, but it also introduces trade-offs. Redis adds network latency and becomes another dependency, while concurrent requests require atomic updates to prevent the limit from being exceeded.

4. Implement a Basic Token Bucket

The token bucket algorithm can be expressed with a few variables:

• capacity: The maximum number of tokens.
• tokens: The currently available tokens.
• refill_rate: The number of tokens added per second.
• last_refill: The last time the bucket was updated. When a request arrives, calculate how much time has passed, replenish the bucket, and check whether at least one token is available.

Here is simplified Python pseudocode:

def allow_request(bucket, now):
    elapsed = now - bucket.last_refill

    bucket.tokens = min(
        bucket.capacity,
        bucket.tokens + elapsed * bucket.refill_rate
    )

    bucket.last_refill = now

    if bucket.tokens >= 1:
        bucket.tokens -= 1
        return True

    return False

This illustrates the algorithm, but it is not a production-ready distributed implementation. A real implementation needs appropriate locking or atomic updates, safe time handling, persistent shared state where required, and a defined policy for storage failures.

For example, two concurrent requests must not both consume the same final token. A shared implementation might use an atomic Redis operation or a server-side script to calculate and update the bucket state as one logical operation.

5. Handle Distributed Systems Challenges

Choosing an algorithm is only part of designing a reliable rate limiter.

Concurrent requests

Two requests might read the same counter before either updates it. Both could be allowed even when only one should pass.

Use an atomic operation or an appropriate server-side script to make checking and updating the rate-limit state a single logical operation.

Store failures

If Redis becomes unavailable, should the application reject requests or temporarily allow traffic?

A fail-open policy favors availability but may allow excessive traffic. A fail-closed policy preserves enforcement but may block legitimate requests during an outage.

The right choice depends on the endpoint, security requirements, and consequences of overload.

Hot keys

A very popular client or endpoint may concentrate traffic on one key or storage partition. Monitor request distribution and latency, and consider partitioning or alternative approaches when necessary.

Monitoring

Track the following metrics:

• Allowed and rejected requests.
• Rate-limiter latency.
• Storage errors and timeouts.
• Unexpected traffic spikes.
• The number of requests exceeding configured limits.

These metrics help engineers detect abuse, diagnose performance problems, and adjust policies.

6. Compare the Main Trade-Offs

There is no universally correct choice. The design should reflect the expected traffic, acceptable latency, required accuracy, and operational constraints.

7. How to Approach This in a System Design Interview

If an interviewer asks you to design a rate limiter, use a structured approach rather than immediately choosing an algorithm.

Step 1: Clarify requirements. Determine whether limits apply per user, API key, IP address, or endpoint. Ask whether bursts are allowed.

Step 2: Estimate traffic. Establish request volume, peak load, and the number of clients whose state must be tracked.

Step 3: Select an algorithm. Explain why a token bucket, fixed window, or sliding window fits the requirements.

Step 4: Draw the architecture. Identify where the limiter runs, where state is stored, and how requests reach application services.

Step 5: Explore failure cases. Discuss concurrency, storage outages, hot keys, and multi-region behavior if relevant.

Step 6: Explain trade-offs. Justify the design and describe what you would change if traffic increased tenfold.

Practicing this complete process is more useful than memorizing a single architecture diagram. You can also use ScaleDojo's System Design challenges to practice applying these concepts to structured design problems.

Conclusion

A rate limiter may appear to be a simple request counter, but a production-ready design involves algorithm selection, distributed state, concurrency, failure handling, and observability.

Start with the requirements, choose the simplest algorithm that satisfies them, and introduce additional complexity only when the workload demands it.

For System Design interviews, focus not only on drawing the architecture but also on explaining the reasoning behind each decision. That is what turns a familiar concept into a defensible engineering design.

Below is a simplified design of how a ride-hailing app can solve it. It's a learning model, not Uber's exact internals, but the building blocks are the ones used in real systems.

The problem in one line

Take a rider's location, find nearby available drivers, pick a good one, and do it fast, even when a whole city is requesting rides at once.

Step 1: Drivers keep sending their location

Every online driver's app sends a GPS update every few seconds. Do the math with example numbers: if 1 million drivers are online and each sends an update every 4 seconds, that's 250,000 updates per second.

This system is write-heavy.

So updates should be tiny messages sent over a persistent connection (WebSocket or MQTT), and they should land in a message queue like Kafka first. The queue absorbs spikes so the location store never gets hammered directly.

Step 2: Keep only the latest location, in memory

For matching you only need to know where a driver is right now. Yesterday's location is useless here. So keep the latest position in a fast in-memory store such as Redis, with an expiry time. If a driver goes offline and stops sending updates, the entry expires and they quietly disappear from the pool.

The full location history can flow from the same queue into cheaper storage for analytics and trip records.

Step 3: Find nearby drivers without scanning everyone

Calculating the distance from the rider to every driver in the city is far too slow. The trick is a geospatial index.Divide the map into cells, and track which drivers are in which cell. Common choices:

• Geohash: encodes a location into a short string, and nearby places share a prefix

• Quadtree: splits busy areas into smaller squares automatically

• Hexagonal grids (like H3, open-sourced by Uber): every neighbour is the same distance away, which makes "nearby" cleaner

To find candidates, look up the rider's cell and its neighbours. That's a few quick lookups instead of a city-wide scan. Then rank the handful of candidates by real road distance or ETA from a maps service, not by straight-line distance. A driver across a river may be close on the map and far by road.

Step 4: Offer the ride, safely

Send the request to the best driver with a short timeout. If they decline or don't answer, move to the next one.ScaleDojo | Blog Articles | Draft for publishing

One thing must never happen: two riders getting the same driver. Make the driver's state change atomic.Move a driver from available to offered using a compare-and-set or a lock in the store, so only one request can win.

• Available: online and free to take a ride. Next state: Offered.

• Offered: a request has been sent and we are waiting for the driver to respond. Next state: On trip if accepted, or back to Available if declined or timed out.

• On trip: passenger on board. Next state: Available when the trip ends.

Step 5: Demand and surge pricing

When more riders than drivers are in one area, prices can rise to balance supply and demand. A stream-processing job counts requests and available drivers per cell over a short time window and publishes a multiplier. It's a nice extension to mention once the core design is solid.

Scaling and failure

• Shard by city or region. A rider in Mumbai never needs drivers in Delhi.

• Keep services stateless behind load balancers so you can add more of them quickly.

• Driver loses signal? The expiry removes them, and the app re-registers when it reconnects.

• Hot spots (say, a stadium after a match) can overload one cell. Use smaller cells or split busy ones dynamically.

• Matching service crashes? State lives in the store, so another instance can pick up and retry.

The trade-offs to mention in an interview

• Update frequency vs cost: more frequent updates mean fresher data and a heavier write load

• Straight-line distance vs ETA: faster to compute vs more accurate

• Strong consistency on driver assignment vs raw speed: here, correctness wins

Want to practice this one hands-on? ScaleDojo has a detailed walkthrough: Design Uber: Real-Time Ride Matching at Global Scale

What is an LLD round, really?

In a low level design interview you are asked to design the classes, interfaces, and relationships for a small, well-defined system. Unlike high level design, there is no talk of load balancers or sharding. The interviewer wants to see clean object-oriented thinking: sensible entities, clear responsibilities, and code structure that survives a changing requirement.

Typically you get 40 to 60 minutes. You are expected to clarify requirements, identify core entities, define their relationships, write key class skeletons or method signatures, and sometimes implement one critical flow in code.

The 10 LLD interview questions

1. Parking lot system The classic. Model vehicles, spots of different sizes, tickets, entry and exit gates, and pricing. Interviewers like to ask: what changes if we add electric vehicle charging spots? A Strategy pattern for pricing and a clean spot-allocation interface usually impress.

2. Elevator system Tests state management and scheduling. You need an elevator state (idle, moving up, moving down), a request queue, and a scheduling algorithm. Be ready to discuss multiple elevators and how a controller assigns requests.

3. Splitwise (expense sharing) Focus on users, groups, expenses, and different split types (equal, exact, percentage). The Strategy pattern fits split types naturally. Expect a follow-up on simplifying debts so fewer transactions are needed.

4. Library management system A good warm-up: books, copies, members, loans, fines, and reservations. The trap is confusing a Book (the title) with a BookCopy (the physical item). Interviewers watch for exactly this.

5. Rate limiter (class design) Here you design the classes behind algorithms like token bucket or sliding window. Define a common RateLimiter interface and plug different algorithms in. This one tests whether you can keep the design extensible.

6. LRU cache A hash map plus a doubly linked list. It looks like a data structure question but it is also an LLD question: how do you make the eviction policy swappable (LRU, LFU, FIFO)?

7. Tic-tac-toe or chess Board, players, moves, and win conditions. For chess, the interesting part is piece behavior: each piece type knows its own valid moves. Polymorphism is the star here.

8. Hotel booking system Rooms, room types, reservations, availability search, and payments. Pay attention to concurrency: two people booking the last room at once is a favorite follow-up.

9. Notification service Channels (email, SMS, push), templates, user preferences, and retries. Observer and Factory patterns both have natural homes here.

10. ATM machine A state machine problem: card inserted, PIN entered, transaction selected, cash dispensed. The State pattern is the textbook answer, and the interviewer will likely ask about failure handling mid-transaction.

What interviewers actually check

● Requirement clarification. Do you ask questions before designing, or do you start coding immediately?

● Entity identification. Are your classes the real nouns in the problem, and does each have one clear responsibility?

● Relationships. Do you choose composition over inheritance where it makes sense, and can you explain why?

● Extensibility. When they say, "Now add a new vehicle type," does your design bend or break?

● Pattern usage. Patterns used where they fit are a plus. Patterns forced everywhere are a red flag.

● Code quality. Meaningful names, small methods, and no giant God class that does everything.

A 40-minute method for any LLD question

1 Minutes 0 to 5: Requirements. Ask who the users are, what the core features are, and what is out of scope. Write the agreed requirements down.

2 Minutes 5 to 12: Entities and relationships. List nouns, group them, and sketch how they relate. Decide what is a class, an interface, or an enum.

3 Minutes 12 to 25: Class skeletons. Write fields and method signatures for the key classes. Focus on public behavior, not private details.

4 Minutes 25 to 35: One critical flow in code. Pick the most important operation (for example, parking a vehicle) and implement it end to end.

5 Minutes 35 to 40: Extend. Add a new requirement to your own design and note what has to change. If the answer is "a lot," refactor.

Common mistakes that cost offers

● Jumping straight into code without clarifying requirements.

● Creating one huge class that holds all the logic.

● Over-engineering with five design patterns when two classes would do.

● Ignoring edge cases like invalid input, full capacity, or concurrent access.

● Never discussing trade-offs or alternatives.

How to practice so it actually sticks

Reading solutions teaches recognition, not recall. You nod along, think "that makes sense," and then freeze in the real interview. The fix is active practice: pick one problem, set a 40-minute timer, and design from a blank page. Only then compare your answer to a reference solution and note the gaps.

Do this daily with a new problem for two weeks and you will feel the difference. Platforms with hands-on labs make this easier because you can design, get feedback, and iterate in one place. If you want a structured way to do it, ScaleDojo has practice LLD interview questions hands-on where each problem is broken into requirement, design, and implementation steps.

Final tip

Whatever method you use, always end a practice session by adding one new requirement to your own design and watching what breaks. That single habit trains the exact skill interviewers test most: designing code that can change. If you want more reps on realistic problems, try the low level design labs and work through two or three questions a week

Why reading LLD write-ups doesn't build the skill

You can read ten solutions to "design a parking lot" and walk away able to explain why each class exists. That's recognition, not production. The actual interview skill is starting from a blank whiteboard and building that same structure yourself, under time pressure, without a reference answer to check against as you go. Those are different skills, and only one of them gets tested live.

The practice method that actually worked

1. Time-box the blank-page struggle. Pick a problem, set a 20-minute timer, and start from nothing-no notes, no reference solutions open. The discomfort of not knowing where to start is the exact discomfort a real interview produces, and getting used to sitting in it matters more than any specific pattern you memorize.

2. Narrate your reasoning out loud, even alone. LLD interviews are graded on reasoning, not just the final class diagram. Practicing silently teaches you to produce a diagram. Practicing out loud teaches you to explain why each class and relationship exists, which is the actual thing being evaluated.

3. Deliberately apply one principle at a time. Instead of trying to remember all of SOLID simultaneously, run through a design once checking only for single responsibility violations, then again checking only for whether your abstractions would survive a specific extension. This catches more than trying to hold every principle in your head at once from the start.

4. Get pushed back on, not just checked. This is the piece hardest to replicate alone. Reading a checklist against your own design tells you if something's technically wrong. It doesn't replicate an interviewer asking "why did you choose composition here instead of inheritance?" and expecting you to defend it on the spot.

A rotation that covers the real range

Don't drill the same problem type repeatedly. Rotate across:

• A "model the real world" problem (parking lot, elevator, library system)

• A concurrency-flavored problem (rate limiter, thread-safe cache)

• A "messy requirements" problem (splitwise-style expense splitting, where clarifying questions matter as much as the classes)

• An extension-focused problem (chess game, card game framework) where you have to justify why your design would survive a plausible new requirement

Where I closed the "get pushed back on" gap

Practicing alone gets you most of the way, but the missing piece-someone specifically telling you where a senior engineer would flag your design-is hard to replicate solo. I used ScaleleDojo's LLD Schema Design Lab for that part specifically: you build the design on an interactive canvas, submit it, and get scored feedback on the actual decisions, which is closer to the live-pushback experience than self-checking against a reference solution.

If your LLD prep has mostly been reading solutions and nodding along, switching to timed, narrated, blank-page practice-even without a person watching-is the single change most likely to show up as improvement in your next real interview.

Here's a breakdown of the principles that come up constantly, and what interviewers are actually listening for when they ask about them.

Encapsulation: more than "make fields private"

Most candidates know encapsulation means hiding internal state. Fewer can explain why it matters in a specific design. If your Order class exposes a public items list that anyone can mutate directly, you've lost control over invariants like "an order can't be empty" or "total must match the sum of item prices." Encapsulation isn't a style rule — it's how you protect the rules your system depends on.

What interviewers listen for: can you point to a specific invariant your encapsulation is protecting, not just say "it's good practice."

Abstraction: hiding complexity, not hiding classes

Abstraction gets confused with encapsulation constantly. Abstraction is about exposing a simple interface while hiding how the work actually happens. A PaymentProcessor interface with a charge(amount) method is abstraction — the caller doesn't need to know if it's calling Stripe, Razorpay, or a mock in tests.

What interviewers listen for: whether your abstractions are stable. If adding a new payment provider means changing code all over your system instead of just adding one new class, the abstraction was drawn in the wrong place.

The Single Responsibility Principle, applied honestly

Everyone can recite "a class should have one reason to change." Almost nobody applies it under interview pressure, because the honest version requires you to notice when you're about to violate it — usually while you're mid-design and it's tempting to just add one more method to a class that already does too much.

A classic tell: a User class that handles authentication, sends emails, and calculates loyalty points. Three reasons to change, three responsibilities, three future bugs waiting to happen when one gets modified and breaks the others.

Open/Closed: designing for the extension you haven't been asked for yet

This is the principle candidates struggle with most, because it requires anticipating change without over-engineering for hypothetical futures that never arrive. The skill isn't "add every extension point possible" — it's identifying the one or two axes your interviewer's problem is actually likely to extend along.

For a parking lot problem, vehicle types are an obvious extension axis (cars, motorcycles, trucks). Pricing strategies are another. If your class hierarchy makes adding a new vehicle type require editing five existing classes, that's a real signal, not a nitpick.

Liskov Substitution: the one that silently breaks systems

This principle rarely gets asked about directly, but it gets violated constantly, and interviewers notice. If a subclass changes the behavior a caller expects from the parent class — a Square that extends Rectangle but breaks setWidth/setHeight independence — you've created a class that's technically a subtype but functionally a landmine.

What interviewers listen for: whether you catch this yourself when sketching an inheritance hierarchy, without being prompted.

Composition over inheritance: knowing when to reach for which

Interviewers love asking "would you use inheritance or composition here?" because there's rarely one right answer -there's a right answer for the specific constraints of the problem. Inheritance models a true "is-a" relationship with shared behavior. Composition models "has-a" and tends to age better when requirements shift, because you're not locked into a rigid hierarchy.

A Car "is-a" Vehicle -inheritance is reasonable. A Car "has-a" Engine -composition, not inheritance, because a car doesn't behave like an engine, it uses one.

Why reading these definitions doesn't make them stick

You can memorize every principle above and still freeze the first time you're asked to design something from scratch, because recognizing a principle in a textbook example and applying it correctly to a novel problem, live, are different skills. The gap only closes with repetition-designing schemas and class hierarchies yourself, getting told specifically where a principle was violated, and doing it again.

That loop is what ScaleDojo's LLD Schema Design Lab is built around progressive missions on an interactive canvas where you apply these principles to real problems and get scored, specific feedback instead of a vague pass/fail.

If OOD principles have been the part of your LLD prep you've been reading about instead of practicing, that's usually the highest-leverage gap to close before your next interview.

"Okay, now how would you make this thread-safe?"

Concurrency isn't a side topic in system design interviews anymore. It's one of the areas that tests whether you can think beyond class diagrams and consider what happens when multiple requests interact with the same system at the same time.

I've gone through enough LLD discussions both as a candidate and while reviewing mock interviews to notice that the same handful of concurrency questions show up repeatedly, even when the underlying design problem is completely different. Here's a breakdown of what those questions are, why they matter, and how to approach them instead of simply memorizing answers.

Why Concurrency Questions Keep Showing Up in LLD Rounds

LLD interviews are meant to simulate real engineering problems, and real applications are almost never single-threaded. The moment you design something like an inventory system, a booking engine, or a cache, you're implicitly deciding how multiple users or processes will interact with the same piece of shared state.

Interviewers care less about whether you can define a mutex and more about whether you notice shared state in the first place.

For example, imagine a ticket-booking system where only one seat is left, and two requests arrive almost simultaneously:

if (seatsAvailable > 0) {
    seatsAvailable--;
    bookSeat();
}

At first glance, the code looks reasonable. But what happens when two threads execute it at the same time? Both threads might see seatsAvailable = 1, both pass the condition, and both attempt to book the same seat. That's a race condition and catching it yourself, before the interviewer has to point it out, is an important LLD skill.

The Concurrency Questions That Come Up Again and Again

1. "Where's the Race Condition in This Design?"

This is often one of the first concurrency questions you'll encounter. Suppose you're designing a ticket-booking system and have:

if (seatsAvailable > 0) {
    seatsAvailable--;
}

The problem is the check-then-act sequence -two threads can perform the check before either one performs the decrement. A similar problem can happen with shared collections, counters, caches, inventory, or account balances.

A useful habit during LLD interviews is to ask yourself:

• What data is shared, and which fields are mutable?

• Can multiple threads modify them?

• Is there a check-then-act operation?

• Is a shared collection being read and written concurrently?

• What happens if two requests arrive at exactly the same time?

You don't want concurrency to be something you think about only after the interviewer asks.

2. "How Would You Make This Method Thread-Safe?"

Once you've identified a race condition, the obvious follow-up is how you'd fix it -and this is where understanding the trade-offs matters.

synchronized is the simplest solution:

public synchronized void bookSeat() {
    // booking logic
}

It's straightforward and useful when the critical section is small and contention isn't a major concern. However, locking a large method can unnecessarily limit concurrency.

ReentrantLock gives you more control:

lock.lock();
try {
    // critical section
} finally {
    lock.unlock();
}

It provides capabilities like timed lock acquisition, fairness policies, and interruptible locking -at the cost of extra complexity and the responsibility of correctly releasing the lock.

Atomic variables like AtomicInteger, AtomicLong, and AtomicReference provide thread-safe operations on individual variables without explicitly locking the whole method:

AtomicInteger counter = new AtomicInteger();
counter.incrementAndGet();

The important interview skill isn't just knowing these APIs -it's explaining why one approach fits the particular problem better than the others.

3. "What's the Difference Between volatile and Atomic Operations?"

This is another area where candidates often get confused. volatile primarily provides visibility guarantees between threads:

private volatile boolean running = true;

When one thread changes running, other threads can observe the updated value. But volatile doesn't make compound operations atomic. Consider:

counter++;

This involves reading the value, adding one, and writing the result three steps that multiple threads can interfere with. An atomic counter avoids this:

AtomicInteger counter = new AtomicInteger();
counter.incrementAndGet();

A useful way to remember the distinction: visibility is not the same as atomicity.

4. "How Would You Design a Thread-Safe Singleton?"

The Singleton pattern is a common LLD exercise because it touches several concurrency concepts at once. One implementation uses double-checked locking:

public class Singleton {

    private static volatile Singleton instance;

    private Singleton() {}

    public static Singleton getInstance() {
        if (instance == null) {
            synchronized (Singleton.class) {
                if (instance == null) {
                    instance = new Singleton();
                }
            }
        }
        return instance;
    }
}

The interesting part isn't memorizing the implementation it's answering when the interviewer asks, "Why does this need volatile?" That's where you need to understand visibility and instruction reordering rather than just reproducing a code snippet.

5. "How Would You Implement Producer-Consumer?"

Producer-consumer problems test something beyond mutual exclusion: coordination between threads. A producer generates work, a consumer processes it, and a queue sits between them:

Producer
   |
   v
[ Queue ]
   |
   v
Consumer

You can implement coordination manually using wait() and notify(), but Java also provides higher-level abstractions like BlockingQueue:

BlockingQueue<Task> queue = new ArrayBlockingQueue<>(100);

Now you can reason about questions like: What happens when the queue is full? What happens when there's nothing to consume? How do producers and consumers coordinate without unsafe access to shared state? An interviewer may also ask about the difference between notify() and notifyAll() -the important part is understanding the coordination model, not memorizing individual methods.

6. "How Does Deadlock Happen, and How Would You Avoid It?"

Consider two threads:

Thread A: Lock A → waiting for Lock B
Thread B: Lock B → waiting for Lock A

Neither thread can continue. This is a classic deadlock scenario, and a common way to reason about it is through its four necessary conditions: mutual exclusion, hold and wait, no preemption, and circular wait.

One common prevention strategy is consistent lock ordering. If your application always acquires Lock A before Lock B-instead of some code acquiring A→B and other code acquiring B→A -you eliminate the circular wait. During an interview, don't just say "use lock ordering." Explain which deadlock condition your fix actually breaks.

7. "ExecutorService vs Raw Threads -What's the Difference?"

An interviewer may ask why you wouldn't simply create a new thread for every task. The honest answer is that creating an uncontrolled number of threads gets expensive fast as workload increases. Java's ExecutorService and thread pools manage worker threads more systematically:

Incoming Tasks
      |
      v
  Thread Pool
 [T1] [T2] [T3] [T4]

Instead of creating a new thread per task, a pool reuses a controlled number of worker threads. You should also understand why different executor configurations exist and what kind of workload each fits - the real question isn't "do you know ExecutorService," it's "why would you choose it here?"

8. "When Would You Use a Reader-Writer Lock?"

Not every shared resource has the same read/write pattern. Imagine a configuration cache where reads vastly outnumber writes. Using a single exclusive lock for every operation unnecessarily restricts concurrent readers. Java's ReentrantReadWriteLock separates read and write access, letting multiple readers proceed concurrently while writes still require exclusivity.

That doesn't automatically make it the right choice, though. The interviewer may push further: "Why is this better than a normal lock for this specific workload?" Your answer should connect the synchronization mechanism to the actual access pattern, not just name the API.

9. "Design a Thread-Safe Rate Limiter"

This problem combines several concurrency concepts at once. Suppose you want to allow 100 requests per minute per user, and hundreds of requests arrive simultaneously. Now you need to think about shared counters, atomic updates, time windows, concurrent access, synchronization, cleanup, throughput, and correctness-all together. A simple counter isn't enough.

The interviewer will likely keep pulling the thread: What happens if two requests arrive at exactly the same time? What happens if you have multiple application servers? Would an in-memory implementation still work? That's what makes rate limiting such a useful LLD problem- a relatively simple requirement quickly opens into concurrency, scalability, and distributed systems trade-offs.

How to Prepare for Concurrency Questions

Reading a list of questions helps you recognize common patterns, but reading alone isn't enough. Try implementing small systems and deliberately testing them under concurrent access:

• Thread-safe counter - build one and hit it with updates from multiple threads.

• Ticket booking -create a booking system and try booking the same seat concurrently.

• LRU cache -implement one and think through concurrent reads and writes.

• Producer-consumer -build a bounded queue and test what happens when producers outrun consumers.

• Rate limiter- implement a simple one and fire simultaneous requests at it.

After every implementation, ask yourself: "What happens if 100 threads execute this code at the same time?" That single question can change the way you approach LLD problems.

The Real Skill Isn't Memorizing Concurrency APIs

It's easy to memorize synchronized, volatile, AtomicInteger, ReentrantLock, deadlock conditions, ExecutorService, and BlockingQueue. But an LLD interview isn't a multiple-choice test. You're given a system, and then the interviewer changes the conditions -two users arrive simultaneously, a resource becomes shared, traffic increases, a method gets called concurrently, the application gets deployed across multiple servers.

That's when you need to reason about what actually happens. The real skill is being able to identify shared state, synchronization requirements, race conditions, and trade-offs while you're designing the system, not after.

From Knowing to Building

Reading about concurrency gives you the vocabulary. Building concurrent systems gives you the intuition -and that difference matters a lot in LLD preparation.

If you want hands-on practice alongside structured LLD problems, ScaleDojo has practical system-design exercises covering areas like LLD, HLD, API design, and GenAI systems.

The goal isn't to memorize every concurrency API. It's to develop the habit of asking, "What happens when multiple things happen at the same time?" Once that question becomes automatic, concurrency gets a lot easier to reason about during LLD interviews.

I want to walk through how to actually think about this, structured the way I'd want it drilled into me if I were prepping: not as trivia to memorize, but as a decision tree you can reason through live.

Start with who's calling the API

Before picking a protocol, the first system design interview question you should be asking yourself is: who are the clients? A mobile app on a flaky network has different needs than an internal service making thousands of calls per second to another internal service. Naming the client type out loud, before you touch the whiteboard, is the single easiest way to sound like you know what you're doing.

REST: correct by default, wrong when you use it as a crutch

REST is the safe pick, and it's usually the right one. Resource-based systems (users, orders, posts) map naturally onto it, it's cacheable at the HTTP layer, and every tool on earth understands it.

Where it breaks down: deeply nested data. If a client needs a post, its author, its comments, and each comment's author, REST forces you into either multiple round trips or an endpoint that dumps way more data than the client asked for. If your system has that shape, be ready for the interviewer to push on it-and have an answer beyond "REST is fine."

GraphQL: solves over-fetching, creates new problems

GraphQL exists specifically to let the client ask for exactly the fields it needs, in one request. It's a strong answer for systems with multiple client types pulling different slices of the same data.

But proposing GraphQL without acknowledging its costs is one of the fastest ways to lose points in a system design interview. Be ready for:

• Caching -a single GraphQL endpoint breaks standard HTTP caching

• The N+1 query problem -a poorly written query can quietly hammer your database

• Rate limiting -every request hits the same endpoint, so limiting by route doesn't work anymore

gRPC: the one people forget to practice

gRPC rarely gets asked about directly, but it's often the right call for service-to-service communication -not client-facing APIs. High-frequency internal calls (an order service hitting an inventory service constantly) benefit from gRPC's binary protocol and HTTP/2 multiplexing in a way REST and GraphQL can't match.

The strongest answer usually isn't "I'd use X." It's layering: REST or GraphQL at the edge for external clients, gRPC internally between services. Saying that out loud signals you're thinking in layers, not reaching for one tool and forcing it everywhere.

A framework worth practicing until it's automatic

1. Who are the clients -web, mobile, third-party, internal services?

2. How nested is the data -flat leans REST, deeply nested and multi-client leans GraphQL?

3. What's the call frequency and latency profile -high-frequency internal leans gRPC?

4. What do you need at the edge -caching, rate limiting, and auth are easier with REST or GraphQL?

Why this only sticks with actual practice

You can read a breakdown like this one and nod along, but the trade-offs only really click once you've had to defend a specific choice out loud, with someone pushing back on it. That's the gap between knowing the difference and being ready to justify one live.

This is exactly what pushed me to build through ScaleDojo's API Design Lab -50 progressive levels that go from basic endpoint structure to pagination, nested resources, and error response modeling, with an AI review that tells you specifically what a senior engineer would have done differently. It's part of a broader set of labs covering HLD, LLD, and GenAI/RAG system design too, if API design isn't the only gap in your prep.

If system design interview questions on API design are the part of your prep you keep skipping, this is the one to stop skipping.