Wait, why can't I just use an AtomicInteger in a singleton?

If you're running a single instance of your application on one server, an AtomicInteger or a ConcurrentHashMap is perfect. It's fast and simple. But the second you scale to two or more nodes behind a load balancer, that strategy falls apart.

Imagine you've set a limit of 100 requests per minute. User A hits Server 1, then Server 2, then Server 1 again. If each server is tracking the count locally, User A could potentially make 100 requests per server. Suddenly, your "100 request limit" is actually 1,000 requests if you have ten nodes. To solve this, we need a shared state—something external to the JVM. Redis is the industry standard here because it's an in-memory store that's fast enough to handle the overhead of every single request checking in.

If I fetch the count and then increment it, won't I have a race condition?

Exactly. This is the classic "read-modify-write" problem. If two requests hit two different servers at the exact same millisecond, they both read the value 49, both think it's under the limit of 50, and both increment it to 50. You've just let 51 requests through.

I usually handle this using Lua scripts in Redis. Redis guarantees that a Lua script runs atomically. Instead of bringing the data to Scala, we send the logic to the data. Here is how I'd structure a simple fixed-window limiter in Scala:

case class RateLimitResult(allowed: Boolean, remaining: Long)

def checkLimit(userId: String, limit: Long, windowSeconds: Int): RateLimitResult = {
  val luaScript = 
    """
    local current = redis.call('INCR', KEYS[1])
    if current == 1 then
      redis.call('EXPIRE', KEYS[1], ARGV[1])
    end
    return current
    """.stripMargin

  // We use the userId and the current minute as the key to create a fixed window
  val windowKey = s"rate_limit:$userId:${System.currentTimeMillis() / 60000}"
  val count = redisClient.eval(luaScript, Seq(windowKey), Seq(windowSeconds.toString)).toLong

  RateLimitResult(count <= limit, limit - count)
}

By using INCR and EXPIRE inside a script, we ensure that the increment and the TTL (Time To Live) are set without any other request sneaking in between.

Fixed windows feel clunky—how do I stop "bursts" at the edge of a minute?

You've hit on the biggest weakness of fixed windows. If a user sends 100 requests at 10:00:59 and another 100 at 10:01:01, they've technically stayed within the "per minute" limit, but they just slammed your API with 200 requests in two seconds. It's a great way to crash a database.

To fix this, I use a Sliding Window Log. Instead of a single counter, we store every request timestamp in a Redis Sorted Set (ZSET). When a request comes in, we remove all timestamps older than the current window, count what's left, and then decide if we allow the new request.

def checkSlidingWindow(userId: String, limit: Long, windowMillis: Long): Boolean = {
  val now = System.currentTimeMillis()
  val windowStart = now - windowMillis
  val key = s"sliding_limit:$userId"

  // Use a Redis pipeline or Lua script to make this atomic
  redisClient.multi {
    // 1. Remove timestamps older than the current window
    redisClient.zremrangebyscore(key, 0, windowStart)
    // 2. Count elements remaining in the set
    val count = redisClient.zcard(key)
    
    if (count < limit) {
      // 3. Add the current request timestamp
      redisClient.zadd(key, now.toDouble, now.toString)
      true
    } else {
      false
    }
  }
}

It's a bit more expensive in terms of memory because you're storing every timestamp, but for most high-value APIs, the precision is worth the cost. If you're dealing with millions of users and billions of requests, you might want to look into the Generic Cell Rate Algorithm (GCRA), but that's a deeper rabbit hole for another time.

📋 Practical Task

Implement a Redis-backed Sliding Window Pruner

You are tasked with optimizing a distributed rate limiter. The current implementation is leaking memory because it adds timestamps to the Redis Sorted Set but doesn't efficiently clean them up.

Write a Scala function pruneAndCheck that takes a userId, a limit, and a windowMillis. The function must:

• Use a Redis Sorted Set (ZSET) where the score and value are both the current timestamp.
• Remove all entries older than currentTime - windowMillis.
• Check if the number of remaining entries is less than the limit.
• If allowed, add the current timestamp to the set and return true. Otherwise, return false.

Constraints: Assume you have a redis client object available with methods zremrangebyscore(key, min, max), zcard(key), and zadd(key, score, member).