All work Case study 01 / 04

FluxLimit

Distributed API rate limiting for Java 21 — the same five-line API from a single JVM to a Redis-backed fleet.

Type
Open-source library
Status
0.1.0 on Maven Central
License
Apache-2.0
Stack
Java 21 · Redis · Lua · Lettuce · Spring Boot · Micrometer · JMH
Links
Source Maven
01 — Context

Rate limiting looks trivial until the second server.

One JVM can count requests in a map. A fleet needs one shared decision per request — atomic, fast, and correct while the servers disagree about what time it is.

Run one in-memory limiter per server and N servers enforce N times your limit. The Java options each cover half of the problem: embedded libraries don’t share state, and wiring Redis up yourself means Lua, clock skew and failure semantics all become your problem.

02 — Approach

Run the algorithm inside the store.

The decision happens where the state lives. In memory that is one atomic map operation; on Redis it is one Lua script, in one round trip, timestamped by the Redis server’s own clock.

That removes the read-modify-write race, and it removes clock-skew handling entirely, because application clocks are never consulted. Moving from one JVM to a fleet is one builder line, with identical semantics. The whole API:

RateLimiter limiter = RateLimiter.builder()
    .tokenBucket(100, Duration.ofMinutes(1))   // or .slidingWindow(...)
    .build();                                  // in-memory by default

RateLimitResult result = limiter.tryAcquire("user:" + userId);
if (!result.allowed()) {
    // 429; result.retryAfter() fills Retry-After
}

tryAcquire never blocks and never throws for a denial — a denial is a result. In Spring Boot it is an annotation, @RateLimit(key = "#{principal.name}"), with standard RateLimit-* and Retry-After headers on every limited response.

03 — Architecture

One interface where the hard part lives.

FluxLimit architecture Application code calls RateLimiter, which validates and prefixes keys, applies the failure policy and delegates one atomic state transition to the RateLimiterStore interface. Two stores implement it: InMemoryStore, a ConcurrentHashMap compute per key inside one JVM, and RedisStore, a Lua script run with EVALSHA in one round trip against a shared Redis that holds the authoritative state and the clock. Application codetryAcquire(key) · @RateLimit RateLimitervalidate · prefix · failure policy · cooldown RateLimiterStore SPIone atomic state transition per check InMemoryStoreConcurrentHashMap.compute() RedisStoreLua via EVALSHA · one round trip Single JVM~4.9M checks/s, in-process Shared Redisauthoritative state + clock (TIME) no read-then-write anywhere standalone · Sentinel · Cluster
Fig. 1Every store runs the algorithm inside its own atomicity primitive, so two servers can never spend the same permit.
  • RateLimiter owns no business logic. It validates, prefixes keys, applies the failure policy, and delegates one atomic state transition to the store.
  • The store is the single SPI. Everything distributed-systems-hard lives behind RateLimiterStore; a new backend is a new module, with no core changes.
  • Modules exist for dependency isolation. fluxlimit-core has no runtime dependencies at all; fluxlimit-redis adds Lettuce; the Spring Boot starter adds Spring. A plain-Java user never pulls Spring.
04 — Engineering

The details that keep it correct.

  1. Integer math, no float drift

    Token counts are stored as scaled integers — micro-tokens — never floats. A bucket refilling one token every two seconds admits exactly one request every two seconds, forever.

  2. A sliding window that errs toward no

    Two counters per key, with the previous window weighted by its remaining overlap. The estimate’s error is bounded — at most 2× the limit, and only under an adversarial burst pattern — and it deliberately rounds toward denying.

  3. One clock

    The Lua scripts read Redis TIME. Application server clocks never enter the decision, so there is no skew to handle anywhere.

  4. Memory that cleans up after itself

    Every Redis key carries a PEXPIRE to the moment its state becomes indistinguishable from absent, so idle keys delete themselves. The in-memory store sweeps stale keys once a minute on one daemon thread — the only background thread, and never on the request path.

  5. An outage costs one timeout per second, not per request

    If Redis is unreachable or slower than the 50 ms command timeout, the limiter answers from the configured failure policy, flags the result degraded(), and bypasses the store for a one-second cooldown before re-probing.

  6. Tested for exactness

    Property-based tests (jqwik) over generated schedules. Sixteen threads hammering one key against a frozen clock, where admitted must equal capacity exactly, not approximately. Testcontainers suites that flush the script cache to force NOSCRIPT recovery, and kill Redis mid-test to exercise the degraded path.

05 — Tradeoffs

What I chose, and what it cost.

Why a Lua script, instead of read, decide, write back?
A read-then-write from two servers can spend the same permit twice. Inside Redis the script runs atomically, in one round trip, with no retries, transactions or locks. The cost: both algorithms exist twice — Java for memory, Lua for Redis — and the same semantics have to be held in both.
Why can’t users plug in their own algorithm?
A user-supplied algorithm couldn’t execute atomically inside Redis. So the algorithm set is sealed, and the honest extension point is the store: implement RateLimiterStore in a new module. New algorithms, like GCRA, arrive as new config members.
Fail open or fail closed?
Either — chosen explicitly. Allowing traffic while the store is down keeps the API up but leaves limits unenforced; denying protects whatever is downstream at the cost of availability. Allow is the default, deny is one property away, and in both cases the result is flagged degraded, counted, and logged once per episode. Never silent.
Why 192 bytes per check, and not zero?
The allocation is the key-prefix concatenation plus two immutable result records — steady-state and young-gen only. Chasing true zero would mean flyweight results and key interning: complexity the numbers don’t yet justify.
Why not just use Bucket4j?
Often you should — it is mature and has more backends. FluxLimit’s niche is per-key limits shared across a fleet in one atomic round trip, with a sliding window as well as a token bucket. Multi-region is deliberately out of scope: per-region independent limits; true global coordination is a different product.
06 — Results

Measured, with the caveats left in.

JMH · JDK 21Result
In-memory, single thread~4.9M checks/s · ~0.2 µs/op
In-memory, 8 threads on one hot key~3.5M checks/s
8 threads contended: FluxLimit (keyed) vs Bucket4j LocalBucket~4.0M vs ~2.4M ops/s
Allocation per check192 B
Redis on loopback, p50 / p99, one client0.25 ms / 2.2 ms
Design target≥ 100k checks/s

JMH 1.37, one fork, consumer hardware: treat absolute values as ±30%; the shapes were stable across runs. Bucket4j and Resilience4j are faster single-threaded — they are measured unkeyed, with no map lookup — so the contended column is the meaningful one. Redis numbers are dominated by loopback round-trip time, which is the claim being verified: one round trip per check. Published to Maven Central as 0.1.0; the API may still move before 1.0.