Rate limiting looks trivial until the second server.
One JVM can count requests in a map. A fleet needs one shared decision per request — atomic, fast, and correct while the servers disagree about what time it is.
Run one in-memory limiter per server and N servers enforce N times your limit. The Java options each cover half of the problem: embedded libraries don’t share state, and wiring Redis up yourself means Lua, clock skew and failure semantics all become your problem.
Run the algorithm inside the store.
The decision happens where the state lives. In memory that is one atomic map operation; on Redis it is one Lua script, in one round trip, timestamped by the Redis server’s own clock.
That removes the read-modify-write race, and it removes clock-skew handling entirely, because application clocks are never consulted. Moving from one JVM to a fleet is one builder line, with identical semantics. The whole API:
RateLimiter limiter = RateLimiter.builder() .tokenBucket(100, Duration.ofMinutes(1)) // or .slidingWindow(...) .build(); // in-memory by default RateLimitResult result = limiter.tryAcquire("user:" + userId); if (!result.allowed()) { // 429; result.retryAfter() fills Retry-After }
tryAcquire never blocks and never throws for a denial — a denial is a result. In Spring Boot it is an annotation, @RateLimit(key = "#{principal.name}"), with standard RateLimit-* and Retry-After headers on every limited response.
One interface where the hard part lives.
RateLimiterowns no business logic. It validates, prefixes keys, applies the failure policy, and delegates one atomic state transition to the store.- The store is the single SPI. Everything distributed-systems-hard lives behind
RateLimiterStore; a new backend is a new module, with no core changes. - Modules exist for dependency isolation.
fluxlimit-corehas no runtime dependencies at all;fluxlimit-redisadds Lettuce; the Spring Boot starter adds Spring. A plain-Java user never pulls Spring.
The details that keep it correct.
Integer math, no float drift
Token counts are stored as scaled integers — micro-tokens — never floats. A bucket refilling one token every two seconds admits exactly one request every two seconds, forever.
A sliding window that errs toward no
Two counters per key, with the previous window weighted by its remaining overlap. The estimate’s error is bounded — at most 2× the limit, and only under an adversarial burst pattern — and it deliberately rounds toward denying.
One clock
The Lua scripts read Redis
TIME. Application server clocks never enter the decision, so there is no skew to handle anywhere.Memory that cleans up after itself
Every Redis key carries a
PEXPIREto the moment its state becomes indistinguishable from absent, so idle keys delete themselves. The in-memory store sweeps stale keys once a minute on one daemon thread — the only background thread, and never on the request path.An outage costs one timeout per second, not per request
If Redis is unreachable or slower than the 50 ms command timeout, the limiter answers from the configured failure policy, flags the result
degraded(), and bypasses the store for a one-second cooldown before re-probing.Tested for exactness
Property-based tests (jqwik) over generated schedules. Sixteen threads hammering one key against a frozen clock, where admitted must equal capacity exactly, not approximately. Testcontainers suites that flush the script cache to force
NOSCRIPTrecovery, and kill Redis mid-test to exercise the degraded path.
What I chose, and what it cost.
- Why a Lua script, instead of read, decide, write back?
- A read-then-write from two servers can spend the same permit twice. Inside Redis the script runs atomically, in one round trip, with no retries, transactions or locks. The cost: both algorithms exist twice — Java for memory, Lua for Redis — and the same semantics have to be held in both.
- Why can’t users plug in their own algorithm?
- A user-supplied algorithm couldn’t execute atomically inside Redis. So the algorithm set is sealed, and the honest extension point is the store: implement
RateLimiterStorein a new module. New algorithms, like GCRA, arrive as new config members. - Fail open or fail closed?
- Either — chosen explicitly. Allowing traffic while the store is down keeps the API up but leaves limits unenforced; denying protects whatever is downstream at the cost of availability. Allow is the default, deny is one property away, and in both cases the result is flagged
degraded, counted, and logged once per episode. Never silent. - Why 192 bytes per check, and not zero?
- The allocation is the key-prefix concatenation plus two immutable result records — steady-state and young-gen only. Chasing true zero would mean flyweight results and key interning: complexity the numbers don’t yet justify.
- Why not just use Bucket4j?
- Often you should — it is mature and has more backends. FluxLimit’s niche is per-key limits shared across a fleet in one atomic round trip, with a sliding window as well as a token bucket. Multi-region is deliberately out of scope: per-region independent limits; true global coordination is a different product.
Measured, with the caveats left in.
| JMH · JDK 21 | Result |
|---|---|
| In-memory, single thread | ~4.9M checks/s · ~0.2 µs/op |
| In-memory, 8 threads on one hot key | ~3.5M checks/s |
8 threads contended: FluxLimit (keyed) vs Bucket4j LocalBucket | ~4.0M vs ~2.4M ops/s |
| Allocation per check | 192 B |
| Redis on loopback, p50 / p99, one client | 0.25 ms / 2.2 ms |
| Design target | ≥ 100k checks/s |
JMH 1.37, one fork, consumer hardware: treat absolute values as ±30%; the shapes were stable across runs. Bucket4j and Resilience4j are faster single-threaded — they are measured unkeyed, with no map lookup — so the contended column is the meaningful one. Redis numbers are dominated by loopback round-trip time, which is the claim being verified: one round trip per check. Published to Maven Central as 0.1.0; the API may still move before 1.0.