Rate limiting APIs without breaking legitimate integrations (part 3)

Here is a question we could not answer well a year ago: what really happens with rate limiting APIs without breaking legitimate integrations? We can answer it now.

Observability first

Every mitigation decision is logged with its reasons, and every log line can be traced to the rule and score components that produced it. When a customer asks why a request was challenged, the answer is a link, not a guess.

Logs stream to the customer’s SIEM within seconds, which also means their security team sees attacks in the same tools they use for everything else.

What actually happens during a flood

The first thing to fail during a layer 7 flood is almost never bandwidth. It is connection slots, worker processes or database connections on the origin — resources measured in hundreds or thousands, not gigabits. An attacker who can make each request expensive only needs a few thousand requests per second.

That is why we score requests before they are proxied. By the time a request reaches your origin, it has been attributed to a client, compared against that client’s history and weighed against the current load on the route it targets.

Why per-route baselines matter

A thousand requests per second to your homepage is Tuesday. A thousand requests per second to your password-reset endpoint is an attack. Global rate limits cannot tell the difference; per-route baselines can.

We learn the normal shape of traffic per route and per hour of the week, so a surge on a sensitive endpoint raises the risk score long before it approaches a global threshold.

The economics behind it

A booter service rents out a 100 Gbps attack for less than the price of a pizza. Defending against it with bandwidth alone is a race you lose. Defending against it by making each malicious request cost more than it earns is a race you win.

This is the entire idea behind never metering attack traffic: our costs scale with filtering, not with your invoice.

Measuring success

We track three numbers for every incident: time to mitigation, the share of attack traffic that reached the origin, and the share of legitimate traffic that was challenged. The first should be under a second, the second under 0.1% and the third under 1%.

Those numbers go into every incident report, and they are the same numbers we are measured against in our SLA.

Challenges beat blocks

Blocking lists go stale within minutes when attackers rotate through residential proxies. Proof-of-work does not care where a request comes from; it only cares whether the client is willing to pay the cost.

For a real browser that cost is a few hundred milliseconds, once per session. For a botnet sending a million requests a minute it is a million puzzles a minute — and at that point the attack stops being cheap.

{ "match": { "path": "/search", "risk": ">= 64" }, "action": "challenge" }

We will follow up with the numbers from the next quarter.