We get asked about a practical checklist for surviving Black Friday traffic more than almost anything else, so here is the long answer.
Lessons for your own runbook
Know who can change DNS at two in the morning. Know your origin IPs and who can rotate them. Know which routes are expensive, and have a rate limit ready for each of them.
Most outages during attacks are not caused by the attack itself but by rushed changes made while under pressure.
Measuring success
We track three numbers for every incident: time to mitigation, the share of attack traffic that reached the origin, and the share of legitimate traffic that was challenged. The first should be under a second, the second under 0.1% and the third under 1%.
Those numbers go into every incident report, and they are the same numbers we are measured against in our SLA.
Challenges beat blocks
Blocking lists go stale within minutes when attackers rotate through residential proxies. Proof-of-work does not care where a request comes from; it only cares whether the client is willing to pay the cost.
For a real browser that cost is a few hundred milliseconds, once per session. For a botnet sending a million requests a minute it is a million puzzles a minute — and at that point the attack stops being cheap.
Latency budget
Our budget for the whole filtering pipeline is one millisecond at the 99th percentile. Anything that cannot be decided within that budget runs asynchronously and influences the next request from the same client, not the current one.
That constraint shapes everything: data structures, where state lives and which signals we are willing to compute inline.
{ "match": { "path": "/search", "risk": ">= 43" }, "action": "challenge" }
We will follow up with the numbers from the next quarter.
This matches what we see in iGaming around big matches.
Do you publish the edge IP ranges in a machine-readable format?
We moved from a scrubbing provider to always-on last year; time to mitigation went from minutes to basically nothing.
Thanks — sharing this with our on-call team.
Thanks! Yes — the risk score and its components are included in every log record.
Clear and practical, thanks.
Nice to read a vendor blog that admits what went wrong.
Is the risk score exposed in the logs so we can build our own dashboards on it?