Designing a risk score that analysts can explain

We get asked about designing a risk score that analysts can explain more than almost anything else, so here is the long answer.

What we changed

We moved the decision from a single threshold to a continuous score, added an explanation to every decision and made every rule testable against historical traffic before it goes live.

The result is fewer late-night pages for our analysts and — more importantly — fewer real users challenged by mistake.

Observability first

Every mitigation decision is logged with its reasons, and every log line can be traced to the rule and score components that produced it. When a customer asks why a request was challenged, the answer is a link, not a guess.

Logs stream to the customer’s SIEM within seconds, which also means their security team sees attacks in the same tools they use for everything else.

Compliance is a side effect

Regulators increasingly ask for evidence of resilience, not just promises. An incident timeline with start, peak, vectors and impact is exactly the evidence DORA and NIS2 ask for — and it falls out of good observability for free.

We export incident reports in formats auditors can file without anyone rewriting them.

Challenges beat blocks

Blocking lists go stale within minutes when attackers rotate through residential proxies. Proof-of-work does not care where a request comes from; it only cares whether the client is willing to pay the cost.

For a real browser that cost is a few hundred milliseconds, once per session. For a botnet sending a million requests a minute it is a million puzzles a minute — and at that point the attack stops being cheap.

Lessons for your own runbook

Know who can change DNS at two in the morning. Know your origin IPs and who can rotate them. Know which routes are expensive, and have a rate limit ready for each of them.

Most outages during attacks are not caused by the attack itself but by rushed changes made while under pressure.

If you run into any of this in your own environment, our SOC is happy to take a look — even if you are not a customer.