On Thursday 6 April 2023 at 16:03 UTC, a HTTP/2 rapid reset targeted a video streaming customer in Southeast Asia. The attack peaked at 206.2 million requests per second and lasted 62 minutes. Traffic originated from 1116 autonomous systems in 81 countries, predominantly hijacked home routers.
| Vector | HTTP/2 rapid reset |
| Peak | 206.2 million requests per second |
| Duration | 62 min |
| Time to mitigation | 0.244 s |
| Attack traffic reaching origin | 0.044% |
| Legitimate traffic challenged | 0.39% |
Timeline
The attack was preceded by a publicly announced sales event. Request rates on the targeted routes exceeded their hourly baseline by a factor of 314 within 26 seconds. The risk score of participating clients crossed the challenge threshold automatically and proof-of-work difficulty rose with origin load.
What the customer saw
No customer-visible impact. The on-call engineer was notified and acknowledged the incident from the dashboard.
Recommendations
- Add a dedicated rate limit for the targeted route.
- Keep origin IPs out of public DNS history.
- Enable authenticated origin pulls.
Is the risk score exposed in the logs so we can build our own dashboards on it?
Is the risk score exposed in the logs so we can build our own dashboards on it?
Our auditors asked for exactly this kind of incident evidence under DORA.
Good question. We will cover that in a follow-up post.
Thanks — sharing this with our on-call team.
We moved from a scrubbing provider to always-on last year; time to mitigation went from minutes to basically nothing.
Nice to read a vendor blog that admits what went wrong.
Is the risk score exposed in the logs so we can build our own dashboards on it?
Nice to read a vendor blog that admits what went wrong.
Do you publish the edge IP ranges in a machine-readable format?