Surviving a 300M+ Req/Day DDoS & EDoS Attack

A Real-World Serverless Postmortem on Google Cloud Run & Edge WAF Architecture

How an AI-orchestrated botnet pushed a 10-year hobby speech synthesis service from $10/month to a $6,750/month WAF billing crisis — and how 5-layer edge defense defeated it for $20/month flat.

Incident Postmortem & Cloud Security Case Study · 2026

Welcome to this postmortem on surviving a 300-million-request-per-day DDoS and Economic Denial of Sustainability (EDoS) attack on Google Cloud Run.

In this presentation, we walk through a real-world multi-week incident against a Japanese speech synthesis web service that had operated peacefully for 10 years at $10/month:

Act 1 (The Incident Onset): How migrating from a fixed-capacity VPS to serverless auto-scaling on Cloud Run turned a CPU-bound synthesis API into an uncapped financial liability.
Act 2 (Perimeter WAF & Proof-of-Work): Deploying Google Cloud Load Balancing and Cloud Armor, battling rapid proxy geo-hopping, and building a custom SHA-256 Proof-of-Work challenge in Web Workers.
Act 3 (The EDoS Pivot & WAF Pricing Trap): How the attacker pivoted to a 300M req/day volumetric flood against lightweight endpoints, triggering a $225/day ($6,750/month) Cloud Armor evaluation fee trap even while blocking 99.9% of requests.
Act 4 (Multi-Layered Edge Victory): Migrating under fire to flat-rate Cloudflare Pro ($20/month) and combining 5 orthogonal signals—ASN blocks, User-Agent hygiene, HTTP/1.1 protocol fingerprinting, sub-threshold rate limiting, and SHA-256 PoW—to achieve 100% availability at zero overage cost.

Agenda & Incident Executive Summary

This presentation is structured into four chronological acts that mirror the real-world incident response timeline:

Act 1 — The Incident Onset: We examine why 10 years of peaceful operation on a fixed-cost VPS ended abruptly after migrating to Google Cloud Run, where cache-busting TTS requests drove CPU utilization to 100% and multiplied weekly compute bills by 4-5x.
Act 2 — Perimeter WAF & Evasion Loops: We contrast the 5-step adaptation cycles of the defender and the attacker, trace the deployment of Google Cloud Load Balancing and Cloud Armor, and see how the botnet defeated country-level geo-blocking across 4 rapid proxy waves and cracked our static JS signatures in 3 hours.
Act 3 — PoW, EDoS Pivot & WAF Pricing Trap: We deploy an asymmetric SHA-256 Proof-of-Work challenge that stops unauthorized TTS synthesis—triggering a shift in the attacker's goal toward pure origin penetration (300M req/day) and a $225/day ($6,750/month) Cloud Armor evaluation fee crisis.
Act 4 — Multi-Layered Edge Victory: We detail the midnight emergency teardown of GCLB, migration to flat-rate Cloudflare Pro ($20/month), traffic decomposition of datacenter vs. residential bots, the 5-layer defense-in-depth architecture, and lessons for defending against AI-assisted adversaries.

Background: A 10-Year Hobby Service Enters the Cloud

Architecture Evolution

10 Years of Peace (2015–2025) Japanese TTS web service running at $10/mo on a single VPS.
Serverless Migration (Mid-2025) Containerized onto Google Cloud Run for zero-ops maintenance.
Hidden Vulnerability Direct public URL (run.app) with no CDN or Edge WAF and uncapped pay-per-use scaling.

Let's look at the background of the target system before the attack began.

10 Years of Peaceful Operation (2015–2025): The application is a solo-maintained Japanese Text-to-Speech (TTS) synthesis service. For a decade, it ran on a single traditional VPS costing roughly $10/month. On a fixed-price VPS, a traffic spike simply saturates the single CPU core and drops connections—it can never bankrupt the operator.
The Serverless Migration (Mid-2025): To eliminate manual Linux OS upgrades and maintenance toil, the service was containerized and migrated to Google Cloud Run.
The Hidden Architectural Vulnerability: Because TTS waveform generation requires heavy DSP computation per request, placing Cloud Run directly on the public internet (ingress = all) without an upstream CDN or WAF meant that every malicious request automatically spun up more container instances and billed directly for vCPU-seconds and network egress.

The Attack Onset: Cache-Busting TTS Flood

Telemetry & Billing Shock

Cloud Run + Cloud Logging Surge Compute/egress and high-volume request logging jumped 15–20x above daily baseline.
100% Cache-Miss Exploitation Bots randomized text, pitch, and speed query parameters on every call.
CPU & Log Saturation Every request forced cold DSP audio synthesis and wrote gigabytes of access logs.

The incident was first discovered not through an uptime alert, but through a Google Cloud billing anomaly notification:

Dual Cost Spike (Cloud Run + Cloud Logging): Within 48 hours, daily charges jumped nearly 20x above the historical baseline. Notably, while Cloud Run CPU & egress (Orange) accounted for ~58% of this initial surge, Cloud Logging (Amber) accounted for ~42%—because every single flood request generated structured container and HTTP request log entries!
Why Application Caching Failed: The attacker specifically targeted the CPU-heavy /synthesize endpoint. Instead of replaying identical URLs (which would hit an in-memory or CDN cache), the botnet randomized the text, pitch, and speed parameters on every single HTTP request.
Maximum Resource Amplification: Because every request had a unique parameter signature, the cache hit ratio dropped to 0%, forcing a full DSP waveform synthesis cycle in Python/C++ on every call.

Initial Triage: Origin Clamping vs. Edge Filtering

Tactic A (max-instances=1 + App Rate Limit): Caps runaway Cloud bill immediately, but starves legitimate users (504 Gateway Timeout) as bots fill the queue.
Tactic B (Dedicated Edge WAF Layer): Drops malicious floods at the Anycast POP (403 Forbidden) so only clean traffic reaches the Cloud Run container.

During the first hour of emergency triage, we implemented two stopgap measures directly on the Cloud Run service:

Clamping max-instances = 1: To stop the financial bleeding immediately, we restricted Cloud Run auto-scaling to a single container instance. While this guaranteed our compute bill could not exceed a single instance's cost, the botnet immediately saturated that single container's request queue, causing 504 Gateway Timeouts for all legitimate users.
In-App IP Rate Limiting: Next, we added an in-memory per-IP rate limiter inside the Python application to return 429 Too Many Requests. However, in a serverless environment without an upstream edge proxy, every 429 response still consumes Cloud Run ingress bandwidth, TLS termination overhead, and container request-handling threads.

Architectural Lesson: You cannot mitigate a volumetric L7 flood inside the origin container. Malicious requests must be dropped at an upstream Anycast Edge WAF before they ever reach the origin.

The Defender's Cycle: 5-Step Iterative Mitigation

Telemetry-Driven Defense Loop

Observe & Extract (Steps 1–2) Inspect attack traffic reaching the origin and isolate shared bot fingerprints (Geo, UA, ASN, HTTP protocol).
Design & Apply (Steps 3–4) Engineer targeted blocking rules that avoid false positives on organic users, then deploy to Edge WAF or API.
Measure & Iterate (Step 5) Evaluate block rate, origin CPU, and cloud billing—looping back to Step 1 whenever residual traffic leaks through.

Throughout this incident, our defense followed a disciplined 5-step operational cycle:

Inspect Attack Traffic Reaching Origin: Analyze request logs that bypassed existing filters and hit our Cloud Run instances.
Identify Common Bot Patterns: Compare malicious requests against legitimate user traffic to find shared attributes (source country, ASN, User-Agent string, request rate, or HTTP protocol version).
Design Blocking Configuration: Formulate a WAF rule or application challenge that blocks the identified pattern without causing false positives for real Japanese users.
Apply Configuration: Roll out the rule to production (Cloud Armor, Cloud Run, or Cloudflare WAF).
Measure Effectiveness: Monitor block rates, container CPU utilization, and billing metrics—and whenever residual attack traffic leaked through, immediately start the next cycle from Step 1.

The Attacker's Cycle: 5-Step Probing & WAF Inference

Assumed Adversary Loop

Flow Analysis & Vector Selection (Steps 1–2) Analyze legitimate user access flows and select an attack method with minimal bot cost and maximum target CPU or billing damage.
Parallel Probing & Rule Inference (Steps 3–4) Test multiple attack patterns in parallel and compare passed (200) vs. blocked (403/429) responses to infer WAF rules.
Blind-Spot Concentration (Step 5) Once the unblocked weak spot is found, concentrate all botnet traffic onto that pattern—and repeat when blocked.

Why did simple single-layer defenses fail so quickly? Because the attacker was executing a parallel 5-step adaptation cycle of their own:

Analyze Legitimate User Flow: Inspect our web app, JavaScript bundles, and API endpoints to understand how real users request audio synthesis.
Select Low-Cost, High-Damage Vector: Choose attack patterns that cost the botnet almost nothing to send while maximizing our CPU load (cache-busting TTS parameters) or cloud bill (300M/day EDoS requests).
Launch Parallel Multi-Pattern Probes: Test diverse combinations of proxy countries, request rates, and headers simultaneously.
Infer WAF Rules via Response Comparison: Compare which probe patterns succeeded (200 OK) versus which were rejected (403 Forbidden or 429 Too Many Requests) to reverse-engineer our exact WAF rules and rate-limit thresholds.
Concentrate Fire on the Weak Point: Shift the entire botnet swarm into the unblocked pattern—and whenever we closed that gap, loop back to find the next blind spot.

Cycle Tug-of-War: Block, CDN & Origin Telemetry

Competing Objectives in Logs

Attacker's Goal: Drive Origin Up Evade WAF filters and randomize parameters so requests penetrate to Origin (Blue) and burn server resources.
Defender's Goal: Drive Block & CDN Up Absorb attacks via Edge Block (Red) and CDN Cache (Amber) while keeping Origin flat with zero false positives.
5-Day Cat-and-Mouse Waveform Every Blue spike marks an attacker bypass; every Red takeover marks a new defense rule sealing the leak.

When the Defender's 5-step cycle and the Attacker's 5-step cycle collide in production, this 5-day traffic telemetry log is the result—a real-time tug-of-war across three metrics:

Attacker's Objective (Maximize Origin — Blue Line): The attacker wants every request to bypass the edge and force the Origin to execute expensive logic. Each time the attacker discovered a WAF blind spot (Day 1, Day 2, and Day 3), Origin traffic spiked to 50k+ requests/hour.
Defender's Objective (Maximize Block & CDN — Red & Amber Lines): Our goal as defenders is to convert that blue Origin line into Edge Blocked (Red) and CDN Cached (Amber)—while never falsely blocking legitimate users.
Convergence by Day 5: Notice how after multiple cat-and-mouse iterations, by Day 5 the Origin (Blue) line is pinned flat near zero (serving only legitimate human traffic), while the Edge Blocked (Red) line smoothly absorbs 100% of the ongoing 30k/hr attack stream.

Perimeter Defense: GCLB & Cloud Armor WAF

Edge Shield Architecture

Private Origin Isolation Locked down run.app direct URL (ingress = internal-and-cloud-load-balancing).
Global Anycast Ingress Placed Google Cloud Load Balancing (GCLB) with Serverless NEG in front of Cloud Run.
Edge L7 WAF Policies Attached Cloud Armor for edge IP rate limiting and geo-filtering before container invocation.

To stop malicious traffic before it could invoke Cloud Run containers, we re-architected the network perimeter:

Locking Down Direct Origin Access: If an attacker knows or discovers the underlying *.run.app URL, they can bypass an external proxy entirely. We reconfigured Cloud Run ingress settings to internal-and-cloud-load-balancing, ensuring any direct request to *.run.app is rejected at Google's fabric edge.
Global External Application Load Balancer (GCLB): We provisioned an Anycast IP frontend backed by a Serverless Network Endpoint Group (NEG) pointing to our Cloud Run service.
Cloud Armor WAF Attachment: With GCLB in place, we attached a Google Cloud Armor security policy to evaluate IP rate limits and geographic rules at Google's edge POPs before forwarding requests to Cloud Run.

Why Geo-Blocking Failed (1/4): Initial Flood from China

Wave 1: Single-Country Origin

95%+ Traffic from China (CN) Initial Cloud Armor telemetry showed almost the entire cache-busting TTS flood originating from Chinese IP ranges.
WAF Rule #1: Block CN Region Deployed an edge geo-filter in Cloud Armor to drop all requests from China (403 Forbidden).
Bypassed in 15 Minutes Attack volume dropped to zero briefly—until the botnet's automated failover shifted proxy regions.

Why Geo-Blocking Failed (2/4): Regional Shift Across Asia

Wave 2: Neighboring Asian Proxies

Instant Regional Failover (+15m) Just 15 minutes after blocking CN, the flood resumed at full volume from surrounding Asian countries.
APAC Stepping-Stone Swarm Traffic poured in simultaneously from Korea (KR), Taiwan (TW), Vietnam (VN), Singapore (SG), Indonesia (ID), Philippines (PH), and India (IN).
WAF Rule #2: Block 10+ Asian Regions Expanded the geo-blacklist across APAC—only to trigger a global rotation within 1 hour.

Why Geo-Blocking Failed (3/4): Global Proxy Mesh (120+ Countries)

Wave 3: Worldwide Escalation

120+ Countries Activated (+1h) Once Asian regions were blocked, the botnet unleashed a global residential proxy pool across the Americas, Europe, Africa, and Oceania.
Country Blacklisting Exhausted Adding individual countries became futile as traffic arrived from US, BR, DE, GB, TR, ZA, and AU.
WAF Rule #3: Allow ONLY Japan (JP) Inverted our WAF policy to block the entire world outside Japan—buying just 4 hours.

Why Geo-Blocking Failed (4/4): Domestic Japanese Proxies

Wave 4: Inside the Perimeter

100% Domestic Japanese IPs (+4h) Within 4 hours of enabling our Japan-only whitelist, the flood returned at full scale—originating entirely inside Japan.
Compromised Home Routers & Proxies Attack traffic routed through domestic consumer fiber ISPs across Hokkaido, Miyagi, Aichi, Osaka, Hiroshima, and Fukuoka.
Geo-Blocking Defeated Blocking Japanese IPs would lock out our own real users—forcing us to move up to application-layer defenses.

Application-Layer Tactics: Static JS Token Spoofing

Why "Secret" JS Logic Failed

Custom Header Verification Added obfuscated frontend JS to compute X-Client-Signature on /synthesize calls.
3 Hours of Relief Simple HTTP flood bots were dropped immediately (403 Forbidden).
Automated AST Reverse-Engineering Within 3 hours, the attacker parsed our JS bundle and replicated the exact HMAC algorithm in their headless bot scripts.

When geographic IP filtering failed, we attempted to distinguish real browsers from raw Python/Go HTTP scripts at the application layer:

Obfuscated JS Header Generation: We updated the web frontend so that before calling POST /synthesize, the browser executed an obfuscated JavaScript routine that generated a dynamic X-Client-Signature header based on request parameters and a timestamp.
Why Security by Obscurity Failed: Because the botnet was executing raw HTTP requests without a DOM, traffic dropped to zero immediately. However, less than 3 hours later, the attack resumed at full volume—with every single bot request carrying a mathematically valid X-Client-Signature header.
Key Takeaway: Any secret or signature algorithm shipped to the client browser in JavaScript can be extracted and re-implemented in Python or Go within hours (especially when attackers use LLMs to de-obfuscate JS ASTs). To stop bots without secret keys, the defense must require unforgeable computational work.

Asymmetric Defense: SHA-256 Proof-of-Work

Cryptographic Cost Inversion

One-Time Challenge Seed Server issues an HMAC-signed seed with a short TTL via GET /challenge.
Browser Web Worker (<50ms) Client finds a nonce where SHA256(seed + nonce) starts with 0000... (~65k hashes).
100% Synthesis Protection Server verifies in O(1) (1 hash), while 1,000 RPS costs the botnet 65M+ hashes/sec.

To defeat automated script spoofing without hurting human UX with intrusive image CAPTCHAs, we implemented an asymmetric SHA-256 Proof-of-Work (PoW) protocol:

Signed Challenge Issuance: Before requesting speech synthesis, the client calls GET /challenge and receives a stateless, HMAC-signed timestamped seed.
Client-Side Hashcash Puzzle: A background Web Worker iterates through nonce values until SHA-256(seed + nonce) begins with k leading zero bits (e.g., 16 bits $\approx 65,536$ hashes). On a normal smartphone or laptop, this completes silently in under 50 milliseconds.
Asymmetric Economic Inversion:
- For the Server: Verifying the proof takes exactly 1 SHA-256 hash ($O(1)$ CPU) plus a fast replay check.
- For the Botnet: Sustaining 1,000 requests/sec now requires computing 65+ million SHA-256 hashes per second, burning the attacker's own CPU cores.

Outcome: Unauthorized calls to /synthesize dropped to 0% permanently. Even if the attacker reads the JavaScript source code, there is no shortcut around SHA-256 preimage resistance.

The Turning Point: How PoW Shifted the Attacker's Goal

A Shift in Adversary Objective

Before PoW: Focused on Calling TTS API Up to this point, every tactic (4 geo-waves & JS cracking) aimed specifically at calling POST /synthesize.
SHA-256 PoW Closed the Synthesis Door Requiring 65,000 hashes per call made mass TTS generation computationally infeasible.
After PoW: "Just Reach the Origin" No longer fixated on the TTS API, the attacker's new goal became sneaking massive request volume past our defenses to hit the Origin on any endpoint.

The Attacker's Pivot: From Resource Theft to Pure EDoS

Strategic Shift in Attack Vector

Phase 1 Blocked (POST /synthesize) SHA-256 PoW made unauthorized TTS generation computationally infeasible.
Phase 2 Retaliation (GET /challenge & /) Botnet pivoted to unauthenticated endpoints that require zero PoW.
300 Million Requests / Day A 100x volumetric surge (~3,500–10,000 RPS) designed purely to inflate cloud request & WAF bills.

Once the SHA-256 Proof-of-Work challenge went live, the attacker realized they could no longer force our server to synthesize audio without burning massive CPU across their own botnet.

From Resource Theft to Pure EDoS: Rather than walking away, the adversary pivoted immediately to Phase 2: Economic Denial of Sustainability (EDoS).
Targeting Lightweight Public Endpoints: Every Proof-of-Work system requires a public endpoint (GET /challenge) and a landing page (GET /) that can be requested before a client solves a puzzle.
300 Million Requests per Day: The botnet unleashed a 100x volumetric surge—scaling from ~3 million requests/day to 300+ million requests/day (~3,500 sustained RPS, peaking at 10,000+ RPS) against / and /challenge, aiming to inflict maximum per-request cloud infrastructure and WAF billing damage.

Cloud Billing Log: How Cloud Armor Became the #1 Cost

This stacked daily billing chart is abstracted directly from our Google Cloud Billing Report across the incident month, and it tells the entire story in four distinct phases:

Days 5–9 (Initial Direct Flood — Orange & Yellow Bars): Before we had a WAF, the cache-busting TTS flood drove up Cloud Run compute (Orange, 34.4% of monthly bill) alongside a massive hidden cost: Cloud Logging (Yellow, 18.0% of monthly bill) from gigabytes of high-velocity request logs.
Days 10–17 (GCLB + Cloud Armor + PoW): Deploying Perimeter WAF and SHA-256 Proof-of-Work successfully crushed Cloud Run compute and logging costs back down near baseline.
Days 24–25 (The Cloud Armor EDoS Explosion — Giant Blue Bar): When the attacker pivoted to a 300M req/day volumetric flood, look at Day 24: Cloud Run (Orange) at the bottom of the bar is almost invisible (~$3/day) because Cloud Armor blocked 99.9% of traffic—yet Google Cloud Armor (Blue) shot up to $205+ in a single day, becoming the #1 cost of the entire month (39.6%)!
Days 26–30 (Post-Cloudflare Cutover): The moment we tore down GCLB/Cloud Armor and migrated to flat-rate Cloudflare Pro, daily GCP billing flatlined back to $2/day.

Midnight Emergency: Tearing Down GCLB & Choosing a Flat-Rate WAF

Realizing that Cloud Armor was accruing $225 every 24 hours, we executed an emergency midnight cutover:

Immediate Teardown of GCLB & Cloud Armor: You cannot simply "pause" an External Application Load Balancer while attack traffic is hitting its Anycast IP. We deleted the GCLB forwarding rules and Cloud Armor security policies outright to stop the per-request billing meter, temporarily clamping Cloud Run back to max-instances = 1.
Evaluating 4 Architectural Candidates:
- Google Cloud Armor: $0.75/M requests evaluated ($\sim$6,750$/mo at 300M req/day).
- AWS WAF / Azure WAF: Identical hyperscaler billing model ($\sim$0.60$/M requests evaluated $\rightarrow $5,400$/mo).
- Self-Hosted Reverse Proxy VPS: Caps cost at $10/mo, but a single VPS network interface immediately saturates under 300M req/day, causing 100% downtime.
- Cloudflare Pro ($20/month Flat Rate): Absorbs and drops L7 DDoS floods across its global Anycast edge with zero per-request evaluation overage fees.

Cloudflare Pro Migration & The 25% Sub-Threshold Leak

Why Per-IP Rate Limits Leaked

Flat $20/mo Cost Secured Cut over DNS to Cloudflare Pro; 75% of the volumetric flood was blocked immediately at $0 overage.
10 Req / 10s Per-IP Threshold Setting stricter per-IP limits risked blocking legitimate Japanese offices & carrier NATs.
Distributed Sub-Threshold Swarm 5,000+ bot IPs throttled down to 6–8 req / 10s each, leaking 4,000 RPS (25%) to origin!

Traffic Forensics: Decomposing the Two Botnet Fleets

Log Forensics Breakthrough: Inspecting Cloudflare request logs by ASN, HTTP Protocol Version, and User-Agent revealed the attack was actually two distinct botnet fleets.
Protocol Mismatch Tell: Fleet B spoofed modern Chrome/Safari User-Agents (Chrome/134), but its proxy software spoke HTTP/1.1 (real browsers always negotiate HTTP/2 or HTTP/3!).

To eliminate the remaining 25% sub-threshold leak without blocking legitimate Japanese users on the same ISPs, we performed deep HTTP telemetry forensics in Cloudflare Analytics and discovered the botnet consisted of two distinct fleets:

Fleet A — Datacenter & Cloud VMs (55% of volume): Originated from commercial cloud/VPS Autonomous System Numbers (ASNs). Since a consumer web app never receives legitimate human browser clicks from headless datacenter ASNs, we blocked entire cloud hosting ASNs outright at the WAF edge.
Fleet B — Hijacked Domestic Residential Proxies (44.5% of volume): Shared the exact same Japanese consumer ISP ASNs as our real users and spoofed modern User-Agent strings (Chrome/134.0).

The Smoking-Gun Protocol Mismatch: Every modern web browser (Chrome, Safari, Firefox, Edge) connecting to Cloudflare over TLS negotiates HTTP/2 or HTTP/3 (QUIC) via ALPN. Fleet B's residential proxy forwarding software, however, hardcoded HTTP/1.1! Blocking or challenging http.request.version == "HTTP/1.1" instantly wiped out Fleet B with zero false positives.

The 5-Layer Defense-in-Depth Architecture

No single rule stopped the attack alone. Victory required stacking 5 orthogonal defense layers in an inverted funnel:

Layer 1 — Network ASN & Tor Filtering (~31% of blocks): Drops all commercial cloud/VPS ASNs plus Tor and anonymous proxy country codes (T1, A1, XX) right at the edge.
Layer 2 — Fingerprint & Header Hygiene (~8% of blocks): Drops requests with spoofed or unknown OS/Browser combinations, missing Sec-Fetch / custom client headers, and /cdn-cgi/trace reconnaissance probes.
Layer 3 — HTTP/1.1 Wire Protocol Block (~19% of blocks): Exploits the ALPN protocol mismatch of residential proxy relays by blocking HTTP/1.1 on /api/*.
Layer 4 — Dual-Speed Edge Rate Limiting (~17% of blocks): Combines a strict burst limit (5 req / 30s on /api/challenge) with a global per-IP ceiling (15 req / 30s across *).
Layer 5 — Page Managed Challenge + API Cookie Gate + SHA-256 PoW (~25% of blocks): Challenges users once on HTML page entry (Managed Challenge), blocks any direct /api/* call lacking the resulting cf_clearance cookie, and verifies SHA-256 PoW before synthesis.

Composite WAF Rules: 10 Custom Rules & 2 Rate Limits

Here is the concrete production rule chain abstracted from our Cloudflare Security Rules and Rate Limiting dashboard, showing how 10 custom rules and 2 rate-limiting rules worked together:

Four Heavy-Lifting Pillars (90% of Mitigated Volume):
1. #2 Block Cloud & Hosting ASNs (245.0k / 28.8%): Drops commercial cloud and datacenter VMs at top priority.
2. #9–10 Page Managed Challenge + API cf_clearance Cookie Gate (210.0k / 24.7%): Because challenging AJAX /api/* endpoints directly breaks frontend apps, Rule #9 applies a Managed Challenge on HTML page entry (~31% Challenge Solve Rate), and Rule #10 blocks any direct /api/* call missing the issued cf_clearance clearance cookie.
3. #3 Block Legacy HTTP/1.1 on /api/* (165.0k / 19.4%): Eliminates residential proxy relays that fail to negotiate HTTP/2 or HTTP/3.
4. RL #1–2 Dual-Threshold Rate Limits (145.0k / 17.1%): Enforces a tight 5 req / 30s cap on /api/challenge alongside a 15 req / 30s global fallback per IP.
Four Surgical Hygiene Guardrails (10% of Mitigated Volume):
- #4–5 Unknown OS / Browser Spoof (32.0k / 3.8%), #6–7 API Client & Sec-Fetch Headers (26.5k / 3.1%), #1 Tor / Anonymous Geo (18.5k / 2.2%), and #8 /cdn-cgi/trace Recon Block (8.0k / 0.9%) caught specialized probe bots that slipped past volumetric filters.

Threat Actor Profile: Anatomy of an AI-Assisted Adversary

Why Their Cycle Ran So Fast

LLM-Speed JS Cracking (<3 Hours) De-obfuscated our custom frontend JS bundle and replicated HMAC signing in under 180 minutes.
Automated A/B Threshold Probing Binary-searched our WAF rate rule and tuned 5,000+ nodes to 6–8 req/10s (just below 10 req/10s).
Adversarial Training Sandbox Used our low-stakes hobby service as a live proving ground to train automated WAF-evasion agents.

In Act 2, we saw the attacker's 5-step adaptation cycle. Why did that cycle execute at superhuman speed against a $10/month hobby speech synthesis service with zero monetizable user data?

AI-Assisted Reverse Engineering: The 3-hour turnaround between deploying our obfuscated JS signature and the botnet replicating the exact signing algorithm strongly points to an LLM-assisted coding workflow (feeding minified JS bundles into an LLM to generate Python/Go request signers).
Automated Rate-Limit Probing: Whenever we adjusted per-IP rate limits, the swarm automatically binary-searched the exact threshold via parallel A/B probes and dialed per-node request rates just underneath it.
Production as an Adversarial Sandbox: Modern botnet operators and AI security researchers routinely use real-world public APIs as live proving grounds to benchmark residential proxy rotation and WAF evasion frameworks before attacking high-value commercial targets.

Summary: 4 Golden Rules for Serverless DDoS Defense

Let's summarize the 4 Golden Rules every cloud architect and solo developer should take away from this incident:

Never Expose Auto-Scaling Serverless Origins Directly: Publicly exposing a pay-per-use endpoint (*.run.app, AWS Lambda Function URLs) without an edge shield gives attackers a blank check drawn on your credit card.
Audit Your WAF's EDoS Pricing Model Before an Attack: Hyperscaler WAFs that charge per evaluated request ($0.60–$0.75 per million) turn volumetric floods into financial disasters even when 99.9% of requests are blocked. Use flat-rate unmetered edge mitigation (such as Cloudflare Pro at $20/mo) when cost predictability matters.
Stack Orthogonal L3–L7 Signals: Geographic IP blocks and simple rate limits fail against distributed residential proxies. Combine Hosting ASN blocks, User-Agent hygiene, and HTTP/2+ ALPN wire protocol checks.
Replace Client-Side Obscurity with Cryptographic Proof-of-Work: In the era of LLM-assisted reverse engineering, obfuscated JS logic survives less than 3 hours. Use asymmetric SHA-256 Proof-of-Work so computational cost scales against the attacker.