Scaling Security Insights: how we achieved a 10x increase in global scanning capacity

Summary

Cloudflare Security Insights system now processes over 120 scans per second, providing frequent insights for all customers. By optimizing Kafka consumers, Postgres queries, and our API, we scaled our throughput 10x without adding hardware.

SpinGraph analysis pending — check back after processing.

• Turning Cloudflare’s threat indicators into real-time WAF rules
• Defend against frontier cyber models: Cloudflare's architecture as customer zero
• Route public traffic to private applications with Cloudflare
• Growing the Cloudflare AI team with talent from Ensemble AI
• Cloudflare DMARC Management is now generally available
• Introducing the Cloudflare One stack: agent-powered deployment