MIR4 Global: massive Azure infrastructure and automated DDoS blocking.
Dedicated operation of a global game with 3,500 servers and 650 databases on Azure, plus a pipeline that finds DDoS sources in logs and adds them to an NSG automatically.
Problem
At this scale, a physical failure or an attack could spread across the global service. Night and weekend DDoS attacks were slow to answer when a person had to block them by hand.
Approach
Involved from the first design stage, I helped set a high-availability layout on Azure Dedicated Host. Mitigation flow logs from Azure DDoS protection are queried on a schedule in Log Analytics to find abnormal TCP SYN sources, and PowerShell adds those IPs to an NSG deny rule.
Implementation
- Azure
- Dedicated Host
- PowerShell
- Log Analytics
- NSG
Scale · 3,500 servers · 650 databases · Massive
MIR4 Global was a global MMORPG running 3,500 servers on Azure.
Plus 650 databases.
A high-availability layout on Azure Dedicated Host was set from the first design stage.
As main engineer, the work covered design advice, incident response, and security audits.
Read every step
- MIR4 Global was a global MMORPG running 3,500 servers on Azure.
- Plus 650 databases.
- A high-availability layout on Azure Dedicated Host was set from the first design stage.
- As main engineer, the work covered design advice, incident response, and security audits.
Result
Detection to block runs with no human step, which removed the delay at night and on weekends. Resource deployment, automated incident fixes, and asset collection were also automated in PowerShell.
- 3,500
- Servers
- 650
- Databases
- 0
- Human steps from detection to block
What I learned
I learned that for the service to survive one physical host failing, spreading dedicated hosts across separate fault zones is the baseline. At this scale, failures and attacks spread across the global service, so such decisions had to be made from the first design stage rather than added during operation. For DDoS, so that response speed was not tied to when someone was awake, finding abnormal sources in the logs and adding them to the NSG block became one automated flow. And the more blocking is automatic, the more each block needs its reason recorded, so a person can review the decision later.
3,500 / 650 — Servers / databases, global game on Azure