GCP Terraform GKE platform.
A module set that keeps capacity during GKE node pool upgrades with SURGE, and opens only what a VPN needs through a firewall baseline, all as code.
Problem
A GKE node upgrade must not reduce serving capacity, and management ports should open only from a VPN.
Approach
The node pool upgrades with SURGE: five new nodes come up first and unavailable stays at zero. The c2sfirewall module opens management ports from the VPN at priority 1000 and denies other tcp and udp outbound at priority 65534.
Implementation
- Terraform
- GCP
- GKE
- Cloud Logging
- Cloud Monitoring
Scale · 22 modules · 3 examples
The node pool upgrades with SURGE: max_surge 5, max_unavailable 0.
Five new nodes come up first. The existing nodes stay in place.
Workloads move to the new nodes. No node drops out during the swap.
Only after a replacement is ready does an old node leave.
The swap finishes. The 3600s soak value is read only under BLUE_GREEN.
Read every step
- The node pool upgrades with SURGE: max_surge 5, max_unavailable 0.
- Five new nodes come up first. The existing nodes stay in place.
- Workloads move to the new nodes. No node drops out during the swap.
- Only after a replacement is ready does an old node leave.
- The swap finishes. The 3600s soak value is read only under BLUE_GREEN.
Result
Old nodes leave only after replacements are ready. Host errors flow from the hostError audit log to a log-based metric, an alert policy, and a Slack channel.
- 22
- Terraform modules
- 5
- Extra nodes first (max_surge)
- 0
- Nodes down at once (max_unavailable)
What I learned
I learned that judging behavior from a setting's name can be wrong. A value like soak time sounds as if it applies to every upgrade, but it is read only under the BLUE_GREEN strategy, so the SURGE example states in a comment that the value has no effect there. Protecting capacity during upgrades also works only when the intent is pinned in numbers, such as bring five new nodes up first (max_surge 5) and take none away (max_unavailable 0), so the result stays the same whoever runs it. The firewall follows the same idea: the module ships a baseline that opens only management ports at priority 1000 and blocks the rest at 65534, so every new environment starts from the same rules.
22 modules — max_surge 5, max_unavailable 0