Every few months, an engineering team reaches out to me for consulting because their Kubernetes cluster has turned into an operational sinkhole. The story is almost identical every time. They are a team of five developers building a B2B SaaS web application. Two years ago, someone on the team read a blog post about service meshes and decided to deploy Amazon EKS.
Now, instead of shipping product features, their most senior engineer spends twenty percent of every sprint debugging CoreDNS latency, upgrading control plane versions before AWS deprecates them, and resolving Helm chart dependencies.
My advice to almost every team with fewer than thirty engineers is simple: pick the boring option. Pick AWS ECS.
#The Hidden Operational Tax of Kubernetes
Kubernetes is a marvelous piece of software for platforms operating at massive organizational scale. When you have four hundred microservices and fifty distinct engineering squads, Kubernetes provides the universal API layer you need.
However, Kubernetes is not just an application runner; it is a full operating system for distributed systems. When you adopt Amazon EKS, you take on several ongoing maintenance commitments:
- 1The Version Treadmill: AWS supports each Kubernetes version for only 14 months. If you fail to upgrade within that window, AWS puts your cluster on Extended Support and bills you an extra $0.60 per cluster hour ($430/month) just to keep running. Upgrading means checking API deprecations, upgrading ingress controllers, updating the AWS VPC CNI plugin, and running node group rollouts.
- 2Cluster Base Cost: Even if you run zero application pods, an EKS control plane costs $73 per month. Add managed node groups, NAT gateways, and load balancers, and your idle cluster baseline starts at $150 to $250 every month.
- 3Cognitive Load: Your developers do not just write TypeScript or Python. They now have to understand Custom Resource Definitions, Horizontal Pod Autoscalers, ServiceAccounts, and IAM Roles for Service Accounts (IRSA).
- 4Networking Complexity: The AWS VPC CNI requires allocating elastic network interfaces and secondary private IP addresses directly from your VPC subnets to every pod. If your subnets run out of IP space during a rolling deployment, pods fail to schedule with cryptic networking errors.
#Why AWS ECS Is the Boring Winner
AWS Elastic Container Service (ECS) is AWS's proprietary container orchestrator. It does not pretend to be an open-source standard. It does one thing exceptionally well: it runs Docker containers on AWS infrastructure with zero fuss.
Here is what you get when you choose ECS with AWS Fargate:
- Zero Control Plane Management: There is no control plane to upgrade, patch, or monitor. AWS manages the orchestrator for free. You pay strictly for the vCPU and memory consumed by your running containers.
- Native AWS Integration: ECS connects directly to AWS Application Load Balancers, CloudWatch logs, AWS Secrets Manager, and IAM Task Roles without third-party operators or sidecars.
- Fargate Serverless Compute: You never patch a Linux EC2 host. You define a task with 0.5 vCPU and 1 GB RAM, and AWS provisions the isolated microVM underneath.
Here is how clean an ECS Task Definition looks in Terraform compared to dozens of YAML manifests:
# ECS Task Definition Example
resource "aws_ecs_task_definition" "api" {
family = "production-api"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
cpu = "256"
memory = "512"
execution_role_arn = aws_iam_role.ecs_execution.arn
task_role_arn = aws_iam_role.ecs_task.arn
container_definitions = jsonencode([
{
name = "api"
image = "${aws_ecr_repository.api.repository_url}:v1.4.2"
essential = true
portMappings = [
{
containerPort = 8080
hostPort = 8080
}
]
logConfiguration = {
logDriver = "awslogs"
options = {
"awslogs-group" = "/ecs/production-api"
"awslogs-region" = "us-east-1"
"awslogs-stream-prefix" = "api"
}
}
}
])
}#Fargate Spot: Slashing Compute Costs by 70%
One of the greatest features in AWS ECS that Kubernetes teams often struggle to implement cleanly is Fargate Spot. With Fargate Spot, AWS sells spare compute capacity at a discount of roughly 70 percent compared to on-demand pricing.
In ECS, you can define a capacity provider strategy that mixes on-demand and spot tasks with a single block of code:
# ECS Service Capacity Provider Strategy
resource "aws_ecs_service" "api" {
name = "production-api"
cluster = aws_ecs_cluster.main.id
task_definition = aws_ecs_task_definition.api.arn
desired_count = 4
capacity_provider_strategy {
capacity_provider = "FARGATE"
weight = 1
base = 1 # Guarantee at least 1 on-demand task always runs
}
capacity_provider_strategy {
capacity_provider = "FARGATE_SPOT"
weight = 3 # Run 75% of remaining scaling tasks on 70% discounted Spot
}
}If AWS reclaims a Fargate Spot task during peak hours, ECS automatically provisions a replacement task or routes incoming traffic to the surviving on-demand task. You achieve enterprise-grade high availability at a fraction of standard cloud costs.
#Observability Without Third-Party DaemonSets
In a Kubernetes cluster, setting up logs and metrics usually requires deploying FluentBit or Promtail as a DaemonSet on every node, configuring RBAC roles, and tuning buffer sizes.
In ECS, logging requires zero agent installation. AWS FireLens integrates directly into the container task definition using Fluent Bit. Logs stream natively to CloudWatch Logs, Datadog, or S3 with zero memory overhead on your nodes. AWS CloudWatch Container Insights provides out-of-the-box dashboards for CPU utilization, memory reservations, and network I/O per service with a single checkbox.
#When Does Kubernetes Actually Make Sense?
I am not anti-Kubernetes. I run Kubernetes clusters for enterprise clients who genuinely need them. Here are the valid technical triggers that justify switching from ECS to Kubernetes:
- Multi-Cloud Portability: If your contractual compliance or enterprise customers require you to deploy on AWS, Google Cloud, and on-premises bare metal with identical deployment tooling.
- Complex In-Cluster Service Mesh: If you genuinely need Istio or Linkerd for mutual TLS, advanced traffic splitting, or distributed request routing across hundreds of distinct services.
- Custom Operator Ecosystem: If your architecture relies heavily on Kubernetes operators like the Prometheus Operator, cert-manager, or Kafka operators.
If your team is operating on a lean budget and wants container orchestration without the EKS price tag, explore my breakdown on [how to run production on $0.64/day](/blog/how-i-run-production-on-0-64-per-day).
If none of those conditions apply to your company today, do yourself a massive favor. Choose AWS ECS. Ship features, delight your customers, and let AWS handle the infrastructure plumbing.
