AWS ECS Tasks and Services

Two objects do most of the work in ECS. A task definition describes what to run; a service keeps a number of tasks running and handles deployments.

Task definitions

A versioned, immutable blueprint. Registering a change creates a new revision. You never edit one in place:

{
  "family": "payments-api",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024",
  "runtimePlatform": { "cpuArchitecture": "ARM64" },
  "executionRoleArn": "arn:aws:iam::111122223333:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::111122223333:role/paymentsTaskRole",
  "containerDefinitions": [{
    "name": "api",
    "image": "111122223333.dkr.ecr.eu-west-1.amazonaws.com/payments-api:1.4.2",
    "portMappings": [{ "containerPort": 8080, "protocol": "tcp" }],
    "environment": [{ "name": "LOG_LEVEL", "value": "info" }],
    "secrets": [{
      "name": "DB_PASSWORD",
      "valueFrom": "arn:aws:secretsmanager:eu-west-1:111122223333:secret:prod/db-AbCdEf:password::"
    }],
    "healthCheck": {
      "command": ["CMD-SHELL", "curl -f http://localhost:8080/healthz || exit 1"],
      "interval": 30, "timeout": 5, "retries": 3, "startPeriod": 60
    },
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/ecs/payments-api",
        "awslogs-region": "eu-west-1",
        "awslogs-stream-prefix": "ecs"
      }
    }
  }]
}

aws ecs register-task-definition --cli-input-json file://task-definition.json

The two IAM roles

The most common ECS mistake, and worth stating plainly:

If image pulls and logging work but your code gets AccessDenied, the permissions are on the execution role and belong on the task role.

Note that the secrets block is resolved by the execution role, not the task role, so the execution role needs secretsmanager:GetSecretValue and, if the secret is KMS-encrypted, kms:Decrypt.

awsvpc networking

On Fargate it is the only option, and on EC2 it is the one to choose. Each task gets its own elastic network interface and private IP, so tasks are addressable like instances and get their own security group.

That IP consumption is the constraint: a large ECS deployment can exhaust a subnet just as EKS can. Size subnets accordingly.

The alternatives, bridge and host, exist for EC2 launch type and involve port mapping on the host, which complicates running several copies of the same task on one instance.

Sizing

Fargate accepts only specific CPU and memory combinations. 512 CPU units (half a vCPU) permits 1, 2, 3 or 4 GB of memory, and nothing else. An invalid pair is rejected at task definition registration with a message listing the valid values.

Set runtimePlatform.cpuArchitecture to ARM64 for Graviton, which is around 20% cheaper.

Services

A service keeps N tasks running, replaces failures, registers targets with a load balancer, and manages rolling deployments:

aws ecs create-service \
  --cluster production \
  --service-name payments-api \
  --task-definition payments-api:42 \
  --desired-count 3 \
  --launch-type FARGATE \
  --network-configuration 'awsvpcConfiguration={subnets=[subnet-0aaa,subnet-0bbb],securityGroups=[sg-0abc],assignPublicIp=DISABLED}' \
  --load-balancers 'targetGroupArn=arn:aws:elasticloadbalancing:...,containerName=api,containerPort=8080' \
  --health-check-grace-period-seconds 60 \
  --deployment-configuration 'deploymentCircuitBreaker={enable=true,rollback=true},maximumPercent=200,minimumHealthyPercent=100'

Three settings there earn their place.

deploymentCircuitBreaker with rollback is the most valuable. Without it, a deployment of a broken image retries indefinitely while the service sits degraded. With it, ECS detects repeated task failures, stops, and reverts to the previous task definition automatically.

minimumHealthyPercent=100 with maximumPercent=200 means capacity never dips during a deploy. New tasks start before old ones stop, at the cost of briefly running double.

healthCheckGracePeriodSeconds stops the load balancer killing tasks that are still starting. Set it above your real start-up time, or a slow-booting application will be terminated and restarted in a loop that looks like a crash.

Deploying a new version

aws ecs update-service --cluster production --service payments-api \
  --task-definition payments-api:43

# Or, to redeploy the same revision (picking up a moved image tag)
aws ecs update-service --cluster production --service payments-api --force-new-deployment

aws ecs wait services-stable --cluster production --services payments-api

services-stable blocks until the deployment settles, which is what a CI pipeline should wait on rather than assuming success.

Debugging a task that will not start

stoppedReason is the field to read first, and it is usually explicit:

aws ecs describe-tasks --cluster production \
  --tasks $(aws ecs list-tasks --cluster production --desired-status STOPPED \
            --query 'taskArns[0]' --output text) \
  --query 'tasks[0].{Stopped:stoppedReason,Containers:containers[].{Name:name,Reason:reason,Exit:exitCode}}'

That second one is a frequent Fargate problem: a task in a private subnet with no NAT gateway and no VPC endpoints cannot reach ECR or Secrets Manager, so it fails before your code runs at all. Interface endpoints for ecr.api, ecr.dkr, secretsmanager and logs, plus a gateway endpoint for S3, resolve it, and cost less than NAT.

Getting a shell in a running task

aws ecs execute-command --cluster production \
  --task <task-id> --container api \
  --interactive --command "/bin/sh"

ECS Exec needs enableExecuteCommand on the service and ssmmessages permissions on the task role. It is the ECS equivalent of kubectl exec and worth enabling before you need it.

Service discovery and scaling

Service Connect or Cloud Map give services stable DNS names so they can find each other without a load balancer per service.

Application Auto Scaling adjusts desiredCount on CloudWatch metrics, typically average CPU, or ALBRequestCountPerTarget, which tracks load far better than CPU for an I/O-bound service.

Quick reference

---

Key takeaways

• Task definitions are immutable. Every change creates a new revision.
• Execution role for the agent, task role for your code. The secrets block uses the execution role.
• awsvpc gives each task its own IP, so subnets can be exhausted like on EKS.
• Fargate CPU and memory must be a valid pair: arbitrary combinations are rejected.
• Enable the deployment circuit breaker with rollback, or a broken image retries forever.
• Set a health check grace period above real start-up time, or slow starters are killed in a loop.
• stoppedReason names the cause. ResourceInitializationError usually means no route to ECR or Secrets Manager.
• Enable ECS Exec before you need it.

---

Check your understanding

---

Frequently Asked Questions

What is the difference between an ECS task definition and a service?

A task definition is an immutable, versioned blueprint describing what to run: the image, CPU and memory, environment, IAM roles, logging and health checks. A service takes a task definition and keeps a specified number of copies running, replacing failures, registering them with a load balancer and managing rolling deployments. You can run a task once without a service, which suits batch jobs, but anything long-lived should be behind a service.

What is the difference between the ECS task role and task execution role?

The execution role is assumed by the ECS agent to pull the image from ECR, write logs to CloudWatch and resolve values in the secrets block. The task role is assumed by your application code for AWS API calls. If image pulls and logging succeed but your code gets AccessDenied, the permissions were added to the execution role and belong on the task role. Note that secrets injection specifically uses the execution role, which surprises people.

Why does my Fargate task fail with ResourceInitializationError?

It cannot reach the AWS services needed before your container starts, typically ECR for the image, Secrets Manager for secrets, or CloudWatch for logs. In a private subnet with no NAT gateway and no VPC endpoints there is no route to any of them. Add interface endpoints for ecr.api, ecr.dkr, secretsmanager and logs, plus a free gateway endpoint for S3 since image layers live there. That is usually cheaper than a NAT gateway as well as faster.

How do I get a shell inside a running ECS task?

aws ecs execute-command --cluster <c> --task <task-id> --container <name> --interactive --command "/bin/sh". It requires enableExecuteCommand on the service and ssmmessages permissions on the task role, both of which are easier to add before an incident than during one. It is the ECS equivalent of kubectl exec and works on Fargate as well as EC2.

How do I deploy a new version of my ECS service?

Register a new task definition revision with the updated image, then aws ecs update-service --task-definition <family>:<revision>. To redeploy the same revision, for instance after moving a mutable image tag, use --force-new-deployment. Always follow with aws ecs wait services-stable in CI so the pipeline fails on a bad deploy rather than reporting success while tasks crash. Enable the deployment circuit breaker with rollback so a broken image reverts automatically.

Why do my ECS tasks keep restarting?

Read stoppedReason from describe-tasks first. It usually names the cause directly. If it says the essential container exited, the application itself is failing and the CloudWatch logs will show why. If it reports failed ELB health checks, the target group check is not passing; the frequent cause is a health check grace period shorter than real start-up time, so the load balancer kills the task before it is ready. OutOfMemoryError means the container exceeded its memory allocation.