The ECS agent could not fetch the image. Read the stopped reason first, because the three variants have different causes:
aws ecs describe-tasks --cluster prod --tasks <task-arn> \
--query 'tasks[0].{stopped:stoppedReason,containers:containers[].reason}'
| In the message | Cause |
|---|---|
i/o timeout, dial tcp | No network route to the registry |
403 Forbidden, denied | Execution role lacks ECR permission |
ResourceInitializationError | Execution role missing, or cannot read a secret |
manifest unknown, not found | The tag does not exist |
Cause 1: No route to the registry
The most common on Fargate, and the least obvious because nothing is misconfigured in an IAM sense.
aws ecs describe-services --cluster prod --services api \
--query 'services[0].networkConfiguration.awsvpcConfiguration'
{
"subnets": ["subnet-0private1"],
"assignPublicIp": "DISABLED"
}
A private subnet with no public IP needs either a NAT gateway or VPC endpoints. Without one, the pull times out.
Option A: NAT gateway. Simple, and it has an hourly charge plus data processing costs.
aws ec2 describe-route-tables \
--filters "Name=association.subnet-id,Values=subnet-0private1" \
--query 'RouteTables[].Routes[?DestinationCidrBlock==`0.0.0.0/0`]'
No 0.0.0.0/0 route via a NAT gateway means no outbound path.
Option B: VPC endpoints. Cheaper at volume and keeps traffic off the internet. ECR needs three, and missing any one of them fails:
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc --vpc-endpoint-type Interface \
--service-name com.amazonaws.eu-west-1.ecr.api --subnet-ids subnet-0private1
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc --vpc-endpoint-type Interface \
--service-name com.amazonaws.eu-west-1.ecr.dkr --subnet-ids subnet-0private1
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc --vpc-endpoint-type Gateway \
--service-name com.amazonaws.eu-west-1.s3 \
--route-table-ids rtb-0private
The S3 gateway endpoint is required because ECR stores image layers in S3. People create the two ECR endpoints, see the pull still fail, and conclude endpoints do not work. Add S3.
You will also want a logs endpoint for CloudWatch, or the task starts and then cannot ship logs.
The interface endpoints need a security group allowing 443 from the task's security group, and private DNS enabled.
assignPublicIp: ENABLED in a public subnet with an internet gateway also works, and is the quickest way to prove the diagnosis.
Cause 2: Execution role permissions
This is the distinction that catches everyone. There are two roles:
- Execution role: used by the ECS agent to pull images and read secrets, before your container exists
- Task role: used by your application code afterwards
A pull failure is always the execution role.
aws ecs describe-task-definition --task-definition api \
--query 'taskDefinition.{exec:executionRoleArn,task:taskRoleArn}'
Attach the managed policy:
aws iam attach-role-policy \
--role-name ecsTaskExecutionRole \
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
For a cross-account ECR pull, the repository policy on the other side must also allow it:
{
"Effect": "Allow",
"Principal": { "AWS": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole" },
"Action": ["ecr:GetDownloadUrlForLayer", "ecr:BatchGetImage", "ecr:BatchCheckLayerAvailability"]
}
ecr:GetAuthorizationToken is account-level and stays in the puller's IAM policy, not the repository policy.
Cause 3: ResourceInitializationError on secrets
ResourceInitializationError: unable to pull secrets or registry auth
The task definition references a Secrets Manager secret or SSM parameter and the execution role cannot read it:
{
"Effect": "Allow",
"Action": ["secretsmanager:GetSecretValue", "kms:Decrypt"],
"Resource": ["arn:aws:secretsmanager:eu-west-1:123456789012:secret:prod/db-*"]
}
kms:Decrypt is needed for a secret encrypted with a customer-managed key, and is frequently forgotten.
The same endpoint problem applies: a private subnet needs a secretsmanager or ssm VPC endpoint too.
Cause 4: The tag does not exist
aws ecr describe-images --repository-name api \
--query 'imageDetails[].imageTags' --output text
A pipeline that pushes :latest and deploys :${SHA} fails here. Note also that ECR image tag immutability, if enabled, causes the push to fail silently in some pipelines, so the tag is simply never created.
Fargate platform version
On Fargate, pulls go out through the task ENI, so everything above applies to the task's own subnet and security group, not the cluster's. A security group with no outbound rules blocks the pull; the default allows all egress, so this only bites where someone has locked egress down.
A checklist
aws ecs describe-tasksand readstoppedReason.i/o timeout→ network path.403→ IAM.ResourceInitializationError→ secrets or execution role.- Private subnet → NAT gateway, or VPC endpoints.
- VPC endpoints for ECR need three:
ecr.api,ecr.dkrand the S3 gateway. - Attach
AmazonECSTaskExecutionRolePolicyto the execution role, not the task role. - Secrets → execution role needs
secretsmanager:GetSecretValuepluskms:Decrypt. - Cross-account ECR → repository policy on the source side as well.
- Confirm the tag exists with
aws ecr describe-images.
Frequently Asked Questions
What is the difference between the ECS task role and the execution role?
The execution role belongs to the ECS agent and is used before your container starts, to pull the image from ECR, fetch secrets referenced in the task definition, and create log streams. The task role is assumed by your application code once it is running, for whatever AWS APIs it calls. Any failure during image pull or secret retrieval is therefore an execution role problem, and adding permissions to the task role will have no effect at all.
Why does my Fargate task fail to pull from ECR in a private subnet?
Because a private subnet with assignPublicIp: DISABLED has no route to ECR unless you provide one. Either add a NAT gateway with a 0.0.0.0/0 route, or create VPC endpoints. The endpoint route catches people because ECR needs three: interface endpoints for ecr.api and ecr.dkr, plus an S3 gateway endpoint, since ECR stores the actual image layers in S3. Creating only the two ECR endpoints leaves the layer download with nowhere to go.
Why do I need an S3 endpoint to pull from ECR?
ECR's API handles authentication and manifests, but the image layers themselves are stored in S3 and downloaded directly from there. So a task in a private subnet can authenticate to ECR successfully through the interface endpoints and then fail while fetching layers, which produces a pull error that looks like the endpoints are not working. The S3 gateway endpoint is free and attaches to a route table rather than a subnet, so it is worth adding as a matter of course.
What does ResourceInitializationError mean?
It occurs before your container starts, while the ECS agent is preparing the task, and almost always means it could not fetch something the task definition references. The usual cause is the execution role lacking secretsmanager:GetSecretValue or ssm:GetParameters for a referenced secret, often together with a missing kms:Decrypt when the secret uses a customer-managed key. In a private subnet it can also mean there is no VPC endpoint for Secrets Manager or SSM, so the request has no route.
How do I quickly test whether it is a network problem or a permissions problem?
Temporarily move the task to a public subnet with assignPublicIp: ENABLED. If the pull then succeeds, the problem is the network path and you need a NAT gateway or VPC endpoints. If it still fails with a 403 or a denial, it is IAM, and the execution role is the thing to look at. This one change separates the two causes in a couple of minutes and avoids a long investigation in the wrong direction.