AWS S3

Amazon S3

S3 stores objects in buckets and addresses them by key. There is no filesystem, no mounting, and no directories, which is what lets it scale effectively without limit, and also what surprises people who treat it like a disk.

For DevOps it ends up everywhere: build artifacts, logs, backups, Terraform state, static sites, and the landing zone for data before anyone decides where it really belongs.

Buckets, keys and the folder illusion

A bucket is the top-level container. Its name must be globally unique across all of AWS, not just your account, which is why bucket creation fails with a name someone else took years ago. Names must be lowercase, DNS-compatible, and 3 to 63 characters.

Include your account ID or organisation name to avoid collisions:

aws s3api create-bucket \
  --bucket acme-artifacts-111122223333 \
  --region eu-west-1 \
  --create-bucket-configuration LocationConstraint=eu-west-1

us-east-1 is the exception that catches everyone: it must not have a LocationConstraint, and passing one fails.

An object has a key, content and metadata. A key like logs/2026/09/app.log looks like a path but is a single flat string. There are no folders. The console renders them as folders for convenience, which is why:

• Deleting a "folder" means deleting every object under that prefix.
• An empty "folder" does not exist unless someone created a zero-byte object.
• Listing is a prefix scan, and it gets slow and expensive over millions of objects.

Prefixes do matter for one real reason: S3 scales request rate per prefix, at roughly 3,500 writes and 5,500 reads per second each. A high-throughput workload writing everything under one prefix will throttle; spreading keys across prefixes fixes it.

Storage classes

Those minimum duration charges are the trap. Delete a Standard-IA object after five days and you are billed for thirty. A lifecycle rule that moves objects to Glacier after seven days, on data routinely deleted at day twenty, costs more than leaving it in Standard.

Intelligent-Tiering is the honest default when you do not know the access pattern. It moves objects between tiers automatically for a small per-object monitoring fee, with no retrieval charges and no minimum duration surprises.

One Zone-IA stores in a single Availability Zone. Use it only for data you can regenerate.

Lifecycle rules

{
  "Rules": [
    {
      "ID": "artifacts-tiering",
      "Status": "Enabled",
      "Filter": { "Prefix": "builds/" },
      "Transitions": [
        { "Days": 30, "StorageClass": "STANDARD_IA" },
        { "Days": 90, "StorageClass": "GLACIER_IR" }
      ],
      "Expiration": { "Days": 365 }
    },
    {
      "ID": "abort-incomplete-uploads",
      "Status": "Enabled",
      "Filter": { "Prefix": "" },
      "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
    }
  ]
}

aws s3api put-bucket-lifecycle-configuration \
  --bucket acme-artifacts-111122223333 \
  --lifecycle-configuration file://lifecycle.json

That second rule is worth adding to every bucket. Failed multipart uploads leave orphaned parts that are billed as storage but never appear in a normal listing. Teams discover them years later as an unexplained line on the bill.

# Find them
aws s3api list-multipart-uploads --bucket acme-artifacts-111122223333

Public access and encryption

Both defaults improved substantially and it is worth knowing what you now get for free on new buckets:

• Block Public Access is on by default, at the account and bucket level. Leave it on. A bucket that genuinely serves public content should sit behind CloudFront with an Origin Access Control, not be exposed directly.
• ACLs are disabled by default under Bucket owner enforced ownership. This is a good thing. Object ACLs were the mechanism behind a long history of accidental public exposure. Use bucket policies and IAM instead.
• Server-side encryption with SSE-S3 is applied automatically to all new objects.

For sensitive data, move to SSE-KMS with a customer-managed key so you control the key policy and see key usage in CloudTrail:

aws s3api put-bucket-encryption \
  --bucket acme-artifacts-111122223333 \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "aws:kms",
        "KMSMasterKeyID": "arn:aws:kms:eu-west-1:111122223333:key/abcd-1234"
      },
      "BucketKeyEnabled": true
    }]
  }'

BucketKeyEnabled matters commercially, without it, every object operation is a separate KMS API call, and on a high-volume bucket the KMS bill can exceed the S3 bill. It typically cuts those calls by over 99%.

Versioning and Object Lock

Versioning keeps every version of an object, so an overwrite or delete is recoverable:

aws s3api put-bucket-versioning \
  --bucket acme-artifacts-111122223333 \
  --versioning-configuration Status=Enabled

Two consequences to plan for. A "delete" becomes a delete marker with the real data still stored and still billed, so lifecycle rules need NoncurrentVersionExpiration or the bucket grows forever. And versioning cannot be turned off once enabled, only suspended.

Object Lock goes further, making objects genuinely immutable for a retention period, even the root user cannot delete them in compliance mode. It must be enabled at bucket creation and is the right tool for backups you want protected against ransomware and against your own mistakes.

Controlling cost

S3 bills for storage, requests, and data transfer out. The usual surprises:

• Data transfer out to the internet is often the largest line. Serving through CloudFront is cheaper at volume and faster.
• Cross-region requests cost more than people expect. Keep compute in the bucket's region.
• Request charges matter at scale. Millions of small objects cost more in requests than in storage.
• Orphaned multipart uploads and old versions accumulate invisibly.

# What is actually in there?
aws s3 ls s3://acme-artifacts-111122223333 --recursive --summarize --human-readable | tail -3

S3 Storage Lens gives a free account-wide dashboard of usage and cost by bucket and prefix, and is the fastest way to find what is actually driving the bill.

Presigned URLs

The right way to let someone upload or download without giving them AWS credentials:

aws s3 presign s3://acme-artifacts-111122223333/reports/q3.pdf --expires-in 3600

The URL grants access to that one object for that period. It is how applications hand files to users without proxying the bytes through their own servers, and how users upload directly to S3 without the application handling the data at all.

Go deeper in this section

Quick reference

---

Key takeaways

• Bucket names are globally unique across all of AWS. Include your account ID.
• There are no folders. Keys are flat strings, and prefixes matter for request-rate scaling.
• IA and Glacier classes carry minimum duration charges. Transitioning short-lived data costs more, not less.
• Add an AbortIncompleteMultipartUpload rule to every bucket: orphaned parts are billed and invisible.
• Leave Block Public Access on and serve public content through CloudFront with an Origin Access Control.
• Enable BucketKeyEnabled with SSE-KMS, or KMS request charges can exceed your storage bill.
• Versioning cannot be disabled, and old versions are billed until a lifecycle rule expires them.

---

Check your understanding

---

Frequently Asked Questions

Why does my S3 bucket name fail with "already exists"?

Bucket names are globally unique across every AWS account, not just yours, so a common name was almost certainly claimed long ago. Names must also be lowercase, 3 to 63 characters and DNS-compatible. Include your account ID or organisation prefix, acme-artifacts-111122223333, which makes collisions essentially impossible. Note also that creating a bucket in us-east-1 must omit LocationConstraint, while every other region requires it.

What S3 storage class should I use?

Standard for actively accessed data. Intelligent-Tiering when the access pattern is unknown or varies, since it moves objects automatically for a small monitoring fee with no retrieval charges or minimum duration penalties. Standard-IA or Glacier only when you are confident objects will live well beyond their minimum duration, 30 days for IA, 90 for most Glacier tiers, 180 for Deep Archive, because deleting earlier still incurs the full charge. One Zone-IA saves money but stores in a single Availability Zone, so reserve it for data you could regenerate.

How do I reduce my S3 bill?

Start with S3 Storage Lens, which shows free account-wide usage by bucket and prefix and usually reveals the cause immediately. Then: add an AbortIncompleteMultipartUpload lifecycle rule to every bucket, since orphaned parts are billed but invisible in listings; expire old noncurrent versions if versioning is on; move genuinely cold data to Glacier where it will outlive the minimum duration; serve public downloads through CloudFront rather than directly, as internet egress is often the largest line; and keep compute in the same region as the bucket.

Is S3 data encrypted by default?

Yes. All new objects are encrypted with SSE-S3 automatically at no cost. For sensitive data, configure SSE-KMS with a customer-managed key so you control the key policy and can see usage in CloudTrail. If you do, enable BucketKeyEnabled, without it every object operation makes a separate KMS API call, and on a busy bucket the KMS charges can exceed the S3 charges entirely.

How do I safely make files publicly available?

Usually you should not make the bucket public at all. Put CloudFront in front with an Origin Access Control, keep Block Public Access enabled, and let the distribution read from the private bucket. You get caching, HTTPS, a custom domain and lower egress costs as well as better security. For individual files shared with specific people, use a presigned URL, which grants time-limited access to one object without credentials or any public configuration.

Can I turn off versioning once it is enabled?

No. Versioning can be suspended but never returned to the unversioned state, and suspending it does not delete versions already stored. They remain and continue to be billed. If storage is growing unexpectedly on a versioned bucket, the cause is almost always retained noncurrent versions and delete markers. Add a lifecycle rule with NoncurrentVersionExpiration and one to remove expired delete markers.