S3: Object Storage Basics

In part five we saw storage tied to servers. But photos, videos, backups and logs don't need a server at all. They need somewhere that keeps them safe and delivers them on request, however large they get. That's S3.

S3 (Simple Storage Service) is one of AWS's oldest and most used services. There's no space to reserve, and you pay for what you store. AWS designs it for 99.999999999% durability, meaning the chance of losing a file is close to nil.

You'll see it everywhere: backups, user files, static sites, logs and data lakes for analysis.

Object storage versus disks #

The point that matters for design: S3 isn't a disk you mount and run a database from, but it's ideal for any file you read or write as a whole.

Buckets and objects #

A bucket:

• Lives in one Region you choose, even though S3 appears as a global service in the console.
• Must have a name unique across all AWS accounts, using lowercase letters, numbers and hyphens.

An object:

• Can be up to 50 TB.
• Its key is its full path, such as photos/2026/trip.jpg.
• There are no real folders. photos/2026/ is part of the name, and the console merely displays it as a folder.

Who can reach it? #

Everything in S3 is private by default. When someone requests an object, AWS combines all the rules: they get through if a rule allows them and no rule denies them.

For example: an EC2 server reading from S3? Give it an IAM role. Another account reading? A bucket policy allows it. This policy makes a whole bucket publicly readable, which is what a static site needs:

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": "*",
    "Action": "s3:GetObject",
    "Resource": "arn:aws:s3:::my-site-bucket/*"
  }]
}

Versioning #

Once it's on, replacing an object creates a new version rather than wiping the old one:

• Writing over a file: the old version stays, and you can go back to it.
• Deleting: adds a delete marker, and the data stays. Remove the marker and the file returns.
• Suspending: doesn't wipe existing versions.

You pay for every version, so pair it with lifecycle rules to clean up old ones.

Replication #

S3 automatically copies new objects to another bucket, even in another account. Versioning must be on at both ends.

Note that existing objects move only with Batch Replication, replication doesn't pass along a chain of buckets, and permanently deleting a version isn't replicated, which protects you from malicious deletes.

Hosting a static site #

S3 can serve a complete HTML site: turn on Static website hosting, set index.html, and make the objects public with a policy like the one above.

That website link only works over HTTP, though. The better choice in production is a private bucket with CloudFront and OAC in front of it, giving you HTTPS and higher speed without opening the bucket.

Reacting to events #

S3 can send a notification when something happens, such as a photo being uploaded. It sends it to SQS, SNS or Lambda directly, or to EventBridge if you want advanced filtering and more destinations. The classic example: for every photo uploaded, a Lambda function makes a thumbnail. We'll apply this in part twenty-two.

Performance #

S3 handles at least 3,500 writes and 5,500 reads per second per prefix, so spreading keys across several prefixes increases capacity. You also have tools for large files and long distances:

Hands-on: versioning rescues a deleted file #

From CloudShell, create a bucket with a unique name and upload a page:

BUCKET=series-files-$RANDOM$RANDOM
aws s3 mb s3://$BUCKET

echo "<h1>Version 1</h1>" > index.html
aws s3 cp index.html s3://$BUCKET/
aws s3 ls s3://$BUCKET/

Try reaching the file from the internet. It returns 403, because everything is private:

curl -s -o /dev/null -w "%{http_code}\n" \
  https://$BUCKET.s3.$AWS_REGION.amazonaws.com/index.html

Now turn on versioning, write over the file, then delete it:

aws s3api put-bucket-versioning --bucket $BUCKET \
  --versioning-configuration Status=Enabled

echo "<h1>Version 2</h1>" > index.html
aws s3 cp index.html s3://$BUCKET/

aws s3 rm s3://$BUCKET/index.html
aws s3 ls s3://$BUCKET/

The file has gone from the list. But look at what really happened:

aws s3api list-object-versions --bucket $BUCKET --prefix index.html \
  --query "{Versions:Versions[].[VersionId,IsLatest],Markers:DeleteMarkers[].[VersionId,IsLatest]}"

Both versions are there, with a delete marker on top. Delete the marker and the file returns:

MARKER=$(aws s3api list-object-versions --bucket $BUCKET --prefix index.html \
  --query "DeleteMarkers[0].VersionId" --output text)

aws s3api delete-object --bucket $BUCKET --key index.html --version-id $MARKER

aws s3 cp s3://$BUCKET/index.html -

It prints Version 2. The file is back as if it had never been deleted.

Summary #

• S3 is object storage with no practical limit, and you pay for what you store.
• A bucket sits in one Region with a unique name, and a key is an object's full path.
• Everything is private by default, so keep Block Public Access on.
• Versioning protects you from accidental deletes and overwrites, and replication copies objects to another bucket.
• For a static site in production: a private bucket with CloudFront.

Test yourself #

Before you move on, try these questions to check you've got this part. Each one is a practical scenario, and the explanation appears once you answer.

But does every file deserve the same price? A file untouched for two years shouldn't cost you the same as a photo requested every second. In part sixteen we save a lot with storage classes and lifecycle rules.