S3 Buckets, Objects, and Versioning

Object storage, not a filesystem

S3 stores objects in buckets. A bucket name is unique across its partition (all of commercial AWS), but the bucket is created in one Region and its data stays there unless you replicate it. A key is the complete object name; / creates prefixes that look like folders but not real directories. You read and write whole objects over HTTPS; there is no in-place edit or file locking.

flowchart LR
  U["App or CLI<br/>PUT object"] --> B["Bucket<br/>devops-assets-123"]
  B --> P["Prefix site/<br/>looks like a folder"]
  P --> O["Object index.html<br/>key + data + metadata"]
  O --> V1["Version v1"]
  O --> V2["Version v2<br/>current"]
  O --> LC["Lifecycle rule<br/>to Glacier after 90 days"]
  %% step U: Clients upload with PUT over HTTPS, using the SDK, CLI or Console.
  %% step B: The bucket lives in one Region and has a globally unique name.
  %% step P: Prefixes are just part of the key; S3 has no real folders.
  %% step O: An object is its data, its key and metadata such as Content-Type.
  %% step V1: With versioning on, overwriting keeps the old version.
  %% step V2: The newest version is what a normal GET returns.
  %% step LC: Lifecycle rules move or expire objects automatically to save cost.

Objects

An object can be up to 5 TB; a single PUT handles up to 5 GB, and larger uploads require multipart upload (recommended from about 100 MB). Metadata describes an object (for example Content-Type), and tags support classification, lifecycle filters and access control. S3 provides strong read-after-write consistency: after a successful write, every read returns the new data. Durability is designed for eleven nines (99.999999999%) by storing data across multiple AZs.

Versioning

Versioning is enabled on a bucket. Writing the same key creates a new version; a normal delete creates a delete marker that hides the object while older versions remain. To really delete, remove a specific version ID. Suspending versioning does not remove old versions, and an object created before versioning has the null version ID. MFA Delete can require MFA to delete versions or change versioning state.

Storage classes and lifecycle

Lifecycle rules transition objects to cheaper classes after a number of days and expire old objects or old versions. Replication (Same-Region or Cross-Region) copies new objects to another bucket and requires versioning on both.

Naming

Bucket names are 3–63 characters of lowercase letters, numbers, dots, and hyphens, starting and ending with a letter or number; uppercase and underscores are invalid. Avoid dots if you need HTTPS virtual-hosted URLs.

Traps: treating prefixes as real folders; thinking a delete on a versioned bucket removes data; uploading a 10 GB file with one PUT.