Changyi's Blog

AWS S3

AWS S3

Amazon S3 (Simple Storage Service)

Overview

Amazon S3 (Simple Storage Service) is an object storage service that allows users to store and retrieve any amount of data, at any time, from anywhere on the web.
It is designed for high durability (99.999999999%), scalability, and low cost, making it ideal for backups, data lakes, static websites, and more.

Amazon S3 Buckets

  • Buckets are containers for objects (files).
  • Each bucket:
    • Must have a globally unique name.
    • Is defined at the region level (e.g., us-east-1, ap-southeast-2).

Object Properties

Each S3 object includes:

  • Object data (content)
    • Max size: 5 TB (5000 GB)
    • For uploads >5 GB, use multi-part upload for reliability.
  • Metadata
    • Key–value pairs (system or user-defined).
  • Tags
    • Up to 10 key/value pairs — useful for security, lifecycle, or cost tracking.
  • Version ID (if versioning is enabled).

Versioning

  • Versioning allows keeping multiple versions of an object in a bucket.
  • Enabled at the bucket level.
  • When you upload a new file with the same key, it creates a new version instead of replacing the old one.

Benefits

  • Protects against accidental deletions.
  • Enables easy rollback to previous versions.

Notes

  • Objects uploaded before versioning are assigned a version ID of "null".
  • Suspending versioning doesn’t delete old versions.

S3 Encryption Methods

S3 supports encryption at rest (data stored) and encryption in transit (data moving).

SSE-S3 (Server-Side Encryption with S3 Managed Keys)

  • Encryption handled by Amazon S3.
  • AES-256 encryption algorithm.
  • Must set header: "x-amz-server-side-encryption": "AES256"

Flow:

  1. You upload the object over HTTPS.
  2. S3 encrypts it automatically with a managed key.
  3. The object is stored encrypted in the bucket.

SSE-KMS (Server-Side Encryption with KMS Managed Keys)

  • Encryption keys are handled by AWS Key Management Service (KMS).
  • Provides user control, audit trail, and key rotation.
  • Must set header: "x-amz-server-side-encryption": "aws:kms"

Flow:

  1. The object is sent with an HTTP/S request.
  2. S3 calls KMS to get a data encryption key (DEK).
  3. The object is encrypted and stored securely.

SSE-C (Server-Side Encryption with Customer-Provided Keys)

  • You manage your own encryption keys outside AWS.
  • S3 does not store the encryption key you provide.
  • Requires HTTPS only.
  • Key must be included in the HTTP header for every upload/download request.

Used when organizations require full control over encryption material.

Client-Side Encryption

  • Encryption performed before sending data to S3.
  • Clients handle both encryption and decryption.
  • AWS SDK (e.g., Amazon S3 Encryption Client) can assist.

Flow:

  1. The client encrypts data locally with its own key.
  2. The encrypted file is sent to S3.
  3. The client decrypts it when downloading.

Best suited for maximum data control and regulatory compliance.

Encryption in Transit (SSL/TLS)

  • Amazon S3 supports:
  • HTTP endpoint (unencrypted)
  • HTTPS endpoint (encrypted with SSL/TLS)

Recommendations

  • Always use HTTPS — it ensures encryption during upload/download.
  • Most AWS SDKs and CLI tools use HTTPS by default.
  • Mandatory for SSE-C, since encryption keys are included in headers.

Best Practices

  • Enable versioning to protect from accidental deletions.
  • Use SSE-KMS for auditability and compliance.
  • Use Lifecycle Policies to transition old data to cheaper storage classes (like Glacier).
  • Enforce HTTPS-only access via S3 bucket policies.
  • Regularly review bucket policies and IAM permissions to maintain least privilege.
← PreviousNext →