Skip to main content

S3 Bucket - Object Storage

Scalable, durable, and secure cloud object storage with versioning, encryption, and access control.

Prerequisite: AWSProvider Configuration

Before creating any AWS resource, you need to configure an AWSProvider that manages credentials and authentication with AWS.

IRSA:

apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: AWSProvider
metadata:
name: production-aws
namespace: default
spec:
region: us-east-1
roleARN: arn:aws:iam::123456789012:role/infra-operator-role
defaultTags:
managed-by: infra-operator
environment: production

Static Credentials:

apiVersion: v1
kind: Secret
metadata:
name: aws-credentials
namespace: default
type: Opaque
stringData:
access-key-id: test
secret-access-key: test
---
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: AWSProvider
metadata:
name: localstack
namespace: default
spec:
region: us-east-1
accessKeyIDRef:
name: aws-credentials
key: access-key-id
secretAccessKeyRef:
name: aws-credentials
key: secret-access-key
defaultTags:
managed-by: infra-operator
environment: test

Verify Status:

kubectl get awsprovider
kubectl describe awsprovider production-aws
warning

For production, always use IRSA (IAM Roles for Service Accounts) instead of static credentials.

Create IAM Role for IRSA

To use IRSA in production, you need to create an IAM Role with the necessary permissions:

Trust Policy (trust-policy.json):

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:infra-operator-system:infra-operator-controller-manager",
"oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
}
}
}
]
}

IAM Policy - S3 Bucket (s3-policy.json):

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:DeleteBucket",
"s3:ListBucket",
"s3:GetBucketVersioning",
"s3:PutBucketVersioning",
"s3:GetBucketEncryption",
"s3:PutBucketEncryption",
"s3:GetBucketPublicAccessBlock",
"s3:PutBucketPublicAccessBlock",
"s3:GetBucketTagging",
"s3:PutBucketTagging",
"s3:GetBucketLifecycleConfiguration",
"s3:PutBucketLifecycleConfiguration",
"s3:GetBucketCors",
"s3:PutBucketCors"
],
"Resource": "*"
}
]
}

Create Role with AWS CLI:

# 1. Get EKS cluster OIDC Provider
export CLUSTER_NAME=my-cluster
export AWS_REGION=us-east-1
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)

OIDC_PROVIDER=$(aws eks describe-cluster \
--name $CLUSTER_NAME \
--region $AWS_REGION \
--query "cluster.identity.oidc.issuer" \
--output text | sed -e "s/^https:\/\///")

# 2. Update trust-policy.json with correct values
cat > trust-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${OIDC_PROVIDER}:sub": "system:serviceaccount:infra-operator-system:infra-operator-controller-manager",
"${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
}
}
}
]
}
EOF

# 3. Create IAM Role
aws iam create-role \
--role-name infra-operator-s3-role \
--assume-role-policy-document file://trust-policy.json \
--description "Role for Infra Operator S3 management"

# 4. Create and attach policy
aws iam put-role-policy \
--role-name infra-operator-s3-role \
--policy-name S3Management \
--policy-document file://s3-policy.json

# 5. Get Role ARN
aws iam get-role \
--role-name infra-operator-s3-role \
--query 'Role.Arn' \
--output text

Annotate Operator ServiceAccount:

# Add annotation to operator ServiceAccount
kubectl annotate serviceaccount infra-operator-controller-manager \
-n infra-operator-system \
eks.amazonaws.com/role-arn=arn:aws:iam::123456789012:role/infra-operator-s3-role
note

Replace 123456789012 with your AWS Account ID and EXAMPLED539D4633E53DE1B71EXAMPLE with your OIDC provider ID.

Overview

Amazon S3 (Simple Storage Service) is a massively scalable object storage service that offers enterprise-class durability, availability, and security. S3 buckets can be used for various use cases: backup, static website files, data lakes, log repositories, and much more.

Features:

  • Unlimited object storage (each up to 5 TB)
  • 99.999999999% (11 nines) annual durability
  • 99.99% availability in multi-AZ
  • Automatic replication between availability zones
  • Versioning for object history control
  • Encryption at rest (AES256 or KMS)
  • Encryption in transit (HTTPS)
  • Public Access Block for security
  • Lifecycle rules for cost optimization
  • CORS for web applications
  • Static website hosting
  • CloudFront integration for CDN
  • Granular control via bucket policies and IAM
  • Access logging
  • No creation cost, pay only for stored data

Quick Start

S3 Bucket Full-Featured:

apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: e2e-test-bucket
namespace: default
spec:
providerRef:
name: localstack
bucketName: e2e-test-bucket-infra-operator

# Enable versioning
versioning:
enabled: true

# Encryption
encryption:
algorithm: AES256

# Block public access
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true

# Tags
tags:
test-type: e2e
component: s3-controller

# Retain bucket after test (for debugging)
deletionPolicy: Delete

Simple S3 Bucket:

apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: e2e-simple-bucket
namespace: default
spec:
providerRef:
name: localstack
bucketName: e2e-simple-bucket

tags:
test-type: e2e
variant: simple

deletionPolicy: Delete

Apply:

kubectl apply -f s3-bucket.yaml

Verify Status:

kubectl get s3bucket e2e-test-bucket
kubectl describe s3bucket e2e-test-bucket
kubectl get s3bucket e2e-test-bucket -o yaml

Configuration Reference

Required Fields

Reference to AWSProvider resource

AWSProvider resource name

Unique S3 bucket name. Must be globally unique across all of AWS.

Rules:

  • Minimum 3 characters, maximum 63
  • Only lowercase letters, numbers, hyphens, and dots
  • Cannot start or end with hyphen or dot
  • Cannot contain IP address (e.g., 192.168.1.1)
  • Cannot contain underscore (_)

Example:

bucketName: my-company-app-data-prod-2025

Optional Fields

Versioning configuration to maintain object history

Example:

versioning:
enabled: true

Details:

  • enabled: true: Keeps all versions of each object
  • enabled: false: No versioning (default)
  • Allows recovery of deleted or overwritten objects
  • Requires more storage
  • Adds storage cost per version

Encryption at rest for the bucket

Example:

encryption:
algorithm: AES256 # or KMS
kmsKeyID: "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012"

Options:

  • AES256: AWS-managed server-side encryption (default, no additional cost)
  • KMS: AWS KMS encryption (requires CMK, additional cost)
  • If using KMS, kmsKeyID is required

Blocks public access to the bucket (highly recommended)

Example:

publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true

Fields:

  • blockPublicAcls: Blocks new public ACLs (recommended: true)
  • ignorePublicAcls: Ignores existing public ACLs (recommended: true)
  • blockPublicPolicy: Blocks public policies (recommended: true)
  • restrictPublicBuckets: Restricts public access via policies (recommended: true)

Rules for automatic transition or deletion of objects based on age

Example:

lifecycleRules:
- id: archive-old-logs
enabled: true
prefix: logs/
filter:
tags:
archived: "true"
transitions:
- days: 30
storageClass: STANDARD_IA
- days: 90
storageClass: GLACIER
- days: 180
storageClass: DEEP_ARCHIVE
expiration:
days: 365
noncurrentVersionTransitions:
- days: 30
storageClass: STANDARD_IA
noncurrentVersionExpiration:
days: 90

Use Cases:

  • Save costs by moving old objects to cheaper classes
  • Delete logs after retention period
  • Archive data for compliance

Configure CORS (Cross-Origin Resource Sharing) for website access

Example:

cors:
corsRules:
- allowedMethods:
- GET
- PUT
- POST
allowedOrigins:
- "https://example.com"
- "https://app.example.com"
allowedHeaders:
- "Authorization"
- "Content-Type"
exposeHeaders:
- "x-amz-meta-*"
maxAgeSeconds: 3000

Usage: Allow AJAX requests from browsers to S3

Configure bucket for static website hosting

Example:

website:
indexDocument: index.html
errorDocument: 404.html

Details:

  • indexDocument: Default file when accessing the bucket
  • errorDocument: File displayed on 4xx errors
  • Bucket MUST have publicAccessBlock: false to work
  • Requires bucket policy allowing GetObject

Key-value pairs to tag the bucket

Example:

tags:
Environment: production
Application: my-app
Team: platform
CostCenter: engineering
ManagedBy: infra-operator

What happens to the bucket when the CR is deleted

Options:

  • Delete: Bucket is deleted from AWS (WARNING: objects will be lost)
  • Retain: Bucket remains in AWS but not managed
  • Orphan: Remove only management but keep bucket

Example:

deletionPolicy: Retain

Status Fields

After the S3 Bucket is created, the following status fields are populated:

Created bucket name

Bucket ARN (Amazon Resource Name) (e.g., arn:aws:s3:::my-bucket)

AWS region where the bucket was created

Whether versioning is enabled on the bucket

Whether encryption is enabled on the bucket

Encryption algorithm used (AES256 or KMS)

Bucket access URL (e.g., https://my-bucket.s3.amazonaws.com)

true when the bucket is created and ready for use

Timestamp of last synchronization with AWS

Examples

Production S3 Bucket with Versioning and Encryption

Example:

apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: production-bucket
namespace: default
spec:
providerRef:
name: production-aws

bucketName: mycompany-app-data-prod

# Enable versioning for recovery
versioning:
enabled: true

# AES256 encryption (no additional cost)
encryption:
algorithm: AES256

# Maximum security against public access
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true

tags:
Environment: production
Application: data-storage
Team: platform
ManagedBy: infra-operator
CostCenter: infrastructure

# Retain bucket when deleting CR (safety)
deletionPolicy: Retain

S3 Bucket for Static Website Hosting

Example:

apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: website-bucket
namespace: default
spec:
providerRef:
name: production-aws

bucketName: mycompany-website-prod

# Configure for website hosting
website:
indexDocument: index.html
errorDocument: 404.html

# Allow CORS requests from browsers
cors:
corsRules:
- allowedMethods:
- GET
- HEAD
allowedOrigins:
- "https://mycompany.com"
- "https://www.mycompany.com"
maxAgeSeconds: 3000

# Basic encryption
encryption:
algorithm: AES256

# Block public access (access via CloudFront/policies)
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true

tags:
Environment: production
Type: static-website
Application: corporate-website

deletionPolicy: Retain

S3 Bucket with Lifecycle Rules for Cost Optimization

Example:

apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: logs-bucket
namespace: default
spec:
providerRef:
name: production-aws

bucketName: mycompany-logs-archive-prod

# Versioning for compliance
versioning:
enabled: true

encryption:
algorithm: AES256

# Lifecycle rules for savings
lifecycleRules:
# Transition to cheaper classes
- id: transition-to-cheaper-storage
enabled: true
prefix: logs/
transitions:
- days: 30 # After 30 days: Standard-IA
storageClass: STANDARD_IA
- days: 90 # After 90 days: Glacier
storageClass: GLACIER
- days: 365 # After 1 year: Deep Archive
storageClass: DEEP_ARCHIVE

# Delete old versions
- id: cleanup-old-versions
enabled: true
noncurrentVersionTransitions:
- days: 30
storageClass: STANDARD_IA
noncurrentVersionExpiration:
days: 90

# Delete incomplete multipart upload objects
- id: cleanup-incomplete-multipart
enabled: true

publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true

tags:
Environment: production
Type: logs-archive
Purpose: compliance

deletionPolicy: Retain

S3 Bucket with CORS for Web Application

Example:

apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: api-assets-bucket
namespace: default
spec:
providerRef:
name: production-aws

bucketName: mycompany-api-assets-prod

# Configure CORS for web application
cors:
corsRules:
- id: allow-frontend-app
allowedMethods:
- GET
- HEAD
- PUT
- POST
- DELETE
allowedOrigins:
- "https://app.mycompany.com"
- "https://dashboard.mycompany.com"
allowedHeaders:
- "*"
exposeHeaders:
- "x-amz-meta-*"
- "ETag"
maxAgeSeconds: 3600

encryption:
algorithm: AES256

versioning:
enabled: true

publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true

tags:
Environment: production
Type: application-assets
Application: web-api

deletionPolicy: Retain

Verification

Verify Bucket Status

Command:

# List all created S3 Buckets
kubectl get s3buckets

# Get detailed information
kubectl get s3bucket production-bucket -o yaml

# Describe bucket (shows events and status)
kubectl describe s3bucket production-bucket

# Watch creation in real-time
kubectl get s3bucket production-bucket -w

Verify on AWS

AWS CLI:

# List buckets
aws s3 ls

# Get versioning configuration
aws s3api get-bucket-versioning --bucket my-company-app-data

# Get encryption configuration
aws s3api get-bucket-encryption --bucket my-company-app-data

# Get public access configuration
aws s3api get-public-access-block --bucket my-company-app-data

# Get lifecycle configuration
aws s3api get-bucket-lifecycle-configuration --bucket my-company-app-data

# Get CORS configuration
aws s3api get-bucket-cors --bucket my-company-app-data

# Get all tags
aws s3api get-bucket-tagging --bucket my-company-app-data

# Check bucket size
aws s3 ls s3://my-company-app-data --summarize --human-readable --recursive

# List objects
aws s3 ls s3://my-company-app-data --recursive

LocalStack:

# For LocalStack testing
export AWS_ENDPOINT_URL=http://localhost:4566

aws s3 ls
aws s3api get-bucket-versioning --bucket my-company-app-data
aws s3api get-bucket-encryption --bucket my-company-app-data

Expected Output

Example:

status:
bucketName: my-company-app-data
bucketArn: arn:aws:s3:::my-company-app-data
region: us-east-1
bucketURL: https://my-company-app-data.s3.amazonaws.com
versioningEnabled: true
encryptionEnabled: true
encryptionAlgorithm: AES256
ready: true
lastSyncTime: "2025-11-22T20:30:15Z"

Troubleshooting

Error: Bucket name already exists

Symptoms: Creation fails with error "BucketAlreadyExists" or "BucketAlreadyOwnedByYou"

Cause: Bucket name already exists and is used by another AWS account or yourself

Solutions:

# Bucket names are globally unique in AWS
# Check if bucket exists
aws s3 ls | grep my-bucket-name

# Use a more unique name (add timestamp, region, etc)
bucketName: my-company-app-data-prod-us-east-1-20250122

# Or check if it's your bucket and reuse it
kubectl patch s3bucket my-bucket \
--type merge \
-p '{"metadata":{"finalizers":[]}}'

Bucket stuck in NotReady

Symptoms: Bucket remains in NotReady state after creation

Common causes:

  1. Insufficient IAM permissions
  2. Bucket quota reached (default: 100 buckets per account)
  3. AWS connectivity issue

Solutions:

# Check operator logs
kubectl logs -n infra-operator-system \
deploy/infra-operator-controller-manager \
--tail=100

# Check detailed status
kubectl describe s3bucket production-bucket

# Check if AWSProvider is ready
kubectl get awsprovider
kubectl describe awsprovider production-aws

# Try force sync with annotation
kubectl annotate s3bucket production-bucket \
force-sync="$(date +%s)" --overwrite

Access denied to bucket

Symptoms: Error "AccessDenied" or "403 Forbidden"

Cause: Insufficient IAM permissions or bucket policy blocking access

Solutions:

# Check IAM permissions of role/user
# IAM policy MUST include s3:* or specific actions

# Check bucket policy
aws s3api get-bucket-policy --bucket my-company-app-data

# Test access
aws s3 ls s3://my-company-app-data

# Verify credentials using
aws sts get-caller-identity

# If using IRSA, check service account
kubectl describe sa infra-operator-controller-manager \
-n infra-operator-system

Cannot delete bucket (contains objects)

Symptoms: Error when deleting: "The bucket you tried to delete is not empty"

Cause: Bucket contains objects that need to be deleted first

Solutions:

# Option 1: Delete objects manually
aws s3 rm s3://my-company-app-data --recursive

# Option 2: Use deletionPolicy: Delete
# This deletes objects automatically when deleting CR
kubectl patch s3bucket production-bucket \
--type merge \
-p '{"spec":{"deletionPolicy":"Delete"}}'

# Option 3: If versioning enabled, delete versions
aws s3api list-object-versions \
--bucket my-company-app-data \
--output json > versions.json

# Then delete each version
aws s3api delete-object --bucket my-company-app-data \
--key <key> --version-id <version-id>

# Only then delete CR
kubectl delete s3bucket production-bucket

High S3 storage costs

Symptoms: AWS bills with unexpected costs

Cause: Data is not transitioned to cheaper classes

Solutions:

# 1. Implement Lifecycle Rules
# Transition to STANDARD_IA after 30 days (~50% of cost)
# Transition to GLACIER after 90 days (~20% of cost)

# 2. Clean up old data
aws s3api delete-objects --bucket my-bucket \
--delete 'Objects=[{Key=old-file.txt}]'

# 3. Check current usage
aws s3 ls s3://my-bucket --summarize --human-readable --recursive

# 4. Use S3 Intelligent-Tiering (automatic)
# Automatically transitions based on access

# 5. Consider S3 Select for efficient queries
# Read only necessary data instead of entire object

Versioning not working

Symptoms: Versioning enabled: false even after configuration

Causes:

  1. Versioning was disabled after enabling
  2. MFA Delete is enabled (requires confirmation)

Solutions:

# Check status
aws s3api get-bucket-versioning --bucket my-company-app-data

# Re-apply configuration
kubectl patch s3bucket production-bucket \
--type merge \
-p '{"spec":{"versioning":{"enabled":true}}}'

# If MFA Delete is enabled, disable it
aws s3api put-bucket-versioning \
--bucket my-company-app-data \
--versioning-configuration Status=Enabled,MFADelete=Disabled

Bucket deletion stuck

Symptoms: CR deletion remains pending indefinitely

Cause: Finalizer cannot delete bucket (objects, permissions, etc)

Solutions:

# Check finalizers
kubectl get s3bucket production-bucket -o yaml | grep finalizers

# View detailed events
kubectl describe s3bucket production-bucket

# Force remove finalizer (WARNING: bucket will remain in AWS)
kubectl patch s3bucket production-bucket \
-p '{"metadata":{"finalizers":[]}}' --type=merge

# Or change deletion policy first
kubectl patch s3bucket production-bucket \
--type merge \
-p '{"spec":{"deletionPolicy":"Retain"}}'

# Then delete CR
kubectl delete s3bucket production-bucket

CORS not working in web application

Symptoms: CORS error in browser: "No 'Access-Control-Allow-Origin' header"

Cause: CORS not configured or origin not allowed

Solutions:

# Check CORS configuration
aws s3api get-bucket-cors --bucket my-company-app-data

# Test CORS request
curl -i -X OPTIONS https://my-company-app-data.s3.amazonaws.com/ \
-H "Origin: https://app.mycompany.com" \
-H "Access-Control-Request-Method: GET"

# Update CORS to allow your origin
kubectl patch s3bucket api-assets-bucket \
--type merge \
-p '{
"spec": {
"cors": {
"corsRules": [{
"allowedMethods": ["GET", "HEAD", "PUT", "POST", "DELETE"],
"allowedOrigins": ["https://app.mycompany.com"],
"allowedHeaders": ["*"],
"maxAgeSeconds": 3600
}]
}
}
}'

Best Practices

Best Practices
  • Enable public access block — Blocks accidental public access, reduces security breach risks
  • Enable versioning — Protects against accidental deletion and overwrites
  • Configure lifecycle policies — Move to cheaper storage classes, expire old versions
  • Enable encryption — AES256 or KMS encryption for all objects
  • Use bucket policies carefully — Restrict access to specific principals and conditions

Use Cases

1. Static Website Hosting

Example:

# React/Vue/Angular frontend served via S3 + CloudFront
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: website-bucket
spec:
providerRef:
name: production-aws
bucketName: mycompany-website
website:
indexDocument: index.html
errorDocument: 404.html
cors:
corsRules:
- allowedMethods: [GET, HEAD]
allowedOrigins: ["https://mycompany.com"]

2. Data Lake for Analytics

Example:

# Store raw data for analysis with Athena/Redshift
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: data-lake
spec:
providerRef:
name: production-aws
bucketName: mycompany-datalake
versioning:
enabled: true
encryption:
algorithm: AES256
lifecycleRules:
- prefix: raw-data/
transitions:
- days: 90
storageClass: GLACIER

3. Backup and Archiving

Example:

# Store long-term backups
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: backup-bucket
spec:
providerRef:
name: production-aws
bucketName: mycompany-backups
versioning:
enabled: true
lifecycleRules:
- transitions:
- days: 7
storageClass: STANDARD_IA
- days: 30
storageClass: GLACIER
- days: 180
storageClass: DEEP_ARCHIVE
expiration:
days: 2555 # 7 years

4. Application Assets and Media

Example:

# Store images, videos, application documents
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: assets-bucket
spec:
providerRef:
name: production-aws
bucketName: mycompany-app-assets
versioning:
enabled: true
cors:
corsRules:
- allowedMethods: [GET, HEAD, PUT, POST]
allowedOrigins: ["https://app.mycompany.com"]
tags:
Type: application-assets