S3 Bucket - Object Storage
Scalable, durable, and secure cloud object storage with versioning, encryption, and access control.
Prerequisite: AWSProvider Configuration
Before creating any AWS resource, you need to configure an AWSProvider that manages credentials and authentication with AWS.
IRSA:
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: AWSProvider
metadata:
name: production-aws
namespace: default
spec:
region: us-east-1
roleARN: arn:aws:iam::123456789012:role/infra-operator-role
defaultTags:
managed-by: infra-operator
environment: production
Static Credentials:
apiVersion: v1
kind: Secret
metadata:
name: aws-credentials
namespace: default
type: Opaque
stringData:
access-key-id: test
secret-access-key: test
---
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: AWSProvider
metadata:
name: localstack
namespace: default
spec:
region: us-east-1
accessKeyIDRef:
name: aws-credentials
key: access-key-id
secretAccessKeyRef:
name: aws-credentials
key: secret-access-key
defaultTags:
managed-by: infra-operator
environment: test
Verify Status:
kubectl get awsprovider
kubectl describe awsprovider production-aws
For production, always use IRSA (IAM Roles for Service Accounts) instead of static credentials.
Create IAM Role for IRSA
To use IRSA in production, you need to create an IAM Role with the necessary permissions:
Trust Policy (trust-policy.json):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::123456789012:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:sub": "system:serviceaccount:infra-operator-system:infra-operator-controller-manager",
"oidc.eks.us-east-1.amazonaws.com/id/EXAMPLED539D4633E53DE1B71EXAMPLE:aud": "sts.amazonaws.com"
}
}
}
]
}
IAM Policy - S3 Bucket (s3-policy.json):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:CreateBucket",
"s3:DeleteBucket",
"s3:ListBucket",
"s3:GetBucketVersioning",
"s3:PutBucketVersioning",
"s3:GetBucketEncryption",
"s3:PutBucketEncryption",
"s3:GetBucketPublicAccessBlock",
"s3:PutBucketPublicAccessBlock",
"s3:GetBucketTagging",
"s3:PutBucketTagging",
"s3:GetBucketLifecycleConfiguration",
"s3:PutBucketLifecycleConfiguration",
"s3:GetBucketCors",
"s3:PutBucketCors"
],
"Resource": "*"
}
]
}
Create Role with AWS CLI:
# 1. Get EKS cluster OIDC Provider
export CLUSTER_NAME=my-cluster
export AWS_REGION=us-east-1
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
OIDC_PROVIDER=$(aws eks describe-cluster \
--name $CLUSTER_NAME \
--region $AWS_REGION \
--query "cluster.identity.oidc.issuer" \
--output text | sed -e "s/^https:\/\///")
# 2. Update trust-policy.json with correct values
cat > trust-policy.json <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${OIDC_PROVIDER}:sub": "system:serviceaccount:infra-operator-system:infra-operator-controller-manager",
"${OIDC_PROVIDER}:aud": "sts.amazonaws.com"
}
}
}
]
}
EOF
# 3. Create IAM Role
aws iam create-role \
--role-name infra-operator-s3-role \
--assume-role-policy-document file://trust-policy.json \
--description "Role for Infra Operator S3 management"
# 4. Create and attach policy
aws iam put-role-policy \
--role-name infra-operator-s3-role \
--policy-name S3Management \
--policy-document file://s3-policy.json
# 5. Get Role ARN
aws iam get-role \
--role-name infra-operator-s3-role \
--query 'Role.Arn' \
--output text
Annotate Operator ServiceAccount:
# Add annotation to operator ServiceAccount
kubectl annotate serviceaccount infra-operator-controller-manager \
-n infra-operator-system \
eks.amazonaws.com/role-arn=arn:aws:iam::123456789012:role/infra-operator-s3-role
Replace 123456789012 with your AWS Account ID and EXAMPLED539D4633E53DE1B71EXAMPLE with your OIDC provider ID.
Overview
Amazon S3 (Simple Storage Service) is a massively scalable object storage service that offers enterprise-class durability, availability, and security. S3 buckets can be used for various use cases: backup, static website files, data lakes, log repositories, and much more.
Features:
- Unlimited object storage (each up to 5 TB)
- 99.999999999% (11 nines) annual durability
- 99.99% availability in multi-AZ
- Automatic replication between availability zones
- Versioning for object history control
- Encryption at rest (AES256 or KMS)
- Encryption in transit (HTTPS)
- Public Access Block for security
- Lifecycle rules for cost optimization
- CORS for web applications
- Static website hosting
- CloudFront integration for CDN
- Granular control via bucket policies and IAM
- Access logging
- No creation cost, pay only for stored data
Quick Start
S3 Bucket Full-Featured:
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: e2e-test-bucket
namespace: default
spec:
providerRef:
name: localstack
bucketName: e2e-test-bucket-infra-operator
# Enable versioning
versioning:
enabled: true
# Encryption
encryption:
algorithm: AES256
# Block public access
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true
# Tags
tags:
test-type: e2e
component: s3-controller
# Retain bucket after test (for debugging)
deletionPolicy: Delete
Simple S3 Bucket:
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: e2e-simple-bucket
namespace: default
spec:
providerRef:
name: localstack
bucketName: e2e-simple-bucket
tags:
test-type: e2e
variant: simple
deletionPolicy: Delete
Apply:
kubectl apply -f s3-bucket.yaml
Verify Status:
kubectl get s3bucket e2e-test-bucket
kubectl describe s3bucket e2e-test-bucket
kubectl get s3bucket e2e-test-bucket -o yaml
Configuration Reference
Required Fields
Reference to AWSProvider resource
AWSProvider resource name
Unique S3 bucket name. Must be globally unique across all of AWS.
Rules:
- Minimum 3 characters, maximum 63
- Only lowercase letters, numbers, hyphens, and dots
- Cannot start or end with hyphen or dot
- Cannot contain IP address (e.g., 192.168.1.1)
- Cannot contain underscore (_)
Example:
bucketName: my-company-app-data-prod-2025
Optional Fields
Versioning configuration to maintain object history
Example:
versioning:
enabled: true
Details:
enabled: true: Keeps all versions of each objectenabled: false: No versioning (default)- Allows recovery of deleted or overwritten objects
- Requires more storage
- Adds storage cost per version
Encryption at rest for the bucket
Example:
encryption:
algorithm: AES256 # or KMS
kmsKeyID: "arn:aws:kms:us-east-1:123456789012:key/12345678-1234-1234-1234-123456789012"
Options:
AES256: AWS-managed server-side encryption (default, no additional cost)KMS: AWS KMS encryption (requires CMK, additional cost)- If using KMS,
kmsKeyIDis required
Blocks public access to the bucket (highly recommended)
Example:
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true
Fields:
blockPublicAcls: Blocks new public ACLs (recommended: true)ignorePublicAcls: Ignores existing public ACLs (recommended: true)blockPublicPolicy: Blocks public policies (recommended: true)restrictPublicBuckets: Restricts public access via policies (recommended: true)
Rules for automatic transition or deletion of objects based on age
Example:
lifecycleRules:
- id: archive-old-logs
enabled: true
prefix: logs/
filter:
tags:
archived: "true"
transitions:
- days: 30
storageClass: STANDARD_IA
- days: 90
storageClass: GLACIER
- days: 180
storageClass: DEEP_ARCHIVE
expiration:
days: 365
noncurrentVersionTransitions:
- days: 30
storageClass: STANDARD_IA
noncurrentVersionExpiration:
days: 90
Use Cases:
- Save costs by moving old objects to cheaper classes
- Delete logs after retention period
- Archive data for compliance
Configure CORS (Cross-Origin Resource Sharing) for website access
Example:
cors:
corsRules:
- allowedMethods:
- GET
- PUT
- POST
allowedOrigins:
- "https://example.com"
- "https://app.example.com"
allowedHeaders:
- "Authorization"
- "Content-Type"
exposeHeaders:
- "x-amz-meta-*"
maxAgeSeconds: 3000
Usage: Allow AJAX requests from browsers to S3
Configure bucket for static website hosting
Example:
website:
indexDocument: index.html
errorDocument: 404.html
Details:
indexDocument: Default file when accessing the bucketerrorDocument: File displayed on 4xx errors- Bucket MUST have
publicAccessBlock: falseto work - Requires bucket policy allowing
GetObject
Key-value pairs to tag the bucket
Example:
tags:
Environment: production
Application: my-app
Team: platform
CostCenter: engineering
ManagedBy: infra-operator
What happens to the bucket when the CR is deleted
Options:
Delete: Bucket is deleted from AWS (WARNING: objects will be lost)Retain: Bucket remains in AWS but not managedOrphan: Remove only management but keep bucket
Example:
deletionPolicy: Retain
Status Fields
After the S3 Bucket is created, the following status fields are populated:
Created bucket name
Bucket ARN (Amazon Resource Name) (e.g., arn:aws:s3:::my-bucket)
AWS region where the bucket was created
Whether versioning is enabled on the bucket
Whether encryption is enabled on the bucket
Encryption algorithm used (AES256 or KMS)
Bucket access URL (e.g., https://my-bucket.s3.amazonaws.com)
true when the bucket is created and ready for use
Timestamp of last synchronization with AWS
Examples
Production S3 Bucket with Versioning and Encryption
Example:
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: production-bucket
namespace: default
spec:
providerRef:
name: production-aws
bucketName: mycompany-app-data-prod
# Enable versioning for recovery
versioning:
enabled: true
# AES256 encryption (no additional cost)
encryption:
algorithm: AES256
# Maximum security against public access
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true
tags:
Environment: production
Application: data-storage
Team: platform
ManagedBy: infra-operator
CostCenter: infrastructure
# Retain bucket when deleting CR (safety)
deletionPolicy: Retain
S3 Bucket for Static Website Hosting
Example:
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: website-bucket
namespace: default
spec:
providerRef:
name: production-aws
bucketName: mycompany-website-prod
# Configure for website hosting
website:
indexDocument: index.html
errorDocument: 404.html
# Allow CORS requests from browsers
cors:
corsRules:
- allowedMethods:
- GET
- HEAD
allowedOrigins:
- "https://mycompany.com"
- "https://www.mycompany.com"
maxAgeSeconds: 3000
# Basic encryption
encryption:
algorithm: AES256
# Block public access (access via CloudFront/policies)
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true
tags:
Environment: production
Type: static-website
Application: corporate-website
deletionPolicy: Retain
S3 Bucket with Lifecycle Rules for Cost Optimization
Example:
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: logs-bucket
namespace: default
spec:
providerRef:
name: production-aws
bucketName: mycompany-logs-archive-prod
# Versioning for compliance
versioning:
enabled: true
encryption:
algorithm: AES256
# Lifecycle rules for savings
lifecycleRules:
# Transition to cheaper classes
- id: transition-to-cheaper-storage
enabled: true
prefix: logs/
transitions:
- days: 30 # After 30 days: Standard-IA
storageClass: STANDARD_IA
- days: 90 # After 90 days: Glacier
storageClass: GLACIER
- days: 365 # After 1 year: Deep Archive
storageClass: DEEP_ARCHIVE
# Delete old versions
- id: cleanup-old-versions
enabled: true
noncurrentVersionTransitions:
- days: 30
storageClass: STANDARD_IA
noncurrentVersionExpiration:
days: 90
# Delete incomplete multipart upload objects
- id: cleanup-incomplete-multipart
enabled: true
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true
tags:
Environment: production
Type: logs-archive
Purpose: compliance
deletionPolicy: Retain
S3 Bucket with CORS for Web Application
Example:
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: api-assets-bucket
namespace: default
spec:
providerRef:
name: production-aws
bucketName: mycompany-api-assets-prod
# Configure CORS for web application
cors:
corsRules:
- id: allow-frontend-app
allowedMethods:
- GET
- HEAD
- PUT
- POST
- DELETE
allowedOrigins:
- "https://app.mycompany.com"
- "https://dashboard.mycompany.com"
allowedHeaders:
- "*"
exposeHeaders:
- "x-amz-meta-*"
- "ETag"
maxAgeSeconds: 3600
encryption:
algorithm: AES256
versioning:
enabled: true
publicAccessBlock:
blockPublicAcls: true
ignorePublicAcls: true
blockPublicPolicy: true
restrictPublicBuckets: true
tags:
Environment: production
Type: application-assets
Application: web-api
deletionPolicy: Retain
Verification
Verify Bucket Status
Command:
# List all created S3 Buckets
kubectl get s3buckets
# Get detailed information
kubectl get s3bucket production-bucket -o yaml
# Describe bucket (shows events and status)
kubectl describe s3bucket production-bucket
# Watch creation in real-time
kubectl get s3bucket production-bucket -w
Verify on AWS
AWS CLI:
# List buckets
aws s3 ls
# Get versioning configuration
aws s3api get-bucket-versioning --bucket my-company-app-data
# Get encryption configuration
aws s3api get-bucket-encryption --bucket my-company-app-data
# Get public access configuration
aws s3api get-public-access-block --bucket my-company-app-data
# Get lifecycle configuration
aws s3api get-bucket-lifecycle-configuration --bucket my-company-app-data
# Get CORS configuration
aws s3api get-bucket-cors --bucket my-company-app-data
# Get all tags
aws s3api get-bucket-tagging --bucket my-company-app-data
# Check bucket size
aws s3 ls s3://my-company-app-data --summarize --human-readable --recursive
# List objects
aws s3 ls s3://my-company-app-data --recursive
LocalStack:
# For LocalStack testing
export AWS_ENDPOINT_URL=http://localhost:4566
aws s3 ls
aws s3api get-bucket-versioning --bucket my-company-app-data
aws s3api get-bucket-encryption --bucket my-company-app-data
Expected Output
Example:
status:
bucketName: my-company-app-data
bucketArn: arn:aws:s3:::my-company-app-data
region: us-east-1
bucketURL: https://my-company-app-data.s3.amazonaws.com
versioningEnabled: true
encryptionEnabled: true
encryptionAlgorithm: AES256
ready: true
lastSyncTime: "2025-11-22T20:30:15Z"
Troubleshooting
Error: Bucket name already exists
Symptoms: Creation fails with error "BucketAlreadyExists" or "BucketAlreadyOwnedByYou"
Cause: Bucket name already exists and is used by another AWS account or yourself
Solutions:
# Bucket names are globally unique in AWS
# Check if bucket exists
aws s3 ls | grep my-bucket-name
# Use a more unique name (add timestamp, region, etc)
bucketName: my-company-app-data-prod-us-east-1-20250122
# Or check if it's your bucket and reuse it
kubectl patch s3bucket my-bucket \
--type merge \
-p '{"metadata":{"finalizers":[]}}'
Bucket stuck in NotReady
Symptoms: Bucket remains in NotReady state after creation
Common causes:
- Insufficient IAM permissions
- Bucket quota reached (default: 100 buckets per account)
- AWS connectivity issue
Solutions:
# Check operator logs
kubectl logs -n infra-operator-system \
deploy/infra-operator-controller-manager \
--tail=100
# Check detailed status
kubectl describe s3bucket production-bucket
# Check if AWSProvider is ready
kubectl get awsprovider
kubectl describe awsprovider production-aws
# Try force sync with annotation
kubectl annotate s3bucket production-bucket \
force-sync="$(date +%s)" --overwrite
Access denied to bucket
Symptoms: Error "AccessDenied" or "403 Forbidden"
Cause: Insufficient IAM permissions or bucket policy blocking access
Solutions:
# Check IAM permissions of role/user
# IAM policy MUST include s3:* or specific actions
# Check bucket policy
aws s3api get-bucket-policy --bucket my-company-app-data
# Test access
aws s3 ls s3://my-company-app-data
# Verify credentials using
aws sts get-caller-identity
# If using IRSA, check service account
kubectl describe sa infra-operator-controller-manager \
-n infra-operator-system
Cannot delete bucket (contains objects)
Symptoms: Error when deleting: "The bucket you tried to delete is not empty"
Cause: Bucket contains objects that need to be deleted first
Solutions:
# Option 1: Delete objects manually
aws s3 rm s3://my-company-app-data --recursive
# Option 2: Use deletionPolicy: Delete
# This deletes objects automatically when deleting CR
kubectl patch s3bucket production-bucket \
--type merge \
-p '{"spec":{"deletionPolicy":"Delete"}}'
# Option 3: If versioning enabled, delete versions
aws s3api list-object-versions \
--bucket my-company-app-data \
--output json > versions.json
# Then delete each version
aws s3api delete-object --bucket my-company-app-data \
--key <key> --version-id <version-id>
# Only then delete CR
kubectl delete s3bucket production-bucket
High S3 storage costs
Symptoms: AWS bills with unexpected costs
Cause: Data is not transitioned to cheaper classes
Solutions:
# 1. Implement Lifecycle Rules
# Transition to STANDARD_IA after 30 days (~50% of cost)
# Transition to GLACIER after 90 days (~20% of cost)
# 2. Clean up old data
aws s3api delete-objects --bucket my-bucket \
--delete 'Objects=[{Key=old-file.txt}]'
# 3. Check current usage
aws s3 ls s3://my-bucket --summarize --human-readable --recursive
# 4. Use S3 Intelligent-Tiering (automatic)
# Automatically transitions based on access
# 5. Consider S3 Select for efficient queries
# Read only necessary data instead of entire object
Versioning not working
Symptoms: Versioning enabled: false even after configuration
Causes:
- Versioning was disabled after enabling
- MFA Delete is enabled (requires confirmation)
Solutions:
# Check status
aws s3api get-bucket-versioning --bucket my-company-app-data
# Re-apply configuration
kubectl patch s3bucket production-bucket \
--type merge \
-p '{"spec":{"versioning":{"enabled":true}}}'
# If MFA Delete is enabled, disable it
aws s3api put-bucket-versioning \
--bucket my-company-app-data \
--versioning-configuration Status=Enabled,MFADelete=Disabled
Bucket deletion stuck
Symptoms: CR deletion remains pending indefinitely
Cause: Finalizer cannot delete bucket (objects, permissions, etc)
Solutions:
# Check finalizers
kubectl get s3bucket production-bucket -o yaml | grep finalizers
# View detailed events
kubectl describe s3bucket production-bucket
# Force remove finalizer (WARNING: bucket will remain in AWS)
kubectl patch s3bucket production-bucket \
-p '{"metadata":{"finalizers":[]}}' --type=merge
# Or change deletion policy first
kubectl patch s3bucket production-bucket \
--type merge \
-p '{"spec":{"deletionPolicy":"Retain"}}'
# Then delete CR
kubectl delete s3bucket production-bucket
CORS not working in web application
Symptoms: CORS error in browser: "No 'Access-Control-Allow-Origin' header"
Cause: CORS not configured or origin not allowed
Solutions:
# Check CORS configuration
aws s3api get-bucket-cors --bucket my-company-app-data
# Test CORS request
curl -i -X OPTIONS https://my-company-app-data.s3.amazonaws.com/ \
-H "Origin: https://app.mycompany.com" \
-H "Access-Control-Request-Method: GET"
# Update CORS to allow your origin
kubectl patch s3bucket api-assets-bucket \
--type merge \
-p '{
"spec": {
"cors": {
"corsRules": [{
"allowedMethods": ["GET", "HEAD", "PUT", "POST", "DELETE"],
"allowedOrigins": ["https://app.mycompany.com"],
"allowedHeaders": ["*"],
"maxAgeSeconds": 3600
}]
}
}
}'
Best Practices
- Enable public access block — Blocks accidental public access, reduces security breach risks
- Enable versioning — Protects against accidental deletion and overwrites
- Configure lifecycle policies — Move to cheaper storage classes, expire old versions
- Enable encryption — AES256 or KMS encryption for all objects
- Use bucket policies carefully — Restrict access to specific principals and conditions
Use Cases
1. Static Website Hosting
Example:
# React/Vue/Angular frontend served via S3 + CloudFront
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: website-bucket
spec:
providerRef:
name: production-aws
bucketName: mycompany-website
website:
indexDocument: index.html
errorDocument: 404.html
cors:
corsRules:
- allowedMethods: [GET, HEAD]
allowedOrigins: ["https://mycompany.com"]
2. Data Lake for Analytics
Example:
# Store raw data for analysis with Athena/Redshift
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: data-lake
spec:
providerRef:
name: production-aws
bucketName: mycompany-datalake
versioning:
enabled: true
encryption:
algorithm: AES256
lifecycleRules:
- prefix: raw-data/
transitions:
- days: 90
storageClass: GLACIER
3. Backup and Archiving
Example:
# Store long-term backups
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: backup-bucket
spec:
providerRef:
name: production-aws
bucketName: mycompany-backups
versioning:
enabled: true
lifecycleRules:
- transitions:
- days: 7
storageClass: STANDARD_IA
- days: 30
storageClass: GLACIER
- days: 180
storageClass: DEEP_ARCHIVE
expiration:
days: 2555 # 7 years
4. Application Assets and Media
Example:
# Store images, videos, application documents
apiVersion: aws-infra-operator.runner.codes/v1alpha1
kind: S3Bucket
metadata:
name: assets-bucket
spec:
providerRef:
name: production-aws
bucketName: mycompany-app-assets
versioning:
enabled: true
cors:
corsRules:
- allowedMethods: [GET, HEAD, PUT, POST]
allowedOrigins: ["https://app.mycompany.com"]
tags:
Type: application-assets