S3 (Simple Storage Service)

Overview

DevCloud S3 provides basic object storage using the filesystem for object data and SQLite for metadata. Objects are stored as files on disk; bucket and object metadata (names, sizes, ETags, content types, timestamps) are tracked in a SQLite database with WAL mode enabled.

Supported APIs

These 37 operations are hand-verified — implemented by the provider, not by the CRUD engine. A sub-resource this provider does not serve (?lifecycle, ?encryption, …) returns a clean AWS error rather than being quietly answered as a bucket listing; fidelity-manifest.md is the per-operation answer.

OperationDescription
ListBuckets / CreateBucket / DeleteBucket / HeadBucketBucket lifecycle
GetBucketLocationReturns the fixed region
ListObjects / ListObjectsV2List objects, with prefix, delimiter and continuation tokens
PutObjectUpload an object (computes MD5 ETag)
GetObject / HeadObjectDownload an object, or read its metadata alone
CopyObjectServer-side copy via x-amz-copy-source
DeleteObject / DeleteObjectsDelete one object, or a batch
CreateMultipartUpload / UploadPart / CompleteMultipartUploadMultipart upload
AbortMultipartUpload / ListMultipartUploads / ListPartsInspect and abandon multipart uploads
GetBucketTagging / PutBucketTagging / DeleteBucketTaggingBucket tags
GetObjectTagging / PutObjectTagging / DeleteObjectTaggingObject tags
GetBucketCors / PutBucketCors / DeleteBucketCorsCORS configuration (stored, not applied)
GetBucketPolicy / PutBucketPolicy / DeleteBucketPolicyBucket policy (stored, not evaluated)
GetBucketAcl / PutBucketAclBucket ACL (stored, not evaluated; a canned full-control ACL is returned when unset)
GetBucketVersioning / PutBucketVersioningVersioning status (stored, not applied)
GetBucketNotificationConfiguration / PutBucketNotificationConfigurationEvent notifications — these do fire

Notification configuration is the one of those that has behaviour behind it: PutObject, CopyObject, CompleteMultipartUpload and DeleteObject emit ObjectCreated:* / ObjectRemoved:Delete events and deliver them to the configured SQS queue or Lambda function, which is what makes the S3 → Lambda integration work.

boto3 Examples

Create a bucket and upload an object

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="http://localhost:4747",
    aws_access_key_id="test",
    aws_secret_access_key="test",
    region_name="us-east-1",
)

# Create bucket
s3.create_bucket(Bucket="my-bucket")

# Upload object
s3.put_object(Bucket="my-bucket", Key="hello.txt", Body=b"Hello, DevCloud!")

# Download object
response = s3.get_object(Bucket="my-bucket", Key="hello.txt")
print(response["Body"].read().decode())  # Hello, DevCloud!

List buckets and objects

# List all buckets
buckets = s3.list_buckets()
for b in buckets["Buckets"]:
    print(b["Name"])

# List objects in a bucket
objects = s3.list_objects(Bucket="my-bucket", Prefix="hello")
for obj in objects.get("Contents", []):
    print(obj["Key"], obj["Size"])

AWS CLI Examples

# Create bucket
aws --endpoint-url http://localhost:4747 s3 mb s3://my-bucket

# Upload file
aws --endpoint-url http://localhost:4747 s3 cp file.txt s3://my-bucket/

# List objects
aws --endpoint-url http://localhost:4747 s3 ls s3://my-bucket/

# Download file
aws --endpoint-url http://localhost:4747 s3 cp s3://my-bucket/file.txt ./downloaded.txt

Known Limitations

The pattern below is worth reading once: several configuration sub-resources round-trip faithfully but change nothing about how requests are served.

  • Versioning is a stored status, not versions. PutBucketVersioning records Enabled, and GetBucketVersioning reads it back, but no version IDs are assigned and overwriting an object still destroys the previous one.
  • ACLs and bucket policies are stored, never evaluated. No request is ever denied by one. There is no object lock.
  • CORS configuration is stored, never applied. DevCloud does not emit Access-Control-* response headers or answer preflight requests from it.
  • Presigned URLs are not verified. SigV4 signatures are not validated anywhere, so a presigned request is served like any other and an expired or forged one succeeds.
  • No server-side encryption (SSE-S3, SSE-KMS) — ?encryption is unimplemented
  • No lifecycle policies, replication, or storage classes
  • Single account model (account ID: 000000000000)