S3 API¶

leBLOB speaks the S3 protocol with SigV4 signatures. This page lists what it supports, what it refuses, and how it refuses. Its compatibility is measured with the ceph/s3-tests suite: 349 tests pass. The list of passing tests is kept in the repository and checked on every change.

Throughout, leBLOB refuses what it can't honour. A request asking for something leBLOB doesn't do, such as server-side encryption with managed keys, gets 501 NotImplemented. Ignoring an encryption header, for instance, would store plaintext while the client believes it is encrypted.

Operations¶

The "Scoped key" column gives the level a scoped key needs on the bucket. A full-access key may do everything. "Anonymous" marks what a request with no signature may do on a public bucket or object.

Service and buckets¶

Listing¶

Objects¶

Multipart uploads¶

The ETag of a multipart object is the MD5 of its parts' MD5s, followed by -N, as on AWS.

Requests¶

Signatures¶

• SigV4 in the Authorization header, and in presigned URLs.
• Payloads: a signed SHA-256, UNSIGNED-PAYLOAD, aws-chunked with signed chunks, and the streaming variants with trailers, signed or not.
• Every x-amz-* header must be covered by the signature, presigned URLs included.
• Clock skew of up to 15 minutes is accepted.
• SigV2 and SigV4A are refused. boto3 presigns with SigV2 unless told otherwise: create its client with config=Config(signature_version="s3v4").
• A request with no signature at all is anonymous. A request with a wrong signature is refused, not treated as anonymous.

Integrity¶

• Content-MD5, x-amz-content-sha256, and x-amz-checksum- with crc32, crc32c, crc64nvme, sha1 and sha256 are verified as the body arrives, stored, and returned with x-amz-checksum-mode: ENABLED.
• A multipart upload's checksum, composite or full-object, is worked out from its parts and checked against what the client claims at completion.

Conditions and ranges¶

• Reads: If-Match, If-None-Match, If-Modified-Since, If-Unmodified-Since.
• Writes: If-Match and If-None-Match. Deletes: If-Match. They are checked against the current version only.
• A single byte range on GET, answered with 206.

Stored headers¶

Content-Type, Content-Encoding, Content-Disposition, Content-Language, Cache-Control, Expires and x-amz-meta-* are kept with the object and returned on reads. Non-ASCII metadata comes back as raw bytes, where S3 encodes it.

Addressing¶

Path-style (/bucket/key) always works. Virtual-hosted style (bucket.s3.example.org) works when the server is started with --domain. Keys are kept byte for byte: foo/, a//b, a/../b, +, % and Unicode are all distinct, valid keys. A key is valid UTF-8, at most 1024 bytes, with no NUL.

Expect: 100-continue and HTTP chunked uploads are handled.

Encryption¶

SSE-C, server-side encryption with a key sent by the client, works over HTTPS. See SSE-C.

Refused with 501¶

These query parameters select an operation leBLOB doesn't have, on a request where it does have another meaning, so leBLOB answers them with 501 NotImplemented. Each is refused only where it isn't supported: ?versioning, ?lifecycle, ?cors, ?acl, ?tagging and ?versionId, for instance, work on the operations listed above.

accelerate, analytics, attributes, encryption, intelligent-tiering, inventory, legal-hold, logging, metrics, notification, object-lock, ownershipControls, partNumber, policy, policyStatus, publicAccessBlock, replication, requestPayment, restore, retention, select, torrent, website.

These headers get a 501 too:

• x-amz-server-side-encryption and its variants, other than the SSE-C customer-key headers;
• x-amz-grant-*, and x-amz-acl with any value but private or public-read;
• x-amz-object-lock-*, and x-amz-bucket-object-lock-enabled with any value but false;
• x-amz-storage-class with any value but STANDARD;
• x-amz-website-redirect-location and x-amz-if-match-*.

A header sent with an operation that wouldn't use it is also refused:

• a copy source on a request that doesn't copy;
• a copy range anywhere but UploadPartCopy;
• tags or a public ACL where they can't be set;
• an encryption key on a request that can't use one.

Not implemented¶

Planned for later, without a date:

• GetObject and HeadObject by partNumber, and GetObjectAttributes.
• Bucket tagging.
• Object Lock and retention.
• Lifecycle transitions, and the x-amz-expiration header.
• Conditional deletes by size or time.
• A small subset of bucket policies.
• SSE-S3 (encryption with keys the server manages).
• Quotas, event notifications and a web console.

Left out unless a real need appears:

• SigV2 and SigV4A.
• ACLs beyond the canned private and public-read.
• Storage classes other than STANDARD, Glacier and restore.
• S3 Select, Inventory, Analytics, website hosting and BitTorrent.
• S3 Express and appendable objects.