Backup and restore¶
A store has two parts: the metadata in meta.db, and the blob files on its volumes. leblob backup copies the metadata. The blobs need copies of their own, either as several copies on several disks or by copying objects to another store.
What a backup holds¶
The backup is a copy of meta.db: buckets, objects, versions, the list of blobs each object is made of, settings such as lifecycle rules and CORS, and the access keys with their secrets.
It holds no blobs. Blob files have random names, so without the metadata they can't be put back together into objects, and losing meta.db loses every object. A backup protects against losing or damaging the metadata. Against losing a disk, keep copies on several disks.
So keep the backup on another disk than meta.db, and treat it as secret, since it holds the keys.
Making a backup¶
This works while the server runs, and writers don't wait for it. It copies the database in one step through SQLite's backup API. The file is created readable by its owner only. An existing file is never replaced.
At 1M keys under load, a backup took 2.5 s, with no visible effect on PUT or GET, measured on the Linux test machine (btrfs on a consumer NVMe).
A backup taken every hour, from cron or a systemd timer, leaves at most the last hour's new writes at risk, as below.
Restoring¶
Stop the server first, and move the damaged meta.db aside along with any meta.db-wal and meta.db-shm. Then:
leblob restore /backups/meta-2026-11-08-14.db --data-dir /srv/leblob
leblob fsck --data-dir /srv/leblob
restore checks the backup's integrity and puts it in place. It refuses to replace an existing meta.db, or a leftover -wal or -shm file, because SQLite would replay a stale write-ahead log over the backup.
After a restore, the store is as it was when the backup was taken:
- Objects written since then are lost. Their blob files become orphans.
- Objects replaced or deleted since then come back as they were, as long as their blobs haven't been collected. Deleted blobs are kept for a grace period of one hour, so a backup less than an hour older than the loss brings back everything except new writes.
- With an older backup, some blobs are missing.
fsckreports them by blob ID. It can't name their keys yet.
Checking a store with fsck¶
compares the metadata with the volumes and exits non-zero if they disagree. It reports:
- blobs the metadata lists that are missing from a volume, or whose file has the wrong size;
- blobs lost for good, after a volume was forgotten with
--accept-loss; - blob files with no metadata, called orphans: wasted space, never visible to clients. Those under a day old may be writes about to commit, and are listed apart without failing the check;
- blobs with fewer copies than asked for, which repair will restore, and which don't fail the check either.
It refuses a path that holds no store, which it would otherwise report as a clean, empty one. It also refuses a volume it can't read.
A crash between writing a blob and committing its metadata leaves an orphan. So does a restore. To remove orphans:
which removes only blob files over a day old, so an upload in progress is never touched.
fsck checks that the files exist and have the right size. Checking their contents is scrub's job. It loads every blob ID into memory: at 10M keys it took 4.51 GiB and 548 s from a cold cache.
Moving the objects themselves¶
To keep a second copy of the data on another machine, copy the objects with an S3 tool, for example rclone sync into another store. The migration guide describes what such a copy keeps and what it loses.