Backup

A complete, restorable backup of a Server is three separate things. Two of them are not in the database dump, and the operator decides where each lives.

| What | Where it lives | In the DB dump? | |---|---|---| | The PostgreSQL database - records, the audit_vault chain, the signing-key registry, provisioning state | Postgres | Yes | | The customer-held encryption keys (encrypted mode) | Wherever you keep them - the Server never stores them | No | | The config-as-code provisioning files | Your PROVISIONING_CONFIG_PATH directory / source control | No |

The split is the point. A database dump without the encryption keys is opaque-but-safe - an attacker who steals it cannot read encrypted payloads. The encryption keys without the database are useless. Back up each on its own schedule, to its own place.

This runbook pairs with the recovery runbook - a backup you have never restored is a hope, not a backup.

Back up the database

scripts/backup.sh - from the agledger-ai/install repository - is the supported path. It writes a timestamped, compressed tarball and prunes to the last --keep N (default 7):

./scripts/backup.sh            # keep last 7
./scripts/backup.sh --keep 30
BACKUP_DIR=/mnt/backups ./scripts/backup.sh

Each tarball contains the database dump and, in almost every case, the key-registry export. Under the hood the script runs pg_dump in custom format against the bundled postgres (or your external DATABASE_URL for Aurora/RDS):

db.dump                  # pg_dump -Fc — the full database, custom format
vault-public-keys.csv    # the signing-key registry, PUBLIC keys only

PostgreSQL backup complete (732K).

The vault-public-keys.csv is public-key metadata only - fingerprints, algorithms, status, activation and retirement dates. Private signing keys are never written to the database and never appear in a backup:

key_id,public_key,algorithm,status,activated_at,retired_at
c4dd3e20388b594d,MCowBQYDK2VwAyEAo95XH8DQ...,Ed25519,active,2026-06-09 15:54:50.700979+00,

It is there so that after a restore you can confirm the registry came back intact (see the recovery runbook). A fresh install carries the one active key above; after a key rotation the CSV lists the active and retired keys, because retired keys still verify records signed before the rotation.

For an external database (Aurora, RDS, Cloud SQL), back up with your provider's snapshot mechanism instead - backup.sh detects an external DATABASE_URL and uses pg_dump directly. The three-way split above is unchanged.

pg_dump refuses to dump a server newer than itself, and distribution packages lag: Ubuntu 24.04 ships client 16 against the PostgreSQL 18 this product is validated on. backup.sh compares the two and falls back to a matching client in a container when the host's is older, so the case that needs your attention is a host with neither an adequate client nor docker. It names the PostgreSQL apt repository lines when it hits that.

A backup that cannot be read is not written. Before reporting success, backup.sh checks that db.dump begins with the archive header pg_restore expects. If it does not, the whole backup directory is removed and the script exits non-zero naming what it found, rather than leaving a plausible tarball whose only symptom appears during a restore, after the stack is down and the database is dropped. The key-registry export gets the same treatment against its header row; a bad one is dropped and the run says so, leaving a one-file tarball with the database dump intact.

Back up the chain off-box, too

Keep a second, database-independent copy of the chain: the NDJSON dump the offline verifier consumes. This is not a replacement for the database backup - it is the artifact you hand an auditor, and the copy that proves intact without a running Server.

./scripts/vault-dump.sh ./chain-backup

The shipped scripts/vault-dump.sh runs the dump tool inside the Server image, so it needs no source checkout or pnpm - only a reachable database. It prints the dumped row counts and the output directory:

{
  "outDir": "/dump",
  "orgId": null,
  "counts": { "audit_vault": 10, "vault_checkpoints": 0, "vault_signing_keys": 1, "org_admin_reads": 0, "org_admin_reads_checkpoints": 0 }
}

It then prints the same counts as a per-file row list:

[...] Vault dump complete
[...]   Directory: /path/to/chain-backup
[...]   audit_vault.ndjson: 10 rows
[...]   org_admin_reads.ndjson: 0 rows
[...]   org_admin_reads_checkpoints.ndjson: 0 rows
[...]   vault_checkpoints.ndjson: 0 rows
[...]   vault_signing_keys.ndjson: 1 rows

It writes five NDJSON files (see the audit runbook for the contents) including vault_signing_keys.ndjson - the public-key registry travels with the dump, so the chain verifies offline with no further inputs. Keep it alongside the db.dump tarball, not instead of it: the database backup is what you restore a running Server from; the NDJSON dump is what proves the chain to someone who does not trust your Server.

The tool that reads this dump is @agledger/verify, and it takes the directory:

npx -y @agledger/verify ./chain-backup

@agledger/cli's verify subcommand is a different tool. It checks one record's audit export (GET /v1/records/{id}/audit-export) and cannot open a dump directory: point it at one and it reports EISDIR. Either verifier is optional: the per-row cose_sign1 bytes verify under any RFC 9052 library, which is the path to use when you want no AGLedger package in the loop at all.

What a backup does not contain