External anchoring

External anchoring periodically writes each record chain's signed checkpoint to S3-compatible object storage, by default under Object Lock in COMPLIANCE mode. Once written, an anchor is immutable for its retention period - neither an attacker nor an administrator of the AGLedger Server can alter or delete it.

This is the layer that makes tamper-evidence hold against the operator of the system itself, not only against an outside attacker.

Why anchoring matters

Every chain entry is Ed25519-signed and hash-linked, so any modification to stored history is detectable on verification - provided the verifier trusts the signing key. An administrator who holds both the database and the active signing key can, in principle, rewrite a chain and re-sign it; the rewritten chain would verify cleanly. Anchoring closes this gap:

Entries covered by an anchor cannot be rewritten without exposure: the rewritten chain no longer matches the immutable checkpoint in the bucket.
Entries written since the last anchor sweep rely on the signature and hash chain alone. The span between sweeps is your tamper-exposure window - that is what VAULT_ANCHOR_INTERVAL_MINUTES sets.

Anchoring detects tampering after the fact; it does not prevent it in real time. Without it, all tamper-evidence layers live in the same PostgreSQL database a privileged attacker would rewrite - which is why a production Server logs a boot warning while anchoring is disabled.

What gets anchored

Checkpoints are per record chain, swept incrementally on the anchor interval: each sweep checkpoints chains that advanced since their last checkpoint, up to 100 per sweep (the remainder rolls to the next sweep). Each anchor object is a small JSON document (about a kilobyte) carrying the checkpoint's canonical COSE_Sign1 envelope (base64), the record ID, chain position, payload hash, signing key ID, the instance ID, a document version, and the anchor timestamp. Record payloads never leave your database - the anchor is a cryptographic commitment, not a copy.

Objects land at a deterministic key inside the configured bucket:

vault-anchors/<instance>/<YYYY-MM-DD>/<recordId>/<chainPosition>.json

where <instance> is AGLEDGER_INSTANCE_ID and the date is the checkpoint's creation date. The installer generates a UUID for that variable and writes it to your .env, so on a scripted install the segment is that UUID rather than a friendly name. Read it back from .env (or from GET /v1/admin/ops-summary) before going looking for objects by prefix.

Because checkpoints are per-chain, the object count scales with how many chains advance per interval, not with a fixed per-tick rate. A low-volume install produces a handful of objects per sweep; a high-volume install produces up to the 100-chain sweep cap per sweep.

Checkpoints are created even when anchoring is disabled - they are the in-database layer of the same control. Enabling anchoring adds the external, immutable copy.

Enable it (Docker Compose)

Anchoring is disabled by default. In your .env:

VAULT_ANCHOR_ENABLED=true
VAULT_ANCHOR_S3_BUCKET=my-org-agledger-anchors
VAULT_ANCHOR_S3_REGION=us-east-1
# VAULT_ANCHOR_S3_ENDPOINT=http://minio:9000   # non-AWS S3-compatible stores (MinIO, etc.)
VAULT_ANCHOR_INTERVAL_MINUTES=360              # tamper-exposure window; see below
VAULT_ANCHOR_OBJECT_LOCK=true                  # COMPLIANCE-mode immutable retention (default)
VAULT_ANCHOR_RETENTION_DAYS=2555               # ~7 years (default)

Credentials come from the standard AWS credential chain (instance role, AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY, or a profile). Two processes touch the bucket:

The worker runs the anchor sweep and the daily sample-verify job: it needs s3:PutObject (plus s3:PutObjectRetention when Object Lock is on) and s3:GetObject.
The API serves the admin anchor endpoints: it needs s3:GetObject and s3:ListBucket.

Grant only s3:PutObject to the worker and every verify surface in this runbook returns s3_error.

Defaults if unset: bucket vault-anchors, region from AWS_REGION (falling back to us-east-1), Object Lock on, retention 2555 days, interval 360 minutes.

Enable it (Kubernetes / Helm)

The chart passes these through extraEnv (there is no dedicated vault.anchor values block):

extraEnv:
  - name: VAULT_ANCHOR_ENABLED
    value: "true"
  - name: VAULT_ANCHOR_S3_BUCKET
    value: my-org-agledger-anchors
  - name: VAULT_ANCHOR_S3_REGION
    value: us-east-1

On EKS, prefer an IRSA role on the worker's service account over static credentials.

Prepare the bucket

Object Lock must be enabled at bucket creation - it cannot be added to an existing bucket:

aws s3api create-bucket \
  --bucket my-org-agledger-anchors \
  --region us-east-1 \
  --object-lock-enabled-for-bucket

AGLedger applies per-object COMPLIANCE-mode retention of VAULT_ANCHOR_RETENTION_DAYS as each anchor is written. In COMPLIANCE mode no one - including the account root - can delete the object or shorten its retention until it expires. That permanence is the point; choose the retention deliberately.

For the strongest posture, put the bucket in a separately-administered account (or a different provider), so no single set of credentials controls both the database and the anchors. A bucket in the same account as the Server is a single trust domain.

Air-gapped installs: point VAULT_ANCHOR_S3_ENDPOINT at MinIO (path-style addressing is handled automatically) and enable MinIO's object locking on the bucket.

Choose the interval

VAULT_ANCHOR_INTERVAL_MINUTES (default 360, clamped to 5–2880, snapped to cron granularity) is a risk-tolerance dial, not a performance knob. It bounds how much recent history has no external evidence yet:

Tighten (15–60 min) for high-value, irreversible workloads where hours of unpinned records is too much exposure.
Default 360 (6 h) is a reasonable posture for most deployments.
Loosen (up to 2880 = 48 h) where object economy matters more than a tight window.

The sweep is incremental, so short intervals stay cheap on the database; the cost of tightening is the permanent Object Lock object count.

Verify and monitor

Posture at a glance, under vault.anchoring in GET /v1/admin/ops-summary:

{"enabled": true, "intervalMinutes": 5, "bucket": "my-org-agledger-anchors",
 "workerEnabled": true, "reconciled": true}

List a record's anchors (recordId is required):

curl -H "Authorization: Bearer $ADMIN_KEY" \
  "$AGLEDGER_API_URL/v1/admin/vault/anchors?recordId=<uuid>"

{"data":[{"key":"vault-anchors/b736ccb2-.../2026-08-10/019fe911-.../1.json",
  "lastModified":"2026-08-10T00:30:18.234Z","size":1043}],
 "total":1,"nextCursor":null,"hasMore":false}

Verify anchors against the database (exposes any divergence between the live chain and the pinned state):

curl -X POST -H "Authorization: Bearer $ADMIN_KEY" -H "Content-Type: application/json" \
  -d '{"recordId": "<uuid>"}' \
  "$AGLEDGER_API_URL/v1/admin/vault/anchors/verify"

{"data":[{"recordId":"019fe911-...","chainPosition":1,"match":true,
  "detail":"Anchor matches database checkpoint"}]}

A daily background job also verifies a random sample of 20 checkpoints against their anchors (checkpoints not yet anchored are skipped, so a run may verify fewer than 20). Two Prometheus counters back alerting, and they are exposed by different processes, so scrape both:

agledger_vault_anchor_upload_total{outcome}, on the worker, which runs the sweep. Alert on outcome="error": a misconfigured bucket or IAM regression silently widens the tamper-exposure window. A healthy sweep increments outcome="success".
agledger_vault_anchor_verify_total{outcome}, on the API, which serves the verify endpoint. Alert on outcome="tamper" (and investigate no_such_key). A matching verify increments outcome="verified".

Scraping only the API leaves the upload counter unread, and that is the one that tells you anchors stopped landing.

Uploads are deliberately non-blocking: an S3 outage never stops notarization. The counter is the signal that anchors are not landing.

In-database checkpoints (anchored or not) are readable at GET /v1/audit-vault/checkpoints. Full request/response shapes are in the API reference.

Honest limitations

Detect, not prevent. An operator with the signing key can still rewrite within the current window; the next sweep exposes everything already pinned.
One bucket = one trust domain. Separate the account administering the anchors from the one running the Server.
COMPLIANCE retention is irreversible. You cannot shorten it after the fact.

Key-compromise runbook - anchors are what distinguish pre-compromise history from a forged rewrite.
Audit runbook - offline chain verification with published keys.
Day-2 operations - ops-summary and alerting.

Validated against API v1.4.0 on 2026-08-09 (Developer Edition, Docker Compose), the first end-to-end run of the anchoring flow since v1.1.0. The air-gapped path is what made it reachable: a MinIO bucket created with object locking, the anchor endpoint pointed at it, a five-minute interval. Run live: records notarized, the sweep firing on its interval, anchor objects landing at exactly the documented key layout, the object body carrying the fields described above, the vault.anchoring posture block, both admin endpoints (transcripts above), and both Prometheus counters incrementing with real label values. The COMPLIANCE-mode retention claims stay desk-reviewed: MinIO object locking exercised the write path, but it is not S3 Object Lock in COMPLIANCE mode.

Validated against API v1.1.0 on 2026-06-10.

Reviewed for API v1.3.4 on 2026-08-03: the anchoring flow is unchanged, and this page was not re-run end to end. Validating it properly needs a bucket created with Object Lock enabled, which cannot be added to an existing bucket, so the COMPLIANCE-mode retention claims stay desk-reviewed. What was confirmed live on a v1.3.4 install: with anchoring disabled, POST /v1/admin/vault/anchors/verify returns match: false with detail: "Vault anchoring is not enabled", and GET /v1/admin/vault/anchors requires a recordId querystring parameter.

Reviewed for API v1.3.3 on 2026-07-20: 1.3.3 hardens audit-vault chain-scan detection (anchors remain what distinguish pre-compromise history from a forged rewrite) and adds opt-in verdict per-actor signatures; the external-anchoring flow is unchanged.