Install AGLedger on Kubernetes
This guide brings up a single AGLedger Server on a generic Kubernetes cluster using the published Helm chart, fronted by TLS, backed by your own PostgreSQL. Every command below is a literal transcript captured against a live Server. On OpenShift the same install takes one extra flag; see the OpenShift section below.
Just evaluating? Start here instead. This page is the production Kubernetes install. To get a signed Server running locally in about five minutes with a single command - bundled PostgreSQL, keys generated for you, no cluster - use the quick install on Docker Compose.
Image tag, chart version, and pinned digests updated to v1.4.0 on 2026-08-09. The Kubernetes transcripts (keygen fingerprint, rollout, readiness gate) carry over from the v0.27.5 live validation on 2026-06-04 (Kubernetes 1.35, Helm 3.20, cosign 3.0); they are version-stable and were not re-run against a live cluster for this release. The v1.4.0 image index digest (
sha256:75c9590f…) and chart digest (sha256:b70d8df5…) are read from the published Docker Hub registry after the release ran.v1.4.0 adds a fourth migration (
004_signing_agility.sql), which carries the per-key algorithm the engine now resolves signatures against. The upgrade stays a rolling image bump, but it runs a schema step, so read Version upgrades in Day-2 operations before you bump. v1.3.4 added the third (003_export_cap_and_checkpoint_immutability.sql).The runtime base is Red Hat UBI 10 (
ubi10/nodejs-24-minimal) as of v1.4.0. Nothing in this guide changes; the container runs as the same non-root uid with the same hardening.Release images are multi-arch (
linux/amd64andlinux/arm64). Pulling by tag selects your platform automatically. If you pin by digest, pin the index digest above rather than a per-architecture child digest, or you will pin one architecture. Verified on Graviton (2026-07-27): the index digest resolves per-node, the arm64 runtime is native (process.arch=arm64,aarch64kernel, no emulation), migrations apply, and signatures produced on arm64 verify against the offline verifier on x86.
The API reference is the OpenAPI document the Server serves at /openapi.json. This guide
does not restate request or response schemas; it links to them.
What you provide
A Server is durable only as far as its database and its signing key. You bring both:
- A PostgreSQL database. AGLedger writes a hash-chained, Ed25519-signed record chain to it.
Use any PostgreSQL you can reach over a direct connection (pg-boss needs
LISTEN/NOTIFY, so no transaction-mode pooler). This guide runs one inside the cluster; production should use a managed or HA PostgreSQL. - An Ed25519 vault signing key. The Server signs every record with it. Generate it once, store it like any other root secret, and reuse it across upgrades - the published verification key is derived from it.
Prerequisites
- A Kubernetes cluster (1.27+) and
kubectlpointed at it helm3.x andcosign3.0+- An ingress controller and a way to issue TLS certificates. This guide uses ingress-nginx and cert-manager; any controller and certificate source work.
- A usable
StorageClass- only if you let the chart run PostgreSQL in-cluster (see step 3). Bundled PostgreSQL provisions a PersistentVolumeClaim against the cluster's default class, or an explicitpostgres.bundled.storageClassName. With neither, the claim staysPendingand the install hangs. Modern EKS (1.30+) ships no default class - setpostgres.bundled.storageClassNameto your provisioner's class, e.g.gp3, with the EBS CSI driver installed.
1. Verify the release
Releases are keyless-signed: GitHub Actions OIDC -> Sigstore/Fulcio -> the public Rekor transparency log. There is no static public key to fetch. A valid signature binds to the GitHub Actions workflow in agledger-ai/agledger-api that built the artifact, verifiable with no access to the source repository. Requires cosign 3.0+. Verify both the image and the chart before you install.
$ cosign verify \
--certificate-identity-regexp '^https://github\.com/agledger-ai/agledger-api/\.github/workflows/.+@refs/tags/v.+$' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
agledger/agledger:1.4.0
$ cosign verify \
--certificate-identity-regexp '^https://github\.com/agledger-ai/agledger-api/\.github/workflows/.+@refs/tags/v.+$' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
registry-1.docker.io/agledger/agledger-chart:1.4.0
Expected - each prints Verification for ... -- and a JSON block. The certificate identity in
the output is the GitHub Actions workflow that built the artifact, and the image digest in the
output (sha256:75c9590f…) is the digest you pin in step 3. To additionally check build
provenance, verify the SLSA Build L3 attestation with slsa-verifier verify-image (full recipe
in the install repo's SECURITY.md at https://github.com/agledger-ai/install).
2. Generate the vault signing key
Generate the Ed25519 key with the Server image itself, then keep the private key safe.
$ kubectl run agledger-keygen --restart=Never --image=agledger/agledger:1.4.0 \
--command -- /nodejs/bin/node dist/scripts/generate-signing-key.js
$ kubectl logs agledger-keygen
VAULT_SIGNING_KEY=<base64 Ed25519 private key>
Public key: MCowBQYDK2VwAyEAPInxjno26azIT9i6GVqJag9QuCJFMoDG96iljTd8fHo=
Fingerprint: dcd0573755f045c6
$ kubectl delete pod agledger-keygen
The fingerprint (dcd0573755f045c6) is the key id the Server later publishes at
/v1/verification-keys. Confirming they match (step 5) is how you prove the Server is signing
with the key you provided.
3. Bring up PostgreSQL and install
Create the namespace and a PostgreSQL the Server can reach. Then install the chart, pointing
database.externalUrl at it and passing the signing key.
Simpler path for evaluation: let the chart run PostgreSQL. Set
postgres.bundled.enabled: trueinvalues.yamland omitdatabase.externalUrl- the chart provisions an in-clusterpostgres:18-alpinewith a 10Gi PersistentVolumeClaim. Bundled PostgreSQL is not recommended for production. It requires a usableStorageClass: with no cluster default and no explicitpostgres.bundled.storageClassName, the claim staysPendingand the install hangs. On EKS 1.30+ (no default class) set the class explicitly.postgres: bundled: enabled: true password: <change-me> # also used in the generated database URL storageClassName: gp3 # required on EKS 1.30+; "" uses the cluster defaultConfirm the claim binds before waiting on the workloads -
kubectl get pvc -n agledgershould show the-pgdataclaimBound, notPending. Choose one database path, not both; the steps below use an external PostgreSQL.
$ kubectl create namespace agledger
$ kubectl apply -f postgres.yaml # your PostgreSQL Deployment + Service
$ kubectl rollout status deploy/agledger-postgres -n agledger --timeout=120s
deployment "agledger-postgres" successfully rolled out
values.yaml:
image:
digest: "sha256:75c9590f353c063735728229f07f941fd063b1ab242caee900406ddc1eb5d20c" # 1.4.0
database:
# sslmode is required under nodeEnv=production. Prefer sslmode=verify-full against a TLS-enabled
# database with config.nodeExtraCaCerts pointing at its CA — `require` encrypts but does NOT
# validate the server certificate (the CA bundle is only consulted under verify-ca/verify-full).
# sslmode=disable only suits a local, non-TLS PostgreSQL like the in-cluster one here.
externalUrl: "postgresql://agledger:<db-password>@agledger-postgres.agledger.svc.cluster.local:5432/agledger?sslmode=disable"
config:
externalUrl: "https://agledger.k8s.example" # the Server's signed issuer identity; use your host
ingress:
enabled: true
className: nginx
annotations:
cert-manager.io/cluster-issuer: agledger-selfsigned
hosts:
- host: agledger.k8s.example
paths: [{ path: /, pathType: Prefix }]
tls:
- secretName: agledger-tls
hosts: [agledger.k8s.example]
$ helm install agledger oci://registry-1.docker.io/agledger/agledger-chart \
--version 1.4.0 --namespace agledger \
--values values.yaml --set secrets.vaultSigningKey=<vault-key>
NAME: agledger
STATUS: deployed
REVISION: 1
The chart runs schema migrations as a pre-install hook, then starts the API and worker.
$ kubectl rollout status deploy/agledger-agledger-chart-api -n agledger --timeout=180s
deployment "agledger-agledger-chart-api" successfully rolled out
$ kubectl rollout status deploy/agledger-agledger-chart-worker -n agledger --timeout=120s
deployment "agledger-agledger-chart-worker" successfully rolled out
4. Create the platform API key
The platform key is the first credential; you use it to provision organizations and agents. It is printed once.
$ kubectl exec deploy/agledger-agledger-chart-api -n agledger -- \
/nodejs/bin/node dist/scripts/init.js --non-interactive
✓ API_KEY_SECRET found in environment
✓ VAULT_SIGNING_KEY found in environment
✓ Platform API key created (ID: ...)
✓ Database query OK
│ Platform Key: agl_plt_<store-this-securely>
5. The readiness gate: up and signing
A Server is ready when it is healthy and signing with your key.
$ kubectl port-forward svc/agledger-agledger-chart -n agledger 3001:80 &
$ curl -s http://localhost:3001/health
{"status":"ok","version":"1.4.0","timestamp":"..."}
$ curl -s http://localhost:3001/health/ready
{"status":"ready","version":"1.4.0","timestamp":"..."}
Confirm the active signing key is the one you generated in step 2 - same fingerprint:
$ curl -s http://localhost:3001/v1/verification-keys
{
"data": [{ "keyId": "dcd0573755f045c6", "algorithm": "Ed25519",
"publicKey": "MCowBQYDK2VwAyEAPInxjno26azIT9i6GVqJag9QuCJFMoDG96iljTd8fHo=",
"status": "active" }],
"envelope": "COSE_Sign1", "signatureAlgorithm": "Ed25519"
}
keyId matches the keygen fingerprint: the Server is signing with your key, COSE_Sign1 / Ed25519.
With no license applied the Server runs as Developer Edition (all features enabled):
$ curl -s -H "Authorization: Bearer <platform-key>" http://localhost:3001/v1/admin/license
{"validity":"unlicensed","tier":"developer",
"features":["custom_schemas","expression_engine","delegation_chains","audit_export",
"compliance_reports","encrypted_mode","entity_references","proposals","federation"],
"source":"none"}
6. TLS
With ingress-nginx and cert-manager, the chart's Ingress requests a certificate and serves it. A self-signed ClusterIssuer is portable and needs no public DNS; for production use an ACME (Let's Encrypt) or your-CA issuer.
$ kubectl get certificate -n agledger
NAME READY SECRET AGE
agledger-tls True agledger-tls ...
$ curl -sk https://agledger.k8s.example/health # through the ingress
{"status":"ok","version":"1.4.0","timestamp":"..."}
External database
The in-cluster PostgreSQL above is the simplest start. To use a managed PostgreSQL, point
database.externalUrl at it with sslmode=verify-full and set config.nodeExtraCaCerts to its CA
bundle (require encrypts without validating the server certificate). External-database licensing
is per database instance. For Amazon Aurora / RDS specifically, see the
AWS install guide.
OpenShift
Add --set openshift.enabled=true to the helm install above. Everything else on this page is
unchanged.
openshift.enabled was introduced in chart 1.4.0, the version this page pins. On an earlier chart
the key does not exist, and Helm accepts an unknown --set key without complaint, so the flag is
silently ignored, the pod-level uid request stays, and you get exactly the admission error below.
helm install agledger oci://registry-1.docker.io/agledger/agledger-chart \
--version 1.4.0 --namespace agledger --create-namespace \
--set openshift.enabled=true \
--values values.yaml
Without it, admission rejects every pod before it starts:
unable to validate against any security context constraint:
runAsUser: Invalid value: 65532: must be in the ranges: [1000700000, ...]
OpenShift's default restricted-v2 SCC assigns each namespace its own uid range and admits pods
with MustRunAsRange. The chart's stock pod security context asks for uid/gid 65532, which is
outside that range on essentially every cluster. The flag drops that block from all three workloads
(the api Deployment, the worker Deployment, and the migrate Job) and lets the platform assign a uid
instead. The bundled PostgreSQL needs no equivalent: it requests no security context at all, which
is exactly what restricted-v2 admits.
What goes is runAsNonRoot: true alongside the two uid/gid keys, so non-root stops being asserted
in the manifest. It still holds, by a different guarantee: on OpenShift restricted-v2 enforces it
directly along with a default seccomp profile, and elsewhere it rests on the image's own
USER 65532:65532. Every container-level control is untouched either way, so each container keeps
its read-only root filesystem, no privilege escalation, and all capabilities dropped.
The image supports an arbitrary uid: it declares USER 65532 for plain Kubernetes and Docker, but
nothing in it is owned by or hardcoded to that uid, its files are world-readable, and it needs no
passwd entry.
./scripts/helm-install.sh sets the flag for you when it sees the security.openshift.io API group
on the cluster, unless you already set openshift.enabled yourself, whether by --set or in a
values file it was passed. The chart also ships
values-openshift.yaml, which sets the same flag for anyone who prefers a values file, though
reaching it means helm pull --untar first:
helm pull oci://registry-1.docker.io/agledger/agledger-chart --version <ver> --untar
helm install agledger ./agledger-chart -n agledger \
-f ./agledger-chart/values-openshift.yaml \
-f values.yaml
What was measured. Under the constraints
restricted-v2imposes on a container (--user 1000700000:0 --read-only --cap-drop ALL --security-opt no-new-privileges), the shipped image applies its migrations, boots healthy, serves the unauthenticated discovery surfaces, mints a platform key, and notarizes a record, so signing works under an assigned uid. The rendered manifests were checked directly: the flag removes the pod-level uid request from all three workloads and leaves container hardening identical to a stock render. Admission itself was not exercised on a live OpenShift cluster; that part follows from the SCC rules.
Run more than one Server
You can run more than one Server and link them so chains reference records across Servers (we call linking Servers federation). Each Server is a full, independent install of this guide with its own database and signing key; linking is configured after both are healthy.
Air-gapped install
Nothing in install or runtime depends on agledger.ai, Docker Hub, or npm at runtime. For a
restricted network, mirror the image (agledger/agledger:1.4.0) and chart
(agledger/agledger-chart:1.4.0) into your internal registry, the cosign verification bundle
ships with each GitHub release; see the install repository's air-gap guide for the offline
cosign verify --new-bundle-format flow, and set image.repository (and image.pullSecrets) to
your registry. The Server makes no outbound calls except the webhook and federation endpoints you
configure.
Uninstall
$ helm uninstall agledger -n agledger
release "agledger" uninstalled
This leaves your database and signing key intact, so a reinstall against the same database and key resumes the same chain.
Next
Keep the Server healthy over time with Day-2 Operations, and set up a backup and recovery routine.