OpenBao administration and recovery
Prerequisite: install, initialize and unseal the server. Run bao commands on your trusted workstation with BAO_ADDR and CA trust configured. Run service/log commands on the Debian server.
This guide bootstraps a named administrator, then covers daily checks, audit logs, backup and upgrade planning. Application access belongs in scoped KV policies, SSH signing policies and Docker AppRole.
1. Create named administrator access
Start authenticated with the initial root token, using the hidden prompt:
Create a deliberately broad administrator-only policy:
bao policy write platform-admin - <<'HCL'
path "*" {
capabilities = ["create", "read", "update", "patch", "delete", "list", "sudo"]
}
HCL
This is full platform administration, not an application policy
platform-admin can manage policies, authentication and secrets, and grant further privileges. Limit it to trusted platform administrators, use separate daily read/write accounts, and prefer OIDC/LDAP with appropriate identity controls for a shared deployment. Never assign it to a container or CI job. Some operations still require a root token, which custodians can generate for a controlled emergency.
Enable userpass only if it is not already mounted (bao auth list shows existing mounts):
Create a named account without placing its password in shell history or process arguments. Bash printf is a builtin; keep shell tracing disabled:
set +x
read -r -s -p 'New OpenBao password for bao-admin: ' PASSWORD
printf '\n'
printf '%s' "$PASSWORD" | bao write auth/userpass/users/bao-admin \
password=- token_policies=platform-admin token_ttl=1h token_max_ttl=8h
unset PASSWORD
Use a strong, unique password. For production human access, evaluate OIDC, LDAP and login MFA rather than treating this password-only example as a complete identity architecture.
Test before revoking bootstrap root
Use a second shell with a separate token-cache path so testing does not overwrite the first shell's root-token cache:
export BAO_ADDR=https://bao.lab.example:8200
export BAO_CACERT=/path/to/kafana-root-ca.crt
export BAO_TOKEN_PATH="$HOME/.bao-admin-token"
unset BAO_TOKEN
bao login -no-print -method=userpass username=bao-admin
bao token lookup -field=policies
bao token lookup -field=ttl
bao auth list
bao policy list
bao audit list
bao operator raft list-peers
The password prompt is hidden. Confirm the token has platform-admin and that these operations succeed. In the UI, choose Username & Password and auth path userpass; verify named login works there too.
Back in the original root-authenticated shell, revoke that shell's initial root token:
Remove any temporary bootstrap-root copies from protected custody once revoked. Do not revoke root first and only then discover that administrator login is broken.
Continue in the tested administrator shell. Token caches contain credentials: exclude them from Git/backups and protect workstation access. Use bao login -no-print to avoid printing tokens. When finished with a session:
2. Inspect access without broadening it
bao auth list
bao secrets list
bao policy list
bao policy read platform-admin
bao token lookup -field=policies
bao token lookup -field=ttl
bao token capabilities secret/data/apps/docker-demo
bao token capabilities checks the currently authenticated token. A runtime policy should normally allow only its specific data/ path; UI browsing may also require metadata/ listing. Do not solve every 403 by handing out platform-admin.
Changing a policy affects tokens that reference it. Changing an auth user's policy mapping affects subsequent logins; do not assume deleting an auth user revokes all their existing tokens. For offboarding, revoke outstanding sessions/tokens as well as the ability to log in, then rotate any static credentials the user could have copied.
For a known token accessor obtained from controlled session tracking (not its secret token value):
Token revocation cannot erase a credential already copied to a file or invalidate an already-issued SSH certificate on a target host. See SSH revocation limits and Docker rotation.
3. Service checks and audit logs
On the Debian server:
sudo systemctl status openbao --no-pager
sudo journalctl -u openbao --since '1 hour ago' --no-pager
sudo du -sh /var/lib/openbao /var/log/openbao
df -h /var/lib/openbao /var/log/openbao
From the administrator workstation:
Service logs describe server behavior; audit logs record API requests/responses. The installation config enables a declarative file audit device named local-file/ at /var/log/openbao/audit.json.
Most sensitive audit values are HMAC-protected by default, but audit logs still contain sensitive operational metadata. Restrict access and ship them to protected central storage. Do not set log_raw=true casually. If all audit destinations fail, OpenBao can block requests: monitor log disk space and delivery independently of OpenBao.
Rotate the audit file
On Debian, install logrotate if needed and create /etc/logrotate.d/openbao:
/var/log/openbao/audit.json {
daily
rotate 14
missingok
notifempty
compress
delaycompress
su openbao openbao
create 0600 openbao openbao
sharedscripts
postrotate
/usr/bin/systemctl kill --kill-whom=main --signal=HUP openbao.service
endscript
}
HUP makes OpenBao reopen its audit file. Do not use copytruncate: truncation during writes can lose audit records. Check without forcing rotation:
After an actual scheduled rotation, verify new requests appear in the new file and central shipping continues. Adjust retention to your requirements. Removing a declarative audit stanza disables that device on reload/restart; config review matters as much as API permissions.
4. Take Raft snapshots
A Raft snapshot is a consistent backup of OpenBao's persisted state, not a backup of the whole Debian machine. An authorized operator can save one from the workstation:
umask 077
mkdir -p "$HOME/openbao-backups"
SNAPSHOT="$HOME/openbao-backups/bao-$(date -u +%Y%m%dT%H%M%SZ).snap"
bao operator raft snapshot save "$SNAPSHOT"
sha256sum "$SNAPSHOT" > "$SNAPSHOT.sha256"
Store snapshots encrypted off-host with access controls, retention and monitoring. The encrypted data remains dependent on the original seal/unseal material. A snapshot alone is not enough to recover if that material is lost.
For scheduled backups, create a separate backup-reader policy instead of saving an administrator token in a cron script:
bao policy write raft-backup - <<'HCL'
path "sys/storage/raft/snapshot" {
capabilities = ["read"]
}
HCL
Attach it through a suitable machine authentication method and manage its token lifecycle. Restore permission belongs to a different privileged role.
Back up these separately, under appropriate access controls:
- Server configuration and systemd overrides.
- TLS configuration, certificates and protected private-key material or its reissuance process.
- Off-host snapshots and integrity records.
- Audit logs, which are not restored by a Raft snapshot.
- Seal provider configuration or independently held Shamir shares. Do not bundle the threshold with the data backup.
5. Plan and test recovery
Restore overwrites cluster state
Do not try a restore on a live server as a routine verification step. A snapshot can contain valid tokens, old credentials and integrations; an isolated recovery environment must not contact production applications. Restoring an older snapshot can reintroduce access you previously revoked.
First retrieve the selected snapshot, verify its checksum, and prepare the compatible OpenBao version, configuration and independently held original seal material. Connect directly to the recovery environment's active node, not the production address or a load balancer. There are two different cases:
Restore into an instance with the same seal material
For an initialized, unsealed recovery instance that already has the snapshot's seal material, authenticate with appropriately privileged access and use the normal compatibility check:
Re-unseal if necessary and reauthenticate using credentials valid in the restored snapshot.
Fresh-instance disaster recovery with Shamir
A brand-new empty data directory cannot be unsealed with the old shares before restoring the snapshot. Its temporary initialization creates different keys. For this case only, in a fully isolated recovery environment:
- Start a compatible single-node Raft instance on an empty data directory with working TLS, matching intended configuration and a recovery-specific address. Confirm
BAO_ADDRpoints there. - Temporarily initialize it, record its temporary shares/root token securely, unseal it and authenticate using that temporary root token. Follow the hidden-prompt workflow from installation.
- Intentionally bypass the temporary seal mismatch to load the selected original snapshot:
- The temporary initialization state is replaced. Submit the required distinct original snapshot's unseal shares, not the temporary shares, using
bao operator unsealprompts. Discard the temporary root-token cache and log in with an administrator identity that existed in the snapshot. If none is usable, custodians may need the controlledoperator generate-rootprocedure. - Verify expected secret versions, auth methods, policies, peers, auditing and clients before connecting restored infrastructure to real workloads. Reassess restored tokens and rotate exposed/stale upstream credentials.
-force does not decrypt data with the wrong keys or rescue a snapshot whose original seal material is lost. Auto-unseal recovery requires the original provider/key to remain available and a provider-specific procedure; do not apply the Shamir recipe blindly. See snapshot commands and the force-restore API.
6. Upgrade deliberately
For this direct .deb installation:
- Read the target release notes, security advisories and upgrading guide. Check supported upgrade paths, plugin compatibility and new defaults.
- Take an off-host snapshot and configuration backup; confirm recovery material is available.
- Verify the new package signature/checksum as in installation.
- Schedule downtime for the single node and arrange unseal custodians.
- Install the verified package, inspect any configuration-file prompts, restart if required and unseal.
- Check status, audit logging, authentication, KV reads, SSH signing and Agent rendering.
On the workstation, submit the required distinct shares, then check:
Substitute the actual verified package name. The installed binary's bao version is not proof of the version a remote server is running; check server status/logs too. Do not downgrade against an upgraded data directory unless upstream explicitly supports it. A rollback may require restoring a pre-upgrade snapshot to a compatible version in a planned recovery process.
For HA, upgrade one node at a time while maintaining quorum; this single-node procedure is not a rolling-cluster runbook.