Credential management

The shared cluster credential

Every Arctic agent authenticates operator requests with OAuth2 client credentials: a client ID (prefixed cli_) and a client secret (prefixed sec_). The cluster shares one of these credentials across all peers and replicates it through gossip, so an operator can point the CLI at any agent with the same client_id and client_secret and be authenticated everywhere.

Because the secret is shared, rotating it is a cluster-wide event with a deadline. The rest of this page covers the two credentials a bootstrap leaves you with, then listing, creating, revoking, and rotating credentials, and the account lockout that protects the token endpoint.

The curl examples on this page use https with -k because the agent serves TLS with a self-signed certificate by default; identity is verified by fingerprint pinning in the CLI, not a CA chain. See TLS and trust.

The two credentials from bootstrap

A bootstrap run (arctic compose apply on a fresh cluster, or arctic bootstrap) mints two credentials with different jobs:

The working credential holds everything routine CLI work needs, but no admin scope and no credential mutation. Day-to-day commands need no extra setup: the CLI uses the saved working credential automatically.

The admin pair is break-glass only. The CLI never writes it into its config; supply it per invocation when you need it, via the environment or flags:

export ARCTIC_CLIENT_ID=cli_YOUR_ADMIN_ID
export ARCTIC_CLIENT_SECRET=sec_YOUR_ADMIN_SECRET
arctic credentials create --scopes read

or:

arctic credentials create --scopes read \
  --client-id cli_YOUR_ADMIN_ID --client-secret sec_YOUR_ADMIN_SECRET

Move break-glass.<cluster-id>.json into your secret manager and delete the file once it is secured. Pass --credentials-file at bootstrap to pick a different destination (the literal value none prints the pair without writing a file).

List credentials

List the API credentials known to the cluster:

arctic credentials list

curl -k -X GET https://AGENT_IP:8080/v1/credentials \
  -H "Authorization: Bearer $TOKEN"

Each entry shows the client_id, its scopes, the license_id, and timestamps. Secrets are never displayed.

Create a credential

Creating a credential requires an explicit scope set. The --scopes flag accepts the read, write, and admin aliases as well as individual scope names:

# Read-only automation
arctic credentials create --scopes read

# Scoped to read-only peer and service access
arctic credentials create --scopes peers.read,services.read

# Full admin: granted only when asked for by name
arctic credentials create --scopes admin

curl -k -X POST https://AGENT_IP:8080/v1/credentials \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"scopes": ["peers.read", "services.read"]}'

Use scoped credentials for automation that only needs part of the API. Grant least privilege; reserve admin for break-glass use.

Revoke a credential

Revoke (tombstone) a credential by its client ID:

arctic credentials revoke cli_THE_CLIENT_ID

curl -k -X DELETE https://AGENT_IP:8080/v1/credentials/cli_THE_CLIENT_ID \
  -H "Authorization: Bearer $TOKEN"

Revocation propagates across the cluster as a signed tombstone, so the credential stops working on every peer.

Revocation also cuts off tokens the credential already issued. Bearer verification checks the issuing credential's status on every request, behind a cache of REVOCATION_CACHE_TTL seconds (default 30; 0 disables the cache), so outstanding tokens start returning 401 within about 30 seconds of the revocation reaching a peer. See the agent reference for the variable.

Rotate the cluster credential

Rotation replaces the shared secret with a new one. Any peer can run it, but it is an admin operation: after a default bootstrap, supply the break-glass pair as shown above.

arctic credentials rotate

arctic credentials rotate --update-config

curl -k -X POST https://AGENT_IP:8080/v1/cluster/credentials/rotate \
  -H "Authorization: Bearer $TOKEN"

When you rotate:

1. A new secret is minted and gossiped to every cluster peer.
2. The previous secret's hash stays accepted for a 24-hour propagation window.
3. After the window, the previous secret expires and stops working everywhere.

The propagation window exists so the new secret has time to reach all peers and so operators have time to pick it up. It is not a grace period you should rely on for day-to-day access.

Recommended rotation procedure

1. Rotate from a peer you control:

   arctic credentials rotate

   Add --update-config if your CLI config stores the credential being rotated (see the note above).

2. Capture the new secret from the command output (or the client_secret in the API response) and store it in your secret manager. It is shown only once. If this is the break-glass credential, the pair in break-glass.<cluster-id>.json (or wherever you moved it) is now stale; replace it.

3. Distribute the new secret to every operator and automation that authenticates against the cluster: CI/CD pipelines, scripts, teammates' CLI configs, and any external tooling.

4. Update each consumer's config before the window closes. Operators who did not run the rotation can write the new secret into their own CLI config and verify access:

   arctic credentials list

5. Confirm convergence within the 24-hour window. Every consumer should be using the new secret well before the previous one expires.

Account lockout

Five consecutive failed attempts with a client_id lock it out of the token endpoint for 15 minutes; locked requests return HTTP 429 (slow_down) without touching credential verification. Only real credential failures count, so a malformed grant type or scope request cannot lock out a misconfigured client. The recovery-token header has the same protection, keyed by hashed source IP.

Locks are held in memory on each agent, so they never span peers and an agent restart clears them. Nothing is locked permanently: wait out the 15 minutes, authenticate against a different peer, or use the recovery token, which bypasses the token endpoint entirely (see Access recovery). Scripts that retry hard on bad credentials will see 429 instead of endless 401s; see troubleshooting for the symptom.

Troubleshooting

Locked out after rotation

If the 24-hour window has closed and you are still using the previous secret, you can no longer authenticate with it. Obtain the current secret from whoever rotated it (or your secret manager) and update your CLI config. If the secret is lost entirely, rotate again from a peer that still has a working credential, or fall back to access recovery.

Operator who rotated is locked out

This happens when the rotated credential was the one stored in the local config and rotate was run without --update-config. Re-run with --update-config, or manually write the new client_secret into the CLI config, while a working token is still available.

Permission denied

Rotating, creating, and revoking credentials require the credentials.write scope, which the scoped working credential saved by bootstrap deliberately does not hold. Supply the break-glass pair via ARCTIC_CLIENT_ID/ARCTIC_CLIENT_SECRET or --client-id/--client-secret. Verify a credential's scopes with arctic credentials list.

HTTP 429 slow_down

The credential (or your source IP, for recovery-token attempts) is locked out after repeated authentication failures. Fix the credential your tooling is sending and wait out the 15-minute lock, or restart the agent to clear it.