Peer management

Add a peer

This section shows you how to add a new Arctic agent to an existing cluster.

Before you start

Ensure you have:

• An existing Arctic cluster with at least one bootstrapped agent
• A new agent that is running and accessible
• The new agent bootstrapped with the same license as your cluster
• Network connectivity between the existing cluster and the new agent

Steps

1. Bootstrap the new agent

If the new agent has not been bootstrapped yet, initialize it with your license:

arctic bootstrap \
  --url http://NEW_AGENT_IP:8080 \
  --license-file license.json \
  --name "new-agent"

curl -X POST http://NEW_AGENT_IP:8080/v1/bootstrap \
  -H "Content-Type: application/json" \
  -d '{"license": LICENSE_JSON, "name": "new-agent"}'

2. Verify the new agent is healthy

Confirm the agent is running and ready:

curl http://NEW_AGENT_IP:8080/livez

Expected response: {"status":"ok",...}

3. Add the peer

From your CLI (configured to connect to an existing cluster agent), add the new peer by its address:

arctic peers add NEW_AGENT_IP:8080

arctic peers add NEW_AGENT_IP:8080 --name "new-agent"

# Get an access token first
TOKEN=$(curl -s -X POST http://EXISTING_AGENT_IP:8080/v1/oauth/token \
  -d "grant_type=client_credentials&client_id=cli_YOUR_ID&client_secret=sec_YOUR_SECRET" \
  | jq -r '.access_token')

# Add the peer. Prefer the endpoints list over the deprecated
# single "address" field; the agent tries each entry in order.
curl -X POST http://EXISTING_AGENT_IP:8080/v1/peers \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"endpoints": ["NEW_AGENT_IP:8080"], "name": "new-agent"}'

4. Verify the peer was added

List all peers to confirm the new agent appears:

arctic peers list

arctic peers list --remote

curl -X GET http://EXISTING_AGENT_IP:8080/v1/peers \
  -H "Authorization: Bearer $TOKEN"

The new peer should appear with is_local: false.

What happens during peer addition

When you add a peer, Arctic performs these steps:

1. Identity exchange: The existing agent contacts the new agent and they exchange public keys.
2. Signature verification: Both agents run an Ed25519 challenge-response and validate license compatibility.
3. Database update: Both agents store each other's peer information.
4. Gossip initialization: The agents begin exchanging heartbeats and synchronize state.

The handshake response includes a transitive-trust voucher. The process typically completes within seconds.

Troubleshooting

Handshake failed: connection refused

The new agent is not accessible. Check:

• Network connectivity: curl http://NEW_AGENT_IP:8080/livez
• Firewall rules allow port 8080
• The agent service is running: systemctl status arctic

Handshake failed: license mismatch

The agents have different licenses. Verify:

• Both agents were bootstrapped with the same license file
• Check license status on both agents: arctic license status

Peer already exists

The peer is already in the cluster. This can happen if:

• You previously added this peer
• Another cluster member already discovered this peer via gossip

Use arctic peers list to see existing peers.

Node limit exceeded

Your license has a maximum node limit. Check your license:

arctic license status

Contact your administrator to upgrade your license if needed.

Update a peer

This section shows you how to update a peer's metadata, such as its human-readable name or QoS toggle.

Before you start

Ensure you have:

• The peer ID of the peer you want to update
• Appropriate permissions (peers.write scope)

Steps

1. Find the peer ID

List peers to find the ID:

arctic peers list

curl -X GET http://AGENT_IP:8080/v1/peers \
  -H "Authorization: Bearer $TOKEN"

Peer IDs are prefixed with peer_.

2. Update the peer

Update the peer's name:

arctic peers update PEER_ID --name "prod-svr-1"

arctic peers update PEER_ID --qos=false

curl -X PUT http://AGENT_IP:8080/v1/peers/PEER_ID \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "prod-svr-1", "qos_enabled": false}'

Both name and qos_enabled are optional; omit a field to leave it unchanged.

3. Verify the update

Confirm the change:

arctic peers get PEER_ID

curl -X GET http://AGENT_IP:8080/v1/peers/PEER_ID \
  -H "Authorization: Bearer $TOKEN"

What can be updated

Other peer attributes (public key, peer ID, etc.) are immutable and cannot be changed.

Propagation

A name change made through the CLI is also synced to your local CLI config. Other cluster members maintain their own view of peer metadata. If you want consistent naming across the cluster, update the peer on each agent.

Troubleshooting

Peer not found

If the peer ID is not found:

1. Verify the peer ID: arctic peers list
2. Check you are connected to the correct agent

Permission denied

If you receive an authorization error:

1. Verify your credentials have the peers.write scope
2. Check that your token has not expired

Remove a peer

This section shows you how to gracefully remove an agent from your Arctic cluster.

Before you start

Understand the difference between removing a remote peer and removing the local peer:

• Remote peer: Removes another agent from your view of the cluster.
• Local peer (remove-self): Gracefully removes the current agent from the entire cluster.

The local peer cannot be removed with peers delete. Use peers remove-self for the local agent.

Remove a remote peer

Use this when you want to remove another agent from your cluster.

1. Find the peer ID

List all peers to find the ID of the peer you want to remove:

arctic peers list

curl -X GET http://AGENT_IP:8080/v1/peers \
  -H "Authorization: Bearer $TOKEN"

2. Delete the peer

Remove the peer by ID:

arctic peers delete PEER_ID

# You will be prompted to confirm. Use --yes to skip confirmation:
arctic peers delete PEER_ID --yes

curl -X DELETE http://AGENT_IP:8080/v1/peers/PEER_ID \
  -H "Authorization: Bearer $TOKEN"

3. Verify removal

Confirm the peer no longer appears in the list:

arctic peers list

Remove the local peer (self-removal)

Use this when you want to gracefully remove the current agent from the cluster. This is useful when decommissioning a node.

1. Remove self

arctic peers remove-self

# You will be prompted to confirm. Use --yes to skip confirmation:
arctic peers remove-self --yes

arctic remove-self --yes

curl -X DELETE http://AGENT_IP:8080/v1/peers/self \
  -H "Authorization: Bearer $TOKEN"

2. What happens

When you remove self:

1. The local peer is marked as deactivated.
2. A signed deactivation announcement is broadcast to all cluster peers.
3. Other peers receive the announcement and remove this peer from their databases.
4. The agent shuts down gracefully.

Cleanup after removal

After removing a peer, you may want to:

Delete associated services

Services involving the removed peer will no longer function. List services and delete them by ID:

# List services, then delete each by ID
arctic services list

arctic services delete SERVICE_ID --yes

Update CLI configuration

If you have the removed peer in your CLI config, remove it:

arctic config delete-peer PEER_ID_OR_NAME

Troubleshooting

Cannot delete local peer

If you try to delete the local peer using peers delete:

Error: cannot delete local peer (use 'arctic peers remove-self' instead)

Use arctic peers remove-self for the local agent.

Peer not found

If the peer ID is not found:

1. Verify the peer ID: arctic peers list
2. The peer may have already been removed by another cluster member

Services still active after removal

Services are not automatically deleted when a peer is removed. You must manually delete services that reference the removed peer.