Runbooks/Secret Rotation

Secret Rotation

Operator runbook for rotating every long-lived credential or signed artefact in an IdentityMesh deployment. Pair with secrets-and-dpapi.md for the storage model.

The full inventory:

SecretWhere it livesRotation cadence (suggested)
License key%CommonApplicationData%\IdentityMesh\license.keyOn expiry / tier change
Relay agent API keyRelay:ApiKey in appsettings.json on each relay host90 days
SQL connection-string credentialsIdentityMesh:SqlConnectionString (Admin API + Sync Engine)90 days, or per AD policy
Connector credentials (LDAP, SQL, SaaS auth tokens)IM_Secrets table (DPAPI-encrypted)Per source-system policy
Service account passwords (SVC_ACCOUNT, API_SVC_ACCOUNT)Windows account / installed service identityPer AD policy
DPAPI master key (implicit)Local Windows profile / machineDon’t rotate — re-provision

Each rotation procedure below is idempotent and can be re-run if interrupted.


License key

Rotate when the key is reissued by sales (tier change, expiry extension, customer rename) or when you suspect the file leaked.

# 1. Place the new license file alongside the old one.
$dataDir = "$env:ProgramData\IdentityMesh"
Copy-Item -Path "C:\Path\To\new-license.key" `
          -Destination "$dataDir\license.key.new"

# 2. Atomic swap — replace the live file in one move so a
#    concurrent license check never sees a torn read.
Move-Item -Path "$dataDir\license.key.new" `
          -Destination "$dataDir\license.key" `
          -Force

# 3. Hot-reload via the Admin API. No service restart required —
#    the license file is re-read on demand.
Invoke-RestMethod -Method Post `
                  -Uri https://<admin-host>/api/license/reload `
                  -UseDefaultCredentials

If you don’t have admin-API access, restart the IdentityMesh services to force the reload:

sc stop  IdentityMeshAdmin
sc stop  IdentityMeshEngine
sc start IdentityMeshAdmin
sc start IdentityMeshEngine

Verify by hitting GET /api/license — Tier, CustomerName, and ExpiresUtc should reflect the new file.

The license file is not a secret in the cryptographic sense — it’s an RSA-signed payload — but treat it as customer data. Never email it; never commit it.


Relay agent API key

Each relay agent authenticates to the Admin API with its own API key, issued when the agent is registered and stored only as a hash on the server. Rotating it is a per-agent operation; no server-side configuration changes.

Phase 1 — issue the new key

  1. In the console open Relay Agents, find the agent and press Reset key (needs roles.write).
  2. Copy the new key from the banner. It is shown once; the agent ID does not change.

The old key stops working the moment you press Reset, so the agent is disconnected until Phase 2 is done. Connectors assigned to it hold their runs in the meantime.

Phase 2 — roll the relay (per host)

$cfg = "$env:ProgramFiles\IdentityMesh Relay Agent\appsettings.json"

# 1. Stop the relay so the file write is uncontended.
sc stop IdentityMeshRelayAgent

# 2. Update the ApiKey field. Use a JSON-aware editor; the
#    snippet below works for the simple flat schema.
$json = Get-Content $cfg -Raw | ConvertFrom-Json
$json.Relay.ApiKey = '<new-key>'
$json | ConvertTo-Json -Depth 10 | Set-Content $cfg -Encoding UTF8

# 3. Start the relay. It reconnects to the hub with the new
#    key on first attempt; check the relay log for "Connected".
sc start IdentityMeshRelayAgent

Phase 3 — confirm

On the Relay Agents page the agent shows Connected with a fresh heartbeat within a minute. If it stays disconnected, the key in appsettings.json was pasted wrong: reset it again and repeat Phase 2.

If a key is lost and the agent was never reset, the same Reset key button recovers it; there is no need to delete and re-register the agent.


SQL connection-string credentials

Two cases — pick the matching procedure.

Rotate the SQL service account’s domain password per your AD policy. No IdentityMesh-side action is needed; the services pick up the new credentials on their next database connection.

If you change the service account (not just the password), re-run the installer’s Repair flow or update the service login manually:

sc config IdentityMeshAdmin obj= "DOMAIN\new-svc-account" password= "<password>"
sc config IdentityMeshEngine obj= "DOMAIN\new-svc-account" password= "<password>"
sc stop IdentityMeshAdmin; sc start IdentityMeshAdmin
sc stop IdentityMeshEngine; sc start IdentityMeshEngine

Case B — SQL auth (User ID=…;Password=…)

# 1. Update the connection string in BOTH appsettings.json files
#    (Admin API and Sync Engine).
foreach ($cfg in @(
    "$env:ProgramFiles\IdentityMesh\api\appsettings.json",
    "$env:ProgramFiles\IdentityMesh\engine\appsettings.json"
)) {
    $json = Get-Content $cfg -Raw | ConvertFrom-Json
    $json.IdentityMesh.SqlConnectionString = '<new-cs>'
    $json | ConvertTo-Json -Depth 10 | Set-Content $cfg -Encoding UTF8
}

# 2. Cycle services together so a half-rotated state never runs.
sc stop IdentityMeshEngine
sc stop IdentityMeshAdmin
sc start IdentityMeshAdmin
sc start IdentityMeshEngine

Verify with /health/ready (returns 200 only when the DB check passes) and GET /api/dashboard.


Connector credentials in IM_Secrets

These are the per-connector passwords / API tokens that the secret store holds DPAPI-encrypted.

# Push a new value through the SecretsCli tool. Run on the
# Sync Engine host so it's encrypted under the right local
# machine key.
secretscli set "secret://ad/svc-imadmin/password"  "<new-password>"
secretscli set "secret://sql/hr-source/password"   "<new-password>"

# Verify
secretscli list

The change is picked up on the next sync run for the affected connector; the running engine reads the secret store each time a connector is activated, so no restart needed.

To find every secret reference an operator might need to rotate, run secretscli list on the Sync Engine host. Each connector’s configuration in the Admin UI shows the secret://… reference it uses, so you can match references to connectors.

Host migration is NOT a rotation. DPAPI blobs only decrypt on the host that wrote them. If you’re moving to a new host, see the DR section of backup-and-restore.md — every secret must be re-pushed via secretscli set from the new host. The blobs in SQL travel; the keys do not.


What you can’t rotate


Cadence summary

FrequencyItems
Per AD policy (e.g. 90 days)Service-account passwords, SQL auth (if used)
90 daysRelay API key
Per source-system policyConnector credentials in IM_Secrets
As neededLicense key (issued by sales)

Document the dates of the last rotation in your CMDB or change-log of choice — IdentityMesh tracks IM_Secrets.UpdatedOn so you can see when each connector secret was last touched, but everything else is operator-owned.