Maintenance Operations

Operator runbook for the manual and periodic maintenance surfaces IdentityMesh exposes: uniqueness re-resolve, duplicate-orphan cleanup (including the two mesh-level modes), audit-backlog purge, export staging / gating, the Management Space export sweep, and the interactive discovery calls behind the connector editor page.

These are the levers you reach for after a bulk import, a messy migration, or an out-of-band change — the things you run deliberately, not the sync that runs itself. Each one is either dry-run-first or read-only-first so you can look before you leap.

Uniqueness re-resolve

IsUnique attributes (samAccountName, mail, userPrincipalName, employeeId, …) are allocated once when a mesh object is first resolved. If duplicates were imported before the uniqueness rule existed — or two sources contributed the same value in the same import window — those duplicates survive: the assign-once path doesn’t revisit an attribute it already set. The symptom is IsUnique attributes that still show collisions after a clean full import.

Re-resolve re-runs the uniqueness allocator across the existing mesh and clears those surviving duplicates.

Periodic

The Sync Engine runs the re-resolve on a background cadence of its own, so a steady deployment self-heals without operator action. The manual endpoint is for when you need it now (post-migration) or want to preview the impact first.

On demand

# Dry-run (default): forecast only, writes nothing.
curl -X POST https://identitymesh.example.com/api/admin/reresolve-uniqueness \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "dryRun": true }'

# Apply: re-allocate and clear duplicates.
curl -X POST https://identitymesh.example.com/api/admin/reresolve-uniqueness \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "dryRun": false }'

POST with no body defaults to dryRun: true — the endpoint never mutates unless you explicitly pass { "dryRun": false }. Run the dry-run, read the forecast (how many values it would re-allocate), then re-run with dryRun false to apply.

The endpoint is gated on rules.write (re-allocating a unique value is a rules-side change). The read-only reconciliation report that surfaces duplicate/orphan symptoms is gated on audit.read (see reconciliation.md) — an auditor can see the drift; a rules operator applies the fix.

Duplicate-orphan cleanup

A duplicate orphan is a management-space row that is an orphaned duplicate of an object already linked to a mesh object — the same source identity represented twice, where one copy carries the live join and the other is dead weight. These accumulate from re-imports after a source-side rename, or from disabled accounts that were re-created upstream. A row an administrator has isolated is never a candidate, and neither is a row that was disconnected while its export hold is still open (see isolate-disconnect.md).

Report first

# CSV of every eligible duplicate for a connector (no row cap).
curl -X GET "https://identitymesh.example.com/api/admin/cleanup-duplicate-orphans/report?connectorId=$CONNECTOR_ID" \
     -H "Authorization: Bearer $TOKEN" \
     -o duplicate-orphans.csv

The report is a dry-run that returns the full eligible set as CSV (SourceLocator, SourceName, ExternalId, IsDisabledAtSource, DuplicateOfMeshObjectId). Review it before applying anything.

Apply

# Dry-run apply (default): count what would be removed.
curl -X POST https://identitymesh.example.com/api/admin/cleanup-duplicate-orphans \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "connectorId": "'"$CONNECTOR_ID"'", "dryRun": true, "requireDisabled": true }'

# Real apply, restricted to duplicates that are disabled at the source.
curl -X POST https://identitymesh.example.com/api/admin/cleanup-duplicate-orphans \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "connectorId": "'"$CONNECTOR_ID"'", "dryRun": false, "requireDisabled": true }'

requireDisabled: true is the safe default posture — it restricts the delete to duplicates whose source object is disabled at the source, so you never remove a duplicate that might still be live upstream. Set it to false only when you have reviewed the report and are certain the duplicates are dead regardless of source state.

Both the apply and the report are gated on rules.write.

Mesh mode

New in this release, the same page has a Mesh mode that looks at the other side of the join. A mesh orphan is a mesh object with no connector link at all that duplicates a linked mesh object of the same type — typically left behind by a Composer rule or an import that ran before a join rule existed, so the real identity got a second, unlinked mesh row.

The match is deliberately narrow: the orphan must agree with a linked object of the same type on the anchor or on an IsUnique attribute. Each deletion is written to the audit log naming the survivor and the rule that matched.

Anything weaker is reported for review and never deleted:

• a match on display name only;
• an ambiguous match (more than one candidate survivor);
• an orphan that has exported attribute snapshots (it has been written to a target, so it is not dead weight).

# Report: what the Mesh mode would delete, and what it leaves for review.
# CSV; objectType is optional, omit it to cover every mesh object type.
curl -X GET "https://identitymesh.example.com/api/admin/cleanup-mesh-orphans/report?objectType=person" \
     -H "Authorization: Bearer $TOKEN" \
     -o mesh-orphans.csv

# Dry-run apply (default): count what would be removed, write nothing.
curl -X POST https://identitymesh.example.com/api/admin/cleanup-mesh-orphans \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "objectType": "person", "dryRun": true }'

# Real apply: delete the matched orphans (review rows are left alone).
curl -X POST https://identitymesh.example.com/api/admin/cleanup-mesh-orphans \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "objectType": "person", "dryRun": false }'

POST with no body defaults to dryRun: true, the same posture as the other cleanup endpoints. Both endpoints are gated on rules.write and rate limited. Run the apply while the engine is quiet (pause it, or wait for a gap between scheduled runs): an import landing mid-apply can link an orphan the report already classified. The admin page shows the applied rows, split into deleted and left-for-review with the reason for each.

Unlinked mode

The Mesh mode above deletes an orphan only when it duplicates a linked object. A third option on the same page, Mesh — every unlinked object of one type, is for the other case: mesh objects that have no connector link and no linked twin either, typically the leftovers of an import that ran before the right join rule existed, or of a connector that was removed. A mesh object with an open export hold (one of its connector objects was disconnected and the hold has not been released) is left alone by this mode.

It is deliberately blunter, so it has guard rails the other modes do not need:

• The object type is required. A blank type is refused (API 400), so a request can never widen to every type in the mesh.
• Three holds are never deleted, and are reported with the reason: an object that has exported attribute snapshots (it has been written to a target); an object Composer created; an object self-service created. On an estate with many exported placeholder people, expect most rows to come back as has exported snapshots holds rather than eligible rows.
• Every deletion is audited with the reason UnlinkedMeshObjectCleanup; there is no survivor to name.

# Report: every unlinked object of the type, eligible or held, with the reason.
curl -X GET "https://identitymesh.example.com/api/admin/cleanup-unlinked-mesh-objects/report?objectType=person" \
     -H "Authorization: Bearer $TOKEN" \
     -o unlinked-mesh-objects.csv

# Dry-run apply (default): count what would be removed, write nothing.
curl -X POST https://identitymesh.example.com/api/admin/cleanup-unlinked-mesh-objects \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "objectType": "person", "dryRun": true }'

# Real apply.
curl -X POST https://identitymesh.example.com/api/admin/cleanup-unlinked-mesh-objects \
     -H "Authorization: Bearer $TOKEN" \
     -H "Content-Type: application/json" \
     -d '{ "objectType": "person", "dryRun": false }'

Same gating (rules.write, rate limited) and the same rule about running it while the engine is quiet. The admin page’s confirm quotes the count, the type and the word every; read it.

Audit-backlog purge

The hourly retention sweep keeps IM_ObjectAudit and IM_AdminAudit trimmed to the Retention.AuditDays window — but it deletes only 1,000 rows per sweep, which is far too slow to clear a multi-ten-million-row backlog (e.g. after a period with retention effectively disabled, or a first upgrade onto retention).

For that one-time bulk catch-up there is a standalone, run-by-hand script in the installed sql folder:

C:\Program Files\IdentityMesh\sql\Manual_PurgeAuditBacklog.sql

It batch-deletes IM_ObjectAudit down to the retention window (default 30 days) in < 5000-row auto-committed batches, with a tunable inter-batch delay so the engine and Admin API can still query the table. It is not part of the installer’s database step and is never run automatically — run it deliberately, off-hours, after reading the header notes:

• Pause services first (Settings > System > Pause All Services, or export staging below).
• Mind the transaction log under FULL recovery — run frequent log backups during the purge, or switch to SIMPLE for the run and take a FULL backup afterward (this breaks the log-backup chain).
• IM_ObjectAudit is the hash-chained tamper-evident table (PreviousRowHash/RowHash). Purging the oldest rows is exactly what retention already does — after a purge, chain verification is meaningful from the earliest retained row forward. This is expected for any retention purge.

To run it, open a command prompt on the IdentityMesh server as an account that can delete from the IdentityMesh database, change to the installed sql folder, and start the script with sqlcmd (it ships with SQL Server Management Studio and with the SQL Server command-line tools):

cd "C:\Program Files\IdentityMesh\sql"
sqlcmd -S <server\instance> -d IdentityMesh -E -i Manual_PurgeAuditBacklog.sql

Replace <server\instance> with your SQL Server, and IdentityMesh with your database name if you changed it during install. Use -U <user> -P <password> instead of -E for SQL authentication. The script prints its progress batch by batch; when it finishes, confirm the oldest remaining row in IM_ObjectAudit is inside the retention window, then resume the services.

After the manual catch-up, the retention sweep maintains steady state on its own. The full retention model — tables, the audit-evidence window vs the operational windows, and verification — is documented in audit-retention.md.

Export staging / gating

By default the engine imports, projects, and exports in one continuous flow. During onboarding, a migration, or a risky change you often want to stage exports: let imports and projections populate the mesh and fill the export queue, review what would be written, then release it to the target systems on your signal. The /api/system family provides the gates. These gates are connector-wide or instance-wide; to stop traffic to or from one connector object while you fix it, use Isolate or Disconnect on that object instead (see isolate-disconnect.md).

Global service pause

curl -X POST https://identitymesh.example.com/api/system/pause  -H "Authorization: Bearer $TOKEN"
curl -X POST https://identitymesh.example.com/api/system/resume -H "Authorization: Bearer $TOKEN"

Sets ServicePaused — the whole engine (imports, projections, exports) stops. Use this for maintenance windows and before the audit-backlog purge. For a named engine instance, pause just that instance with POST /api/instances/{id}/pause (and /resume); see multi-instance for named-instance workload distribution.

Export-only staging pause

# Hold all exports; imports and projections keep running and the queue fills.
curl -X POST https://identitymesh.example.com/api/system/exports/pause  -H "Authorization: Bearer $TOKEN"
# Release: exports flush to targets.
curl -X POST https://identitymesh.example.com/api/system/exports/resume -H "Authorization: Bearer $TOKEN"

Sets ExportsPaused. Unlike the global pause, the mesh keeps building — nothing is written to any target connector until you resume. This is the core staging switch: stage the whole mesh, review, then flush when ready.

Per-connector allow-list

While staging is on, you can let a single connector export live (a canary target) while the rest stay held:

curl -X POST https://identitymesh.example.com/api/system/connectors/$CONNECTOR_ID/exports/allow    -H "Authorization: Bearer $TOKEN"
curl -X POST https://identitymesh.example.com/api/system/connectors/$CONNECTOR_ID/exports/disallow -H "Authorization: Bearer $TOKEN"

An allow-listed connector (ExportsAllowed:{connectorId} = true) flushes its queue even while ExportsPaused is on; every other connector stays paused. Verify the canary, then allow the next, or lift the global staging pause to release everything.

Staging state

curl -X GET https://identitymesh.example.com/api/system/staging-state -H "Authorization: Bearer $TOKEN"

{
  "exportsPaused": true,
  "pendingExports": 5120,
  "allowedConnectorIds": ["9c1c..."],
  "pendingByConnector": [
    { "connectorId": "9c1c...", "pending": 5000 },
    { "connectorId": "00000000-0000-0000-0000-000000000000", "pending": 120 }
  ]
}

pendingByConnector tells you how many items each connector would flush on release. Queue items with no connector (rare) are split out under the empty GUID (00000000-0000-0000-0000-000000000000) so they’re visible rather than hidden inside a total — investigate any non-zero null-connector bucket before releasing.

Drain-worker tuning

A dedicated export drain worker in the Sync Engine empties the queue on its own cadence. Tune its throughput when a large staged batch needs to flush faster (or slower, to spare a fragile target):

curl -X GET https://identitymesh.example.com/api/system/export-drain-settings -H "Authorization: Bearer $TOKEN"

curl -X PUT https://identitymesh.example.com/api/system/export-drain-settings \
     -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
     -d '{ "parallelism": 8, "intervalSeconds": 30 }'

parallelism is clamped to 1–32 (default is min(CPU, 8)), intervalSeconds to 5–3600 (default 60). The engine reads these live each drain cycle — no restart needed.

Import-run timeout

curl -X GET https://identitymesh.example.com/api/system/import-run-timeout -H "Authorization: Bearer $TOKEN"

curl -X PUT https://identitymesh.example.com/api/system/import-run-timeout \
     -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
     -d '{ "minutes": 120 }'

Caps how long a single import run may execute before the engine abandons it. Clamped to 1–1440 minutes (default 60). Raise it for a slow, very large first import; lower it to fail fast on a wedged source.

Staging pattern

The supported end-to-end pattern:

1. POST /api/system/exports/pause — stage. Imports and projections run; the export queue fills; nothing reaches a target.
2. Review — GET /api/system/staging-state for queue depth per connector, plus the mesh-object export-preview in the Admin UI, plus a reconciliation report.
3. Release progressively — allow a canary connector, verify the real writes landed, then either allow the rest or POST /api/system/exports/resume to flush everything.

Permissions

The write endpoints are additionally rate-limited — they are operator-scale actions, not high-frequency calls.

Management Space export sweep

Normally only an import re-evaluates what a connector should export: the engine compares the freshly imported Management Space row against the projection and queues whatever drifted. If the drift came from somewhere else — a projection rule edited, a template applied, a target changed by hand, a delete-mode change — nothing triggers a re-evaluation until the next import happens to touch that object.

The export sweep, new in this release, re-evaluates every object of a connector from the stored Management Space without running an import, and queues only genuine drift. Items it queues carry the reason ManagementSpaceSweep so they are distinguishable in the queue.

On demand

In the Admin UI, the connector’s queue page has a Re-evaluate exports button beside Run Now. The card polls while the sweep runs and reports how many objects were scanned and how many were queued, or what went wrong.

# Start a sweep for one connector (202 Accepted; requires sync.execute).
curl -X POST https://identitymesh.example.com/api/connectors/$CONNECTOR_ID/export-sweep \
     -H "Authorization: Bearer $TOKEN"

# Poll its state and result (requires runs.read).
curl -X GET https://identitymesh.example.com/api/connectors/$CONNECTOR_ID/export-sweep \
     -H "Authorization: Bearer $TOKEN"

Scheduled

IdentityMesh:ManagementSpaceExportSweepIntervalMinutes = 0

Default 0 means off. Set a positive interval to sweep every connector on that cadence. The sweep honours the same pause keys as the export drain (ServicePaused, ExportsPaused), so a staged deployment does not start queueing behind your back.

A sweep queues work; it does not write to any target. Whether the queued items flush is still governed by the staging switches above.

Interactive discovery

The connector editor in the Admin UI (and relay agents, for relay-routed connectors) uses a set of live discovery calls so an operator can point at a source and browse it before committing a connector config. They connect to the live source with the supplied (or secret-referenced) credentials and return what they find; they never write. All are gated on connectors.write.

For a relay-routed connector these run on the relay agent against the remote source rather than on the API host — see relay-agent.md. The credentials the discovery calls use are resolved the same way the connector will resolve them at runtime, so a successful browse is also a credential sanity check.

Related

• reconciliation.md — the read-only drift/orphan/dangling report (and the operations posture dashboard) that tells you whether you need these operations. Its source-drift counts also show rows left behind for an object type a connector no longer imports, which the orphan cleanup removes.
• audit-retention.md — the automatic retention sweep the manual audit-backlog purge complements.
• relay-agent.md — where discovery and export run for relay-routed connectors.