Troubleshooting

Start here for any mirror or analysis that doesn't behave:

kubectl describe clustermirror <name>
kubectl describe upgradeanalysis <name>
kubectl -n mirops logs deploy/mirops-controller-manager

Debugging order

Check resource conditions and report state first

kubectl describe shows the operator's conditions and destination errors. Then verify the report service and destination credentials.

No ClusterMirror, or the Web UI says there's none

The chart creates no ClusterMirror — nothing is mirrored until you create one. See Create a ClusterMirror.

The mirror shows a syncError

The last rebuild failed, or it succeeded but its report couldn't be written. The message says which. For a report failure, status.reportState is failed and status.reportError has the detail — check the destination as in remote-write failure. The mirror retries on its next rebuild; bump mirops.io/refresh to retry now.

An analysis stays on Analyzing

Upgrade analysis is off in this install, so the operator isn't processing UpgradeAnalysis resources. Turn it on:

helm upgrade mirops oci://ghcr.io/miropshq/charts/mirops --namespace mirops \
  --reuse-values --set upgrade.enabled=true

The mirror report's upgrade.enabled says which mode the operator runs in; the Web UI reads it and shows "Upgrade analysis is turned off".

Decision is ERROR

The analysis didn't run — a configuration error. The most common cause is targetVersion not being higher than the current cluster version (downgrades aren't supported). status.reason explains it.

Report shows a remote-write failure

status.reportState: failed with a message in status.reportError. Check, in order:

1. Credentials/identity — IRSA role (S3) or Workload Identity client-id (Azure) on the operator's ServiceAccount, or a static-credentials Secret.
2. Permissions — the identity can write to the bucket/container (e.g. Storage Blob Data Contributor).
3. Names — bucket/region/key or accountName/containerName/blobName are correct.
4. Network — the operator can reach the endpoint.

Then re-run (bump the mirops.io/refresh annotation or edit the spec) — on a ClusterMirror, the next rebuild retries on its own.

"AI unavailable — showing base analysis only"

The verdict is still valid (base analysis). Check status.aiError:

• Invalid/empty API key or no credit — fix the Secret.
• Truncated response (parse error) — maxTokens is too small; raise spec.ai.maxTokens (e.g. 8192), especially with remediation.

Then Re-analyze — the AI re-runs (raising maxTokens invalidates any cached result).

The Web UI is slow or shows the old plugin

The plugin is served from an image in the cluster. Rebuild/push the plugin image and kubectl -n headlamp rollout restart deploy/headlamp, then hard-refresh the browser (the browser caches the plugin bundle).

The report reads back as a 404 briefly

Right after an analysis, the export may still be in flight — the report endpoint returns a clean 404 and the UI keeps polling. This resolves on its own once the report is written; only a real storage failure surfaces as an error.

The same happens to a file-destination mirror report right after the operator restarts or changes leader: the report lives in the pod, so it's missing until the first rebuild. A pvc, s3 or blob destination keeps it across restarts.

mirops scan exits 2

The CLI couldn't evaluate, and says why on stderr. Exit 2 is never a pass. The usual causes:

Message says	Fix
namespace "…" isn't in mirror / isn't in mirror … or in this change	A typo in -n / MIROPS_NAMESPACE, or the mirror's scope leaves that namespace out.
the mirror report is … old (--max-mirror-age …)	The operator stopped rebuilding the mirror, or the pipeline reads a stale copy. Check the ClusterMirror's lastSync and syncError.
is a Helm chart source / Kustomize source / template expressions / Terraform source	-f got a source, not what will be applied. Render it first: helm template …, kustomize build …, terraform show -json plan.out.
get credentials / no EC2 IMDS role found / AccessDenied	The runner can't read the report — see Credentials.
MIROPS_UPGRADE=true needs MIROPS_UPGRADE_SOURCE / upgrade analysis is off	Set the analysis report URL, or turn upgrade analysis on (Helm upgrade.enabled=true).
nothing to scan	Give -f, -n or --upgrade.

A file-destination report lives inside the operator's pod: a runner outside the cluster can't read it — use s3, blob, or the reports service over HTTP.

mirops scan -f blocks a deploy (exit 1)

That's the gate working: the change would break something running. Each ✗ BLOCK line names the object and why — a volume that is Pending or missing, a Service that doesn't exist, something still in use that the change deletes. Fix the change, or fix the dependency first (bind the volume, deploy the Service). To try the stage without failing the pipeline, run it without --enforce: it reports the same findings and exits 0.

Could not assume role with OIDC in GitHub Actions

The role's trust policy doesn't match the token GitHub sends. Repositories using GitHub's immutable OIDC subject send repo:<org>@<org-id>/<repo>@<repo-id>:ref:refs/heads/<branch> instead of repo:<org>/<repo>:…. Check yours with gh api repos/<org>/<repo>/actions/oidc/customization/sub and allow that prefix in the role's token.actions.githubusercontent.com:sub condition. When this step fails, any later mirops scan fails too, with get credentials errors.

Still stuck?

Open an issue on the relevant repository (operator, CLI, plugin) with the CR status, the operator logs, and what you expected.