Symptom
After merging a .gitignore change from a dev branch to the prod branch and performing a POST /version/sync on the Prod Leader, a persistent "Pending Commit" appears in the Prod Leader's global Git dropdown. The Prod Leader is configured in GitOps mode (read-only) so the pending commit does not resolve on subsequent syncs and cannot be committed via the UI.
Additionally, the Dev Leader does not show the same pending commit despite having the same .gitignore change applied.
Environment
- Cribl Stream - Distributed deployment with GitOps mode enabled
- Dev Leader managed separately with a remote Git repo
- Affected versions: 4.17.1, 4.18.2 (and likely others)
Resolution
- Restart the Dev Leader service (a full Cribl service restart, not just an API reload) to trigger the self-healing check.
- After restart, Cribl will rewrite
.gitignorewith the complete correct entries for that version. This diff will appear as a pending commit in the Dev Leader UI.
- After restart, Cribl will rewrite
- In the Dev Leader UI, commit the
.gitignorediff and push it to the dev branch in the remote Git repo. - Open a PR merging the dev branch into the prod branch in your remote Git repo. Merge it.
- Sync the Prod Leader via
POST /version/sync.- After the sync completes, refresh the Prod Leader UI. The "Pending Commit" should be gone and the
.gitignoreon prod now matches what Cribl expects, so the self-healing check produces no diff.
- After the sync completes, refresh the Prod Leader UI. The "Pending Commit" should be gone and the
Cause
On every Cribl service restart (not a reload), Cribl compares the .gitignore file against its internal hardcoded ruleset for that version under the ### CRIBL SECTION -- DO NOT REMOVE ### block. If any entries are missing, Cribl rewrites the file to include all expected rules.
In a GitOps setup:
- The Prod Leader restarts on every
POST /version/syncso the self-healing check runs, Cribl rewrites.gitignorewith the missing rules, and since the Prod Leader is read-only, this change cannot be committed via the UI, resulting in a persistent "Pending Commit". - The Dev Leader, in this specific case, was reloaded via API (not fully restarted), so the self-healing check never ran and no pending commit appeared on the Dev Leader.
- The root cause is that the
.gitignorepushed from dev to main (or prod) was missing entries from the### CRIBL SECTION -- DO NOT REMOVE ###block that Cribl expects for that version. Every sync triggers a restart, Cribl detects the incomplete rules, rewrites.gitignore, and the cycle repeats.
The .gitignore sample published here, .gitignore File, is out of sync with the entries Cribl actually enforces at runtime. Users who use this sample as a reference when setting up GitOps will have an incomplete .gitignore that Cribl continuously self-heals on every restart, producing a persistent "Pending Commit" on a read-only GitOps Prod Leader that can never be resolved without fixing the .gitignore itself.
The best approach is to let Cribl generate the authoritative .gitignore for your version by performing a full service restart on the Dev Leader and committing the resulting file.
Additional Information
- The
### CRIBL SECTION -- DO NOT REMOVE ###and### CUSTOM SECTION -- DO NOT REMOVE ###header lines in.gitignoreare required as without them, Cribl cannot identify its managed block and will rewrite the entire file on restart. See CRIBL Section. - Only add custom exclusions under
### CUSTOM SECTION -- DO NOT REMOVE ###and never modify entries under the CRIBL SECTION.
