Cribl License Error “Invalid license token: failed to decode”

Symptom

After upgrading a Cribl Leader from an earlier 4.15.x release to a later 4.16.x–4.18.x release, the following symptoms may occur:

• The licensing page or logs show the error message: Invalid license token: failed to decode.
• The deployment suddenly appears to be running with only a single Worker Group; additional Worker Groups seem to disappear from the UI.
• Workers that previously belonged to other Worker Groups report an invalid worker group error and disconnect from the Leader.
• Log entries similar to the following appear in the Leader logs:
  { "channel": "license", "level": "warn", "message": "failed to load license", "error": { "message": "Invalid license token: failed to decode" } }

These symptoms typically appear immediately after applying or reloading a license and restarting the Leader following an upgrade.

Environment

• Component: Leader (distributed deployment with Worker Groups and Workers)
• Deployment type: Self-managed
• Version: Observed after upgrading from 4.15.x to 4.18.1; behavior introduced by a JWT library change in 4.16.0 that makes license parsing stricter.
• License type: Standard or Enterprise license applied via licenses.yml or the config.license parameter.

Resolution

1. Confirm the error in the logs.
   1. On the Leader host, open the Cribl log file.
   2. Search for messages containing both failed to load license and Invalid license token: failed to decode to confirm that license decoding is failing.
2. Locate how the license is being supplied.
   1. Check whether the license is configured in config.license (for example, via environment variable, Helm values, or other deployment configuration).
   2. Check whether a licenses.yml file is present under the Cribl configuration directory on the Leader.
3. Inspect the license string for extra characters or line breaks.
   1. Obtain the raw license value that is being used (from config.license or from licenses.yml).
   2. Verify that the license string:
      1. Starts with eyJ (typical for a JWT).
      2. Is a single, continuous line with no embedded line breaks or YAML folding.
      3. Does not include any trailing or leading characters that are not part of the original license (for example, an extra % character at the end like ...1eWViW8A%).
4. Correct the license string.
   1. If you find any extra characters (such as %) or whitespace at the end of the license value, remove them so that only the exact JWT license string remains.
   2. If the license is in licenses.yml, ensure that:
      1. The license appears under the licenses: array as a single-line string.
      2. There is no YAML line folding and no trailing characters after the JWT.
   3. If the license is supplied via config.license, update the configuration to use the corrected, single-line JWT value only.
5. Restart the Leader service.
   1. Restart the Leader process or pod so that it reloads the updated license configuration.
   2. Wait for the Leader UI to become available again.
6. Verify that licensing and Worker Groups are restored.
   1. In the Leader UI, confirm that the license now shows the expected edition (for example, Enterprise or Standard) with the correct limits.
   2. Confirm that all expected Worker Groups are visible again.
   3. On one or more Workers, verify that they successfully connect to their configured Worker Groups and no longer report invalid worker group errors.

If the error persists after validating that the license string is intact and correctly formatted, collect a fresh diagnostic bundle from the Leader and contact Cribl Support for further investigation.

Cause

This can be caused by:

• A malformed license string in licenses.yml or config.license, such as an extra % character appended to the end of the JWT value.
• Line breaks, YAML folding, or whitespace that alter the license string so that it no longer matches the original JWT issued by Cribl.
• The stricter JWT parsing introduced when the underlying JWT library was replaced in Cribl Stream 4.16.0, which now rejects malformed license strings that may have been silently accepted in earlier versions (for example, 4.15.x).

When the license decode fails, the Leader drops the invalid license, falls back to the embedded Free license, and enforces the Free license limits (including a limit of one Worker Group). This fallback behavior explains why additional Worker Groups disappear and Workers in those groups report invalid worker group after the error appears.

Additional Information

• When no valid Standard or Enterprise license is available, Cribl falls back to an embedded Free license that enforces stricter limits, including a maximum of one Worker Group. This is expected licensing behavior once the license fails to decode.
• For general background or licensing limits and license distribution from Leaders to Workers, see Manage Licenses for Cribl On-Prem Deployments | Cribl Docs.