This message originated from Cribl Community Slack.
Click here to view the original link.
I have a dict that contains base64-encoded fields. How can I decode them all without knowing their names?
Solved
Dictionary Base64 Decoding Without Knowing Field Names
Best answer by David Maislin
You can decode all base64-encoded fields in a dictionary in Cribl Stream using an expression that iterates over the object’s keys and decodes each value. Since you don’t know the field names in advance, use Object.keys and a loop in an Eval function.
Here’s an example for use in an Code function which you can update for your use case too.
let decoded = {};
for (let k of Object.keys(dict)) {
try {
decoded[k] = C.Decode.base64(dict[k]);
} catch (e) {
decoded[k] = dict[k]; // fallback if not valid base64
}
}
decoded- Replace dict with your actual field name.
- This will attempt to base64-decode every value; if decoding fails (e.g., not valid base64), it leaves the original value.
- You can assign the result to a new field or overwrite the original.
for (let k of Object.keys(dict)) {
try {
dict[k] = C.Decode.base64(dict[k]);
} catch (e) {
// leave as is
}
}
dict
You can use the Mask function with a wildcard field to decode all patterns that match encoding across all fields. In this example I am using _raw, but it can be
Links for this message:
Screenshot 2026-02-05 at 10.18.48 AM.png
Screenshot 2026-02-05 at 10.18.48 AM.png
* too. Can also be done in the Code function.Links for this message:
Screenshot 2026-02-05 at 10.18.48 AM.png
Screenshot 2026-02-05 at 10.18.48 AM.png
You can decode all base64-encoded fields in a dictionary in Cribl Stream using an expression that iterates over the object’s keys and decodes each value. Since you don’t know the field names in advance, use Object.keys and a loop in an Eval function.
Here’s an example for use in an Code function which you can update for your use case too.
let decoded = {};
for (let k of Object.keys(dict)) {
try {
decoded[k] = C.Decode.base64(dict[k]);
} catch (e) {
decoded[k] = dict[k]; // fallback if not valid base64
}
}
decoded- Replace dict with your actual field name.
- This will attempt to base64-decode every value; if decoding fails (e.g., not valid base64), it leaves the original value.
- You can assign the result to a new field or overwrite the original.
for (let k of Object.keys(dict)) {
try {
dict[k] = C.Decode.base64(dict[k]);
} catch (e) {
// leave as is
}
}
dictJust a few more examples:
Below can decode base64-encoded substrings/values when the field names are unknown:
- Mask with wildcards: apply a regex and a replace expression over many fields using a wildcard. This leverages the expression engine to call
C.Decode.base64(...)inline, as seen in Screenshot 2026-02-05 at 10.14.55AM.png and proof.png. - Code function that iterates
__eand decodes values programmatically, optionally guided by a lookup to decide which keys or patterns to decode.
- Loads a lookup
decode-keys.csvcontaining keys (or regex patterns) that should be decoded and optional encoding (e.g.utf16leorutf8). - Iterates all top-level fields of the event.
- If a key matches the lookup, and the value looks like base64, decodes it.
// Load inline lookup once per expression evaluation.
// CSV columns: key,encoding
// Example rows:
// commandline,utf16le
// payload,utf8
// ^secret_.+,utf8 // regex-style pattern row if using a regex-oriented lookup file
(function() {
const lk = C.LookupIgnoreCase('decode-keys.csv', 'key'); // exact or ignore-case match
// If you need regex keys, use a separate file and C.LookupRegex with a different primaryKey.
const lkRegex = C.LookupRegex('decode-keys-regex.csv', 'key');
// helper to detect plausible base64 (simple heuristic; tighten as needed)
const looksB64 = (s) => typeof s === 'string' &&
s.length >= 8 &&
/^[A-Za-z0-9/+]+=*$/.test(s) &&
(s.length % 4 === 0);
// Get shallow keys. For nested objects, recurse or use a traversal strategy.
Object.keys(__e).forEach((k) => {
if (k.startsWith('__')) return; // skip internal fields
const v = __e[k];
if (v == null) return;
// Only attempt to decode strings (or arrays of strings)
const tryDecode = (val, enc) => {
try {
return C.Decode.base64(val, enc || 'utf8');
} catch (e) {
return val; // leave original if decoding fails
}
};
// Decide encoding from lookup; first exact/ignore-case
let enc = lk.match(k, 'encoding');
if (!enc) {
// Try regex lookup if no exact match
enc = lkRegex.match(k, 'encoding');
}
if (enc) {
// If value is string and looks like base64, decode
if (typeof v === 'string' && looksB64(v)) {
__e[k] = tryDecode(v, enc);
}
// If value is array, try element-wise
else if (Array.isArray(v)) {
__e[k] = v.map((el) => (typeof el === 'string' && looksB64(el)) ? tryDecode(el, enc) : el);
}
// If value is object, you can recurse if you need deep decode
// else leave as-is
}
});
})();- This leverages
__eto mutate the event in place as documented C.Lookup/C.LookupIgnoreCase/C.LookupRegexusage.- If you need deep traversal across nested fields, wrap the
Object.keysloop in a recursive walker or switch to the Code function’s__e.__traverseAndUpdatestyle pattern (if available in your runtime).
- Known textual pattern in many fields: Mask with a wildcard and a regex that captures the base64 portion; decode in Replace Expression.
- Unknown keys with policy-driven selection: Eval with
__eiteration and lookups to drive which keys to decode and with which encoding.
This is my use case:
I have an ldap export I need to parse. Every line is a key-value pair. If the format is key: string, it's text, if the format is key:: string, the value is base64. What I've done so far is use the parser function twice, once with the regex:
<?<_NNAME_0>\S+):: (?<_VALUE_0>)\r\r<?<_NNAME_0>\S+)[^:]: (?<_VALUE_0>)\r\rNo, but after you parse, then you can use Mask or Code
Yeah
You have a field called
base64_fields? Can you provide me a sample event?You can DM me an event
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.