Cribl Stream HEC Pipeline for Double-Encoded JSON Arrays

Symptom

Cribl Stream receives an HEC event whose body field contains a JSON array encoded as a string. The array is not available for direct unrolling. For example, the parsed envelope contains body similar to this value:

"["timestamp":"2042-42-42T18:00:00.000","transaction_id":"..."]"

The escaped quotes indicate that body is a string, not a native array.

Environment

• Cribl Stream pipeline processing HTTP Event Collector (HEC) events
• HEC payloads with a JSON-encoded array inside the body field

Resolution

1. Navigate to Processing and select Pipelines.
2. Create or import a pipeline.
3. Add a Parser function in Extract mode with type JSON and source field _raw.
4. Add an Eval function that assigns events = JSON.parse(body).
5. Add an Unroll function with source expression events and destination field my_event.
6. Add an Eval function that assigns _raw = JSON.stringify(my_event).
7. Remove the scratch fields events, my_event, body, headers, and __json.
8. Save the pipeline and verify that each array element produces a separate event.

Cause

This can be caused by:

• The HEC envelope stores the inner array as escaped JSON text in body.
• Unroll receives a string until JSON.parse(body) converts it into an array.

Additional Information

• To promote event fields to the top level, add a Parser in Reserialize mode or expand my_event.* with a final Parser.
• To retain the HEC application header, assign my_event.app = headers['x-app-name'] before cleanup.
• To flatten nested extracted_data arrays, add another Unroll function or reduce the array into an object with Eval.
• For the Unroll function’s source-expression and destination-field behavior, see Cribl Stream Unroll documentation.
• For JSON-array handling based on parsed _raw content, see Cribl Stream JSON Unroll documentation.