Skip to main content
Question

Crowdstream - Bytes out size doubled

  • March 11, 2025
  • 9 replies
  • 39 views

Hi Everyone

We have integrated Azure Storage Account and Azure Application Gateway with CrowdStream (Cribl Stream) via EventHub. Since logs are in nested JSON format, we used unroll function to convert them into individual events before forwarding them to CrowdStrike NGSIEM.

Currently, we are observing that Bytes Out size is twice that of Bytes In. While we understand that event count should double, our concern is why Bytes Out size is also doubling compared to Bytes In. Additionally, we have not observed any duplicate events in CrowdStrike NGSIEM.

9 replies

Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

A sample of the original event, and the resulting event(s) would help a lot. My best guess without any samples is that your unroll is keep bits of the original event in each and every resulting "unrolled" event.


  • Author
  • New Participant
  • March 11, 2025

Hi Jon,

Thanks for the response, As we couldnt capture sample input and output log. attached the random log files


Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

Can you share the pipeline(s)?


  • Author
  • New Participant
  • March 11, 2025

created 1 Pipeline with function unroll and attached it to source pre-processing.

925_2e4f2b39fc5e4913a32b1ae44f5dfb7c.png

Please find attached screenshot for reference.


Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

I'm not clear why you're seeing this behavior, but try adding an Eval function before the Unroll. In the Eval, drop the __raw field. That's a double underscore raw. Let me know if that helps.

Edit: I've confirmed internally that __raw is included in the internal metrics and volume accounting. Use the Eval mentioned above to remove it to avoid this mistake. I'll raise a ticket to for the product team to look into this.


  • Author
  • New Participant
  • March 11, 2025

HI Jon,

We did add eval first then unroll, Now i dont see —raw field in output log, However still I observe increase in Bytes Out size.

Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

Do you have multiple outputs? If you're sending data to more than one destination, your output will be appropriately higher. What does the preview screen's inspection show you? (The bar chart icon at the top of the preview pane)


  • Author
  • New Participant
  • March 11, 2025
No we dont have multiple output, one source has one destination. Also in inspection preview no much different in events IN and events OUT.

Jon Rust
Forum|alt.badge.img
  • Employee
  • March 11, 2025

Hard to tell from these screenshots, but I don't see anything obvious. I'd open a support ticket: support@cribl.io