Skip to main content
Question

Passing data to another host with no extra meta data

  • Mar 11, 2025 3:17 AM UTC
  • 35 replies
  • 368 views

Hello all. I feel this should be simple, but not working for me. I receive lots of different data types on one port. For one particular type, I want to just forward it on, completely untouched, to a different host on a different port as JSON. I am using the TCP JSON output. The issue is, Cribl is setting the metadata and parsing fields, such as _raw. I don't want any parsing, no fields, no metadata- just the raw data. I must be missing something? Thanks in advance for any suggestions.

35 replies

Shawn Cannon
Forum|alt.badge.img+1
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

Did you try sending it through the passthru pipeline?


What destination are you sending to?If it's not Splunk, you can drop fields you don't want going out the door before you transmit.


post processing pipeline/Pack


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

If I use passthru pipeline, Crible sets metadata fields host and _time


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

so can't use that


Shawn Cannon
Forum|alt.badge.img+1
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

you can remove host and _time fields


Shawn Cannon
Forum|alt.badge.img+1
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

use a post processing pipelne to do that


Shawn Cannon
Forum|alt.badge.img+1
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

and the eval function


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

I guess I will have to mask the _raw at the beginning also?


Shawn Cannon
Forum|alt.badge.img+1
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

_raw is the raw event I would think you need that


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

that will work I guess, but I thought there must be a simpler way


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

I need the data untouched, so no field name


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

it comes in as raw json, like {data1:"blah"}


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

in the output I have "_raw":"{bl;ah.."


Shawn Cannon
Forum|alt.badge.img+1
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

ok guess i thought _raw was the raw data. Iwill wait for someone from Cribl to chime in.


Michael Katz

You can also create a pipeline that doesn't do anything other than remove the fields you don't want passed on. I do that in Packs, becaus they're more portable, but to each his own.


Michael Katz

use that instead of a pass-thru. it acts like a pass-thru


Michael Katz

but gives you more control.


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

Thanks for the suggestion of using post-processing. That fixed the metadata stuff.


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

However, I still have a _raw field


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

so my incoming message looks like {json_field1:blah...


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

I need to get rid of the field name _raw


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

outgoing looks like raw: {json....


  • Author
  • Known Participant
  • Mar 11, 2025 4:12 AM UTC

I tried masking it


Hey Steve, What source is receiving the data? I'd need to doublecheck - there is probably a better way to do this - but you could just create a 2-function pipeline to parse out the fields from _raw and then remove _raw, _time, and host: