Skip to main content
Question

Passing data to another host with no extra meta data

  • March 11, 2025
  • 35 replies
  • 80 views

Hello all. I feel this should be simple, but not working for me. I receive lots of different data types on one port. For one particular type, I want to just forward it on, completely untouched, to a different host on a different port as JSON. I am using the TCP JSON output. The issue is, Cribl is setting the metadata and parsing fields, such as _raw. I don't want any parsing, no fields, no metadata- just the raw data. I must be missing something? Thanks in advance for any suggestions.

35 replies

Shawn Cannon
Forum|alt.badge.img+1
  • Inspiring
  • March 11, 2025

Did you try sending it through the passthru pipeline?


  • Employee
  • March 11, 2025

What destination are you sending to?If it's not Splunk, you can drop fields you don't want going out the door before you transmit.


  • Employee
  • March 11, 2025

post processing pipeline/Pack


  • Author
  • Known Participant
  • March 11, 2025

If I use passthru pipeline, Crible sets metadata fields host and _time


  • Author
  • Known Participant
  • March 11, 2025

so can't use that


Shawn Cannon
Forum|alt.badge.img+1
  • Inspiring
  • March 11, 2025

you can remove host and _time fields


Shawn Cannon
Forum|alt.badge.img+1
  • Inspiring
  • March 11, 2025

use a post processing pipelne to do that


Shawn Cannon
Forum|alt.badge.img+1
  • Inspiring
  • March 11, 2025

and the eval function


  • Author
  • Known Participant
  • March 11, 2025

I guess I will have to mask the _raw at the beginning also?


Shawn Cannon
Forum|alt.badge.img+1
  • Inspiring
  • March 11, 2025

_raw is the raw event I would think you need that


  • Author
  • Known Participant
  • March 11, 2025

that will work I guess, but I thought there must be a simpler way


  • Author
  • Known Participant
  • March 11, 2025

I need the data untouched, so no field name


  • Author
  • Known Participant
  • March 11, 2025

it comes in as raw json, like {data1:"blah"}


  • Author
  • Known Participant
  • March 11, 2025

in the output I have "_raw":"{bl;ah.."


Shawn Cannon
Forum|alt.badge.img+1
  • Inspiring
  • March 11, 2025

ok guess i thought _raw was the raw data. Iwill wait for someone from Cribl to chime in.


Michael Katz
  • Employee
  • March 11, 2025

You can also create a pipeline that doesn't do anything other than remove the fields you don't want passed on. I do that in Packs, becaus they're more portable, but to each his own.


Michael Katz
  • Employee
  • March 11, 2025

use that instead of a pass-thru. it acts like a pass-thru


Michael Katz
  • Employee
  • March 11, 2025

but gives you more control.


  • Author
  • Known Participant
  • March 11, 2025

Thanks for the suggestion of using post-processing. That fixed the metadata stuff.


  • Author
  • Known Participant
  • March 11, 2025

However, I still have a _raw field


  • Author
  • Known Participant
  • March 11, 2025

so my incoming message looks like {json_field1:blah...


  • Author
  • Known Participant
  • March 11, 2025

I need to get rid of the field name _raw


  • Author
  • Known Participant
  • March 11, 2025

outgoing looks like raw: {json....


  • Author
  • Known Participant
  • March 11, 2025

I tried masking it


Hey Steve, What source is receiving the data? I'd need to doublecheck - there is probably a better way to do this - but you could just create a 2-function pipeline to parse out the fields from _raw and then remove _raw, _time, and host: