Skip to main content
Solved

Data Stream Noise Filtering: Seeking Better Architecture For Complex Conditions

  • April 29, 2026
  • 12 replies
  • 3 views
D
This message originated from Cribl Community Slack.
Click here to view the original link.

I am considering how to tune noise out of a stream of data in a route. I started with: source == 'pan' && _raw.includes('THREAT,vulnerability') && !(_raw.includes('SMB: User Password Brute Force Attempt(40004)') && raw.includes('adm<user>')) However, I can forsee this getting to be pretty long and granular. There must be a better way to architect this. Any suggestions?

Best answer by Jon Rust

There are nearly endless ways to get this done. One would be to keep the source == pan match, but move the rest of the logic into the pipeline

12 replies

Jon Rust
Forum|alt.badge.img
  • Jon Rust
  • Employee
  • Answer
  • April 29, 2026
There are nearly endless ways to get this done. One would be to keep the source == pan match, but move the rest of the logic into the pipeline
D
OMG I am such a spaz, of course!!!

Links for this message:
image.png
image.png
Jon Rust
Forum|alt.badge.img
  • Jon Rust
  • Employee
  • April 29, 2026
Another trick is to hide the code in a global variable
D
can't get the spaz meme to work in slack
D
hmm, yes...I have examples of both variable and drops.
D
Thanks, I feel dumb now
Jon Rust
Forum|alt.badge.img
  • Jon Rust
  • Employee
  • April 29, 2026
lol nah, you're good :slightly_smiling_face:
D
ok one last question. Just putting in a drop for now. Should the match of a drop be a final?

Links for this message:
image.png
Jon Rust
Forum|alt.badge.img
  • Jon Rust
  • Employee
  • April 29, 2026
final has no impact on Drop. It's just a vestigial chunk of code left over from the template engine for functions
Jon Rust
Forum|alt.badge.img
  • Jon Rust
  • Employee
  • April 29, 2026
i like to turn it on so it stands out in the display better :slightly_smiling_face:
justin_cribl
  • justin_cribl
  • Employee
  • April 29, 2026
Could you parse a field and then use a lookup to keep or drop specific logs?
D
Totally doable but is that CPU efficient versus dropping them?
Terms & ConditionsAccessibility statement