we're doing a POC for cribl and I'm wondering if anyone is sending PerfmonMetrics through Cribl. I tried it by filtering the route for__inputId=='splunk:in_splunk_tcp' && index=='win_em_metrics' through the passthru pipeline and even though I"m seeing them in the capture before the destination they aren't showing up in the metrics index win_em_metrics like I'd expect.
Question
Is anyone sending PerfmonMetrics through Cribl?
do you see them as Metrics in Cribl? They should get that little M icon
no but they're coming in from the UF through normal port 9997 communications so they shouldn't get converted to metrics until they hit the props/transforms on the indexers since they are regular PerfmonMetrics inputs I would think.
Well, and now you learn one of the most important things in this POC 
Cribl is like a HF. It does all the work. The indexers will consider everything from Cribl as "processed" and will only save to disk
no IDX props/transforms are getting applied here
well damn. so basically any data source we bring through cribl, we need to redo everything the TA on the indexer would have done.
correct
I recall we had a customer with Splunk App for Infrastructure where we send to HEC so that the app could still convert the logs to metrics in Splunk.
ok. we're using IT work essentials but the same props/transforms would apply so I might try that. I'm wondering if there's any real benefit to it sending it through Cribl in that windows metrics use case though or is it technically just acting as a passthrough and maybe adding an additional Cribl field like cribl_pipe
Let me know if it works and perhaps you could come up with a cool use case :slightly_smiling_face:
Start at the start: Why are you sending through Cribl? :slightly_smiling_face:
That usually means dropping events or aggregating data, or removing partial unnecessary data from an event... which means you'd at least need to parse the event in Cribl, to figure what you want to do with it
agreed except for the passthru still "cooking" the data from the indexer point of view. I was thinking initially that we'd be able to just send whatever datasource we wanted to Cribl and just use passthru to decide if we could streamline, drop or transform the data until I got my bubble burst with the information about indexers not apply their normal props/transforms to that sourcetype anymore because it's now cooked which in essence means we'd have to redo all the index time extractions and transforms if we were to do that. I can see where it makes sense on data sources that don't have good TA's already or even no TA's for it but it just means we need to pick and choose more carefully about what we plan on sending through Cribl.
Cause everything is better with Cribl! Even with passthru, just being able to manage everything in the UI, see real-time preview of events in the stream, and make the life of a Splunk admin better, it all adds up to value that can't be measured with just a calculation.
I mean, most TAs do pretty limited index-time work. Set timestamp, maybe split sourcetype using a few regex, and that's often it, everything else is search time
that brings up a question, is there anyway to send from Cribl but not "cook" the data so the indexers still do their normal transforms other than sending it via HEC?
Well, you can send it as syslog or anything comparably ugly :slightly_smiling_face:
but I'd really not advise to do that :slightly_smiling_face:
and there's a dirty hack to force Splunk to re-parse, but you're clearly stepping into unsupported territory there
some setting in inputs.conf that effectively tells Splunk to send received data to a different point in the processing pipeline
Yeah, I crashed my Splunk doing that hack lol!
Honestly, I'd once bite into the sour apple (literal translation of German saying), and adapt your TAs, and that's it
it's usually less work than you think
I'm going through the admin training now on http://university.cribl.io now to get better handle on efficiently doing that so we'll see how it goes.
if you're coming from Splunk, and you think of "doing all that shit in props + transforms" - it's SOOO much easier in Cribl 
I am coming from splunk. But I've already done the props/transforms work for most things so I'm mainly not wanting to break currently working data sources in the process of moving them to Cribl until I get a good handle on Cribl event manipulation so all the relevant fields still get a extracted properly.
Sign up
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.