do you see them as Metrics in Cribl? They should get that little M icon

no but they're coming in from the UF through normal port 9997 communications so they shouldn't get converted to metrics until they hit the props/transforms on the indexers since they are regular PerfmonMetrics inputs I would think.

Well, and now you learn one of the most important things in this POC

Cribl is like a HF. It does all the work. The indexers will consider everything from Cribl as "processed" and will only save to disk

no IDX props/transforms are getting applied here

well damn. so basically any data source we bring through cribl, we need to redo everything the TA on the indexer would have done.

correct

alright, thanks.

I recall we had a customer with Splunk App for Infrastructure where we send to HEC so that the app could still convert the logs to metrics in Splunk.

ok. we're using IT work essentials but the same props/transforms would apply so I might try that. I'm wondering if there's any real benefit to it sending it through Cribl in that windows metrics use case though or is it technically just acting as a passthrough and maybe adding an additional Cribl field like cribl_pipe

Let me know if it works and perhaps you could come up with a cool use case

Start at the start: Why are you sending through Cribl?

Ideally to show how Cribl can save us money/ingest

That usually means dropping events or aggregating data, or removing partial unnecessary data from an event... which means you'd at least need to parse the event in Cribl, to figure what you want to do with it

agreed except for the passthru still "cooking" the data from the indexer point of view. I was thinking initially that we'd be able to just send whatever datasource we wanted to Cribl and just use passthru to decide if we could streamline, drop or transform the data until I got my bubble burst with the information about indexers not apply their normal props/transforms to that sourcetype anymore because it's now cooked which in essence means we'd have to redo all the index time extractions and transforms if we were to do that. I can see where it makes sense on data sources that don't have good TA's already or even no TA's for it but it just means we need to pick and choose more carefully about what we plan on sending through Cribl.

Cause everything is better with Cribl! Even with passthru, just being able to manage everything in the UI, see real-time preview of events in the stream, and make the life of a Splunk admin better, it all adds up to value that can't be measured with just a calculation.

I mean, most TAs do pretty limited index-time work. Set timestamp, maybe split sourcetype using a few regex, and that's often it, everything else is search time

that brings up a question, is there anyway to send from Cribl but not "cook" the data so the indexers still do their normal transforms other than sending it via HEC?

Well, you can send it as syslog or anything comparably ugly

but I'd really not advise to do that

and there's a dirty hack to force Splunk to re-parse, but you're clearly stepping into unsupported territory there

what "hack" are you referring to?

just curious

some setting in inputs.conf that effectively tells Splunk to send received data to a different point in the processing pipeline

Yeah, I crashed my Splunk doing that hack lol!

gotcha

Honestly, I'd once bite into the sour apple (literal translation of German saying), and adapt your TAs, and that's it

it's usually less work than you think

I'm going through the admin training now on http://university.cribl.io now to get better handle on efficiently doing that so we'll see how it goes.

if you're coming from Splunk, and you think of "doing all that shit in props + transforms" - it's SOOO much easier in Cribl

I am coming from splunk. But I've already done the props/transforms work for most things so I'm mainly not wanting to break currently working data sources in the process of moving them to Cribl until I get a good handle on Cribl event manipulation so all the relevant fields still get a extracted properly.