How do you get the workers to send metrics (like CPU usage, etc.) to Splunk?

Enable the internal Cribl Metrics input to send them through the pipelines.

Since I didn't set up the Cribl env, I will have to look to see how that leader was set up, I guess to see how to do that for the workers.

Not sure what you mean by the leader sending metrics. The leader doesn't send out any metrics.

0

These are going to the cribl_logs index. The cpuPerc is what I'm most interested in.

I think....

So, this is what we have that i would consider the metrics going to a metrics index, but there is nothing in the index (which is a metrics index):

Those are only logs. Not actual metrics in a metrics format. Metrics need to go to a metrics index. If what you have there is sufficient then you won't need the cribl metrics source enabled but you'll want the cribl logs source enabled instead.

I didn't set that up, BTW, so please don't think I know why it was set up this way.

Is the cribl metrics source enabled ?

How do I find that out?

Go into the inputs for each worker group to see if that input is on.

From the Sources -> Cribl Internal -> CriblMetrics, they are enabled. Is that what you mean?

yes

So if they are already enabled, why would I not see the metrics in that index?

Thank you for your help, BTW. I truly appreciate it.

hard to say, i've only seen a small section of your configuration. Maybe a route for them isn't setup? Maybe they aren't even going thru that pipeline.

Route is there.

I recommend filing a support case so we can review the entire config in depth and possibly conduct a zoom call if necessary. Please include a diag bundle from a worker node (not the leader node) for our review.

Pipeline:

Gov't won't let out diags that aren't completely cleaned and then they have to go through a clearance process. But I can file a support case, and do a screen share.