Anyone know the proper method to extract something prior to pipelines to use for filtering?Case: I'm looking to use kubernetes.namespace_name as a field to compare against to send through different routes. It's nested inside _raw and I can't seem to get it to be referenced without doing some nasty .search() or .match() that could match with other data streams coming through that source
Question
Anyone know the proper method to extract something prior to pipelines to use for filtering?
Solid thinking
Use a pre-proc pipeline, or a pipeline chain
assuming _raw is a JSON object. That allows you to avoid a generic search/match of the data by being able to reference specific fields using JSON syntax. Does that help?
_raw is a JSON object indeed, so this worked like a charm! I had totally assumed it wouldn't want functions like that being run in the filter, but i'm glad it lets it.
To Jon: That wasn't necessary for this go-round but I'll keep that in the back pocket for all the other data sources
<@U034GKZLVB8> any field that has a blue vertical bar on the left side allows JS expressions to be used.
if i'm going to reference things inside the object more than once, i'll usually set-up a pre-proc to set `__parsed` => `JSON.parse(_raw)` to save cycles
I did know that, just expected some limitations on some of those fields that weren't in the pipelines. Love the freedom.And another helpful tip from Jon as well.Always appreciate the expedient help!
Sign up
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.