If I am reading the docs correctly, when converting from json to CSV, I have to define all the fields. Since all of the fields are really already known, is there a way to output all the fields first without having to define all of them? We are working on converting some Azure logs, and there are LOTs of properties.field(s). Hoping there is another way without us having to define all of these in the "Fields to serialize" field within the Serialize function.
Question
Query on simplifying JSON to CSV conversion.
All fields must be listed, in the order you require.Copy-pasta works for the list. Possibly in your analysis tool of choice you can get a list?
create a temp function to serialize into K=Vand another to use regex to pull all the field names. In the preview window, copy-pasta the list
hold on, trying it out
```{ "conf": { "output": "default", "streamtags": [], "groups": {}, "asyncFuncTimeout": 1000, "functions": [ { "filter": "true", "conf": { "mode": "extract", "type": "json", "srcField": "raw", "dstField": "_raw" }, "id": "serde" }, { "filter": "true", "conf": { "type": "kvp", "fields": [ "!", "!cribl_breaker", "" ], "dstField": "raw", "cleanFields": false, "srcField": "_raw" }, "id": "serialize" }, { "filter": "true", "conf": { "source": "_raw", "iterations": 100, "overwrite": false, "regex": "/(?<fields>[-a-zA-Z0-9]+)=/g" }, "id": "regex_extract" }, { "filter": "true", "conf": { "add": [ { "disabled": false, "name": "fields", "value": "fields.join(',')" } ] }, "id": "eval" } ] }, "id": "list_fields"}```
» my sample data started as stringified JSON» i parsed it into actual JSON» then serialized into K=V» then regex extract for all field names (strings leading up to `=`), with global flag set» then eval to join the fields array into a list
obviously nested json would present a problem, but then i don't know how you'd plan on making that CSV in the first place
how would you do nested in CSV?
not sure. I think we are doing this for another source, so I need to look. We will tackle this next week
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.