General

OK I have another REST collector question. My collect section I am using POST with body. The body is JSON and it has to provide the time there (start and end). I was trying `${earliest}` and `${latest}` but it does not seem to work. I think it is expecting the date to be in a certain format. This collector is currently a bash script but I was thinking it would work as a REST collector. The bash script sets and start and end variable like this:start="$(date --date=-1days '+%F %H:%M:%S')"end="$(date --date=last-minute '+%F %H:%M:%S')"How can I do this with the REST collector? Ideas?

Hey everyone! General question to see if I'm missing something. We're about to configure a Health check that is going to be ran across dozens of endpoints and will inevitably expand in the future even more. Looking to make sure this is as painless as possible so I'm trying to figure out the "best practice" for doing this. We're essentially monitoring overall health pages exposed by some of our containerized applications. All the urls will be formatted the same (<environment>.http://site.com/status/json.php|site.com/status/json.php). Ideally we'd have a way for this to dynamically update based on some outside source, second best would be for us to maintain a lookup on cribl of all the environments. Just looking for a way that I don't need to manually create sources and rely on my dev ops teams to tell me when new environments are spun up. Any thoughts?

Is there a way to order/sort the JSON keys prior to hitting the destination? I thought prepending the key names with an underscore would bring them on the top but they show up at the bottom of the JSON structure. The ordering looks the way we want it in the pipeline view but come out differently in the actually output.

I want to duplicate a certain set of sourcetypes to a second index (for reasons). I'm typically processesing data through a pipeline and adding my index to the data and then sending off to Splunk via HEC. What's the best way to do this? Do I just need a second route, with a duplicate pipeline configured that then set a different index?

To those of you that are using QRoC/QRadar if I am sending the data to QRoC as simple syslog, on the IBM Qradar side what should I select as a log source type? I went looking through the list and didnt see anything jump out at me though I did see a few that could be considered generic. What I would like to know is what have others used?

REST collector question. I used a standalone test box to get my REST collector working like I wanted. authentication, discover and collect worked great. When I moved it into my distributed environment, it errors out but does pull in some events. The discovery and collect phases are distrubuted across my 5 workers. It seems like the authentication token may not be working properly when it runs on more than one worker? I am guessing of course. I get errors like 503 Service Temporaily Unavailable and 500 Internal Server Error. This does not happen when I run it from my single standalone test server. I may need some assistance trying to figure this one out.

TCP syslog flow to cribl is reporting that TCP window is full

how do people handle secrets that need to be used in scripts that Cribl may run?

Can I create a lookup file from data I receive from a collector? I have two fields in a JSON array that I want to use to create/update a lookup file every time the collector is run.

I'm using a code function in my pipeline for some extra custom logic, but exporting that as yml config is kinda ugly. Is there any chance I can load a .js file up and just tell the code function to run the js file? This would make local testing of my custom js a much nicer experience. I am doing this with samples for my test cases where I have the sample file locally and I load it up to the cribl server, and reference the file in my samples.yml. I'd love to do basically the same thing with a code function.

Need some help with a time issue. The raw event starts with this:03-05-2023 14:40:27 0DF0 PACKETThe time on in (before processing is correct), but post processing is incorrect.Pre-processing is 2023-03-05 (correct) but post processing is 2023-03-05

Hi all,Setting up my first HEC source within Cribl. I'm sending json from Corelight to this new HEC source. Should the HEC services endpoint within Cribl be "/services/collector/event" or "/services/event" ?

Does anyone know how to migrate Splunk Cloud Archive frozen data to Azure blob (or even AWS)? All the information I found was on putting hot data into Azure/AWS.

hello all, is it possible to change the max size of cribl.log?it's rotating too fast, at the moment

We are trying to ingest logs from azure blob storage and for that, we exported environmental variables (as suggested by cribl team and exported NODE_EXTRA_CA_CERTS)them in each worker and the that was successful, all the workers in the source turned green in UI and we can see data ingestion happening. Now we want to do it in a permanent way like exporting environmental variables in .sh file in /etc/profile.d and then terminated the session for the changes to apply and i can see exported env variable when i do printenv but when in see them in UI i see workers are red.

I'd like to generate logging messages when a Cribl worker node goes off-line and also when it recovers operation. The CriblLogs internal data source generates an event with a "channel" value of "MetricsStore", a "message" value of "active messages" and a "numMetrics" value of 1 when a worker begins to recover operation, and other numMetrics values such as 212,193,303 and 304 when the worker is running. Those events appears to be valid sources to generate one or more log entries for worker recovery. Where are the numMetrics codes documented? I tried searching the Cribl documentation site and the web but didn't find anything about them.Finding an event generated when a worker goes off-line proved to be more difficult. There is a file on the leader node that records incidents in which a Cribl worker goes off-line. That file is: /opt/cribl/config-volume/state/kvstore/default/InactiveWorkerStore/state.json.I couldn't find any events similar to the entries in that file in the CriblLo

Are there any plans to implement something like server classes in splunk some day to cover the problem described here: https://curious.cribl.io/t/cribl-architecture-question/740?We also really stuck in between a pragmatic and proper solution with cribl.

Hey,Can we merge two APIs response in Cribl? I have implemented discover and used the data from discover in collect but in the response, I am getting the data of collect only. So can we get the data of discover and collect in the response?

Is there a way to set a retention period for filesystem destination or is it necessary to build custom skripts outside of cribl?

Anyone integrate with Microsoft cloud app security?

Good morning all. Does the S3 Collector pull every file from the s3 bucket on each run? Or, does it have some state to tell what has/hasn't been pulled?

Hello, small question about index time fields, if I want to create a new field with my logs coming from a syslog (eval function on my cribl), I have to put a fields.conf file on my splunk instance which declares all the new fields I will create from cribl ?example on fields.conf :[my_field]INDEXED=true[my_field2]INDEXED=trueIs this the only prerequisite for not having an error during indexing? As for the fields.conf, do you have to put it on the SH and IDX also in a distributed environment?

:question: With Cribl GitOps - how can we avoid script collector running ONLY in production environment?

Is there a way to force the Worker nodes (using Docker) to communicate with the leader over a specific TLS version?

I'm trying to test 2 different index mapping in Elastic. How do I send output to these 2 indexes? Doesn't seem like I can do with the "Output" when defining routes (no quick connect).