Solved

HiWe have setup event hub to send data stream to cribl and our now trying to extract the json into individual events. We have tried the event breaker JSON Array method and JSON unroll method. Neither seems to work. Has anyone tried this successfully or have pointers?Thank you

Can I use self-signed certificate with a REST collector? Does Cribl perform any TLS certificate validity checks?

If I need to accept an SNMP trap from a source, and send it to multiple trap destinations, will Cribl spoof the source IP and send the trap “un-altered” to the final destinations? Meaning will it preserve the source IP address from the original trap IP header?

Not a question but rather something useful. I have data that is in key:value format, using key=value in the extract format did not work. I was able to extract every key and value with this one regex extract.(?<_NAME_0>.*):\s(?<_VALUE_0>.*)

i have a use case where we have a nfs share that applications writes logs too i would like to monitor these while being written to (file monitor?) but distribute the load across workers. is that possible?

Hi guys,I’m testing cribl stream for the 1st time by getting a splunk UF to forward to a Worker on the same centos7 host (ip: 10.0.2.9).When checking Live Data, I only see events from /opt/splunkforwarder/var/log/splunk/metrics.logI was expecting to see events from /var/log/audit/audit.log and /var/log/secure which are listed as being monitored files:/opt/splunkforwarder/bin/splunk list monitorYour session is invalid. Please login.Splunk username: adminPassword:Monitored Directories:$SPLUNK_HOME/var/log/splunk/opt/splunkforwarder/var/log/splunk/audit.log/opt/splunkforwarder/var/log/splunk/btool.log/opt/splunkforwarder/var/log/splunk/conf.log/opt/splunkforwarder/var/log/splunk/configuration_change.log/opt/splunkforwarder/var/log/splunk/dfm_stderr.log/opt/splunkforwarder/var/log/splunk/dfm_stdout.log/opt/splunkforwarder/var/log/splunk/first_install.log/opt/splunkforwarder/var/log/splunk/health.log/opt/splunkforwarder/var/log/splunk/license_usage.log/opt/splunkforwarder/var/log/sp

Hi Guys,I have setup a test environment on my laptop using VBOX and with a Centos7 edge leader node and worker using ports 9000(ui) and 4200 for internal comms.I’ve installed the worker node using curl. Seems there is an issue with the worker node connecting to the leader as shown below. From the worker node, telnet to the leader nodes port 9000 and 4200 connect fine. But not sure what additional stuff to check based on the below messages:{"time":"2022-05-23T00:50:14.256Z","cid":"api","channel":"output:DistWorker","level":"info","message":"attempting to connect","host":"10.0.2.4","port":4200,"tls":false}{"time":"2022-05-23T00:50:14.256Z","cid":"api","channel":"output:DistWorker","level":"debug","message":"will retry to connect","nextConnectTime":1653267018346}{"time":"2022-05-23T00:50:14.256Z","cid":"api","channel":"output:DistWorker","level":"debug","message":"connecting","host":"10.0.2.4","port":4200,"tls":false}{"time":"2022-05-23T00:50:14.264Z","cid":"api","channel":"input:D

I am new to the distributed world, so please bear with me. I have a distributed setup with 2 worker groups(default & 2ndgroup). I have assigned nodes to each of the above 2 groups. In the leader ui when I navigate into the "2ndgroup" and make changes to my pipelines and commit and distribute the changes, it gets pushed to my git repo(remote) & the worker nodes.When I further navigate into the worker nodes and do a live capture of events and make changes to the pipelines, how do I push these changes to the git remote and further when I view it in the leader node under group view reflect the changes that were done at the node level? May be I am missing something obvious, can somebody shed some light on what I am not getting right in this flow?

So I have a script that will collect data from an API. The data is returns is CSV formatted, the first line will be the list of fields. How do you tell Stream to use that first line as the headers?

HiAn interesting topic came up yesterday which I didn’t have a good answer too, around how many events are in flight in Stream at any point in time.It was a discussion around potential for data loss, and if something adverse were to happen the worker, what the impact would be. I imagine there is up to one event being processed per worker thread at any point in time, but is there a buffer of events before the worker thread picks it up? If so, how much, and if not, does the thread just block the network socket until it’s done processing?Supplementary to that - what happens when a destination goes down? Persistent queues aside, how much will Cribl buffer before applying back-pressure to the pipeline?And probably scenarios I haven’t considered. How much data is it theoretically possible to have in flight at any point in time, and thus risk of loss in case of catastrophe?

One of the neatest features of the product is the live preview. This works great for relatively small events.If I have a massive event with a pretty big intensive pipeline to boot, things time out and it really is a PITA to debug my code. How do people work through such a work flow?I usually increase the timeout from 10 seconds to 60 seconds and "play" the pipeline, but I wanted to know if people had some other nifty hacks.Thank you!b1scu1t

Hi, not sure if this is a FREE license thing or something I have wrong with docker. Here is my docker compose.version: 3.5services: master: image: ${CRIBL_IMAGE:-cribl/cribl:latest} environment: - CRIBL_DIST_MODE=master - CRIBL_DIST_MASTER_URL=tcp://criblmaster@0.0.0.0:4200 - CRIBL_VOLUME_DIR=/share/homes/chris/cribl:/opt/cribl/config-volume ports: - 19000:9000 hostname: cribl_leader volumes: - /share/homes/chris/cribl:/opt/cribl/config-volume workers: image: ${CRIBL_IMAGE:-cribl/cribl:latest} depends_on: - master hostname: cribl_worker environment: - CRIBL_DIST_MODE=worker - CRIBL_DIST_MASTER_URL=tcp://criblmaster@0.0.0.0:4200 ports: - 9000Every-time I modify the compose and redeploy, I have to re-input config (user / password, registration, etc). The homes DIR is on the host and has open permissions. I do see data being written in there. Is there a setting I may be missing? Thank you Chris

As a Stream user, if I deploy Edge, do I need to have a leader for each of them? Or do they work from the same leader that I use for Stream today?

when i try to upgrade splunk app via search head deployer i get an error that it fails to apply bundle because cribl is busyError while deploying apps to first member, aborting apps deployment to all members: Error while updating app=cribl on target=https://10.150.22.183:8089: Non-200/201 status _code=500; {"messages":[{"type":"ERROR","text":"\n In handler localapps: Error installing application: Failed to copy: /opt/splunk/var/run/splunk/bundle_tmp/eb494662a4bfa 723/cribl/bin to /opt/splunk/etc/apps/cribl/bin. 2 errors occurred. Description for first 2: [{operation:\"copying source to destination\", error:\"Text file busy\", src:\" /opt/splunk/var/run/splunk/bundle_tmp/eb494662a4bfa723/cribl/bin/cribl\", dest:\"/opt/splunk/etc/apps/cribl/bin/cribl\"}, {operation:\"transfering contents from source to d estination\", error:\"Text file busy\", src:\"/opt/splunk/var/run/splunk/bundle_tmp/eb494662a4bfa723/cribl/bin\", dest:\"/opt/splunk/etc/apps/cribl/bin\"}]"}]}

When adding a GeoIP function to a pipeline, where do I upload .mmdb files (or where are they stored if I grabbed a pack that includes one)?

I am kinda being nudged into a move from a Standalone to a distributed setup. So have to start playing in the lab with this before I plan it out in production. Does any cribl’er have any write ups or discussions here where such a thing has been talked over that I can review before embarking on this adventure.My end goal is to have a Cribl Stream Distributed + A handful of Edge clients to start with(and a ton of Splunk Universal Forwarders sending out stuff to the worker node).

Is there a pattern that I can use to suppress Splunk logs based on data that is part of the _raw field? So far I have come up with a pipeline that uses Regex Extract, Suppress, and Eval function. I am extracting the key:value pair that I need from the _raw field, utilize it in my suppression function, and then remove it from the logs before sending the event to Splunk. While this seems to be working, I wonder if there is a better way to achieve this especially if we want to add additional key:value pairs to suppress on in the future? I was looking into the Parser function that allows me to extract all key:value pairs from the _raw field but I could not find a way to remove all those from the event before sending it to Splunk.

Can i do this in the product today or would it have to be an alert developed outside of Cribl Stream?

I’ve been experimenting with “Next Processing Queue” in the Advanced Settings for a SPlunk HEC Destination, and I can’t seem to tell any difference when I change the value. I’ve used the default value (indexQueue) as well as parsingQueue, but I don’t see any difference once the data gets to Splunk.What exactly is this setting supposed to do?

We are enabling persistent queuing (PQ) on our destinations and notices that the Max File size is defaulted to 1MB. With the amount of data we are processing that will lead to A LOT of small files so we were thinking of increasing that to a Max File Size of 100MB. However, before we do that - we wanted to know if there was any potential impact to Cribl if the PQ files are to large ?When Cribl is reading these files to send to the destination, does it read the entire file into memory ?

When i check the monitoring page for sources, we still see data, and we see logs for the deleted collector. The logs button is greyed out though.

Is it possible to stream logs and events to Cribl Stream, send a copy of that event to S3 (or similar) for archive, and then process/parse/enrich the logs before sending them to another destination? The use cases I’m thinking of are as follows:(Primary Use Case) If a security incident occurs and we need to present evidence in court, how do we protect the forensic integrity of an event/log/etc. if the event is being changed in flight before going to an archive or analysis tool?If we needed to replay a series of logs for some reason or another, it might be helpful to do so from the source events, unmodified by Cribl.The downstream, long-term storage solution may also serve as a repository of data for other teams to use. Providing unmodified events for those teams may be preferred in that case.Thanks!

Is there a good way to enable HEC destinations where the HEC input on the Splunk side has Indexer acknowledgment enabled?

I have a field that includes IP and Port (i.e. 0.0.0.0:1234) and I want to split them into separate fields for further analysis. What’s the best way to do that?

Does GitOps only make changes via the UI read-only or are changes via API read-only also?