Community Discussions

I am trying to send logs from cribl to OpenObserve.OpenObserve accepts data in a JSON array format.e.g.[ { "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus" }, { "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus" }]What is the best way to do this?

Hello everyone,I have the following problem. I'm designing a distributed deployment. I have to create an environment for data reduction but before that I need to send the data to a blob storage, for compliance reason, as you can imagine. Now, all of these is achieved but I have some doubts. When I use the Azure blob storage as only destination, all the 100% of the data goes in there but, the Monitoring Page, is telling me that the data that comes in are the same as the data that goes out (sometimes, the data that goes out are even more), even with compression enable. As said, I have the g-zip compression enabled on the Azure Destination page. I've checked the storage and I found out that the data are really compressed. So, my point is the following: when I try to estimates the possible number of workers and their capabilities, should I trust the monitoring page and consider DATA IN = DATA OUT (since the data is not processed by any pipeline at all) or I c

Hi,We are currently working on updating our Cribl Edge Agents. While our Linux agents successfully auto-update through the UI, we are encountering issues with our Windows agents. Specifically, the Windows agents display an error during the download process and fail to update.Manual updates are possible by logging into the system and running the update script; however, the UI-based update consistently fails. We have confirmed that port 4200 is open, and there are no indications of blocked or dropped traffic. Additionally, all platform-specific packages, including the corresponding hash files, have been prepared and placed in the appropriate Cribl temporary directory.Given these conditions, what could be causing this issue?

I am new to cribl and trying to setup Splunk HEC destination from cribl sanbox stream instance. I have provisioned a free splunk cloud instance and setup HEC. While I can send data via: curl -k "https://<host>:8088/services/collector" -H "Authorization: Splunk 38aa4a38-8fd5-4faa-afc4-9b3533ac39c7" -d "{"event": "Hello, world!", "sourcetype": "manual"}"I am unable to run destination test from cribl with the exact endpoint or with /services/collector/event. Everytime the request time out with error:2025-02-20T14:23:57.956αtime:2025-02-20T14:23:57.956Zαcid:w0αchannel:output:splunkαlevel:errorαmessage:error while flushingαioName:splunk_hecαioType:destination{}error:αmessage:request timeoutαstack:Error: request timeout at /opt/cribl/bin/cribl.js:15:9721630 at new Promise (<anonymous>) at p.doHttpRequest (/opt/cribl/bin/cribl.js:15:9721585) at runNextTicks (node:internal/process/task_queues:60:5) at process.processTimers (node:internal/timers:509:9)

Hi Anyone/supportYesterday, started Cribl-Cloud (stream), just for testing, now my Splunk License is chocking with data from it Crible Cloud :-(.Tried to delete all things I could delelte, still chocking my Splunk license. Deactivating my subscriiption says: We're sorry to see you go!We are processing the cancellation of your Cribl.Cloud organization. This process takes up to 30 days and you will get an email 7 days before the cancellation is final. If you didn't mean to cancel or having second thoughts, contact support@cribl.io.How can I interrupt this data-stream. I had to stop my Splunk Enterprise instance for this…Hope someone can help out.Ashley Pietersen

I am trying to pull in DNS Logs from Cisco Umbrella. I have my S3 Connector configured to pull down the logs every 5 minutes. The problem is that it pulls down all the logs. I only want the new logs. I have tried adding the follow to my Conf section of the JSON configuration but it doesn't appear to work."trackProgress": true,      "trackMode": "file",      "trackField": "s3.object.etag",      "useLastModifiedTime": true

Hi, I have gone through the cribl university user and admin courses and setting up the product is quite easy however am constantly stuck trying to parse or extract strings of data into fields so that we can search or use the information.For systems which send in JSON or CSV it is very simple, however most of ours are embedded Linux or network devices which give something like the following:<183> 02/06/2025:00:24:20 GMT NYDC1VPX01-DMZ 0-PPE-0 : default SSLLOG SSL_HANDSHAKE_SUCCESS 9838002 0 : SPCBId 33168208 - ClientIP 10.1.1.130 - ClientPort 57486 - VserverServiceIP 10.200.80.80 - VserverServicePort 443 - ClientVersion TLSv1.2 - CipherSuite "TLS1-AES-256-CBC-SHA" - Session New - HandshakeTime 47 msShowWhat combination of functions can split this out into fields such as hostname, clientip, clientport, etc.I have tried using ChatGPT to help and can get regex and grok expressions from the AI however these don't work when applying to cribl. I d

Hi EveryoneWe have integrated Azure Storage Account and Azure Application Gateway with CrowdStream (Cribl Stream) via EventHub. Since logs are in nested JSON format, we used unroll function to convert them into individual events before forwarding them to CrowdStrike NGSIEM.Currently, we are observing that Bytes Out size is twice that of Bytes In. While we understand that event count should double, our concern is why Bytes Out size is also doubling compared to Bytes In. Additionally, we have not observed any duplicate events in CrowdStrike NGSIEM.

I read that Cribl has the ability to update lookup tables using REST API and with a search string, but I'm looking for a way to do it during the processing stage. Is this possible? I see a Redis function and read something about that. If the values go there, is there a way to automatically lookup the results?

Hi everyoneI have different locations which need to have separate cribl stream installed on each location .. Which means standalone deployment. In this case how the license will be deployed .. Will it have separate enterprise license applied to each Cribl instance ?

I have installed cribl standalone on a VM , but need to change it to leader ..Do I need a worker node or only a leader can do the processing work as a stream ?If worker node is required, can the same VM be a leader and worked node together ?

Hello,I'm new to Cribl stream and am having difficulty translating the API spec from the documentation. I can get all the curls working fine, but I don't know how they translate to cribl. For example, if I'm trying to use cribl for this type of curl:curl --location --request GET 'https://public-api.cloud.meater.com/v1/devices' \--header 'Authorization: Bearer <JWT>'What would my discover and collect headers look like to include the "Authorization: Bearer <JWT>?Cribl is authenticating correctly (I managed to figure that out), but all my requests result in a 400. My authorize expression is `Bearer ${token}`Any help/insight would be greatly appreciated.

I want to create new fields from values in _raw. See below. I want to create a series of folders based off the year, month, date and hour. 2025 (parent folder) - 01(subfolder) - 12(sub under 01) - 23(subfolder under 12)._raw:"2025-01-12 23:59:56","Another Value","Dummy Data","","Something Really Cool"I'm using Regex Extraction and I can see the selection in the preview as Group 1, Group 2, etc.However, I can not convert these into individual fields. Any suggestions?

Hi,Is there a way to find out which worker node processed an event, does it inject a field or can we configure it?Reason is we have some issues and this could narrow it down to a particular worker node.Cheers,Jay

Hi,I would like to secure the communication between the Cribl managed Leader node and the Hybrid Workers and edge nodes currently located on Prem . I understand that the communication between the Leader node and Hybrid Workers / Edge nodes is already encrypted ( port 443 ) . How are those entities being currently authenticated ? I can also see that the recommendation is to implement mutual authentication but not sure about the best way to go about it ?Any advice would be be very much appreciated .Thank You

We are having Hybrid Cribl Cloud Setup where leader resides in Cloud. Is there a way to Get Cribl Cloud leader's Internal/metrics logs to our Splunk

I am attempting to record which server logs are being collected, e.g. last seen, and I haven't found that capability yet. I am thinking I need something custom, maybe write to a file or a lookup. The cribl documentation doesn't provide much in the way of specifics when it comes to implementing scripts, except an ominous warning. I am new to the product and any help is greatly appreciated.

I would like define a Readis connection at the beginning of pipeline and reuse by a code functions through all an event live-time.Something like "C.Lookup" expression.PS: A Redis function is not applicable for my needs :).

Hi ,I have been trying to add an edge node into my default fleet but being denied because of permission error . I have run the script provided by the leader into an Ubuntu EC2 instance with a userwith privileged access but still failing ( see Screenshot ) . Can anyone assist please ? Thanks in advance

Hi ,I ' m looking to add AWS S3 as a destination in Cribl while using a set of Hybrid Workers.I have followed the below instructions but yet failing to successfully test the connection . https://cribl.io/blog/securely-connecting-aws-s3-destination-to-cribl-cloud-and-hybrid-workers/The logs are showing the following errors even though the AWS Roles / Permissions / Policies have been correctly applied . The resources also appear to be sufficient on the Hybrid Workers nodes . This has worked fine with the Cribl Cloud managed workers .Can anyone please advise ?

Hi I am having some problem to get the host name out of some sample log files, the format is RCF 3164. And thru the documentation it says Cribl will try to parse that itself, do I need to decelerate that it syslog as a source?

I 'm trying to add AWS hosted Hybrid workers into our instance of Cribl cloud ( Leader ) .The Workers VMs have been built using the CriblPrivateBuild from Amazon Marketplace .All the firewall policies are in place already . As per the instruction , I invoked the script from the leader and pasted it into our Hybrid Worker nodes but getting the following error ?Can anyone advise please ?Much Appreciated

L.s.,Maybe easy answer for all of you . We have got an HEC input and when i capture the live data i see as host the Cribl worker which is recieving the data. Why is that host filled?In the message itself there is also a host, but the right one. So i send the message in _raw to Splunk and delete the rest (also the wrong host). But tadaa.. there are two host in Splunk. The reciever Cribl and the one from the message.Any clue why?Thanks in advanceJari

Hi,Maybe a simple answer (i hope). For a route we want to filer the host.name, but there are a lot of host in the list so a wildcard is the best way to filter.So doing like 'drnms10*.dmz.somewhere.nl' in the filter for the servers matching with this wildcard. But with a filter 'host == 'drnms10*.dmz.somewhere.nl'' wil not work. What is the best way to do this?I tried with a C.lookup and a lookup file.. it wil work. But stil have top maintain a list. And is a C.lookup not very slow in a filter?Thanks for the answer :)greetsJari

Hi,I' m working on a solution that consist of Cribl sending logs to an Onprem syslog server . Is there a way to confirm whether Cribl has established communication with the destination endpoint ( like a ping feature) . The monitoring section in Cribl is showing data is flowing but we can't find anything on the syslog server . Any assistance will be greatly appreciated.