Skip to main content

Products

Get answers, share tips, and explore best practices across the entire Cribl product suite.

  • 765 Topics
  • 0 Replies
765 Topics
Angelo Michele Pizzi
Angelo Michele PizziNew Participant
asked in Stream
Logging level "silly" takes my infrastructure down

Hi everyone,Does any of you ever tried to use the Logging level  known as silly? I’ve tried to use it for troubleshooting an issues on my env, but after less than 10 minutes, my three worker nodes were experiencing memory issues. One of my nodes went completely down, the other two remained with less than 1 GB of RAM (starting from 9 GB of free space in the beginning). I just want to know your experience/opinion on this, is it possible that my memory was not enough? 

151
kprior
Raffaele
RaffaeleNew Participant
asked in Stream
Duplicate field values in Splunk events from Cribl

Hello,I’m using Cribl Cloud to pull JSON events from an Azure Event Hub and forward them to Splunk via HEC. Each incoming event contains a nested array field called records, for example:{ "records": [ { "FileName": "xx", "FileType": "xx", "NetworkMessageId": "xx", "RecipientEmailAddress": "xx", "RecipientObjectId": "xx", "ReportId": "xx", "SHA256": "xx", "SenderDisplayName": "xx", "SenderObjectId": "x", "SenderFromAddress": "x", "FileSize": x, "Timestamp": "xx", "TimeGenerated": "xx", "_ItemId": "xx", "TenantId": "xx", "_TimeReceived": "xx", "_Internal_WorkspaceResourceId": "xx", "Type": "xx" }, { "FileName": "xx", "FileType": "xx", "NetworkMessageId": "xx", "RecipientEmailAddress": "xx", "RecipientObjectId": "xx", "ReportId": "xx", "SHA256": "xx", "SenderDisplayName": "xx", "SenderObjectId": "x", "SenderFromAddress": "x", "Fil

333
Angelo Michele Pizzi
ashu
ashuNew Participant
asked in Stream
CIDR range support in no proxy

HI all just wanted to confirm if CIDR ranges are supported in no proxy setting where i want to mention CIDR range of leader node in worker node settings instead of directly providing leader node IP.Is there any additional settings needed to make this work? 

252
ashu
J
Jonathan RauNew Participant
asked in Search
Cannot POST /api/v1/search/jobs

In both the API Reference (with authenticated session) and via Python, I started running into an issue with submitting queries to the POST /search/jobs API to submit a remote query to Cribl Search.Despite the fact that it's a POST endpoint, and I'm using the headers generated by the API Reference, I receive the following error<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><title>Error</title></head><body><pre>Cannot POST /api/v1/search/jobs</pre></body></html>In the API Reference this is the payload I am POST'ing with application/json as the accept and Content-Type - the Auth comes from my session automatically{ "query": "cribl dataset=\"cribl_internal_logs\" | limit 10 "}In Python, I attempted to recreate it as well, I can confirm that the Client ID and Client Secret are valid and I receive a valid token. Here is the relevant portion of that script.def generateBearerToken() -> str: url = "https://

191
rcalvert
R
redukNew Participant
asked in Stream
Populating lookup table using incoming events

As the title says, I’m looking to create and update a lookup table based on incoming events, somewhat similar to the outputlookup function in splunk. Is there any way to update or create a table based on fields from events passing through a pipeline?

164
chris.daniels
D
Daniel JordanKnown Participant
asked in Stream
Notifications to Microsoft Teams

Looking for a configuration example for sending webhook notifications to Teams channel.

913
cbreshears
V
VenkatNew Participant
asked in General
Query on SIEM parsing

Hi All,We are managing Crowdstrike NGSIEM  in our network and all the data sources are routed to Cribil and from Cribil , we are forwarding the logs to NGSIEM. Data source → Cribil →  NGSIEM I understand , we require parsers in NGSIEM to read the relevent logs received from the  datasources but I wish to know ,is there any parser concept present in cribil to onboard the logs from the different datasources.    

281
B
Louise Tang
Louise TangKnown Participant
asked in General
How do I create a REST API Collector that needs to reference a dynamic bearer token?

This is an example of how we can reference a value provided by Discover Type HTTP Request in the REST API Collector which will correctly save a bearer token value which will expire (default time 5 minutes) using a named Token attribute: token, Authorization header `Bearer $(token)` and Authentication expression `Bearer $(token)` The initial authorization request:curl -X POST 'http://localhost:9000/api/v1/auth/login' -H "Content-Type: application/json" -d '{ "username": "admin", "password": "<password value here>" }' Run a Discover request to generate a bearer token: The above request will return a value something like: {"token":"abcdefgUNIQUEPASSWORDTOKENabcdefgUNIQUEPASSWORDTOKENabcdefgUNIQUEPASSWORDTOKENabcdefgUNIQUEPASSWORDTOKENabcdefgUNIQUEPASSWORDTOKENabcdefgUNIQUEPASSWORDTOKEN","forcePasswordChange":false} The Collect layer will use the information pulled during discover to pull the following: The Encode operation is not necessary but provided for illustration purposes.‘ite

511
Louise Tang
N
nada28.7.1996@gmail.comNew Participant
asked in Stream
enrich data with API

hello,i need options on how to enrich data coming from zabbix as a source on cribl stream before it reaches the destination by API or Http requestthe data is on database i need to call the database by API to fetch the data using fields in the data payload from zabbix and enrich the data received from zabbix then forward it to destinationnote : i want to get the data on fly and the API call will use attributed from the data payload this is the URL that i want to use and will take the massagegroup and the application value from the data payload to call the needed data to be enrichedhttp://IP:25050/3i_service.php?MESSAGEGROUP=X&APPLIKATION=Yi tried by code on the pipeline to do httprequest but it didn't work

260
N
J
jaycriblNew Participant
asked in Stream
Is there a list of links for cribl upgrade files?

Had a situation where the automatic upgrade was not working, so had to manually upgrade.From the logs it was trying to download ARM version not X64 version.Fortunately managed to figure out the correct weblink by changing the file name slightly: https://cdn.cribl.io/dl/4.10.1/cribl-4.10.1-45136dbb-linux-x64.tgz.Are these links listed anywhere on the cribl website, or how would we manually upgrade?

271
J
J
Jari WeststrateParticipating Frequently
asked in Stream
filter expresion and route order

Hello,I noticed a strange thing maybey anyone can explain.I have 3 syslog sources: syslog:514: ,syslog:51402: and syslog:51403:Three routes for these sources, each with a different pipeline attached. The order off the routes are 514 then 51402 and at last 51403.The filter in de routes are __inputId.startsWith('syslog:514:') &&  __srcIpPort.includes('10.10.1.112')__inputId.startsWith('syslog:51402:') && __srcIpPort.includes('10.10.1.112')__inputId.startsWith('syslog:51403:')&&  __srcIpPort.includes('10.10.1.112')(same host with different syslog outputs)In this order the 514 port gets a load off events, the 51402 lesser and teh 51403 nothing. That surpised me because when a check the source side i can capture 51403 events from this sourceip.For test i moved the 514 route after the 51403 route. Just a feeling. but what??.. the 51403 gets suddenly l the events it was missing. And here it gets strange. Just as if the 514 filter takes al the events from the 51403. But

216
Jon Rust
A
Arnold JohnsonNew Participant
asked in Stream
Bootstrap worker from cribl cloud leader UI access

HiI setup a worker node on premise by running the script captured from cribl leader cloud. I enabled UI access for worker node. I can open the link in browser but what will be the credentials for it ?Thanks

72
A
A
a.pietersen@eremote.nlParticipating Frequently
asked in General
Cribl Edge (standalone): Windows Metrics - Splunk, do not result in actionable information

I am new with Cribl.Cribl Edge (standalone, installed on Windows 10 laptop for test/evaluation): Windows Metrics do not return actionable information when forwarding to a standalone Splunk Enterprise instance. What am I doing wrong or what I am missing here?Windows Eventlog cribl-source is correctly received in json. But adding the Windows Metrics cribl-source to the same (working) Splunk Destination (default forwarding to port 9997) it shows:All relevant metric_names seems to be received correctly, but no Values to find. Besides, do not understand the metric_type "g". I would expect a value-field in bytesI have tried many things sofar, including Cloud Stream, searching in the docs and and start some basic first lessons on Cribl Universty. Neither an overall and understandable Google search explanation about type g. Stuck now!Does somebody has a clue, suggestion, tip or direction to search in other sources?ThanksAshleyP

204
A
P
Prabhat SharmaNew Participant
asked in General
Best way to send logs from Cribl to OpenObserve

I am trying to send logs from cribl to OpenObserve.OpenObserve accepts data in a JSON array format.e.g.[ { "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus" }, { "kubernetes.annotations.kubectl.kubernetes.io/default-container": "prometheus", "kubernetes.annotations.kubernetes.io/psp": "eks.privileged", "kubernetes.container_hash": "quay.io/prometheus/prometheus@sha256:4748e26f9369ee7270a7cd3fb9385c1adb441c05792ce2bce2f6dd622fd91d38", "kubernetes.container_image": "quay.io/prometheus/prometheus:v2.39.1", "kubernetes.container_name": "prometheus" }]What is the best way to do this?

151
Jon Rust
Angelo Michele Pizzi
Angelo Michele PizziNew Participant
asked in Stream
Distributed deployment planning with compressing g-zip Destination?

Hello everyone,I have the following problem. I'm designing a distributed deployment. I have to create an environment for data reduction but before that I need to send the data to a blob storage, for compliance reason, as you can imagine. Now, all of these is achieved but I have some doubts. When I use the Azure blob storage as only destination, all the 100% of the data goes in there but, the Monitoring Page, is telling me that the data that comes in are the same as the data that goes out (sometimes, the data that goes out are even more), even with compression enable. As said, I have the g-zip compression enabled on the Azure Destination page. I've checked the storage and I found out that the data are really compressed. So, my point is the following: when I try to estimates the possible number of workers and their capabilities, should I trust the monitoring page and consider DATA IN = DATA OUT (since the data is not processed by any pipeline at all) or I can consider the quantity of byt

113
Jon Rust
J
jbos@svb.nlNew Participant
asked in Edge
Updating Windows Agents Through the UI

Hi,We are currently working on updating our Cribl Edge Agents. While our Linux agents successfully auto-update through the UI, we are encountering issues with our Windows agents. Specifically, the Windows agents display an error during the download process and fail to update.Manual updates are possible by logging into the system and running the update script; however, the UI-based update consistently fails. We have confirmed that port 4200 is open, and there are no indications of blocked or dropped traffic. Additionally, all platform-specific packages, including the corresponding hash files, have been prepared and placed in the appropriate Cribl temporary directory.Given these conditions, what could be causing this issue?

171
Jon Rust
S
Somnath BhattacharyaNew Participant
asked in Stream
Unable to perform Destination test with Splunk HEC

I am new to cribl and trying to setup Splunk HEC destination from cribl sanbox stream instance. I have provisioned a free splunk cloud instance and setup HEC. While I can send data via: curl -k "https://<host>:8088/services/collector" -H "Authorization: Splunk 38aa4a38-8fd5-4faa-afc4-9b3533ac39c7" -d "{"event": "Hello, world!", "sourcetype": "manual"}"I am unable to run destination test from cribl with the exact endpoint or with /services/collector/event. Everytime the request time out with error:2025-02-20T14:23:57.956αtime:2025-02-20T14:23:57.956Zαcid:w0αchannel:output:splunkαlevel:errorαmessage:error while flushingαioName:splunk_hecαioType:destination{}error:αmessage:request timeoutαstack:Error: request timeout at /opt/cribl/bin/cribl.js:15:9721630 at new Promise (<anonymous>) at p.doHttpRequest (/opt/cribl/bin/cribl.js:15:9721585) at runNextTicks (node:internal/process/task_queues:60:5) at process.processTimers (node:internal/timers:509:9) at async p.buildRecProm

161
Jon Rust
A
Ashley PietersenNew Participant
asked in General
Yesterday started Cribl-Cloud (stream), now my Splunk License is chocking with data from it

Hi Anyone/supportYesterday, started Cribl-Cloud (stream), just for testing, now my Splunk License is chocking with data from it Crible Cloud :-(.Tried to delete all things I could delelte, still chocking my Splunk license. Deactivating my subscriiption says: We're sorry to see you go!We are processing the cancellation of your Cribl.Cloud organization. This process takes up to 30 days and you will get an email 7 days before the cancellation is final. If you didn't mean to cancel or having second thoughts, contact support@cribl.io.How can I interrupt this data-stream. I had to stop my Splunk Enterprise instance for this…Hope someone can help out.Ashley Pietersen

135
A
S
Steve BennettParticipating Frequently
asked in Stream
Full Read of S3 Bucket for only new items

I am trying to pull in DNS Logs from Cisco Umbrella. I have my S3 Connector configured to pull down the logs every 5 minutes. The problem is that it pulls down all the logs. I only want the new logs. I have tried adding the follow to my Conf section of the JSON configuration but it doesn't appear to work."trackProgress": true,      "trackMode": "file",      "trackField": "s3.object.etag",      "useLastModifiedTime": true

204
S
J
jaycriblNew Participant
asked in Stream
Having hard luck with parsing logs - tips?

Hi, I have gone through the cribl university user and admin courses and setting up the product is quite easy however am constantly stuck trying to parse or extract strings of data into fields so that we can search or use the information.For systems which send in JSON or CSV it is very simple, however most of ours are embedded Linux or network devices which give something like the following:<183> 02/06/2025:00:24:20 GMT NYDC1VPX01-DMZ 0-PPE-0 : default SSLLOG SSL_HANDSHAKE_SUCCESS 9838002 0 : SPCBId 33168208 - ClientIP 10.1.1.130 - ClientPort 57486 - VserverServiceIP 10.200.80.80 - VserverServicePort 443 - ClientVersion TLSv1.2 - CipherSuite "TLS1-AES-256-CBC-SHA" - Session New - HandshakeTime 47 msShowWhat combination of functions can split this out into fields such as hostname, clientip, clientport, etc.I have tried using ChatGPT to help and can get regex and grok expressions from the AI however these don't work when applying to cribl. I don't see many questions on this topic

114
Jon Rust
K
kiran kNew Participant
asked in Stream
Crowdstream - Bytes out size doubled

Hi EveryoneWe have integrated Azure Storage Account and Azure Application Gateway with CrowdStream (Cribl Stream) via EventHub. Since logs are in nested JSON format, we used unroll function to convert them into individual events before forwarding them to CrowdStrike NGSIEM.Currently, we are observing that Bytes Out size is twice that of Bytes In. While we understand that event count should double, our concern is why Bytes Out size is also doubling compared to Bytes In. Additionally, we have not observed any duplicate events in CrowdStrike NGSIEM.

79
Jon Rust
O
openwebspaceNew Participant
asked in General
Updating a lookup table with a pipeline function

I read that Cribl has the ability to update lookup tables using REST API and with a search string, but I'm looking for a way to do it during the processing stage. Is this possible? I see a Redis function and read something about that. If the values go there, is there a way to automatically lookup the results?

82
O
A
Arnold JohnsonNew Participant
asked in Stream
Cribl License

Hi everyoneI have different locations which need to have separate cribl stream installed on each location .. Which means standalone deployment. In this case how the license will be deployed .. Will it have separate enterprise license applied to each Cribl instance ?

402
A
A
Arnold JohnsonNew Participant
asked in Stream
Cribl leader and worker node on same VM

I have installed cribl standalone on a VM , but need to change it to leader ..Do I need a worker node or only a leader can do the processing work as a stream ?If worker node is required, can the same VM be a leader and worked node together ?

143
Jon Rust
R
RexNew Participant
asked in Stream
Having difficulty translating API header to Cribl

Hello,I'm new to Cribl stream and am having difficulty translating the API spec from the documentation. I can get all the curls working fine, but I don't know how they translate to cribl. For example, if I'm trying to use cribl for this type of curl:curl --location --request GET 'https://public-api.cloud.meater.com/v1/devices' \--header 'Authorization: Bearer <JWT>'What would my discover and collect headers look like to include the "Authorization: Bearer <JWT>?Cribl is authenticating correctly (I managed to figure that out), but all my requests result in a 400. My authorize expression is `Bearer ${token}`Any help/insight would be greatly appreciated.

53
Jon Rust

Badge winners

Show all badges
Terms & Conditions