Skip to main content

Stream

Ask and answer questions about routing, shaping, and optimizing observability and security data.

  • 282 Topics
  • 601 Replies
282 Topics
J
jaycriblNew Participant
asked in Stream
Is there a list of links for cribl upgrade files?

Had a situation where the automatic upgrade was not working, so had to manually upgrade.From the logs it was trying to download ARM version not X64 version.Fortunately managed to figure out the correct weblink by changing the file name slightly: https://cdn.cribl.io/dl/4.10.1/cribl-4.10.1-45136dbb-linux-x64.tgz.Are these links listed anywhere on the cribl website, or how would we manually upgrade?

51
J
J
Jari WeststrateParticipating Frequently
asked in Stream
filter expresion and route order

Hello,I noticed a strange thing maybey anyone can explain.I have 3 syslog sources: syslog:514: ,syslog:51402: and syslog:51403:Three routes for these sources, each with a different pipeline attached. The order off the routes are 514 then 51402 and at last 51403.The filter in de routes are __inputId.startsWith('syslog:514:') &&  __srcIpPort.includes('10.10.1.112')__inputId.startsWith('syslog:51402:') && __srcIpPort.includes('10.10.1.112')__inputId.startsWith('syslog:51403:')&&  __srcIpPort.includes('10.10.1.112')(same host with different syslog outputs)In this order the 514 port gets a load off events, the 51402 lesser and teh 51403 nothing. That surpised me because when a check the source side i can capture 51403 events from this sourceip.For test i moved the 514 route after the 51403 route. Just a feeling. but what??.. the 51403 gets suddenly l the events it was missing. And here it gets strange. Just as if the 514 filter takes al the events from the 51403. But

36
J
A
Arnold JohnsonNew Participant
asked in Stream
Bootstrap worker from cribl cloud leader UI access

HiI setup a worker node on premise by running the script captured from cribl leader cloud. I enabled UI access for worker node. I can open the link in browser but what will be the credentials for it ?Thanks

12
A
A
Angelo Michele PizziNew Participant
asked in Stream
Distributed deployment planning with compressing g-zip Destination?

Hello everyone,I have the following problem. I'm designing a distributed deployment. I have to create an environment for data reduction but before that I need to send the data to a blob storage, for compliance reason, as you can imagine. Now, all of these is achieved but I have some doubts. When I use the Azure blob storage as only destination, all the 100% of the data goes in there but, the Monitoring Page, is telling me that the data that comes in are the same as the data that goes out (sometimes, the data that goes out are even more), even with compression enable. As said, I have the g-zip compression enabled on the Azure Destination page. I've checked the storage and I found out that the data are really compressed. So, my point is the following: when I try to estimates the possible number of workers and their capabilities, should I trust the monitoring page and consider DATA IN = DATA OUT (since the data is not processed by any pipeline at all) or I can consider the quantity of byt

03
J
S
Somnath BhattacharyaNew Participant
asked in Stream
Unable to perform Destination test with Splunk HEC

I am new to cribl and trying to setup Splunk HEC destination from cribl sanbox stream instance. I have provisioned a free splunk cloud instance and setup HEC. While I can send data via: curl -k "https://<host>:8088/services/collector" -H "Authorization: Splunk 38aa4a38-8fd5-4faa-afc4-9b3533ac39c7" -d "{"event": "Hello, world!", "sourcetype": "manual"}"I am unable to run destination test from cribl with the exact endpoint or with /services/collector/event. Everytime the request time out with error:2025-02-20T14:23:57.956αtime:2025-02-20T14:23:57.956Zαcid:w0αchannel:output:splunkαlevel:errorαmessage:error while flushingαioName:splunk_hecαioType:destination{}error:αmessage:request timeoutαstack:Error: request timeout at /opt/cribl/bin/cribl.js:15:9721630 at new Promise (<anonymous>) at p.doHttpRequest (/opt/cribl/bin/cribl.js:15:9721585) at runNextTicks (node:internal/process/task_queues:60:5) at process.processTimers (node:internal/timers:509:9) at async p.buildRecProm

01
J
S
Steve BennettParticipating Frequently
asked in Stream
Full Read of S3 Bucket for only new items

I am trying to pull in DNS Logs from Cisco Umbrella. I have my S3 Connector configured to pull down the logs every 5 minutes. The problem is that it pulls down all the logs. I only want the new logs. I have tried adding the follow to my Conf section of the JSON configuration but it doesn't appear to work."trackProgress": true,      "trackMode": "file",      "trackField": "s3.object.etag",      "useLastModifiedTime": true

14
S
J
jaycriblNew Participant
asked in Stream
Having hard luck with parsing logs - tips?

Hi, I have gone through the cribl university user and admin courses and setting up the product is quite easy however am constantly stuck trying to parse or extract strings of data into fields so that we can search or use the information.For systems which send in JSON or CSV it is very simple, however most of ours are embedded Linux or network devices which give something like the following:<183> 02/06/2025:00:24:20 GMT NYDC1VPX01-DMZ 0-PPE-0 : default SSLLOG SSL_HANDSHAKE_SUCCESS 9838002 0 : SPCBId 33168208 - ClientIP 10.1.1.130 - ClientPort 57486 - VserverServiceIP 10.200.80.80 - VserverServicePort 443 - ClientVersion TLSv1.2 - CipherSuite "TLS1-AES-256-CBC-SHA" - Session New - HandshakeTime 47 msShowWhat combination of functions can split this out into fields such as hostname, clientip, clientport, etc.I have tried using ChatGPT to help and can get regex and grok expressions from the AI however these don't work when applying to cribl. I don't see many questions on this topic

04
J
K
kiran kNew Participant
asked in Stream
Crowdstream - Bytes out size doubled

Hi EveryoneWe have integrated Azure Storage Account and Azure Application Gateway with CrowdStream (Cribl Stream) via EventHub. Since logs are in nested JSON format, we used unroll function to convert them into individual events before forwarding them to CrowdStrike NGSIEM.Currently, we are observing that Bytes Out size is twice that of Bytes In. While we understand that event count should double, our concern is why Bytes Out size is also doubling compared to Bytes In. Additionally, we have not observed any duplicate events in CrowdStrike NGSIEM.

09
J
A
Arnold JohnsonNew Participant
asked in Stream
Cribl License

Hi everyoneI have different locations which need to have separate cribl stream installed on each location .. Which means standalone deployment. In this case how the license will be deployed .. Will it have separate enterprise license applied to each Cribl instance ?

02
A
A
Arnold JohnsonNew Participant
asked in Stream
Cribl leader and worker node on same VM

I have installed cribl standalone on a VM , but need to change it to leader ..Do I need a worker node or only a leader can do the processing work as a stream ?If worker node is required, can the same VM be a leader and worked node together ?

03
J
R
RexNew Participant
asked in Stream
Having difficulty translating API header to Cribl

Hello,I'm new to Cribl stream and am having difficulty translating the API spec from the documentation. I can get all the curls working fine, but I don't know how they translate to cribl. For example, if I'm trying to use cribl for this type of curl:curl --location --request GET 'https://public-api.cloud.meater.com/v1/devices' \--header 'Authorization: Bearer <JWT>'What would my discover and collect headers look like to include the "Authorization: Bearer <JWT>?Cribl is authenticating correctly (I managed to figure that out), but all my requests result in a 400. My authorize expression is `Bearer ${token}`Any help/insight would be greatly appreciated.

03
J
J
jaycriblNew Participant
asked in Stream
Find which worker node processed an event

Hi,Is there a way to find out which worker node processed an event, does it inject a field or can we configure it?Reason is we have some issues and this could narrow it down to a particular worker node.Cheers,Jay

01
H
A
Allen McLeodNew Participant
asked in Stream
A working example of a script in Stream - Help me keep track of my goats

I am attempting to record which server logs are being collected, e.g. last seen, and I haven't found that capability yet. I am thinking I need something custom, maybe write to a file or a lookup. The cribl documentation doesn't provide much in the way of specifics when it comes to implementing scripts, except an ominous warning. I am new to the product and any help is greatly appreciated.

11
D
P
Pawel KwiatkowskiInspiring
asked in Stream
Redis. How to use a Redis query in a code function?

I would like define a Readis connection at the beginning of pipeline and reuse by a code functions through all an event live-time.Something like "C.Lookup" expression.PS: A Redis function is not applicable for my needs :).

03
B
F
Fredrik PalmqvistNew Participant
asked in Stream
Syslog parsing

Hi I am having some problem to get the host name out of some sample log files, the format is RCF 3164. And thru the documentation it says Cribl will try to parse that itself, do I need to decelerate that it syslog as a source?

01
J
J
Jari WeststrateParticipating Frequently
asked in Stream
host in hec events

L.s.,Maybe easy answer for all of you . We have got an HEC input and when i capture the live data i see as host the Cribl worker which is recieving the data. Why is that host filled?In the message itself there is also a host, but the right one. So i send the message in _raw to Splunk and delete the rest (also the wrong host). But tadaa.. there are two host in Splunk. The reciever Cribl and the one from the message.Any clue why?Thanks in advanceJari

03
J
J
Jari WeststrateParticipating Frequently
asked in Stream
filter expresion in route with wildcard

Hi,Maybe a simple answer (i hope). For a route we want to filer the host.name, but there are a lot of host in the list so a wildcard is the best way to filter.So doing like 'drnms10*.dmz.somewhere.nl' in the filter for the servers matching with this wildcard. But with a filter 'host == 'drnms10*.dmz.somewhere.nl'' wil not work. What is the best way to do this?I tried with a C.lookup and a lookup file.. it wil work. But stil have top maintain a list. And is a C.lookup not very slow in a filter?Thanks for the answer :)greetsJari

06
J
E
emmanuel.manu@edwardjones.comNew Participant
asked in Stream
Getting an error message:Validation error (WrongType@[events])

I am getting the following error in stream from REST Collection. message:Validation error (WrongType@[events]) : argument 'between.startTime' with value 'StringValue{value='`${C.Time.strftime(new Date((earliest * 1000.toISOString()}`'}' is not a valid 'DateTime'Here is the schema for GraphSQL and Collect POST body i am using. What am i doing wrongly? can any help with setting the correct time value.GraphSQLbetween: { startTime: "`${C.Time.strftime(new Date((earliest * 1000.toISOString()}`" endTime: "`${C.Time.strftime(new Date((latest * 1000.toISOString()}`"}Collect POST bodybetween: { startTime: \"`${C.Time.strftime(new Date((earliest * 1000.toISOString()}`\" endTime: \"`${C.Time.strftime(new Date((latest * 1000.toISOString()}`\" }

01
J
M
Martin ZivnyNew Participant
asked in Stream
Statistic data from Zabbix event streaming to mysql reporting server

Hi, i'd like know what is recommended method to save Zabbix event data (source) to mysql (destination). I understand that mysql is not typical and preffered destination but thats my use case. Thank you

01
J
S
Srinath DayalaKnown Participant
asked in Stream
we are removing some fields using Eval function but the bytes_in and bytes_out are still the same.

we are removing some fields using Eval function but the bytes_in and bytes_out are still the same. But, in the pipeline statistics we can see the bytes_out is reduced but it is not reflecting in the Monitoring → Flows.

03
J
S
Srinath DayalaKnown Participant
asked in Stream
what is the equivalent of Splunk should_linemerge in Cribl?

I am getting data into Cribl and it is by default breaking on each line (also when there is no timestamp). So, i have added manual event breaking based on timestamp. But it still the same behavior. Is there a way to disable line breaking on each line?

07
S
D
Dinesh RajaNew Participant
asked in Stream
Configure REST Collector to fetch SIEM events from Akamai

Akamai has the SIEM API which allows to capture security events generated on the Akamai platform. We have been getting these events to Splunk via Splunk Add-on earlier https://splunkbase.splunk.com/app/4310/ and decided to try Cribl REST collector. After giving collect URL's & required credentials in collector's config, I can't see the events from Akamai and getting 400 error. not sure what's getting missed. "type": "https://problems./-/pep-authn/request-error", "title": "Bad request", "status": 400, "detail": "Authorization header missing",

04
A
F
FeGNew Participant
asked in Stream
log and sourcetype reporting in splunk

I want to report on logs ingested with Cribl in the Splunk environment. The logs will remain stored on the Cribl side, but the reporting will be done in Splunk. How can I achieve this?The logs are NOT forwarded to Splunk.thank you in advance for your answers

01
R
N
nada28.7.1996@gmail.comNew Participant
asked in Stream
Split AZURE event Hub records into individual events

I have tried Event Breaker on this type of data but it didn't work, it can't separate the record into individual events note: this is the live data streamed from Azure event hub and this is the event breaker configuration but if I use the same pipeline or the event breaker at the knowledge with the data in that shape it works fine and splits every record into an event I tried to use unroll but also didn't work so any suggestion on how can i solve this issue?thanks,

010
Y
E
Ezell EdwardsNew Participant
asked in Stream
Has anyone had experience integrating Proofpoint logs into Cribl Stream? If so, could you share any

I been trying to find some formal documentation on bringing Proofpoint logs into Cribl steam and wonder if anyone had some experience or documentation they could share. At the time of this writing, there's not a cribl available for proofpoint logging.

01
J

Badge winners

Show all badges
Terms & ConditionsCookie settings