Stream

Does Stream support receiving Syslog that uses Octet-Counting Framing? For example, from the default configuration in Corelight?

Hi,IMHO, a group of functions or routes should contains a filter also.What you think?

A customer asked this: We have multiple syslog senders routing their messages to Cribl. Each of the senders has an event format that's specific to the source. And, we are routing those events to multiple destinations which also require their own format, such as CEF format for our security tool, JSON format for our logging tool.What's the best way to handle the processing of these events, so they're correctly processed and formatted correctly for the destination?Here is the example scenario:We have devices sending via syslog 514 ie (f5s, Cisco ASA, Cisco routers, cisco switches, storage gear, Palos, etc.) All the syslog messages would be different based on the vendor. Some (like the PAN) have a single device sending multiple different formats.We're doing filtering, enrichment, reformatting on the events. We're prioritizing this effort based on volumes.Once processed, events are sent to different destinations. Lets say we have 3: one requires JSON, one CEF, and one requires a single

Hello,Currently experiencing an issue when trying to configure the authentication section of the sentinel destination.Have been following the Cribl guides but it is not clear on what should be entered into the Login URL, OAuth Secret and Client ID.I am guessing the Login URL can be found from the App Registration in Azure and clicking on Endpoints?Client ID of the App Registration is not accepted in the Cribl UI.Any suggestions?Thanks :)

How do you reference a scheduled collector inside a route?

Can I remove the time stamp from Raw events and keep only the remaining message for raw?

I have a JSON file that I am ingesting into CRIBL. I have created a pipeline with the following functions “JSON Unroll” which is then followed by the “Parser” function with the following settings (Operation mode = extract; Type=JSON Object, Source Field = _raw).My single event now has a new field created called “records:” with what appears to be multiple arrays for each record (there are in excess of 250) with thisrecords:{}0:a account_id: xxxxxxxxxxa account_name: xxxxxxxxa owner: xxxxxxxxxa cost: xxxxxxxxxxa group: xxxxxxxxxxxxa reporter: xxxxxxxxxxxxx{}1:a account_id: yyyyyyyyyyya account_name: yyyyyyyyyyya owner: yyyyyyyyyyya cost: yyyyyyyyyyya group: yyyyyyyyyyya reporter: yyyyyyyyyyy{}2:a account_id: zzzzzzzzzzzzza account_name: zzzzzzzzzzzzza owner: zzzzzzzzzzzzza cost: zzzzzzzzzzzzza group: zzzzzzzzzzzzza reporter: zzzzzzzzzzzzzetc…I am running CRIBL version 3.5.1. I am trying to separate each of these records into an individual event. So in my example above {}0 with its 6 f

What the best way to convert a sample data file into a datagen?

In the https://docs.cribl.io/stream/sources-snmp-traps/ I don't see description regarding to the SNMPv3. So I have question if the Cribl support the SNMPv3 integrations?

Hi guys,I am new to Cribl and I was wondering whether there is an option in Cribl to store the data in a local storage.So, if there is a way, a clear step by step procedure with the prerequisites would be really helpful.Also, a little clarity on using the "replay function" along with this local storage would be nice.Thanks in advance.

gonna sound funny but a Popular vendor in our telecom world does support sending webhook notifications for alarms/alerts (but the problem it only sends to 80 / 443 / 8443 (no custom port) i dont think cribl cloud supports these ports (but is there a online proxy like ngrok or something that i can send/relay to cribl cloud (i dont wanna cloud ---> punch hole through FW to allow 443/8443 and then back to criblCloud at least in this simple POC approachOriginal Thread on https://cribl-community.slack.com/archives/C022FME74PM/p1695069411755299

Hello,In a Cribl Distributed Deployment, with one Worker Group, and 2 Worker Nodes in this Worker Group, with a Source Splunk TCP enabled, and TLS enabled from Leader Node, It is not possible to use a unique certificate and passphrase for each Worker Node.It is possible, to have each Worker Node in a different Worker Group. But that means all the settings must be the same (Sources, Destination, Routes, Pipelines). That means in the end that the use of Leader Node is useless.Is that correct?Can we improve that? Or make an enhancement?Thank you!Best regards,Vali

There is an Exec source. Is it possible to have a similar function as a destination, ie. execute a command?

I am seeing the below error messages:error: { [-]message: Unspecified GSS failure. Minor code may provide more information: No credentials cache found (filename: /tmp/krb5cc_176874)stack: Error: Unspecified GSS failure. Minor code may provide more information: No credentials cache found (filename: /tmp/krb5cc_176874)}level: errormessage: Authentication process failedIt seems the Kerberos tickets are being removed from the /tmp directory.  How do I change that?

Random question, and doesn’t really affect me or customer base atm, but is something that ALWAYS comes up with Splunk SaaS. When we say that that Cribl.Cloud instance is deployed within a VPC, is this VPC individual to the customer, or are they a tenant?Original Post is from https://cribl-community.slack.com/archives/C022FME74PM/p1664401586748939

Hello Cribl gurus - could you square https://cribl.io/blog/how-to-architect-your-logstream-to-logstream-data-flows/ with https://docs.cribl.io/stream/destinations-cribl-tcp/#reqs & https://docs.cribl.io/stream/destinations-cribl-http#reqs? We have an on-prem distributed deployment and a Cloud hybrid deployment, and we would like to forward data from one source from the on-prem workers to a hybrid Cloud deployment to continue our Cribl PS engagement, but it’s unclear if this is possible and if so how to achieve it. To be clear, our on-prem and cloud instances/deployments’ have different tokens. Please advise what options, if any, we have to connect the 2 instances together in terms of the on-prem source of interest.

Trying to implement a new data source but getting the error "Error 404-Not Found"Also getting an error saying "Failed to send anonymized telemetry metadata. Data flow has been stopped."

Hello,After upgrading my forwarders to the latest version of 9.1, I am no longer able to send to Cribl Stream. I am using the outputs.conf referenced here to send to my on-prem Stream instances:[tcpout]disabled = falsedefaultGroup = cribl[tcpout:cribl]server = STREAM_IP:PORTsendCookedData = trueforceTimebasedAutoLB = falsenegotiateProtocolLevel = 0However, no data is making it through. When I look in the Log section of my Source, I do see this error message:Any idea what I can do for this issue?

I have changed the auth token on my leader's distributed settings page and the workers in my worker group can no longer connect. what is the proper way to update the worker node's settings to reflect the new auth key?

Best way to extract fields from this data where the the field and value are separated by a : and the group is delimited by a , ?{"action": "update", "body": "New token value generated", "name": "user1", "timestamp": "2023-08-24T19:33:22+00:00", "userID": "1"}

Currently I'm ingesting Windows Classic and Windows Sysmon XML using Splunk UF, and apply corresponding packs to convert to JSON and some custom modifications. Until few days everything worked, but now we are receiving empty or broken messages.I'm running Splunk UFs 9.0.4/Splunk Cloud 9.0.2209.4 and Cribl 4.2.2.Config on UFs, as recommended in the docs. On Cribl Worker s2s is v4.[tcpout] disabled = false defaultGroup = cribl-worker-1[tcpout:cribl-worker-1] sendCookedData = true server=x.x.x.x:9997I noticed that Cribl defaults to fallback event breaker. No error/warnings in Cribl logs.

I can't get any of the data destinations to work. I have tried this with ElasticSearch, AWS S3 and Splunk. The message I get when I try a Test payload is "Error: 400-Bad Request Output … does not exist!"The logs don't have any information either.I'm running stream in a docker container. I got the same results on Cribl cloud instance as well.

Can Cribl Stream read AWS S3 tags? I know there are multiple places where you can add tags to parts of the stream, such as in the Source or in a Pipeline, but I want my stream to be able to read the tags that are assigned via S3 in order to filter them. If that's not possible, would I be able to call a Lambda function and use that to manage the tags instead?

We don't want our Cloud org to be removed, can that action be undone? We don't want to lose our configurations or any of the data being processed.

Hello, I’m trying to creation a S3 destination and trying to figure out how I can add index value to the file name.  How do I pass the index field from the source to the S3 destination file name?