Stream

I am attempting to record which server logs are being collected, e.g. last seen, and I haven't found that capability yet. I am thinking I need something custom, maybe write to a file or a lookup. The cribl documentation doesn't provide much in the way of specifics when it comes to implementing scripts, except an ominous warning. I am new to the product and any help is greatly appreciated.

I would like define a Readis connection at the beginning of pipeline and reuse by a code functions through all an event live-time.Something like "C.Lookup" expression.PS: A Redis function is not applicable for my needs :).

Hi I am having some problem to get the host name out of some sample log files, the format is RCF 3164. And thru the documentation it says Cribl will try to parse that itself, do I need to decelerate that it syslog as a source?

L.s.,Maybe easy answer for all of you . We have got an HEC input and when i capture the live data i see as host the Cribl worker which is recieving the data. Why is that host filled?In the message itself there is also a host, but the right one. So i send the message in _raw to Splunk and delete the rest (also the wrong host). But tadaa.. there are two host in Splunk. The reciever Cribl and the one from the message.Any clue why?Thanks in advanceJari

Hi,Maybe a simple answer (i hope). For a route we want to filer the host.name, but there are a lot of host in the list so a wildcard is the best way to filter.So doing like 'drnms10*.dmz.somewhere.nl' in the filter for the servers matching with this wildcard. But with a filter 'host == 'drnms10*.dmz.somewhere.nl'' wil not work. What is the best way to do this?I tried with a C.lookup and a lookup file.. it wil work. But stil have top maintain a list. And is a C.lookup not very slow in a filter?Thanks for the answer :)greetsJari

I am getting the following error in stream from REST Collection. message:Validation error (WrongType@[events]) : argument 'between.startTime' with value 'StringValue{value='`${C.Time.strftime(new Date((earliest * 1000.toISOString()}`'}' is not a valid 'DateTime'Here is the schema for GraphSQL and Collect POST body i am using. What am i doing wrongly? can any help with setting the correct time value.GraphSQLbetween: { startTime: "`${C.Time.strftime(new Date((earliest * 1000.toISOString()}`" endTime: "`${C.Time.strftime(new Date((latest * 1000.toISOString()}`"}Collect POST bodybetween: { startTime: \"`${C.Time.strftime(new Date((earliest * 1000.toISOString()}`\" endTime: \"`${C.Time.strftime(new Date((latest * 1000.toISOString()}`\" }

Hi, i'd like know what is recommended method to save Zabbix event data (source) to mysql (destination). I understand that mysql is not typical and preffered destination but thats my use case. Thank you

we are removing some fields using Eval function but the bytes_in and bytes_out are still the same. But, in the pipeline statistics we can see the bytes_out is reduced but it is not reflecting in the Monitoring → Flows.

I am getting data into Cribl and it is by default breaking on each line (also when there is no timestamp). So, i have added manual event breaking based on timestamp. But it still the same behavior. Is there a way to disable line breaking on each line?

Akamai has the SIEM API which allows to capture security events generated on the Akamai platform. We have been getting these events to Splunk via Splunk Add-on earlier https://splunkbase.splunk.com/app/4310/ and decided to try Cribl REST collector. After giving collect URL's & required credentials in collector's config, I can't see the events from Akamai and getting 400 error. not sure what's getting missed. "type": "https://problems./-/pep-authn/request-error", "title": "Bad request", "status": 400, "detail": "Authorization header missing",

I want to report on logs ingested with Cribl in the Splunk environment. The logs will remain stored on the Cribl side, but the reporting will be done in Splunk. How can I achieve this?The logs are NOT forwarded to Splunk.thank you in advance for your answers

I have tried Event Breaker on this type of data but it didn't work, it can't separate the record into individual events note: this is the live data streamed from Azure event hub and this is the event breaker configuration but if I use the same pipeline or the event breaker at the knowledge with the data in that shape it works fine and splits every record into an event I tried to use unroll but also didn't work so any suggestion on how can i solve this issue?thanks,

I been trying to find some formal documentation on bringing Proofpoint logs into Cribl steam and wonder if anyone had some experience or documentation they could share. At the time of this writing, there's not a cribl available for proofpoint logging.

I have a need to send json formatted data to a beats (lumberjack) input. I am new to cribl and wondered if there was a pre-configured output that could be used to send lumberjack protocol based json to a receiver?My current SIEM solution has a json parser, but is using a Beats receiver as the only way to receive it.Thanks,

Hello, I would like to ask one thing: is it possible to use some database (SQL etc.) as a data source for lookup? One of our customers has a database that contains data that he would like to use to enrich events processed on Cribl Stream. The idea is that he would read the data from the database into Cribl Stream using a suitable connector, save it as a CSV file and then use that as the source for a lookup to use in the pipeline. Since I don't have experience with something like this yet, I would like to ask: Is there anyone who has experience with this? Is this a feasible way? And if so, is it suitable or are there better ones? What is there to be careful about performance, load on Cribl and the database, etc.? (Enrichment would involve about 200 events per second, the database is in the lower tens of thousands of rows, and the data in the lookup should be refreshed once per hour.) Any experience, ideas or recommendations would be greatly appreciated.Thank you very much.Lukas Mecir

Hi,I'm trying to setup a Splunk HEC within Cribl Stream, and I'm encountering the error "malformed HEC event." I've encountered that error when setting up HECs in Splunk, and to correct the problem I have to go to Source type > Select Source Type > Structured > _json when editing the HEC's settings. I've been looking and reading search results, but so far I haven't found the equivalent setting in Cribl Stream. Where can I find that setting for the HEC please?

We are planning to enable gitops workflow for production and from the docs we understood that the production environment will be “read only” if the gitops workflow is set to “push”. What if there is outage with “git” and we can’t do the release? What is the alternative to push the code without git?

How I can Parse Array of array in cribl stream. Each array element contains multiple strings.

I am trying to forward an exact duplicate of the syslog message. I have the route going thru passthru to not change the data. But I am getting the timestamp and original message host added to the message. Thoughts?Source: Syslog on 514/tcp/udp.Pipeline: passthruSource Data:<164>:Jul 23 12:34:15 CDT: %ASA-auth-4-987654: Group = 1234567890, Username = efghijk, IP = aaa.bbb.ccc.ddd, Session disconnected. Session Type: LAN-to-LAN, Duration: 0h:30m:29s, Bytes xmt: 0, Bytes rcv: 0, Reason: Idle TimeoutDestination Data:<164>Jul 23 12:34:15 lll.mmm.nnn.ppp :Jul 23 12:34:15 CDT: %ASA-auth-4-987654: Group = 1234567890, Username = efghijk, IP = aaa.bbb.ccc.ddd, Session disconnected. Session Type: LAN-to-LAN, Duration: 0h:30m:29s, Bytes xmt: 0, Bytes rcv: 0, Reason: Idle Timeout

The cribl service should not be run as root, some users have run into file ownership problems if they start the service as root and subsequently chown the cribl home directory as described in the docs. If the files in $CRIBL_HOME are owned by user cribl and the service still won't start, check that the files in /tmp/cribl_stream are also owned by the cribl user.If not, you can use this command to change the owner. You may need to use sudo to do this if your user does not have permission.chown -R cribl /tmp/cribl_stream

I have an application which is running on a Virtual Machine & this app is responsible for generating syslog messages.My goal is to forward these syslog messages to a source configured on Cribl Cloud.I went through the documentation, however I am not able to understand that how can I forward these messages.I know the idea is to send the messages to some IP address and Port, but where do I find this IP and port number?Also, while configuring the source on cribl, what is the purpose of IP and Port there? Do I need to point it to my Virtual Machine?

You know how as you get better at Cribl Pipelines, you refactor as you learn better ways to do stuff? What's the coolest thing you've figured out how to do? Include a screenshot if you can.

Hi, I have an API endpoint which require parameter to use a time string in the format of %Y-%m-%d %H:%M, which e.g. look something like this: https://host/api/v1.0/endpoint?start=2024-06-21 14:00&end=2024-06-21 15:00However when I put this on the URL, it always get converted to a value of 2024-06-21%2014:00 using example above. The API endpoint is unable to accept %20 and always ended up throwing error.Is there a way to avoid space being converted in Discover/Collect URI in REST API collector?

I'm getting 400 responses on a POST Body that I'm trying to pass through the state.latestTime within the POST body. I would like to know what is being posted.I don't seem to be able to get a debug log out of my REST Collector to see what it is POSTing. What it's deriving from state.latestTime.Anyone know where I can find this? The debug logs seem to only show the URL, not the POST Body

I have a REST API Collector which look at retrieving a number of records containing an ID which then need to be fed into a collection task as input for a 2-stage API call.While the Discover task has successfully retrieved the records with the ID, and I can see that in debug log the collection task has pick up the ID and insert correctly into Collect URI using variable, the tasks simply get cancelled afterwards.Does anyone have experience resolving similar issue before or have some idea what this might have gone wrong?