Stream

Is there a way that we can search data in Devo using the API?

I am getting the following error in Splunk when sending an event from Stream:04-20-2023 04:21:09.000 +0000 INFO IndexWriter [7972 indexerPipe] - idx=goat: found event timestamped=1681964469123 is out of range. Using 1680307200.Here’s what _time looks like, in Stream, when viewing the data:{“_raw”: “”,“_time”: 1681964469123,“cribl_breaker”: “”,“index”: “”,“source”: “”,“sourcetype”: “”,“cribl_pipe”: “”}

I have a REST collector discovering events but not collecting them when scheduled. It works Ad-hoc, though.Any idea what could be causing this? I do not see errors in the logs.

What's the best way to move a sample data capture into a pack? Seems that exporting it isn't consistent.

I have a stream instance hosted locally and have several machines hosted externally. Is the ideal solution to install edge on all the external machines and then forward to an edge on one of the machines with a VPS connection that can then forward everything to stream?

As we know, we have the default route where there goes any data that was not taken by previous routes.Since that route has a filter true how can I find what data is going to that route? Capture is not helping of course because it has filtered trueAny other method except for negating all filters from previous routes?Original Question: https://cribl-community.slack.com/archives/CPYBPK65V/p1688625589301539

Double checking on this, what is the expected behavior of Cribl if the Redis cluster is unavailable for some time? To my understanding events skip the Redis function (after default 30seconds) and processes through the rest of the pipeline. Is that how it is?

How do I write a database query for a DB Collector that uses state-tracking for MS SQL?

Hi,I recently installed Cribl and today when I tried to log back in I couldn't, I'm locked out! I have CLI access so I was hoping there was a way to reset my password through the CLI?Thank you!

So, this question has been bothering me for quite some time now. While I am a big fan of Cribl and I really enjoy working with their products and showing/explaining them to others I still wonder every now and then what value Stream would provide to a customer, who already has a well-maintained and functioning logstash for routing/transforming data.If I think about it the following points come to my mind, but if someone here has more/different reasons I would be glad to hear them!Stream's replay functionScalabilityLogstash works as a single instance, Cribl can be clustered to infinityStream is easier to maintainNo grepping around in config files for that one transformation command you need to changePipelines are easier to understand/maintain than logstash filesLess complex to get started with for new users/adminsVisualization of data flows"Debuggability" Being able to

When we check monitoring/sources we see metrics for the average thruput for both Events Per Second and Bytes Per Second for the sourcesHowever when we check monitoring/routes we see metrics for the average thruput for only Events Per Second and and NOT Bytes Per Second for the sources

Here at Cribl we are committed to making our customers' lives easier. One of the requests we had was how we can quickly ship new versions of our products into the Amazon Marketplace for customers who cannot use our SaaS offering Cribl.Cloud. Today we are announcing the availability of Cribl Amazon Machine Images or AMI’s for both x86_64 and ARM64  platforms. These two AMI’s can be used as part of a Free Tier for up-to 1TB/data per day. Customers can also elect to bring their own license and convert these images to paid Enterprise or Standard versions. This would also work for deploying hybrid Cribl worker nodes in our customer’s own AWS environment.The CloudFormation templates will still be available via our GitHub repo https://github.com/criblio/cribl-aws-cloudformation-templates .  Navigate to the AWS Marketplace and select the AMI archi

GOAT Mind, I am using a packet broker to aggregate my syslog data, then push it to Cribl Stream, which is running on Red Hat 9. I have had to spoof the MAC to the NIC and give it the accepting IP address that the Packet Broker is expecting to receive the data. Next I have had to modify kernel parameters so the data can be seen at the NIC.I can see the data at the NIC via tcpdump and ngrep. However, nether of the Crible Stream Sources Syslog or UDP are able to pick up the data, whether I use the specific IP for the NIC or leave it wide open (0.0.0.0).I have gone as far as SELinux running in permissive mode and disabled the firewalld, all with no luck with the source picking up data.Has anyone run into this issue and if so, how did they solve it?Scott

IMO, the Cribl logging need improvements.While in logs can see errors it is hard find which a pipeline, function causing it.My suggestion is to add infromation to each ERROR line about a pipeline and fuction releated to an error.

Hi All, i am facing an issue in Cribl data script, Actually the job is completed based on the logs, but it stuck saying 2 tasks inflight,We are using script option to pull the data, We have handled all the scenarios to prevent the issues, but after sometime, the job is getting stuck, As we can see in the above screenshot, totally 55 tasks( 1 discovery + 54 collector), and it says, 53 completed. But when I downloaded the logs where condition is ""message": "collect task completed","  based on this log count, inclusive of 0 to 53 totally 54. and all of them is completed but why the job didnt stopped.

I am confused about the new Cribl permissions system, I am in the process of configuring some roles for a local user, I am very happy with the fine grained controls available, especially allowing me to control individual endpoint access, the new permissions system appears to be much more coarse and losing a lot of functionality available in roles? Seems to be a "dumbing down" of the permissions system that takes away a lot of control?Hope I am misunderstanding this.Thanks for your help

The default format of exporting pipelines is the JSON. As the YML is used in config files, IMO, also exporting to this format should be possible.

Hi Team,I am a Cribl Certified Engineer and have been working on a use case which is described below, but I haven't achieved the expected output.The use case involves an S3/Blob Storage where we have one bucket containing 2-3 folders. The source should be a specific folder, and from that folder, I need to ingest data to the required destination.In S3/Blob, I can see that regex can be used for matching file names to download and process, with the default being .*. Can I change it to /foldername/.*? Will this work? If yes, can I have a whitepaper or some documentation on this?

The question is focusing more towards your smaller companies, what are the usual problems your smaller customers present for first time setups?

Can i read azure event hub tags in cribl during the preprocess pipeline

I wanted to ansiblize creating new cribl instances and have them fully up and functional/ready to login however this is my first barrier to entry. I want to untar cribl to /opt/cribl, run my playbook and then when I login it goes straight to the normal logged in page as I already accepted the agreement. Similar to Splunk's —accept-license

I am a long time Cribl & Splunk user, I have been on this platform for almost 5+ years now, and I have made my share of stupid stupid mistakes but learnt a lot about both Cribl & Splunk. In my journey to build a more resilient Cribl + Splunk environment with the constraints I do have($$ + time), I am constantly trying to make things a bit better and I introduced a change in my Splunk UF deployment. I set up some routing entries in my output.conf so that all index=_internal traffic is shipped directly to the Splunk IDX. The thinking was this allowed me to monitor my SplunkUF's and get ahead of any problems the Cribl workers may have in processing the events that we really care for. This change introduced some unintentional side effects which lead me to understand sourcePQ(sPQ) much better. My nodes ships out metrics periodically 24/7 and events on a much more systematic basis. Some of the nodes get very chatty during certain hours of the day and totally ra

Has anyone used a Transparent proxy when using Hybrid workers with CRIBL.Cloud ?We are having a weird issue where the workers work and communicate to CRIBL.Cloud Leader as expected but when the CRIBL.Cloud leader gets rebooted after an upgrade all the worker nodes get this error when trying to talk to the CRIBL.Cloud leader:Connection error: Client network socket disconnected before secure TLS connection was established

what's an alternate input plugin for logs received from beats