Community Discussions

Is it possible to stream logs and events to Cribl Stream, send a copy of that event to S3 (or similar) for archive, and then process/parse/enrich the logs before sending them to another destination? The use cases I’m thinking of are as follows:(Primary Use Case) If a security incident occurs and we need to present evidence in court, how do we protect the forensic integrity of an event/log/etc. if the event is being changed in flight before going to an archive or analysis tool?If we needed to replay a series of logs for some reason or another, it might be helpful to do so from the source events, unmodified by Cribl.The downstream, long-term storage solution may also serve as a repository of data for other teams to use. Providing unmodified events for those teams may be preferred in that case.Thanks!

I am new to the distributed world, so please bear with me. I have a distributed setup with 2 worker groups(default & 2ndgroup). I have assigned nodes to each of the above 2 groups. In the leader ui when I navigate into the "2ndgroup" and make changes to my pipelines and commit and distribute the changes, it gets pushed to my git repo(remote) & the worker nodes.When I further navigate into the worker nodes and do a live capture of events and make changes to the pipelines, how do I push these changes to the git remote and further when I view it in the leader node under group view reflect the changes that were done at the node level? May be I am missing something obvious, can somebody shed some light on what I am not getting right in this flow?

Is there a way to set up a Destination as a POST request (w/authentication)? I tried using Code and modifying the code for DNS lookup on this page (https://cribl.io/blog/extending-cribl-building-custom-functions/). I got errors about using Promise.

I’m changing the ip on a couple of my workers - anything that I need to change or be aware of? My assumption is that Stream/Edge will just pickup the change and it will appear on the leader with those changes. Nothing else is changing.

Is there a URL or API endpoint I can use to check the status of a Stream Worker or Leader? If so, what HTTP Status codes does the endpoint return?

Hi all. I have Cribl replacing syslog-ng, but in order to listen on 514 I needed to run it as root. Is there a work-around to drop privileges or switch to user cribl after start-up?

Is there a way that we can search data in Devo using the API?

I am getting the following error in Splunk when sending an event from Stream:04-20-2023 04:21:09.000 +0000 INFO IndexWriter [7972 indexerPipe] - idx=goat: found event timestamped=1681964469123 is out of range. Using 1680307200.Here’s what _time looks like, in Stream, when viewing the data:{“_raw”: “”,“_time”: 1681964469123,“cribl_breaker”: “”,“index”: “”,“source”: “”,“sourcetype”: “”,“cribl_pipe”: “”}

I have a REST collector discovering events but not collecting them when scheduled. It works Ad-hoc, though.Any idea what could be causing this? I do not see errors in the logs.

What's the best way to move a sample data capture into a pack? Seems that exporting it isn't consistent.

This discussion has been moved.

I'm here to give you the 411 on new features of v.4.1.1 with updates to Stream, Edge, Search, and AppScope. Remember, this is just the new feature list, so, it is kind of a big deal. Stream (release notes):This release provides the following improvements: Introducing Stream Projects (Beta). This feature enables Cribl admins to create isolated spaces for teams and users to share and manage their data.New native MSK (Amazon Managed Streaming for Kafka) Source and Destination support accessing Kafka topics via IAM roles.The Mask Function now supports toggling individual Masking Rules on and off, to enable iterative development and debugging. It also now supports the SHA512 hash algorithm.The Redis Function now supports mutual TLS.Destinations' Post-Processing > System fields can now include cribl_route, identifying data's path through a given Route or QuickConnect.Cribl.Cloud users on all plans can now generate diagnostic bundles and send them to Cribl Support, directly from Stream's G

A week from today!  Sunil Kimar Gudivada will work through REST API integration for ServiceNow and Smartsheet.You know you don't want to miss that!

I’m using a code function in my pipeline for some extra custom logic, but exporting that as yml config is kinda ugly. Is there any chance I can load a .js file up and just tell the code function to run the js file? This would make local testing of my custom js a much nicer experience. I am doing this with samples for my test cases where I have the sample file locally and I load it up to the cribl server, and reference the file in my samples.yml. I’d love to do basically the same thing with a code function. Original message in Cribl Community Slack Message:https://cribl-community.slack.com/archives/C02A9EQV61J/p1683553724131069

It’s May, hi, 4.1.2 fixes the problem, it’s May (to the tune of TSwift’s Anti-hero ) Sorry everyone @here, we had to. It just fit so well. Even with  Cribl v.4.1.2 being a “maintenance release”, the team managed to squeeze in a few new features as well as a ton of fixes. AppScope 1.3.3 removes our support for two Go versions that the Go project no longer supports: 1.9 and 1.10. This makes the AppScope code less complicated and more robust. Related issue: #1452.AppScope 1.3.3 was also able to solve the following:1461 - In libc musl environments, attaching to an already-running but idle process now works normally1412 - Node.js apps compiled as Position Independent Executables (PIE) no longer seg fault when scoped.Edge now contains the Prometheus Edge Scraper source. Edge is edgier than ever before thanks to the new Prometheus Edge Scraper Source. In addition to the functionality already supported in the existing Prometheus Scraper, this Source is designed to work seamlessly in Kubernetes

Cribl is representing 413. Shout out to Springfield, MA! While we do love Massachusetts, today we are talking about the new release of Cribl v4.1.3.Downloads are now available at: https://cribl.io/download/ AppScopeRelease Notes: https://github.com/criblio/appscope/releases/tag/v1.3.4Fixes:1499 AppScope no longer exposes the backtrace symbol. Before this fix, certain scoped applications would segfault when they called backtrace.1491 When you run AppScope in a Kubernetes pod, the pod now starts normally even when the pod definition is missing the label section.1481 When scoping Java apps that use SSL and that are run on certain JREs, AppScope no longer causes the scoped app to segfault.EdgeRelease Notes: https://docs.cribl.io/edge/release-notes/release-v413/New Feature:A new native Amazon Security Lake Destination is now generally available.Corrections:CRIBL-17602 The Kubernetes Logs Source now continues reading log files after the container runtime rotates them.CRIBL-17907 The Kubernet

Thank you to @CriblNinja:A couple examples of real world use of Lookup Tables.https://medium.com/p/80c72a569e27Original Message: https://cribl-community.slack.com/archives/CPYBPK65V/p1687549304158039

I have a stream instance hosted locally and have several machines hosted externally. Is the ideal solution to install edge on all the external machines and then forward to an edge on one of the machines with a VPS connection that can then forward everything to stream?

I'm trying to assign a value to a field whose name is a value of another field. I expected __e[fieldName] to work, but it doesn't seem to. I get foo2 , but not foo1

As we know, we have the default route where there goes any data that was not taken by previous routes.Since that route has a filter true how can I find what data is going to that route? Capture is not helping of course because it has filtered trueAny other method except for negating all filters from previous routes?Original Question: https://cribl-community.slack.com/archives/CPYBPK65V/p1688625589301539

We just wanted to say thank you to everyone that has been asking and answering questions on Cribl Curious. Each month we will reward our top three contributors who have asked amazing questions and provide insightful answers that have helped fellow members. This month we even have a tie for 3rd place! 🏆 Top Contributors for the Month of June 🏆1st – Jeff G , 8 Points2nd – Bryan Rapp, 7 Points3rd  - Pawel Kwiatkowski, 4 Points3rd – Qian Zhao, 4 PointsTo express our gratitude for your contributions, we have a special gift for each of our top contributors. Please keep an eye on your inbox, as we'll be reaching out to you shortly with the details. You've earned it!You might be asking; how do I get points in Cribl Curious?Discussions:Posting a question1 pointGetting an UpGoat (like) reaction1 pointGetting an Awesome reaction2 pointsAnswer a question (first answer per question)1 pointYour answer being accepted3 points There are also Badges you can earn that award points. When you are awarded

Hi, Using Cribl TCP > Cribl TCP and on the destination side getting a “message:sender state change” with a “stage: STALE”. Also seeing messages of “reason: marked as stale”. What is causing this and is this something to be concerned about ? Found this digging around as on the Cribl TCP destination we are seeing the “Blocked status” on the charts tab showing as 1 however all servers are green under the status. The final destination is also Green and we cannot find any reason why the Blocked status is enabled. Original message in Cribl Community Slack Message: https://cribl-community.slack.com/archives/CPYBPK65V/p1683177474390609

Double checking on this, what is the expected behavior of Cribl if the Redis cluster is unavailable for some time? To my understanding events skip the Redis function (after default 30seconds) and processes through the rest of the pipeline. Is that how it is?

We had the first CriblCon on Monday. We have kept the party going with the CooLabs so if in Las Vegas, you can still stop by and get demos, talk to the experts, or just hang out. But we wanted to give you more. Today we are releasing Cribl v4.2.0 and AppScore v1.4.1! Below are some of the features I am personally excited for but make sure to check the release notes to learn more.Download Today!In version 4.2.0, we give you more control. Members and Permissions adds finer-grained access control and authorization, allowing you to assign access and capabilities to users independently at the Organization, Product, Group/Fleet, and resource levels.Stream:Release NotesStream Projects - We're proud to announce that Stream Projects is officially released! Now out of beta, Projects opens up Streams to new teams, users, and use cases by providing isolated workspaces (Projects) where users can build pipelines safely, securely, and collaboratively.We've added a new Microsoft Azure Sentinel Destina

Besides bragging rights, is there anything else I can do with my points? Funny you should ask. We are currently building the Community Redemption center. Once you earn a new level, you will be sent a code to redeem Community Swag. Please let us know what you would be wanting swag wise.Community T-Shirts?Sling Bags (like those found at CriblCon)?Tumblers?Stickers?Sweatshirts? Donations to a cause?Call with a Community Team Member / Product Team Member / one of the Co-Founders?