General

I'm new to Cribl, so at this point we have single-instance Cribl Stream installed on premise. We just got a new SE, and he stated I need to move the Leader Node to the cloud as it's easier for him to troubleshoot. I have a cloud account created.I presume this is the distributed deployment, but the menus described in the documentation don't seem to align with what I see in the UI. https://docs.cribl.io/stream/deploy-distributed/ I'm not confident I have the correct software. Can Stream be converted into a worker node? Do I need the distributed deployment? Can anyone point me to a video that describes the process? I've looked on YouTube, but I've not had any luck.

Is there a way to move a worker to another group via GUI? And when not, why not?!

Hello, I would like to ask about Cribl software support policy. In other words - where I can find info about how long is particular Cribl release supported and about End of support dates? Is it for Cribl available something like this docu for Splunk - https://www.splunk.com/en_us/legal/splunk-software-support-policy.html? Thanks in advance for help. Best regards Lukas Mecir

In raw there is a field called origin="123456"I want to extract this from the events how can I do this?

Good morning Cribl folks! Does Cribl Stream handle native delivery to Azure Data Explorer (ADX / Kusto)?

Hi team, I need help with File Monitor source strange behaviour. Configured FM source (Cribl Stream single instance version 4.0.3) for one log file containing logs from shell backup script running once every day at 7PM. Backup script produces 6 lines/events during its operation. No advanced option used here, manual discovery with allowlist specifying log file name and Collect from end checked. Quick connect through simple pipeline adding metadata (index, sourcetype) and sending events to Splunk single instance destination. This setup works fine most of the time, 6 events read and sent to Splunk every day correctly. But from time to time it happens that at scheduled time not complete content of the log file is read and sent. The remaining lines from one day are then sent and ingested to Splunk together with the events from next run. It happens irregularly. I can not find any single reason for this, any idea? Original message in Cribl Community Slack Message:https://cribl-co

We are following this document, https://docs.cribl.io/stream/usecase-rest-ms-graph/ We have tested bringing in Azure User and Device data in the past with no issue. We are having a problem figuring out the best Collect URL to use. We've tried many different ones. We've tested our URLs to Microsoft's Graph Explorer and we are not receiving the expected output in Cribl. We have also set all the permissions that a Splunk App would require, https://docs.splunk.com/Documentation/AddOns/released/MSO365/ConfigureappinAzureAD

How would maintenance impact data processing, etc. (i.e. for scale-up procedure)?AMIs must be upgraded every X months; how does that impact the application (Stream)?Have you seen any auto-scaling ? How does scaling down impact the application? Will data be lost or will it complete its processing before shutting down?Concerned about what happens to data in flow when instances are downLooking for details on public cloud use casesFor the infrastructure scaling sideAfter disaster recovery, how do they switch back to the original state?

Is it possible for a member company to purchase a single license for the entire business group and share the license with the rest of the companies?

Has anyone experienced system metrics crippling the 4200 port? What are some best practices or documentation to combat this and just setting configurations for system metrics in general?

What's the best way to reference a Worker Group Level Lookup File inside a pack? Would it be: $CRIBL_HOME/groups/<groupname>/data/lookups/LookupFileName.csv ?

Does Cribl have any documentation for pulling Workday report logs?

does anyone know how to 'unflatten' fields in Cribl Stream? For example: source.ip is a flattened field that I need to be modified as json: '{ "source": { "ip": "whatever" } }' Original message in Cribl Community Slack Message:https://cribl-community.slack.com/archives/C02A9EQV61J/p1698617495203379

Hello, we are triying to setup a Splunk Search collector, against an on-prem splunk with a self signed certificate. Is it possible to disable certificate validation?

https://docs.cribl.io/stream/usecase-replay-s3/#which-format Starting in version 4.3, Cribl Stream supports replaying data that has been exported as Parquet, using either the S3 Collector or the Filesystem Collector.Meanwhile, the Azure Blob Storage and Google Cloud Storage Collectors support ingesting data in Parquet format, but do not support replay.I am glad to see that Parquet is now supported for S3 replay.  Is replay actually a feature in itself or just a technique implemented via a Source with different handling?How can Azure Blob and Google Cloud Storage collectors support ingesting but not support replay?

There's something about filters in an eval function (and probably other functions as well) that I either don't understand or that don't work as advertised. I have some events, all of them already have a field state. In some of these events, the value is a straight number. In some events, the field value contains an epoch time and a value, delimited by a pipe, e.g. 1696254765000|5.333333333333333. To get that value, I though I'd just use an eval and filter on events that have a pipe, like seen in the screenshot. However, that seems to remove the field for all events that do not match the filter (events that have just a number in state). I created a state_copy field to illustrate this: the top event only has state_copy, it does not have a pipe in the field value, and the second event has

Hi all,Is there a cribl equivalent to Splunk’s rex mode=sed? I’ve tried replace_regex but I kept hitting a brick wall.I’m trying to take a timestamp with an indeterminate number of spaces and replace those spaces with a single space in a pipeline (using ‘eval’ function)Data looks like this:timestamp: “2023-10-03     09:29:00”Currently I’ve done the really hacky fix of extracting both the date and time and smooshing them together (eval date + ’ ' time)Thanks!

Hi All, I am having some issues when enabling a Smart Persistent Queue for a Splunk TCP source (TLS disabled). When the persistent queue is enabled the thruput drops by 2 orders of magnitude (from ~20k eps to ~200eps). The Cribl worker nodes are running on VMs, could this be due to the storage not being fast enough? If this is the reason why would the thruput drop so dramatically? Does enabling persistent queues on a source use much CPU?I don’t have any other source persistent queues enabled currently, but do have some on destinations that are working fine.

For a Raw HTTP source, if I put that behind a load balancer, what would I use for a health check?

Using "Webhook" Destination: Can we transform events to be used as HTTP requests?Background - we are trying to send alerts from the Cribl Internal (Logs) source.  We use a pipeline to reduce these events down to only Cribl errors and any Cribl Internal Logs that we care about.  However, we need a way to send this data in the correct format to something like  Slack "Incoming Webhooks."2 Problems with Sending Events as HTTP Requests Using "Webhook" Destination:Slack has a specific format and keys for the JSON body data -  https://api.slack.com/messaging/webhooks#advanced_message_formatting There are certain keys that may only appear in the JSON that Slack receives like "text".This is also problematic when we can't control if cribl_pipe is appended to the top-level fieldsThe JSON data must be indi

Does anyone have an example of a working configuration to receive Palo Alto logs over HTTPS?

Anyone have a good set up for querying Azure AD via say graph to access specific user object parameters? Looking to replace our current method, which brings in only about 6-8 fields out of 20 something available. Original message in Cribl Community Slack Message: https://cribl-community.slack.com/archives/C02A9EQV61J/p1680717534405409

Hi I am trying to setup Cribl Stream leader on Fargate with EFS as a persistent storage for the leader using Cribl_Volume_Dir environment variable. The container keeps running with unhealthy health check. Anyone has a working task definition for me to refer to?

Hi, I am trying to ingest sysmon logs via the Elastic api. But i do not see any live data but instead i get all dropped counts. Can anyone help?

Quick question. If I have a field called like "srcIP" with a value of 1.2.3.4, and sent it to a system like Splunk or Elasticsearch, would it see the srcIP field as a string or as an IP field? Because I would want to ensure that there is a way for the values to be the right type and be converted if a change is needed.