Skip to main content

Products

Get answers, share tips, and explore best practices across the entire Cribl product suite.

  • 759 Topics
  • 0 Replies
759 Topics
I
Ihor SerohinNew Participant
asked in Stream
File Monitor UTF-16BE

Hi.I need to continuously read events from log file which is UTF-16 encoded. Is any way to convert collected data to UTF-8?Before that I used File System Collector which could use "Custom Command" option. Using "/usr/bin/iconv -f UTF16BE -t UTF8" solves the problem. But there is no such option for "File Monitor" source.

01
K
G
guifueNew Participant
asked in General
Is it possible to disable certificate validation when connecting a Splunk Search Collector?

Hello, we are triying to setup a Splunk Search collector, against an on-prem splunk with a self signed certificate. Is it possible to disable certificate validation?

02
G
B
Brian WassermanNew Participant
asked in General
Is replay actually a feature in itself or just a technique implemented via a Source with different

https://docs.cribl.io/stream/usecase-replay-s3/#which-format Starting in version 4.3, Cribl Stream supports replaying data that has been exported as Parquet, using either the S3 Collector or the Filesystem Collector.Meanwhile, the Azure Blob Storage and Google Cloud Storage Collectors support ingesting data in Parquet format, but do not support replay.I am glad to see that Parquet is now supported for S3 replay.  Is replay actually a feature in itself or just a technique implemented via a Source with different handling?How can Azure Blob and Google Cloud Storage collectors support ingesting but not support replay?

06
T
E
Erin SweeneyEmployee
asked in General
Looking for help with Filters and Eval Functions

There's something about filters in an eval function (and probably other functions as well) that I either don't understand or that don't work as advertised. I have some events, all of them already have a field state. In some of these events, the value is a straight number. In some events, the field value contains an epoch time and a value, delimited by a pipe, e.g. 1696254765000|5.333333333333333. To get that value, I though I'd just use an eval and filter on events that have a pipe, like seen in the screenshot. However, that seems to remove the field for all events that do not match the filter (events that have just a number in state). I created a state_copy field to illustrate this: the top event only has state_copy, it does not have a pipe in the field value, and the second event has both state and state_copy.I would have expected the filter on the eval function to leave the top event untouched.I know I can solve it e.g. with a regex extraction instead of split() in eval, so I'm not

02
E
T
Tony REmployee
asked in General
Is there a cribl equivalent to Splunk’s rex mode=sed?

Hi all,Is there a cribl equivalent to Splunk’s rex mode=sed? I’ve tried replace_regex but I kept hitting a brick wall.I’m trying to take a timestamp with an indeterminate number of spaces and replace those spaces with a single space in a pipeline (using ‘eval’ function)Data looks like this:timestamp: “2023-10-03     09:29:00”Currently I’ve done the really hacky fix of extracting both the date and time and smooshing them together (eval date + ’ ' time)Thanks!

01
D
M
Mike DupuisParticipating Frequently
asked in General
Using Persistent Queue drops thruput

Hi All, I am having some issues when enabling a Smart Persistent Queue for a Splunk TCP source (TLS disabled). When the persistent queue is enabled the thruput drops by 2 orders of magnitude (from ~20k eps to ~200eps). The Cribl worker nodes are running on VMs, could this be due to the storage not being fast enough? If this is the reason why would the thruput drop so dramatically? Does enabling persistent queues on a source use much CPU?I don’t have any other source persistent queues enabled currently, but do have some on destinations that are working fine.

01
B
S
Shawn CannonKnown Participant
asked in General
Using a load balancer with RAW HTTP source, what to use for a health check?

For a Raw HTTP source, if I put that behind a load balancer, what would I use for a health check?

02
X
B
benjamin.raderNew Participant
asked in General
Using "Webhook" Destination: Can we transform events to be used as HTTP requests?

Using "Webhook" Destination: Can we transform events to be used as HTTP requests?Background - we are trying to send alerts from the Cribl Internal (Logs) source.  We use a pipeline to reduce these events down to only Cribl errors and any Cribl Internal Logs that we care about.  However, we need a way to send this data in the correct format to something like  Slack "Incoming Webhooks."2 Problems with Sending Events as HTTP Requests Using "Webhook" Destination:Slack has a specific format and keys for the JSON body data -  https://api.slack.com/messaging/webhooks#advanced_message_formatting There are certain keys that may only appear in the JSON that Slack receives like "text".This is also problematic when we can't control if cribl_pipe is appended to the top-level fieldsThe JSON data must be individual events in JSON with "ContentType: application/json".This is problematic, it seems, because the events are batched using NDJSON, JSON array, or even with the custom option (image below) t

03
J
V
vuphamNew Participant
asked in General
Does anyone have an example of a working configuration to receive Palo Alto logs over HTTPS?

Does anyone have an example of a working configuration to receive Palo Alto logs over HTTPS?

03
T
D
Dinesh RajaNew Participant
asked in Stream
Collect and Send S3 logs via Cribl to Splunk

Hello All,I'm new to Cribl and basically a Splunk Admin & developer. Been working on Cribl migration project for a while.My requirement is to collect data from a S3 bucket Via Cribl and apply some cool stuffs than send it to Splunk for indexing.Now, I have established the connection with my S3 bucket from Cribl stream. I'm looking for some assistance to optimize & send data to Splunk.Can anyone guide me the steps & tricks for same?TIA.

01
T
T
Tony REmployee
asked in General
Anyone have a good set up for querying Azure AD?

Anyone have a good set up for querying Azure AD via say graph to access specific user object parameters? Looking to replace our current method, which brings in only about 6-8 fields out of 20 something available. Original message in Cribl Community Slack Message: https://cribl-community.slack.com/archives/C02A9EQV61J/p1680717534405409

01
T
J
Jason RehmNew Participant
asked in Stream
syslog output to ALB showing TCP endpoint errors

I'm working with our security team who's wanting me to send events to his AWS ALB syslog destination which has multiple syslog nodes behind it.He's getting events, but the cribl UI is consistantly showing the destination as experiencing issues. It's going through a loop of successfully establishing a connection, 1 minute later the sender disconnects it, 1 minute later it restablishesI do not experience this same thing when I use a non-loadbalanced syslog destination.Does anyone have thoughts on how to resolve or even troubleshoot this? Would using TLS help?

05
B
L
Laurent ZannettacciNew Participant
asked in Stream
Clibl_route field added but not configured in the system fields destination settings

Hello,I noticed yesterday that the cribl_route system field is always added to all events even if deactivated in the system fields post processing settings of my destination (Splunk).I am running a Stream cloud instance since more than a year and never had this issue before.Anyone else has the issue ?

01
T
G
Grant ForresterNew Participant
asked in Stream
Recommended Config for Persistent Queuing with Kubernetes Deployment?

Hello,We are exploring running a self-hosted Cribl deployment on Kubernetes (EKS+Fargate) using the provided Cribl Helm charts. We've got a leader and worker pod running. We are now looking at persistent queuing.My question is: what's the recommended configuration for ensuring no data loss in the case of a worker pod terminating with events still in the persistent queue?We've been exploring the use of extraVolumeMounts and persistent volumes backed backed by EFS. We found this technique in this blog post (https://cribl.io/blog/persistent-queueing-using-the-logstream-workergroup-chart/) but we see it is not recommended in the the docs (https://docs.cribl.io/stream/deploy-kubernetes#using-persistent-storage-for-persistent-queueing). We were wondering if there are better solutions that allow for a more automated recovery of events sitting in the persistent queue of the lost pod?

01
T
P
Pawel KwiatkowskiInspiring
asked in Stream
An import/export of code to a yml format

An export of code is done to a json fromat only while on servers is used a yml fromat.IMHO, it should be possible to do import/export also in yml format. :)

01
T
J
Joel YueNew Participant
asked in General
Cribl Stream Leader on Fargate, the container keeps running with unhealthy health check.

Hi I am trying to setup Cribl Stream leader on Fargate with EFS as a persistent storage for the leader using Cribl_Volume_Dir environment variable. The container keeps running with unhealthy health check. Anyone has a working task definition for me to refer to?

01
T
W
Warner ChengNew Participant
asked in Stream
Hi Cribl Streamer,

From the Data Routes, Apache has a yellow triangle. The Pipeline is Apache_Access.I tried other pipelines and it did not make a difference. Any pointer for me is appreciated.

03
W
J
Joel YueNew Participant
asked in General
Injesting sysmon logs via Elastic API and events are getting dropped

Hi, I am trying to ingest sysmon logs via the Elastic api. But i do not see any live data but instead i get all dropped counts. Can anyone help?

01
I
W
wcall_criblNew Participant
asked in General
Would my destination see the srcIP field as a string or as an IP field?

Quick question. If I have a field called like "srcIP" with a value of 1.2.3.4, and sent it to a system like Splunk or Elasticsearch, would it see the srcIP field as a string or as an IP field? Because I would want to ensure that there is a way for the values to be the right type and be converted if a change is needed.

09
I
W
Warner ChengNew Participant
asked in Stream
What are the steps to create worker group via CLI?

What are the steps to create worker group via CLI?

03
W
T
Tony REmployee
asked in General
How do I find out what IPs I should allow for Cribl.Cloud?

Our Splunk Cloud environment requires IP whitelists in order to access or ingest data. If we have a Cribl cloud org created, is there a way to get the IPs that would be required to whitelist in Splunk Cloud?

12
M
T
Tony REmployee
asked in General
Which TLS certs are used for syslog in Cribl Cloud?

In a cribl cloud deployment, what certificate do we use for TLS for syslog?

02
M
S
Shane DanielsEmployee
asked in Stream
How many worker groups can a Cribl Stream leader support?

How many worker groups can a Cribl Stream leader support?

01
B
J
JP BourgetKnown Participant
asked in General
Any reason my leader doesn't want to grab 4.03?

Any reason my leader doesn't want to grab 4.03?

02
B
J
Johan WogerInspiring
asked in Stream
How can I download a list of worker nodes in JSON format?

I have a big deployment with hundreds of worker nodes. Is there an easy way to download a full list of worker nodes in json format?

03
B

Badge winners

Show all badges
Terms & ConditionsCookie settings