General

Hi Experts,i am doing some sizing calculation using  https://criblsizing.info/ what is the per node -2 referring to ?

Hello, I have my Cribl setup in GitOps mode (https://docs.cribl.io/stream/gitops/). I also have it setup to auth against Azure AD (https://docs.cribl.io/stream/usecase-azure-ad/). Further, I have defined several Azure AD groups including one that I have assigned the GitOps role to (https://docs.cribl.io/stream/gitops/#user-role). When I attempt to follow the instructions (https://docs.cribl.io/stream/gitops/) about obtaining a Bearer token and use an account that is only in Azure AD, I always get “Invalid user or password”. However

I’m really having a hard time with the GitOps sync. I’ve repeatedly followed the steps to generate a bearer token and am always getting “Forbidden” when I attempt to make the production leader sync. This has worked in the past.mkdir -p ~/.authcurl http://<Leader-URL-or-IP>:9000/api/v1/auth/login -H 'Content-Type: application/json' -d "{\"username\":\"<username>\",\"password\":\"<password>\"}" 2>/dev/null | jq -r .token > ~/.auth/tokenexport JWT_AUTH_TOKEN=`cat ~/.auth/token`export AUTH_HEAD="Authorization:Bearer `cat ~/.auth/token`"curl -X POST "http://<Leader-URL-or-IP>:9000/api/v1/version/sync" -H "accept: application/json" -H "${AUTH_HEAD}" -d "ref=prod&deploy=true"Any suggestions?

For a JSON log using the default Cribl JSON event breaker, do you know what the name is of the top most object?  For instance, _raw is an object, but what is the name of the object containing _raw?

In the monitoring section, how is Free Memory calculated per worker?

Hello,Currently testing the process of sending logs to the Chronicle destination.How do i retrieve the data from the "_raw" field.I came across a blog post that seemed to be relevant to my use case, where it mentioned using the log text field as "_raw."https://cribl.io/blog/google-chronicle-ingestion/However, despite following the instructions, I have not been successful in getting it to work.Appreciate any help.

Does anyone have a splunk universal forwarder config they typically use for forwarding?Original Question: https://cribl-community.slack.com/archives/CPYBPK65V/p1690293855973699Original Author: Matt

HiI have DB Input. Adhoc it is working fine, but not scheduled. I can see different in the logs in the fields filtered.But I do not see an filter settings. I do not see any filter in the scheduler.Original Message: https://cribl-community.slack.com/archives/CPYBPK65V/p1688712420207369

Is there any guide available for deploying Cribl standalone on a Azure Kubernetes Service?Original Question: https://cribl-community.slack.com/archives/CPYBPK65V/p1688623502905129

As a part of a cribl log processing pipeline, I want to add a function to remove any JSON keys that have empty strings or empty objects. I have a serialization function, such as this: - id: serde filter: "true" disabled: null conf: mode: reserialize type: json srcField: text fieldFilterExpr: value !=="" fields: [] remove: - .*but I don't want to reserialize, as it converts a JSON object during my pipeline into a JSON string. Is there another built-in function I can use to achieve this? Example input:{"my-log": {}, "my-other-log": "", "my-third-log": false} Expected output:{"my-third-log": false}

o365 message trace input: changed from basic auth to oauth and the input fails now with this error{ "time": "2023-04-10T23:00:12.531Z", "cid": "w1", "channel": "TaskExecutor", "level": "error", "message": "failed to execute task", "jobId": "1681167600.54.scheduled.CD_o365_msg_traces:MessageTrace", "taskId": "collect.1", "host": "hostname", "reason": { "message": "Non-whitespace before {[.\nLine: 3\nColumn: 1\nChar: 60", "stack": "Error: Non-whitespace before {[.\nLine: 3\nColumn: 1\nChar: 60\n at v (/opt/cribl_w/bin/cribl.js:14:9579357)\n at p._ [as write] (/opt/cribl_w/bin/cribl.js:14:9580223)\n at y.write (/opt/cribl_w/bin/cribl.js:14:17538090)\n at S._transform (/opt/cribl_w/bin/cribl.js:14:17539431)\n at S.Transform._write (node:internal/streams/transform:205:23)\n at writeOrBuffer (node:internal/streams/writable:391:12)\n at _write (node:internal/streams/writable:332:10)\n

does cirbl support MDT cisco yet Yang/netconf collection Original message in Cribl Community Slack Message:https://cribl-community.slack.com/archives/C02A9EQV61J/p1680730994732489

I am running a distributed environment, and when I use the API Reference to query an endpoint it returns Status Code 404. How can I resolve this?

Hello, this is probably simple, but I can’t get it to work. I just want to remove IPV6 encapsulation from the host field of events arriving at Cribl via syslog. So, existing host field is ::ffff:10.1.1.1, I want it to be just 10.1.1.1. For some reason I can’t get this to work, Splunk shows a null value for host when I try to EVAL, MASK, REGEX, etc. Anyone have a way to do this?

I am trying to send the count or average of certain fields instead of the whole events for a specific time period.

I’ve cloned user, and am now trying to add more GET polices. However, looks like ‘GET’ /system/something, ‘POST’ etc can’t be added as NewPolicy - only GroupFull etc available.Can this be done in the UI?or only using POST /system/roles?or do I need to post new Policy , POST /system/policies?

FYI, I was unable to sign up to the site via the normal account sign up.Never got any confirmation emails, tried multiple email accounts (they are were all hosted with google if that has a bearing on anything)Had to use “OpenID Connect” and sign up with Google

Has anyone connected up an AWS Opensearch Service instance to Cribl? I created a local backend user in my domain and gave it what I thought were good permissions but I still get a 401 error when I test the connection. Can anyone share what they did to get this to work?

I am trying to capture login attempts (successful/unsuccessful) to Cribl. I can see that the cribl.log file contains logs for these (with a type of “auth”).I have enabled CriblLogs as a source.The log level for channel “auth” is set to Info.When I do a capture on the CriblLogs source I don’t see these “auth” events, I do however see (some) other events - any suggestions as to what I might be doing wrong?Thanks.

I am currently bringing in a bunch of Apache logs. The sites are divided up in /var/log/httpd (eg. /var/log/httpd/site1/access.log, /var/log/httpd/site2/access.log). It would be amazing to set the base folder, the max depth, the allow list, and the host segment of the path. Is there a way to read in the path of the file to set the host field? Currently doing this in Splunk with the “host_segment” function in the inputs.conf file.

OK so I am trying to pull from the Microsoft Graph API. To do that, I must first POST to get a bearer token. I assume I can use discover type of HTTP Request to get it. A JSON response will be returned with the field access_token containing what I need.How do I use the data from access_token as a collect header for Authorization? This is a first for me so I need some hand holding here.Thanks!

I was previously using SPLUNK to get data via a SPLUNK add-on that used an API query to pull down detections from the CrowdStrike platform. We are moving away from SPLUNK and trying to use CRIBL to get CrowdStrike data. CrowdStrike have a SIEM connector but this requires additional infrastructure and I am trying to see if I can use CRIBL to query CrowdStrike via the API to pull this information down. Has anyone attempted or tried this? I was stuck with the client id and key with the REST API, but I think this needs to be done separately and only use an authentication token (bearer) to make the connection.

The Microsoft Windows Events pack is currently dropping "SourceName=AD FS Auditing" events from the Security logs. I found the two lines that are not properly filtering the events.Pipeline: Windows Classic Events, "Final Cleanup" lines 27 & 28, (Serialize & Eval).I turned it off, but still working to get the events to not drop and convert to json.Q: Wondering if this filter is currently being updated/corrected?

Hi all, this is my first post on this forum, so hello. We have just begun to use CRIBL and I am still in a learning phase. I am wondering if there are more parsers that can be added to the library or whether you have to create them yourself? As an example I have a CRIBL instance that is receiving json data from another CRIBL instance (with the data having been prebaked SPLUNK) for a Cisco ASA log. The content of the log entry is in either the raw or Message field and I would like to extract into key value pairs. I have tried the some functions but it looks like I have to do custom regex (and I am not good with regex).

Can someone assist in explaining the difference between setting the Cribl General TLS settings versus the distributed TLS settings and what server certificates need to be applied? Where do we need to put the server.pem certs versus the web certs? Currently I am using mutual authentication and each host has their own self-signed certificates.