Search

Cribl Search Masking Redaction Handling Inquiry

This message originated from Cribl Community Slack.Click here to view the original link.How do you handle masking/redacting in Cribl Search?

116

2 days ago

Kiwi169New Participant

Graph at What Time a Service Reached 1,000 Total Requests, or When a Service

This message originated from Cribl Community Slack.Click here to view the original link.Is there a way to do a cumulative sum on log events in Search? Or even with metrics? I'm not seeing anything mentioning cumulative in the stats functions in the docs. I would like to graph at what time a service reached 1,000 total requests, or when a service first had 100 unique clients IPs. I currently only have the events with this data, but if I need to convert these to metrics in Stream and send them over to Search as well I can, I'm just not sure Search supports cumulative queries

scott.bossi856Participating Frequently

Can Search Send Data Directly to HTTP Listeners or Through Stream

This message originated from Cribl Community Slack.Click here to view the original link.can search send data to http listeners, or does it have to go through stream?

156

scott.bossi856Participating Frequently

Enable Column Click to Change Search Table Sorting

This message originated from Cribl Community Slack.Click here to view the original link.Should I be able to click on a column in search table to change the sort on rows?

162

jp.lumsdaine917New Participant

How to See Concurrency Metrics? Almost Every Dashboard Panel Is Getting

This message originated from Cribl Community Slack.Click here to view the original link.How can I see concurrency metrics? Almost every dashboard panel is getting queued, but I don't see how many searches are actually being run at once so I dont know how much adjustment to usage groups I need to do

213

Jonathan RauNew Participant

Cannot POST /api/v1/search/jobs

In both the API Reference (with authenticated session) and via Python, I started running into an issue with submitting queries to the POST /search/jobs API to submit a remote query to Cribl Search.Despite the fact that it's a POST endpoint, and I'm using the headers generated by the API Reference, I receive the following error<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><title>Error</title></head><body><pre>Cannot POST /api/v1/search/jobs</pre></body></html>In the API Reference this is the payload I am POST'ing with application/json as the accept and Content-Type - the Auth comes from my session automatically{ "query": "cribl dataset=\"cribl_internal_logs\" | limit 10 "}In Python, I attempted to recreate it as well, I can confirm that the Client ID and Client Secret are valid and I receive

551

4 months ago

Norman MorrisNew Participant

Best way to repopulate S3 data into Splunk?

We are wanting to use Cribl to repopulate cloud trail logs from S3 into Splunk on-demand for review/audit/analysis purposes. Ideally, we would be able to request from within Splunk, but we could also query within Cribl to pull the data if necessary. Are there any best practices or use-cases that you can provide?

411

Jari WeststrateParticipating Frequently

Regex in Filter Expresion

Hi,Excuse me for my "not knowing", maybe a easy answer for all of you.I am in the process of migrating our Heavy Forwaders to Cribl . One off the inputs is for Cisco IOS devices. The props.conf has a Regex in it to change the sourcetype with help of a transforms. The props uses a regex to select the events.Where and how do i put the regex which is used in the props.conf for filtering the events. Do i have to place this regex in the "filter expresion" in the route? Or in a pre-pipeline? And how do i do it?Thanks in advancegreetingsJari

502

Mitchell FangNew Participant

Search Configuration help - CloudTrail S3 bucket path and search

I'd like to search a CloudTrail S3 bucket. What I want to search is "in all accounts" and only in "US-EAST-2" region for event name:"Assume Role". I'm not sure if I configured the Dataset correctly and/or how to do the search. Regarding the Dataset bucket path. This is the S3 folder layout: AWSLogs/subfolder/<ACCOUNT>/CloudTrail/<REGION>/YYYY/MM/DDIn the Dataset config, i'm not sure what to put in the bucket path. Especially for the <Account> and <Region> parts My guess is something like this AWSLogs/subfolder/${ACCOUNT}/CloudTrail/${REGION}/${_time:%Y}/${_time:%m}/${_time:%d}Regarding search: What I want to search is "in all accounts" and only in "US-EAST-2" region for event name:"Assume Role" When I did: dataset="CloudTrail_DATASET_NAME" | limit 1000 it returns no results. Not sure what I'm missingThanks

231

Abhishek ChristianNew Participant

Search Queries

How to search ipv4 range through Cribl search. What would be the syntaxI want to know whether we can search fields from raw logs or only parsed fields can be searched?

372

Santhosh MParticipating Frequently

How to skip the default parsing of JSON input to the HTTP(S) Cribl source in Streams?

Dear all,Coming up with yet another question as I couldn't find similar from the history of available requests. I have created the HTTP RAW Source and able to push the data and process it in the pipeline. However, I have observed that when I POST the valid JSON format(all the attributes with double quotes), Cribl http source is parsing them into individual fields after the source process, and if the posted content is not JSON parsable, then the content is available under the key "_raw".As per my requirement, I would like to keep the input JSON object as is, since I was trying to validate the whole json with the schema stored in the Knowledge. If it's not getting parsed then I was able to run my use case as expected. In other case where the JSON is parsed into individual fields, I couldn't find any internal key which holds all these variable to format them back to JSON object to validate against the schema.Hence, reaching out for any sugge

604

benjamin.raderNew Participant

Using JSON Paths

I'm attempting to extract values from a JSON string field. However, it seems, that none of the below methods are working for referencing or obtaining the value using a JSON path or dot notation type of approach. I've even tried the "extract_json" function, but to no avail.KQL in Azure has the bag_unpack function, but I noticed this is not supported in Cribl Search.Below are my queries and results.| extend json=parse_json(_raw)| extend test1=_raw._raw.kubernetes.container_name| extend test2=json._raw.kubernetes.container_name| extend test3=json['_raw']['kubernetes']['container_name']| extend test4=extract_json("$._raw.kubernetes.container_name", json, typeof(string))| limit 10

512

Santhosh MParticipating Frequently

How to access the Regexes saved in the Knowledge?

Hello all,I have recently started working with Cribl and my question may seem to be easier one, however, I couldn’t find an option to do this in right way and hence reaching out for help.In one of my use case I am getting a json from source and my requirement is to check whether the attributes in the object are matching with some of the existing Regexes in Crible knowledge section. I was expecting something like below,For ex: To access Schema, we use C.Schema(‘Schema Name’).Validate()In the same way can we do C.Regexes(‘Name of the regex’).test()Please advise on this with any suggestions. Thanks in advance!

607

Tony REmployee

Use of Project with nested fields

I have a field that is nested. I can search with the full name without an issue but when I try to use it with project, I do not get any values back. I have tried to wrap the field with single and double quotes. That results in the value for the field to be the name of the field.The goal is to get a list of all the different values of the payload.comment.commentID field. What do I need to do to use the payload.comment.commentID field? dataset="MyDataSet" sourcetype="MySourceType" payload.comment.commentID=* | project payload.comment.commentID

301

Thulasi KrishnanNew Participant

Will my Cribl Cloud URL always have a prefix of "main-" ?

My Cribl Cloud URL seems to prefix a main- when navigating to leader URL from the Cribl Cloud home page. Is this prefix standard for all URLs? Are there scenarios where this might be a different prefix?Original Question asked on https://cribl-community.slack.com/archives/C03HK1K33FA/p1685111839249629

191

Erin SweeneyEmployee

Is there any info on Search costs when searching AWS S3 buckets?

Is there any info on Search costs when searching AWS S3 buckets? Specifically on how data transfer works. For example, does Search hit S3 from US East and is $.01 per GB against my AWS account? Is Search caching at all? https://aws.amazon.com/s3/pricing/?p=pm&c=s3&z=4 Original post was from https://cribl-community.slack.com/archives/C03HK1K33FA/p1678133798883439

389

Tony REmployee

Most direct method of finding your completed scheduled searches?

Very excited about scheduled searches! Couple of questions:What’s the most direct method of finding your completed scheduled searches? Is it search->history->scheduled?I don’t see a way to see anything but the most recent runs of a search. Am I missing something?Original Message was from https://cribl-community.slack.com/archives/C03HK1K33FA/p1685111839249629

337

Abdullah ZubairNew Participant

Cribl Search to query S3 bucket - "Error initializing task queue"

Hi guys, we are trying to configure Cribl Search to query S3 bucket but have been facing error that says "Error initializing task queue" .... "Access Denied"We are seeing the Cribl Stream worker AssumeRole into our AWS account for the existing S3 Source Collector that we have configured previously but, we’re not seeing any Cloudtrail logs in our SIEM to show that the Cribl Search role (new role that we created in AWS to configure Cribl Search only) is able to AssumeRole into our account i.e we’re not seeing any STS logs in SIEM for Cribl Search role.Can anyone provide help on this?

463

Matthew JochymNew Participant

Any plans to make a more mobile friendly search page or even an app?

242

Tony REmployee

Can search read gzip files?

482

Johan WogerEmployee

Restoring git commit after reverting from remote repo

I was a bit careless and i pointed my stream deployment to a remote repo that had a bunch of old commits. Now my leader pulled down those old changes and they were deployed to my worker nodes. I reverted back from the remote repo but still have no access to my old commits which should still be living on the leaders local git. How do i fix this?

301

shemreader shemreaderNew Participant

victoriametrics destination

HiI am new to cribl and i am trying to collect data from kafka, aggregate it and send it to victoriametricsfor that I have configured Prometheus destination http://xx.xx.xx.xx:8480/insert/0/prometheus/api/v1/import/prometheuswhen I run test all is ok but I don't see data in victoriametricsany idea why ?

100

Aaron HoferNew Participant

How performant is Cribl Search when reading frozen Splunk buckets from S3?

I see it's possible for Cribl Search to read frozen Splunk buckets from S3 per this article below. Has anyone used it for reading Splunk frozen buckets, how fast are searches on multi-GB frozen buckets? Thanks https://community.cribl.io/kb/articles/30-search-splunk-backups-in-place-with-cribl-search?utm_source=community-search&utm_medium=organic-search&utm_term=splunk+frozen

280