Skip to main content

Search

Ask and answer questions about searching telemetry data in place, without needing to move or index it.

  • 18 Topics
  • 46 Replies
18 Topics
N
Norman MorrisNew Participant
asked in Search
Best way to repopulate S3 data into Splunk?

We are wanting to use Cribl to repopulate cloud trail logs from S3 into Splunk on-demand for review/audit/analysis purposes. Ideally, we would be able to request from within Splunk, but we could also query within Cribl to pull the data if necessary. Are there any best practices or use-cases that you can provide?

01
J
J
Jari WeststrateParticipating Frequently
asked in Search
Regex in Filter Expresion

Hi,Excuse me for my "not knowing", maybe a easy answer for all of you.I am in the process of migrating our Heavy Forwaders to Cribl . One off the inputs is for Cisco IOS devices. The props.conf has a Regex in it to change the sourcetype with help of a transforms. The props uses a regex to select the events.Where and how do i put the regex which is used in the props.conf for filtering the events. Do i have to place this regex in the "filter expresion" in the route? Or in a pre-pipeline? And how do i do it?Thanks in advancegreetingsJari

12
J
M
Mitchell FangNew Participant
asked in Search
Search Configuration help - CloudTrail S3 bucket path and search

I'd like to search a CloudTrail S3 bucket. What I want to search is "in all accounts" and only in "US-EAST-2" region for event name:"Assume Role". I'm not sure if I configured the Dataset correctly and/or how to do the search. Regarding the Dataset bucket path. This is the S3 folder layout: AWSLogs/subfolder/<ACCOUNT>/CloudTrail/<REGION>/YYYY/MM/DDIn the Dataset config, i'm not sure what to put in the bucket path. Especially for the <Account> and <Region> parts My guess is something like this AWSLogs/subfolder/${ACCOUNT}/CloudTrail/${REGION}/${_time:%Y}/${_time:%m}/${_time:%d}Regarding search: What I want to search is "in all accounts" and only in "US-EAST-2" region for event name:"Assume Role" When I did: dataset="CloudTrail_DATASET_NAME" | limit 1000 it returns no results. Not sure what I'm missingThanks

01
J
A
Abhishek ChristianNew Participant
asked in Search
Search Queries

How to search ipv4 range through Cribl search. What would be the syntaxI want to know whether we can search fields from raw logs or only parsed fields can be searched?

02
J
S
Santhosh MParticipating Frequently
asked in Search
How to skip the default parsing of JSON input to the HTTP(S) Cribl source in Streams?

Dear all,Coming up with yet another question as I couldn't find similar from the history of available requests. I have created the HTTP RAW Source and able to push the data and process it in the pipeline. However, I have observed that when I POST the valid JSON format(all the attributes with double quotes), Cribl http source is parsing them into individual fields after the source process, and if the posted content is not JSON parsable, then the content is available under the key "_raw".As per my requirement, I would like to keep the input JSON object as is, since I was trying to validate the whole json with the schema stored in the Knowledge. If it's not getting parsed then I was able to run my use case as expected. In other case where the JSON is parsed into individual fields, I couldn't find any internal key which holds all these variable to format them back to JSON object to validate against the schema.Hence, reaching out for any suggestions in this. Thanks in advance!NOTE: Would be

04
S
B
benjamin.raderNew Participant
asked in Search
Using JSON Paths

I'm attempting to extract values from a JSON string field. However, it seems, that none of the below methods are working for referencing or obtaining the value using a JSON path or dot notation type of approach. I've even tried the "extract_json" function, but to no avail.KQL in Azure has the bag_unpack function, but I noticed this is not supported in Cribl Search.Below are my queries and results.| extend json=parse_json(_raw)| extend test1=_raw._raw.kubernetes.container_name| extend test2=json._raw.kubernetes.container_name| extend test3=json['_raw']['kubernetes']['container_name']| extend test4=extract_json("$._raw.kubernetes.container_name", json, typeof(string))| limit 10

02
B
S
Santhosh MParticipating Frequently
asked in Search
How to access the Regexes saved in the Knowledge?

Hello all,I have recently started working with Cribl and my question may seem to be easier one, however, I couldn’t find an option to do this in right way and hence reaching out for help.In one of my use case I am getting a json from source and my requirement is to check whether the attributes in the object are matching with some of the existing Regexes in Crible knowledge section. I was expecting something like below,For ex: To access Schema, we use C.Schema(‘Schema Name’).Validate()In the same way can we do C.Regexes(‘Name of the regex’).test()Please advise on this with any suggestions. Thanks in advance!

07
S
T
Tony REmployee
asked in Search
Use of Project with nested fields

I have a field that is nested. I can search with the full name without an issue but when I try to use it with project, I do not get any values back. I have tried to wrap the field with single and double quotes. That results in the value for the field to be the name of the field.The goal is to get a list of all the different values of the payload.comment.commentID field. What do I need to do to use the payload.comment.commentID field? dataset="MyDataSet" sourcetype="MySourceType" payload.comment.commentID=* | project payload.comment.commentID

01
D
T
Thulasi KrishnanNew Participant
asked in Search
Will my Cribl Cloud URL always have a prefix of "main-" ?

My Cribl Cloud URL seems to prefix a main-  when navigating to leader URL from the Cribl Cloud home page. Is this prefix standard for all URLs? Are there scenarios where this might be a different prefix?Original Question asked on https://cribl-community.slack.com/archives/C03HK1K33FA/p1685111839249629

01
J
E
Erin SweeneyEmployee
asked in Search
Is there any info on Search costs when searching AWS S3 buckets?

Is there any info on Search costs when searching AWS S3 buckets?   Specifically on how data transfer works.  For example, does Search hit S3 from US East and is $.01 per GB against my AWS account?  Is Search caching at all? https://aws.amazon.com/s3/pricing/?p=pm&c=s3&z=4 Original post was from https://cribl-community.slack.com/archives/C03HK1K33FA/p1678133798883439

29
D
T
Tony REmployee
asked in Search
Most direct method of finding your completed scheduled searches?

Very excited about scheduled searches!   Couple of questions:What’s the most direct method of finding your completed scheduled searches?  Is it search->history->scheduled?I don’t see a way to see anything but the most recent runs of a search.  Am I missing something?Original Message was from https://cribl-community.slack.com/archives/C03HK1K33FA/p1685111839249629

07
D
A
Abdullah ZubairNew Participant
asked in Search
Cribl Search to query S3 bucket - "Error initializing task queue"

Hi guys, we are trying to configure Cribl Search to query S3 bucket but have been facing error that says "Error initializing task queue" .... "Access Denied"We are seeing the Cribl Stream worker AssumeRole into our AWS account for the existing S3 Source Collector that we have configured previously but, we’re not seeing any Cloudtrail logs in our SIEM to show that the Cribl Search role (new role that we created in AWS to configure Cribl Search only) is able to AssumeRole into our account i.e we’re not seeing any STS logs in SIEM for Cribl Search role.Can anyone provide help on this?

03
A
M
Matthew JochymNew Participant
asked in Search
Any plans to make a more mobile friendly search page or even an app?

Any plans to make a more mobile friendly search page or even an app?

02
M
T
Tony REmployee
asked in Search
Can search read gzip files?

Can search read gzip files?

02
D
J
Johan WogerInspiring
asked in Search
Restoring git commit after reverting from remote repo

I was a bit careless and i pointed my stream deployment to a remote repo that had a bunch of old commits. Now my leader pulled down those old changes and they were deployed to my worker nodes. I reverted back from the remote repo but still have no access to my old commits which should still be living on the leaders local git. How do i fix this?

01
K
J
Jonathan RauNew Participant
asked in Search
Cannot POST /api/v1/search/jobs

In both the API Reference (with authenticated session) and via Python, I started running into an issue with submitting queries to the POST /search/jobs API to submit a remote query to Cribl Search.Despite the fact that it's a POST endpoint, and I'm using the headers generated by the API Reference, I receive the following error<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><title>Error</title></head><body><pre>Cannot POST /api/v1/search/jobs</pre></body></html>In the API Reference this is the payload I am POST'ing with application/json as the accept and Content-Type - the Auth comes from my session automatically{ "query": "cribl dataset=\"cribl_internal_logs\" | limit 10 "}In Python, I attempted to recreate it as well, I can confirm that the Client ID and Client Secret are valid and I receive a valid token. Here is the relevant portion of that script.def generateBearerToken() -> str: url = "https://

00
J
S
shemreader shemreaderNew Participant
asked in Search
victoriametrics destination

HiI am new to cribl and i am trying to collect data from kafka, aggregate it and send it to victoriametricsfor that I have configured Prometheus destination http://xx.xx.xx.xx:8480/insert/0/prometheus/api/v1/import/prometheuswhen I run test all is ok but I don't see data in victoriametricsany idea why ?

00
S
A
Aaron HoferNew Participant
asked in Search
How performant is Cribl Search when reading frozen Splunk buckets from S3?

I see it's possible for Cribl Search to read frozen Splunk buckets from S3 per this article below. Has anyone used it for reading Splunk frozen buckets, how fast are searches on multi-GB frozen buckets? Thanks https://community.cribl.io/kb/articles/30-search-splunk-backups-in-place-with-cribl-search?utm_source=community-search&utm_medium=organic-search&utm_term=splunk+frozen

10
A

Badge winners

Show all badges
Terms & ConditionsCookie settings