Skip to main content

Is there any info on Search costs when searching AWS S3 buckets?   Specifically on how data transfer works.  For example, does Search hit S3 from US East and is $.01 per GB against my AWS account?  Is Search caching at all? https://aws.amazon.com/s3/pricing/?p=pm&c=s3&z=4


Original post was from https://cribl-community.slack.com/archives/C03HK1K33FA/p1678133798883439

On the AWS side, you pay for API calls to the bucket. Assuming the bucket is in the same region as the Cribl.Cloud instance, there is no data transfer fee. Assuming you're using S3 standard storage tier, the API calls we would be making:
LIST: $0.005/1000 requests
GET: $0.0004/1000 requests


My region via the overview page says us-west-2.  Does this mean it is to my advantage to host my s3 in us-west-2?


Yes, otherwise you're paying extra money to search that bucket.


Okay, thanks.  Also, are there any logs showing the buckets and/or objects being hit from Search?  I only saw logs from stream, edge, etc.


S3 access logs should be visible in the Search Status page, and when searching the internal Search logs themselves


Use case you mention above,  if I am not searching, will I get charged api hits? consistantly?


you only get charged API hits when you actively run a search. There's no background or periodic updates that occur at this time.


This definitely helps with understanding how it works, thanks! 


we've actually added to the docs some information for clarity here: https://docs.cribl.io/search/set-up-s3#data-transfer-charges


Reply