Hi! I was looking into using cribl for sending in data from amazon kinesis firehose into cribl. But there is no format for the data to be send mentioned in the documentation. And there is no event breaker support also. So what format should be followed? is it ndjson or is there anything I'm missing?
Question
For bringing Amazon Kinesis Firehose in to Cribl, what format should I use?
Hi <@U05ALUDRLD6>, is this the documentation you have seen?https://docs.cribl.io/stream/sources-kinesis-firehose/
Only this:```This Source supports gzip-compressed inbound data when the Content‚ÄëEncoding: gzip connection header is set.```
And, right at the beginning, it says that you should use Kinesis' HTTP endpoint destination option.Sorry, but I don't have experience with Kinesis Firehose. Is that information not adequate?
Our Amazon Kinesis Firehose Source expects HTTP inbound data.
For that range, I too will need to defer to a Cribl colleague how has more practical experience ingesting from Firehose.
Any format, pick http or http_raw sources, define event breakers if needed, done.
NDJSON is fine for that.
Sign up
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.