Cribl Search v2 Does Not Decompress Extensionless Gzip Objects in Amazon S3

Symptom

Cribl Search v2 does not decompress gzip-compressed Amazon S3 objects whose object keys have no file extension, even when the object metadata lists ContentType: application/x-gzip.

Environment

• Cribl Search with a v2 Federated Dataset
• Gzip-compressed Amazon S3 objects with no file extension and ContentType: application/x-gzip

Resolution

Use a v1 dataset to read extensionless gzip objects. To use a v2 dataset, have the S3 object keys include a gzip file extension.

1. In Cribl Search, navigate to Data > Datasets.
2. Open the Amazon S3 Federated Dataset you want to use, or select Add Dataset.
3. Set Dataset type to Federated Dataset.
4. Set Dataset Provider to the Amazon S3 Dataset Provider.
5. Set Type to v1.
6. Configure Bucket path, Region, and Partitioning scheme for your S3 data.
7. Click Save.
8. Search the objects using the v1 dataset.
9. To use a v2 dataset, ask your Amazon S3 or object-storage administrator to copy the objects so their S3 object keys include a gzip file extension.
10. Search the copied objects using the v2 dataset.

Cause

This can be caused by:

• Cribl Search v2 relies on the file extension to detect compression. It does not decompress extensionless objects, even when the object metadata lists ContentType: application/x-gzip.

Additional Information

For Amazon S3 dataset setup details, see Connect Cribl Search to Amazon S3.