Hi @Dinesh Raja, what does the epoch timestamp in the filename represent? When the file was created? The first event timestamp? If you can provide this info, we can help with a solution.

Hello @Brendan Dalpe
Thanks for the response. Yes, the epoch timestamp represents "file creation time" .

@Dinesh Raja, you can use a path extractor in the Collector to set _time on the filename.

https://docs.cribl.io/stream/collectors-s3/#path-extractors

Something like this should work for you based on this path:

s3://BUCKETNAME/APPNAME/ENV/ak-913478-1701139960-008071-ds.gz

Path: /${appname}/${env}/${file}
Token: file
Extractor Expression: {"_time": value.match(/ak-\d+-(\d+)-\w+/)[1]}

This is making some assumptions that the file path is consistent, but should help get you going in the right direction. With _time set to the filename, the earliest/latest in your collection job has something to key off and won't need to grab all files. Hope this helps.