Skip to main content
Question

Does Collect has a checkpoint option?

  • March 11, 2025
  • 1 reply
  • 54 views
M

Hello!I am using HTTP Discover and Collect with Login Authentication, to take the notifications.log.Then I pass them to a route and send them to Splunk.Question: Does Collect has a checkpoint option? Currently, every run on collect, gets the entire file every time, so getting duplicates.

    1 reply

    T

    the REST Collector does not have a checkpoint option. when you say notifications.log, are you referring to an endpoint/API or a file? there are some options to not collect duplicates with either endpoint or file collection, such as <https://docs.cribl.io/stream/collectors-schedule-run#about-partitions-and-tokens|leveraging Partitions and Tokens> if you're able to separate files by date for example

    Terms & ConditionsAccessibility statement