Skip to main content

Cribl Stream Datagen Fails to Replace eventTime Timestamps

Jessica Bracken

Symptom

With Datagen timestamp extraction set to auto, Timestamp anchor unset, and Length at 150:

  • Events with eventTime within the first 150 characters show the timestamp template in _raw during Live Capture instead of the generated timestamp.
  • Events with eventTime beyond the first 150 characters retain the original sample data timestamp in eventTime.

The timestamp template should not remain in _raw. The event should either receive the generated timestamp or retain its original value when no match occurs:

"eventTime":"${timestamp:%Y-%m-%dT%H:%M:%S.%LZ}"

Environment

  • Cribl Stream Datagen template created from sample data containing eventTime.
  • Datagen timestamp extraction set to auto, Timestamp anchor unset, and Length 150.

Resolution

  1. Navigate to Data Routes or Pipelines.
  2. Select Import Data in the right Preview pane.
  3. Load the sample data.
  4. Select Create a Datagen File.
  5. On the resulting screen, set Timestamp type to auto.
  6. Enter \\"eventTime\\":\\" in Timestamp anchor.
  7. Leave Length at 150.
  8. Select Save as Datagen File.
  9. Navigate to Data > Sources.
  10. Open the existing Datagen source.
  11. Update Data generator file to use the newly created Datagen file.
  12. Select Save.
  13. Select Commit & Deploy.
  14. Run a live capture on the source.
  15. Verify in Live Capture that all events have the generated timestamp in eventTime and that the timestamp template is not present in _raw.

Cause

Cause: Not yet determined. The default Length of 150 does not reach eventTime when it falls beyond the first 150 characters. The case does not establish why timestamp replacement also failed for events with eventTime inside the scan window. Setting an explicit anchor resolved both behaviors.

Additional Information

Terms & ConditionsAccessibility statement