Why does Cribl Stream Syslog Destination throttling allow an initial burst before enforcing the configured rate limit?

Question

Why does a Cribl Stream Syslog Destination with Throttling configured (for example, 100 KB/s) send a large burst of traffic for the first few minutes — significantly exceeding the configured rate  — before settling into a flat line at the throttle value? Is this a bug, and can I make the throttle behave as a strict instantaneous cap from the first byte?

Environment

• Cribl Stream
• Version: All Cribl Versions and Releases
• Destination: Syslog (TCP) Destination with Throttling (bytes per second) set to a non-zero value (for example, 100 KB/s).

Answer

This behavior is expected in the current implementation of Syslog Destination throttling. The throttle is implemented as a token-bucket rate limiter with a large default burst window (approximately one hour), not as a strict instantaneous cap from the first byte.

Key points:

• The Syslog Destination builds a token bucket with:
  bucketSize = (bucketSizeMultiplier ?? 3600) * rate
tokensPerInterval = rate
interval = 'second'

• bucketSizeMultiplier is not exposed in the Syslog Destination configuration, so it defaults to 3600 (one hour’s worth of tokens).

• The bucket starts full, which means at startup there is a reserve of ~1 hour of traffic at the configured rate that can be sent immediately before the limiter’s refill rate (the configured throttle) dominates.

For example, with a throttle of 100 KB/s (≈102,400 B/s):

• Bucket capacity ≈ 3600 × 102,400 ≈ 368.6 MB at startup.
• Over the first 3 minutes, the bucket also refills (≈180 × 102,400 ≈ 18.4 MB), for a total of ~387 MB available.
• In a real case, the destination saw ~382 MB over ~3 minutes (per-minute measurements like 3,233 KB/s and 2,630 KB/s), then a flat ~102 KB/s — matching the theoretical capacity almost exactly.

Because of this:

• The initial burst is by design and is not a defect in the Syslog Destination.
• The Throttling field should be understood as a sustained average rate with a large initial burst allowance, not as a strict cap from the first byte.
• Persistent Queue (PQ) / backpressure mode does not change the burst: it governs what happens once throttling and downstream backpressure are engaged (block, drop, or queue to disk), but it is independent of the token-bucket’s burst capacity.

Additional Information

Practical implications when configuring throttling:

• When you set a Syslog Destination throttle, plan for the destination to potentially receive up to roughly one hour of traffic at that rate as a startup burst after idle periods or replay jobs.
• If the downstream system cannot tolerate that burst, you may need to:
  • Lower the configured throttle rate to reduce burst size, and/or
  • Avoid starting large high-volume replays all at once against the same Syslog Destination.