Skip to main content

Splunk Universal Forwarders 10.4.0 Cannot Connect to Cribl Splunk TCP Source

  • Nov 20, 2026 9:51 PM UTC
  • 0 replies
  • 2 views

Jessica Bracken

Symptom

New Splunk Universal Forwarders running version 10.4.0 may fail to send data to a Cribl Splunk TCP Source even though older forwarders with an earlier version of Splunk using the same general deployment pattern continue working normally. In this example, data buffered locally on the forwarders and no events were received by Cribl from the affected hosts.

  • New Splunk Universal Forwarders do not deliver events to the target Cribl Splunk TCP Source.
  • Data buffers locally on the forwarder instead of being forwarded.
  • Existing forwarders on older Splunk versions continue to send successfully while newly deployed 10.4.0 forwarders fail.
  • Cribl shows repeated source-side connection closures and compression-related errors.
  • Splunk forwarder logs show connections opening and then immediately closing, followed by output queue backpressure or retry behavior.

Environment

  • Cribl Stream with a Splunk TCP Source configured with S2S v4
  • Splunk Universal Forwarders version 10.4.0 on the affected systems
  • TLS 1.3 in use during the forwarder-to-Cribl connection path

Example Log Indicators

Splunk Universal Forwarder

INFO AutoLoadBalancedConnectionStrategy - Connected to idx=xxx.xxx.xxx.xxx:9997 INFO AutoLoadBalancedConnectionStrategy - Connection to xxx.xxx.xxx.xxx:9997 closed. Connection closed by server. WARN AutoLoadBalancedConnectionStrategy - Applying quarantine to ip=xxx.xxx.xxx.xxx port=9997 WARN TailReader - Could not send data to output queue (parsingQueue), retrying...

Cribl Stream

"message":"closed connection" "error":{"message":"Compression is not enabled on this Source. Enable compression in Advanced Settings > Compression."}

Resolution

Set useClientSSLCompression = false in the global tcpout stanza of outputs.conf, then restart Splunk Universal Forwarder on the affected hosts.

[tcpout] useClientSSLCompression = false

Verification Steps

  1. Run the following command on an affected forwarder to confirm the active output configuration.
    splunk btool outputs list --debug
  2. Check whether useClientSSLCompression is set to true in the active configuration output.
  3. After changing the setting to false and restarting Splunk, review splunkd.log for a successful connection pattern where the S2S sender is created with compressed=0 and the forwarder remains connected to the Cribl worker.

Expected Result After Fix

After disabling client SSL compression, the forwarder establishes and maintains the connection to the Cribl Splunk TCP Source successfully, and data resumes flowing normally from the affected hosts.

Cause

The affected forwarders had useClientSSLCompression = true enabled in outputs.conf. In Splunk Universal Forwarder 10.4.0 with TLS 1.3, as per the standards of the protocol SSL compression is not supported. This results in the forwarder attempting a compression mode that Cribl rejects for the connection, causing the session to be closed before data can flow normally.

In Splunk version 10.2.0 this was not using TLSv1.3 it was using TLSv1.2 which supports TLS compression.

Debug logging on the forwarder confirmed the behavior with messages indicating that SSL compression was requested but not available with TLS 1.3, and that S2S sender creation reflected the compression state being attempted.

  • Log entries in Splunk and Cribl that illustrated the Splunk forwarder was attempting to connect to Cribl using TLS compression:

06-29-2026 10:48:29.592 +0100 DEBUG DistributedClient [347318 TcpOutEloop] - SSL compression requested but not available with TLS 1.3, using S2S compression instead 06-29-2026 10:48:29.592 +0100 DEBUG S2SSender [347318 TcpOutEloop] - Creating S2SSender: lwf=1, compressed=1, protocolLevel=6, channelLimit=300, channelReapInterval=60000, channelTTL=300000, channelReapLowater=10

 

{"time":"2026-06-23T16:02:00.704Z","cid":"w1","channel":"input:in_splunk_tcp","level":"error","message":"closed connection", "ioType":"source","ioName":"splunk","src":"xxx.xxx.xxx.xxx:10623", "error":{"message":"Compression is not enabled on this Source. Enable compression in Advanced Settings > Compression."

  • Log entry in Splunk that illustrated the Splunk forwarder was attempting to connect to Cribl with no TLS compression:

06-30-2026 09:04:11.181 +0100 DEBUG S2SSender [384649 TcpOutEloop] - Creating S2SSender: lwf=1, compressed=0, protocolLevel=6, channelLimit=300, channelReapInterval=60000, channelTTL=300000, channelReapLowater=10 06-30-2026 09:04:11.181 +0100 INFO AutoLoadBalancedConnectionStrategy [384649 TcpOutEloop] - Connected to idx=xx.xxx.xxx.xxx:xxxx:x, pset=0, reuse=0. autoBatch=1

Additional Information

  • To avoid impact to production ingestion during testing, validating with a separate Splunk TCP Source and port is a safe troubleshooting approach.
  • If deeper troubleshooting is needed, temporarily increasing Splunk logging to debug can help confirm whether compression negotiation is involved.
  • Data ingestion stops after upgrading heavy forwarder to version 10.4.x