This message originated from Cribl Community Slack.
Click here to view the original link.
In general... should I need to import the default Cribl certificate when attempting to connect one of our AWS EC2 Linux-based Splunk Heavy Forwarders to a Splunk TCP Source hosted in a Cribl Cloud's AWS workgroup? We've confirmed that we have connectivity using netcat, but when I configure the TA in question to send data to the Cribl workgroup (using the FQDN in outputs.conf), the IP addresses fail after 3 attempts and are quarantined.
I've reviewed the splunkd logs on the heavy forwarder, but they have yielded little so far.
Solved
Splunk Heavy Forwarder Fails To Connect To Cribl Cloud: IP Addresses Quarantined After 3 Attempts
Best answer by Jon Rust
Apologies, i thought you were asking about Splunk Fwd -> Splunk Cloud.
Yes, you'll need to get the certs from Splunk Cloud if you plan on using S2S. I'd strongly encourage you use HEC though.
https://docs.cribl.io/stream/securing-import-certs/
Splunk Cloud requires the client to present a valid cert when using S2S. HEC doesn't have that requirement.
Right
IIRC there should be a Forwarder output app you can download from the cloud instance / control plane that would have all the certs and configs
Uh, wait - do you mean a Cribl-created Forwarder output app? I don't see anything on our Cribl Cloud instance. Looks like I'll have to go the manual route.
Apologies, i thought you were asking about Splunk Fwd -> Splunk Cloud.
Yes, you'll need to get the certs from Splunk Cloud if you plan on using S2S. I'd strongly encourage you use HEC though.
https://docs.cribl.io/stream/securing-import-certs/
To clarify: I'm trying to route data from a Splunk Heavy Forwarder to a Cribl workgroup. Are you saying the preferred mechanism is to have the HF send to a Cribl HEC?
For Splunk -> Cribl use S2S
For Cribl -> Splunk use HEC
sorry for confusion
There won't be any special certs needed for Splunk -> Cribl tho. Install your cert on the receiving end (Cribl), and if needed, CA certs on the client side so it can validate. If you're using a well-known, public CA, you shouldn't even need that.
it should Just Work(tm)
I wish that were the case! This has been very frustrating. Even with a PEM file containing the Cribl default issuer, intermediate and server certs... I can't connect.
Here's outputs.conf stanza:
[tcpout:cribl_cloud_aws_saas]
server = correct worker group FQDN:9997
useSSL = true
compressed = false
sendCookedData = true
enableOldS2SProtocol = true
#clientCert = $SPLUNK_HOME/etc/apps/TA-QualysCloudPlatform/local/fa_cribl_cert_bundle.pem01-14-2026 09:33:31.957 -0800 ERROR TcpOutputFd [1256231 TcpOutEloop] - Connection to host=correct-server-ip:9997 failed. sock_error = 104. SSL Error = No errorI had to comment out the cert file because it generated this error - even though the file's contents are in fact formatted correctly with ------BEGIN .... etc.
01-14-2026 09:15:51.407 -0800 ERROR SSLCommon [1252267 indexerPipe] - Can't read key file /opt/splunk/etc/apps/TA-QualysCloudPlatform/local/fa_cribl_cert_bundle.pem SSL error code=151441516 message="error:0906D06C:PEM routines:PEM_read_bio:no start line"You don't need nor want a client cert UNLESS you require mutual TLS -- where the server side is also required to validate the connection
That's not normally needed
Server side says "here's a cert saying who i am"
client side, if validation is enabled: "okay, let me compare against my CA certs"
if validation is not enabled, client side just says "okie doke" and continues
In troubleshooting, try turning off validation. Does that work? Then proceed to turning it back on. If it fails then, you've got a CA cert issue. You'll need to tell splunk where to look to find the correct CA cert(s)
I will try turning off validation, but I did just replace the FQDN with the ingress IPs, and I have a new error:
- 01-14-2026 10:04:30.099 -0800 WARN AutoLoadBalancedConnectionStrategy [1262091 TcpOutEloop] - Cooked connection to ip=54.186.151.149:9997 timed out
Connection timeout is usually a network issue
on the client side, attempt connect from the command line using openssl and/or netcat (aka nc)
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.