Symptom
A Cribl Kafka Source connects to the bootstrap endpoint, joins the consumer group, and receives a partition assignment, but Events In remains at zero. Worker logs show SASL handshake failures or repeated reconnects when the consumer connects to partition-leader brokers.
Environment
- Cribl Stream
- Kafka Source using SASL/Kerberos authentication
- Kafka cluster with separate bootstrap and broker data listeners
Resolution
- Verify the broker addresses advertised to Kafka clients.
- Verify that each advertised broker listener is reachable from the Cribl worker.
- Verify the security protocol configured for the broker data listener.
- Verify the broker-side Kerberos service principal.
- Verify the client principal and keytab configured for the Cribl Source.
- Verify the configured Kerberos service name matches the broker principal.
- Verify that the expected SASL mechanism, such as GSSAPI, is enabled on the data listener.
- Verify that any proxy or intermediary passes the SASL exchange without altering it.
- Test authentication to a partition-leader broker after bootstrap succeeds.
- Verify non-zero Events In after broker authentication and fetch succeed.
Cause
This issue can occur when:
- The bootstrap endpoint is reachable while one or more advertised broker data endpoints are not.
- A broker listener advertises an incorrect hostname, port, or security protocol.
- The broker service principal or client keytab is stale or does not match the configured service name.
- The required SASL mechanism is disabled on the data listener.
- A proxy or intermediary changes or terminates the SASL exchange.
Additional Information
Successful bootstrap, group coordination, and partition assignment do not prove that the consumer can authenticate to the broker that leads each assigned partition. Confirm the direct broker fetch path separately.