Cribl Stream Leader Does Not Listen on Port 4200 After Restart Due to Insufficient Memory

Symptom

After a Cribl Stream Leader restart or configuration change, port 4200 is not listening and Workers or Edge Nodes cannot reconnect to the Leader.

The Leader logs can include:

Shutdown:CB:Failed name: SocketRouter Error: Cannot read properties of undefined (reading '_connections')

The logs can also show RpcNoWorkersRegisteredError: No workers registered or out-of-memory errors in $CRIBL_HOME/log/cribl_stderr.log.

Environment

• Cribl Stream in a self-hosted distributed deployment
• A Cribl Stream Leader managing many Worker Groups, Fleet mappings, or Edge Nodes
• Leader memory that is insufficient for the resource demand during a cold start

Resolution

1. Verify whether the Leader is listening on port 4200:
   ss -tulpn | grep 4200
2. Review $CRIBL_HOME/log/cribl_stderr.log for out-of-memory errors.
3. Remove unused Worker Groups and fleets.
4. Consolidate temporary or duplicate Worker Groups.
5. Increase the Leader's memory allocation to match the managed deployment size.
6. Restart the Cribl Stream Leader service.
7. Verify that port 4200 is listening on the Leader.
8. Verify that Workers and Edge Nodes reconnect to the Leader.
9. Monitor Leader memory and CPU usage during startup.

If the Leader cannot remain available long enough to update the configuration in the UI, use the approved configuration-management procedure to remove unused groups from local/cribl/groups.yml before restarting the service.

Cause

This can be caused by:

• Insufficient Leader memory for the number of configured Worker Groups, Fleet mappings, and Edge Nodes.
• Excessive unused or temporary groups increasing cold-start process and configuration-load demand.
• A configuration change or restart forcing a cold start that exceeds the Leader's available resources.

Additional Information

• Port 4200 carries distributed Leader-to-Worker communication. If the Leader itself is not listening, investigate the Leader process and host resources before troubleshooting firewall connectivity.
• For sizing guidance, see Cribl Stream Sizing and Scaling.
• For port requirements, see Cribl Stream ports and protocols.