Cribl Stream Service Fails to Start on Linux with OpenSSL openssl.cnf Permission Denied

Symptom

After upgrading Cribl Stream workers on Linux (for example, from 4.17.0 to 4.17.1), the Cribl service on one or more worker nodes fails to start under systemd.

When you run:

sudo systemctl status cribl -l

you see output similar to:

cribl.service - Systemd service file for Cribl Stream; Loaded: loaded (/etc/systemd/system/cribl.service; enabled; vendor preset: disabled); Drop-In: /etc/systemd/system/cribl.service.d; └─override.conf; Active: activating (auto-restart) (Result: exit-code) ...; Process: 26809 ExecStart=/opt/cribl/bin/cribl start (code=exited, status=13); Main PID: 26809 (code=exited, status=13)

and Cribl log or journal messages similar to:

/opt/cribl/bin/cribl: OpenSSL configuration error; 000E7964B67F0000:error:8000000D:system library:BIO_new_file:Permission denied:../deps/openssl/openssl/crypto/bio/bss_file.c:67:calling fopen(/etc/ssl/openssl.cnf, rb)

Additional observed symptoms:

• Affected worker nodes appear offline or missing from the Cribl Stream UI.
• Other worker nodes (with different OpenSSL configuration state) continue to run normally.

Environment

• Cribl Stream 4.17.x
• Deployment type: self-managed workers

Resolution

To resolve the issue, ensure that the OpenSSL configuration file used by Node.js/Cribl is present and readable by the Cribl service user (typically cribl).

1. Connect to an affected worker node using SSH.
2. Check the current Cribl service status and confirm the OpenSSL permission error: sudo systemctl status cribl -l
3. Inspect the OpenSSL configuration file path used in the error message (commonly /etc/ssl/openssl.cnf): ls -l /etc/ssl/openssl.cnf

   a. Note whether this path is:

   • a regular file, or
   • a symbolic link pointing to another file (for example, /etc/pki/tls/openssl.cnf).
4. If /etc/ssl/openssl.cnf is a symbolic link, inspect the target file: ls -l /etc/pki/tls/openssl.cnf
5. Verify that the file being opened by OpenSSL is not empty and is world-readable (mode 644 is usually sufficient): sudo stat /etc/ssl/openssl.cnf; sudo stat /etc/pki/tls/openssl.cnf
6. If the file is not readable by the Cribl service user (for example, permissions too restrictive), update the permissions so it is readable: sudo chmod 644 /etc/ssl/openssl.cnf

   If /etc/ssl/openssl.cnf is a symlink, you may need to adjust the permissions on the target, for example:

   sudo chmod 644 /etc/pki/tls/openssl.cnf
7. If an empty, zero-byte OpenSSL configuration file was recently created by a system script or cleanup job and is not actually required, work with your OS or security team to either remove the empty file or replace it with a valid configuration file and set appropriate permissions.
8. Restart the Cribl service: sudo systemctl restart cribl
9. Confirm that the service stays active: sudo systemctl status cribl -l
10. In the Cribl Stream UI, verify that the worker node is now online and reporting correctly in its worker group.
11. Repeat these steps on any additional worker nodes that exhibit the same OpenSSL permission error.

Cause

This can be caused by:

• An OpenSSL configuration file (for example, /etc/ssl/openssl.cnf or its target /etc/pki/tls/openssl.cnf) that exists on the system but is not readable by the Cribl service user.
• A system cleanup or hardening task that creates a new, empty /etc/ssl/openssl.cnf file on some servers or changes permissions on the OpenSSL configuration file, making it unreadable by non-root service users.
• Differences in OpenSSL configuration state across worker nodes (some with a symlinked configuration file, others with a regular file or different permissions), leading to only certain nodes failing to start after an upgrade.

When the file does not exist at all, Node.js or OpenSSL can proceed without loading it, and Cribl Stream starts normally. When the file exists but is not readable, Node.js or OpenSSL fails with a BIO_new_file: Permission denied error, which prevents the Cribl service from starting.

Additional Information

For additional background on similar OpenSSL configuration errors in Node.js, see the Node.js GitHub issue on BIO_new_file: Permission denied errors.