Skip to main content

Why do we see this message when using Cribl v4.4.0 with S2S V4 and Splunk forwarder version 9.1.x:

{
"time": "2023-12-19T15:16:15.549Z",
"cid": "w0",
"channel": "input:in_splunk_tcp",
"level": "error",
"message": "closed connection",
"src": "10.10.10.10:62453",
"error": {
"message": "Unsupported op-code 249.",
"stack": "Error: Unsupported op-code 249.\n at g._processData (/opt/cribl/bin/cribl.js:15:15721335)\n at g._transform (/opt/cribl/bin/cribl.js:15:15717080)\n at Transform._write (node:internal/streams/transform:175:8)\n at writeOrBuffer (node:internal/streams/writable:392:12)\n at _write (node:internal/streams/writable:333:10)\n at Writable.write (node:internal/streams/writable:337:10)\n at TLSSocket.ondata (node:internal/streams/readable:766:22)\n at TLSSocket.emit (node:events:513:28)\n at TLSSocket.emit (node:domain:489:12)\n at TLSSocket.<anonymous> (/opt/cribl/bin/cribl.js:15:12649570)\n at addChunk (node:internal/streams/readable:324:12)\n at readableAddChunk (node:internal/streams/readable:297:9)\n at Readable.push (node:internal/streams/readable:234:10)\n at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)"
},
"r": 410398909,
"b": 1374112951
}
{
"time": "2023-12-19T15:16:15.546Z",
"cid": "w0",
"channel": "input:in_splunk_tcp",
"level": "error",
"message": "Failed to parse S2S payload",
"src": "10.10.10.10:62453",
"error": {
"message": "Unsupported op-code 249.",
"stack": "Error: Unsupported op-code 249.\n at g._processData (/opt/cribl/bin/cribl.js:15:15721335)\n at g._transform (/opt/cribl/bin/cribl.js:15:15717080)\n at Transform._write (node:internal/streams/transform:175:8)\n at writeOrBuffer (node:internal/streams/writable:392:12)\n at _write (node:internal/streams/writable:333:10)\n at Writable.write (node:internal/streams/writable:337:10)\n at TLSSocket.ondata (node:internal/streams/readable:766:22)\n at TLSSocket.emit (node:events:513:28)\n at TLSSocket.emit (node:domain:489:12)\n at TLSSocket.<anonymous> (/opt/cribl/bin/cribl.js:15:12649570)\n at addChunk (node:internal/streams/readable:324:12)\n at readableAddChunk (node:internal/streams/readable:297:9)\n at Readable.push (node:internal/streams/readable:234:10)\n at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)"
}
}

An "Unsupported op-code 249." is seen when you have "compressed = true" in the outputs.conf on your Splunk Forwarders, with Splunk version 9.1.x and using S2S version 4 both on your Cribl source and Splnuk forwarder together with SSL communications from your Splunk forwarders to the Cribl workers.

As per the Splunk documentation:
compressed = <boolean>
* Whether or not forwarders and receivers communicate with one another in compressed format.
* A value of "true" means the receiver communicates with the forwarder in compressed format.
* If set to "true", you do not need to set the 'compressed' setting to "true" in the inputs.conf file on the receiver for compression of data to occur.
* This setting applies to non-SSL forwarding only. For SSL forwarding, Splunk software uses the 'useClientSSLCompression' setting.
* Default: false

Setting "compressed = true" in the outputs.conf, if you are using SSL to send from your Splunk forwarders this not required as SSL compression is already being used.

This was not an issue in S2S version 3 as version 3 was not checking for this parameter being set but S2S version 4 does.

Further testing from our QA team revealed that having the setting of "compressed = true" while using S2S version 3 and SSL to send from your Splunk forwarders actually has a negative performance impact on amount of throughput that can be achieved, in addition to this they also confirmed that disabling this setting does not increase the amount of data on the actual wire neither.

If you are sending from Splunk forwarders directly to Splunk indexers then the Splunk indexers will accept the data formatted this way, Cribl workers will not and cannot accept the data formatted this way.

If you are using SSL + Splunk 9.1.x + S2S v4 the fix is to set "compressed = false" in the outputs.conf as per the Splunk docs.

We have raised this with our dev team to investigate if we can just flag the issue in our logs and still permit the data through for processing rather than dropping the data from the forwarders, our internal ticket for this is CRIBL-21845 which is still under review.

Reply


Terms & ConditionsCookie settings