Skip to main content
Solved

AWS Workers Failing To Deliver Data To Dynatrace With 401 Unauthorized Error

  • April 29, 2026
  • 3 replies
  • 5 views
H
This message originated from Cribl Community Slack.
Click here to view the original link.

Hi guys, I have workers running on-premises and data is being sent successfully from the source to the destination. I also have some workers running in AWS. These workers are able to receive data, but for some reason the Dynatrace destination keeps returning 401 Unauthorized, so the data is not being delivered to Dynatrace. I confirmed the following: If I SSH to any AWS worker, I can successfully send data to Dynatrace using curl with the same endpoint and auth token. Firewall connectivity is open. outputs.yml is synced correctly from the Leader to the AWS workers. The AWS workers are using a specific RHEL 8 image. I tried the suggested checks related to fapolicyd and SELinux, and I also tested with SELinux disabled, but the issue still persists. As soon as I configure the destination and route data to it, the destination fails with a 401 error. Has anyone faced a similar issue before, or can anyone help troubleshoot what might be causing this?

Best answer by wdelarocha352

{"time":"2026-04-24T12:45:13.062Z","cid":"w0","channel":"criblsecret","level":"error","message":"Secret decrypt failed","path":"/outputs/dynatrace/token","error":{"message":"error:1C800064:Provider routines::bad decrypt","stack":"Error: error:1C800064:Provider routines::bad decrypt\n    at Decipheriv.final
This means the worker can’t decrypt the Dynatrace token stored for that output, so it’s almost certainly sending a bad/empty token and Dynatrace is replying 401. Curl works because you’re using the raw token but Stream is failing earlier when it tries to decrypt the token In the Cribl UI, open the Dynatrace destination, re-paste the token, save, then Commit and Deploy. Send test traffic and watch the worker log to see if the Secret decrypt failed for /outputs/dynatrace/token disappears. If not, I’d suspect a cribl.secret mismatch on those workers and consider reinstalling them so they pick up the right cribl.secret

3 replies

A
  • agates182
  • Employee
  • April 29, 2026
Can you send the specific logs for said destination? (Destinations > $MY_DYNATRACE > Logs)
H
{"time":"2026-04-24T12:45:10.904Z","cid":"w0","channel":"Limits","level":"info","message":"reloading limits"}
{"time":"2026-04-24T12:45:11.554Z","cid":"w0","channel":"server","level":"info","message":"initializing data insights metrics manager","enabled":false,"enableFieldStats":false}
{"time":"2026-04-24T12:45:11.555Z","cid":"w0","channel":"DataInsightsMetricsManager","level":"info","message":"Data Insights is disabled"}
{"time":"2026-04-24T12:45:11.577Z","cid":"w0","channel":"functions","level":"info","message":"starting lister","context":"cribl"}
{"time":"2026-04-24T12:45:11.578Z","cid":"w0","channel":"collectors","level":"info","message":"starting lister"}
{"time":"2026-04-24T12:45:11.578Z","cid":"w0","channel":"executors","level":"info","message":"starting lister"}
{"time":"2026-04-24T12:45:11.739Z","cid":"w0","channel":"redis:RedisCacheManager","level":"info","message":"Cache config changed. Resetting cache."}
{"time":"2026-04-24T12:45:11.739Z","cid":"w0","channel":"redis:RedisClientManager","level":"info","message":"Connection config changed. Rebalancing clients."}
{"time":"2026-04-24T12:45:13.056Z","cid":"w0","channel":"criblsecret","level":"error","message":"Secret decrypt failed","path":"/outputs/ekobs_np_loglens-aws-destination/token","error":{"message":"error:1C800064:Provider routines::bad decrypt","stack":"Error: error:1C800064:Provider routines::bad decrypt\n    at Decipheriv.final (node:internal/crypto/cipher:184:29)\n    at f.decrypt (/opt/obsv/cribl/bin/cribl.js:21:6949510)\n    at Object.g (/opt/obsv/cribl/bin/cribl.js:21:15223629)\n    at Object.validate (eval at N (/opt/obsv/cribl/bin/cribl.js:21:428240), <anonymous>:3:6070327)\n    at Object.e (/opt/obsv/cribl/bin/cribl.js:21:8588651)\n    at M.decrypt (/opt/obsv/cribl/bin/cribl.js:21:15226359)\n    at P.loadFromDirs (/opt/obsv/cribl/bin/cribl.js:21:6930480)\n    at async Object.load (/opt/obsv/cribl/bin/cribl.js:21:13690590)"}}
{"time":"2026-04-24T12:45:13.062Z","cid":"w0","channel":"criblsecret","level":"error","message":"Secret decrypt failed","path":"/outputs/dynatrace/token","error":{"message":"error:1C800064:Provider routines::bad decrypt","stack":"Error: error:1C800064:Provider routines::bad decrypt\n    at Decipheriv.final (node:internal/crypto/cipher:184:29)\n    at f.decrypt (/opt/obsv/cribl/bin/cribl.js:21:6949510)\n    at Object.g (/opt/obsv/cribl/bin/cribl.js:21:15223629)\n    at Object.validate (eval at N (/opt/obsv/cribl/bin/cribl.js:21:428240), <anonymous>:3:6070327)\n    at Object.e (/opt/obsv/cribl/bin/cribl.js:21:8588651)\n    atM.decrypt (/opt/obsv/cribl/bin/cribl.js:21:15226359)\n    at P.loadFromDirs (/opt/obsv/cribl/bin/cribl.js:21:6930480)\n    at async Object.load (/opt/obsv/cribl/bin/cribl.js:21:13690590)"}}
W
  • wdelarocha352
  • New Participant
  • Answer
  • April 29, 2026
{"time":"2026-04-24T12:45:13.062Z","cid":"w0","channel":"criblsecret","level":"error","message":"Secret decrypt failed","path":"/outputs/dynatrace/token","error":{"message":"error:1C800064:Provider routines::bad decrypt","stack":"Error: error:1C800064:Provider routines::bad decrypt\n    at Decipheriv.final
This means the worker can’t decrypt the Dynatrace token stored for that output, so it’s almost certainly sending a bad/empty token and Dynatrace is replying 401. Curl works because you’re using the raw token but Stream is failing earlier when it tries to decrypt the token In the Cribl UI, open the Dynatrace destination, re-paste the token, save, then Commit and Deploy. Send test traffic and watch the worker log to see if the Secret decrypt failed for /outputs/dynatrace/token disappears. If not, I’d suspect a cribl.secret mismatch on those workers and consider reinstalling them so they pick up the right cribl.secret
Terms & ConditionsAccessibility statement