would it be possible to use the same encryption key in an Edge fleet and in a Stream worker group ?

Are you talking about event-level encryption, like using `C.Crypto.encrypt()` ?

yes i am

Then, yes, considering you use the same keys on all instances

Ok I thought cribl used different secrets for each and every group/fleet

maybe I missed something ...

Well there's multiple things, but cribl.secret isn't the same es the encryption key for data

You can even use the Cribl app on Splunk to do the decryption in Splunk while searching

Playing devil's advocate here... been on the infosec side too... Do they mean encrypting the data in transit (i.e. TLS/SSL)?

That's a huge performance penalty for encrypting the actual data itself and would prevent you from doing granular filtering of data in S3 if you ever wanted to replay it.

And distributing the keys to every Edge node would in theory allow an attacker to just decrypt the data if they were able to obtain the key pairs.

data we are dealing with are sensitive and need to be stored on WORM S3 for 10 years. Replying the data is actually not a big deal because of the unencrypted metadata added to each event. I am encrypting _raw field only.Regarding the last point I am considering using an external KMS such as Hashicorp Vault.Last but not least it is a very specific use case, we are talking about 4 Edge clients here for a daily volume around 100MB

transit is already secured with TLS

Ah, okay. I feel way better about your use case here.

Hashicorp Vault is good, but your Edge clients would need to be able to speak with it to get the `cribl.secret` file.

We don't support external symmetric key storage yet

are the encryption keys stored in cribl.secret?

ah :slightly_smiling_face:

No, but they work together

So if I understand correctly, since each workerGroup/fleet has it's own secret I cannot use the exact same encryption key between my fleet and my worker group

Per my understanding, the `cribl.secret` is only used to store the encrypted representation of the actually used encryption key

It would make no sense to use some randomly generated secret to encrypt the data, because then you wouldn't be able to decrypt it later (without pulling the cribl.secret from all your boxes)

Ok but when you generate an Encryption key for a Edge fleet you have no way to import it to your Worker Group Security settings. But again maybe I missed something ...

Because the leader node is creating a secret for each group/fleet

Okay, maybe I'm the one talking bullshit here, lemme try something

If you find a way to do it next round is on me

Okay, I gotta admit this seems a little extra complicated to me

So what you should be able to do is create a key, and then download the key bundle, and manually place it on the leader for the other group

If that group already has encrypted stuff in its config, you might have to reset that

it's not a very pretty solution, but should work

I don't think you can because the secret also used to generate a randomized user/pwd to communicate with the nodes of the group/fleet when you teleport to them, per my understanding

so if you override the secret file of a group I am pretty sure that you break everything

maybe not used to generate it, but possibly to store it

It seems to me nobody thought about "encrypt to store in S3 and later decrypt when replaying", where those two actions don't happen on the same worker group...