This message originated from Cribl Community Slack.
Click here to view the original link.
Hi,
I have an on prem stream single instance and i want to install Cribl edge on several servers to collect and forward logs to this stream instance.
In this case what is the right architecture here? Can this Cribl stream single instance act as a leader for the edge nodes? or they require a separate leader ?
If i install edge as a single instance on each machine i wish to collect from, does it consume a lot of resources? or should i install a separate leader node and all edge nodes connect to it?
Solved
Cribl Edge Installation Architecture Query For Resource Management And Leader Node Configuration
Best answer by xpac
A "piece of Cribl" can be a worker, an Edge node, a leader, but only one of those. If you want to centrally manage your Edge nodes, you'll need a leader AND a stream worker. However, nothing says you can't do that on the same instance, if the load on the leader is light, it will just be a little cumbersome. Or do it in docker or whatever container solution.
Either way - if you're getting started with Edge, I'd advise to connect them to a leader from the beginning. How many nodes are you planning?
A "piece of Cribl" can be a worker, an Edge node, a leader, but only one of those. If you want to centrally manage your Edge nodes, you'll need a leader AND a stream worker. However, nothing says you can't do that on the same instance, if the load on the leader is light, it will just be a little cumbersome. Or do it in docker or whatever container solution.
Either way - if you're getting started with Edge, I'd advise to connect them to a leader from the beginning. How many nodes are you planning?
I'd say the resource consumption does not really differ between standalone Edge and Leader-connected, but I never benchmarked this. Either way Edge should not consume more than one CPU core at max
So it is not possible to manage cribl edge worker nodes with a stream single instance (stream instance to act as leader for the edge nodes as well as a stream single instance)
or it is not documented if its possible or not? I cannot find anything the proves or denies it yet.
I cannot have a separate server for the leader for now, the approach is the one mentioned above if possible or to have edge single instances for each server i am going to collect from.
I am planning on having 5-6 edge nodes.
It's not possible. One instance of Cribl can be a leader (which means no data processing), or a worker, or an Edge node
I'd really consider trying to use that one server with two Cribl instances, if you currently can't get a new box
Else youll have to manage each Edge node manually, including updates etc. I'd try to avoid that
you're welcome :slightly_smiling_face:
Sign up
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.