PROPOSAL: Reduce init container memory from 500Mi to 64Mi

[mkuratczyk]

It'd be great to get some feedback from users who changed the default init container resources - what did you change them to? I've tested successfully deployed RabbitMQ with a 50Mi init container using:

• rabbitmq:4.1-management
• bitnami/rabbitmq:4.1
• rabbitmq.packages.broadcom.com/vmware-tanzu-rabbitmq:4.1.0

Given that an init container failure would be a terrible user experience, I'd rather have some headroom, hence 64Mi.

When originally developed, it wasn't clear what would need to be done in the init container. With hindsight, there's very little to do and the init phase hasn't changed in a long time.

We also assumed that the resources requested for the init container would effectively be reused by the actual rabbitmq container later. Turns out that's not the case: kubernetes/kubernetes#124282 init container resources are considered even after the init container completes its task.

This closes #960

[zvlb]

We tested with this resources:

resources:
  limits:
    cpu: 10m
    memory: 50Mi
  requests:
    cpu: 10m
    memory: 50Mi

All working good.

I think we can install:

initContainerCPU    string = "20m"
initContainerMemory string = "64Mi"

without any potential issues for users

[Zerpet]

I like this proposal 👍 The init container doesn't do anything special, and it uses very basic shell commands like cp and mv.

[zvlb]

pls add changes to initContainerCPU before merge it)

[mkuratczyk]

@Zerpet I've also pushed a change in CPU from 100m to 20m. Still ok with this?

[Zerpet]

20m is 0.02 CPU 🤔 I understand the intention of lowering this value so drastically, after reading again kubernetes/kubernetes#124282 and revisiting Pod QoS documentation. I'm not opposed to this change, since there are claims that 20m CPU JustWorks™

[mkuratczyk]

Yeah, I even tried what happens if I try to use 20m to start RabbitMQ itself. I didn't wait long enough to see it running but nothing was crashing, it was just starting very very slowly... :)
Should not be a problem at all to just move some files around.

[zvlb]

@mkuratczyk @Zerpet Thank you!

Сan I know when to expect a release with these changes?
The last release was on January 24th and I'm a little scared to guess how long to wait for the next one. Maybe there is some kind of release plan?

[mkuratczyk]

There's just not that much to release usually. The Operator as it is, seems to be working well for a lot of people.
We can discuss within the team tomorrow and let you know.

One thing you I'd love to hear your input for is whether it's a problem that this change, as currently implemented, would restart all those clusters you have, since changes to the StatefulSet definition cause a restart. We could make some tweaks to prevent this I think

[zvlb]

I understand that restarting all RabbitMQ clusters is an unpleasant situation and could indeed cause dissatisfaction among users. However, in my case, I don’t consider this a problem. If someone deployed a RabbitMQ cluster with a single instance and expects 100% SLA—that’s the problem of whoever set it up that way. Besides the reboot, such RabbitMQ installations cannot be considered stable, as a pod can be terminated for a multitude of reasons, and changes to the StatefulSet are just one of them!

If the RabbitMQ cluster is deployed with more than one node, updating the StatefulSet should proceed without performance issues.

[mkuratczyk]

Version 2.13.0 is now available:
https://github.com/rabbitmq/cluster-operator/releases/tag/v2.13.0