Updating kube-reserved and system-reserved to better fit usage

[Cyper-Madsen]

We have experienced a few nodes going down, and we tracked it to memory usage, and that 1 pod used all the memory that was available for the system, and as Bottlerocket-os don't set a default system-reserved and that kube-reserved is set to less then it is using.

My findings on kube-reserved is that it is set using the formula "memory_to_reserve = max_num_pods * 11 + 255" and the 11 is properly for the containerd.socket and the 255 is for containerd, but when I look at the memory usage cgroup the sockets are using on average 18Mib of memory, so the larger the node and the more pods that is deployed there should be allocated more to kube-reserved for the system to stay stable.

My recommendation for kube-reserved is changing the formula to memory_to_reserve = max_num_pods * 18 + 255

For system-reserved as it is not set we it is quite easy for the system to max out, I recommend that it is set to 500Mib to allow for bursting and keep vital node services running, from my finding it is just using around 200Mib, but it could depend on usage of the system so setting this ekstra high looks to be normal.