Use cgroups for limiting memory usage

[Williangalvani]

fix #2393
fix #2387
fix #2378

[Williangalvani]

it seems good now. I'd appreciate some more testing =]
Btw, you can add a cgroup column to htop, but it does mess up the columns spacing =[

[joaoantoniocardoso]

We extensively tested (@Williangalvani and me) for two hours changing the memory limit, as well as playing with other controls not used on this PR, like the memory.high, memory.swap.high and memory.swap.max.

As a next improvement, we should be able to configure the memory.high, and move this run_service.sh to some robust application (I heard Rust?) instead of a bash script, possibly having a frontend to show the cgroups stats and events, and allow the command line, env variables, and the cgroups controls to be configured.

[joaoantoniocardoso]

Useful commands inside the docker:

# Check if any process has been hitting memory limits and/or has been killed by the OOM.
cat /sys/fs/cgroup/$DOCKER_CGROUP/*/memory.events
# Check the current memory of any service:
cat /sys/fs/cgroup/$DOCKER_CGROUP/<cgroupname>/memory.*current

...and RedHat has a good resource about how to monitor it here. The most useful command is, from outside the docker:

systemd-cgtop

[Williangalvani]

time to get this in?

[patrickelectric]

We should test the ssh_command is working, if not, we should abort this process and continue the start script.
The reason behind this is that the start script is the root of the entire BlueOS, it can't fail.

[Williangalvani]

done

[patrickelectric]

We should create a variable for /sys/fs/cgroup/$DOCKER_CGROUP, since it's used so many times

[Williangalvani]

done

[Williangalvani]

@patrickelectric can you try it on docker compose? I'm having some iptables issues

[ES-Alexander]

Would be nice if we could include some notes about this in the BlueOS-core dev docs, if/when it gets merged in.

[patrickelectric]

@patrickelectric can you try it on docker compose? I'm having some iptables issues

Everything appears to be fine with compose