Failing to stop nmeof-gw container when removing nvmeof-gw service resulting SIGKILL nvmeof-gw systemd service

[roysahar]

Removing nvmeof-gw -
ceph orch rm nvmeof.nvmeof-pool --force

Resulting systemd service exit code 137 (128+9) ==> SIGKILL:

Feb 26 08:23:50 a-rhel9-node2 bash[1771785]: time="2024-02-26T08:23:50-05:00" level=warning msg="StopSignal SIGTERM failed to stop container ceph-ec7951fe-d263-11ee-8562-005056b4ea89-nvmeof-nvmeof-pool-a-rhel9-node2-dfydij in 10 seconds, resorting to SIGKILL"
.....
Feb 26 08:23:50 a-rhel9-node2 systemd[1]: ceph-ec7951fe-d263-11ee-8562-005056b4ea89@nvmeof.nvmeof-pool.a-rhel9-node2.dfydij.service: Main process exited, code=exited, status=137/n/a
Feb 26 08:23:50 a-rhel9-node2 systemd[1]: ceph-ec7951fe-d263-11ee-8562-005056b4ea89@nvmeof.nvmeof-pool.a-rhel9-node2.dfydij.service: Failed with result 'exit-code'.
Feb 26 08:23:50 a-rhel9-node2 systemd[1]: Stopped Ceph nvmeof.nvmeof-pool.a-rhel9-node2.dfydij for ec7951fe-d263-11ee-8562-005056b4ea89.

[caroav]

I failed to reproduce it with:
[root@multiESXiM2a ~]# docker images REPOSITORY TAG IMAGE ID CREATED SIZE quay.io/ceph/nvmeof-cli 1.1.0 8565e0c679b3 22 hours ago 258MB quay.io/ceph/nvmeof 1.1.0 ee72d0fbfa7d 22 hours ago 638MB quay.io/barakda1/ceph 47ea673ae9ebf51b2ebc505093bd7272422045e4 57d5a95370c2 2 days ago 1.3GB

@roysahar can you retry and update?

[roysahar]

@caroav I have a recreation - FYI.