OpenSSH broken after updating to 8.9p1-2.ph4

[ufoonline]

Describe the bug

OS: Photon OS 4.0
Latest know working OpenSSH Version:
openssh-clients-8.8p1-3.ph4.x86_64
openssh-server-8.8p1-3.ph4.x86_64
openssh-8.8p1-3.ph4.x86_64

Latest avaiable OpenSSH package:
openssh-server x86_64 8.9p1-2.ph4 photon-updates 1.14M 1196581
openssh-clients x86_64 8.9p1-2.ph4 photon-updates 4.83M 5061405
openssh x86_64 8.9p1-2.ph4 photon-updates 0.00b 0

Tested kernel:
5.10.142-1.ph4-esx
5.10.190-3.ph4-esx

After the upgrade:
1 - systemctl daemon-reload is not triggered
2 - the SSHd daemon is down
3 - If you manually start the daemon you will not be able to log in and the following error will be logged:
2023-09-14T06:26:26.681618+00:00 SRVNAME sshd[4675]: Server listening on 0.0.0.0 port 22.
2023-09-14T06:26:26.681856+00:00 SRVNAME sshd[4675]: Server listening on :: port 22.
2023-09-14T06:26:34.935768+00:00 SRVNAME sshd[4685]: Connection closed by 127.0.0.1 port 38882 [preauth]
2023-09-14T06:26:49.518524+00:00 SRVNAME sshd[4693]: [module:pam_lsass]pam_sm_authenticate: failed [error code:40017]
2023-09-14T06:26:49.524349+00:00 SRVNAME sshd[4693]: [module:pam_lsass]pam_sm_authenticate: failed [error code:40017]
2023-09-14T06:26:49.531977+00:00 SRVNAME sshd[4691]: Accepted keyboard-interactive/pam for support from 127.0.0.1 port 59004 ssh2
2023-09-14T06:26:49.532557+00:00 SRVNAME audit[4692]: SECCOMP auid=4294967295 uid=50 gid=50 ses=4294967295 subj=unconfined pid=4692 comm="sshd" exe="/usr/sbin/sshd" sig=31 arch=c000003e syscall=13 compat=0 ip=0x7f0d09dc8192 code=0x0
2023-09-14T06:26:49.532698+00:00 SRVNAME audit[4692]: ANOM_ABEND auid=4294967295 uid=50 gid=50 ses=4294967295 subj=unconfined pid=4692 comm="sshd" exe="/usr/sbin/sshd" sig=31 res=1
2023-09-14T06:26:49.532996+00:00 SRVNAME sshd[4691]: fatal: privsep_preauth: preauth child terminated by signal 31

Reproduction steps

1. Upgrade openssh package from 8.8p1-3.ph4 to 8.9p1-2.ph
2. systemctl-daemon reload
3. systemct start ssh
   ...

Expected behavior

Would be possibile to log-in.

Additional context

No response

[dcasota]

similar to #1480

[ufoonline]

Hello,

If I understood well the workaround that has been put in place by the user was to switch from sshd.socket to sshd.service, I did it but had no improvements:
root@SRVNAME [ /etc/tdnf/locks.d ]# systemctl disable --now sshd.socket
Removed /etc/systemd/system/sockets.target.wants/sshd.socket.
root@SRVNAME [ /etc/tdnf/locks.d ]# systemctl daemon-reload
root@SRVNAME [ /etc/tdnf/locks.d ]# systemctl enable --now sshd.service
root@SRVNAME [ /etc/tdnf/locks.d ]# systemctl status sshd.socket
● sshd.socket
Loaded: loaded (/usr/lib/systemd/system/sshd.socket; disabled; vendor preset: enabled)
Active: inactive (dead) since Fri 2023-09-15 12:56:59 UTC; 15s ago
Listen: [::]:22 (Stream)
Accepted: 12; Connected: 1;

Sep 14 07:18:54 SRVNAME systemd[1]: Listening on sshd.socket.
Sep 15 12:56:59 SRVNAME systemd[1]: sshd.socket: Succeeded.
Sep 15 12:56:59 SRVNAME systemd[1]: Closed sshd.socket.
root@SRVNAME [ /etc/tdnf/locks.d ]# systemctl status sshd
● sshd.service - OpenSSH Daemon
Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2023-09-15 12:57:01 UTC; 16s ago
Main PID: 4186 (sshd)
Tasks: 1 (limit: 9543)
Memory: 1.0M
CGroup: /system.slice/sshd.service
└─4186 sshd: /usr/sbin/sshd -D [listener] 0 of 10-60 startups

Sep 15 12:57:01 SRVNAME systemd[1]: Started OpenSSH Daemon.
Sep 15 12:57:01 SRVNAME sshd[4186]: Server listening on 0.0.0.0 port 22.
Sep 15 12:57:01 SRVNAME sshd[4186]: Server listening on :: port 22.
Sep 15 12:57:08 SRVNAME sshd[4192]: Accepted keyboard-interactive/pam for XXX\xxxxxx from 10.xxx.xxx.xxx port 57402 ssh2
Sep 15 12:57:08 SRVNAME sshd[4192]: fatal: privsep_preauth: preauth child terminated by signal 31
root@SRVNAME [ /etc/tdnf/locks.d ]#

Best Regards

[dcasota]

Hi @ufoonline , I see your point. Unfortunately the latest available openssh packages are not backported to 4.0, and the 8.9p1 with all bugs - and your issue mentioned - is the latest in 4.0. Assuming you've tested the distro update, actually I would stay on 8.8p1 or upgrade to Ph5.0 (+distro update).