SSLClientServerTest.* tests fail with OpenSSL 3.2.1

[mgorny]

When attempting to run the test suite on Gentoo Linux amd64, I'm seeing most of the SSLClientServerTest.* tests fail:

[  FAILED  ] 6 tests, listed below:
[  FAILED  ] SSLClientServerTest.ClientCertPresent
[  FAILED  ] SSLClientServerTest.ClientEncryptedCertPresent
[  FAILED  ] SSLClientServerTest.MemoryClientCertPresent
[  FAILED  ] SSLClientServerTest.MemoryClientEncryptedCertPresent
[  FAILED  ] SSLClientServerTest.TrustDirOptional
[  FAILED  ] SSLClientServerTest.CustomizeServerSSLCtx

This is on 548dfff (but I also reproduce the same result on 0.15.3).

Detailed results:

[----------] 8 tests from SSLClientServerTest
[ RUN      ] SSLClientServerTest.ClientCertPresent
test.cc:5160: Failure
Value of: res
  Actual: false
Expected: true
[  FAILED  ] SSLClientServerTest.ClientCertPresent (7 ms)
[ RUN      ] SSLClientServerTest.ClientEncryptedCertPresent
test.cc:5160: Failure
Value of: res
  Actual: false
Expected: true
[  FAILED  ] SSLClientServerTest.ClientEncryptedCertPresent (7 ms)
[ RUN      ] SSLClientServerTest.MemoryClientCertPresent
test.cc:5243: Failure
Value of: res
  Actual: false
Expected: true
[  FAILED  ] SSLClientServerTest.MemoryClientCertPresent (6 ms)
[ RUN      ] SSLClientServerTest.MemoryClientEncryptedCertPresent
test.cc:5243: Failure
Value of: res
  Actual: false
Expected: true
[  FAILED  ] SSLClientServerTest.MemoryClientEncryptedCertPresent (7 ms)
[ RUN      ] SSLClientServerTest.ClientCertMissing
[       OK ] SSLClientServerTest.ClientCertMissing (5 ms)
[ RUN      ] SSLClientServerTest.TrustDirOptional
test.cc:5308: Failure
Value of: res
  Actual: false
Expected: true
[  FAILED  ] SSLClientServerTest.TrustDirOptional (6 ms)
[ RUN      ] SSLClientServerTest.SSLConnectTimeout
[       OK ] SSLClientServerTest.SSLConnectTimeout (1006 ms)
[ RUN      ] SSLClientServerTest.CustomizeServerSSLCtx
test.cc:5428: Failure
Value of: res
  Actual: false
Expected: true
[  FAILED  ] SSLClientServerTest.CustomizeServerSSLCtx (7 ms)
[----------] 8 tests from SSLClientServerTest (1056 ms total)

Please let me know how can I help you debug this.

Full output: test.txt

[mgorny]

This is with OpenSSL 3.2.1, Brotli 1.1.0.

[mgorny]

This seems specific to OpenSSL 3.2.1. With 3.1.5, they work.

[yhirose]

@mgorny thanks for the report. It's interesting. Are you able to run the tests with 3.2.0 to see what will happen?

[mgorny]

Sure, just done that. I've getting the same failures as with 3.2.1.

[yhirose]

Thanks a lot. It seems like something has changed between 3.1.x and 3.2.0 about certificate handling... At this point, I have no idea why the problem came from. I'll do some research about 3.2 when I have time. Please let me know if you find any helpful information regarding this matter.
Thanks!

[mgorny]

The release notes are here: https://www.openssl.org/news/openssl-3.2-notes.html

My first thought was "The default SSL/TLS security level has been changed from 1 to 2", but I've tried adding SSL_CTX_set_security_level(..., 1) everywhere and that didn't help. I don't see anything else that would look like a breaking change.