Enhanced observability for TLS ClientHello: ciphers, supported groups, key shares and sigalgs

[martinschmatz]

Your Feature Request

OpenSSL allows to get insights into some of the contents of a TLS ClientHello, e.g the SNI (here, here with implementation here).

What is proposed is to extend this functionality to ciphers, supported groups, key shares and sigalgs.

What are you trying to do?

The added functionality would allow routing with fine granularity pending the capabilities the client indicates in the ClientHello.

In particular, it allows a L4 (tcp) passthrough for some of the supported groups with L7 (https) TLS termination for others. It also allows to take client key shares into consideration when deciding which of the client supported groups should be used for groups considered to have 'equal security level'. Last-not-least, it enables fine grain selection of certificate types (beyond the RSA vs ECC distinction).

All of that is relevant in the context of rapidly upcoming PQC operation modes.

The affected files will be payload.c for ciphers, supported groups, key shares and sigalgs, plus also struct buffer in buf-t.h for key shares.

I have implementations for all of the above described functionalities which - if there's interest - I'd be happy to share.

The implementation is straight forward by simply mimicking what's already available for e.g. SNI, with the exception of the extraction of the key share code points which are not found sequentially in the ClientHello buffer and hence require some (small) additional intermediate buffer space.

Let me add that if the functionality is not used, the related code sections would not be executed and hence no performance penalty occurs.

Output of haproxy -vv

haproxy -vv
HAProxy version 2.9.6-9eafce5 2024/02/26 - https://haproxy.org/
Status: stable branch - will stop receiving fixes around Q1 2025.
Known bugs: http://www.haproxy.org/bugs/bugs-2.9.6.html
Running on: Linux 6.1.19 #1 SMP PREEMPT_DYNAMIC Sat Mar 25 09:19:55 UTC 2023 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = generic
  CC      = cc
  CFLAGS  = -O3 -fno-omit-frame-pointer -DTCP_USER_TIMEOUT=18 -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment
  OPTIONS = USE_THREAD=1 USE_LINUX_TPROXY=1 USE_GETADDRINFO=1 USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1 USE_TFO=1 USE_PROMEX=1 USE_PCRE2=1 USE_PCRE2_JIT=1
  DEBUG   = -DDEBUG_STRICT -DDEBUG_MEMORY_POOLS

Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE -LIBATOMIC +LIBCRYPT +LINUX_CAP +LINUX_SPLICE +LINUX_TPROXY +LUA +MATH -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL -OPENSSL_AWSLC -OPENSSL_WOLFSSL -OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX -PTHREAD_EMULATION -QUIC -QUIC_OPENSSL_COMPAT +RT +SHM_OPEN +SLZ +SSL -STATIC_PCRE -STATIC_PCRE2 -SYSTEMD +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL -ZLIB

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_TGROUPS=16, MAX_THREADS=256, default=128).
Built with OpenSSL version : OpenSSL 3.2.1 30 Jan 2024
Running on OpenSSL version : OpenSSL 3.2.1 30 Jan 2024
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
OpenSSL providers loaded : default oqsprovider
Built with Lua version : Lua 5.4.6
Built with the Prometheus exporter as a service
Built with network namespace support.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.42 2022-12-11
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 11.4.1 20230605 (Red Hat 11.4.1-2)

Available polling systems :
      epoll : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result OK
Total: 3 (3 usable), will use epoll.

Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
         h2 : mode=HTTP  side=FE|BE  mux=H2    flags=HTX|HOL_RISK|NO_UPG
       fcgi : mode=HTTP  side=BE     mux=FCGI  flags=HTX|HOL_RISK|NO_UPG
  <default> : mode=HTTP  side=FE|BE  mux=H1    flags=HTX
         h1 : mode=HTTP  side=FE|BE  mux=H1    flags=HTX|NO_UPG
  <default> : mode=TCP   side=FE|BE  mux=PASS  flags=
       none : mode=TCP   side=FE|BE  mux=PASS  flags=NO_UPG

Available services : prometheus-exporter
Available filters :
        [BWLIM] bwlim-in
        [BWLIM] bwlim-out
        [CACHE] cache
        [COMP] compression
        [FCGI] fcgi-app
        [SPOE] spoe
        [TRACE] trace

[wlallemand]

Hello, don't hesitate to share your patches, people could benefit from this indeed, and it would be simpler for your own sake to have this in mainline!

[martinschmatz]

Great! Will do....

Just FYI - I identified a person in my organization more knowledgeable in the process of open-sourcing (in particular when it comes to also generating tests and documentation updates) who has started to work on the relevant PRs, to be submitted in the next couple of weeks.