You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are multiple bugs about server-state-file where when used, a reload would not pick up changes to ports (e.g. #2103) or when enabling TLS/SSL for backends (e.g. #2401).
Maybe HAProxy could detect these changes on reload and not load the state for these servers? Load everything else where the server and port matches as well as TLS/SSL enabled.
Maybe a HAProxy option, or side tool, to generate the server-state-file from the latest haproxy.cfg on disk and then overlay it with the existing server state info removing/discarding servers that doesn't exist - echo 'show servers updated-state' | socat stdio /path/to/socket > /your/state/file or similar.
What are you trying to do?
We're using server-state-file to reduce the downtime on HAProxy reload. Unfortunately, sometimes port changes are made or TLS/SSL enabled for backends and that's not picked up without completely removing the state file then forcing a reload.
Unfortunately, on servers where there are a lot of sites, this causes downtime to the rest even though it's changes for a single site.
Output of haproxy -vv
$ haproxy -vv
HAProxy version 2.6.16-1ppa1~focal 2023/12/13 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2027.
Known bugs: http://www.haproxy.org/bugs/bugs-2.6.16.html
Running on: Linux 5.4.0-173-generic #191-Ubuntu SMP Fri Feb 2 13:55:07 UTC 2024 x86_64
Build options :
TARGET = linux-glibc
CPU = generic
CC = cc
CFLAGS = -O2 -g -O2 -fdebug-prefix-map=/build/haproxy-Uooydt/haproxy-2.6.16=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -Wshift-negative-value -Wshift-overflow=2 -Wduplicated-cond -Wnull-dereference -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment
OPTIONS = USE_PCRE2=1 USE_PCRE2_JIT=1 USE_OPENSSL=1 USE_LUA=1 USE_SLZ=1 USE_SYSTEMD=1 USE_PROMEX=1
DEBUG = -DDEBUG_STRICT -DDEBUG_MEMORY_POOLS
Feature list : -51DEGREES +ACCEPT4 +BACKTRACE -CLOSEFROM +CPU_AFFINITY +CRYPT_H -DEVICEATLAS +DL -ENGINE +EPOLL -EVPORTS +GETADDRINFO -KQUEUE +LIBCRYPT +LINUX_SPLICE +LINUX_TPROXY +LUA -MEMORY_PROFILING +NETFILTER +NS -OBSOLETE_LINKER +OPENSSL -OT -PCRE +PCRE2 +PCRE2_JIT -PCRE_JIT +POLL +PRCTL -PROCCTL +PROMEX -QUIC +RT +SLZ -STATIC_PCRE -STATIC_PCRE2 +SYSTEMD +TFO +THREAD +THREAD_DUMP +TPROXY -WURFL -ZLIB
Default settings :
bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Built with multi-threading support (MAX_THREADS=64, default=4).
Built with OpenSSL version : OpenSSL 1.1.1f 31 Mar 2020
Running on OpenSSL version : OpenSSL 1.1.1f 31 Mar 2020
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports : TLSv1.0 TLSv1.1 TLSv1.2 TLSv1.3
Built with Lua version : Lua 5.3.3
Built with the Prometheus exporter as a service
Built with network namespace support.
Support for malloc_trim() is enabled.
Built with libslz for stateless compression.
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Built with PCRE2 version : 10.34 2019-11-21
PCRE2 library supports JIT : yes
Encrypted password support via crypt(3): yes
Built with gcc compiler version 9.4.0
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Available multiplexer protocols :
(protocols marked as <default> cannot be specified using 'proto' keyword)
h2 : mode=HTTP side=FE|BE mux=H2 flags=HTX|HOL_RISK|NO_UPG
fcgi : mode=HTTP side=BE mux=FCGI flags=HTX|HOL_RISK|NO_UPG
<default> : mode=HTTP side=FE|BE mux=H1 flags=HTX
h1 : mode=HTTP side=FE|BE mux=H1 flags=HTX|NO_UPG
<default> : mode=TCP side=FE|BE mux=PASS flags=
none : mode=TCP side=FE|BE mux=PASS flags=NO_UPG
Available services : prometheus-exporter
Available filters :
[CACHE] cache
[COMP] compression
[FCGI] fcgi-app
[SPOE] spoe
[TRACE] trace
these actually are not bugs but design limitations that were overlooked by then. The state file doesn't include the config settings, only the last ones, so when they differ, some arbitration has to be performed to figure which one is the most likely valid :-/
I think there's an issue somewhere mentioning the need to extend the file format to place config stuff there so that upon reload there's no doubt about what has changed. The algorithm is simple: if the previous config differs from the new one, the new config must be used because it was part of the cause for the reload. Otherwise the last state must be used.
Your Feature Request
There are multiple bugs about
server-state-file
where when used, a reload would not pick up changes to ports (e.g. #2103) or when enabling TLS/SSL for backends (e.g. #2401).Maybe HAProxy could detect these changes on reload and not load the state for these servers? Load everything else where the server and port matches as well as TLS/SSL enabled.
Maybe a HAProxy option, or side tool, to generate the
server-state-file
from the latesthaproxy.cfg
on disk and then overlay it with the existing server state info removing/discarding servers that doesn't exist -echo 'show servers updated-state' | socat stdio /path/to/socket > /your/state/file
or similar.What are you trying to do?
We're using
server-state-file
to reduce the downtime on HAProxy reload. Unfortunately, sometimes port changes are made or TLS/SSL enabled for backends and that's not picked up without completely removing the state file then forcing a reload.Unfortunately, on servers where there are a lot of sites, this causes downtime to the rest even though it's changes for a single site.
Output of
haproxy -vv
Doesn't appear to be much progress with #953
The text was updated successfully, but these errors were encountered: