Report "http2: server sent GOAWAY and closed the connection; LastStreamID=33695, ErrCode=INTERNAL_ERROR, debug="""

[ningyougang]

We are using nghttp2 in istio

• istio version: 1.17.2
• envoy version: 1.25.4
• nghttp2 version: 1.51.0

Because there some security issues in nghttp2

• GHSA-x6x3-gv8h-m57q
• HTTP/2 memory leak
• HTTP/2 Rapid Reset

After cherry-pick above patches's commits in nghttp2 codes of 1.51.0 branch in my local and added patch using git diff 00399695cb3ea9715162b6a15c0d7b185ac384d2 > cve.patch and finally did benchmark test under 30K RPS using vegeta, report below info

Requests      [total, rate, throughput]         819706, 1352.41, 1351.79
Duration      [total, attack, wait]             10m6s, 10m6s, 2.509ms
Latencies     [min, mean, 50, 90, 95, 99, max]  1.02ms, 233.567ms, 21.9ms, 680.424ms, 856.186ms, 1.361s, 3.502s
Bytes In      [total, mean]                     452171750, 551.63
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           99.95%
Status Codes  [code:count]                      0:377  200:819329  
Error Set:
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=19629, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=31541, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=19949, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=30493, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=19455, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=18705, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=20885, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=20629, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=32087, ErrCode=INTERNAL_ERROR, debug=""
Get "https://myhost.com": http2: server sent GOAWAY and closed the connection; LastStreamID=19631, ErrCode=INTERNAL_ERROR, debug=""
...

It seems server side sends GOAWAY frame to client somtimes, i don't know if this is normal behavior, so i asked here.
(Fyi, without cherry-pick the security commits, server sent GOAWAY and closed the connection issue does not exist)

There has a similar issue: nghttp2 1.57.0 limits responses to 33 per second here
Not sure my issue has relation with this similar issue

[ningyougang]

It seems under some condition(e.g. meets the rate limit condition), sever side will send goaway iframe: 72b4af6#diff-5b2a44d10e274206481b47626335d2cc6061865744d38e1886b24b0f890f3036R4475

[tatsuhiro-t]

I do not see any issues between h2load and nghttpd. I do not know what istio, envoy or vegata are doing.
It is better to ask them about this.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.