HPE_INVALID_HEADER_TOKEN on https requests

[ejoebstl]

Version: v10.16.3
Platform: alpine:3.10 in docker
Subsystem: http_parser: '2.9.2'

HTTP request crashes with HPE_INVALID_HEADER_TOKEN.

Script to reproduce:

https.get('https://www.dezeen.com/2015/08/04/walden-raft-provides-seclusion-french-lake-elise-morin-florent-albinet-henry-david-thoreau-cabin/', (resp) => { }).on("error", (err) => {
  console.log(err);
});

Output:

{ Error: Parse Error
    at TLSSocket.socketOnData (_http_client.js:442:20)
    at TLSSocket.emit (events.js:198:13)
    at TLSSocket.EventEmitter.emit (domain.js:466:23)
    at addChunk (_stream_readable.js:288:12)
    at readableAddChunk (_stream_readable.js:269:11)
    at TLSSocket.Readable.push (_stream_readable.js:224:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:94:17) bytesParsed: 1087, code: 'HPE_INVALID_HEADER_TOKEN' }

I think it's related to http_parser: '2.9.2'. It does not happen with a slightly older build which uses http_parser: '2.8.0'.

Likely related to #27711 (comment).

The mentioned workaround does not work here, as the --http-parser option is not available on node v10.

[richardlau]

Node.js 10.16.3 still uses http_parser 2.8.0. @sam-github has a PR to bump the http_parser version to 2.9.1 (#30471) but it hasn't landed on 10.x yet.

Unless Node.js in docker is linking to an external http_parser?

[ejoebstl]

To clarify: This is the build installed on alpine linux 3.10. Notice nodejs 10.16.3 and http_parser 2.9.2 below.

> docker run -it alpine:3.10 sh
/ # apk update
/ # apk add nodejs
(1/7) Installing ca-certificates (20190108-r0)
(2/7) Installing c-ares (1.15.0-r0)
(3/7) Installing libgcc (8.3.0-r0)
(4/7) Installing http-parser (2.9.2-r0)
(5/7) Installing libstdc++ (8.3.0-r0)
(6/7) Installing libuv (1.29.1-r0)
(7/7) Installing nodejs (10.16.3-r0)
Executing busybox-1.30.1-r2.trigger
Executing ca-certificates-20190108-r0.trigger
OK: 32 MiB in 21 packages
/ # 
/ #
/ # node --version
v10.16.3
/ # node
> process
process {
  title: 'node',
  version: 'v10.16.3',
  versions:
   { http_parser: '2.9.2',
     node: '10.16.3',
     v8: '6.8.275.32-node.54',
     uv: '1.29.1',
     zlib: '1.2.11',
     brotli: '1.0.7',
     ares: '1.15.0',
     modules: '64',
     nghttp2: '1.39.2',
     napi: '4',
     openssl: '1.1.1c',
     icu: '64.2',
     unicode: '12.1',
     cldr: '35.1',
     tz: '2019a' },
  arch: 'x64',
  platform: 'linux',
[......]

Details here. Is it worth raising an issue with the maintainer? I guess they are using an external http_parser.

[sam-github]

Also repros with node w/http-parser@2.9.1

% ./out/Release/node                    
> https.get('https://www.dezeen.com/2015/08/04/walden-raft-provides-seclusion-french-lake-elise-morin-florent-albinet-henry-david-thoreau-cabin/', (resp) => { }).on("error", (err) => {console.log(err);}), null;
null
> { Error: Parse Error
    at TLSSocket.socketOnData (_http_client.js:454:20)
    at emitOne (events.js:116:13)
    at TLSSocket.emit (events.js:211:7)
    at addChunk (_stream_readable.js:263:12)
    at readableAddChunk (_stream_readable.js:250:11)
    at TLSSocket.Readable.push (_stream_readable.js:208:10)
    at TLSWrap.onread (net.js:601:20) bytesParsed: 605, code: 'HPE_INVALID_HEADER_TOKEN' }

> process.versions
{ http_parser: '2.9.1',
  node: '8.16.3-pre',
  v8: '6.2.414.78',
  uv: '1.23.2',
  zlib: '1.2.11',
  ares: '1.10.1-DEV',
  modules: '57',
  nghttp2: '1.39.2',
  napi: '4',
  openssl: '1.0.2s',
  icu: '60.1',
  unicode: '10.0',
  cldr: '32.0',
  tz: '2017c' }

Does not repro with 2.8.1 (build from v10.x-staging)

@nodejs/http-parser PTAL

[indutny]

This appears to be the same problem with Incapsula server as before:

Set-Cookie: ___utmvaskuOpzY=uda\u0001zMYa; path=/; Max-Age=900

[indutny]

See #29589

[sam-github]

@indutny @nodejs/http I'm still not clear, is this a bug or not? Do you have a link to the "before" issue?

It's clearly a change in the behaviour of the http-parser, before #30471 http parser 2.8.0 doesn't error, after with 2.9.1 it does error.

Is that change a regression?

If its not a regression, is it enough of a change to block release in an LTS version? cc: @nodejs/lts

If the failure here is a result of the security fixes nodejs/http-parser#469 or nodejs/http-parser#458, we might need to add a --cve-revert to #30471, since its those sec issues that prompted me to PR updates to the http-parser into LTS branches.

[indutny]

My opinion here is that fixing this bug generally and without a runtime flag means likely security issue for most users. Incapsula is sending blatantly incorrect header value and we are right to reject it. HTTP/1.1 is very brittle as the protocol data is mixed with the protocol itself. It would be unwise to loosen our requirements since it could potentially lead to request smuggling and other nasty security vulnerabilities.

However, I do believe that we have to re-introduce the lenient parsing to llhttp and in fact I just opened a PR for this: nodejs/llhttp#33 . Not sure if it should be a blocker for a release in LTS version, though.

[ianp]

I would like to be able to set this on a per request basis, via a insecureParser option (similar to the existing maxHeaderSize) to http.request. Happy to have a go at creating a PR for it myself (now that @indutny has done all of the hard work!), should I open a new issue for this or just reference this one?

[sam-github]

@ianp you don't need to open an issue to open a PR. @addaleax was interested in doing something similar to what you suggest,but I suspect would be happy for you to do it instead. Nobody can call dibs on a feature, so if you can implement it quickly, its yours.

[addaleax]

@ianp Yes, feel free to go ahead! You can use Refs: https://github.com/nodejs/node/issues/30515 in the commit message if you want to refer to this issue 👍

[lundibundi]

Closing, see #27711 (comment).