Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

request disconnect event does not trigger for certain HTTP methods under node v16 #4315

Open
FaHeymann opened this issue Nov 9, 2021 · 6 comments
Labels
bug Bug or defect

Comments

@FaHeymann
Copy link

Support plan

  • is this issue currently blocking your project? (yes/no): no
  • is this issue affecting a production system? (yes/no): no

Context

  • node version: v16.12.0 / v15.14.0
  • module version with issue: 20.2.1
  • last module version without issue: n/a
  • environment (e.g. node, browser, native): node
  • used with (e.g. hapi application, another framework, standalone, ...): n/a
  • any other relevant information:

What are you trying to achieve or the steps to reproduce?

It seems like the disconnect event of requests does only fire for GET requests but not for other HTTP methods starting from nodeJS v16. See also the test script below. With node v15 it fires for all of them. In both versions the finish event also seems to not trigger for GET. I don't know why there are differences in the bahaviour between http methods in the first place.

const Hapi = require('@hapi/hapi');
const net = require('net');
const { setTimeout } = require('timers/promises');

const check = async (method) => {
  console.log(method)
  const server = Hapi.server({
    port: 8000,
    host: '0.0.0.0',
  })

  server.ext('onRequest', (request, h) => {
    request.events.on('disconnect', () => { console.log('disconnect') });
    request.events.on('finish', () => { console.log('finish') });

    return h.continue;
  })

  server.route({
    method: method,
    path: '/test',
    handler: async (request, h) => {
      await setTimeout(1000);
      return h.response();
    },
  });
  
  await server.start();

  const socket = new net.Socket();
  socket.connect({
    host: server.info.host,
    port: server.info.port,
  });

  await new Promise ((resolve) => {
    socket.on('connect', function () {
      socket.write(`${method} /test HTTP/1.1\r\n\r\n`, async () => {
        socket.destroy();
        await setTimeout(100);
        await server.stop();
        resolve();
      });
    });
  });
  console.log('');
}

(async () => {
  await check('GET');
  await check('POST');
  await check('PUT');
  await check('DELETE');
})();

What was the result you got?

Node 16:

GET
disconnect

POST
finish

PUT
finish

DELETE
finish

Node 15:

GET
disconnect

POST
finish
disconnect

PUT
finish
disconnect

DELETE
finish
disconnect

What result did you expect?

I expected the disconnect event to fire in all cases.

@FaHeymann FaHeymann added the support Questions, discussions, and general support label Nov 9, 2021
@kanongil
Copy link
Contributor

kanongil commented Nov 9, 2021

Thanks for the report. Node v16 has a change that causes issues for requests with payloads. We have worked to support it fully, but it seems that it is still causing trouble.

@kanongil
Copy link
Contributor

kanongil commented Nov 10, 2021

Looking at the docs, hapi does not guarantee that 'disconnect' is called. In fact the v15 behaviour could be considered a bug:

'disconnect' - emitted when a request errors or aborts unexpectedly.

Maybe you are using the event for something that it is not intended to handle?

Edit: I misread your code. A 'disconnect' event should definitely happen.

@kanongil kanongil added bug Bug or defect and removed support Questions, discussions, and general support labels Nov 10, 2021
@kanongil
Copy link
Contributor

kanongil commented Nov 10, 2021

Hmm, this could be a node bug. It seems that http.IncomingMessage no longer emits 'aborted' events once the payload has been delivered. It also seems that node will now deliver 'error' emits (with Error: aborted) after the 'close' event!

It seems node wants to deprecate 'aborted'.

Working on a patch that works around this issue by not relying on the 'aborted' event (or property).

@kanongil
Copy link
Contributor

There does not appear to be a non-hacky way to work around this issue with node v16+. I have reported the underlying issue here: nodejs/node#40775.

@ronag
Copy link

ronag commented Nov 10, 2021

It also seems that node will now deliver 'error' emits (with Error: aborted) after the 'close' event!

Do you have a repro for that? It shouldn't happen.

@kanongil
Copy link
Contributor

It also seems that node will now deliver 'error' emits (with Error: aborted) after the 'close' event!

Do you have a repro for that? It shouldn't happen.

I don't think it actually does. I likely misread the initial testing I was doing.

Cruikshanks added a commit to DEFRA/water-abstraction-import that referenced this issue Jun 8, 2023
https://eaflood.atlassian.net/browse/WATER-4039

Phew! Somehow when looking into why there were errors during an import process we've ended up needing to upgrade Hapi!

In order to better debug issues with the NALD import process we wanted to add a new POST endpoint that would allow us to trigger it manually. But when we did we saw the request logged but nothing was actually firing. The handler would not be called and the request would not disconnect leaving Postman whirring away.

We quickly spotted `GET` requests were fine it was just `POST` failing. Hmm 🤔

Some Googling later and we found we were not alone

- [POST request not completing on postman](https://stackoverflow.com/a/72642389/6117745)
- [Hapi js server does not respond to my post route](https://www.reddit.com/r/node/comments/syq6t0/hapi_js_server_does_not_respond_to_my_post_route/?utm_source=share&utm_medium=web2x&context=3)
- [hapijs/hapi#4315 disconnect event does not trigger for certain HTTP methods under node v16)

It seems the root cause is running Hapi on Node v16 which we are doing in our local dev environment. In our AWS environments we're running v14 but we intend to upgrade as soon as SROC supplementary billing is shipped.

So, we know this project is fine in AWS running v14 which concurs with those posts. But we also know apps like [water-abstraction-service](https://github.com/DEFRA/water-abstraction-service) are handling `POST` requests fine in our local dev environment. The difference is the version of Hapi; they are running v21 but the import service is running v18.

This change updates Hapi to v21 (and anything else needed along the way) in order to resolve this issue and get the project ready for upgrading all our environments to v16.
Cruikshanks added a commit to DEFRA/water-abstraction-import that referenced this issue Jun 8, 2023
https://eaflood.atlassian.net/browse/WATER-4039

Phew! Somehow when looking into why there were errors during an import process we've ended up needing to upgrade Hapi!

In order to better debug issues with the NALD import process we wanted to add a new POST endpoint that would allow us to trigger it manually. But when we did we saw the request logged but nothing was actually firing. The handler would not be called and the request would not disconnect leaving Postman whirring away.

We quickly spotted `GET` requests were fine it was just `POST` failing. Hmm 🤔

Some Googling later and we found we were not alone

- [POST request not completing on postman](https://stackoverflow.com/a/72642389/6117745)
- [Hapi js server does not respond to my post route](https://www.reddit.com/r/node/comments/syq6t0/hapi_js_server_does_not_respond_to_my_post_route/?utm_source=share&utm_medium=web2x&context=3)
- [request disconnect event does not trigger for certain HTTP methods under node v16](hapijs/hapi#4315)

It seems the root cause is running Hapi on Node v16 which we are doing in our local dev environment. In our AWS environments, we're running v14 but we intend to upgrade as soon as SROC supplementary billing is shipped.

So, we know this project is fine in AWS running v14 which concurs with those posts. But we also know apps like [water-abstraction-service](https://github.com/DEFRA/water-abstraction-service) are handling `POST` requests fine in our local dev environment. The difference is the version of Hapi; they are running v21 but the import service is running v18.

This change updates Hapi to v21 (and anything else needed along the way) in order to resolve this issue and get the project ready for upgrading all our environments to v16.

The upgrade from version 18->19 introduces a breaking change as per the v19 release notes hapijs/hapi#4017 - _"No joi schema compiler is loaded by default and must be explicitly loaded"_. 

This breaking change produces the error `error: AssertError: Cannot set uncompiled validation rules without configuring a validator` which is fixed by explicitly loading joi.

**Notes**

- Bump `@hapi/hapi` 3 major versions
- Fix issues caused by updating `@hapi/hapi` (The upgrade from version 18->19 introduces a breaking change "No joi schema compiler is loaded by default and must be explicitly loaded". This commit fixes that issue as per the v19 release notes: hapijs/hapi#4017)

---------

Co-authored-by: Jason Claxton <30830544+Jozzey@users.noreply.github.com>
Co-authored-by: Jason Claxton <jason.claxton@defra.gov.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug or defect
Projects
None yet
Development

No branches or pull requests

3 participants