You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have a long-running Bull job which processes a big CSV file, validates each line and creates a new Bull job for every row in the CSV for further processing.
When we send SIGTERM signal to the process running this job, we ideally want that it should not process any new jobs, but still complete the pending job and then close the app.
My suggestion is to do queue.pause(true) to only pause the processing of new jobs locally, but still allow adding new jobs. Or provide an option in the BullModuleOptions/BullModuleAsyncOptions to control this behaviour.
Copy the process ID from process listening on ... log
Run the script sh test.sh <pid>
The test.sh script will publish a job in the "long-job" queue, which will internally start creating a job in the "internal-jobs" queue. It will create one job every second for the next 10 seconds. After 3 seconds the test.sh will kill the node process, which triggers the error.
Expected behavior
We shouldn't get the Connection is closed error. Instead the internal jobs should get created successfully. The node process should only stop processing any new jobs, it should still be allowed to add new jobs.
Package version
10.1.1
Bull version
4.12.2
NestJS version
10.0.0
Node.js version
20.10.0
In which operating systems have you tested?
macOS
Windows
Linux
Other
No response
The text was updated successfully, but these errors were encountered:
Allows customising the onApplicationShutdown behaviour by providing a
callback in the `BullModuleOptions`. For eg. if someone doesn't want to
`close()` the queues instead just `pause(true)` them
Resolvesnestjs#2069
Is there an existing issue for this?
Current behavior
We have a long-running Bull job which processes a big CSV file, validates each line and creates a new Bull job for every row in the CSV for further processing.
When we send
SIGTERM
signal to the process running this job, we ideally want that it should not process any new jobs, but still complete the pending job and then close the app.But we observed that if we try to create any new jobs inside the active job after
SIGTERM
event, it fails withError: Connection is closed
redis error. We believe that's becausequeue.close()
gets called on application shutdown -https://github.com/nestjs/bull/blob/197079daa3400b9d0b487a5ace7f27f12762b920/packages/bull/lib/bull.providers.ts#L43C3-L47C5
My suggestion is to do
queue.pause(true)
to only pause the processing of new jobs locally, but still allow adding new jobs. Or provide an option in theBullModuleOptions
/BullModuleAsyncOptions
to control this behaviour.Minimum reproduction code
https://github.com/akshatflx/bull-graceful-shutdown
Steps to reproduce
process listening on ...
logtest.sh <pid>
The test.sh script will publish a job in the "long-job" queue, which will internally start creating a job in the "internal-jobs" queue. It will create one job every second for the next 10 seconds. After 3 seconds the test.sh will kill the node process, which triggers the error.
Expected behavior
We shouldn't get the Connection is closed error. Instead the internal jobs should get created successfully. The node process should only stop processing any new jobs, it should still be allowed to add new jobs.
Package version
10.1.1
Bull version
4.12.2
NestJS version
10.0.0
Node.js version
20.10.0
In which operating systems have you tested?
Other
No response
The text was updated successfully, but these errors were encountered: