bug(hubble): Hubble restarts periodically due to "neon::types::Deferred` was dropped without being settled"

[manan19]

What is the bug?

Hubble restarts periodically with the following shutdown log

hubble-hubble-1  | {"message":"{\"level\":50,\"time\":1715696876137,\"pid\":42,\"hostname\":\"562c55704677\",\"reason\":\"Unhandled Rejection\",\"err\":{\"type\":\"Error\",\"message\":\"`neon::types::Deferred` was dropped without being settled\",\"stack\":\"Error: `neon::types::Deferred` was dropped without being settled\"},\"msg\":\"shutting down hub\"}\n","timestamp":"2024-05-14T14:27:56.137Z","type":"out","process_id":0,"app_name":"hubble"}

How can it be reproduced? (optional)
Not sure

Additional context (optional)
Some of the times it seems to coincide with pruning but it doesn't always restart at every pruning.

[sds]

Not 100% sure what the cause is, but based on these docs we either need to:

• Figure out where we're not settling the Deferred object
• Handle unhandled promise rejections (i.e. don't rely on the default Node.js behavior of terminating the process)