Replies: 1 comment 5 replies
-
Hello and thanks for the report. Background: Robustness against terminated R parallel process is still on the roadmap for Futureverse. As you've noticed, there are many feature requests around this topic. There are lots of things that need to be in place to handle this, e.g. we need to figure how it should work if a backend does not support it. Then there will be requests for automatically retrying, etc. For now, we "only" detect failed workers and throw That said, I think in the case of FWIW, there's work like mirai, for which there will be a future.mirai backend, that, part of the core framework, attempts to autostart parallel tasks when they fail. future.mirai is "around the corner" - we just need mirai to settle in and find it's true identity as it's under heavy, active development right now. |
Beta Was this translation helpful? Give feedback.
-
This is related to
It is complicated slightly by the use of the
promises
package, so I cannot just callvalue
to regain the lost forked process.I am using
promises
to orchestrate amulticore
scenario: when I set (say)workers=20
, I need to be able to queue up more than 20 without freezing the main process. I might try to keep track of this manually, or I can usepromises::future_promise
to do the orchestration for me.Reprex:
This is emulating a real-world use-case where occasionally the child process dies unexpectedly.
Unsurprisingly, when I
ps_kill
the process, I am passed the error:but then checking the number of free processes, it is still decremented:
I have found that generally (I cannot yet say "always") I can run
resetWorkers(plan())
to regain this:but in unsupervised use, when I may not know exactly when a child process dies, I may need to try to harvest failed workers preemptively. For that,
resetWorkers
errors.I can easily catch this (
try
,tryCatch
).Questions:
try(resetWorkers(plan()), silent=TRUE)
?workers
tasks does not freeze the main process, and I have better/easier awareness and control over the situation of failed processes?Beta Was this translation helpful? Give feedback.
All reactions