Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Release the input futures as soon as we submit the combiner task. But…
… really, redo how we release resources in general. This is a followup to CL 265489523, which "only" released the input futures as soon as the combiner task finished running (which had often happened even before that CL but hadn't if the combiner returned a Future that was still pending). That CL was good enough for practical purposes, but I wanted to better understand how we release resources. This CL standardizes on AggregateFuture.releaseResources() as the way to null out all fields[*], merging logic from releaseResources(), releaseResourcesAfterFailure(), and AsyncCallableInterruptibleTask.setValue(...). As part of that, it merges AggregateFuture and AggregateFutureState/RunningState into a single object. [*] OK, except seenExceptions, which gets its own handling. As a bonus, I believe that this CL clears seenExceptions earlier than it used to be cleared in the CombinedFuture case. Specifically, it clears it when all inputs are done, rather than when the combiner task has finished running. It turns out that blindly nulling out fields is too aggressive, so we need to be careful in 2 cases: 1. CombinedFuture.releaseResources() can't null out `task` until the future is done or the task is done running. That's because it may need to interrupt the task. To handle this, I don't null out `task` in releaseResources() unless isDone(). To ensure that `task` still gets nulled out as soon as it's done running, I null it out directly in afterRanInterruptibly(). (OK, this is another exception to my claim that releaseResources() handles nulling out "all" fields....) 2. Even if the output future is done, processCompleted() sometimes needs access to the original futures in order to see whether any of them failed. To handle this, I store them in the listener and pass them through to processCompleted() (when necessary). The changes to prod code are net negative in line count, at least ignoring the added comments. I've also added a couple tests, only one of which passed before this CL. And I think the model for when fields are nulled out is overall clearer after this CL. So hopefully this CL is a step forward, despite the complexity of the changes and the remaining complexity in the code. (I also included a few unrelated simplifications, like not bothering to check collectsValues before calling collectOneValue(...).) (Aside: This CL's releaseResources() is like our proposed afterCommit() API but different. First, releaseResources() may be called even before set() or setAsync() in the CombinedFuture case. Second, CombinedFuture may rely on the fact that it's called twice in some cases: It's called once when all inputs complete, but it doesn't null out `task`, and then it can be called again if the output is cancelled, at which point it *does* null out `task`. But that probably doesn't matter too much because the task was probably handed to an executor in the meantime, so CombinedFuture is unlikely to hold the final reference to it. Anyway, for more discussion of afterCommit(), see #2886) [] RELNOTES=n/a ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=266130757
- Loading branch information