Fork Join Parallelism for Workflows

[QuietRocket]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

1. Is this request related to a challenge you're experiencing?

Some workflow configurations have high response latency because they necessitate multiple orthogonal requests to execute sequentially. The latency can be greatly reduced via simple fork-join parallelism. For example, multiple LLM generations in parallel with HTTP requests.

2. Describe the feature you'd like to see

Add Fork and Join blocks.

• Place a Fork anywhere an LLM/HTTP block could be placed.
• The output is a series of pathways which can be executed in parallel.
• In a pathway, only variables defined before the Fork can be accessed.
• Pathways potentially have a constrained selection of blocks. For example, for chat workflows, "Answer" blocks might need to be disabled since execution order isn't deterministic.
• In order for the configuration to be valid, all parallel pathways which originate from a Fork must end up in a Join block.
• Join block waits for all pathways to finish, then continue sequential execution.

3. How will this feature improve your workflow or experience?

Improves response latency by performing orthogonal tasks in parallel.

4. Additional context or comments

Fork Join diagram courtesy of this paper.

5. Can you help us with this feature?

• I am interested in contributing to this feature.