You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
/kind feature
I want sequence node to merge the responses from steps(could select which steps).
But I can't find a way to do it.
according to document and cmd/router/main.go
Ensemble: routes the request to multiple models and then merge the responses
and in main.go, Ensemble Node use same input in gorountines
and I observe that sequence node only return result from final step, then check main.go.
for i := range currentNode.Steps {
...
responseBytes, statusCode, err = executeStep(step, graph, request, headers)
...
}
return responseBytes, statusCode, nil
I suppose there is no such feature. (or is there any workaround?)
Describe the solution you'd like
specify responses to be merged in yaml file of InferenceGraph sequence node.
add a field in InferenceStep
beMerged: bool (optional)(if not explictly set true, response from this step is not merged)
and let Sequence Node collects these responses and merge them maybe like Ensemble Node
{${STEP_NAME}: ${RESPONSE_FROM_STEP}, ...}
if no other step response wanted, maintain same structure
{
"prediction": "3"
}
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
Links to the design documents:
[Optional, start with the short-form RFC template to outline your ideas and get early feedback.]
[Required, use the longer-form design doc template to specify and discuss your design in more detail]
The text was updated successfully, but these errors were encountered:
/kind feature
I want sequence node to merge the responses from steps(could select which steps).
But I can't find a way to do it.
according to document and cmd/router/main.go
Ensemble: routes the request to multiple models and then merge the responses
and in main.go, Ensemble Node use same input in gorountines
and I observe that sequence node only return result from final step, then check main.go.
I suppose there is no such feature. (or is there any workaround?)
Describe the solution you'd like
specify responses to be merged in yaml file of InferenceGraph sequence node.
add a field in InferenceStep
beMerged: bool (optional)(if not explictly set true, response from this step is not merged)
and let Sequence Node collects these responses and merge them maybe like Ensemble Node
{${STEP_NAME}: ${RESPONSE_FROM_STEP}, ...}
Example:
if some step response is wanted
stepName could be empty, then use index(which start from 0) instead
if no other step response wanted, maintain same structure
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
Links to the design documents:
[Optional, start with the short-form RFC template to outline your ideas and get early feedback.]
[Required, use the longer-form design doc template to specify and discuss your design in more detail]
The text was updated successfully, but these errors were encountered: