What does mean "size of prediction is not equal to the size of instances" in batcher? #3540
Unanswered
hyunjoon1015
asked this question in
Q&A
Replies: 1 comment
-
It means number of inputs not equal to the number of predictions. That is for example, if you provide three instances(or inputs), the prediction should contain three output values. During batching multiple inputs are combined an sent as a single request and if the prediction output does not match the input length, you will get this error. For this 3 instances {
"instances": [
[1, 2],
[1, 2],
[1, 2]
]
} you should get 3 prediction like below, but if it does not match the input length you will get the error. {
"predictions": [1, 2, 3]
} Note: This is just an example input format. The format may change depending on how the model consumes the input. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello guys,
I want to use batcher in my model prediction, so I'm following custom model serving guide ( https://kserve.github.io/website/0.9/modelserving/v1beta1/custom/custom_model/#create-your-custom-model-server-by-extending-model-class)
But when I add batcher in isvc yaml and try to send request, curl command gets response like,
In case of batcher agent container, it looks working fine.
The log of agent batcher was,
So, what does "size of prediction is not equal to the size of instances" mean??
Beta Was this translation helpful? Give feedback.
All reactions