Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support suspending InferenceService #3675

Open
tenzen-y opened this issue May 10, 2024 · 1 comment
Open

Support suspending InferenceService #3675

tenzen-y opened this issue May 10, 2024 · 1 comment
Assignees

Comments

@tenzen-y
Copy link
Member

tenzen-y commented May 10, 2024

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]
Similar to batch/v1 Job, I would like to support suspending InferenceService.
The suspending semantics allows us to appropriately manage capacity planning in the hybrid cluster (mixed cluster of Training / Inference).

Also, the suspending feature allows us to manage all capacity allocation by Kueue.

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

xref: kubernetes-sigs/kueue#1603

Links to the design documents:
[Optional, start with the short-form RFC template to outline your ideas and get early feedback.]
[Required, use the longer-form design doc template to specify and discuss your design in more detail]

@tenzen-y
Copy link
Member Author

@yuzisun As we discussed in the last KubeCon, I would like to support Suspending semantics.
cc: @terrytangyuan I heard that you are interested in this feature in another place.

I will create the design doc, first.
/assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant