Move volume dynamic provisioning scheduling to beta #67432

lichuqiang · 2018-08-15T11:40:43Z

What this PR does / why we need it:

Combine feature gate VolumeScheduling and DynamicProvisioningScheduling into one
Add allowedTopologies description in kubectl

Special notes for your reviewer:
Wait until related e2e and downside plugins are ready.

/hold

Release note:

Move volume dynamic provisioning scheduling to beta (ACTION REQUIRED: The DynamicProvisioningScheduling alpha feature gate has been removed. The VolumeScheduling beta feature gate is still required for this feature)

lichuqiang · 2018-08-15T11:41:26Z

/sig storage
/kind feature
/assign @msau42

msau42 · 2018-08-15T16:57:40Z

pkg/controller/volume/persistentvolume/pv_controller.go

+	// Scheduler signal to the PV controller to start dynamic
+	// provisioning by setting the "annSelectedNode" annotation
+	// in the PVC
+	if _, ok := claim.Annotations[annSelectedNode]; ok {


The annotation needs to move to beta too. I would actually not prefix the annotation with "beta" either, it will make going to GA easier.

msau42 · 2018-08-16T20:50:27Z

pkg/printers/internalversion/describe.go

@@ -3482,6 +3485,38 @@ func describeStorageClass(sc *storage.StorageClass, events *api.EventList) (stri
 	})
 }

+func printAllowedTopologies(w PrefixWriter, topologies []api.TopologySelectorTerm) {


Can you paste an example of the output?

Similar to that of node affinity

msau42 · 2018-08-23T23:48:16Z

/lgtm

We need to wait for these PRs to go in first:
#66843
#67530

msau42 · 2018-08-23T23:52:59Z

/assign @liggitt @saad-ali

liggitt · 2018-08-24T02:33:55Z

pkg/features/kube_features.go

@@ -396,7 +390,6 @@ var defaultKubernetesFeatureGates = map[utilfeature.Feature]utilfeature.FeatureS
 	RunAsGroup:                                  {Default: false, PreRelease: utilfeature.Alpha},
 	VolumeSubpath:                               {Default: true, PreRelease: utilfeature.GA},
 	BalanceAttachedNodeVolumes:                  {Default: false, PreRelease: utilfeature.Alpha},
-	DynamicProvisioningScheduling:               {Default: false, PreRelease: utilfeature.Alpha},


shouldn't this change to true and beta, not be removed completely?

We want to merge this with the VolumeScheduling feature gate, which is already beta

According to the feature gate discussion here, when a flag goes GA, we have to deprecate it before removing, to support upgrade cases.

However, this is an alpha feature gate, so we don't support upgrade. I think it is fine to remove/merge it in this case. @liggitt wdyt?

if the gate never graduated from alpha, we can drop it on a release boundary (though it's release-note worthy).

based on offline discussion, the code gated by DynamicProvisioningScheduling is also opt-in based on storageclass attributes. if that's the case, I see no downside to collapsing onto the VolumeScheduling

Making sure all entry points to the feature are covered by:

VolumeScheduling feature gate

Opt-in behavior via StorageClass.VolumeBindingMode field

Scheduler:

Feature gate check is at the beginning of the predicate

In predicate call, we check for delayed binding setting and treat it as legacy (immediate binding) scenario otherwise

Assume/bind volumes are feature gated

PV controller:

Defer to the scheduler is controlled by feature gate and StorageClass check

API server:

StorageClass field controlled by feature gate

@lichuqiang can you update the release note saying that the feature gate has been merged with VolumeScheduling.

sorry being nitpicky, can you word it as "ACTION REQUIRED: The DynamicProvisioningScheduling alpha feature gate has been removed. The VolumeScheduling beta feature gate is still required for this feature"

msau42 · 2018-08-28T18:02:41Z

The other two prs have merged, so this can go in.
/hold cancel

msau42 · 2018-08-28T18:03:29Z

/lgtm

…ng into one

msau42 · 2018-08-29T17:03:05Z

/lgtm

/assign @saad-ali @liggitt
for approval

saad-ali

/approve
/milestone v1.12

liggitt · 2018-08-29T20:11:56Z

/approve

k8s-ci-robot · 2018-08-29T20:12:04Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lichuqiang, liggitt, msau42, saad-ali

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~api/OWNERS~~ [liggitt]
~~docs/api-reference/OWNERS~~ [liggitt]
~~pkg/apis/OWNERS~~ [liggitt]
~~pkg/controller/volume/persistentvolume/OWNERS~~ [msau42,saad-ali]
~~pkg/features/OWNERS~~ [liggitt,saad-ali]
~~pkg/printers/OWNERS~~ [liggitt]
~~pkg/volume/util/OWNERS~~ [saad-ali]
~~plugin/pkg/auth/OWNERS~~ [liggitt]
~~staging/src/k8s.io/api/OWNERS~~ [liggitt]
~~test/integration/scheduler/OWNERS~~ [liggitt,saad-ali]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-github-robot · 2018-08-29T22:19:32Z

Automatic merge from submit-queue (batch tested with PRs 67745, 67432, 67569, 67825, 67943). If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

The mentioned feature gate is now collapsed into `VolumeScheduling`. xref: kubernetes/kubernetes#67432

* Update docs for fields allowed at root of CRD schema (#9973) * add plugin docs and examples (#10053) * docs update to promote TaintNodesByCondition to beta (#9626) * HPA Specificity Improvements (#8757) Updated the HPA docs to reference the `autoscaling/v2beta2` API version, and added documentation about the new fields. * adjust docs for pod ready++ (#10049) * Remove --cadvisor-port - has been deprecated since v1.10 (#10023) Change-Id: Id2a685473a243aef492a98ff450759f39e362557 * Add Documentation for Snapshot Feature (#9948) * Add documentation for snapshot feature * Update volume-snapshots.md * Add dry-run to api-concepts (#10033) * kubeadm-init: Update the offline support section (#10062) The update includes the following things (in mind with Kubernetes 1.12): - Remove the 1.8 image versions - Add the 1.10 image versions that were missing until now - Include a comment for the missing arch suffixes in 1.12 Signed-off-by: Rostislav M. Georgiev <rostislavg@vmware.com> * Say bye to `DynamicProvisioningScheduling` (#10157) The mentioned feature gate is now collapsed into `VolumeScheduling`. xref: kubernetes/kubernetes#67432 * Update ResourceQuota per PriorityClass state for 1.12 (#10229) * TokenRequest and TokenRequestProjection now beta (#10161) xref: kubernetes/kubernetes#67349 * Change feature state for kms provider to beta. (#10230) KMS Provider will be graduating to beta in v1.12, reflecting this change on the website. * coredns default (#10200) * Promote ShareProcessNamespace to beta in docs (#9996) * Add CoreDNS details to DNS Debug docs (#10201) * add coredns details * address nits, add query logging section * Update docs with topology aware dynamic provisioning (#9939) * Document topology aware volume binding feature * update for readability * Update storage-classes.md * comma splice * don't abbreviate * HPA Algorithm Information Improvements (#9780) * Update HPA docs with more algorithm details The HPA docs pointed to an out-of-date document for information on the algorithm details, which users were finding confusing. This sticks a section on the algorithm in the HPA docs instead, documenting both general behavior and corner cases. * Add glossary info, HPA docs on quantities People often ask about the quantity notation when working with the metrics APIs, so this adds a glossary entry on quantities (since they're used elsewhere in the system), and a short explantation in the HPA walkthough. * Information about HPA readiness and stabilization This adds information about the new changes to HPA readiness and stabilization from kubernetes/enhancements#591, and other minor changes that landed in Kubernetes 1.12. * Update horizontal-pod-autoscale.md * Audit 1.12 doc (#9953) * audit 1.12 document * remove legacy audit feature kubernetes/kubernetes#65862 * update feature gate doc * MountPropagation is now GA (#10090) * RuntimeClass documentation (#10102) * RuntimeClass documentation * Update runtime-class.md * Add documentation for Scheduler performance tuning (#10048) * Add documentation for Scheduler performance tuning * Update scheduler-perf-tuning.md * TTL controller for cleaning up finished resources (#10064) * TTL controller for cleaning up finished resources * Address comments * Update ttlafterfinished.md * Bump quota configuration api version (#10217) * Incremental update from master (#10278) * fix invalid href of cloud controller manager (#10240) * fix invalid yaml format (#10238) * update storage-limits doc with Azure disk part (#10224) update storage-limits doc with Azure disk part fix comments * Update kubelet-config-file.md (#10222) Update link to KubeletConfiguration struct. * fix a trivial misspelling (#10244) * Fix cassandra-statefulset.yaml indent level (#10243) * Mention minimum etcd versions (#10208) Source: https://groups.google.com/d/msg/kubernetes-dev/jMPA4JzKiY4/HIx2ugvLBAAJ * fix 404 error (#10250) * Small verb tweak (#10190) Present participle, ftw. * Add AnchorJS logic for header links (#10155) * Add AnchorJS JavaScript * Remove existing inpage_heading logic * Remove underline from anchor tags * Use single icon and add touch visibility * Use paragraph link icon for AnchorJS * Update Sass to use code formatting in docsContent headers * Update header size coverage to H3-H6 * fix broken link in kubefed.md (#10254) * Update the version numbers for the X-Remote-Extra- and Impersonate-Extra- key fixes (#9827) The fix was cherry picked into 1.11.3, 1.10.7, and 1.9.11: kubernetes/kubernetes#67162 kubernetes/kubernetes#67163 kubernetes/kubernetes#67164 * fix typo (#10168) * fix typo * addressing comments. * Update setup-ha-etcd-with-kubeadm.md * fix typos (#10252) * fix description of contribute guide (#10253) * describe truncate feature about advanced audit (#10236) * describe truncate feature about advanced audit * Update audit.md * docs update to promote ScheduleDaemonSetPods to beta (#9923) * Dynamic volume limit updates for 1.12 (#10211) * add a placeholder commit * Update docs for csi volume limits * Update storage-limits.md * Add "MayRunAs" value among other GroupStrategies (#9888) * Add CoreDNS details to the customize DNS doc (#10228) * Add CoreDNS details to the customize DNS doc Rewrite the document to include more details about CoreDNS, since it's now the default from v1.12 * Address comments * Improve doc wording * Fix link * Update dns-custom-nameservers.md * Update dns-custom-nameservers.md * Fix secrets docs in 1.12 branch (#10056) * Fix secrets docs * Update secret.md * Revert CoreDNS Docs (#10319) * Revert "Add CoreDNS details to DNS Debug docs (#10201)" This reverts commit 462817a. * Revert "Add CoreDNS details to the customize DNS doc (#10228)" This reverts commit e7319ee. * Revert "coredns default (#10200)" This reverts commit 698e93b. * Add CRI installation instructions page Added cri-installation page with CRI installation instructions Referenced it from kubeadm-init and install-kubeadm pages. * kubeadm: update API types documentation for 1.12 (#10283) v1alpha2 -> v1alpha3 MasterConfiguration -> [new-api-types] * TokenRequest feature documentation (#10295) * AdvancedAuditing is now GA (#10156) xref: kubernetes/kubernetes#65862 `AdvancedAuditing` feature is GA in 1.12. This PR adjusts the related docs. * update runtime-class.md (#10332) * update runtime-class.md * Update runtime-class.md * Document cross-authorizer permissions for creating RBAC roles (#10015) * Document cross-authorizer permissions for creating RBAC roles * Update rbac.md * kubeadm: update authored content for 1.12 (reference docs and cluster creation) (#10348) * kubeadm: update authored content in reference docs for 1.12 * kubeadm: add time frame in create-cluster-kubeadm for 1.12 * add AllowedProcMountTypes and ProcMountType to docs (#9911) Signed-off-by: Jess Frazelle <acidburn@microsoft.com> * kubeadm: add new command line reference (#10306) Add: - placeholder files - include place holder files - include "renew" sub command - add missing tabs for "alpha phase kubelet" * Documenting SCTP support in Kubernetes (#10279) * Documenting SCTP support in Kubernetes Service, Endpoint, NetworkPolicy and Pod * Updates based on comments on the PR * kubectl expose update with SCTP support * Updated according to comments in the PR * Revert "kubectl expose update with SCTP support" This reverts commit 0d5a1e6. * TLS Bootstrap and Server Cert Rotation feature documentation (#10232) * TokenRequest feature documentation * line wrapping to make review not insane * update content for GA without major refactor * Update kubelet-tls-bootstrapping.md * Add clarifications for volume snapshots (#10296) * Update kubadm ha installation for 1.12 (#10264) * Update kubadm ha installation for 1.12 Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * update stable version Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * Update stacked control plane for v1.12 (#2) * use v1alpha3 Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * more v1alpha3 (#4) * updates Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * Document how to run in-tree cloud providers with kubeadm (#10357) Change-Id: Iab6b996a830503d74a6eb0c507c5f8ca7a39235b * kubeadm reference doc for release 1.12 (#10359) * Revert "Revert "Add CoreDNS details to DNS Debug docs (#10201)"" This reverts commit bb30f4d. * Revert "Revert "Add CoreDNS details to the customize DNS doc (#10228)"" This reverts commit bc23d45. * Revert "Revert "coredns default (#10200)"" This reverts commit 7f4350d. * add missing instruction for ha guide (#10374) Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * kubeadm - Ha upgrade updates (#10340) * Update HA upgrade docs * Adds external etcd HA upgrade guide Signed-off-by: Chuck Ha <ha.chuck@gmail.com> * copyedit * more edits * add runasgroup in psp (#10076) * update KubeletPluginsWatcher feature gate (#10205) * generated 1.12 docs * Building Multi-arch images with Manifests (#10379) In 1.12, a variety of images used in a typical kubernetes installation have started to using manifests to better support environments with arm or ppc64le architectures. For example all images used with kubeadm by default have manifests, another would be all the tests in the conformance test suite. Here we capture the best practices for everyone to start using manifests in their own workflows. Change-Id: I5ba4c5fe55ffc9486a8251760f3352be4f2e1494 * Upgrade docs for v1.12 (#10344) * generated assets and docs * remove 1.7 * update 1.12 * update plugin documentation under docs>tasks>extend-kubectl (#10259) * update plugin documentation under docs>tasks>extend-kubectl * Update kubectl-plugins.md

k8s-ci-robot requested review from deads2k and jayunit100 August 15, 2018 11:41

k8s-ci-robot assigned msau42 Aug 15, 2018

k8s-ci-robot added sig/storage Categorizes an issue or PR as relevant to SIG Storage. kind/feature Categorizes issue or PR as related to a new feature. labels Aug 15, 2018

msau42 reviewed Aug 15, 2018

View reviewed changes

msau42 reviewed Aug 16, 2018

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 23, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 23, 2018

k8s-ci-robot assigned liggitt and saad-ali Aug 23, 2018

liggitt reviewed Aug 24, 2018

View reviewed changes

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 28, 2018

k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Aug 28, 2018

k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Aug 29, 2018

lichuqiang added 4 commits August 29, 2018 10:30

combine feature gate VolumeScheduling and DynamicProvisioningScheduli…

b4a57f6

…ng into one

related test update

4c43d62

generated files

795b213

describe allowedTopologies

eefd337

k8s-ci-robot added release-note-action-required Denotes a PR that introduces potentially breaking changes that require user action. and removed release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Aug 29, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 29, 2018

saad-ali approved these changes Aug 29, 2018

View reviewed changes

k8s-ci-robot added this to the v1.12 milestone Aug 29, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 29, 2018

k8s-github-robot merged commit 37b2929 into kubernetes:master Aug 29, 2018

msau42 mentioned this pull request Aug 29, 2018

Remove feature tag from dynamic provisioning topology tests #68034

Merged

tengqm added a commit to tengqm/website that referenced this pull request Aug 31, 2018

Say bye to DynamicProvisioningScheduling

4f0b9f6

The mentioned feature gate is now collapsed into `VolumeScheduling`. xref: kubernetes/kubernetes#67432

tengqm mentioned this pull request Aug 31, 2018

Say bye to DynamicProvisioningScheduling kubernetes/website#10157

Merged

k8s-ci-robot pushed a commit to kubernetes/website that referenced this pull request Sep 10, 2018

Say bye to DynamicProvisioningScheduling (#10157)

dc9d71a

The mentioned feature gate is now collapsed into `VolumeScheduling`. xref: kubernetes/kubernetes#67432

detiber pushed a commit to detiber/kubernetes-website that referenced this pull request Sep 19, 2018

Say bye to DynamicProvisioningScheduling (kubernetes#10157)

5e022aa

The mentioned feature gate is now collapsed into `VolumeScheduling`. xref: kubernetes/kubernetes#67432

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move volume dynamic provisioning scheduling to beta #67432

Move volume dynamic provisioning scheduling to beta #67432

lichuqiang commented Aug 15, 2018 •

edited

lichuqiang commented Aug 15, 2018

msau42 Aug 15, 2018

msau42 Aug 16, 2018

lichuqiang Aug 17, 2018

msau42 commented Aug 23, 2018

msau42 commented Aug 23, 2018

liggitt Aug 24, 2018

msau42 Aug 24, 2018

msau42 Aug 27, 2018

liggitt Aug 28, 2018

msau42 Aug 28, 2018

msau42 Aug 28, 2018

lichuqiang Aug 29, 2018

msau42 Aug 29, 2018

lichuqiang Aug 29, 2018

msau42 commented Aug 28, 2018

msau42 commented Aug 28, 2018

msau42 commented Aug 29, 2018

saad-ali left a comment

liggitt commented Aug 29, 2018

k8s-ci-robot commented Aug 29, 2018

k8s-github-robot commented Aug 29, 2018

Move volume dynamic provisioning scheduling to beta #67432

Move volume dynamic provisioning scheduling to beta #67432

Conversation

lichuqiang commented Aug 15, 2018 • edited

lichuqiang commented Aug 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msau42 commented Aug 23, 2018

msau42 commented Aug 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msau42 commented Aug 28, 2018

msau42 commented Aug 28, 2018

msau42 commented Aug 29, 2018

saad-ali left a comment

Choose a reason for hiding this comment

liggitt commented Aug 29, 2018

k8s-ci-robot commented Aug 29, 2018

k8s-github-robot commented Aug 29, 2018

lichuqiang commented Aug 15, 2018 •

edited