Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DataSource and TypedLocalObjectReference #67087

Merged
merged 11 commits into from Aug 29, 2018

Conversation

xing-yang
Copy link
Contributor

@xing-yang xing-yang commented Aug 7, 2018

What this PR does / why we need it:
This PR adds TypedLocalObjectReference in the core API and adds DataSource in PersistentVolumeClaimSpec.

It also enables feature gate for VolumeSnapshotDataSource.

This is part of the CSI snapshot design proposal to support restoring a volume from a snapshot:
kubernetes/community#2495

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
kubernetes/enhancements#177

Special notes for your reviewer:

Release note:

Added support to restore a volume from a volume snapshot data source. 

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 7, 2018
@k8s-github-robot k8s-github-robot added the kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API label Aug 7, 2018
@xing-yang
Copy link
Contributor Author

@jingxu97

@xing-yang
Copy link
Contributor Author

@saad-ali

@neolit123
Copy link
Member

/ok-to-test
/sig api-machinery
/sig storage

@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/storage Categorizes an issue or PR as relevant to SIG Storage. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 7, 2018
@fedebongio
Copy link
Contributor

/assign @lavalamp

xing-yang added a commit to xing-yang/external-provisioner that referenced this pull request Aug 10, 2018
This PR adds changes to support restoring a volume from a snapshot.

Note: Mark it WIP until the DataSource addition in core API
(kubernetes/kubernetes#67087) and snapshot API/controller changes
(kubernetes-csi/external-snapshotter#7) are merged.
@xing-yang xing-yang force-pushed the datasource branch 2 times, most recently from 61fc09a to 0a59a42 Compare August 16, 2018 20:16
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 18, 2018
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 19, 2018
@saad-ali
Copy link
Member

/lgtm
Looks good from sig-storage side.
Needs API Approval. CC @kubernetes/api-approvers
/assign @thockin

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 20, 2018
Copy link
Member

@thockin thockin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is missing the readiness gate portion. Do you really want to merge this without that?

The risk is that you don't get the readiness done in time, and the release cuts with a half-cooked feature. We're trying to do less in-progress development in master to avoid this problem.

Should this wait for the readiness feature to be mergeable? Either as part of this PR or as a pre-req for this.

Name string
// Kind of the referent.
// +optional
Kind string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lavalamp @kubernetes/sig-api-machinery-api-reviews I need a consult.

Is "kind" sufficient here? In general a kind holds something like "Endpoints" while apiVersion holds "extensions/v1beta1". In a context like this, I don't care which API version, but I do care which group.

Copy link
Member

@liggitt liggitt Aug 21, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kind, API group, and name are the three bits of info needed to unambiguously reference a particular object (assuming you don't care what API version the thing chasing the reference chooses to use).

for example:

// RoleRef contains information that points to the role being used
type RoleRef struct {
// APIGroup is the group for the resource being referenced
APIGroup string `json:"apiGroup" protobuf:"bytes,1,opt,name=apiGroup"`
// Kind is the type of resource being referenced
Kind string `json:"kind" protobuf:"bytes,2,opt,name=kind"`
// Name is the name of resource being referenced
Name string `json:"name" protobuf:"bytes,3,opt,name=name"`
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Jordan, that's what I assumed. I saw that ObjectReference has a field "APIVersion" which seemed wrong... I didn't know this struct, but it makes more sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added APIGroup.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we will add the APIGroup in the struct. I assumed it is required and user has to specify the APIGroup here for the data source.

@xing-yang
Copy link
Contributor Author

@thockin We have another PR on the readiness gate depending on this PR: #67625. Do you want us to combine them into one PR? Thanks.

@xing-yang
Copy link
Contributor Author

The pull-kubernetes-kubemark-e2e-gce-big test failure was not related to this PR. I'll re-run it.
Looks like another feature gate change just got merged, so I'll need to resolve the conflict and re-submit.

@xing-yang
Copy link
Contributor Author

xing-yang commented Aug 28, 2018

@thockin Jing and I discussed about this and want to make the following changes to the DataSource comments. Let us know what you think. Thanks!

// This field requires the VolumeSnapshotDataSource alpha feature gate to be enabled and
// currently VolumeSnapshot is the only supported data source. If the provisioner can support
// VolumeSnapshot data source, it will create a new volume and data will be restored to the volume
// at the same time. If the provisioner does not support VolumeSnapshot data source, the volume
// will not be created. In the future, we plan to support more data source types and the behavior of
// the provisioner may change.

@jingxu97
Copy link
Contributor

@thockin The comment we have now only describes the provisioner behavior for the only supported data source type, VolumeSnapshot. For this case, the provisioner will create volume and put data in volume at the same time. For the data sources that will be added in the future, it might require an external data populator to handle the data copy. In this case, things will get more complicated. For example, although provisioner does not fully support the data source (it cannot copy data directly), it still creates the volume and let the data populator to handle the data copying. In case of data copy failure, an empty volume might be left there but PVC should be marked as not ready. We need more discussion to finalize the design and cover all the cases. When that feature is ready, we will come back and change the comments here. Please let us know if there is any problem with it. Thank you!

@xing-yang
Copy link
Contributor Author

/test pull-kubernetes-e2e-gce

@xing-yang
Copy link
Contributor Author

/test pull-kubernetes-kubemark-e2e-gce-big

@thockin
Copy link
Member

thockin commented Aug 29, 2018

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Aug 29, 2018
Copy link
Member

@saad-ali saad-ali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: saad-ali, thockin, xing-yang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@saad-ali
Copy link
Member

/milestone v1.12

@k8s-ci-robot k8s-ci-robot added this to the v1.12 milestone Aug 29, 2018
@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here: https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md.

@k8s-github-robot k8s-github-robot merged commit d97ece0 into kubernetes:master Aug 29, 2018
xing-yang added a commit to xing-yang/external-provisioner that referenced this pull request Aug 30, 2018
This PR adds changes to support restoring a volume from a snapshot.

Note: The DataSource addition in core API
(kubernetes/kubernetes#67087) and snapshot API/controller changes
(kubernetes-csi/external-snapshotter#7) are merged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/storage Categorizes an issue or PR as relevant to SIG Storage. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet