Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metadata to kubelet eviction event annotations #64213

Merged

Conversation

dashpole
Copy link
Contributor

What this PR does / why we need it:
Add annotations to kubelet eviction events. Annotations include
"offending_containers" : comma-seperated list of containers.
"offending_containers_usage": comma-seperated list of usage.
"starved_resource": v1.ResourceName of the starved resource

Special notes for your reviewer:
Adding annotations to events required changing the EventRecorder interface to add a AnnotatedEventf function, which can add annotations to an event.

Release note:

NONE

/assign @dchen1107
cc @mwielgus @schylek @kgrygiel

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels May 23, 2018
@dashpole
Copy link
Contributor Author

/sig node
/kind feature

@k8s-ci-robot k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. kind/feature Categorizes issue or PR as related to a new feature. labels May 23, 2018
@dashpole
Copy link
Contributor Author

/retest

@dchen1107
Copy link
Member

I am ok that you introduce an annotation to the event object, and also ok to have some structure to the annotation message for eviction event. The question is that if you are expecting VPA or other tool to parse that message to make some controlling decision. If yes, could you please add a test with a comment, so that later the other developer would break that dependency without any warning.

LGTM

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 23, 2018
@dashpole
Copy link
Contributor Author

@dchen1107 I added e2e_node testing for this to the eviction tests. It checks that for each evicted pod:

  • The pod has an event with event.Reason == "Evicted"
  • The event has an annotation with Key: "starved_resource", Value: the expectedStarvedResource for that test.
  • The event has an annotation with Key: "offending_containers", Value: The name of each offending container in the pod.
  • The event has an annotation with Key: "offending_containers_usage", Value: The usage of each offending container in the pod. The value must be able to be parsed as a Quantity, and must be larger than the container's request.

@dashpole
Copy link
Contributor Author

/retest

@schylek
Copy link
Contributor

schylek commented May 29, 2018

@dchen1107 Can we get this change in despite code slush?

@dchen1107
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 29, 2018
@dchen1107 dchen1107 added this to the v1.11 milestone May 29, 2018
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dashpole, dchen1107

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 29, 2018
@dashpole
Copy link
Contributor Author

/priority critical-urgent

@k8s-ci-robot k8s-ci-robot added the priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. label May 29, 2018
@dchen1107 dchen1107 added status/approved-for-milestone and removed priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. labels May 29, 2018
@dashpole
Copy link
Contributor Author

/priority critical-urgent

@k8s-ci-robot k8s-ci-robot added the priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. label May 29, 2018
@k8s-github-robot
Copy link

[MILESTONENOTIFIER] Milestone Pull Request: Up-to-date for process

@dashpole @dchen1107

Pull Request Labels
  • sig/node: Pull Request will be escalated to these SIGs if needed.
  • priority/critical-urgent: Never automatically move pull request out of a release milestone; continually escalate to contributor and SIG through all available channels.
  • kind/feature: New functionality.
Help

@dashpole
Copy link
Contributor Author

/retest

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel comment for consistent failures.

@k8s-github-robot
Copy link

/test all [submit-queue is verifying that this PR is safe to merge]

@k8s-github-robot
Copy link

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit 15cd355 into kubernetes:master May 30, 2018
@dashpole dashpole deleted the eviction_event_annotation branch May 30, 2018 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note-none Denotes a PR that doesn't merit a release note. sig/node Categorizes an issue or PR as relevant to SIG Node. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants