Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release-4.14] UPSTREAM: <carry>: OCPBUGS-32473: fix cpu manager cpuset check #1951

Open
wants to merge 1 commit into
base: release-4.14
Choose a base branch
from

Conversation

openshift-cherrypick-robot

This is an automated cherry-pick of #1915

/assign MarSik

NONE

…k in workload partitioned env

(this can be squashed to  04070bb UPSTREAM: : add management support to kubelet)

Workload partitioning makes the separation between reserved and workload cpus more strict. It is therefore expected the reserved cpus are NOT part of the default cpuset and the existing check was overzealous.

First execution of kubelet after reboot never gets here as the cpuset is computed on line 209. However a kubelet restart without reboot skips this code, recovers from state file and runs the check on line 220.

This was uncovered by decoupling the cpu manager state file cleanup from kubelet restart, doing it only once at reboot as part of OCPBUGS-24366
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: Jira Issue OCPBUGS-31348 has been cloned as Jira Issue OCPBUGS-32473. Will retitle bug to link to clone.
/retitle [release-4.14] UPSTREAM: : OCPBUGS-32473: fix cpu manager cpuset check

In response to this:

This is an automated cherry-pick of #1915

/assign MarSik

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot changed the title [release-4.14] UPSTREAM: <carry>: OCPBUGS-31348: fix cpu manager cpuset check [release-4.14] UPSTREAM: <carry>: OCPBUGS-32473: fix cpu manager cpuset check Apr 19, 2024
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: the contents of this pull request could not be automatically validated.

The following commits could not be validated and must be approved by a top-level approver:

Comment /validate-backports to re-evaluate validity of the upstream PRs, for example when they are merged upstream.

@openshift-ci-robot openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 19, 2024
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: This pull request references Jira Issue OCPBUGS-32473, which is invalid:

  • release note text must be set and not match the template OR release note type must be set to "Release Note Not Required"

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This is an automated cherry-pick of #1915

/assign MarSik

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link

openshift-ci bot commented Apr 19, 2024

@openshift-cherrypick-robot: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-agnostic-ovn-cmd 0c72b20 link false /test e2e-agnostic-ovn-cmd

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@MarSik
Copy link

MarSik commented Apr 23, 2024

/retest-required

@mrunalp mrunalp added backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. and removed backports/unvalidated-commits Indicates that not all commits come to merged upstream PRs. labels Apr 29, 2024
@mrunalp mrunalp added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 29, 2024
@mrunalp
Copy link
Member

mrunalp commented Apr 29, 2024

/jira refresh

@openshift-ci-robot
Copy link

@mrunalp: This pull request references Jira Issue OCPBUGS-32473, which is invalid:

  • release note text must be set and not match the template OR release note type must be set to "Release Note Not Required"

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link

openshift-ci bot commented Apr 29, 2024

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by: openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@MarSik
Copy link

MarSik commented Apr 30, 2024

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Apr 30, 2024
@openshift-ci-robot
Copy link

@MarSik: This pull request references Jira Issue OCPBUGS-32473, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.14.z) matches configured target version for branch (4.14.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-31348 is in the state Closed (Done-Errata), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-31348 targets the "4.15.z" version, which is one of the valid target versions: 4.15.0, 4.15.z
  • bug has dependents

No GitHub users were found matching the public email listed for the QA contact in Jira (schoudha@redhat.com), skipping review request.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@lyman9966
Copy link

/label qe-approved

% oc get node
NAME STATUS ROLES AGE VERSION
ci-ln-dhznmmb-72292-mdmmw-master-0 Ready control-plane,master 101m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-master-1 Ready control-plane,master 101m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-master-2 Ready control-plane,master 101m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-worker-a-n5msl Ready worker 89m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-worker-b-kw284 Ready worker 89m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-worker-c-6d9qw Ready worker 89m v1.27.14+df6456e

% oc label node ci-ln-dhznmmb-72292-mdmmw-worker-c-6d9qw node-role.kubernetes.io/worker-cnf=""
node/ci-ln-dhznmmb-72292-mdmmw-worker-c-6d9qw labeled

% oc create -f - <<EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
name: worker-cnf
labels:
machineconfiguration.openshift.io/role: worker-cnf
spec:
machineConfigSelector:
matchExpressions:
- {
key: machineconfiguration.openshift.io/role,
operator: In,
values: [worker-cnf, worker],
}
paused: false
nodeSelector:
matchLabels:
node-role.kubernetes.io/worker-cnf: ''
EOF

% oc get mcp
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
master rendered-master-7c1a31533da09208320625bac8c99994 True False False 3 3 3 0 103m
worker rendered-worker-644beb08662e2b1d718fa4a0f03f2c74 True False False 2 2 2 0 103m
worker-cnf False True False 1 0 0 0 21s

apply the performance profile:
apiVersion: performance.openshift.io/v2
kind: PerformanceProfile
metadata:
annotations:
performance.openshift.io/ignore-cgroups-version: "true"
name: performance
spec:
cpu:
isolated: "3"
reserved: "0-2"
machineConfigPoolSelector:
machineconfiguration.openshift.io/role: worker-cnf
nodeSelector:
#node-role.kubernetes.io/worker: ""
node-role.kubernetes.io/worker-cnf: ""

When the node become ready, apply the kubeletconfig:
apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
name: kubelet-test
spec:
kubeletConfig:
cpuManagerPolicy: static
reservedSystemCPUs: 0-2
machineConfigPoolSelector:
matchLabels:
machineconfiguration.openshift.io/role: worker-cnf

verify nodes comes back in Ready state:
% oc get node
NAME STATUS ROLES AGE VERSION
ci-ln-dhznmmb-72292-mdmmw-master-0 Ready control-plane,master 128m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-master-1 Ready control-plane,master 128m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-master-2 Ready control-plane,master 128m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-worker-a-n5msl Ready worker 116m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-worker-b-kw284 Ready worker 116m v1.27.14+df6456e
ci-ln-dhznmmb-72292-mdmmw-worker-c-6d9qw Ready worker,worker-cnf 116m v1.27.14+df6456e

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label May 29, 2024
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: This pull request references Jira Issue OCPBUGS-32473, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.14.z) matches configured target version for branch (4.14.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-31348 is in the state Closed (Done-Errata), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-31348 targets the "4.15.z" version, which is one of the valid target versions: 4.15.0, 4.15.z
  • bug has dependents

Requesting review from QA contact:
/cc @lyman9966

In response to this:

This is an automated cherry-pick of #1915

/assign MarSik

NONE

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot requested a review from lyman9966 May 29, 2024 11:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet