MGMT-17700: Assign none platform node-ips based on connected addresses and etcd restrictions #6257
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@ori-amizur: This pull request references MGMT-17700 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci
bot
added
the
size/XL
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: ori-amizur The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6257 +/- ##
==========================================
+ Coverage 68.25% 68.43% +0.17%
==========================================
Files 244 245 +1
Lines 35869 36060 +191
==========================================
+ Hits 24483 24676 +193
+ Misses 9223 9205 -18
- Partials 2163 2179 +16
|
|
/test edge-subsystem-kubeapi-aws |
|
@ori-amizur: This pull request references MGMT-17700 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
openshift-ci
bot
added
size/XXL
openshift-ci
bot
added
size/XL
|
/test? |
|
@ori-amizur: The following commands are available to trigger required jobs:
The following commands are available to trigger optional jobs:
Use
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
/test edge-e2e-metal-assisted-none |
|
/test edge-e2e-metal-assisted |
ori-amizur
force-pushed
the
MGMT-17700
branch
2 times, most recently
from
b75c07c
to
366ca49
Compare
|
/retest-required |
|
/retest-required |
|
/lgtm |
|
/retest-required |
…s and etcd restrictions When none platform is in use, if there is ambiguity in node-ip assignment, then incorrect assignment might lead to installation failure. This happens when etcd detects that the socket address from an etcd node does not match the expected address in the peer certificate. In this case etcd rejects such connection. Example: assuming two networks - net1 and net2. master node 1 has 1 address that belongs to net1. master node 2 has 2 addresses. one that belongs to net 1, and another that belongs to net 2 master node 3 has 1 address that belongs to net 1. If the selected node-ip of master node 2 belongs to net 2, then when it will create a connection with any other master node, the socket address will be the address that belongs to net 1. Since etcd expects it to be the same as the node-ip, it will reject the connection. This can be solved by node-ips selection that will not cause such a conflict. Node ips assignment should be done through ignition. To correctly set bootstrap ip, the machine-network for the cluster must be set to match the selected node-ip for that host. MGMT-17701: Add capability to calculate none platform node-ips based on L3 connected addresses and connectivity Calculate the node-ips for none platform cluster. Node ip calculation is actually calculation of the node-ip, hint, and cidr. Node-ip is the ip address that exists on the host that was selected as the node-ip. Hint is actually an IP address that does not exist on the host, but must belong to the subnet of the node-ip. Cidr is the subnet in cidr notation that the node-ip and hint belong to. Node-ip calculation is either done for all cluster hosts or none of them. MGMT-17702: Modify ignition to use calculated node-ips for none platform In order to set node-ip, ignition is modified. A file called /etc/default/nodeip-configuration contains the hint as was set by node-ip generation. In addition the bootstrap-ip is set in ignition as was set in node-ip generation. This is set as environment variable called "OPENSHIFT_INSTALL_BOOTSTRAP_NODE_IP". MGMT-17703: Modify machine-network based on calculated node-ips The etcd in boostrap uses he machine-network to set the IP address. Therefore during install-config generation the machine-network may be replaced by the cidr from the node-ip generation.
|
/test edge-e2e-ai-operator-ztp |
|
@ori-amizur: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
/lgtm |
openshift-merge-bot
bot
merged commit 8a63485
into
openshift:master
|
[ART PR BUILD NOTIFIER] This PR has been included in build ose-agent-installer-api-server-container-v4.17.0-202405271211.p0.g8a63485.assembly.stream.el9 for distgit ose-agent-installer-api-server. |
When none platform is in use, if there is ambiguity in node-ip assignment, then incorrect assignment might lead to installation failure. This happens when etcd detects that the socket address from an etcd node does not match the expected address in the peer certificate. In this case etcd rejects such connection.
Example: assuming two networks - net1 and net2.
master node 1 has 1 address that belongs to net1.
master node 2 has 2 addresses. one that belongs to net 1, and another that belongs to net 2 master node 3 has 1 address that belongs to net 1. If the selected node-ip of master node 2 belongs to net 2, then when it will create a connection with any other master node, the socket address will be the address that belongs to net 1. Since etcd expects it to be the same as the node-ip, it will reject the connection.
This can be solved by node-ips selection that will not cause such a conflict. Node ips assignment should be done through ignition. To correctly set bootstrap ip, the machine-network for the cluster must be set to match the selected node-ip for that host.
MGMT-17701: Add capability to calculate none platform node-ips based on L3 connected addresses and connectivity
Calculate the node-ips for none platform cluster. Node ip calculation is actually calculation of the node-ip, hint, and cidr. Node-ip is the ip address that exists on the host that was selected as the node-ip.
Hint is actually an IP address that does not exist on the host, but must belong to the subnet of the node-ip.
Cidr is the subnet in cidr notation that the node-ip and hint belong to. Node-ip calculation is either done for all cluster hosts or none of them.
MGMT-17702: Modify ignition to use calculated node-ips for none platform
In order to set node-ip, ignition is modified. A file called /etc/default/nodeip-configuration contains the hint as was set by node-ip generation.
In addition the bootstrap-ip is set in ignition as was set in node-ip generation. This is set as environment variable called "OPENSHIFT_INSTALL_BOOTSTRAP_NODE_IP".
MGMT-17703: Modify machine-network based on calculated node-ips
The etcd in boostrap uses he machine-network to set the IP address. Therefore during install-config generation the machine-network may be replaced by the cidr from the node-ip generation.
List all the issues related to this PR
What environments does this code impact?
How was this code tested?
Checklist
docs, README, etc)Reviewers Checklist
/cc @tsorya
/cc @paul-maidment