Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Nodes status not ready for Fleet in Rancher 2.9-head #2334

Open
mmartin24 opened this issue Apr 17, 2024 · 3 comments
Open

Nodes status not ready for Fleet in Rancher 2.9-head #2334

mmartin24 opened this issue Apr 17, 2024 · 3 comments
Labels
kind/bug
Milestone
v2.9.0

Comments

@mmartin24
Copy link
Collaborator

mmartin24 commented Apr 17, 2024 •
edited by thehejik

Issue

In Rancher 2.9-head, clusters in Fleet display status not ready even when they are.This happens for both fleet-local and fleet-default and it was working fine in previous versions:
Image

Taking a look at clusters.fleet.cattle.io it seems Nodes Ready are missing in 2.9-head (fleet:v0.10.0-rc.11)

→ k get clusters.fleet.cattle.io -n fleet-local local                     
NAME    BUNDLES-READY   LAST-SEEN              STATUS
local   1/1             2024-04-17T05:05:37Z   

→ k describe clusters.fleet.cattle.io -n fleet-local local | grep "Ready Nodes"
→

...while this is ok in Rancher 2.8.3 with fleet:v0.9.2:

➜  ~ k get clusters.fleet.cattle.io -n fleet-local local                         
NAME    BUNDLES-READY   NODES-READY   SAMPLE-NODE   LAST-SEEN              STATUS
local   1/1             1/1           mmartinsuse   2024-04-17T05:03:56Z   

➜  ~ k describe clusters.fleet.cattle.io -n fleet-local local | grep "Ready Nodes"
    Non Ready Nodes:       0
    Ready Nodes:              1
    Ready Nodes:    1/1

Here you can see full info on yamls for both versions:

notreadynodesinstatus_290.yaml 
apiVersion: fleet.cattle.io/v1alpha1
kind: Cluster
metadata:
  annotations:
    objectset.rio.cattle.io/applied: H4sIAAAAAAAA/4xSTW/bMAz9KwPPTresX4mBHYquGIoBPbS7FT0wEu1okSlBopIagf/7IDtJvSItcpPI9574nriFhgQ1CkK5BWR2gmIcx3x1i7+kJJKcBePOFIpYOjPuq9FQQmWJZKJsikIBig/BbsMUJvV6BSX44NYmGseG6xFkPS2+/Dasf9yeqMbYEJRgnUJ7Ejh6VHQYeuB1BahAvds/pqEo2HgoOVlbgMUF2T6DBhlraohlJLxzPdEmeovt+3k+5bzDnmhliXGZ5/+OM5zP53OcXdJ0WlWz86vLC7qaKlLneD2bXlzPq0V1DcWQtabwJgIlrM5j9n1khg8Sip5Uvxk1sdxUlWEjbS6w0zS++0AVhUD6ZwqG6ye1JJ2s4fq+Znco372SSjlxKJ/3HOL8cE5a1PLu1QeKcdjA5y2sqN0PNUqjnyan5SmguAAl3DMUsEabKBNBQiJ46V66AjZk6qVAOe1euq4YnDyMHA+yk5HxSWyjUAMFrNKCbh1Xpn4iFUj2oU1yQ/WNI6iHT/IUlBQPieaDxShPRLzfvfFv5M9x/Eio2wenqRc+4EaNCOW3AsJRXBiDugI0RRNI99xfRh7Ju4G+22Yot92O9V87UHQpKLp1iaW3MFbqIY2J0XA9nJ02lSHdX9jJG8oFv0TedcKhnHjFbsP9eYNGbry3Az/nlpoGQ3v01b1E13X/AgAA//+yS93xzwQAAA
    objectset.rio.cattle.io/id: fleet-cluster
    objectset.rio.cattle.io/owner-gvk: provisioning.cattle.io/v1, Kind=Cluster
    objectset.rio.cattle.io/owner-name: local
    objectset.rio.cattle.io/owner-namespace: fleet-local
  creationTimestamp: "2024-04-16T08:26:51Z"
  generation: 2
  labels:
    management.cattle.io/cluster-display-name: local
    management.cattle.io/cluster-name: local
    name: local
    objectset.rio.cattle.io/hash: f2a8a9999a85e11ff83654e61cec3a781479fbf7
    provider.cattle.io: k3s
  name: local
  namespace: fleet-local
  resourceVersion: "56237"
  uid: 6d6ce1cc-fcd5-4b02-a155-ea8e5529fc1b
spec:
  agentAffinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - preference:
          matchExpressions:
          - key: fleet.cattle.io/agent
            operator: In
            values:
            - "true"
        weight: 1
  agentNamespace: cattle-fleet-local-system
  clientID: pcl48bd7b2jsxfrmrppsjzznjrt6jk8jtpmphjf52wkrgww77fw9qp
  kubeConfigSecret: local-kubeconfig
  kubeConfigSecretNamespace: fleet-local
status:
  agent:
    lastSeen: "2024-04-16T14:50:37Z"
    namespace: cattle-fleet-local-system
  agentAffinityHash: f50425c0999a8e18c2d104cdb8cb063762763f232f538b5a7c8bdb61
  agentDeployedGeneration: 0
  agentMigrated: true
  agentNamespaceMigrated: true
  apiServerCAHash: a72b2cd1d6c6bb9d8cbdb13104481a456eacd31a26bc9fc09df0fb5b
  apiServerURL: https://10.43.0.1:443
  cattleNamespaceMigrated: true
  conditions:
  - lastUpdateTime: "2024-04-16T08:27:47Z"
    status: "True"
    type: Processed
  - lastUpdateTime: "2024-04-16T08:28:13Z"
    status: "True"
    type: Ready
  - lastUpdateTime: "2024-04-16T08:35:31Z"
    status: "True"
    type: Imported
  - lastUpdateTime: "2024-04-16T08:27:50Z"
    status: "True"
    type: Reconciled
  desiredReadyGitRepos: 0
  display:
    readyBundles: 1/1
  namespace: cluster-fleet-local-local-1a3d67d0a899
  readyGitRepos: 0
  resourceCounts:
    desiredReady: 0
    missing: 0
    modified: 0
    notReady: 0
    orphaned: 0
    ready: 0
    unknown: 0
    waitApplied: 0
  summary:
    desiredReady: 1
    ready: 1
readynodesinstatus_283.yaml 
 apiVersion: fleet.cattle.io/v1alpha1
kind: Cluster
metadata:
  annotations:
    objectset.rio.cattle.io/applied: >-
      H4sIAAAAAAAA/4xSTW/bMAz9KwPPTresX4mBHYquGIoBPbS7FT0wEu1okSlBopIagf/7IDtJvSItcpPI9574nriFhgQ1CkK5BWR2gmIcx3x1i7+kJJKcBePOFIpYOjPuq9FQQmWJZKJsikIBig/BbsMUJvV6BSX44NYmGseG6xFkPS2+/Dasf9yeqMbYEJRgnUJ7Ejh6VHQYeuB1BahAvds/pqEo2HgoOVlbgMUF2T6DBhlraohlJLxzPdEmeovt+3k+5bzDnmhliXGZ5/+OM5zP53OcXdJ0WlWz86vLC7qaKlLneD2bXlzPq0V1DcWQtabwJgIlrM5j9n1khg8Sip5Uvxk1sdxUlWEjbS6w0zS++0AVhUD6ZwqG6ye1JJ2s4fq+Znco372SSjlxKJ/3HOL8cE5a1PLu1QeKcdjA5y2sqN0PNUqjnyan5SmguAAl3DMUsEabKBNBQiJ46V66AjZk6qVAOe1euq4YnDyMHA+yk5HxSWyjUAMFrNKCbh1Xpn4iFUj2oU1yQ/WNI6iHT/IUlBQPieaDxShPRLzfvfFv5M9x/Eio2wenqRc+4EaNCOW3AsJRXBiDugI0RRNI99xfRh7Ju4G+22Yot92O9V87UHQpKLp1iaW3MFbqIY2J0XA9nJ02lSHdX9jJG8oFv0TedcKhnHjFbsP9eYNGbry3Az/nlpoGQ3v01b1E13X/AgAA//+yS93xzwQAAA
    objectset.rio.cattle.io/id: fleet-cluster
    objectset.rio.cattle.io/owner-gvk: provisioning.cattle.io/v1, Kind=Cluster
    objectset.rio.cattle.io/owner-name: local
    objectset.rio.cattle.io/owner-namespace: fleet-local
  creationTimestamp: '2024-04-15T15:27:36Z'
  generation: 4
  labels:
    management.cattle.io/cluster-display-name: local
    management.cattle.io/cluster-name: local
    name: local
    objectset.rio.cattle.io/hash: f2a8a9999a85e11ff83654e61cec3a781479fbf7
    provider.cattle.io: k3s
  managedFields:
    - apiVersion: fleet.cattle.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        f:spec:
          f:clientID: {}
      manager: fleetcontroller
      operation: Update
      time: '2024-04-15T15:28:08Z'
    - apiVersion: fleet.cattle.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:objectset.rio.cattle.io/applied: {}
            f:objectset.rio.cattle.io/id: {}
            f:objectset.rio.cattle.io/owner-gvk: {}
            f:objectset.rio.cattle.io/owner-name: {}
            f:objectset.rio.cattle.io/owner-namespace: {}
          f:labels:
            .: {}
            f:management.cattle.io/cluster-display-name: {}
            f:management.cattle.io/cluster-name: {}
            f:name: {}
            f:objectset.rio.cattle.io/hash: {}
            f:provider.cattle.io: {}
        f:spec:
          .: {}
          f:agentAffinity:
            .: {}
            f:nodeAffinity:
              .: {}
              f:preferredDuringSchedulingIgnoredDuringExecution: {}
          f:agentNamespace: {}
          f:kubeConfigSecret: {}
          f:kubeConfigSecretNamespace: {}
          f:redeployAgentGeneration: {}
      manager: rancher
      operation: Update
      time: '2024-04-17T05:03:36Z'
    - apiVersion: fleet.cattle.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          f:agent:
            f:lastSeen: {}
            f:readyNodeNames: {}
            f:readyNodes: {}
      manager: fleetagent
      operation: Update
      subresource: status
      time: '2024-04-17T05:03:56Z'
    - apiVersion: fleet.cattle.io/v1alpha1
      fieldsType: FieldsV1
      fieldsV1:
        f:status:
          .: {}
          f:agent:
            .: {}
            f:namespace: {}
            f:nonReadyNodeNames: {}
            f:nonReadyNodes: {}
          f:agentAffinityHash: {}
          f:agentDeployedGeneration: {}
          f:agentMigrated: {}
          f:agentNamespaceMigrated: {}
          f:apiServerCAHash: {}
          f:apiServerURL: {}
          f:cattleNamespaceMigrated: {}
          f:conditions: {}
          f:desiredReadyGitRepos: {}
          f:display:
            .: {}
            f:readyBundles: {}
            f:readyNodes: {}
            f:sampleNode: {}
          f:namespace: {}
          f:readyGitRepos: {}
          f:resourceCounts:
            .: {}
            f:desiredReady: {}
            f:missing: {}
            f:modified: {}
            f:notReady: {}
            f:orphaned: {}
            f:ready: {}
            f:unknown: {}
            f:waitApplied: {}
          f:summary:
            .: {}
            f:desiredReady: {}
            f:ready: {}
      manager: fleetcontroller
      operation: Update
      subresource: status
      time: '2024-04-17T05:05:00Z'
  name: local
  namespace: fleet-local
  resourceVersion: '421030'
  uid: a0638893-a113-4795-9710-3894266786c4
spec:
  agentAffinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - preference:
            matchExpressions:
              - key: fleet.cattle.io/agent
                operator: In
                values:
                  - 'true'
          weight: 1
  agentNamespace: cattle-fleet-local-system
  clientID: tkrh9fc9v7w2fprqwv8kr7npfc47r9qjsn8tvt42rlzk4pdgk8m6h8
  kubeConfigSecret: local-kubeconfig
  kubeConfigSecretNamespace: fleet-local
  redeployAgentGeneration: 3
status:
  agent:
    lastSeen: '2024-04-17T05:03:56Z'
    namespace: cattle-fleet-local-system
    nonReadyNodeNames: null
    nonReadyNodes: 0
    readyNodeNames:
      - mmartinsuse
    readyNodes: 1
  agentAffinityHash: f50425c0999a8e18c2d104cdb8cb063762763f232f538b5a7c8bdb61
  agentDeployedGeneration: 3
  agentMigrated: true
  agentNamespaceMigrated: true
  apiServerCAHash: 74b638d12ece4ad275286c5252e8899e58e3210fe23c0a337ca1b44b
  apiServerURL: https://10.43.0.1:443
  cattleNamespaceMigrated: true
  conditions:
    - lastUpdateTime: '2024-04-17T05:05:00Z'
      status: 'True'
      type: Ready
    - lastUpdateTime: '2024-04-17T05:05:00Z'
      status: 'True'
      type: Processed
    - lastUpdateTime: '2024-04-17T05:03:36Z'
      status: 'True'
      type: Imported
    - lastUpdateTime: '2024-04-15T15:28:08Z'
      status: 'True'
      type: Reconciled
  desiredReadyGitRepos: 0
  display:
    readyBundles: 1/1
    readyNodes: 1/1
    sampleNode: mmartinsuse
  namespace: cluster-fleet-local-local-1a3d67d0a899
  readyGitRepos: 0
  resourceCounts:
    desiredReady: 0
    missing: 0
    modified: 0
    notReady: 0
    orphaned: 0
    ready: 0
    unknown: 0
    waitApplied: 0
  summary:
    desiredReady: 1
    ready: 1

Let's please return this value so the UI can correctly reflect this value

Test steps / environment

  • Deployed Rancher v2.8.3 / Fleet 0.9.2 and 2.9-head (v2.9-0dc6561b4a0cbcda67199cf86d45d28da10ac1f5-head) / fleet:v0.10.0-rc.11
  • Single node K3S_VERSION=v1.27.10+k3s1
  • After Rancher is fully installed, observe how in Continous Delivery > fleet-local > Clusters

Observed behavior:

  • Nodes Ready is 0 in Rancher v2.9-head and 1 in Rancher v2.8.3
@mmartin24 mmartin24 changed the title Nodes status not ready fir Fleet in Rancher 2.9-head Nodes status not ready for Fleet in Rancher 2.9-head Apr 17, 2024
@thehejik
Copy link

thehejik commented Apr 17, 2024 •
edited

To summarize, clusters.fleet.cattle.io resource in v0.10 doesn't contain status.agent.readyNodes object anymore which is probably mandatory for rancher UI.

@thehejik
Copy link

Related rancher issue rancher/dashboard#10542

@mmartin24
Copy link
Collaborator Author

Also related: #1973

@kkaempf kkaempf added this to the v2.9.0 milestone Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug
Projects
Fleet
Status: Needs QA review
Milestone
v2.9.0
Development

No branches or pull requests
3 participants
@kkaempf @thehejik @mmartin24