Looking for a way to limit kluctl to a namespace

[kassah]

I'm looking to throw out using helm directly, but want to try kluctl cli to deploy my single helm chart that is baked into a deploy container with only access to a single namespace.

As best I can figure out, discriminators require uniqueness across the cluster namespaces. I'm looking for something along the lines of helm's --namespace flag that causes helm to store it's data all within a specific namespace and apply the namespace to all namespace supporting objects by default.

Is there a similar config flag for kluctl?

[codablock]

What you're looking for is a combination of overrideNamespace and args. You'd define an arg in .kluctl.yaml, e.g. env_name or namespace and then use that arg in overrideNamespace: {{ args.env_name }}. This way you can pass the namespace via the CLI (by using -a) or by configuring them via target definitions.

[kassah]

This didn't quite work out for me. I deployed using simple-helm example modifying the .kluctl target environment arg to be "ops605-test" to match the current namespace that the kubectl has access to.

What I get when I try to deploy then is:

root@deploy-manual-update:/kluctl-examples/simple-helm# kluctl -t simple-helm deploy
✓ Your kluctl version is up-to-date
✓ Loading kluctl project
✓ Initializing k8s client
✗ namespaces "kube-system" is forbidden: User "system:serviceaccount:ops605-test:deploy-user" cannot get resource "namespaces" in API group "" in the namespace "kube-system"
root@deploy-manual-update:/kluctl-examples/simple-helm# 

Is there a way to add the namespace to the discriminator so it doesn't try to access kube-system namespace?

[codablock]

Ah, I misunderstood you. Kluctl is internally trying to identify the cluster via the UID of the kube-system namespace. I should maybe make this optional and print out a warning in case it fails.

I wonder, do you have permissions for LIST? Can you try kubectl get ns -oyaml and check if if works and if yes, if it contains metadata.uid per namespace?

[codablock]

@kassah Can you try the current devel release? It contains #988, which should finally fix these permission issues.

[kassah]

Ah, I misunderstood you. Kluctl is internally trying to identify the cluster via the UID of the kube-system namespace. I should maybe make this optional and print out a warning in case it fails.

I wonder, do you have permissions for LIST? Can you try kubectl get ns -oyaml and check if if works and if yes, if it contains metadata.uid per namespace?

Looks like nope:

root@deploy-manual-update:/# kubectl get ns -oyaml
apiVersion: v1
items: []
kind: List
metadata:
  resourceVersion: ""
Error from server (Forbidden): namespaces is forbidden: User "system:serviceaccount:ops605-test:deploy-user" cannot list resource "namespaces" in API group "" at the cluster scope
root@deploy-manual-update:/# 

The permissions it has are granted by a namespaced RoleBinding to ClusterRole "edit" (note that it can't see the RoleBinding with it's access):

apiVersion: v1
items:
- apiVersion: rbac.authorization.k8s.io/v1
  kind: RoleBinding
  metadata:
    creationTimestamp: "2024-02-28T00:21:05Z"
    name: deploy-access
    namespace: ops605-test
    resourceVersion: "615445820"
    uid: 3bc5f425-84ff-40ad-b128-8638a6d778cf
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: edit
  subjects:
  - kind: ServiceAccount
    name: deploy-user
    namespace: ops605-test
kind: List
metadata:
  resourceVersion: ""

Will reply separately to the PR and instruction to try the devel release response.

[kassah]

So I tried the 2.24.0-devel version built from commit 301d535.

I had to modify the simple-helm example so that it didn't create the namespace (since it already existed, and the account diddn't have permissions to that)

It did succeed with warnings:

root@deploy-manual-update:/kluctl-examples/simple-helm# kluctl deploy -t simple-helm
✓ Loading kluctl project
✓ Initializing k8s client
⚠ Not enough permissions to write to the result store.
✓ Rendering templates
✓ Rendering Helm Charts
✓ Building kustomize objects
✓ Postprocessing objects
✓ Writing rendered objects
⚠ Getting remote objects by discriminator: Failed with 85 errors
✓ Getting 2 additional remote objects
✓ Getting namespaces
✓ nginx: Applied 2 objects. Encountered 1 warnings.

New objects:
  ops605-test/Service/nginx
  ops605-test/Deployment/nginx

Warnings:
  at least one permission error was encountered while gathering objects by discriminator labels. This might result in orphan object detection to not work properly
  ops605-test/Deployment/nginx: autopilot-default-resources-mutator:Autopilot updated Deployment ops605-test/nginx: defaulted unspecified resources for containers [nginx] (see http://g.co/gke/autopilot-defaults)
✓ The diff succeeded, do you want to proceed? (y/N) y
✓ nginx: Applied 2 objects. Encountered 1 warnings.

New objects:
  ops605-test/Service/nginx
  ops605-test/Deployment/nginx

Warnings:
  at least one permission error was encountered while gathering objects by discriminator labels. This might result in orphan object detection to not work properly
  ops605-test/Deployment/nginx: autopilot-default-resources-mutator:Autopilot updated Deployment ops605-test/nginx: defaulted unspecified resources for containers [nginx] (see http://g.co/gke/autopilot-defaults)
root@deploy-manual-update:/kluctl-examples/simple-helm#

[kassah]

The existing PR diddn't quite make this as smooth as I would think. I filed a feature request for my idea of how I would think that should feel and work #991

Thanks for your awesome work on this tool!