Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add --why flag for zarf dev find-images #2309

Merged
merged 23 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
2f6f363
feat: add --why for zarf dev find-images
waveywaves Feb 16, 2024
4e04f8d
Merge branch 'main' into en/2272
Racer159 Feb 23, 2024
27f96e5
Merge branch 'main' into en/2272
Racer159 Feb 23, 2024
32f9667
Merge branch 'main' into en/2272
lucasrod16 Feb 23, 2024
d9debc0
apply changes from code review
waveywaves Feb 23, 2024
c1d7528
append to whyResources []string{}
waveywaves Feb 23, 2024
3fd5771
marshal only once to yaml instead of json then yaml
waveywaves Feb 23, 2024
8f41236
read file and update expected output to be a regex
waveywaves Feb 23, 2024
18c2ce1
Merge branch 'main' into en/2272
lucasrod16 Feb 28, 2024
5e9029c
minor tweaks
waveywaves Feb 28, 2024
406e234
Update src/test/e2e/13_find_images_test.go
lucasrod16 Feb 29, 2024
3973525
Remove message in error assertion
lucasrod16 Feb 29, 2024
0e1ce5b
Merge branch 'en/2272' of https://github.com/waveywaves/zarf into en/…
lucasrod16 Feb 29, 2024
7d9f72c
Merge branch 'main' into en/2272
lucasrod16 Feb 29, 2024
bd18217
use yamls and not the resources which are parsed
waveywaves Mar 1, 2024
812b112
use yamls and not resources
waveywaves Mar 1, 2024
69f6a09
Merge branch 'main' into en/2272
Racer159 Mar 4, 2024
e809a44
Merge branch 'main' into en/2272
Racer159 Mar 4, 2024
01c3f3f
Update src/test/packages/13-find-images/helm-charts-find-images-why-e…
waveywaves Mar 4, 2024
903261c
Update src/test/e2e/13_find_images_test.go
waveywaves Mar 4, 2024
fdf0169
Merge branch 'main' into en/2272
Racer159 Mar 4, 2024
4976a73
Update src/test/e2e/13_find_images_test.go
waveywaves Mar 5, 2024
260ed65
Merge branch 'main' into en/2272
lucasrod16 Mar 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Expand Up @@ -20,6 +20,7 @@ zarf dev find-images [ PACKAGE ] [flags]
--kube-version string Override the default helm template KubeVersion when performing a package chart template
-p, --repo-chart-path string If git repos hold helm charts, often found with gitops tools, specify the chart path, e.g. "/" or "/chart"
--set stringToString Specify package variables to set on the command line (KEY=value). Note, if using a config file, this will be set by [package.create.set]. (default [])
--why string Find the location of the image given as an argument and print it to the console.
```

## Options inherited from parent commands
Expand Down
2 changes: 2 additions & 0 deletions src/cmd/dev.go
Expand Up @@ -267,6 +267,8 @@ func init() {
devFindImagesCmd.Flags().StringToStringVar(&pkgConfig.CreateOpts.SetVariables, "set", v.GetStringMapString(common.VPkgCreateSet), lang.CmdDevFlagSet)
// allow for the override of the default helm KubeVersion
devFindImagesCmd.Flags().StringVar(&pkgConfig.FindImagesOpts.KubeVersionOverride, "kube-version", "", lang.CmdDevFlagKubeVersion)
// check which manifests are using this particular image
devFindImagesCmd.Flags().StringVar(&pkgConfig.FindImagesOpts.Why, "why", "", lang.CmdDevFlagFindImagesWhy)

devLintCmd.Flags().StringToStringVar(&pkgConfig.CreateOpts.SetVariables, "set", v.GetStringMapString(common.VPkgCreateSet), lang.CmdPackageCreateFlagSet)
devLintCmd.Flags().StringVarP(&pkgConfig.CreateOpts.Flavor, "flavor", "f", v.GetString(common.VPkgCreateFlavor), lang.CmdPackageCreateFlagFlavor)
Expand Down
1 change: 1 addition & 0 deletions src/config/lang/english.go
Expand Up @@ -384,6 +384,7 @@ $ zarf package pull oci://ghcr.io/defenseunicorns/packages/dos-games:1.0.0 -a sk
CmdDevFlagRepoChartPath = `If git repos hold helm charts, often found with gitops tools, specify the chart path, e.g. "/" or "/chart"`
CmdDevFlagGitAccount = "User or organization name for the git account that the repos are created under."
CmdDevFlagKubeVersion = "Override the default helm template KubeVersion when performing a package chart template"
CmdDevFlagFindImagesWhy = "Find the location of the image given as an argument and print it to the console."

CmdDevLintShort = "Lints the given package for valid schema and recommended practices"
CmdDevLintLong = "Verifies the package schema, checks if any variables won't be evaluated, and checks for unpinned images/repos/files"
Expand Down
49 changes: 49 additions & 0 deletions src/pkg/packager/prepare.go
lucasrod16 marked this conversation as resolved.
Show resolved Hide resolved
Expand Up @@ -6,6 +6,7 @@ package packager

import (
"fmt"
"github.com/goccy/go-yaml"
"os"
"path/filepath"
"regexp"
Expand Down Expand Up @@ -37,10 +38,12 @@ type imageMap map[string]bool
func (p *Packager) FindImages() (imgMap map[string][]string, err error) {
lucasrod16 marked this conversation as resolved.
Show resolved Hide resolved
repoHelmChartPath := p.cfg.FindImagesOpts.RepoHelmChartPath
kubeVersionOverride := p.cfg.FindImagesOpts.KubeVersionOverride
whyImage := p.cfg.FindImagesOpts.Why

imagesMap := make(map[string][]string)
erroredCharts := []string{}
erroredCosignLookups := []string{}
whyResources := []string{}

cwd, err := os.Getwd()
if err != nil {
Expand Down Expand Up @@ -158,6 +161,15 @@ func (p *Packager) FindImages() (imgMap map[string][]string, err error) {
for _, image := range annotatedImages {
matchedImages[image] = true
}

// Check if the --why flag is set
if whyImage != "" {
whyResourcesChart, err := findWhyResources(yamls, whyImage, component.Name, chart.Name, true)
if err != nil {
message.WarnErrf(err, "Error finding why resources for chart %s: %s", chart.Name, err.Error())
}
whyResources = append(whyResources, whyResourcesChart...)
}
}

for _, manifest := range component.Manifests {
Expand Down Expand Up @@ -193,6 +205,15 @@ func (p *Packager) FindImages() (imgMap map[string][]string, err error) {
message.Debugf("%s", contentString)
yamls, _ := utils.SplitYAML(contents)
resources = append(resources, yamls...)

// Check if the --why flag is set and if it is process the manifests
if whyImage != "" {
whyResourcesManifest, err := findWhyResources(yamls, whyImage, component.Name, manifest.Name, false)
if err != nil {
message.WarnErrf(err, "Error finding why resources for manifest %s: %s", manifest.Name, err.Error())
}
whyResources = append(whyResources, whyResourcesManifest...)
}
}
}

Expand Down Expand Up @@ -268,6 +289,13 @@ func (p *Packager) FindImages() (imgMap map[string][]string, err error) {
}
}

if whyImage != "" {
if len(whyResources) == 0 {
message.Warnf("image %q not found in any charts or manifests", whyImage)
}
return nil, nil
}

fmt.Println(componentDefinition)

// Return to the original working directory
Expand Down Expand Up @@ -356,6 +384,27 @@ func (p *Packager) processUnstructuredImages(resource *unstructured.Unstructured
return matchedImages, maybeImages, nil
}

func findWhyResources(resources []*unstructured.Unstructured, whyImage, componentName, resourceName string, isChart bool) ([]string, error) {
foundWhyResources := []string{}
for _, resource := range resources {
bytes, err := yaml.Marshal(resource.Object)
if err != nil {
return nil, err
}
yaml := string(bytes)
resourceTypeKey := "manifest"
if isChart {
resourceTypeKey = "chart"
}

if strings.Contains(yaml, whyImage) {
fmt.Printf("component: %s\n%s: %s\nresource:\n\n%s\n", componentName, resourceTypeKey, resourceName, yaml)
foundWhyResources = append(foundWhyResources, resourceName)
}
}
return foundWhyResources, nil
}

// BuildImageMap looks for init container, ephemeral and regular container images.
func buildImageMap(images imageMap, pod corev1.PodSpec) imageMap {
for _, container := range pod.InitContainers {
Expand Down
53 changes: 53 additions & 0 deletions src/test/e2e/13_find_images_test.go
lucasrod16 marked this conversation as resolved.
Show resolved Hide resolved
@@ -0,0 +1,53 @@
package test

import (
"os"
"path/filepath"
"regexp"
waveywaves marked this conversation as resolved.
Show resolved Hide resolved
"testing"

"github.com/stretchr/testify/require"
)

func TestFindImages(t *testing.T) {
t.Log("E2E: Find Images")

t.Run("zarf test find images success", func(t *testing.T) {
t.Log("E2E: Test Find Images")

testPackagePath := filepath.Join("examples", "dos-games")
expectedOutput, err := os.ReadFile("src/test/packages/13-find-images/dos-games-find-images-expected.txt")
require.NoError(t, err)

stdout, _, err := e2e.Zarf("dev", "find-images", testPackagePath)
require.NoError(t, err)
require.Contains(t, stdout, string(expectedOutput))
})

t.Run("zarf test find images --why w/ helm chart success", func(t *testing.T) {
t.Log("E2E: Test Find Images against a helm chart with why flag")

testPackagePath := filepath.Join("examples", "helm-charts")
expectedOutput, err := os.ReadFile("src/test/packages/13-find-images/helm-charts-find-images-why-expected.txt")
require.NoError(t, err)

stdout, _, err := e2e.Zarf("dev", "find-images", testPackagePath, "--why", "curlimages/curl:7.69.0")
require.NoError(t, err)
match, err := regexp.MatchString(string(expectedOutput), stdout)
require.NoError(t, err)
require.True(t, match)
})
waveywaves marked this conversation as resolved.
Show resolved Hide resolved

t.Run("zarf test find images --why w/ manifests success", func(t *testing.T) {
t.Log("E2E: Test Find Images against manifests with why flag")

testPackagePath := filepath.Join("examples", "manifests")
expectedOutput, err := os.ReadFile("src/test/packages/13-find-images/manifests-find-images-why-expected.txt")
require.NoError(t, err)

stdout, _, err := e2e.Zarf("dev", "find-images", testPackagePath, "--why", "httpd:alpine3.18")
require.NoError(t, err)
require.Contains(t, stdout, string(expectedOutput))
})

}
@@ -0,0 +1,7 @@
components:

- name: baseline
images:
- defenseunicorns/zarf-game:multi-tile-dark
# Cosign artifacts for images - dos-games - baseline
- index.docker.io/defenseunicorns/zarf-game:sha256-0b694ca1c33afae97b7471488e07968599f1d2470c629f76af67145ca64428af.sig
@@ -0,0 +1,197 @@
component: demo-helm-charts
chart: podinfo-oci
resource:

apiVersion: v1
kind: Pod
metadata:
annotations:
appmesh.k8s.aws/sidecarInjectorWebhook: disabled
helm.sh/hook: test-success
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
linkerd.io/inject: disabled
sidecar.istio.io/inject: "false"
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo-oci
app.kubernetes.io/version: 6.4.0
helm.sh/chart: podinfo-6.4.0
name: podinfo-oci-service-test-[a-zA-Z0-9]+
spec:
containers:
- command:
- sh
- -c
- |
curl -s ${PODINFO_SVC}/api/info | grep version
env:
- name: PODINFO_SVC
value: podinfo-oci.podinfo-from-oci:9898
image: curlimages/curl:7.69.0
name: curl
restartPolicy: Never

component: demo-helm-charts
chart: podinfo-git
resource:

apiVersion: v1
kind: Pod
metadata:
annotations:
appmesh.k8s.aws/sidecarInjectorWebhook: disabled
helm.sh/hook: test-success
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
linkerd.io/inject: disabled
sidecar.istio.io/inject: "false"
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo-oci
app.kubernetes.io/version: 6.4.0
helm.sh/chart: podinfo-6.4.0
name: podinfo-oci-service-test-[a-zA-Z0-9]+
spec:
containers:
- command:
- sh
- -c
- |
curl -s ${PODINFO_SVC}/api/info | grep version
env:
- name: PODINFO_SVC
value: podinfo-oci.podinfo-from-oci:9898
image: curlimages/curl:7.69.0
name: curl
restartPolicy: Never

component: demo-helm-charts
chart: podinfo-git
resource:

apiVersion: v1
kind: Pod
metadata:
annotations:
appmesh.k8s.aws/sidecarInjectorWebhook: disabled
helm.sh/hook: test-success
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
linkerd.io/inject: disabled
sidecar.istio.io/inject: "false"
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo-git
app.kubernetes.io/version: 6.4.0
helm.sh/chart: podinfo-6.4.0
name: podinfo-git-service-test-[a-zA-Z0-9]+
spec:
containers:
- command:
- sh
- -c
- |
curl -s ${PODINFO_SVC}/api/info | grep version
env:
- name: PODINFO_SVC
value: podinfo-git.podinfo-from-git:9898
image: curlimages/curl:7.69.0
name: curl
restartPolicy: Never

component: demo-helm-charts
chart: podinfo-repo
resource:

apiVersion: v1
kind: Pod
metadata:
annotations:
appmesh.k8s.aws/sidecarInjectorWebhook: disabled
helm.sh/hook: test-success
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
linkerd.io/inject: disabled
sidecar.istio.io/inject: "false"
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo-oci
app.kubernetes.io/version: 6.4.0
helm.sh/chart: podinfo-6.4.0
name: podinfo-oci-service-test-[a-zA-Z0-9]+
spec:
containers:
- command:
- sh
- -c
- |
curl -s ${PODINFO_SVC}/api/info | grep version
env:
- name: PODINFO_SVC
value: podinfo-oci.podinfo-from-oci:9898
image: curlimages/curl:7.69.0
name: curl
restartPolicy: Never

component: demo-helm-charts
chart: podinfo-repo
resource:

apiVersion: v1
kind: Pod
metadata:
annotations:
appmesh.k8s.aws/sidecarInjectorWebhook: disabled
helm.sh/hook: test-success
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
linkerd.io/inject: disabled
sidecar.istio.io/inject: "false"
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: podinfo-git
app.kubernetes.io/version: 6.4.0
helm.sh/chart: podinfo-6.4.0
name: podinfo-git-service-test-[a-zA-Z0-9]+
spec:
containers:
- command:
- sh
- -c
- |
curl -s ${PODINFO_SVC}/api/info | grep version
env:
- name: PODINFO_SVC
value: podinfo-git.podinfo-from-git:9898
image: curlimages/curl:7.69.0
name: curl
restartPolicy: Never

component: demo-helm-charts
chart: podinfo-repo
resource:

apiVersion: v1
kind: Pod
metadata:
annotations:
appmesh.k8s.aws/sidecarInjectorWebhook: disabled
helm.sh/hook: test-success
helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
linkerd.io/inject: disabled
sidecar.istio.io/inject: "false"
labels:
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: cool-release-name-podinfo
app.kubernetes.io/version: 6.4.0
helm.sh/chart: podinfo-6.4.0
name: cool-release-name-podinfo-service-test-[a-zA-Z0-9]+
spec:
containers:
- command:
- sh
- -c
- |
curl -s ${PODINFO_SVC}/api/info | grep version
env:
- name: PODINFO_SVC
value: cool-release-name-podinfo.podinfo-from-repo:9898
image: curlimages/curl:7.69.0
name: curl
restartPolicy: Never
waveywaves marked this conversation as resolved.
Show resolved Hide resolved