Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactor and purify the OCI library within Zarf. #2235

Merged
merged 144 commits into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from 142 commits
Commits
Show all changes
144 commits
Select commit Hold shift + click to select a range
0e1c13e
WIP, OCI tests passing
AustinAbro321 Jan 23, 2024
d6649a4
WIP
AustinAbro321 Jan 23, 2024
4dfa44c
WIP, tests passing
AustinAbro321 Jan 23, 2024
c904cb0
WIP, tests passing
AustinAbro321 Jan 23, 2024
8fb51c1
merge
AustinAbro321 Jan 24, 2024
38308c2
moving concurrency tools to helpers
AustinAbro321 Jan 24, 2024
d13e396
inital progress bar abstraction
AustinAbro321 Jan 24, 2024
4b18cb6
WIP, tests passing
AustinAbro321 Jan 24, 2024
236ce23
WIP, move message to helpers
AustinAbro321 Jan 24, 2024
5592aaf
Allowing nil progress bar
AustinAbro321 Jan 24, 2024
9b529db
Cleaner implemention of allowing nil
AustinAbro321 Jan 24, 2024
7e845e5
WIP, moving to prgoressbar interface
AustinAbro321 Jan 24, 2024
02a650d
refactoring message bar in oci
AustinAbro321 Jan 25, 2024
ebd36f0
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Jan 29, 2024
ec0bc52
WIP, testing works, progress bar numbers not right
AustinAbro321 Jan 29, 2024
4c718ee
WIP, tests are working. Zarf out of push.go
AustinAbro321 Jan 29, 2024
7c9444c
arch fix
AustinAbro321 Jan 30, 2024
d24d38a
WIP Refactoring zarf out, tests passing
AustinAbro321 Jan 30, 2024
d1bbde0
WIP, tests passing
AustinAbro321 Jan 30, 2024
4a60079
WIP, refactoring out zarf
AustinAbro321 Jan 30, 2024
06e52f9
WIP, refactoring zarf out of oci
AustinAbro321 Jan 30, 2024
d851249
fixing platform
AustinAbro321 Jan 30, 2024
7655d32
add media type
AustinAbro321 Jan 30, 2024
9fc13db
move transport to helpers
AustinAbro321 Jan 30, 2024
289f02d
move transport to helpers actually
AustinAbro321 Jan 30, 2024
433b0b2
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Jan 30, 2024
41a83dc
WIP pulling progressbar out of render, breaking skeletons on purpose
AustinAbro321 Jan 31, 2024
9d3ecb3
WIP inital use of concurrency tools
AustinAbro321 Jan 31, 2024
0e0f5a5
WIP, tests passing creating io in helpers
AustinAbro321 Jan 31, 2024
dac2b73
WIP, de-zarfifying
AustinAbro321 Jan 31, 2024
4c7626f
adding logging interface
AustinAbro321 Jan 31, 2024
05b03af
Refactor / rename
AustinAbro321 Jan 31, 2024
b8b0e6b
accounting for nil
AustinAbro321 Jan 31, 2024
7bd1135
lint
AustinAbro321 Jan 31, 2024
9cb21d7
moving ctx out of orasremote
AustinAbro321 Feb 1, 2024
644a7fa
remove comments
AustinAbro321 Feb 1, 2024
31d7769
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 1, 2024
dff24c8
Abstracting zarf logic into ZarfOrasRemote
AustinAbro321 Feb 1, 2024
744e5bf
simplyfing concurrency...hopefully
AustinAbro321 Feb 1, 2024
73e4a53
renaming PlatformForArch
AustinAbro321 Feb 1, 2024
8e4b93c
ZOCI
AustinAbro321 Feb 1, 2024
957f99a
refactors
AustinAbro321 Feb 1, 2024
677e2dc
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 1, 2024
b342d41
Apply suggestions from code review
AustinAbro321 Feb 2, 2024
29c7b55
refactors
AustinAbro321 Feb 2, 2024
21f5e44
refactor
AustinAbro321 Feb 2, 2024
8295a9a
changing modifier order in newremote
AustinAbro321 Feb 2, 2024
2f53378
refactor
AustinAbro321 Feb 2, 2024
81b0004
rename
AustinAbro321 Feb 2, 2024
1c66287
rename
AustinAbro321 Feb 2, 2024
ac36250
removing with insecure option
AustinAbro321 Feb 2, 2024
75847ce
lint
AustinAbro321 Feb 2, 2024
552fccc
removing unnecessary var
AustinAbro321 Feb 2, 2024
471a0fe
take publish out of oci, make mediatype a parameter
AustinAbro321 Feb 2, 2024
561412e
rework user agent
AustinAbro321 Feb 2, 2024
d224d2f
go mod tidy, remove comments
AustinAbro321 Feb 2, 2024
4d3cee1
simplifying filter
AustinAbro321 Feb 2, 2024
4f09a08
update dependencies
AustinAbro321 Feb 5, 2024
fa11785
reordered function names for api
AustinAbro321 Feb 5, 2024
40d968f
comment
AustinAbro321 Feb 5, 2024
22bfe15
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 6, 2024
c0acb3c
getting rid of finish function
AustinAbro321 Feb 6, 2024
15f31e0
getting rid of finish function
AustinAbro321 Feb 6, 2024
e026f4a
Taking channels out of pure oci package
AustinAbro321 Feb 6, 2024
252b364
merge
AustinAbro321 Feb 7, 2024
f2aff80
Apply suggestions from code review
AustinAbro321 Feb 7, 2024
66b7b14
refactor
AustinAbro321 Feb 7, 2024
5cf4f4d
refactor
AustinAbro321 Feb 7, 2024
09d02f3
use right layers
AustinAbro321 Feb 7, 2024
f81502d
refactor filter
AustinAbro321 Feb 7, 2024
67db377
updating oci version
AustinAbro321 Feb 7, 2024
e2db18c
removing unnecessary writeFile function
AustinAbro321 Feb 7, 2024
79b1c05
switching helpers.writefile to os.writefile
AustinAbro321 Feb 7, 2024
3c10035
refactor / delete functions
AustinAbro321 Feb 7, 2024
e814ead
getting rid of pull layers function
AustinAbro321 Feb 7, 2024
da5d0ff
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 7, 2024
5b8ba50
delete unused file, settle down permissions
AustinAbro321 Feb 7, 2024
579e0f8
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 8, 2024
f024a9f
lint
AustinAbro321 Feb 8, 2024
4274711
moving progresswriter and logger to helpers
AustinAbro321 Feb 8, 2024
849aeff
Update src/pkg/oci/common.go
AustinAbro321 Feb 8, 2024
85db8a4
Update src/pkg/oci/push.go
AustinAbro321 Feb 8, 2024
a944089
handle error, refactor
AustinAbro321 Feb 8, 2024
af258f5
Merge branch 'move-oci-lib-out' of github.com:defenseunicorns/zarf in…
AustinAbro321 Feb 8, 2024
13fc297
Make copy package opinated to zarf
AustinAbro321 Feb 8, 2024
c9d8369
take copy opts out of oras remote
AustinAbro321 Feb 8, 2024
c354814
rename
AustinAbro321 Feb 9, 2024
f43b05d
adding constants for file permissions
AustinAbro321 Feb 9, 2024
0f844ef
naming commonly used file permission modes
AustinAbro321 Feb 9, 2024
e2a5b78
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 9, 2024
dc1eef9
rename
AustinAbro321 Feb 9, 2024
2149a37
moving and renaming getcopyopts
AustinAbro321 Feb 9, 2024
a5ebef6
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 9, 2024
b2c1036
Remove include as an option to zoci copy package, fix package destina…
AustinAbro321 Feb 9, 2024
f1cbd1a
Merge branch 'move-oci-lib-out' of github.com:defenseunicorns/zarf in…
AustinAbro321 Feb 9, 2024
1a5653c
remove unncessary error check
AustinAbro321 Feb 9, 2024
e0e72b5
change logger to follow slog interface
AustinAbro321 Feb 12, 2024
e3d4930
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 12, 2024
fe17afe
rename io permissions
AustinAbro321 Feb 12, 2024
d80c3a1
using oras remote logger in zoci, composing transport with roundtripper
AustinAbro321 Feb 12, 2024
876f73e
re-adding base
AustinAbro321 Feb 13, 2024
7aad117
lint
AustinAbro321 Feb 13, 2024
543f001
make logger a pointer
AustinAbro321 Feb 13, 2024
c81ca36
turning log.go into progress.go
AustinAbro321 Feb 13, 2024
dd68dba
fix roundtripper
AustinAbro321 Feb 13, 2024
338f0ac
rename
AustinAbro321 Feb 13, 2024
2956b47
simplified layersfromrequestedcomponents
AustinAbro321 Feb 13, 2024
ef5b270
unneeded code in test
AustinAbro321 Feb 13, 2024
7ed05a6
added media type to index
AustinAbro321 Feb 14, 2024
86486b9
lint
AustinAbro321 Feb 16, 2024
9539983
merge
AustinAbro321 Feb 20, 2024
b604441
update title
AustinAbro321 Feb 20, 2024
85e8482
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 20, 2024
03a9ff7
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 20, 2024
f97c833
rename
AustinAbro321 Feb 21, 2024
27f30c1
Merge branch 'move-oci-lib-out' of github.com:defenseunicorns/zarf in…
AustinAbro321 Feb 21, 2024
476bade
merge
AustinAbro321 Feb 21, 2024
0a980cf
lint
AustinAbro321 Feb 21, 2024
443fda0
Apply suggestions from code review
AustinAbro321 Feb 22, 2024
cca1703
moving variables to layout
AustinAbro321 Feb 22, 2024
5133448
Merge branch 'move-oci-lib-out' of github.com:defenseunicorns/zarf in…
AustinAbro321 Feb 22, 2024
2a5de67
merge
AustinAbro321 Feb 22, 2024
679eaf3
fix
AustinAbro321 Feb 22, 2024
2a7eb7d
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 26, 2024
ca2a018
lint
AustinAbro321 Feb 26, 2024
9943d61
spelling
AustinAbro321 Feb 26, 2024
9572087
Merge branch 'main' into move-oci-lib-out
lucasrod16 Feb 26, 2024
fe9577d
rename
AustinAbro321 Feb 27, 2024
f090179
function rename
AustinAbro321 Feb 27, 2024
fc31c89
more correct progress total
AustinAbro321 Feb 27, 2024
3101a7e
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Feb 27, 2024
765f152
update comments
AustinAbro321 Feb 28, 2024
332f32f
moving out validate reference as it is unnecessary
AustinAbro321 Feb 28, 2024
30ecbe6
Merge branch 'main' into move-oci-lib-out
lucasrod16 Feb 29, 2024
9d70650
correcting progres bar total
AustinAbro321 Feb 29, 2024
f967713
Merge branch 'move-oci-lib-out' of github.com:defenseunicorns/zarf in…
AustinAbro321 Feb 29, 2024
ec54512
fixing error progress bar logic in push
AustinAbro321 Feb 29, 2024
dc714a4
Merge branch 'main' into move-oci-lib-out
Racer159 Mar 4, 2024
f1f1439
switching back to map
AustinAbro321 Mar 4, 2024
948216e
Merge branch 'move-oci-lib-out' of github.com:defenseunicorns/zarf in…
AustinAbro321 Mar 4, 2024
74709bc
Merge branch 'main' into move-oci-lib-out
AustinAbro321 Mar 4, 2024
a10b117
Merge branch 'main' into move-oci-lib-out
Racer159 Mar 4, 2024
ea06cdd
back to 1 media type!
AustinAbro321 Mar 4, 2024
8613b74
Merge branch 'main' into move-oci-lib-out
Racer159 Mar 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 5 additions & 4 deletions src/cmd/initialize.go
Expand Up @@ -22,6 +22,7 @@ import (
"github.com/defenseunicorns/zarf/src/pkg/packager/sources"
"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/defenseunicorns/zarf/src/pkg/zoci"

"github.com/spf13/cobra"
)
Expand Down Expand Up @@ -91,7 +92,7 @@ func findInitPackage(initPackageName string) (string, error) {

// Create the cache directory if it doesn't exist
if utils.InvalidPath(config.GetAbsCachePath()) {
if err := utils.CreateDirectory(config.GetAbsCachePath(), 0755); err != nil {
if err := utils.CreateDirectory(config.GetAbsCachePath(), helpers.ReadExecuteAllWriteUser); err != nil {
message.Fatalf(err, lang.CmdInitErrUnableCreateCache, config.GetAbsCachePath())
}
}
Expand Down Expand Up @@ -119,7 +120,7 @@ func downloadInitPackage(cacheDirectory string) (string, error) {
}

var confirmDownload bool
url := oci.GetInitPackageURL(config.CLIVersion)
url := zoci.GetInitPackageURL(config.CLIVersion)

// Give the user the choice to download the init-package and note that this does require an internet connection
message.Question(fmt.Sprintf(lang.CmdInitPullAsk, url))
Expand All @@ -138,11 +139,11 @@ func downloadInitPackage(cacheDirectory string) (string, error) {

// If the user wants to download the init-package, download it
if confirmDownload {
remote, err := oci.NewOrasRemote(url, oci.PlatformForArch(config.GetArch()))
remote, err := zoci.NewRemote(url, oci.PlatformForArch(config.GetArch()))
if err != nil {
return "", err
}
source := sources.OCISource{OrasRemote: remote}
source := &sources.OCISource{Remote: remote}
return source.Collect(cacheDirectory)
}
// Otherwise, exit and tell the user to manually download the init-package
Expand Down
3 changes: 2 additions & 1 deletion src/cmd/tools/helm/repo_add.go
Expand Up @@ -30,6 +30,7 @@ import (
"strings"
"time"

"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/gofrs/flock"
"github.com/pkg/errors"
"github.com/spf13/cobra"
Expand Down Expand Up @@ -216,7 +217,7 @@ func (o *repoAddOptions) run(out io.Writer) error {

f.Update(&c)

if err := f.WriteFile(o.repoFile, 0600); err != nil {
if err := f.WriteFile(o.repoFile, helpers.ReadWriteUser); err != nil {
return err
}
fmt.Fprintf(out, "%q has been added to your repositories\n", o.name)
Expand Down
5 changes: 3 additions & 2 deletions src/cmd/tools/helm/repo_index.go
Expand Up @@ -26,6 +26,7 @@ import (
"os"
"path/filepath"

"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/pkg/errors"
"github.com/spf13/cobra"

Expand Down Expand Up @@ -100,7 +101,7 @@ func index(dir, url, mergeTo string) error {
var i2 *repo.IndexFile
if _, err := os.Stat(mergeTo); os.IsNotExist(err) {
i2 = repo.NewIndexFile()
i2.WriteFile(mergeTo, 0644)
i2.WriteFile(mergeTo, helpers.ReadAllWriteUser)
} else {
i2, err = repo.LoadIndexFile(mergeTo)
if err != nil {
Expand All @@ -110,5 +111,5 @@ func index(dir, url, mergeTo string) error {
i.Merge(i2)
}
i.SortEntries()
return i.WriteFile(out, 0644)
return i.WriteFile(out, helpers.ReadAllWriteUser)
}
3 changes: 2 additions & 1 deletion src/cmd/tools/helm/repo_remove.go
Expand Up @@ -27,6 +27,7 @@ import (
"os"
"path/filepath"

"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/pkg/errors"
"github.com/spf13/cobra"

Expand Down Expand Up @@ -72,7 +73,7 @@ func (o *repoRemoveOptions) run(out io.Writer) error {
if !r.Remove(name) {
return errors.Errorf("no repo named %q found", name)
}
if err := r.WriteFile(o.repoFile, 0600); err != nil {
if err := r.WriteFile(o.repoFile, helpers.ReadWriteUser); err != nil {
return err
}

Expand Down
18 changes: 10 additions & 8 deletions src/cmd/tools/zarf.go
Expand Up @@ -21,6 +21,8 @@ import (
"github.com/defenseunicorns/zarf/src/pkg/oci"
"github.com/defenseunicorns/zarf/src/pkg/packager/sources"
"github.com/defenseunicorns/zarf/src/pkg/pki"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/defenseunicorns/zarf/src/pkg/zoci"
"github.com/defenseunicorns/zarf/src/types"
"github.com/sigstore/cosign/v2/pkg/cosign"
"github.com/spf13/cobra"
Expand Down Expand Up @@ -181,14 +183,14 @@ var downloadInitCmd = &cobra.Command{
Use: "download-init",
Short: lang.CmdToolsDownloadInitShort,
Run: func(_ *cobra.Command, _ []string) {
url := oci.GetInitPackageURL(config.CLIVersion)
url := zoci.GetInitPackageURL(config.CLIVersion)

remote, err := oci.NewOrasRemote(url, oci.PlatformForArch(config.GetArch()))
remote, err := zoci.NewRemote(url, oci.PlatformForArch(config.GetArch()))
if err != nil {
message.Fatalf(err, lang.CmdToolsDownloadInitErr, err.Error())
}

source := &sources.OCISource{OrasRemote: remote}
source := &sources.OCISource{Remote: remote}

_, err = source.Collect(outputDirectory)
if err != nil {
Expand All @@ -204,13 +206,13 @@ var generatePKICmd = &cobra.Command{
Args: cobra.ExactArgs(1),
Run: func(_ *cobra.Command, args []string) {
pki := pki.GeneratePKI(args[0], subAltNames...)
if err := os.WriteFile("tls.ca", pki.CA, 0644); err != nil {
if err := os.WriteFile("tls.ca", pki.CA, helpers.ReadAllWriteUser); err != nil {
message.Fatalf(err, lang.ErrWritingFile, "tls.ca", err.Error())
}
if err := os.WriteFile("tls.crt", pki.Cert, 0644); err != nil {
if err := os.WriteFile("tls.crt", pki.Cert, helpers.ReadAllWriteUser); err != nil {
message.Fatalf(err, lang.ErrWritingFile, "tls.crt", err.Error())
}
if err := os.WriteFile("tls.key", pki.Key, 0600); err != nil {
if err := os.WriteFile("tls.key", pki.Key, helpers.ReadWriteUser); err != nil {
message.Fatalf(err, lang.ErrWritingFile, "tls.key", err.Error())
}
message.Successf(lang.CmdToolsGenPkiSuccess, args[0])
Expand Down Expand Up @@ -278,10 +280,10 @@ var generateKeyCmd = &cobra.Command{
}

// Write the key file contents to disk
if err := os.WriteFile(prvKeyFileName, keyBytes.PrivateBytes, 0600); err != nil {
if err := os.WriteFile(prvKeyFileName, keyBytes.PrivateBytes, helpers.ReadWriteUser); err != nil {
message.Fatalf(err, lang.ErrWritingFile, prvKeyFileName, err.Error())
}
if err := os.WriteFile(pubKeyFileName, keyBytes.PublicBytes, 0644); err != nil {
if err := os.WriteFile(pubKeyFileName, keyBytes.PublicBytes, helpers.ReadAllWriteUser); err != nil {
message.Fatalf(err, lang.ErrWritingFile, pubKeyFileName, err.Error())
}

Expand Down
2 changes: 1 addition & 1 deletion src/extensions/bigbang/bigbang.go
Expand Up @@ -465,7 +465,7 @@ func addBigBangManifests(YOLO bool, manifestDir string, cfg *extensions.BigBang)
return err
}

if err := utils.WriteFile(path, out); err != nil {
if err := os.WriteFile(path, out, helpers.ReadWriteUser); err != nil {
return err
}

Expand Down
2 changes: 1 addition & 1 deletion src/extensions/bigbang/flux.go
Expand Up @@ -62,7 +62,7 @@ func getFlux(baseDir string, cfg *extensions.BigBang) (manifest types.ZarfManife
fluxKustomization.Patches = append(fluxKustomization.Patches, krustytypes.Patch{Path: absFluxPatchPath})
}

if err := utils.WriteYaml(kustomizePath, fluxKustomization, 0600); err != nil {
if err := utils.WriteYaml(kustomizePath, fluxKustomization, helpers.ReadWriteUser); err != nil {
return manifest, images, fmt.Errorf("unable to write kustomization: %w", err)
}

Expand Down
4 changes: 2 additions & 2 deletions src/internal/packager/helm/post-render.go
Expand Up @@ -15,6 +15,7 @@ import (
"github.com/defenseunicorns/zarf/src/internal/packager/template"
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/defenseunicorns/zarf/src/types"
"helm.sh/helm/v3/pkg/action"
"helm.sh/helm/v3/pkg/releaseutil"
Expand Down Expand Up @@ -61,8 +62,7 @@ func (r *renderer) Run(renderedManifests *bytes.Buffer) (*bytes.Buffer, error) {
}
path := filepath.Join(tempDir, "chart.yaml")

// Write the context to a file for processing
if err := utils.WriteFile(path, renderedManifests.Bytes()); err != nil {
if err := os.WriteFile(path, renderedManifests.Bytes(), helpers.ReadWriteUser); err != nil {
return nil, fmt.Errorf("unable to write the post-render file for the helm chart")
}

Expand Down
2 changes: 1 addition & 1 deletion src/internal/packager/helm/repo.go
Expand Up @@ -204,7 +204,7 @@ func (h *Helm) DownloadPublishedChart(cosignKeyPath string) error {

// Download the file into a temp directory since we don't control what name helm creates here
temp := filepath.Join(h.chartPath, "temp")
if err = utils.CreateDirectory(temp, 0700); err != nil {
if err = utils.CreateDirectory(temp, helpers.ReadWriteExecuteUser); err != nil {
return fmt.Errorf("unable to create helm chart temp directory: %w", err)
}
defer os.RemoveAll(temp)
Expand Down
29 changes: 13 additions & 16 deletions src/internal/packager/images/pull.go
Expand Up @@ -12,13 +12,13 @@ import (
"os"
"path/filepath"
"strings"
"sync"

"github.com/defenseunicorns/zarf/src/config"
"github.com/defenseunicorns/zarf/src/pkg/layout"
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/defenseunicorns/zarf/src/pkg/transform"
"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/google/go-containerregistry/pkg/crane"
"github.com/google/go-containerregistry/pkg/logs"
"github.com/google/go-containerregistry/pkg/name"
Expand Down Expand Up @@ -67,7 +67,7 @@ func (i *ImageConfig) PullAll() ([]ImgInfo, error) {
logs.Warn.SetOutput(&message.DebugWriter{})
logs.Progress.SetOutput(&message.DebugWriter{})

metadataImageConcurrency := utils.NewConcurrencyTools[ImgInfo, error](len(i.ImageList))
metadataImageConcurrency := helpers.NewConcurrencyTools[ImgInfo, error](len(i.ImageList))

defer metadataImageConcurrency.Cancel()

Expand Down Expand Up @@ -123,7 +123,7 @@ func (i *ImageConfig) PullAll() ([]ImgInfo, error) {
}

// Create the ImagePath directory
if err := utils.CreateDirectory(i.ImagesPath, 0755); err != nil {
if err := utils.CreateDirectory(i.ImagesPath, helpers.ReadExecuteAllWriteUser); err != nil {
return nil, fmt.Errorf("failed to create image path %s: %w", i.ImagesPath, err)
}

Expand Down Expand Up @@ -178,16 +178,13 @@ func (i *ImageConfig) PullAll() ([]ImgInfo, error) {
spinner.Success()

// Create a thread to update a progress bar as we save the image files to disk
doneSaving := make(chan int)
errorSaving := make(chan int)
var progressBarWaitGroup sync.WaitGroup
progressBarWaitGroup.Add(1)
doneSaving := make(chan error)
updateText := fmt.Sprintf("Pulling %d images", imageCount)
go utils.RenderProgressBarForLocalDirWrite(i.ImagesPath, totalBytes, &progressBarWaitGroup, doneSaving, errorSaving, updateText, updateText)
go utils.RenderProgressBarForLocalDirWrite(i.ImagesPath, totalBytes, doneSaving, updateText, updateText)

// Spawn a goroutine for each layer to write it to disk using crane

layerWritingConcurrency := utils.NewConcurrencyTools[bool, error](len(processedLayers))
layerWritingConcurrency := helpers.NewConcurrencyTools[bool, error](len(processedLayers))

defer layerWritingConcurrency.Cancel()

Expand Down Expand Up @@ -318,8 +315,8 @@ func (i *ImageConfig) PullAll() ([]ImgInfo, error) {

onLayerWritingError := func(err error) error {
// Send a signal to the progress bar that we're done and wait for the thread to finish
errorSaving <- 1
progressBarWaitGroup.Wait()
doneSaving <- err
<-doneSaving
message.WarnErr(err, "Failed to write image layers, trying again up to 3 times...")
if strings.HasPrefix(err.Error(), "expected blob size") {
message.Warnf("Potential image cache corruption: %s - try clearing cache with \"zarf tools clear-cache\"", err.Error())
Expand All @@ -331,7 +328,7 @@ func (i *ImageConfig) PullAll() ([]ImgInfo, error) {
return nil, err
}

imageSavingConcurrency := utils.NewConcurrencyTools[digestInfo, error](len(refInfoToImage))
imageSavingConcurrency := helpers.NewConcurrencyTools[digestInfo, error](len(refInfoToImage))

defer imageSavingConcurrency.Cancel()

Expand Down Expand Up @@ -382,8 +379,8 @@ func (i *ImageConfig) PullAll() ([]ImgInfo, error) {

onImageSavingError := func(err error) error {
// Send a signal to the progress bar that we're done and wait for the thread to finish
errorSaving <- 1
progressBarWaitGroup.Wait()
doneSaving <- err
<-doneSaving
message.WarnErr(err, "Failed to write image config or manifest, trying again up to 3 times...")
return err
}
Expand Down Expand Up @@ -418,8 +415,8 @@ func (i *ImageConfig) PullAll() ([]ImgInfo, error) {
}

// Send a signal to the progress bar that we're done and wait for the thread to finish
doneSaving <- 1
progressBarWaitGroup.Wait()
doneSaving <- nil
<-doneSaving

return imgInfoList, nil
}
Expand Down
3 changes: 2 additions & 1 deletion src/internal/packager/images/push.go
Expand Up @@ -15,6 +15,7 @@ import (
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/defenseunicorns/zarf/src/pkg/transform"
"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/google/go-containerregistry/pkg/crane"
"github.com/google/go-containerregistry/pkg/logs"
v1 "github.com/google/go-containerregistry/pkg/v1"
Expand Down Expand Up @@ -55,7 +56,7 @@ func (i *ImageConfig) PushToZarfRegistry() error {
httpTransport.ResponseHeaderTimeout = 10 * time.Second
progressBar := message.NewProgressBar(totalSize, fmt.Sprintf("Pushing %d images to the zarf registry", len(i.ImageList)))
defer progressBar.Stop()
craneTransport := utils.NewTransport(httpTransport, progressBar)
craneTransport := helpers.NewTransport(httpTransport, progressBar)

pushOptions := config.GetCraneOptions(i.Insecure, i.Architectures...)
pushOptions = append(pushOptions, config.GetCraneAuthOption(i.RegInfo.PushUsername, i.RegInfo.PushPassword))
Expand Down
5 changes: 3 additions & 2 deletions src/internal/packager/kustomize/build.go
Expand Up @@ -6,8 +6,9 @@ package kustomize

import (
"fmt"
"os"

"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"sigs.k8s.io/kustomize/api/krusty"
krustytypes "sigs.k8s.io/kustomize/api/types"
"sigs.k8s.io/kustomize/kyaml/filesys"
Expand Down Expand Up @@ -39,5 +40,5 @@ func Build(path string, destination string, kustomizeAllowAnyDirectory bool) err
return fmt.Errorf("problem converting kustomization to yaml: %w", err)
}

return utils.WriteFile(destination, yaml)
return os.WriteFile(destination, yaml, helpers.ReadWriteUser)
}
5 changes: 3 additions & 2 deletions src/internal/packager/sbom/catalog.go
Expand Up @@ -28,6 +28,7 @@ import (
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/defenseunicorns/zarf/src/pkg/transform"
"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
v1 "github.com/google/go-containerregistry/pkg/v1"
)

Expand Down Expand Up @@ -59,7 +60,7 @@ func Catalog(componentSBOMs map[string]*layout.ComponentSBOM, imageList []transf
defer builder.spinner.Stop()

// Ensure the sbom directory exists
_ = utils.CreateDirectory(builder.outputDir, 0700)
_ = utils.CreateDirectory(builder.outputDir, helpers.ReadWriteExecuteUser)

// Generate a list of images and files for the sbom viewer
json, err := builder.generateJSONList(componentSBOMs, imageList)
Expand Down Expand Up @@ -151,7 +152,7 @@ func (b *Builder) createImageSBOM(img v1.Image, src string) ([]byte, error) {
imageCachePath := filepath.Join(b.cachePath, layout.ImagesDir)

// Ensure the image cache directory exists.
if err := utils.CreateDirectory(imageCachePath, 0700); err != nil {
if err := utils.CreateDirectory(imageCachePath, helpers.ReadWriteExecuteUser); err != nil {
return nil, err
}

Expand Down
3 changes: 2 additions & 1 deletion src/internal/packager/sbom/tools.go
Expand Up @@ -13,6 +13,7 @@ import (
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/exec"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/defenseunicorns/zarf/src/types"
)

Expand Down Expand Up @@ -49,7 +50,7 @@ func OutputSBOMFiles(sourceDir, outputDir, packageName string) (string, error) {
return "", err
}

if err := utils.CreateDirectory(packagePath, 0700); err != nil {
if err := utils.CreateDirectory(packagePath, helpers.ReadWriteExecuteUser); err != nil {
return "", err
}

Expand Down
3 changes: 2 additions & 1 deletion src/pkg/cluster/data.go
Expand Up @@ -18,6 +18,7 @@ import (
"github.com/defenseunicorns/zarf/src/pkg/message"
"github.com/defenseunicorns/zarf/src/pkg/utils"
"github.com/defenseunicorns/zarf/src/pkg/utils/exec"
"github.com/defenseunicorns/zarf/src/pkg/utils/helpers"
"github.com/defenseunicorns/zarf/src/types"
corev1 "k8s.io/api/core/v1"
)
Expand All @@ -28,7 +29,7 @@ func (c *Cluster) HandleDataInjection(wg *sync.WaitGroup, data types.ZarfDataInj
defer wg.Done()

injectionCompletionMarker := filepath.Join(componentPath.DataInjections, config.GetDataInjectionMarker())
if err := utils.WriteFile(injectionCompletionMarker, []byte("🦄")); err != nil {
if err := os.WriteFile(injectionCompletionMarker, []byte("🦄"), helpers.ReadWriteUser); err != nil {
message.WarnErrf(err, "Unable to create the data injection completion marker")
return
}
Expand Down