Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add --reproducible flag to flux push artifact #4769

Merged
merged 1 commit into from May 8, 2024

Conversation

frekw
Copy link
Contributor

@frekw frekw commented May 7, 2024

This makes the pushed artifact have the exact same hash if the contents are the same.

E.g

flux push artifact oci://repo/image:tag1 --source deploy --revision="test" --path=deploy --reproducible
flux push artifact oci://repo/image:tag2 --source deploy --revision="test" --path=deploy --reproducible

will both result in the same sha hash, tagged with tag1 and tag2.

This is useful when producing flux artifacts in a monorepo setup where you don't want to unnecessarily push new artifacts unless something has actually changed.

Copy link

@jgranstrom jgranstrom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the only way to make the builds idempotent

@@ -202,6 +204,12 @@ func pushArtifactCmdRun(cmd *cobra.Command, args []string) error {
Annotations: annotations,
}

if pushArtifactArgs.reproducible {
zeroTime := time.Unix(0, 0)
meta.Revision = ""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not Ok, a reproducible build is for a Git SHA.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if we annotate the artifact with the git SHA then the image digest (i.e the sha256:9328ca7d5a003880b9a31140436cf43bead8fa289fba55103e661e5aaf9ad1f3) will always differ regardless of it the contents differ or not. This is only for circumstances where you need the digest to be exactly the same for the same contents, akin to https://github.com/GoogleContainerTools/kaniko?tab=readme-ov-file#flag---reproducible

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not Ok, a reproducible build is for a Git SHA.

That's not true, a reproducible build is for an image digest, commit SHAs are completely unrelated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could remove the meta.Revision = "" it we allow passing in an empty revision if reproducible is set;

diff --git a/cmd/flux/push_artifact.go b/cmd/flux/push_artifact.go
index d98361af..7a5063fe 100644
--- a/cmd/flux/push_artifact.go
+++ b/cmd/flux/push_artifact.go
@@ -151,7 +151,7 @@ func pushArtifactCmdRun(cmd *cobra.Command, args []string) error {
 		return fmt.Errorf("--source is required")
 	}
 
-	if pushArtifactArgs.revision == "" {
+	if pushArtifactArgs.revision == "" && !pushArtifactArgs.reproducible {
 		return fmt.Errorf("--revision is required")
 	}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say that reproducible should be just that, reproducible by default, what would the default behavior end up being here? As in would it be a bit unpredictable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not having a revision goes against what I think Flux OCI should be, IMO every object managed by Flux should allow tracing back to its Git origin. If you don't care about this, then feel free to set the revision to some static value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefanprodan fair enough, pushed an update that only changes meta.Created 👍

@frekw frekw force-pushed the feat/reproducible-push branch 3 times, most recently from 55720ba to ea12aa7 Compare May 7, 2024 19:29
@stefanprodan stefanprodan changed the title Add --reprodicuble flag to flux push artifact Add --reproducible flag to flux push artifact May 8, 2024
@stefanprodan
Copy link
Member

There is a typo in the commit message, please fix it to --reproducible

@frekw
Copy link
Contributor Author

frekw commented May 8, 2024

Whoops, sorry, typo should be fixed now! 🙏

@stefanprodan
Copy link
Member

Can you please squash and rebase with upstream main.

This makes the pushed artifact have the exact same hash if the contents
are the same.

E.g
```
flux push artifact oci://repo/image:tag1 --source deploy --revision="test" --path=deploy --reproducible
flux push artifact oci://repo/image:tag2 --source deploy --revision="test" --path=deploy --reproducible
```

will both result in the same sha hash, tagged with `tag1` and `tag2`.

This is useful when producing flux artifacts in a monorepo setup where
you don't want to unnecessarily push new artifacts unless something has
actually changed.

Signed-off-by: frekw <fredrik@warnsberg.se>
@frekw
Copy link
Contributor Author

frekw commented May 8, 2024

Of course!

@stefanprodan stefanprodan added the area/oci OCI related issues and pull requests label May 8, 2024
@stefanprodan stefanprodan mentioned this pull request May 8, 2024
59 tasks
Copy link
Member

@stefanprodan stefanprodan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Thanks @frekw 🏅

@stefanprodan stefanprodan merged commit c1ff78c into fluxcd:main May 8, 2024
8 checks passed
@frekw frekw deleted the feat/reproducible-push branch May 8, 2024 11:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/oci OCI related issues and pull requests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants