Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e tests #1931

Open
krancour opened this issue May 2, 2024 · 2 comments
Open

e2e tests #1931

krancour opened this issue May 2, 2024 · 2 comments

Comments

@krancour
Copy link
Member

krancour commented May 2, 2024

I use these besboke examples to validate a broad swath of Kargo functionality prior to release. It's as of fairly recently that I updated the examples such that they can all run concurrently.

I'd like to automate these so they run at least nightly, if not during every CI run.

Proposal:

  1. Update the examples to be something more like example templates so that branch names and such can be set dynamically on a per-test basis. This will avoid us needing to do something more heavy-handed like forking the GitOps repo with the test manifests for every test.
  2. Provision a KinD or k3D cluster in Actions.
  3. Install Kargo and its pre-reqs on that cluster.
    1. This is probably going to require that we start pushing un-validated images to a remote repo as part of the process
  4. Create shell script to assert successful installation
  5. Create a shell script to drive much of the example setup via Kargo CLI.
  6. Assert success two ways:
    1. With shell script using CLI (to maximize e2e coverage of the CLI binary)
    2. With Go tests (because we'll be able to make more detailed assertions this way)
  7. Create a shell script to drive example tear-down via Kargo CLI. We will cover more functionality this way than if we just tear down implicitly via the Action simply concluding and taking the KinD or k3d cluster with it.
  8. Assert success of tear-down two ways:
    1. Shell script
    2. Go tests
@hiddeco
Copy link
Contributor

hiddeco commented May 2, 2024

This is probably going to require that we start pushing un-validated images to a remote repo as part of the process

You can quite easily load container images into a KinD cluster, which means you do not have to push things to a registry at all.

@krancour
Copy link
Member Author

krancour commented May 2, 2024

You can quite easily load container images into a KinD cluster, which means you do not have to push things to a registry at all.

I forgot all about that!

I suppose that will require that the build and e2e tests occur in a single job, but that's ok.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants