[Test Case] - Test the Get Started guides

[majakurcius]

Summary

The purpose of the test is to make sure all the Get Started guides are up to date and working properly.

Test Steps

1. Download the kubeconfig from Kyma.
2. Go through the Get Started guides and make sure they are up to date and working properly.

• [Test Case] - Get Started guides - Quick install #15621
• [Test Case] - Get Started guides - Deploy and expose a Function #15616
• [Test Case] - Get Started guides - Deploy and expose a microservice #15615
• [Test Case] - Get Started guides - Trigger a workload with an event #15623
• [Test Case] - Get Started guides - Observability #15619

CAUTION: Remember to test BOTH PATHS - CLI and Kyma Dashboard ❗ Just because one is working doesn't necessarily mean the other must be too. There might be new UI changes that haven't been accounted for, or some backend case that was not implemented in the UI, or a bug/typo in a CLI command, and so on.

Expected Results

All the tutorials are up to date and working properly.

[Disper]

Suggested approach by @ameteiko:

How I would approach it, is:

1. base the GSGs on the human readable End-to-End tests (preferably in Gherkin) written and supported by POs
2. come-up with an automation test suite that runs the E2E tests from the step 1 on every GSG change and on every release
3. establish a procedure for Technical Writers that reflects the scenarios from the E2E test suite in the GSGs

By doing this there is just one point of failure, which is the conformance of the human verbalisation in the GSG to the E2E test.
And an additional complexity of establishing the infrastructure and the orchestration for such a test suite that does not yet exists.

[Disper]

Two additional approaches

• (Katakoda-like approach) Katakoda could be a nice tool that shows how to use Kyma, however it has been closed to public access recently. There is an alternative to it https://killercoda.com/creators but it would need to be further investigated.
• (Very pragmatic approach) First, make sure that the parts of Get Started Guides are tested in the code. Either by having a separate test for the whole flow, or by finding already existing parts of code that is testing the same aspect as are steps in GSG. Add some comments that will remind people that are changing anything in the code to also verify if GSG are up to date. The work here can be split in the same way as GSG are split:
  • Quick install
  • Deploy and expose a Function
  • Deploy and expose a microservice
  • Trigger a workload with an event
  • Observability

[Disper]

I will remove test-case label as this test case was split into a smaller tickets that are linked in the issue description.