Adapt strategy for automated testing

[s-pic]

What's the problem you want solved?

• At some places we test too much using e2e tests. Generally speaking e2e tests require a lot of effort to write and to maintain. They should only test actual and important user flows and the the interaction between all or many elements of the software stack.
  See https://martinfowler.com/articles/practical-test-pyramid.html, https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html and https://kentcdodds.com/blog/write-tests
• We lack unit and integration tests for components/services. Those are easier to write/maintain, they offer a faster feedback loop and isolate broken features.

Is there a solution you'd like to recommend?

Adapt our testing strategy.

• Focus on good effort/value proportions.
• Demonstrate how we can test single components to replace e2e tests which took over that role.
• Write UI-Tests for components which are complex/reusable
• Write e2e tests for the most important user flows.
• consider remove tests for features added by 3rd party libraries. It is argued that if a library is used, it has to be trusted to be tested and production-ready in advance

Tasks

• Compare relevant libraries for unit-/integration-testing react components -> see Adapt strategy for automated testing  #341 (comment)
• Implement UI-tests for a couple of components that are re-used across the app, e.g. the WebGl map OR that are designed to be re-used (the AutoCompleteGeocoder).
  Doing this in https://github.com/FixMyBerlin/fixmy.frontend/tree/testing/extend_component_unit_tests
• Determine out what e2e tests we can ditch and which flows actually need e2e tests
  Adapt strategy for automated testing  #341 (comment)

[s-pic]

Sorry @ciex , I assigned you accidently

[s-pic]

From what I read React Testing Library looks more promising.
For an introductive article that also shows/compares the syntax and aproaches used in both libs,
check https://claritydev.net/blog/testing-react-components-enzyme-vs-react-testing-l/

Pro React Testing Libary

• React Testing Libraries supports Hooks out-of-the-box by using
  a sibling repo: https://github.com/testing-library/react-hooks-testing-library.

• Enzyme does not offer it yet: React hooks support checklist enzymejs/enzyme#2011

Furthermore:

Rather than tests focusing on the implementation (Enzyme),
tests are more focused on user behavior (react-testing-library).
...
The best thing about this mindset change is you are thinking about your users within your tests, thinking about how they interact with it rather than thinking about how the props and state objects look (implementation).
...
Focusing on testing users behavior rather than the implementation allows you to easily refactor code going forward. You want your user experience to be the same regardless of your implementation details…

https://medium.com/@boyney123/my-experience-moving-from-enzyme-to-react-testing-library-5ac65d992ce

Contra

The only tradeoff (if you can call that) is that I have to mock way more stuff using react-testing-library than using Enzyme. It makes tests more verbose, but... I think I like it better! It helps me isolate what to test from a component.

https://dev.to/yuritoledo/enzyme-or-react-testing-library-and-why-3gk6

[s-pic]

Regarding

Determine out what e2e tests we can ditch and which flows actually need e2e tests

• IMHO okay are
  • tests within src/pages/Reports/tests/form.e2e.test.js that check that different mechanisms (routing, geolocation api)
    or components (e.g. searchbar and map) work well together. Actually I am a bit unsure about the latter part, I think we will in how far we can replace e2e tests with integration tests, react testing library seems very capable to test high level component that are not shallow-rendered!
• I am still sure that e2e tests are a mujst-have where we test how our client and our api work together (login flow, fetching of map data, etc.).
• We should remove tests testing the bahaviour within a single component, e.g. here (not doable, see FixMyBerlin/fixmy.frontend@5cb133d) or here

• Do we still need to run e2e test files in src/pages/KatasterKI/tests/e2e?

[s-pic]

Nach unserem intensivem Brainstorming und dem Beschluss

• Cypress Tests wieder zum Laufen zu bekommen (Fallse Negatives)
• eine Art Smoke-Test zu schreiben, der bestimmte Seiten je nach Städte-Config lädt und nach Fehlern in der Konsole sucht
  kann das hier m.E. geschlossen werden