Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flakiness often after animations between screens #1703

Open
milesingrams opened this issue Mar 19, 2024 · 2 comments
Open

Flakiness often after animations between screens #1703

milesingrams opened this issue Mar 19, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@milesingrams
Copy link

Describe the bug
We very frequently have tests flake/fail on button press/tap. The failures are random and can occur on almost any button press. Seems like the press is executed by maestro but doesn't register in the app since things are moving. Adding -waitForAnimationToEnd after every step slightly improves the success rate. The failures tend to happen more frequently after an animation/motion like navigating between React Navigation pages or when the button is on a swipeable action sheet. Not sure what the best way to resolve would be but perhaps the ability to add a per action "minimumWait" time before executing the action after the element is found.

For instance:

- tapOn:
  text: My Button
  minimumWait: 200

would wait an additional 200ms after My Button is found to execute the tap.

Another potential solution would be to retry an action if the UI doesnt change after executing the action. Or retry previous step when the next step fails.

To Reproduce

Have an app with lots of moving/animating parts and execute actions right after navigating pages or things move.

Expected behavior

Here is an example. In this example the Acknowledge button tap didn't execute despite being found and visible.

Screen Shot 2024-03-19 at 1 46 19 PM

This should work consistently without flaking.

Environment information (please complete the following information):

  • Maestro version: Maestro Cloud Default
  • Platform: iOS version 15.5
  • Framework: Expo
  • Device model and OS version: Maestro Cloud Default
  • Simulator, emulator or physical device: Maestro Cloud Default
  • Host: Maestro Cloud Default
@milesingrams milesingrams added the bug Something isn't working label Mar 19, 2024
@milesingrams milesingrams changed the title [Maestro version, e.g. v1.2.3] Flakiness often after animations between screens Mar 25, 2024
@Fishbowler
Copy link
Contributor

is waitForAnimationToEnd useful here? Or will it never settle to something a bit more static?

@milesingrams
Copy link
Author

It definitely has helped, but i believe mostly cause it introduces a slight additional delay which allows the UI to settle.
After investigating more it seems the main root cause is React Navigation animations which cause a high rate of flakiness post transition.By turning off animations in React Navigation we've had a huge improvement in failure rate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants