Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix timeout progress bar for cy.wait #7882

Merged
merged 12 commits into from Jul 20, 2020
Merged

Fix timeout progress bar for cy.wait #7882

merged 12 commits into from Jul 20, 2020

Conversation

panzarino
Copy link
Contributor

@panzarino panzarino commented Jul 3, 2020

User facing changelog

Fixes a bug where the timeout progress bar would not reflect the actual timeout for cy.wait().

How has the user experience changed?

Before:

ezgif-6-621a66abf4bc

After:

ezgif-6-19978dd83f9f

PR Tasks

  • Have tests been added/updated?
  • Has the original issue been tagged with a release in ZenHub?

@cypress-bot
Copy link
Contributor

cypress-bot bot commented Jul 3, 2020

Thanks for taking the time to open a PR!

@cypress
Copy link

cypress bot commented Jul 3, 2020



Test summary

7896 0 130 0


Run details

Project cypress
Status Passed
Commit a1de94d
Started Jul 20, 2020 5:19 AM
Ended Jul 20, 2020 5:27 AM
Duration 07:58 💡
OS Linux Debian - 10.1
Browser Multiple

View run in Cypress Dashboard ➡️


This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

Copy link
Member

@jennifer-shehane jennifer-shehane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does now reflect the default requestTimeout of cy.wait() in the progress bar.

I feel like this is likely related, but if you do cy.wait() with just a number timeout, the progress bar is not accurate either (bug before this PR). Where the progress bar shows it has about 10secs, but really I've set it to expire at 5 secs.

Feel free to ignore and open a new issue if you think this is outside the scope of this work.

I think there should be some tests for this PR though. 🙃

Also, I'm a bit curious how the progress bar shows for different requestTimeout + responseTimeout. So...say I have a requestTimeout waiting for the XHR to make a request for 20 secs - it take 10 secs to request, and THEN I have a responseTimeout waiting 30 secs for it to respond - it takes 20 secs to respond.

  • Does the progress bar show 50 secs of countdown?
  • When does the progress bar resolve? At the request resolution? At the response resolution?
  • Does it do the request countdown and then reset a new progress bar for response countdown?

This is the only command that combines 2 timeouts I believe, so this is a unique concern here.

@panzarino
Copy link
Contributor Author

@jennifer-shehane I've fixed the issue regarding passing a number to cy.wait() (my original implementation used the wrong timeout). I've also added tests for ensuring the log timeout is properly set with the requestTimeout and responseTimeout.

There are a couple of ways that we can go about designing the progress bar updates:

I just pushed changes that look like the following, where the progress bar will keep its position but slow down to reflect the updated timeout:

ezgif-6-1045ab4e3064

Alternatively, we can make sure that the progress bar always reflects the actual percentage complete. For example (shown in the gif below), if we're at 2000ms out of 3000ms (66.67%) and the timeout is increased to 5000ms, the percentage complete reverts to 40%.

ezgif-6-3e8f4a8b01f0

To test this design yourself, replace lines 80-86 of command-model.js with the following:

this.timeout = props.timeout

Finally, we can reset the progress bar to the beginning and have it progress to the end of the time remaining in the updated timeout:

ezgif-6-67ad218f0436

To test this design yourself, replace lines 80-86 of command-model.js with the following:

const timeElapsed = Date.now() - new Date(this.wallClockStartedAt).getTime()

this.timeout = props.timeout - timeElapsed
this.wallClockStartedAt = new Date().toJSON()

Here is the test case that I used to generate the examples:

it('tests', () => {
  const start = Date.now()
  let pass = false

  cy.on('command:retry', () => {
    if (Date.now() - start > 2000 && !pass) {
      pass = true

      // trigger request to move onto response timeout verification
      const win = cy.state('window')

      xhrGet(win, '/foo')
    }
  })

  cy
  .server({ delay: 3000 })
  .route('GET', '*', {}).as('fetch')
  .wait('@fetch', { requestTimeout: 3000, responseTimeout: 5000 })
})

Personally I think that designs 1 and 2 make the most sense. Design 1 is the most visually appealing (but might move too slow if near the end and the timeout changes to something huge like 30000) and design 2 makes sense since it actually reflects the real percentage complete (and has the most simple implementation).

@jennifer-shehane
Copy link
Member

@panzarino I think that Design 1 looks the least 'buggy'. While maybe still not completely clearly communicated why the progress bar slows down, the other 2 look more like glitches, where someone may think there was a bug with wait running twice. Thanks for putting the effort in to implement this thoroughly!

Copy link
Contributor

@JessicaSachs JessicaSachs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some code cleanliness nits.

More importantly, I found a bug with how the animation is being calculated and reset...

  • When you toggle the parent component, the latest timer state is lost. (see the gif)

issue-7881

  • If you let the timer run out and then toggle the parent component, then the animation correctly stays completed.

  • When the tests are all passing, the timeout still counts down if you expand the succeeding tests.

Let me know if you agree w those as issues. I'm around to pair tomorrow

packages/reporter/src/commands/command-model.ts Outdated Show resolved Hide resolved
packages/reporter/src/commands/command-model.ts Outdated Show resolved Hide resolved
@panzarino
Copy link
Contributor Author

So the main issue with the first design is that it doesn't look great with the default timeouts. It starts moving so slow that users might actually think something got stuck.

ezgif-6-a89e08fdd21f

The second option alleviates this issue, where the progress bar actually reflects the percentage until it times out. Since there is another command logged simultaneously when the progress changes, I don't think that users will think there is an issue and the command is running twice.

In addition, the requestTimeout unusually doesn't even come close to actually timing out (anecdotally). More often, the command is waiting on the response and the responseTimeout is quite large, and if most users are modifying it they are increasing it rather than decreasing (again, anecdotally).

Here's an example of what a more realistic use case might look like:

Option 1

ezgif-6-2861955016cc

Option 2

ezgif-6-497283c60de2

Most realistically though, is when the command is not alone and is contained in a suite of tests (the user might be debugging with only, but in that case they'll understand the request is working). In this case, the test won't even be expanded until isLongRunning becomes true. This is what that looks like:

ezgif-6-47c6f69dda64

It's not even discernible which version I used in that demo. Therefore I suggest that we use option 2 to simplify the code and display the real percentage complete.

@JessicaSachs regarding the issue with toggling, during our brief discussion on this issue earlier I believe I said that was introduced when I added a key to the progress bar to make it rerender. That is not the case. I had previously thought this was a non issue because when toggling the test (not the hook) on and off the progress bar would remain correct. That's because that toggle actually does cause a rerender, while clicking the hook does not. I'll file a separate issue and PR.

@jennifer-shehane
Copy link
Member

@panzarino Yeah I can see your point. Design 2 is fine with me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Timeout progress indicator for cy.wait is incorrect if timeout option is not passed
3 participants