Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log browser console errors experienced when running TestCafe tests. #1030

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

richardxia
Copy link
Member

I'm having a terrible time debugging or reproducing the TestCafe errors in master.

No matter what I do, I can't reproduce the errors locally, even when I check out the master branches of both repos and I reset my database. When I look at the video recording of the browser test failures in Sauce Labs, I can see that in the "Add resource phone number" always triggers a "Sorry! An error occurred." message when the test tries to hit the Save button, but I have no idea what's causing it. According to the code, that error message appears if any of the multiple HTTP request promises fails. It looks like we do log the promise error to the console, but that's not possible for us to view from the current Travis CI output or the TestCafe output.

This PR is an attempt at getting TestCafe to print out any console errors that it sees in the browser to the command line console (i.e. Travis CI) so that we have more useful debug information. This isn't the cleanest way of doing things, since it pollutes the console output for even tests that succeed with stuff that looks like this:

 Edit Resource
Captured console.log() messages:
[]
Captured console.warn() messages:
[]
Captured console.error() messages:
["React-Hot-Loader: misconfiguration detected, using production version in non-production environment.","React-Hot-Loader: Hot Module Replacement is not enabled.","InvalidValueError: setIcon: not a string; and not an instance of PinView; and no url property; and no path property","React-Hot-Loader: misconfiguration detected, using production version in non-production environment.","React-Hot-Loader: Hot Module Replacement is not enabled.","React-Hot-Loader: misconfiguration detected, using production version in non-production environment.","React-Hot-Loader: Hot Module Replacement is not enabled.","InvalidValueError: setIcon: not a string; and not an instance of PinView; and no url property; and no path property"]
 ✓ Edit resource name

However, I want to submit this as a PR so that I can at least see why the current "Add resource phone number" test fails and whether this gives us more useful information before I clean it up a bit more.

.afterEach(async t => {
const { log, warn, error } = await t.getBrowserConsoleMessages();

console.log('Captured console.log() messages:');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like (later) we could put these behind a conditional if log, warn, etc. is empty or null (since these appear to be arrays).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, although unfortunately, our app constantly prints stuff to console.log() and console.error() even during normal behavior. For instance, every page load on a development server causes the "React-Hot-Loader: misconfiguration detected, using production version in non-production environment." error message. It looks like the right way to do this kind of thing in TestCafe is with a custom Reporter, so I might port this code into a reporter once I actually get it working.

I'm still trying to debug the issue, since I still apparently have not logged enough information to get the real answer.

@richardxia richardxia force-pushed the add-extra-debug-info-to-testcafe branch from 40cddbb to 0763584 Compare March 22, 2021 02:10
@richardxia richardxia force-pushed the add-extra-debug-info-to-testcafe branch from a468ac9 to 4e388f0 Compare March 22, 2021 02:33
@richardxia
Copy link
Member Author

Sigh. Apparently logging the response body doesn't help at all because we don't actually return any information in the response body when an API call fails.

Trying two more things:

  1. Printing the API URL endpoint as part of the rejected Promise message, so that even if the body is empty, we know which request failed
  2. Printing the logs from the askdarcel-api Docker container if the TestCafe tests fail

@richardxia
Copy link
Member Author

I've stopped being able to reproduce the error, which is annoying for this PR, but it does seem like restarting the other failing PRs is getting them to pass now, so I'm at least happy with that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants