Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Playwright integration #676

Merged
merged 8 commits into from
May 11, 2023
Merged

Improve Playwright integration #676

merged 8 commits into from
May 11, 2023

Conversation

baev
Copy link
Member

@baev baev commented May 10, 2023

  1. add message & trace to the steps with errors
  2. add repetition parameter to test results
  3. move history id calculation to the end of the test and include all
    the parameters (except included) to it.
  4. rework testing. Use fork instead of spawn and message results to the
    parent (instead of using stdio)
  5. configure allure reporter for the tests
  6. add additional metadata (steps, attachments) to the test results

Context

Checklist

Sorry, something went wrong.

@baev baev added the type:improvement Improvement or request label May 10, 2023
@github-actions github-actions bot added theme:api Javascript API related issue theme:playwright labels May 10, 2023
@epszaw epszaw self-requested a review May 10, 2023 13:40
@baev baev added type:improvement Improvement or request and removed type:improvement Improvement or request labels May 10, 2023
Co-authored-by: Vladimir Semyonov <20096510+vovsemenv@users.noreply.github.com>
Copy link
Member

@epszaw epszaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Tested locally. Works as before. @vovsemenv could you please check the PR's revision with your playwright examples?
Left some comments. Good to fix/discuss, but not necessary.
Great job! 👍

}

writeCategoriesDefinitions(categories: Category[]): void {
writeJson("categories.json", "misc", categories);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this ok, that sort of the writer methods send data through ipc and another ones interacts with fs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep

allureTest.addParameter("Project", project.name);
}
if (project.repeatEach > 1) {
allureTest.addParameter("Repetition", `${test.repeatEachIndex + 1}`);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it reasonable to create enum with parameters types (like Labels or Statuses)?

@@ -375,4 +375,16 @@ const appendStep = (parent: ExecutableItemWrapper, step: TestStep) => {
}
};

const getStatusDetails = (error: TestError): StatusDetails => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, we can move the utilty to commons package. Looks pretty useful and we can use it in pair with the utility, which returns status by given error

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TestError is Playwright-specific. But anyway, the rework is planned to support multiple errors

expect(results.tests).toEqual([
expect.objectContaining({
fullName: "a.test.ts#should be skipped 1",
status: "skipped",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use Status enum instead of string here and next

});

test("should report stderr", async ({ runInlineTest }) => {
test.skip(true, "stderr is not reported by playwright for some reason");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's wrong with the test? Should we keep it skipped?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know. So I decided to simply ignore it at the moment and deal with that later. This PR is only the first one of many

Copy link
Contributor

@vovsemenv vovsemenv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks fine, works on my example pw tests
image

Copy link
Contributor

@vovsemenv vovsemenv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks fine, works on my example pw tests
image

@baev baev merged commit 758af40 into master May 11, 2023
@baev baev deleted the rft-pw branch May 11, 2023 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme:api Javascript API related issue theme:playwright type:improvement Improvement or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants