Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@wdio/cucumber-framework: Filter specs /w Cucumber Tag Expression before spawning workers #10330

Merged

Conversation

nextlevelbeard
Copy link
Member

@nextlevelbeard nextlevelbeard commented May 7, 2023

fixes #8253

Proposed changes

Makes use of CucumberJS API to read the content of the Feature files used by the Cucumber framework to then filter the specs based on the provided Tag Expression.

We do this before spawning any workers, optimizing heavily the filtering process for less optimal hardware and from even moderate amounts of specs.

Before this change, a worker was spawned for each spec file, the worker would skip the file and then terminate, impacting performance heavily.

The filtering implementation is compatible with latest changes to the Gherkin spec (like the addition of the Rule).

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

Checklist

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • I have added proper type definitions for new commands (if appropriate)

Further comments

Reviewers: @webdriverio/project-committers

@nextlevelbeard
Copy link
Member Author

@christian-bromann Mind having a look at the failing jobs, seems like the new dependencies have trouble being imported with the browser-runner?

Copy link
Member

@christian-bromann christian-bromann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make sure we keep Cucumber related functionality in the Cucumber package.

packages/wdio-config/package.json Outdated Show resolved Hide resolved
packages/wdio-config/src/lib/ConfigParser.ts Outdated Show resolved Hide resolved
packages/wdio-config/src/lib/ConfigParser.ts Outdated Show resolved Hide resolved
@christian-bromann
Copy link
Member

Mind having a look at the failing jobs, seems like the new dependencies have trouble being imported with the browser-runner?

I am looking into fixing the build

Copy link
Member

@christian-bromann christian-bromann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something happened here that reverted some changes from the past, can we revert them?

@nextlevelbeard
Copy link
Member Author

Something happened here that reverted some changes from the past, can we revert them?

Apologies, differences between my origin and upstream, should be fixed

@nextlevelbeard
Copy link
Member Author

nextlevelbeard commented May 8, 2023

I moved the implementation over to @wdio/cucumber-framework.
It has an init() function where we can alter the specs, before workers pick them up.

I noticed if we have 4 files total and when filtering by tag we get 1 file, the reporter output still spits out 4 files.

npx wdio my.conf --cucumberOpts.tagExpression="@myTagThatExistsInOneFileOutOfFour"

Execution of 4 workers started at [redacted]

[0-0] RUNNING in chrome - file:///features/Login.feature
[0-0] PASSED in chrome - file:///features/Login.feature

 "spec" Reporter:
------------------------------------------------------------------
Running: chrome (v113.0.0.0) on darwin
Session ID: etc

» /features/MyFeature.feature
MyFeature
User can do stuff
   ✓ Given I do stuff

1 passing (9.7s)


Spec Files:      1 passed, 4 total (25% completed) in 00:00:14 

This did not happen when filtering was done in @wdio/config, would report only one worker.

Is this intended behavior?

@nextlevelbeard nextlevelbeard changed the title wdio-config: Filter specs /w Cucumber Tag Expression before spawning workers @wdio/cucumber-framework: Filter specs /w Cucumber Tag Expression before spawning workers May 10, 2023
@nextlevelbeard
Copy link
Member Author

This should be ready to go, despite the Linux job failing
Leaving in your hands

Copy link
Member

@christian-bromann christian-bromann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some suggestions, I like it is now all within the cucumber adapter package.

packages/wdio-cucumber-framework/src/index.ts Outdated Show resolved Hide resolved
packages/wdio-cucumber-framework/src/index.ts Outdated Show resolved Hide resolved
Comment on lines 52 to 53
const require = createRequire(import.meta.url)
const EventDataCollector = require('@cucumber/cucumber/lib/formatter/helpers/event_data_collector').default
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems Cucumber now has ESM support so maybe we can update this and now import the EventDataCollector via import statement

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't think this is possible yet. See TypeStrong/ts-node#1585
Since EventDataCollector is still not exported from @cucumber/cucumber I need to

import { EventDataCollector } from '@cucumber/cucumber/lib/formatter/helpers'
// or
import { EventDataCollector } from '@cucumber/cucumber/lib/formatter/helpers/index.js'

But I'm getting

No known conditions for "./lib/formatter/helpers" specifier in "@cucumber/cucumber" package

Added a comment to the file.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cheers! I saw an export for all lib files but then realised that it was only for require imports:

    "./lib/*": {
      "require": "./lib/*.js"
    },

😞

@nextlevelbeard
Copy link
Member Author

While on the subject, WDIO should really have a flag to pass the tags @christian-bromann
Now more than ever.

Try typing

--cucumberOpts.tagExpression="@myTag"

three times fast 😅

--tags "@myTag"

Is where it's at.

@christian-bromann
Copy link
Member

While on the subject, WDIO should really have a flag to pass the tags

Feel free to raise a PR

Copy link
Member

@christian-bromann christian-bromann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks for updating the Cucumber deps 🎉 👍

@christian-bromann christian-bromann added the PR: Bug Fix 🐛 PRs that contain bug fixes label May 10, 2023
@christian-bromann christian-bromann merged commit 048aa07 into webdriverio:main May 10, 2023
6 checks passed
@HannaTarasevich
Copy link
Contributor

HannaTarasevich commented May 11, 2023

Hello @christian-bromann @nextlevelbeard ,
I've just raised a bug that is connected to this merged PR.
Please take a look #10367

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR: Bug Fix 🐛 PRs that contain bug fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[🐛 Bug]: Number of workers created not filtered by cucumber tags
3 participants