Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: create config package for config validation #18589

Merged
merged 75 commits into from
Nov 9, 2021
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
5443611
(feat): create config package for config validation in server & driver
emilyrohrbough Oct 20, 2021
045fde3
cleanup
emilyrohrbough Oct 20, 2021
14632cc
.
emilyrohrbough Oct 20, 2021
b92b3b1
add tests to config
emilyrohrbough Oct 21, 2021
dcfaa5b
clean up
emilyrohrbough Oct 21, 2021
a8e405e
missed update
emilyrohrbough Oct 21, 2021
c7440d1
Merge branch 'develop' into add-config-validation
emilyrohrbough Oct 21, 2021
acd4b77
fix obj overrding issue
emilyrohrbough Oct 21, 2021
8fb03ab
tests
emilyrohrbough Oct 21, 2021
f149559
dependencies
emilyrohrbough Oct 21, 2021
e723b03
update missed import
emilyrohrbough Oct 21, 2021
d843170
fixes one of the two test issues
emilyrohrbough Oct 22, 2021
a38fb6a
Merge branch 'develop' into add-config-validation
emilyrohrbough Oct 22, 2021
3fdd15c
dang. thought this had already been changed.
emilyrohrbough Oct 22, 2021
4bfcfae
clean up
emilyrohrbough Oct 22, 2021
5e52676
clean pu
emilyrohrbough Oct 22, 2021
d0a9967
Merge branch 'develop' into add-config-validation
emilyrohrbough Oct 22, 2021
82b1119
...
emilyrohrbough Oct 25, 2021
ba6d586
Merge branch 'develop' into add-config-validation
emilyrohrbough Oct 25, 2021
00fed44
Merge branch 'develop' into add-config-validation
emilyrohrbough Oct 25, 2021
91dfd3f
Merge branch 'develop' into add-config-validation
emilyrohrbough Oct 25, 2021
6826069
improve import variable name
emilyrohrbough Oct 26, 2021
f449b9d
clean up pkg.json
emilyrohrbough Oct 26, 2021
c64a652
add the codeframe / UI error logic for test overrides
emilyrohrbough Oct 28, 2021
eae59f8
reset this bit.
emilyrohrbough Oct 28, 2021
63d18cf
fix codeframe for mocha suites vs tests & nested blocks
emilyrohrbough Oct 29, 2021
402a393
inlint test config overrides with correct stack trace
emilyrohrbough Nov 1, 2021
f279e36
typo
emilyrohrbough Nov 1, 2021
904c22b
Merge branch 'develop' into add-config-validation
emilyrohrbough Nov 1, 2021
1af43ac
fix global config state
emilyrohrbough Nov 1, 2021
b4ce459
Merge branch 'develop' into add-config-validation
emilyrohrbough Nov 1, 2021
c3ec924
Merge branch 'develop' into add-config-validation
emilyrohrbough Nov 1, 2021
762470c
reset config as expected
emilyrohrbough Nov 1, 2021
3486ab1
fix compile/types test
emilyrohrbough Nov 2, 2021
eafacca
re-add common-tags used in tests
emilyrohrbough Nov 2, 2021
a12e5d2
dont lint compiled code.
emilyrohrbough Nov 2, 2021
ffc43c5
.
emilyrohrbough Nov 2, 2021
9e37c63
dep not dev dep
emilyrohrbough Nov 2, 2021
cd13f0e
Merge branch 'develop' into add-config-validation
emilyrohrbough Nov 2, 2021
2b517be
have not been able to reproduce test failures locally
emilyrohrbough Nov 2, 2021
953dacd
fix the typsescript compiling
emilyrohrbough Nov 3, 2021
e05ceb3
clean up tsc
emilyrohrbough Nov 3, 2021
d4d4354
Merge branch 'develop' into add-config-validation
emilyrohrbough Nov 3, 2021
39dadca
fix slow test threshold.
emilyrohrbough Nov 3, 2021
766502c
Merge branch 'add-config-validation' of https://github.com/cypress-io…
emilyrohrbough Nov 3, 2021
4a34493
fix
emilyrohrbough Nov 3, 2021
f0676e2
clean up
emilyrohrbough Nov 3, 2021
2e27112
aAH. fix broken system tests
emilyrohrbough Nov 4, 2021
ab89639
fix types & update snapshots
emilyrohrbough Nov 4, 2021
530e19d
Update navigation_spec.js
emilyrohrbough Nov 4, 2021
b351eda
PR comments
emilyrohrbough Nov 4, 2021
35fc415
intented for this to release with 9.0.0
emilyrohrbough Nov 4, 2021
8bdfd57
remove @packages/config build script & move all from src to lib
emilyrohrbough Nov 5, 2021
619ee85
update gitignore for compile .js files in lib
emilyrohrbough Nov 5, 2021
a28cd4f
Merge branch 'develop' into add-config-validation
emilyrohrbough Nov 5, 2021
c0a47ca
handle before/beforeEach hooks
emilyrohrbough Nov 8, 2021
eaead0d
fix event emitting & retries
emilyrohrbough Nov 8, 2021
4905d32
Merge branch 'add-config-validation' of https://github.com/cypress-io…
emilyrohrbough Nov 8, 2021
5cb2f5e
typo
emilyrohrbough Nov 8, 2021
f582b3c
fix import
emilyrohrbough Nov 8, 2021
f0d538a
revert local change that was checked in
emilyrohrbough Nov 8, 2021
b804ea3
revert change that would require Product Review & alters the behavior…
emilyrohrbough Nov 8, 2021
1052df3
fix driver errors that highlighted the beforeeach hook issue
emilyrohrbough Nov 8, 2021
537e583
update snapshots
emilyrohrbough Nov 8, 2021
84f53d1
fix runner:test:before:run:asyn event exec
emilyrohrbough Nov 8, 2021
d97e70f
Merge branch '9.0-release' into add-config-validation
emilyrohrbough Nov 8, 2021
2e62654
missed merge conflict
emilyrohrbough Nov 8, 2021
5bad91c
fix unit tests
emilyrohrbough Nov 8, 2021
b6b3e7c
system tests
emilyrohrbough Nov 8, 2021
db58a8c
.
emilyrohrbough Nov 8, 2021
5ddf2dd
update snapshot
emilyrohrbough Nov 8, 2021
f554233
snapshot
emilyrohrbough Nov 8, 2021
e4c351d
snapshots
emilyrohrbough Nov 8, 2021
7341dec
ahh firefox throws window.Error differently
emilyrohrbough Nov 9, 2021
1414fd7
missed from merging in matts node changes
emilyrohrbough Nov 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/config/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"private": true,
"main": "lib/index.js",
"scripts": {
"build-prod": "sc --project .",
"build-prod": "tsc --project .",
"clean": "rm lib/options.js",
"test": "yarn test-unit",
"test-debug": "yarn test-unit --inspect-brk=5566",
Expand Down
6 changes: 1 addition & 5 deletions packages/config/tsconfig.json
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
{
"extends": "../ts/tsconfig.json",
"compilerOptions": {
"allowJs": true,
"outDir": "./lib",
},
"include": [
"src/*",
"lib/*",
],
}
Original file line number Diff line number Diff line change
Expand Up @@ -693,7 +693,7 @@ describe('src/cy/commands/navigation', () => {
})

it('does not support file:// protocol', {
baseUrl: '',
baseUrl: null,
}, (done) => {
cy.on('fail', (err) => {
expect(err.message).to.contain('`cy.visit()` failed because the \'file://...\' protocol is not supported by Cypress.')
Expand Down Expand Up @@ -1438,7 +1438,7 @@ describe('src/cy/commands/navigation', () => {
cy.visit('http://google.com:3500/fixtures/generic.html')
})

it('throws attemping to visit 2 unique ip addresses', function (done) {
it('throws attempting to visit 2 unique ip addresses', function (done) {
const $autIframe = cy.state('$autIframe')

const load = () => {
Expand Down
6 changes: 3 additions & 3 deletions packages/driver/cypress/integration/commands/request_spec.js
Original file line number Diff line number Diff line change
Expand Up @@ -820,7 +820,7 @@ describe('src/cy/commands/request', () => {
})

it('throws when url is not FQDN', {
baseUrl: '',
baseUrl: null,
}, function (done) {
cy.stub(cy, 'getRemoteLocation').withArgs('origin').returns('')

Expand All @@ -840,7 +840,7 @@ describe('src/cy/commands/request', () => {
})

it('throws when url is not FQDN, notes that configFile is disabled', {
baseUrl: '',
baseUrl: null,
configFile: false,
}, function (done) {
cy.stub(cy, 'getRemoteLocation').withArgs('origin').returns('')
Expand All @@ -860,7 +860,7 @@ describe('src/cy/commands/request', () => {
})

it('throws when url is not FQDN, notes that configFile is non-default', {
baseUrl: '',
baseUrl: null,
configFile: 'foo.json',
}, function (done) {
cy.stub(cy, 'getRemoteLocation').withArgs('origin').returns('')
Expand Down
2 changes: 1 addition & 1 deletion packages/driver/src/cy/commands/cookies.ts
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ export default function (Commands, Cypress, cy, state, config) {
// stuff, or handling this in the runner itself?
// Cypress sessions will clear cookies on its own before each test
Cypress.on('test:before:run:async', () => {
if (!Cypress.config.experimentalSessionSupport) {
if (!Cypress.config('experimentalSessionSupport')) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was this causing a bug? since before this condition would always pass

return getAndClear()
}
})
Expand Down
5 changes: 3 additions & 2 deletions packages/driver/src/cy/testConfigOverrides.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,6 @@ function mutateConfiguration (testConfig: ResolvedTestConfigOverride, config, en
// note: must return as an object to meet the dashboard recording API
export function getResolvedTestConfigOverride (test): ResolvedTestConfigOverride {
let curParent = test.parent

const testConfigList = [{
overrides: test._testConfig,
invocationDetails: test.invocationDetails,
Expand Down Expand Up @@ -144,7 +143,9 @@ class TestConfigOverride {

const resolvedTestConfig = test._testConfig || {}

this.restoreTestConfigFn = mutateConfiguration(resolvedTestConfig, config, env)
if (Object.keys(resolvedTestConfig.unverifiedTestConfig).length > 0) {
this.restoreTestConfigFn = mutateConfiguration(resolvedTestConfig, config, env)
}
}
}

Expand Down
5 changes: 0 additions & 5 deletions packages/driver/src/cypress.ts
Original file line number Diff line number Diff line change
Expand Up @@ -398,16 +398,12 @@ class $Cypress {
return this.runner.onRunnableRun(...args)

case 'runner:test:before:run':
// get back to a clean slate
this.cy.reset(...args)

if (this.config('isTextTerminal')) {
// needed for handling test retries
this.emit('mocha', 'test:before:run', args[0])
}

this.emit('test:before:run', ...args)

break

case 'runner:test:before:run:async':
Expand All @@ -431,7 +427,6 @@ class $Cypress {
}

break

case 'cy:before:all:screenshots':
return this.emit('before:all:screenshots', ...args)

Expand Down
42 changes: 20 additions & 22 deletions packages/driver/src/cypress/cy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -706,36 +706,34 @@ export default {
},

// reset is called before each test
reset (_attrs, test) {
const s = state()

const backup = {
window: s.window,
document: s.document,
$autIframe: s.$autIframe,
specWindow: s.specWindow,
activeSessions: s.activeSessions,
}
reset (test) {
try {
const s = state()

const backup = {
window: s.window,
document: s.document,
$autIframe: s.$autIframe,
specWindow: s.specWindow,
activeSessions: s.activeSessions,
}

// reset state back to empty object
state.reset()
// reset state back to empty object
state.reset()

// and then restore these backed up props
state(backup)
// and then restore these backed up props
state(backup)

queue.reset()
queue.clear()
timers.reset()
queue.reset()
queue.clear()
timers.reset()

try {
testConfigOverrides.restoreAndSetTestConfigOverrides(test, Cypress.config, Cypress.env)
// reset mocha slow test threshold
test.slow(Cypress.config('slowTestThreshold'))

cy.removeAllListeners()
} catch (err) {
fail(err)
}

return cy.removeAllListeners()
},

addCommandSync (name, fn) {
Expand Down
69 changes: 33 additions & 36 deletions packages/driver/src/cypress/runner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ const betweenQuotesRe = /\"(.+?)\"/

const HOOKS = 'beforeAll beforeEach afterEach afterAll'.split(' ')
const TEST_BEFORE_RUN_ASYNC_EVENT = 'runner:test:before:run:async'
// event fired before hooks and test execution
const TEST_BEFORE_RUN_EVENT = 'runner:test:before:run'
const TEST_AFTER_RUN_EVENT = 'runner:test:after:run'

Expand Down Expand Up @@ -915,9 +916,9 @@ const _runnerListeners = (_runner, Cypress, _emissions, getTestById, getTest, se
// })

/**
* Mocha retry event is only fired in Mocha version 6+
* https://github.com/mochajs/mocha/commit/2a76dd7589e4a1ed14dd2a33ab89f182e4c4a050
*/
* Mocha retry event is only fired in Mocha version 6+
* https://github.com/mochajs/mocha/commit/2a76dd7589e4a1ed14dd2a33ab89f182e4c4a050
*/
_runner.on('retry', (test, err) => {
test.err = $errUtils.wrapErr(err)
Cypress.action('runner:retry', wrap(test), test.err)
Expand Down Expand Up @@ -1134,7 +1135,7 @@ export default {
}

const onRunnable = (r) => {
// set defualt retries at onRunnable time instead of onRunnableRun
// set default retries at onRunnable time instead of onRunnableRun
return _runnables.push(r)
}

Expand Down Expand Up @@ -1235,6 +1236,7 @@ export default {
// this prevents afterEach hooks that exist at a deeper level than the failing one from running
// we will always skip remaining beforeEach hooks since they will always be same level or deeper
test._skipHooksWithLevelGreaterThan = runnable.titlePath().length

setHookFailureProps(test, runnable, err)
test.fn = function () {
throw err
Expand All @@ -1244,7 +1246,7 @@ export default {
}

if (willRetry && isAfterEachHook) {
// if we've already failed this attempt from an afterEach hook then we've already enqueud another attempt
// if we've already failed this attempt from an afterEach hook then we've already enqueued another attempt
// so return early
if (test._retriedFromAfterEachHook) {
return noFail()
Expand Down Expand Up @@ -1371,10 +1373,9 @@ export default {
}

// closure for calculating the actual
// runtime of a runnables fn exection duration
// runtime of a runnables fn execution duration
// and also the run of the runnable:after:run:async event
let lifecycleStart
let wallClockStartedAt = null
let wallClockEnd = null
let fnDurationStart = null
let fnDurationEnd = null
Expand All @@ -1384,7 +1385,7 @@ export default {
// when this is a hook, capture the real start
// date so we can calculate our test's duration
// including all of its hooks
wallClockStartedAt = new Date()
const wallClockStartedAt = new Date()

if (!test.wallClockStartedAt) {
// if we don't have lifecycle timings yet
Expand All @@ -1395,27 +1396,13 @@ export default {
test.wallClockStartedAt = wallClockStartedAt
}

const isHook = runnable.type === 'hook'

// if this isn't a hook, then the name is 'test'
const hookName = runnable.type === 'hook' ? getHookName(runnable) : 'test'
const hookName = isHook ? getHookName(runnable) : 'test'

// set hook id to hook id or test id
const hookId = runnable.type === 'hook' ? runnable.hookId : runnable.id

// if we haven't yet fired this event for this test
// that means that we need to reset the previous state
// of cy - since we now have a new 'test' and all of the
// associated _runnables will share this state
if (!fired(TEST_BEFORE_RUN_EVENT, test)) {
fire(TEST_BEFORE_RUN_EVENT, test, Cypress)

// this is the earliest we can set test._retries since test:before:run
// will load in testConfigOverrides (per test configuration)
const retries = Cypress.getTestRetries() ?? -1

test._retries = retries
}

const isHook = runnable.type === 'hook'
const hookId = isHook ? runnable.hookId : runnable.id

const isAfterEachHook = isHook && hookName.match(/after each/)
const isBeforeEachHook = isHook && hookName.match(/before each/)
Expand Down Expand Up @@ -1519,22 +1506,32 @@ export default {
})
}

cy.state('duringUserTestExecution', false)

// our runnable is about to run, so let cy know. this enables
// us to always have a correct runnable set even when we are
// running lifecycle events
// and also get back a function result handler that we use as
// an async seam
cy.setRunnable(runnable, hookId)

// TODO: handle promise timeouts here!
// whenever any runnable is about to run
// we figure out what test its associated to
// if its a hook, and then we fire the
// test:before:run:async action if its not
// been fired before for this test
return testBeforeRunAsync(test, Cypress)
return Promise.try(() => {
if (!fired(TEST_BEFORE_RUN_EVENT, test)) {
cy.reset(test)
test.slow(Cypress.config('slowTestThreshold'))
test._retries = Cypress.getTestRetries() ?? -1
fire(TEST_BEFORE_RUN_EVENT, test, Cypress)
}

cy.state('duringUserTestExecution', false)

// our runnable is about to run, so let cy know. this enables
// us to always have a correct runnable set even when we are
// running lifecycle events
// and also get back a function result handler that we use as
// an async seam
cy.setRunnable(runnable, hookId)
})
.then(() => {
return testBeforeRunAsync(test, Cypress)
})
.catch((err) => {
// TODO: if our async tasks fail
// then allow us to cause the test
Expand Down