Support --tag argument #5164

jennifer-shehane · 2019-09-19T00:20:03Z

close Ability to add tag to run to display in 'Runs' list in Dashboard UI #2561

TODO

Bump json-schema endpoints to 2.2.0 for postRunRequest and postRunResponse
How should we handle cypress run --tag nightly and then running cypress run --tag staging in parallel, where the run will become a single run. I think that it should attribute both staging and nightly to the single run. We would be posting tags: ["nightly"] in one POST and then tags: ["staging"] in the 2nd POST. https://github.com/cypress-io/cypress-services/issues/1812

User facing changelog

Cypress now accepts a --tag argument to be used to distinguish separate runs. Any tags assigned will be shown on the run in the Cypress Dashboard.

Additional details

Accepting tag as singular, but always passing it as an Array to the cypress-api as tags
I tested this through staging with the Dashboard and confirmed it does display the tag in the Dashboard, so it is passed along correctly (tested with 1 and multiple tags)

How has the user experience changed?

Options are alphabetized + Tag option added to `--help`

Error when passing space delimited `--spec`

Error when passing space delimited `--tag`

Tag added to beginning of recording run stdout

Displays as false by default.

1 tag:

2 tags:

Dashboard displays the tags in the Runs list (only implementation right now)

PR Tasks

Have tests been added/updated?
Has a PR for user-facing changes been opened in cypress-documentation?
[NA] Have API changes been updated in the type definitions?
[NA] Have new configuration options been added to the cypress.schema.json?

cypress · 2019-09-19T00:36:02Z

Test summary

3551 • 0 • 45 • 0

Run details


Project	cypress
Status	Passed
Commit	`15a8aa9`
Started	Dec 4, 2019 7:33 PM
Ended	Dec 4, 2019 7:36 PM
Duration	03:40 💡
OS	Linux Debian - 9.8
Browser	Multiple

View run in Cypress Dashboard ➡️

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

…s description

…ag and --spec flags dynamically

- show err if passed without record flag - sanitize args into comma separated string - display tag in record errors

# Conflicts: # cli/__snapshots__/cli_spec.js # cli/lib/cli.js

# Conflicts: # cli/__snapshots__/cli_spec.js # cli/lib/cli.js # cli/lib/exec/run.js # packages/server/lib/modes/run.js # packages/server/lib/util/args.js

# Conflicts: # packages/server/lib/modes/run.js

jennifer-shehane · 2019-11-13T17:44:28Z

Need to make an update based on this comment https://github.com/cypress-io/cypress-services/pull/1566#discussion_r339094829 to change sending tag to tags to the cypress-api

We are calling it tag to the user in the Test Runner (as this matches the same convention as the spec flag. But it appears when we communicate to the API later, it is changed to specs, so I am fine with us using tags here since I believe we've already normalized it into an array.

/pull/5164/files#diff-f558bdd26d5a0b4043bae0b9dc251e9fR291

- this will require a bump to json-schemas repo

# Conflicts: # cli/lib/cli.js # cli/test/lib/cli_spec.js

jennifer-shehane · 2019-11-26T08:46:31Z

I tested this in staging and it is working, so the failed tests are likely just some weird snapshot diffs that need reconciling.

# Conflicts: # packages/server/lib/modes/run.js

bahmutov · 2019-12-04T18:31:46Z

packages/server/test/unit/args_spec.coffee

+  context "--tag", ->
+    it "converts to array", ->
+      options = @setup("--run-project", "foo", "--tag", "nightly,production,build")
+      expect(options.tag[0]).to.eq("nightly")


could we use single assertion here for complete message on error

expect(options.tag).to.deep.equal(['nightly', 'production', 'build'])

packages/server/test/unit/args_spec.coffee

bahmutov

Reviewed, ran tests, added more tests, tested against staging - all seems 👍

* Alphabetize cli options for my own sanity * begin added tag flag * Fix some minor grammar in cli help output + be more specific for specs description * update snapshot based on alphabetization change * update snapshot to include --tag in help output * update logic for pulling out space delimited args to look through --tag and --spec flags dynamically * Support and pass along tag flag to run and record - show err if passed without record flag - sanitize args into comma separated string - display tag in record errors * fix some tests/snapshots where 'tag' was missing * Actually try passing in tag through tests to ensure it prints. * Merge branch 'develop' into issue-2561-tags # Conflicts: # cli/__snapshots__/cli_spec.js # cli/lib/cli.js # cli/lib/exec/run.js # packages/server/lib/modes/run.js # packages/server/lib/util/args.js * Send 'tags' as an array to backend API * Update json-schemas to query against 2.2.0 of postRun - this will require a bump to json-schemas repo * update test to reflect tags array * update snapshot to display nightly tag * rearrange args to alphabetical order in specs * Add tags to runResponses / remove tag from incorrect instance post * Fix failing specs / snapshots * Update error messages + snapshots * Fix snapshot that no longer displays tag arg * fix args unit test * remove extra slash * add a few more cli tests * another test just in case * a quick unit test for displayFlags utility Co-authored-by: Gleb Bahmutov <gleb.bahmutov@gmail.com>

jennifer-shehane added 2 commits September 18, 2019 20:18

Alphabetize cli options for my own sanity

4f9736a

begin added tag flag

b53e203

jennifer-shehane added 7 commits September 19, 2019 14:15

Fix some minor grammar in cli help output + be more specific for spec…

c0bd5c3

…s description

update snapshot based on alphabetization change

dd0d6da

update snapshot to include --tag in help output

fefe604

update logic for pulling out space delimited args to look through --t…

aef8f03

…ag and --spec flags dynamically

Support and pass along tag flag to run and record

0b0be25

- show err if passed without record flag - sanitize args into comma separated string - display tag in record errors

fix some tests/snapshots where 'tag' was missing

397472a

Merge branch 'develop' into issue-2561-tags

27ebb5e

jennifer-shehane requested review from brian-mann and a team September 19, 2019 21:03

jennifer-shehane changed the title ~~[WIP] Support --tag argument~~ Support --tag argument Sep 19, 2019

jennifer-shehane added 4 commits September 24, 2019 17:37

Actually try passing in tag through tests to ensure it prints.

665b543

Merge branch 'develop' into issue-2561-tags

ec1df15

# Conflicts: # cli/__snapshots__/cli_spec.js # cli/lib/cli.js

Merge branch 'develop' into issue-2561-tags

c520ee4

# Conflicts: # cli/__snapshots__/cli_spec.js # cli/lib/cli.js # cli/lib/exec/run.js # packages/server/lib/modes/run.js # packages/server/lib/util/args.js

Fix merge conflicts

c163612

jennifer-shehane changed the title ~~Support --tag argument~~ [WIP] Support --tag argument Oct 25, 2019

Merge branch 'develop' into issue-2561-tags

0b67a52

# Conflicts: # packages/server/lib/modes/run.js

jennifer-shehane added 6 commits November 13, 2019 13:27

Send 'tags' as an array to backend API

df65175

Update json-schemas to query against 2.2.0 of postRun

e9bad26

- this will require a bump to json-schemas repo

update test to reflect tags array

7fb4b03

Merge branch 'develop' into issue-2561-tags

7fdd304

# Conflicts: # cli/lib/cli.js # cli/test/lib/cli_spec.js

update snapshot to display nightly tag

71c908a

rearrange args to alphabetical order in specs

b67dba5

jennifer-shehane added 3 commits November 26, 2019 16:11

Merge branch 'develop' into issue-2561-tags

87b8c73

# Conflicts: # packages/server/lib/modes/run.js

Merge branch 'develop' into issue-2561-tags

7942365

Add tags to runResponses / remove tag from incorrect instance post

3b0e411

Fix failing specs / snapshots

546abe3

jennifer-shehane changed the title ~~[WIP] Support --tag argument~~ Support --tag argument Nov 27, 2019

jennifer-shehane and others added 3 commits November 28, 2019 15:27

Update error messages + snapshots

3b7f1a5

Fix snapshot that no longer displays tag arg

4492f8b

Merge branch 'develop' into issue-2561-tags

71d0c24

bahmutov reviewed Dec 4, 2019

View reviewed changes

packages/server/test/unit/args_spec.coffee Outdated Show resolved Hide resolved

bahmutov added 5 commits December 4, 2019 13:33

fix args unit test

ff2e7f9

remove extra slash

723de4c

add a few more cli tests

d58dde0

another test just in case

aaf8028

a quick unit test for displayFlags utility

15a8aa9

bahmutov approved these changes Dec 4, 2019

View reviewed changes

jennifer-shehane merged commit 692a3c0 into develop Dec 5, 2019

cypress-bot bot mentioned this pull request Dec 5, 2019

Ability to add tag to run to display in 'Runs' list in Dashboard UI #2561

Closed

jennifer-shehane mentioned this pull request Jan 14, 2020

3.8.0 run report has puzzling 'undefined' lines #5995

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support --tag argument #5164

Support --tag argument #5164

jennifer-shehane commented Sep 19, 2019 •

edited

cypress bot commented Sep 19, 2019 •

edited

jennifer-shehane commented Nov 13, 2019 •

edited

jennifer-shehane commented Nov 26, 2019

bahmutov Dec 4, 2019

brian-mann Dec 4, 2019

bahmutov left a comment

Support --tag argument #5164

Support --tag argument #5164

Conversation

jennifer-shehane commented Sep 19, 2019 • edited

TODO

User facing changelog

Additional details

How has the user experience changed?

Options are alphabetized + Tag option added to --help

Error when passing space delimited --spec

Error when passing space delimited --tag

Tag added to beginning of recording run stdout

Dashboard displays the tags in the Runs list (only implementation right now)

PR Tasks

cypress bot commented Sep 19, 2019 • edited

Test summary

Run details

jennifer-shehane commented Nov 13, 2019 • edited

jennifer-shehane commented Nov 26, 2019

bahmutov Dec 4, 2019

Choose a reason for hiding this comment

brian-mann Dec 4, 2019

Choose a reason for hiding this comment

bahmutov left a comment

Choose a reason for hiding this comment

jennifer-shehane commented Sep 19, 2019 •

edited

Options are alphabetized + Tag option added to `--help`

Error when passing space delimited `--spec`

Error when passing space delimited `--tag`

cypress bot commented Sep 19, 2019 •

edited

jennifer-shehane commented Nov 13, 2019 •

edited