WIP: feat(cypress_test): introduce rule

[Toxicable]

Prototype of rule for running intergration tests with Cypress.

Right now it's only for cypress run (CI version) since there's another issue (see below) with cypress open (dev version)
Also it only works with bazel run since with bazel test due to the issues with bazel wanting to write files (see below)

These two issues are blocking the use of bazel test
cypress-io/cypress#1281

This one is blocking cypress open
cypress-io/cypress#1925

[Globegitter]

@Toxicable this seems quite exciting also for us, is there a reason this is in rules_typescript rather than rules_nodejs? As in this could also just be used with normal js, right?

[Toxicable]

@Globegitter I wanted to ensure this worked with TS, but you're right it should be in rules_nodejs.
Since we're blocked by Cypress on a few issues for now I just wanted to show it somewhat working.

[Toxicable]

So I tried doing a bit more work on this but ran into a few road blocks.

I don't know how to write a file (cypress.json) as part of of the test setup, right now you can see that im using fs.writeFile which will throw if used with bazel test.
I was able to add the file as part of the manifest

  all_data += [
    Label("//tools/cypress-rule:cypress_runner.js"),
    Label("//tools/cypress-rule:cypress.json"),
  ]

But the path that bazel provides me with is back to the source file, but I need the cypress.json to be copied into the directory with the test files aka: Execute root, I don't know how to get this file there.
Some advise on how to get around this would be appciated.

To work around the Cypress FS issues I was planning on using a fork in the meanwhile

[Globegitter]

@Toxicable if you have a look here: https://docs.bazel.build/versions/master/test-encyclopedia.html#initial-conditions there is apparently an env variable TEST_UNDECLARED_OUTPUTS_DIR that gives you a writable location.

I know other users of bazel had a similiar issue with jest snapshot tests and there is some discussion around that here: jestjs/jest#6143

Otherwise maybe a custom rule could be written that tells bazel to create this file and then possibly you could just write into it. But that would be quite a bit more effort.

It also seems that rules_webtesting is writing files here: https://github.com/bazelbuild/rules_webtesting/blob/09aba6c9b969607c39f10826ed5c50c66364a473/go/screenshotter/screenshotter_test.go but it just seems to write it to the OS tmp dir or a location provided by the env var TEST_TMPDIR.

[Toxicable]

@Globegitter Thanks for the resources!
So looks like TEST_UNDECLARED_OUTPUTS_DIR is a directory under the exe root unfortunately, meaning I cant put the cypress.json there either.

I think a custom rule is going to have to be the way to proceed here

[Globegitter]

@Toxicable I see you are making some progress with the native rule. Is it working now? OR at least looking more promising?

[Toxicable]

@Globegitter I made some progress; this is my latest: https://github.com/bazelbuild/rules_typescript/blob/e664c44ae372e0efd8b758b930364ff318081f49/internal/cypress_test/cypress_test.bzl
But im hitting two issues at the moment

• The cypress_bin runfiles are not being added to the runfiles for my rule, therefore the start script for cypress_bin is there but node isn't
• only the .d.ts files are there, not the .js (edited)

Unfortunately I cant use fine grained deps to do this just yet since our repo isn't upgraded due to some issues with debugging nodejs code, however it should still be doable without it.

[Globegitter]

@Toxicable yeah runfiles can sometimes be a bit tricky, I commented on an issue related to that once (can't find it anymore) but the api is a bit confusing and redundant. I will try and make some time to test this out myself in the coming days. Have you tried adding it to transitive_files maybe?

[Toxicable]

@Globegitter No luck on that one just yet.
I found that if I used ctx.actions.run instead of DefaultInfo I can execute the cypress binary without issues, but I don't see how to make that work with bazel test properly since I have no outputs, I havn't found any examples doing this either.
It would appear the only way to do tests is with a script + DefaultInfo

For the first issue:
I have both these set:

          collect_data = True,
          collect_default = True,

which do: https://docs.bazel.build/versions/master/skylark/lib/ctx.html#runfiles

Whether to collect the default runfiles from the dependencies in srcs, data and deps attributes.

But the cypress executable is not in srcs, data or deps, nor do I see how I'm meant to add cypress to those attriibutes or how it's done in the karma rule im looking at.

For the second:
I think this is whats meant to collect up the .js files

  files = depset(ctx.files.srcs)
  for d in ctx.attr.deps:
    if hasattr(d, "node_sources"):
      files += d.node_sources
    elif hasattr(d, "files"):
      files += d.files

which I can see when iterating it
d in my case is a <target //tools/cypress-rule/test:test_lib, keys:[OutputGroupInfo]>
that only has d.files which is: depset([<generated file tools/cypress-rule/test/index.spec.d.ts>])
and that index.spec.d.ts is in the right place, but I don't know where it's js files are at

I can see here: https://github.com/bazelbuild/rules_typescript/blob/master/internal/karma/ts_web_test.bzl#L82
That this expand_path_into_runfiles function is used which may resolve the .d.ts into .js (?), but user_entries is never added to the runfiles so I don't think it grabs those files.

[alexeagle]

@Toxicable I'm going to close this PR now that rules_nodejs is a monorepo with packages/ subdirs. Want to introduce the cypress subdirectory there?

[Toxicable]

I've opened a ticket in the rules_nodejs repo: bazelbuild/rules_nodejs#607
To track the issues with implementing this rule