Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix test being potentially muddied by local ENV #21586

Merged
merged 1 commit into from Nov 8, 2023

Conversation

AliSoftware
Copy link
Contributor

@AliSoftware AliSoftware commented Oct 19, 2023

Checklist

  • I've run bundle exec rspec from the root directory to see all new and existing tests pass
  • I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
  • I see several green ci/circleci builds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.
  • I've added or updated relevant unit tests.

Motivation and Context

When running bundle exec rspec, if you have an environment variable like SENTRY_AUTH_TOKEN defined in your local environment (e.g. via your ~/.zshrc), its value will be picked up when running rspec, and that will make the expectations of the upload_symbols_to_sentry spec fail.

This is because the spec is testing cases where the user did not provide a token (and expect the action to raise an error in such cases), but when you have the ENV var defined on your Mac, it makes the action being run by the test picking it up as the token so the action doesn't fail and the test doesn't test what's expected of it.

Description

Stub ENV#[] to return nil for all those tests so that it does not risk being polluted by the local environment when bundle exec rspec is being run.

PS: Note that I suspect many other specs are subject to a similar issue of accidentally picking up the local environment. This is just the only one that kept causing me trouble because I happen to have SENTRY_AUTH_TOKEN defined in my ~/.zshrc, but I'm sure if I had other env vars associated with some actions' ConfigItem(env: …) defined in my local env that would make other tests fail for the same reason. That being said, I don't have the bandwidth to investigate all the potential cases where this could happen, so I figured I'd just fix the one that caused me trouble (and let others apply a similar patch in the other specs if they encounter a similar issue on their end on other test cases)

Testing Steps

Run the following:

export SENTRY_AUTH_TOKEN=abc
bundle exec rspec

If you run these lines on master (before this PR) you should end up with the following test failure:

Failed examples:

rspec ./fastlane/spec/actions_specs/upload_symbols_to_sentry_spec.rb:4 # Fastlane Fastlane::FastFile sentry fails with no API key or auth token
rspec ./fastlane/spec/actions_specs/upload_symbols_to_sentry_spec.rb:32 # Fastlane Fastlane::FastFile sentry returns uploaded dSYM path using API key
rspec ./fastlane/spec/actions_specs/upload_symbols_to_sentry_spec.rb:60 # Fastlane Fastlane::FastFile sentry returns uploaded dSYM paths

If you run those same lines on this PR's branch, those tests should now pass.

Copy link
Member

@mollyIV mollyIV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix 💯

I figured I'd just fix the one that caused me trouble (and let others apply a similar patch in the other specs if they encounter a similar issue on their end on other test cases)

Maybe we could create a parent issue that'd list down actions that require ENV fixing in tests? 🤔

@AliSoftware
Copy link
Contributor Author

Maybe we could create a parent issue that'd list down actions that require ENV fixing in tests? 🤔

Good point. I just opened this issue to keep track.

@AliSoftware AliSoftware merged commit 30b67fe into master Nov 8, 2023
7 checks passed
@AliSoftware AliSoftware deleted the fix-test-local-env branch November 8, 2023 15:57
AliSoftware referenced this pull request in wordpress-mobile/release-toolkit Apr 24, 2024
The tests passed on my machine but failed in CI, see
https://buildkite.com/automattic/release-toolkit/builds/1602#018f0986-a43b-49d1-884d-50c62e8af094

It turns out I had a `BUILDKITE_TOKEN` in my environment which I had
forgotten about...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants