Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[trainer] Return Test Plan Configuration in JUnit output #20016

Conversation

sebskuse
Copy link
Contributor

@sebskuse sebskuse commented Feb 25, 2022

Checklist

  • I've run bundle exec rspec from the root directory to see all new and existing tests pass
  • I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
  • I've read the Contribution Guidelines
  • I've updated the documentation if necessary.

Motivation and Context

We've started looking at using Test Plan configurations so that we can run our whole UI test suite against different dynamic font sizes. However, there is an issue when using fastlane + trainer to generate JUnit: There's no way to determine which Test Plan configuration the failure originated from - you just get duplicated <testsuite>s. We'd like to be able to raise the configuration that failed, and the information simply isn't there right now. For reference, this is how it appears within Xcode when you open the xcresult:

Screenshot 2022-02-25 at 16 22 56

And how it appears in the JUnit file after this change:

Screenshot 2022-02-25 at 17 06 37

Description

The Test Plan configuration name is already available to us (via ActionTestableSummary), it's just not being used anywhere yet. This PR makes a couple of changes:

  • Retrieve the name from the ActionTestableSummary, and set it on the returned test results row as configuration_name.
  • Looking at the JUnit spec, there's obviously nothing particularly specific to this case, so I settled on using a property on the testsuite, as defined in the spec called Configuration.

Testing Steps

The new field is now available on the xcresult based tests in test_parser_spec, so I have updated those, and I've added a new test within junit_generator_spec that verifies the output (also some extra coverage as currently there are no tests for xcresult -> junit).

@google-cla
Copy link

google-cla bot commented Feb 25, 2022

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

For more information, open the CLA check for this pull request.

Copy link
Member

@joshdholtz joshdholtz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thank you so much for the contribution 🙌

@joshdholtz joshdholtz merged commit a74e0eb into fastlane:master Feb 26, 2022
@fastlane-bot
Copy link

Hey @sebskuse 👋

Thank you for your contribution to fastlane and congrats on getting this pull request merged 🎉
The code change now lives in the master branch, however it wasn't released to RubyGems yet.
We usually ship about once a week, and your PR will be included in the next one.

Please let us know if this change requires an immediate release by adding a comment here 👍
We'll notify you once we shipped a new release with your changes 🚀

@fastlane-bot
Copy link

Congratulations! 🎉 This was released as part of fastlane 2.205.0 🚀

@fastlane fastlane locked and limited conversation to collaborators May 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants