Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JUnitXMLReportPlugin in JUnitXmlTestsListener.scala is not adding skipped attribute to the testsuite root element #2198

Closed
ghost opened this issue Sep 4, 2015 · 4 comments

Comments

@ghost
Copy link

ghost commented Sep 4, 2015

Hi,

JUnitXML document in testsuite root element could contains skipped="numberofskippedtests" so other plugins can read this info (like jenkins) and get how many skipped tests are.

Looking into the (http://www.scala-sbt.org/release/sxr/sbt/JUnitXmlTestsListener.scala.html) this attribute is not being added
val result = <testsuite hostname={hostname} name={name}
tests={tests + ""} errors={errors + ""} failures={failures + ""}
time={(duration/1000.0).toString} >

So, could we add this attribute?

@eed3si9n
Copy link
Member

eed3si9n commented Sep 4, 2015

Sure. Send us a pull request.

@ashleymercer
Copy link
Contributor

ashleymercer commented Nov 29, 2016

I'd like to have a go at picking this up since it's currently causing me pain. A couple of questions:

  • should I be working against 0.13 or the 1.0 branch?
  • the approach taken by Ant is documented on this bug report - I need to read more background but it seems there's some incompatibility between XML reports in Junit 3 and 4. Will report back here once I've worked out what's going on, but in the meantime can we assume Junit 4 compatiblity in sbt (and, presumably, downstream tools?)
  • an alternative strategy would be to delegate to the specific test library to handle this. Specs2 and scalatest already both have support for outputting Junit XML so it would remove some duplication - although I might need some help getting this to work correctly, and we'd need to decide what to do if a different (non-specs/non-scalatest) framework is being used

@eed3si9n
Copy link
Member

Generally our preference for new feature is sbt 1.0 especially if it breaks binary compatibility, but give that it's not out yet, if you want to use the feature sooner you'd have to send us a PR first to 0.13, and then forward port it to 1.0.

ashleymercer added a commit to ashleymercer/sbt that referenced this issue Nov 29, 2016
…g tests

Fixes sbt#2198 Ref #PULL

Generated Junit-style XML reports now include a count of ignored,
skipped and pending tests; and individual tests are correctly flagged
with the <skipped/> element.
@ashleymercer
Copy link
Contributor

Pull request against 0.13 above, the change is very small so it should be easy for me to forward-port to 1.0

  • junit reports don't distinguish between ignored, skipped and pending tests, so we treat all three as "skipped" in the XML report
  • I've opted for the minimal change for now - modifying the existing listener, rather than trying to forward to the correct test library if it's present - to avoid breaking anything

ashleymercer added a commit to ashleymercer/sbt that referenced this issue Dec 1, 2016
…g tests

Fixes sbt#2198 Ref sbt#2854

Generated Junit-style XML reports now include a count of ignored,
skipped and pending tests; and individual tests are correctly flagged
with the <skipped/> element.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants