README

TestNG Plugin for Jenkins
=============

https://wiki.jenkins-ci.org/display/JENKINS/testng-plugin

This plugin allows you to publish TestNG results. TestNG result xml file
contains more information than the JUnit report xml file. This plugin
exposes those extra information in graph and table reports.

This plugin makes it possible to import TestNG XML reports from each
build into Hudson. The data is parsed using the output generated using
org.testng.reporters.XMLReporter. The results are displayed with a trend
graph and all details about which tests that failed are also presented.


Release Notes
-------

### 1.15
* Fixed: JENKINS-32746 - Change loop determination condition in order to check all loaded builds (Julien Herr)
* Fixed: JENKINS-26971 - Looking at failed TestNG result causes an exception (Vladimir Sitnikov)
* PR-35 - Show "failed configurations" earlier than "failed tests" (Vladimir Sitnikov)
* Added: JENKINS-51341 - Display suite name at main page and at method result page (Vladimir Sitnikov)

### 1.14
* Added: Update to new parent POM (jglick)
* Added: JENKINS-27121 - Pipeline compatibility (jglick)
* Fixed: General code quality fixes (faisal-hameed)
* Added: Added some basic form validation

### 1.13
* Fixed: JENKINS-34822 - TestNG plugin has incorrect default threshold values

### 1.12
* Fixed: JENKINS-34711 - NullPointerException after Update to 1.11 of plugin

### 1.11
* Added: Add ability to configure failed/skipped test thresholds (ChrisEverling) (see JENKINS-20985)

### 1.10
* Upped compatible Jenkins version to v1.554.3
* Added: MIT license
* Added: Add property to fail build on config fail (damienbiggs)
* Added: Make sure TestNGTestResultBuildAction caches counts and loads results lazily (Jesse Glick)

### 1.9.1
* Make duration calculation for PackageResults more accurate
* Retain white-space formatting in reporter output (steveims)

### 1.9
* Fixed the duration values to be more accurate by accounting for tests executing in parallel
* Fixed: JENKINS-24176 - Test result trend breaks lazy-loading (Jesse Glick)
* Fix default value of testng.reportFilenamePattern (juherr)

### 1.8
* Fixed: Added back package names in certain places where they were helpful (see JENKINS-19217)

### 1.7
* Fixed: JENKINS-23285 - All links on trend graph map are wrong on "testngreports" subpage
* Fixed: JENKINS-19217 - Change class names to not contain package names on result pages

### 1.6
* Fixed: JENKINS-19353 - Exception error message newlines are escaped
* Fixed: JENKINS-20968 - Doesn't resolve parameters in "TestNG XML report pattern" field
* Moved some configuration into Advanced section and cleaned up help sections
* Added: JENKINS-20438 - Make it an option to show failed builds in trend graph
* Added: New option to mark build as unstable on finding skipped config or test methods

### v1.5
* Fixed: JENKINS-19157 - Make st:bind work when Jenkins has jQuery plugin installed
* Add failure details to failed configuration methods on TestNG results page

### v1.4
* Bumped up supported Jenkins version to v1.526 or later
* Integrated TestNG plugin results into email-ext plugin

### v1.3
* Added: JENKINS-11899 - Print all URLs as HTML anchors (when escaping description and exception message)
* Added: JENKINS-12275 - TestNG plugin should contribute to Dashboard View plugin
* Fixed: JENKINS-18750 - NPE in testng graph
* Fixed: JENKINS-17564 - Reporter output being escaped incorrectly

### v1.2
* Added: Error details to failed test list
* Fixed: JENKINS-18498 - hudson.tasks.test.AggregatedTestResultAction$1 cannot be cast to hudson.tasks.test.TestResult
* Fixed: JENKINS-16832 - Do not show failed builds in trend graph
* Fixed: JENKINS-18206 - Total duration of a test class should include the times for the configuration methods

### v1.1
* Fixed: JENKINS-16592 - Invalid URL in testngreport on tests execution history chart

### v1.0
* Added: Implement AbstractTestResultAction
* Removed: deprecated class BuildIndividualReport (deprecated since v0.23)
* Removed: deprecated fields isRelativePath, results, fullName (were deprecated since v0.23 or earlier)
* Changed: duration in results to be seconds instead of milliseconds (as we now implement AbstractTestResultAction)
* Removed: Commented out Age column from reports as it wasn't symmetrical to Junit reports. Needs re-implementation
* Fixed: JENKINS-16374 - Allow concurrent builds for Projects using TestNG plugin to proceed

### v0.33
* Upped compatible Jenkins version to v1.447
* Fixed: JENKINS-15119 - Duration sorting is incorrect in TestNG report
* Fixed: JENKINS-12046 - TestNG shouldn't look for result files if build was aborted (again!)
* Added: JENKINS-10832 - make testng reporter-output field viewable from testng-plugin generated reports (Dan Alvizu)

### v0.32
* Fixed: JENKINS-12648 - Results should be reported for failed/aborted builds as well (Reverted fix for JENKINS-12046 in v0.30)

### v0.31
* Fixed: JENKINS-12187 - Ignore testng XMLs (if any) from previous builds
* Fixed: JENKINS-12494 - NPE while parsing testng xml with empty full stacktrace and no short stacktrace (Benoit Guerin)
* Fixed: JENKINS-12567 - failed tests and configurations shouldn't mark the build as unstable (Christoph Kutzinski)

### v0.30
* Fixed: JENKINS-12046 - TestNG shouldn't look for result files if build was aborted
* Fixed: JENKINS-11158 - Don't fail build if result files cannot be found
* Added: More logging around what the plugin is doing
* Improved TestNG trend graph accuracy. It now shows all builds, even ones for which there were no TestNG results

### v0.29

* Fixed: JENKINS-11236 - TestNG plugin links to wrong document (kohsuke)
* Fixed: JENKINS-11045 - Hold TestNG Results in memory using weak references
* Performace improvements (kutzi)
* Improve Load time for graphs (Kutzi) + trivial cleanup

### v0.28
* Bumped up supported Jenkins version to v1.403 or later
* Fixed: JENKINS-9918 - Limit the number of rows in method execution order table

### v0.27
* Added: JENKINS-10882 - Add duration in the XML API for test-result, package and classes [Nicolas De Loof]
* Added: JENKINS-10018 - New graph for test method details page

### v0.26
* Added: JENKINS-8926 - Add group name to class test methods page
* Added: JENKINS-9969 - Store/Display user provided custom test name

### v0.25
* Fixed: JENKINS-10000 - Config methods for classes with no test methods are not displayed
* Added: JENKINS-10001 - Store/Display group names (if any) for test methods
* Added: JENKINS-10098 - Add test/suite names to class results page

### v0.24
* Fixed: JENKINS-9742 - Test Report exposes data via the Remote APIs now

### v0.23
* Fixed: JENKINS-9835 - Remove isRelativePath option
* Fixed: JENKINS-9836 - Start storing testng-results.xml with build artifacts
* Moved help information from configuration options to help.html
* Fixed: JENKINS-9839 - Fix graph refresh after a build is deleted
* Added: JENKINS-9837 - capture parameters used to run the test method
* Changed to start appending '_x' where 1<=x<=n to test method names in urls
to disambiguate multiple tests with same name from same class. This stops
using the UUID that was being associated with all the test methods.

(NOTE: There is a slight break in backward compatibility in this release.
Significant code changes and features have been introduced in this version.
Older builds might face some issues, though the chances are very low. Also, the
urls for older data for test methods will change)

### v0.22
* A whole lot of code cleanup. Removed a lot of unnecessary classes.

### v0.21
* Have checkboxes for escaping test description and exception message selected by default
* Changes to how exceptions are displayed. No need to repeat message, highlight exception class etc
* Removed code supposed to make this work as Maven plugin. Was all unnecessary code.
* Fixed NPE because of GroupTestRun

### v0.20
 * New: JENKINS-9605 - Add option to not HTML-sanitize localized messages of exceptions

### v0.19
 * Fixed: JENKINS-9083 - fix urls in results trend chart

### v0.18
 * Fixed: JENKINS-9083 - handle jenkins context path in URLs properly
 * Updated pom dependency to jenkins 1.399 (make it simpler to release plugins)

### v0.17
 * Fixed: JENKINS-9299 - test results trend chart not showing
 * Updated pom dependency to jenkins 1.396

### v0.14
 * Fixed: JENKINS-8828 - Exception message is not displayed correctly for failed tests
 * Fixed: JENKINS-7916 - FormatUtil method is using TimeUnit.toMinutes which is not defined in Java 1.5

### v0.10
* Fixed: diff value was display as --3 instead of 3 and -3 instead of +3

### v0.9
* Fixed: Checked in the missing sidebar icon files

### v0.8
* New: added diff and age column for all test results
* New: list failures/skipped tests under "Latest Test Results" summary section

### v0.7
* Fixed: Plugin fails to find testng-results.xml file when project is configured to checkout sources from SVN SCM.

### v0.5

* Fixed: parser test cases were failing with IOException
* Fixed: Updated colors for Trend Graph to match JUnit graph and bar colors
* Fixed: Better display of content on the test method result page
* Fixed: Open/Close js behavior on Package results page
* Fixed: Removed a lot of dead code and unnecessary files
* Fixed: Trend Graph is shown for Projects and not for Builds
* Changed: TestNG Results link at Project page doesn't take you to latest build results anymore
* New: Trend graph has tooltips and links to different builds

### v0.4

* New: Initial release