Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Before Class reported as Broken even it passed in retry #912

Open
Apzal opened this issue Apr 19, 2023 · 0 comments
Open

Before Class reported as Broken even it passed in retry #912

Apzal opened this issue Apr 19, 2023 · 0 comments
Labels
theme:testng TestNG related issue type:bug Something isn't working

Comments

@Apzal
Copy link

Apzal commented Apr 19, 2023

Describe the bug
I have added retry plugin in my build.gradle, which retries when any test fails. The issue is when beforeclass annotated method fails in first run its reported as broken in Allure report but in retry if its get passed the status of before class is ignored and the report still shows as failed.

To Reproduce
Steps to reproduce the behavior:

  1. Add retry plugin in build.gradle file
plugins {
    id 'java'
    id 'io.qameta.allure'
    id "org.gradle.test-retry" version "1.5.2"
}

allure {
    def allureVersion = "2.15.0"
    version = allureVersion
}

test {
    useTestNG()
            {
        
                String groupName = System.getenv().getOrDefault('groups', 'REGRESSION')
                String testName = groupName
                includeGroups groupName
          
                setPreserveOrder(true)
                setParallel("classes")
                setThreadCount(10)
                setSuiteName("Test Suite")
                setTestName(testName)
                useDefaultListeners = true
                outputDirectory = file("$projectDir/TestNgReport")

            }
    retry {
        failOnPassedAfterRetry = false
        maxFailures = 200
        maxRetries = 2
    }
    reports.html.setDestination(file("$projectDir/GradleReport"))
    reports.junitXml.mergeReruns = true
}
  1. Run the TestNg test with before class which fails in first run and passes in second run. Build will be suceeded
  2. Generate Allure report
  3. Report shows before class as broken and Test as passed

Expected behavior
Before class should not be reported as a failed test instead report as passed.

Screenshots

image

Desktop (please complete the following information):

  • OS: Mac Monterey
  • Browser : Chrome
  • Version 112.0.5615.49

Additional context

@baev baev added type:bug Something isn't working theme:testng TestNG related issue triage labels Aug 23, 2023
@baev baev removed the triage label Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
theme:testng TestNG related issue type:bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants