[Bug]: Testsigma on-site instance integration with Jenkins CI [this bug is raised as suggested in discord app]

[manognamukthinuthalapati]

Is there an existing issue for this?

• I have searched the existing issues

What is the current behavior?

Hi Team,
I am trying to integrate Testsigma [opensource] with Jenkins and I followed the popular reference available in testsigma official site.
https://testsigma.com/docs/continuous-integration/jenkins/
I think the information related to the Testsigma Test Plan Plugin should be updated in the document. As per the provided doc, there is no field named Testsigma App API End Point which you will find in the latest version of the Testsigma Test Plan Plugin. Since I am unable to give correct input in the specified field, CI pipeline is failing with multiple errors.

If I use localhost as hostname in input url getting the below error:
JENKINS CONSOLE OUTPUT:
************Started Testsigma Testplan execution************* Testsigma API Endpoint:[https://localhost:443/api/v1/execution_results?auth-token=d98cbaa725d6449081e22e556ca0efa7](https://localhost/api/v1/execution_results?auth-token=d98cbaa725d6449081e22e556ca0efa7) Provided TestPlanID:305 Max wait time in minutes:180 Polling Interval:5 minutes Report file path:./report.xml Test plan Id:305 FATAL: Certificate for doesn't match any of the subject alternative names: [[local.testsigmaos.com](http://local.testsigmaos.com/)][javax.net](http://javax.net/).ssl.SSLPeerUnverifiedException: Certificate for doesn't match any of the subject alternative names: [[local.testsigmaos.com](http://local.testsigmaos.com/)]
Nginix is loading certificates from local.testsigmaos.com and any certificate isn't generated manually using keygen here then how should I correct this certificate error?

Following some references, as suggested I use http://app.testsigma.com/ instead of 'localhost' as hostname then getting below error:
JENKINS CONSOLE OUTPUT:
************Started Testsigma Testplan execution************* Testsigma API Endpoint:https://app.testsigma.com/api/v1/execution_results/40 Provided TestPlanID:305 Max wait time in minutes:180 Polling Interval:5 minutes Report file path:./report.xml Test plan Id:305 TOKEN MISSING ERROR: Build step failed with exception java.lang.RuntimeException: Failed : HTTP error code : 401
Though I am passing the API key it is TOKEN MISSING error and I am pretty much sure that providing app.testsigma.com as hostname is not a right way but I just tried it. Maybe that bad url input is causing 401 error here.
There is a difference in API Token of a free instance and API key of a Testsigma Cloud instance.

Instead of using Testsigma Test Plan Plugin, I just tried to run the REST API given in Test Plan page. The API is working as expected when I tested in API Tester but when I build a job in Jenkins to run the same API, I got an issue again:

JENKINS CONSOLE OUTPUT:
C:\Users\LENOVO\.jenkins\workspace\Demo>curl -X POST -H "Content-type: application/json" -H "Accept:application/json" -H "Authorization: Bearer d98cbaa725d6449081e22e556ca0efa7" -u XXXX:XXXX [https://localhost:443/api/v1/test_plan_results?testPlanId=281](https://localhost/api/v1/test_plan_results?testPlanId=281) % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 curl: (60) schannel: SNI or certificate check failed: SEC_E_WRONG_PRINCIPAL (0x80090322) - The target principal name is incorrect. More details here: https://curl.se/docs/sslcerts.html curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.

I wonder why the REST API which worked in API Tester is throwing an error while running from Jenkins. So I switched to last trial and used the powershell file provided in Testsigma site and try to run pipeline job in Jenkins. However, no luck on it.
JENKINS CONSOLE OUTPUT:
No of polls: 36 Polling Interval:100 Junit report file path: ./junit-report.xml Code: Description: Error encountered in executing a test plan. Please check if the test plan is already in running state.
Though Test Plan is not in running state, pipeline is showing always the same issue.

What is the expected behavior?

I tried various ways expecting to see the Test Plan results in Jenkins CI for open-source on-premise instance of testsigma server but none of them haven't worked well. Now I just want to understand how can we integrate this opensource on-premise instance of Testsigma with Jenkins. It is fine either if we use a Plugin/REST API/pre-existing pipeline script but ultimate goal is to do Jenkins CI for it.

Steps To Reproduce

Ruuning Jenkins CI with Testsigma Test Plan Plugin
Running the Job using REST API
Running the CI pipeline with given powershell script

Version

Testsigma Community Edition

[GitBBrepository]

Anybody solved that issue? I have the same problem.