💖 Python Jira needs CI maintenance help

[ssbarnea]

That is a call for help on fixing JIRA CI project so we can test and merge change. I am personally against merging any change that does not pass CI but reality is that CI degrated considerably in the last years and cannot be fixed in one hour or two, probably not even an entire weekend.

If you care about its future, there are two ways you can pick

• Raise PRs that are addressing the CI issues and not by removing or disabling tests.
• Use the sponsor option to show your support, maybe you can convince me to go back to jira instead of the other projects
• Become a maintainer yourself. Well that is the best and if you can deal with first bullet, you have my approval.

PS. Sadly but I am unable to review and PR that does not report as green.

[truthbyron]

Hi @ssbarnea, I believe this is failing because of Deprecation of basic authentication with passwords for Jira and Confluence APIs, given the Travis runs of my PR:

E           jira.exceptions.JIRAError: JiraError HTTP 403 url: https://pycontribs.atlassian.net/rest/auth/1/session
E           	text: Basic auth with password is not allowed on this instance

I was able to create tie a test Jira account to the testing cloud instance, create an API token, and authenticate using a basic auth tuple of (email, token) and get a totally different set (and way more) errors, so I think that method works.

Is there any way we could get a dedicated token generated for the test and admin user and change to using email address instead of username for this block

jira/tests/tests.py

Lines 116 to 134 in 5c27f6d

	if "CI_JIRA_ADMIN" in os.environ:
	self.CI_JIRA_ADMIN = os.environ["CI_JIRA_ADMIN"]
	else:
	self.CI_JIRA_ADMIN = "ci-admin"
	if "CI_JIRA_ADMIN_PASSWORD" in os.environ:
	self.CI_JIRA_ADMIN_PASSWORD = os.environ["CI_JIRA_ADMIN_PASSWORD"]
	else:
	self.CI_JIRA_ADMIN_PASSWORD = "sd4s3dgec5fhg4tfsds3434"
	if "CI_JIRA_USER" in os.environ:
	self.CI_JIRA_USER = os.environ["CI_JIRA_USER"]
	else:
	self.CI_JIRA_USER = "ci-user"
	if "CI_JIRA_USER_PASSWORD" in os.environ:
	self.CI_JIRA_USER_PASSWORD = os.environ["CI_JIRA_USER_PASSWORD"]
	else:
	self.CI_JIRA_USER_PASSWORD = "sd4s3dgec5fhg4tfsds3434"

I'm not sure how technically able to help I am, but I'm happy to research more/help if you have time to give any direction (otherwise I'll keep poking around as I can). Thanks for this project and for your time.

[csullivannet]

@ssbarnea I've run this as well, setting the environment variables for CI_JIRA_ADMIN, CI_JIRA_ADMIN_PASSWORD, CI_JIRA_USER, CI_JIRA_USER_PASSWORD to my own username (my e-mail) and my API token. I was able to successfully authenticate when running python -m tox locally.

Further, I disabled the check here to see whether I'm the admin:

jira/tests/tests.py

Lines 163 to 165 in 5c27f6d

	if not self.jira_admin.current_user():
	self.initialized = 1
	sys.exit(3)

This resulted in being able to make the API calls which returned 403. This shows that I was an authenticated but not authorized user. This gives me confidence that this will resolve this authentication issue.

Updating these two lines to use the token instead of password should fix the authentication issue, (although it may be possible the username must be replaced with the e-mail):

jira/tests/tests.py

Line 124 in 5c27f6d

self.CI_JIRA_ADMIN_PASSWORD = "sd4s3dgec5fhg4tfsds3434"

jira/tests/tests.py

Line 134 in 5c27f6d

self.CI_JIRA_USER_PASSWORD = "sd4s3dgec5fhg4tfsds3434"

Unfortunately neither I nor @truthbyron have access to the ci-admin user, so we cannot create this token. The token can be generated here when logged in as ci-admin: https://id.atlassian.com/manage/api-tokens

[ssbarnea]

@truthbyron @csullivannet I added both of you as side-admins on https://pycontribs.atlassian.net/ -- please use your powers with extra attention. My hope is that you will find time to fix the CI issues. Reply here if you need my help again on other aspects.

We are better off removing the hardcoded credentials from the code and replacing them with a fast fail when they are not defined. These variables are defined on Travis, so the values from code are likely not even used.

I also sent you invite to github team, which I suppose will allow you to configure travis credentials at https://travis-ci.com/pycontribs/jira/settings in case you need to update them.

Merging changes will still need review and I am more than happy to help with these once I see travis reporting green.

[csullivannet]

@ssbarnea Thanks, I've been able to update the build to use a new token I generated for the ci-admin/ci-user users. This has fixed authentication. It does appear that some tests may now be legitimately broken, but I'll need to investigate further to confirm. A preliminary glance shows almost all the errors are due to the missing name field, which is likely due to GDPR compliance updates server side.

[truthbyron]

Thank you both @ssbarnea and @csullivannet ! I hope to able to get back to this mid-next week and help fix any legitimate broken tests.

[csullivannet]

@ssbarnea @truthbyron I have the CI returned to green by #909

It's likely there still remains code broken by Jira's GDPR compliance updates that's not covered by the tests.

[truthbyron]

Nice @csullivannet! I'll be able to do some review/testing tomorrow.

[Addono]

So over the last month or so sometimes a new solution has come up. I will just leave them here as food for thought for anyone who wants to pick it up.

Testing against Jira Cloud - old pipeline:

• PRs do not have a pipeline if they do not supply credentials themselves. Repo maintainers need to first check over the PR manually to see if there is any bad intent to steal the CI secrets. If not, then the PR branch is pushed to a branch inside the repo, which will run the CI.
  Yes, this is tedious, however a bot could help with this. To me, the real advantage of this approach is that the old pipeline can be kept.

Testing against Jira Server:

• Use atlas-cli to spin up a Jira instance with a 3 day development license. (I have been working on this quite a while back on this. The issue I ran into is that 1) a jira-run-standalone server does not come with the jira-software plugin installed. There exists an API, however it's undocumented. So instead my next step would be to use robotic process automation to click through the GUI in a headless browser to get a license. In other projects I had success by creating an archive of the data folder of the Jira instance and just unzip it in the CI pipeline. However, none of the tests in that project were writing to the Jira instance. Hence, having the license expired wasn't an issue. 2) I couldn't find any commit since the last stable release where this library's tests would pass against my local Jira server instance. At some point I started cherry-picking patches from 896 - GDPR Update and CI fixes #909, but even that wasn't enough to get it to run. (If anyone feels like going down this road, then feel free to ping me and I can show where I left of and discuss your approach.)

[ssbarnea]

Both approaches are needed, but the 2nd one can be easier to implement by using a test container. We need to be sure that this container starts with ready to use jira instance without any modifications.

Once we have it, we can make use of https://github.com/tox-dev/tox-docker to allow it to start/stop it for testing.

[Addono]

That was also my gut feeling, however no Atlassian SDK containers existed last time I checked - which was ~2 months ago. So I put a Dockerfile together, which can start a Jira standalone instance. However, then you still need to click through the UI to enable Jira Software and create a test user. So, a Jira instance "without modifications" is not enough in order to be able to run the test suite.
https://github.com/Addono/docker-atlassian-sdk/blob/master/Dockerfile

The Jira SDK seems quite stable - last release was mid 2019 - so I wouldn't expect it to require a lot of maintenance, hence maintaining it here would be an option - leaving it up to others requires trusting them.

Tox-docker looks cool, probably cleaner to use that than running the container detached as I was fiddling with 😅.

[brainwane]

Since you mentioned sponsorship: Just saw that this package is eligible for funding via Tidelift in case that is of interest to maintainers here.

[valentijnscholten]

I've been having good experiences with https://vcrpy.readthedocs.io/en/latest/ to write integration tests with a JIRA API, for example in https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/unittests/test_jira_import_and_pushing_api.py
Might be less usefull if the goal of you tests is to keep up with changes on the JIRA side.

[studioj]

i would be glad to spend some time here, some one wants to get me up to speed?

i would also be ok to spend time to make this integration tests less needy => mocking out the responses
then the only real thing to validate is the authentication mechanism and 1 or 2 very simple tests

@Addono maybe a stupid suggestion couldn't we do a docker commit after doing the setup manually?
just getting started so probably shooting in the wild :-) I'll investigate if https://hub.docker.com/r/atlassian/jira-software could be used, maybe in combination with vcrpy that @valentijnscholten suggested

[Addono]

Hi @studioj, it has been a while since I was active here. So not entirely sure if I'm up to date with the latest changes.

In general, the best way to get up to speed is to read back a bit. I wrote plenty of long-form comments which describe the rather involved situation we're in. #909 could also be of interest.

... to get the CI to pass again three issues needs to be resolved:

1. Fixing the codebase and test-suite to be compatible with the latest version of Jira. As no fixes [were] merged, the drifting API specs cause the current test suite to fail.
2. Provisioning of a Jira instance to test against. Previously, a public Jira Cloud instance was used. However, the credentials to this instance can only be used for branches inside the repository. As giving credentials to this Jira instance to CI runs against PRs from untrusted sources would risk having them exposed.
3. Updating the CI to make use of the changes made to accommodate (1) and (2).

Dockerizing Jira is task (2), which I took a shot at here: #942. So you can build on that.

I'll investigate if https://hub.docker.com/r/atlassian/jira-software could be used

If I remember correctly, the issue is to automatically provision a Jira instance with a test license. The default Docker images can provision a demo license, but that requires clicking through the UI and logging in. Instead I then opted for containerizing the atlassian-cli, see https://github.com/Addono/docker-jira-software-standalone.

couldn't we do a docker commit after doing the setup manually?

Reason being that the demo license is only valid for a short while, so now your CI depends on someone keeping releasing new versions of your image. It's probably not a dealbreaker, so if that's how you get it to work then by al means try, but not having to worry about that would be better 😄 .

[studioj]

thx for the answer. Would it be an idea the maintainers are open to, this vcr.py?

It doesnt seem like an easy thing to dockerize jira and have a full blown running instance to test against.

I also read Atlassian is not providing licenses for jira server and wants to end of life it by 2024 ... see their FAQ

about the forks and CI system, would it be maybe logical to have people merge their forked branch first to a branch of the repo.
and then run a second PR to get it into master? it's this issue you're referencing, right?

maybe having a vcr.py test validating PR's from forks and a full blown test against a jira server if to master?

[Addono]

Would it be an idea the maintainers are open to, this vcr.py?

Just quickly glanced over it, I wouldn't now for sure. To me, it seems viable for writing integration tests. However, the old tests were end-to-end/system tests, so this would change the testing strategy.

Probably at this point anything which makes releasing new versions reliably is fine.

I also read Atlassian is not providing licenses for jira server and wants to end of life it by 2024 ...

Then one could argue that the project drops support for Jira server, however, the original CI ran against a cloud hosted instance, and then we ran into issues to protect the credentials.

about the forks and CI system, would it be maybe logical to have people merge their forked branch first to a branch of the repo. and then run a second PR to get it into master?

Yeah, I suggested something along those lines earlier.

[studioj]

i'll first try to document the steps needed to get the tests up and running locally :-) can't really find too much docu on that

[Addono]

AFAIK this is all there is.
https://github.com/pycontribs/jira#setup

Be adviced, the tests are likely not to pass, since this tests against a Jira server instance rather than Jira Software Cloud. So at least you need to enable Jira Software.

[studioj]

ok cool, i got some tests up and running via tox (new to tox) (seems to be the jira cloud tests)

• needed to get a ubuntu vm (yes I'm a windows user)
• needed to install npm
• needed to install pyenv
• needed to install pykerberos
• for each env i needed to add this in the tox.ini:
  python -m pip install pytest-cov flaky keyring requests-mock

=> i cant find where the test requirements are stored :-S so dont know really where to add it. (maybe just contributing.rst?)

tests are failing now because of basic auth being deprecated, is that the correct state?

ok seems running the tests on a local instance can be done by setting env:
CI_JIRA_URL=http://localhost:2990/jira
CI_JIRA_ADMIN=admin
CI_JIRA_ADMIN_PASSWORD=admin
CI_JIRA_USER=jira_user
CI_JIRA_USER_PASSWORD=jira

[Addono]

This is where the test dependencies are defined:

jira/setup.cfg

Lines 76 to 97 in 7fa3a45

	test =
	docutils>=0.12
	flaky
	MarkupSafe>=0.23
	mypy
	oauthlib
	pre-commit # MIT
	py >= 1.4
	pytest-cache
	pytest-cov
	pytest-instafail
	pytest-sugar
	pytest-timeout>=1.3.1
	pytest-xdist>=1.14
	pytest>=5.0.0,<6.0 # MIT
	PyYAML>=5.1 # MIT
	requests_mock # Apache-2
	requires.io # UNKNOWN!!!
	tenacity # Apache-2
	wheel>=0.24.0 # MIT
	xmlrunner>=1.7.7 # LGPL
	yanc>=0.3.3 # GPL

If you would install the package using pip install jira[test], then these deps should be installed (🤞 ).

[studioj]

trying to use your docker image to at least get the same results as the CI system ... but i'm not there yet :-)

the docker is now running on windows and i can login via the browser => but I can't run the tests yet

In my ubuntu vm its not working locally yet. 😒

• just found out that starting the docker can take more than 10 mins (=> maybe deploying an image which was started already could speed stuff up) => https://phoenixnap.com/kb/how-to-commit-changes-to-docker-image
• Edit: 64 failed, 22 passed, 42 skipped, 20 xfailed, 1 warning, 3 errors => yet a step closer

Once I have that docker up and running and have the same state as the CI I'll open a PR with a shell script to help people join the effort.

• probably the test.local file is a good location to add that part?
  My time spent should help people :-)

[studioj]

Am I correct to assume the CI in fix_ci doesnt run on the jira cloud instance?

just commited this which could make getting started a little bit easier
studioj@f760c30

[Addono]

Am I correct to assume the CI in fix_ci doesnt run on the jira cloud instance?

Yes, that's correct.

[ssbarnea]

I invited @Addono @adehad and @studioj to the repository, giving them write access. (see invitation from github)

Please use them with care and wait for reviews on your own changes. First action you should take is to rebase existing open PRs and see if you can fix them, request them to be fixed or just close them if they do not make sense.

And yep, the invitation is still open. For release, ping me and I can probably reconfigure the publish pipelines.

[adehad]

Just echoing that we still can use all the help we can get !

We are definitely in better shape than we were, i.e. we have a passing test pipeline ( #1017 ).
But there is still a lot of work on how we support and test against Jira Data Center / Server (self-hosted) and Jira Cloud (Atlassian hosted) moving forward, especially as their APIs diverge.

[ssbarnea]

If anyone is willing to show some love to jira library and help review/merge/abandon current PR and issues, ping zbr on #pycontribs on irc and so an sync and do a weekend clenaup.

[adehad]

@ssbarnea

can I ask if the credentials for the Cloud instance ci-user and ci-admin:

    def set_jira_cloud_details(self):
        self.CI_JIRA_URL = "https://pycontribs.atlassian.net"
        self.CI_JIRA_ADMIN = "ci-admin"
        self.CI_JIRA_ADMIN_PASSWORD = 
        self.CI_JIRA_USER = "ci-user"
        self.CI_JIRA_USER_PASSWORD =

exist as GitHub Actions Secrets?

I would like to propose they exist as:

CI_JIRA_CLOUD_URL
CI_JIRA_CLOUD_ADMIN
CI_JIRA_CLOUD_USER
CI_JIRA_CLOUD_ADMIN_TOKEN
CI_JIRA_CLOUD_USER_TOKEN

It looks like @csullivannet had setup these API tokens at some point on the old Travis system?

There also looks like there may be an OAUTH key, which perhaps could instead be put into a secret?

jira/tests/tests.py

Lines 30 to 40 in eb80088

	OAUTH = False
	CONSUMER_KEY = "oauth-consumer"
	KEY_CERT_FILE = "/home/bspeakmon/src/atlassian-oauth-examples/rsa.pem"
	KEY_CERT_DATA = None
	try:
	with open(KEY_CERT_FILE, "r") as cert:
	KEY_CERT_DATA = cert.read()
	OAUTH = True
	except Exception:
	pass

But it might be an either/or situation.

I'm working on #1045 - which is hopefully setting some infrastructure for testing on both Cloud and Server instances.

[ssbarnea]

@adehad ping me again when you can on irc and I can give you extended access to both the repo and the test instance, so you perform maintenance work like updating the tokens, test accounts.

[adehad]

as an update I've got #1107 (and #1109) in a place where the testing on Jira Cloud works for me locally and I've set up the necessary variables in this repo so it should be able to run on GitHub Actions. I'm not sure I can get the new GA to run on that branch without it first existing on master, so I may try and get it working on a fork first. The current file I've setup attempts to wait for the Server CI to finish before running the Cloud CI as an added precaution.

[ssbarnea]

I am afraid I do not have the time to look at what is happening with the code itself and I just want to handle its maintenance to those that have the time and drive needed.

In order to avoid accidents I will require not to merge changes without another core review and I do expect cores to do perform reviews of changes proposed by others. This can easily be achieved by adding them to CODEOWNERS file, so they would be automatically assigned to reviews.

Based on recent activity I am inclined to believe that @adehad and @studioj would first two to add, if they agree. Still having only two is bad as if one is in unavailable, the other one would not be able to merge his own changes to master. Anyone else interested? If anyone thinks it may not be a good idea to add someone, please reach me in private.

In general I am inclined to trust more github accounts that have real names and publicly identifiable information about their owners (like companies where they work, social links). I don't really want to endup in the news because jira library was used to inject some malware ;)

[Zyzyx]

Hi everyone, I can make time to help with this. I have a few features to work on ahead of me too.

[adehad]

For complete transparency I want to report on the sudden Jira Cloud failures a couple months ago.

It seems that some users had created 'Team' managed projects in our Jira instance. This resulted in the test framework (with some of its hard coded assumptions) failing. ie it was trying to use a permission scheme of another project. This resulted in the failures we saw.

What I have done is removed the permission for anyone creating these Team managed projects. Therefore only company managed projects can be made on that instance, which is done by our CI users.

To be honest I am not really sure why we have so many public users on the pycontribs jira. Currently there are around 300.

@ssbarnea do let me know if this affects anyone to your knowledge.