Clarify (and possibly improve) issue states. #6655
Comments
marmarek
added
T: task
Even though I answered "no," I don't think we're in disagreement. The question is ambiguous. I chose to interpret it as asking whether it's expected that an issue with the "Release 4.0 updates" milestone would be closed without ever updating R4.0. In other words, if such an issue gets closed, but no 4.0 update ever drops for it, is that a mistake? I believe it would be, so I answered "no." Due to the ambiguity of the question, I think you and I are both correct.* I chose to interpret the question this way because it was asked by @mattmccutchen, who (I believe) is familiar enough with our issue tracker to have observed that there is generally a time lag between issue closure and update availability. I figured he wasn't expecting that a 4.0 update would immediately be available as soon as #6321 was closed. Rather, I figured he was wondering why there was still no 4.0 update five months after it was closed. So far, there have only been 4.1 updates for it, the last one on Mar 16, with no indication of any 4.0 updates. This suggested to me that either the milestone was wrong (should have been 4.1 instead) or that the 4.0 updates were forgotten, since usually the time lag is not that long. Are you saying that, in this case, the time lag just is that long, and 4.0 updates are still forthcoming for #6321?
This description matches my understanding of our system. We're on the same page. I think it's fine as is, but it would be more intuitive for others (users and contributors unfamiliar with the current system) if issues were closed at step 7 instead. If you think that's easy to implement and wouldn't be disruptive to developer work, then it might be a worthwhile improvement. It would probably also be simpler to communicate ("issues are closed when updates are in stable" vs. the entire explanation you just wrote above). * There is another way I could be wrong, namely if we had a policy of sometimes closing issues on the 4.0 milestone by fixing them in 4.1+ (but not in 4.0) without altering the 4.0 milestone. In fact, I seem to recall that we may have done this at least a few times, but probably without thinking through the implications and certainly without having a "policy" of doing so. We should probably decide whether the appropriate course of action, in such cases, is to change the milestone to the release receiving the actual fix or to do something else. |
andrewdavidwong
added
C: other
project management
|
Thanks Marek for filing this issue and Andrew for adding your point of view. Andrew is basically right about my position. I understand that the Qubes project has a ton of things that need to be done and very limited staff time; I tried to help a bit with that by donating some money a few months ago, and I'm very grateful that you're doing as much as you are. But I'd think that backporting already developed fixes to the version of Qubes OS that most(?) users are actually using would typically be more valuable (and thus a higher priority) than fixing more issues, especially for trivial backports like #6321. I even wonder if it would make sense to make a habit of developing the R4.1 and R4.0 updates at the same time in cases such as #6321 where it seems unlikely that a problem would be detected in R4.1 that would result in work on the R4.0 backport having been wasted. I'd be happy to help with backports if I can, but I imagine it's more work for Marek to review the accuracy of a backport from me than to just do it himself. Whatever priorities and tracking mechanism you all decide on, better documenting it would be greatly appreciated. I know that with your limited time, you need users to "meet you halfway" on many of these things by doing research themselves, but it's good to improve the documentation when we can. Maybe I can submit a PR for that once you make a decision. Would the "reporting bugs" page be a good place for the information? That's the main place I looked for a policy on backports before asking. I certainly don't think the https://github.com/QubesOS/qubes-issues home page (which currently has the link to the "pending backports" query that Marek mentioned) is obvious enough to most users. |
Thank you for your understanding and support!
Yes, this is the page where we'll want to document the policy governing when issues with certain milestones are closed, once it's decided. |
Hmm, in fact it may not be that simple change. To work reliably, it would require disabling github's automatic closing issues with special words in commit messages (but also - we do want to keep mentioning which commit fixes which issue).
I realize the sample size is small, but also I haven't found a single project that closes issues (on github specifically) only when the change gets released. In our case, when the change is released, we already have a comment from builder-github bot (and also labels are updated). So, it should should be clear when it happens. I think there are two states when it isn't clear:
Maybe we can have some extra (human readable) labels, instead of opaque filter(*) ? Something like "backport-pending", or "release-pending"? We can probably pretty easily automate adding/removing them. (*) I've just found the filter is not perfect - it picks up issues as "pending backport" when the change in R4.1 is relevant to both dom0 and VM, but in R4.0 to VM only (and was already backported). In this case |
Ah, it sounds like we're already doing more than most other projects, then.
Yes, that's a great idea. However, I think we should also have automated comments for these in addition to the labels, since many people seem to ignore even the clear human-readable labels. Also, I don't think changing the labels triggers an email notification, whereas a comment does, so this could be important for people who primarily use email for interaction or archiving.
We could always get more fine-grained with our label system, but I don't know whether the additional complexity would be worth it. I think the deciding factor should be whether (and to what extent) it supports your work and the work of other developers. Making the developer workflow transparent to outside observers has value, but it's not as important as the work itself and should never impede it. |
Labels can have also tooltips and they also show up in changes history (not only this right pane).
I kind of prefer label here for this very reason. Label like "backport pending" is IMO more useful as an issue state (if someone finds such issue, to see where is it in the process of hitting R4.0), than an event (whether issue is a backport candidate, should be clear from the very beginning - based on its milestone and affected version). On the other hand, when related package is uploaded to a repository (an event, state change), we already have a comment about it.
Given the corner case mentioned earlier, I think an explicit "backport pending" label has an extra value: it can be manually adjusted if the automatic assignment somehow fails one way or another. With the current system, I need to handle such cases manually (either by ignoring such issues each time, or by using other place to note things to backport); or by adjusting issues metadata in a confusing way (like changing the labels manually, to values not really matching repository state...). |
Yes, I know (and have added many tooltips to our labels), but that doesn't stop people from ignoring them. 🙂
Ah, ok. Fair enough then.
Sounds worthwhile to have this new label, then. |
|
Created the backport pending label with this description:
(Please feel free to improve the description.) |
|
I've revamped the Issue Tracking page (previously known as "How to Report Bugs and Other Issues") and added a Backports section that documents what we discussed here. @marmarek, if you need to do other things on your end (e.g., automate application of the Also, if what I've done is not what was intended or is not sufficient to close this issue, please reopen. |
#6321 (comment)
In fact, yes. Issue states in github are quite limited (just "open"/"closed"), so it doesn't allow to express all the possible states:
Currently, we close issue at the step 3. Then, when updates gets released, issue automatically get appropriate labels:
r4.1-*-cur-testr4.1-*-stabler4.0-*-cur-testr4.0-*-stableBased on those labels, it's possible to select issues waiting for step 6 - see https://github.com/QubesOS/qubes-issues#issues-by-release.
We choose to close issues at step 3 mostly because github provides automation for it for free (just mention "Fixes QubesOS/qubes-issues#...` in the commit message). If we decide to close issue at a different stage, we'd need to write automation for that ourselves (but it isn't much work).
This indeed may be confusing for some (issue is closed, but fix is not yet available for some setups). But given the number of states a fix can be in, and just 2 states in github, there will also be some discrepancies. The current workflow is also friendly for developers - when issue is "open" it should be a thing to work on.
At the very least, we should have this documented. But maybe we could improve it somehow?
The text was updated successfully, but these errors were encountered: