Issue Ice Box Policy

[DanVanAtta]

I would like to propose an ice-box policy for the issue queue. The motivation behind this is that any operations queue that is allowed to grow becomes an overhead in of itself. If interesting I could try to dig up some links to talks that go into this in some detail.

Proposal:

• anything older than N days will be marked with 'ice box - revisit later' and closed

Reason and Benefits:

• in one sense it is admitting that if we do not fix something after 'n' days, we are not very likely to fix it in 'n+1' days, so we should save ourselves the overhead of the active tracking
• breath of fresh air to the queue, we'll be able to see active issues better, from there we can resolve anything not actionable, then our queue will be short and all actionable; we'll be on a path to being able to keep it clear.

Wait, won't we be losing sight of problems?

• We are not fixing these problems anyways, clicking on the "ice box" label and finding closed items is pretty easy to do; that is the least difficult part if someone is actually fixing 'old' bugs
• Keeping every problem we are not fixing in active view is not helpful for tracking the things that are new or that we can fix.
• Keeping unresolved problems in active view builds up a negative culture where things are actively ignored; by providing an SLA we will have some additional motivation to try and address the problem before the 'n' days is up, nobody likes to admit defeat and close an issue with an ice-box.

[Cernelius]

I disagree. I believe unresolved issues, comprising issues deemed not worth to be resolved, should be tracked forever (or up to, say, 100 years).
Also I believe it should not matter at all if an issue was opened today or 10 years ago.

[Cernelius]

For example, if someone opens an issue about this:
https://forums.triplea-game.org/topic/989/aa-revised-minor-bug/44?lang=it&page=3&loggedin=true
I believe it should be tracked forever, like having it still open 20+ years from now, otherwise the matter would be forgotten, and all the discussion and research effort wasted.

Anyways, I believe that having a clear and well documented timeline is better than just arbitrarily close issues just because they are old, so that, in such a case, people at least know how long their issues will be tracked, if not fixed, in the moment they have to decide if to spend the time (all the effort reading rulebooks and addenda etc.) making it at all.

What I'm saying is never closing issues just because they are old, but if you are going to do that, a clear and documented objective timeline is advisable.

Also, in this case, instead of "Ice Box" I suggest the label: "Not Fixed", beside the others (for example, "Bug"+"Not Fixed"), to distinguish all the issues that have been closed for different reasons than being fixed or invalid.

[Cernelius]

For example, in case of bugs:

An invalid bug would be closed with no label.

A valid and fixed bug would be closed with the "bug" label.

A bug that has been tracked somewhere else (in a new issue) would be closed with the "bug" label.

A valid but not fixed or only partially fixed bug that was closed for whatever other reasons would be closed with the "bug" + "not fixed" label. Probably, partially fixed bugs should always have a new issue opened for whatever remains of them, so the "not fixed" would practically apply only to totally not fixed ones that have not been reopened somewhere else.

A possibility would also be to add a "fixed" label for the bugs closed for being fixed, but I guess that would imply going through all the already closed bugs and adding such a label to them, to keep coherency. Another advantage of such a label would be to distinguish the valid bugs that have been closed because tracked in new issues (thus not fixed at that issue). Or maybe it would be better not having a "fixed" label, but, instead, a "duplicate" label, to indicate all those bugs closed for having been reopened in a new issue, as well as more recent ones closed because someone opened a duplicate.

If a new issue is opened that tracks all that is being tracked in an issue labelled "bug"+ "not fixed", the "not fixed" label should be removed from it, and preferably the new issue referenced.

I think the matter of tracking them forever is only for the bugs. For whatever else, it seems reasonable to close them after some time, instead.

Of course, these are just hand-off suggestions. It's up to the developers to manage the label as it works smoother for them.

[DanVanAtta]

The working definitions of labels/status are not meant to be changed here. To review:

• closed + ice-box label: means there are actionable items remaining
• closed and no ice-box label: no actionable items remaining (the issue may contain useful reference/historical information, but never should there be a new issues saying "look at closed issues #xyz, finish the tasks in there." Instead such an issue like that should state specifically what needs to be done and would use the closed issue as a reference link)
• open: new, actionable items remain and can be worked on

Partially fixed items should be (re-)open(ed), or can be closed if all remaining items are tracked in new individual tasks.

What would change in this proposal is that we have a policy for using 'ice-box' more consistently and apply it to older items so we can filter those from active view. That will reduce overhead and let us keep our ticket queue to zero. It is not nice to think that no matter what, we can't complete all tasks in our task queue, that represents a failure of the task queue that it cannot be completed.

Just as we do not lose track of ice-box issues now, we won't lose track of them later by expanding and formalizing the usaeg. Those tickets will always be available behind this link: https://github.com/triplea-game/triplea/issues?utf8=%E2%9C%93&q=is%3Aclosed+is%3Aissue+label%3A%22ice+box+-+close+and+revisit+later%22+

The tickets are also not 'unworthy of a fix', those items whatever they would be are simply closed (I generally think most problems are UX problems instead of user problems. If lots of invalid bugs were a problem, I'd look to add a label for it so we can start analyzing them and find ways to fix root causes, like poor design that makes it hard to tell what the computer was doing, correct or not).

[DanVanAtta]

Looking at the back of the queue, we would benefit quite a lot from a policy that helps us close out discussions/old issues. Of course, closing out with a special label makes those items still easily found and we can bulk re-open and delete the label if we think it does not work out.

[DanVanAtta]

A question to be decided what should 'n' be, how long before we ice-box an issue?

[DanVanAtta]

@Cernelius thank you for the feedback and challenge to the idea. Your suggestions are actually I think almost similar, but instead of how you would view closed + not labelled, I think am suggesting that to be closed + ice-box label.

Added assignments to dev group; this is a pretty notable change in process.

[ssoloff]

My SWAG is 6-12 months.

[RoiEXLab]

I don't have a strong opinion on this.
@ssoloff's suggestion seems reasonable to me.

[ron-murhammer]

No strong opinion on this. Honestly I tend to look through the issue queue starting at most recent so don't think this would really change things much at least for me. Agree with something around 6-12 months. Maybe could consider leaving anything marked as a bug but close anything else?

[panther2]

My bad that I have overlooked this discussion and now am confronted with a lot of issues having been "ice-boxed" and closed.

Anyway, I see now that many rules issues have been ice-boxed, some rules issues are still active and some rules issues are active but have not even been commented on, maybe some or all of them just awaiting their "ice-box"-label.

As you know that I am spending a lot of time dealing with rules issues, clarifying on rules, discovering as well core rule violations as minor issues and so on it is not very encouraging to read that those won't get fixed or maybe at best get the "ice box" label and will be closed (which apparently means that they won't get fixed either).

@Cernelius brought a good example when mentioning this thread:
https://forums.triplea-game.org/topic/989/aa-revised-minor-bug/

It turned out that the engine handles some procedures really wrong (not only in one game but in v2 OOB, v2 LHTR, most likely v1; v3 and v4 are affected, too), thus violating core principles of A&A games (far from being "minor"). We spent quite some time on sorting all of this out. Some resulting issues here on GitHub appear to be more than appropriate (pending due to this ice box-discussion).

However - if no one else cares all of this is pointless and a big waste of time.

So @DanVanAtta and @ron-murhammer I am asking you for some advice on how to deal with rules issues in the future. I love to contribute the way I can - but I would hate to do it only for some "ice-box".
Maybe it is about time to adjust my focuses.

I am aware that this is all about setting priorities. That is why I am not questioning your work but mine.

Thank you for your help.

[Cernelius]

I believe this is a good example of an issue that, in my opinion, makes no sense to close on any timeline basis, as I don't see how anything about it changes wether it was opened yesterday or 5 years ago (unless, of course, the issue is proved invalid or something relevant actually changed meanwhile, but this is another matter):
#1148
I understand that, by being labelled, it is somewhat still tracked (and I still believe a label like "Not Fixed" would be better telling), but what I'm saying is that I wouldn't see a reason for deprioritizing it anyways, if this process is substantially doing that.
Anyways, I think it can make sense to always close issues after 10 years, but I'd say no less than that.
Also, if this is the case, I tend to think that a process of keeping opening (basically cloning) new issues, in substitution of ice-boxed ones (or are we supposed to ask the issue to be de-ice-boxed, in case?), is due to be more confusing than just bumping/refreshing the same issue, eventually.

[DanVanAtta]

@panther2 I think largely it is about perspective, being realistic, and us identifying what is in effect not working and trying new tactics until we find something effective.

The math of more incoming issues than we are fixing, and us tackling the top of the queue rather than working existing items is a problem. The math implies that if we do not fix a new issue when it comes in then we will never get to it. We will simply always be in fire-fighting mode, fixing the new and easy, letting the arguably more important tasks to sink deeper and deeper in the backlog.

Ultimately the hope is that the team will become more efficient, we will drive the issue queue to zero, we will keep it there, and we'll have capacity to then work on the deep back-log (ie: ice-box).

For sure, ice-box label is not a good thing. But please respect that it is exactly the same if we close or leave those issues open, we are not getting those issues done either, it is us simply failing. The number of ice-box tickets is a reasonable metric to know how far backlogged we are, it is a number that when large means the project is off-track.

So why ice-box?

• highlights the process failure. We would not be talking about old game rule bugs identified if it weren't recently touched
• helps relieve pressure so we can better tackle soon to be ice-boxed issues and avoid the new ice-boxed issues
• keeps us a bit honest, when looking to ice-box something, there are some things that were simply closed becuase there was no follow-up to be had/done.
• starts a conversation about priority, in essence that perhaps we should be working on old items first instead of jumping on new things.
• introduces a concept of a SLA for getting things done. We are bad about letting things trail off, linger, or be half finished.

Where does that leave us? Good question. Surely just auto-renewing an issue is not going to be helpful, that gives priority to the loud and diligent over the important (which can often be the same thing, but not always). It is worthwhile though when a task has 5 items, 3 are done, to recreate a new ticket with just the last 2 items. We should keep in mind there is overhead to just read an issue, and that if someone has to go through 4 or 5 very long pages of back and forth responses, code investigations just to find out what to do, that nobody is likely going to do that. To solve this problem technical requirements can be created and captured, in essence we want a queue of those so that we can more easily spend time working rather than catching up on a long windy conversation thread (which can create confusion, "make it work" seems clear, but when an engineer can describe 5 different things that can all be called "working", it's a problem as not everyone will also think all 5 of those things are 'working').

I think at this point we should focus our discussion on how to prevent new issues from getting ice-boxed, what will our strategy be to acheive that? We also have a discussion here I think of what to do about the items that have been ice-boxed. We had about 20 or so before, we're at around 70 now. We need to discuss that both in general, and for specific categories like the game rule bugs.

[DanVanAtta]

IMO the label is probably better called "over-SLA", that captures the intent more. The closing of over SLA tickets is to highlight the reality. If you get beyond the emotional reaction, i think we we can see the outcome is the same and the hope is it'll create some urgency to fix our issue handling operations to be able to handle the incoming work load.

[Cernelius]

So, if an issue is being tracked in a closed ice-boxed issue, can I open it again in a new issue, if, for example, I believe this matter is very important and should be fixed, or for whatever other reasons, like if I want to fix it myself, or should that not be done, or rather done by replying to (bumping) the closed ice-boxed issue already there?

Is all this ice-box thing, comprising the timing, documented somewhere?

To me it seems this is just creating a twilight zone of things that are closed but not really.

[panther2]

Thank you, @DanVanAtta , for your open and self- (meaning project-) critical remarks.

So I will continue rule-(bug-)hunting, reducing my expectations of issues getting fixed, but for the fun of doing it, hoping that the situation will improve one day.

[DanVanAtta]

@Cernelius to finally rseponse:

So, if an issue is being tracked in a closed ice-boxed issue, can I open it again in a new issue

'issue' can be used to track 'Impacts' or 'incidents'. For example, any time you experience or encounter a bug, you could create a brand new bug report and leave it to the developers to identify the reports as the same bug. That would be a bit annoying and adding a message that the issue happened again would help.

The idea is that we want a histogram of how many times an event/issue/bug/problem happens. If there is a one-off that happened 4 months ago, then perhaps it is better we let it slide from active tracking so we can focus elsewhere.

At the same time the intent is to also then focus on fixing any new impacts as they come up, so if such a problem were to be updated or re-opened with a new occurrence, then we would try to focus on it.

Long answer long, open issues to track events or tasks that you would like completed. Events that do not repeat after a few months we will ice-box and tasks that we simply do not get to after months we will also ice-box.

In another way the ice-box is an admission of defeat, either we live with it taking someone to actively go back and fix those, or we find a more effective way to get those tasks done (given that the tasks were not done after so long, it's evidence that we have not succeeded and there is no real evidence that given more time we will fix those issues).

Is all this ice-box thing, comprising the timing, documented somewhere?

The exact cut-off and timing are TBD here. The suggestion time range is somewhere from 3 months to 6 months. In one way this is setting a SLA or expectation on the TripleA team of when they should try to get things fixed (or not, in which case we should look at why we are failing to get things done)

To me it seems this is just creating a twilight zone of things that are closed but not really.

Ideally it's an official 'twilight zone', if that makes sense. In essense any closed issue should truly be dead and nothing further left to do. Otherwise it should be re-opened. On the other hand, there is a set of things where we can't bring ourselves to complete the issue, and there are things left over but we are simply not getting to it. In that case 'ice-box' is the one label to say, "this is closed, but there's stuff left to do that we failed to get done"

Hopefully that all makes sense and answers questions/concerns @Cernelius. I'm happy to further discuss some scenarios.

[Cernelius]

I see that you have removed the "ice box - close and revisit later" label from all issues that are not resolved "Impact: Bad Game Rules" label, reopening them. Or at least I've seen it being done in the following cases:
#2367
#1867
#1645

I wonder if this is a change or exception in your process, in that such labelled issues would never be put in ice box? Is the ice box process actually fully documented anywhere, yet? Please, do so, if not.

In case, I agree to that and, if so, this is one of those cases too:
#1148

So, you may want to remove the "ice box" from this one too, and reopen it.

In any case, I believe that issue should receive the "Impact: Bad Game Rules" label too.

[DanVanAtta]

@Cernelius an issue that is open for months is effectively dead, whether open or closed. A long queue of open issues that are routinely skipped over causes additional inefficiencies. For example, simply reviewing the list of open issues then takes hours when that time could be spent instead solving issues. The idea is to try and get it so that the outflow (issues being resolved) is greater than inflow and such that the queue is near zero.

It looks like #1148 could be solved. That is a good example of such inefficiency. Understanding that issue takes considerable time, verifying the issue still exists also takes considerable, all to the effect of verifying it is a valid issue without actually resolving it.

If an issue is concise and still current, it can be re-opened with a comment stating it's been verified as current. It's also an option to concisely state a problem and reference an existing (and closed) issue that relates to the same problem but was not solved.

The goal is in part to have a short and sweet call to action that states what the problem is, is current, can be understood in minutes and then worked on right away. Reading pages of back and forth discussion defeats that and lends to issues that take significant time to understand and then detract from us solving current and latest problems. It tends to be the case that significant problems do come up multiple times and so ongoing issues tend to get surfaced again.

There are many open source projects that use a bot to automatically close issues that grow beyond a certain age. Given the sensitivities it's been a bit easier to just focus on incoming and ongoing projects. With that being the case, be aware that really old issues are effectively closed because they are stale and are not going to be worked on. If something has sat for 11 months, the chance of it being worked on in the 12th month, 13th month, or beyond basically decreases.

I personally think TripleA should have as accurate of game rules as it can. Picking and choosing battles those are great issues to resolve. I'd suggest if an issue can be verified as current, and the problem is cleanly stated, then re-opening it with a comment that it is still current is a good move. If the issue contains a lot of discussion, then re-summarizing the problem as concisely as possible and then referencing the previous problem is a good way to go. Note that there is a template for github issues that is meant to try and guide this to make it easier. For bugs and problems it tends to be best if the problem is stated, how to reproduce it, a "notice" line that states when the problem and what problem should be noticed, and an "expected" statement to say what should have happened. This done in as few words as possible is ideal.

I hope that answer/discussion offers some help

[DanVanAtta]

I elided your questions @Cernelius , my apologies.

The ice-box label is still there. It'd be great for a bot to automatically apply it and close it after a given time period. For now I'm not concerned by it as there are bigger priorities than to close older issues. I'm not looking at the tail of the queue anyways and am actively ignoring it for now. That is a problem, but one has to pick their battles.

Any bad game rules, as mentioned above, I recommend re-summarizing in a concise issue or simply re-opening if it is already concisely stated.

[DanVanAtta]

@Cernelius I'll be more concise finally:

• a long bug queue is of negative value
• lots of time and no fix is evidence there will be no fix
• the negative value prevents new issues from being fixed, compounding problems

Ice box is a way to stop that bleeding. If we get the issue queue down to zero, then we'd re-open all the ice-boxed issues and fix those. We've been holding pretty steady with the current queue, at some point it may make sense to apply the tourniquet again and close old issues so that we might actually get closer to zero.