Stability bots spam

[annevk]

See, e.g., #5195, for a recent example of too many comments by w3c-bots.

[jugglinmike]

@bobholt I expect that this regression is due to w3c/prbuildbot#12, but only due to the timing between the merge and the incorrect behavior. I can't find any justification for this in the change set itself. I'm probably just being thick, though. I could use a second opinion; do you see what I'm missing?

The most likely cause is an incompatible change in the brittle relationship between the independent "title as identifier" functions, but they remain consistent:

• In WPT: https://github.com/w3c/web-platform-tests/blob/e9fc3cfd3559f900ca8dd71323f041fea2ae028e/check_stability.py#L587-L599
• In prbuildbot: https://github.com/w3c/prbuildbot/blob/61f6626f992b85674e15ed600967346c5f7689db/github.py#L20-L40

So it's not clear to me why the prbuilbot no longer recognizes its own comments, as specified here: https://github.com/w3c/prbuildbot/blob/61f6626f992b85674e15ed600967346c5f7689db/github.py#L125-L131

[bobholt]

@jugglinmike: Curious. I also don't see why that change would break that behavior, though I've seen it happening in another branch. I thought it was due to a change I made there, but it does coincide timing-wise with merging in your fix.

I'm having an issue getting my test server stood back up, but when it is, I'll take a peek at what's going on.

[bobholt]

Running a test build. I've added logging to try to see what it's expecting and what it's really getting. May not get a final answer today, but I'll dive in again first thing in the morning.

[bobholt]

So. This is unrelated to your change, @jugglinmike.

The prbuildbot is now looking for a title string matching # Ci_Stability.Sh Product=Chrome (unstable) #.

"That's odd," I said. It turns out the way this title string is originally determined is via partitioning environment variables received from Travis: https://github.com/w3c/prbuildbot/blob/master/travis.py#L68

And so this seemingly-innocuous change I noticed when rebasing my current working branch current master this afternoon seems to be the culprit: 8196377

It seems like that change by @gsnedders is true and good, and that the title partitioning logic in prbuildbot's travis.py needs to be revisited.

I can't look at it today, but I'll jump on it first thing in the morning if it's still outstanding.

[annevk]

See also #5208. wpt-pr-bot is also affected. Is that still under @tobie's control?

[tobie]

See also #5208. wpt-pr-bot is also affected. Is that still under @tobie's control?

Must be some API change. :-/

[tobie]

So for #5208 and wpt-pr-bot specifically, this looks like a rare racing condition: a new commit was pushed to the branch while the initial pull request was still being processed.

Guarding against this is something I systematically do nowadays but that I hadn't about back when I wrote wpt-pr-bot. It's probably not too difficult to fix, but I'd rather spend time elsewhere unless this becomes a recurrent problem.

[bobholt]

I've opened a PR to resolve the new-comments-on-every-push behavior of the prbuildbot here.

[jugglinmike]

Resolved via w3c/prbuildbot#13.