github-contribs is broken

[lourot]

github-contribs's tests turned red yesterday, because the unofficial interface to discover pull-requests seems to have changed.

So the last time we know for sure that the interface hadn't changed yet is October 10th 9:38AM UTC.

We need to:

• fix github-contribs and pin the latest version here in ghuser,
• re-crawl the recent days of all users,
• re-crawl completely all users created since October 10th 9:38AM UTC.

[brillout]

Seems like GitHub's UI doesn't rely on any fined grained pull request "html endpoint" anymore but instead fetches bigger HTML parts that include the PRs.

The good news is that we can get a list of PRs with graphql:

{
  user(login: AurelienLourot) {
     id,
     pullRequests(first:100) {
      edges {
        node {
          id,
          repository {
            id
            url
          }
        }
      }
    }
  }
}

I checked with my account and does seem to exhaustively list all PRs.

This endpoint is not enough though as we need repoS where I contributed without any PR but by having received write permissions from the owner.

I'll investigate more later.

[brillout]

We don't know how to properly set the header to be able to get these bigger HTML parts. Plus the structure of these bigger HTML parts are very likely to change.

So we will use the graphql api to get the list of all pull requests.

[lourot]

I got the big HTML parsing working. Not the best solution but this unblocks us for the short-term. Updated todo-list:

• fix github-contribs,
• pin the latest version here in ghuser, // done on October 13th at 5:04PM UTC
• re-crawl the recent days of all users,
• re-crawl completely all users created since October 10th 9:38AM UTC.