Server switch to the new codebase

[espadrine]

A switch of the image server at img.shields.io to the gh-badges codebase is scheduled somewhere during January 23.

This issue is meant to detail and argue with the finer details of this switch.

The new codebase uses a very similar design, also Heroku-based. The switch will only require to git push -f to the Heroku repository.

• Currently, the Heroku server uses a Redis plug-in. Was that necessary? I believe it no longer will be.
• Currently, http://img.shields.io/ redirects to http://badges.github.io/shields/, while the new codebase serves its own webpage. I would be fine with either serving a main page from Heroku or redirecting. However, the current page you are redirected to gives no information related to the exact use abilities and purpose of the new image server, and so is inadequate (see this for a definition of what I feel is needed in the index page.). The current index page used in b.adge.me could be used there, for its functional aspects, although a better design (and, obviously, a different logo!) would be welcome.
• Users of b.adge.me will be left with the current version , until they will be redirected. For that purpose, I will set up a separate branch called badgeme.
• Is HTTPS an issue?

[nathany]

HTTPS is quite important. But we can add it shortly after, yes?

[espadrine]

@nathany I never set up anything special for b.adge.me, and HTTPS seems to work (Heroku magic?), so I assume that it probably will work the same, but I remember reading an issue about DNS-wise HTTPS issues.

[chadwhitacre]

Currently, the Heroku server uses a Redis plug-in. Was that necessary?

Yes, it is used for caching. We cache two things:

1. HTTP calls to resolve travis-ci | foo to build | passing
2. PNG generation

I believe it no longer will be.

Okay. Are we giving up caching altogether? Or are we doing that in-memory now? Or ... ?

[espadrine]

@whit537 Caching happens in-memory. I do cache PNG generation; however, I don't cache HTTP calls to vendors, because they're meant to be live. However, that is both an easy fix and something I don't have a strong opinion for.

How long did you keep that cache? How big did that cache end up being?

[chadwhitacre]

Currently, http://img.shields.io/ redirects to http://badges.github.io/shields/, while the new codebase serves its own webpage.

We decided at #52 (comment) to keep the API server separate from the marketing/admin site.

• http://img.shields.io/ <- image server
• http://shields.io/ < - marketing/admin pages

I think it'd be fine for now to swap out the existing homepage with the existing gh-badges homepage.

[chadwhitacre]

How long did you keep that cache?

From the current implementation:

The first user to request a button always has to wait for the transform.

Subsequent users never have to wait for the transform. Rather than make them wait, we proceed with the last transform we did, no matter how stale it is. Before we do, however, we'll kick off a new transform in a thread, so that the next user after the thread completes will get a fresh transform.

This means that the average staleness of the buttons we serve will depend on the frequency with which they're requested. For active projects this will approach the TRANSFORM_TIMEOUT; for inactive projects, infinity. This is appropriate: stale buttons for stale projects, fresh buttons for fresh projects.

The TRANSFORM_TIMEOUT is currently set to 60 seconds, so for active projects the cache approaches 60 seconds.

How big did that cache end up being?

Well, we're on the free plan, which is 5MB, and we're using 4.78 MB at the moment.

[chadwhitacre]

Here's a comparison of img.shields.io-old and gh-badges for the Gittip vendor (the only one in common with a heavy HTTP call):

root@loader:~/wrk# ./wrk http://b.adge.me/gittip/Gittip.png
Running 10s test @ http://b.adge.me/gittip/Gittip.png
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.47s   671.44ms   5.19s    50.00%
    Req/Sec     0.75      0.50     1.00     75.00%
  23 requests in 10.00s, 46.79KB read
  Socket errors: connect 0, read 0, write 0, timeout 19
Requests/sec:      2.30
Transfer/sec:      4.68KB
root@loader:~/wrk# ./wrk http://img.shields.io/gittip/Gittip.png
Running 10s test @ http://img.shields.io/gittip/Gittip.png
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    94.69ms   73.96ms 443.28ms   85.25%
    Req/Sec    53.87     12.70    88.00     68.85%
  1103 requests in 10.00s, 2.53MB read
Requests/sec:    110.28
Transfer/sec:    259.51KB
root@loader:~/wrk#

[chadwhitacre]

Worked through merging the two existing APIs (#100) in IRC. Plan is for @espadrine to wrap up that work and publish gh-badges on http://img.shields.io/. Then we'll replace the http://shields.io/ page with the homepage from gh-badges, and then bring back the redirect from img.shields.io to shields.io. Sounds like @espadrine is planning to bring back vendor caching as well.

[espadrine]

Done.

[chadwhitacre]

✨

[chadwhitacre]

root@loader:~/wrk# ./wrk http://img.shields.io/gittip/Gittip.png
Running 10s test @ http://img.shields.io/gittip/Gittip.png
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    97.27ms   67.56ms 292.97ms   77.63%
    Req/Sec    56.64     18.94    99.00     72.37%
  1130 requests in 10.00s, 2.24MB read
Requests/sec:    112.99
Transfer/sec:    229.12KB
root@loader:~/wrk#

[chadwhitacre]

👍

[chadwhitacre]

Reticketed homepage dance as #110.