Pluggable certificate storage (following on from #284)

[hmarr]

Duplicate of #330, which I accidentally opened from the wrong branch. Unfortunately there doesn't seem to be a way to change the source branch, so I've had to re-create the PR.

---

Following on from the work @btwotch did in #284, this builds on their PR to address @oec's comment (I encountered a similar issue).

Recap

To recap, the current implementation will regenerate TLS certificates every request. Generating certificates is computationally expensive, so this leads to performance issues when there are many requests to the same host.

@btwotch made it possible to add a custom certificate store to alleviate this issue. However, multiple concurrent requests to the same host would result in the certificate being generated multiple times in parallel.

Changes in this PR

This PR adapts the certificate store's interface. Rather than having Store() and Load() as separate functions, a single function - Fetch() is provided with the cache key (hostname) and a function that returns the certificate to cache, and it handles the storage and retrieval internally.

While a simple storage implementation may mimic the previous behaviour (leaving the described problem in place), an alternative implementation of the certificate storage might take a lock on the
hostname when generating the certificate, meaning only one certificate will be generated and other clients will wait until it is present in the store.

Benchmark

Using a simple in-memory certificate store that takes a lock for each hostname while generating the certificate, I got the following results from some simple load testing using vegeta.

Without the certificate store

Note the mean latency of 1.3s, p99 latency of 11.8s, and success rate of 11.2%.

Requests      [total, rate]            250, 50.17
Duration      [total, attack, wait]    17.624958347s, 4.983325s, 12.641633347s
Latencies     [mean, 50, 95, 99, max]  1.262955939s, 0s, 11.787397416s, 13.733232781s, 14.133813748s
Bytes In      [total, mean]            315096, 1260.38
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  11.20%
Status Codes  [code:count]             0:222  200:28

With the certificate store

Note the mean latency of 0.3s, p99 latency of 0.8ms, and success rate of 99.6. Quite an improvement!

Requests      [total, rate]            250, 50.13
Duration      [total, attack, wait]    5.257647755s, 4.987435s, 270.212755ms
Latencies     [mean, 50, 95, 99, max]  305.004131ms, 193.990013ms, 780.066772ms, 842.741389ms, 857.624176ms
Bytes In      [total, mean]            2796573, 11186.29
Bytes Out     [total, mean]            0, 0.00
Success       [ratio]                  99.60%
Status Codes  [code:count]             0:1  200:249

Credit to @btwotch for the foundation of this PR - their original commit was cherry-picked and used as the starting point.

If you think it'd be valuable, I'd also be happy to add a PR to include the in-memory storage implementation I described in this comment.

[elazarl]

Thanks, looks clean and reasonable.

I'd love to see another PR of an.in memory cache

[suxianbaozi]

how to use this...

[btwotch]

how to use this...

Haven't used it, but this test looks like a good example: 9718da4#diff-1ad863f10fb8c2c1ed4cbd0c2316a75ece3f5c78458506ca8c95b5f03533236aR800

[suxianbaozi]

how to use this...

Haven't used it, but this test looks like a good example: 9718da4#diff-1ad863f10fb8c2c1ed4cbd0c2316a75ece3f5c78458506ca8c95b5f03533236aR800

thanks a lot