Add ETag support to the Storage implementation

[everettraven]

As a follow up to #113, we should add ETag support to the HTTP server. This will allow clients to poll for changes using the ETag and only fetch all the catalog contents when something has changed. This would likely be middleware that we wrap our HTTP handler with.

Acceptance Criteria

• The Storage implementation is updated to use middleware to enable ETag support on the HTTP server. This can be custom middleware or middleware provided by an external package

[joelanford]

This might require some thinking:

• Do we actually need an Etag (is last-modified header sufficient?)
• Do we compute it on the fly? What would that do to response times?
• Do we compute it once when we generate the all.json file? If so, where do we store the etag value, and how do we avoid races between reading the etag and reading the file? Maybe a RWMutex would need to come into play?

[everettraven]

I played around with this a bit and I think as long as we continue to use a fileserver the Last-Modified header should be sufficient. If we switch to having a custom server implementation that processes the information differently than serving a file, using the ETag header would likely be needed for client side caching.

[m1kola]

Serving files is a perfect use case for Last-Modified/If-Modified-Since.

If we switch to having a custom server implementation that processes the information differently than serving a file, using the ETag header would likely be needed for client side caching.

I think even if we decide to implement server side filtering of all.json we can still rely on file modification date. As long as underlying file is the same - request with the same parameters should return the same result.

ETag would require doing the filtering on the server side to calculate some sort of hash. If we rely on file modification date we can avoid doing this even for endpoints that do some sort of processing of all.json. Edit: Thinking a bit more about this. I think we can do a hash of file modification date and use it as an ETag, but this makes little sense given that we can just use Last-Modified/If-Modified-Since.

[everettraven]

Closing this issue as obsolete as it has been discussed and consensus reached that the combination of using the pairing of Last-Modified / If-Modified-Since headers will be sufficient for implementing client side caching logic.

For more context, see the discussion here: https://kubernetes.slack.com/archives/C0181L6JYQ2/p1694786005446749