New tutorial: Customizable serverless Golang proxy for GCS

[domZippilli]

This tutorial (and accompanying code) offers a customizable, containerized HTTP proxy for GCS. It shows how to:

1. Serve secure static content with it, such as within a VPC or only from a certain CIDR block (you cannot do this with vanilla GCS)
2. Modify the content to be lowercase
3. Modify the content to be gzip-encoded
4. Modify the content to be dynamically translated from the Translate API

Moreover, the point is to equip the reader with ways to have a serverless proxy in front of object storage to satisfy lots of use cases they might think of.

[lukaszkostrzewa]

Thanks, @domZippilli, for the great tutorial!

I am wondering what are the advantages of using Golang proxy you wrote over using e.g. NGINX with mounted GCS bucket with FUSE? If customization / content transformation is not required, are there any other concerns that should be taken into account? Performance perhaps?

Thanks in advance for your comments on this 😃

[domZippilli]

Good question. Ostensibly, you could host a custom web server out of GCS using a FUSE with httpd, or any other web server. Caveats in the gcsfuse repo about "beta-quality" (still there today) notwithstanding... I think that would be fine, depending on the requirements, preferences of the engineering team, and most importantly, the results of an integration test.

On the last point, I would want to test such a configuration carefully for performance and cost concerns. I think this is generally true of any integration of multiple software and services, and the more there are, the more there is to test. For example, if you're scaling to zero and need to cold start, will that work well with a more robust proxy and a FUSE? Or more importantly, does the proxy/web server use the filesystem with certain assumptions that don't translate well to object storage? Look at this issue where a more robust HTTP library in Go required rewind seeks. I'm sure the FUSE driver (gcsfuse) supports seek, but will it work exactly the way Nginx wants it to, efficiently, with no/minimal side effects?

I think the advantage of using Nginx or something like that is that it's "battle-tested" and its interactions with web browsers and other clients are very well-understood. So, if the integration tests are thorough and they pass on criteria of not just no errors, but good performance and cost profile... go for it!