Skip to content
This repository has been archived by the owner on Feb 25, 2022. It is now read-only.

Massive refactor of helix.vcl #7

Merged
merged 3 commits into from
Mar 4, 2019

Conversation

drwilco
Copy link
Contributor

@drwilco drwilco commented Jan 31, 2019

This started out as a refactor to fix the handling of redirects from runtime for static files that are larger than the limit. It then quickly evolved to reworking how restarts are done, which is documented in the large comment at the top.

Other things were fixed along the way, mostly related, so not split out into separate changes:

  • Deduplicating code in vcl_miss and vcl_pass
  • Deduplicating X-URL and X-Orig-URL
  • Rename X-Static to X-Request-Type
  • Improve handling of proxy strains
  • Use less resources (headers/variables)
  • Be consistent about where to set the backend
  • Split Image out into a separate request type
  • Leak less data in headers to origins

unset req.http.Cookie;

# Sanitize user input. `urlencode` leaves alphanumeric and `-._~`
set req.http.X-Strain = regsuball(urlencode(req.http.Foo), {"%.."}, "_");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

req.http.foo = req.http.X-Strain?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drwilco I think @ejthurgo is correct. should we rename this?

/**
* Set the `X-From-Edge` header that the above sub checks.
*
* Should be called from the top of `vcl_recv`/`vcl_miss`
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function accesses bereq. Should this be vcl_miss/vcl_pass?

if (!bereq.http.X-From-Edge) {
declare local var.data STRING;
set var.data = strftime({"%s"}, now) + "," + server.datacenter;
set bereq.http.Secure-From-Edge =
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be X-From-Edge

*
* Should be called from the top of `vcl_recv`/`vcl_miss`
*/
sub hlx_set_from_edge {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, I'm not sure this is ever called?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drwilco can we delete this function?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not superfluous, we just need to reference it.
I believe it should be called here:

set bereq.url = req.http.X-Orig-Url;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Adding.

@trieloff
Copy link
Contributor

@drwilco thank you for the PR. I'll wait for @ejthurgo's points to be addressed before I merge.

Copy link
Contributor

@tripodsan tripodsan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drwilco do you think you have some time to address the changes? otherwise I will just follow @ejthurgo advice and then merge in this PR.

*
* Should be called from the top of `vcl_recv`/`vcl_miss`
*/
sub hlx_set_from_edge {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drwilco can we delete this function?

unset req.http.Cookie;

# Sanitize user input. `urlencode` leaves alphanumeric and `-._~`
set req.http.X-Strain = regsuball(urlencode(req.http.Foo), {"%.."}, "_");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@drwilco I think @ejthurgo is correct. should we rename this?

@trieloff
Copy link
Contributor

@tripodsan I spoke with @ejthurgo last week and he said that he wasn't entirely done with his review.

# object is in the cache.
#
# So `req.url` is what should be left after filtering query string parameters,
# doing normalizations, etc. Then `req.http.X-Backend-URL` is a transformation of

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that backend transformation could result in any normalization. and therefore cache efficiency?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't. All normalization should be done on req.url and req.http.Host.

set req.http.X-Trace = req.http.X-Trace + "; vcl_miss";
unset bereq.http.X-Orig-Url;
/**
* Do all BackEnd REQuest changes. One sub to be called from both vcl_miss and

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BackEnd REQuest
backend request

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did that capitalization to explain where bereq comes from :)

set bereq.http.host = "adobeioruntime.net";
} elsif (req.backend == F_GitHub) {
set bereq.http.host = "raw.githubusercontent.com";
set bereq.http.Host = req.http.X-Orig-Host;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we stash the original host in req.http.X-Orig-Host, however, we never actually change req.http.host
Is this risk management against future changes to host in vcl_recv, or just an oversight/relic from an older version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk management vs future changes

@adobe-bot
Copy link
Contributor

🎉 This PR is included in version 1.0.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants