Enabling gzip compression on Fastly

[ryanbetts]

So, as @trieloff was mentioning, turning on the default Fastly gzip compression interferes with ESI processes. I double checked and sure enough it does.

More info here:

• Docs: https://docs.fastly.com/guides/basic-configuration/enabling-automatic-gzipping
• Help thread: https://support.fastly.com/hc/en-us/community/posts/360040168131-Conditionally-Disabling-GZIP-in-Order-to-Serve-ESI

I spoke with Fastly's support and they recommend the following:

"So you can enable gzip in the UI of the service. for ESI you will want to add this snippet to vcl_fetch:

/* Turn on ESI for root GETs */
if (req.url == "/esi_example.html") {
    set beresp.do_esi = true; /* Do ESI processing */
    esi;
    set beresp.ttl = 5s; /* Sets the TTL on the HTML above */
}

That should set up the gzip and ESI functionality.

If you are using shielding you will want to add this to vcl_recv:

if (req.http.Fastly-FF) {
    set req.esi = false;
}

I hope that helps."

From speaking with @filmaj I understand that Helix is responsible for generating all the varnish files so I thought I'd kick this over to you. How best can I go about enabling gzip using the above changes?

[tripodsan]

at least we could ensure to disable ESI for static requests.

[tripodsan]

also, ensure to enable it in fastly:

[tripodsan]

Doing this, disabled the ESI processing for the static includes, and it breaks the site:

[trieloff]

We use ESI at the moment for following content types:

• text/html: because you might use ESI inside your HTL
• text/javascript: for automatic statification of ES6 module imports
• text/css: for automatic statification of CSS
  i.e. in almost every scenario where compression is useful (it's not useful for images).

If we build something like I outlined here: adobe/helix-pipeline#365 (comment) (move the ESI processing to Runtime) we would be able to have Gzip across the board.

[tripodsan]

If we build something like I outlined here: adobe/helix-pipeline#365 (comment) (move the ESI processing to Runtime) we would be able to have Gzip across the board.

or if we put the helix-specific service as an origin behind the real service, as mentioned here:
adobe/helix-pipeline#365 (comment) ;-)