RenderCache Second Alpha Fixes a Crawler Redirect Loop Behind HTTPS Edges

RenderCache’s first alpha had a bug that only shows up in a real deployment. RenderCache sits behind nginx or a Cloudflare Worker that handles TLS. On a cold page it passes the crawler’s request on to the app, and in alpha 1 it rebuilt the forwarding headers from its own plain-HTTP connection on the way. So the app was told the crawler came in over HTTP. Apps that force HTTPS redirected it, the crawler came back the same way, and went round again, on exactly the pages not cached yet.

The second alpha, 0.1.0-alpha.2, went out on 18 October and keeps the visitor’s address, scheme and host when the request comes from a trusted edge: an address in the new trusted_proxies setting, loopback by default, or a request carrying the edge token. Anyone else still can’t set them.

The review that found it turned up a few smaller things too. Search Console’s URL Inspection and Rich Results Test now get the cached page, because Google-InspectionTool is in the default crawler list. Pages outside the sitemap no longer use up their 256 slots until a restart; the least recently requested idle page gives its slot away and keeps its saved copy. Tracking parameters such as utm_source and gclid share one snapshot with the plain URL, and the fixed /api bypass became a list you can set.

It’s still an evaluation release for Linux on x86-64. CI ran the tests with real Chrome, the container and nginx routing before the release was published, and old config files keep working. The code is on GitHub under Apache 2.0.