Content expanders are not purged

[tiberiuichim]

plone.restapi/src/plone/restapi/cache/paths.py

Lines 10 to 15 in 1c4d79e

	CONTEXT_ENDPOINTS = [
	"/@actions",
	"/@breadcrumbs",
	"/@comments",
	"/@navigation",
	]

https://github.com/plone/volto/blob/95012fc5f8804982c6ea4e65e81efc2de163b204/packages/volto/src/config/index.js#L201

[davisagli]

@tiberiuichim I'm much more likely to work on an issue if you take a few extra minutes to explain the problem you're trying to solve, rather than just dumping some links to code.

It's not obvious to me what's wrong with the current configuration. The @types and @navroot endpoints are not included in the purging here, but they do not typically change when a content item is edited (one depends on the type, which never changes, and the other depends on data inherited from above).

Also, you're pointing to where they are loaded as expanders, which means they are part of the main content get, and purging the separate endpoints doesn't accomplish anything. Right?

[tiberiuichim]

@tiberiuichim I'm much more likely to work on an issue if you take a few extra minutes to explain the problem you're trying to solve, rather than just dumping some links to code.

You're right, sorry about this.

It's not obvious to me what's wrong with the current configuration. The @types and @navroot endpoints are not included in the purging here, but they do not typically change when a content item is edited (one depends on the type, which never changes, and the other depends on data inherited from above).

The current configuration is insufficient to purge the most typical use case, when Volto is used.

Also, you're pointing to where they are loaded as expanders, which means they are part of the main content get, and purging the separate endpoints doesn't accomplish anything. Right?

Right, the expanded version should be in context paths. I don't know if this is documented in Volto's "api expanders" chapter, because I presume that all websites that want to use varnish will need to monkey patch the context paths list.

[tiberiuichim]

Maybe I'm wrong, and the proper place to fix this would be in plone.volto.

[davisagli]

@tiberiuichim I hadn't really thought through the fact that the querystring is part of the cache key. I was thinking that purging the base path of the content item was enough, but now it seems obvious that it isn't. In our sites we usually have varnish configured to do no caching for logged in users and only cache for a minute or two for unauthenticated users, so that's probably why I haven't had to take a close look at this before.

It's annoying if we have to keep the querystring for purging in sync with the volto configuration. It would be much nicer if we had a way to purge all URLs with the context path, regardless of the querystring. Maybe we can do this with ykey but it'll take some work.

[tiberiuichim]

The mistral chatbot suggested that I could use a ban operation in Varnish to purge all URLs that match a prefix:

sub vcl_recv {
    if (req.method == "BAN") {
        if (req.http.X-Ban-Url) {
            ban("req.url ~ " + req.http.X-Ban-Url);
            return (synth(200, "Banned"));
        } else {
            return (synth(400, "Ban header missing"));
        }
    }
}

But that makes it dangerous: you edit the language root object and suddenly the whole cache is purged.

[tiberiuichim]

Here's the "patch" that works for my case:

from plone.restapi.cache import paths
paths.CONTEXT_ENDPOINTS = [
    "?expand=translations,breadcrumbs,navigation,actions&expand.navigation.depth=3",
    "/?expand=translations,breadcrumbs,navigation,actions&expand.navigation.depth=3",
]