-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC 100: Enhancing headless support in Wagtail core #100
base: main
Are you sure you want to change the base?
Conversation
I had to add extra code to get redirects to work in the API - would that be helpful to have included? |
@ahosgood I would assume so? Redirects is a contrib module so not as "core" as some of the more fundamental aspects of the CMS, but certainly something that many sites would consider core CMS functionality. |
I think it would be good to somehow include the pattern for JSON rendering using normal Page routing. Essentially each Page path can have a different request header and then return JSON instead of HTML. It's quite an intuitive approach and aligns with how other CMSs (e.g. Adobe Experience Manager) can provide their APIs. It's not going to make sense for every installation but worth reviewing as part of this RFC. In the abstract, this also could align with applications that may want to build out HTMX style applications, still returning HTML but providing a partial render of 'inner' HTML based on request headers. |
|
||
### Images | ||
|
||
- Add first-class support for responsive images and multi-format image generation in API responses. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we include focal points?
Some frontends will handle the images. Instead of Wagtail providing various renditions, Wagtail would only supply the original image and leave the resizing to the frontend application. In this scenario it makes sense to include the focal point information in the API response.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep. I see we support accessing the focal point in templates so feels like appropriate feature parity.
| Redirects | Full support | No support | | ||
|
||
## Gaps to address | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've numbered my feedback. I hope it helps to keep discussion structured. There is no specific order.
I'm not sure if my points are in line with the goal of this RFC. I just mention them as I've experienced them as points to address in my headless projects.
1. All page URLs endpoint
In a frontend built step, all pages need to be build. Wagtail could use an endpoint listing all page URLs (no other fields needed, without pagination). One call to built them all.
2. Invalidate/purge
Because the frontend acts as a cache, it needs to be invalidated/purged. In Wagtail, it is necessary to track which object is used on which page so that when content is mutated, the correct frontend components/pages are invalidated/purged.
3. OpenAPI specification and/or JSON schema
I really like to have an OpenAPI specification or JSON schema available. This makes frontend development easier. Could Wagtail provide a schema for all its endpoints? Via introspection of Models and streamfields?
FYI: I my headless projects I roll my own DRF endpoints and DRF Spectacular for this.
4. Point Wagtail admin live
button to the frontend
Override Page.get_url_parts
to point the Wagtail admin live
button to the frontend URL.
5. Override Page.serve to serve the API response
I'd like to override the Page serve method to serve the API response. This gives a one-to-one mapping with the frontend application and makes development much easier. I think many headless projects could benefit from this.
class APIMixin:
def serve(self, request, *args, **kwargs):
request.is_preview = getattr(request, "is_preview", False)
serializer = ...
# A DRF view gives us nice html or json output.
# The default render class should be listed first.
@api_view(("GET",))
@renderer_classes([JSONRenderer, BrowsableAPIRenderer])
def view(req):
return Response(serializer(self, context={"request": request}).data)
return view(request)
This is also a (partial) solution for the userbar. Example: it is easy to switch between https://nl.wagtail.space/ and https://cms.wagtail.space/
6. The way to expose Settings (and Snippets)
Site settings defining a menu, header, footer, contact details, or what not, should be exposed via REST API endpoints. Wagtail could define a best practice.
7. Translation of hardcoded strings
There are strings marked for translation in Django/Wagtail, and Django has the JavaScriptCatalog view. Many frontends come with their own translation solutions.
This might duplicate the way to lookup a specific translation, or lead to duplicated definitions.
If the decision is to use Django JavaScriptCatalog view, frontend translations might end-up in the backend.
8. Forms
Regular Django forms and Wagtail Form Builder forms.
Both need to re-implement rendering in the frontend.
Form validation has logic in backend (to clean/validate the submitted data) and frontend (for direct feedback).
Forms that validate input on-the-fly against the backend and use the error messages supplied by the backend have a single point of truth. To me, this works best.
9. RoutablePageMixins
The RoutablePageMixin mixin provides a convenient way for a page to respond on multiple sub-URLs with different views. The page is the starting point for fixed URL patterns.
I'm not sure how Wagtail API suggests to handle these. I guess point 5 is a possible solution?
| Userbar | Full support | No support | | ||
| Content checks | Full support | No support | | ||
| Redirects | Full support | No support | | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should/can we add the Wagtail accessibility checker in here? It's a really useful tool and while it works in the CMS, it might be a good idea to have the ability to add it to the frontend as well. Of course that also means there will be a requirement for validating logged in Wagtail users in the frontend and including some JS and CSS.
Maybe I'm overthinking it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe it's part of the "Content checks" and "Userbar" items. The accessibility checker is one of the content checks, and we currently piggyback on the userbar (the Wagtail button in the frontend) to implement it 🙂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I’ll update the RFC to make sure it’s more explicit.
Headless was on the agenda for Wagtail Space NL but this only involved incorporating the areweheadlessyet.org website into the Wagtail documentation. The resulting PR is here: wagtail/wagtail#12039 |
| [StreamField](https://areweheadlessyet.wagtail.org/streamfield) | Full support | TBC | | ||
| Userbar | Full support | No support | | ||
| Content checks | Full support | No support | | ||
| Redirects | Full support | No support | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With a headless site, the backend only needs to store the redirects. Wagtail currently does this. The frontend needs to be able to query the redirects from the headless cms and apply them. I would say that wagtail provides what a headless cms needs to provide for handling redirects at this juncture. I have a next.js front end that query's redirects from wagtail at build time and it works just fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not entirely sure it does 🤔
I had to use some custom code for finding a page by URL when that page had a redirect: https://github.com/nationalarchives/ds-wagtail/blob/develop/etna%2Fapi%2Furls.py#L162-L195
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't go through the trouble of querying redirects page by page. I just grab all redirects and renderer them as pages that redirect or update the redirect list on my CDN at build time. You might also consider graphql/grapple instead of interacting with the API directly. It's much easier to get all your data per page in a single query that way vs making oodles of rest calls everytime you generate a page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dopry ah I think you’re right, I didn’t realise there was already a built-in endpoint to fetch the redirects.
@ahosgood if I understand the code you’re sharing, it returns a page that’s associated to a redirect and matches the provided html_path
? That feels useful but more than I’d expect is needed to get to feature parity (which is what this RFC is about). In that scenario, it seems like the site’s front-end would return a 200 with the page data for the redirect path?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dopry I'm not sure I've understood fully.
Our headless CMS is a separate service to the frontend and they are built and deployed independently so building Wagtail doesn't interact with the frontend code.
Also, redirects can be added at any time by updating a Wagtail slug so I don't think computing all redirects at build time is a possibility for us.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@thibaudcolas Essentially, yes.
We had two choices:
- Return the page data even when requesting the "old" path (redirect just in the API)
- Return the data needed for the frontend to be able to redirect
In the frontend I have made a simple switch for both of these options because I wasn't sure which was going to be best for us. https://github.com/nationalarchives/ds-frontend/blob/main/app/wagtail/routes.py#L231-L239
As you can see, I also had some (now commented out) code to return redirects to external sites as well which is also possible in Wagtail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ahosgood We generate the redirects when we build the frontend. We're using next.js on Netlify. In our nextjs config we have a function for the next.js redirects key that queries the redirects from grapple. When it get's published on netlify, their Next.JS integration pushes the redirects to their CDN. We also render the redirects as pages just in case for some unforseen reason the redirect isn't in the CDN it's a static page. We are querying redirects through grapple/graphql. @thibaudcolas I'm am not sure that the native wagtail API provides a list of redirects. They are available in grapple/graphql interface.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The API can expose redirects (https://docs.wagtail.org/en/stable/reference/contrib/redirects.html#api) but it sounds like your use case is to query all the redirects and then build your service using them whereas ours is not built statically.
Our frontend just queries the Wagtail API directly so I can add a redirect to Wagtail at any point and the frontend can use it straight away.
The non-headless Wagtail responds to a URI whilst taking redirects into consideration. The API does not (without having to make multiple calls) so I would argue that it is needed for parity.
Another long-standing issue with the current API is that we cannot easily generate an OpenAPI specification I would say that we must have a documented or even out of the box way to generate specifications for our APIs if we want to truly say we have headless support. |
|
||
## Open Questions | ||
|
||
TODO |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some questions to consider.
- Would we want to consider a more open schema driven response structure ever? We kind of have this with the meta/data response objects but maybe go further with alignment to something like https://jsonapi.org/ or https://www.w3.org/TR/json-ld/ or similar.
- What kind of story to we want to tell when it comes to emerging community practice such as HTMX/Turbo style applications? Or do we consider this not really 'headless'?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
While API documentation is nice to have, I don't think it is essential for building a headless site. It shouldn't be something that blocks a consistent approach to headless previews and ensuring the API exposes all data (pages, snippets, redirects), and functions (crud, search, etc.) necessary for a headless client.
-
To me headless implies my CMS is no longer rendering/serving html. With AHAH (htmx, hotwire, et al) html is still being rendered at the server, so I wouldn't consider it headless. If you had a server responsible for just rendering that was sending hotwire/htmx to the client and not calling on wagtail directly, then that would be headless in my book. I feel like AHAH support should be it's own distinct thing, and it likely to be specific to the AHAH framework in play.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dopry The JSON and OpenAPI specifications are valuable not only for generating documentation but also for generating TypeScript types and even complete client applications. From my experience, using machine-generated types is a must. When the Wagtail data structures change, updating the frontend application is straightforward.
These schemas and types also enhance development with code editor autocompletion, which boosts productivity and reduces the likelihood of errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@allcaps it's a productivity boon, but not a requirement to implement a headless site. You can create a headless site without it. In my experience adding and removing properties from frontend data access layers is trivial. Figuring out what properties need to be added/removed/modified from the templates when that happens is the time sink, and schema documentation doesn't resolve that particular problem.
View as an HTML document
Requires further work. This is based on my experience building a copy of the bakerydemo with Next.js: bakerydemo-nextjs.