Collaborative Editing

[dmonad]

Hello Gutenberg community 👋,

In the coming months, I want to help to make the Gutenberg editor more collaborative. Automattic is kind enough to sponsor my work.

A little about me: I'm Kevin, the author of Yjs - a popular framework for building collaborative applications. I'm also the author of several collab extensions for other popular editors like ProseMirror, Quill, and CodeMirror.

I learned from experience that collaborative editing is quite an invasive feature that touches many aspects of the application.

• Undo/Redo: before we didn't have to consider remote changes. Yjs offers an UndoManager that we could integrate.
• Revisions: A single revision can be produced by multiple users in a single session.
• Changing content: We always need to assume that another user is currently working on a document. Hence we should only change a document through Yjs.

Ideally, I'd like to make Yjs a first-class citizen of the Gutenberg editor, which custom blocks can use to offer their own collaborative experience (e.g., a drawing app using tldraw).

I'm currently meeting with people in this space to learn more about Gutenberg.

I opened this discussion for three reasons:

1. I want to offer my input if you have anything to discuss related to this technology.
2. Track my works.
3. Offer my thoughts about the current state of collaborative editing.

WebRTC

The current collaborative implementation uses WebRTC (using y-webrtc) to connect peers directly with each other. The idea is that the backend doesn't need to understand "real-time collaborative updates", the clients can simply sync directly with each other.

I advise against this because..

1. It is unclear which client should initialize content. This leads to content duplication.
2. It is unclear which client should "save" content to the backend. This leads to loss of data.
3. Auth*: It is unclear how we can ensure that a connected client has access to a certain document (read-only vs read-write).
4. Not all clients can connect to each other directly. Some need a "TURN Server" which the user would need to set up (there is no PHP implementation).

Ideally, the WordPress instance would be responsible for distributing document updates (every keystroke is synced to the backend, like in Google Docs). However, the backend is currently limited to simple REST calls, and it doesn't understand Yjs documents.

I got a lot of pushback when I initially suggested that we should simply put a (nodejs) server in-between WordPress and the client for the collaborative experience. We want to offer collaborative editing to as many users as possible.

Milestone 1 - collaborative editing using the existing backend.

My first milestone is to enhance the existing backend to understand that there can be conflicts that will be resolved eventually.

We could teach the backend that every document is associated to a list of Yjs documents.

When a user "saves" a document (through autosave), it will send the Yjs document alongside the JSON representation to the backend. In case of a conflict, we might end up with multiple "conflicting" Yjs documents on the backend. The client will be able to merge these Yjs documents.

The server will publish changes to the document through the existing autosave functionality. This progressive enhance will give all users a stable (albeit slow) collaborative experience. We may still put a websocket server in-between WordPress and the client to get a faster, more scalable experience that doesn't require autosave.

Protocol

             ┌────────┐        ┌─────────┐                         ┌────────┐
             │Client A│        │WordPress│                         │Client B│
             └────┬───┘        └────┬────┘                         └──┬─────┘
                  │                 │ Yjs state:                      │
                  │                 │ v1: "helo"                      │
                  │                 │                                 │
                  │                 ├──┐(send initial state)          │
                  │              ┌──│  └─────────────────────────────►│
                  │◄─────────────┘  │                                 │
                  │                 │                                 │
                  │                 │            (save, remove v1) ┌──┤ Insert " world" at position 4
    Insert "l" at ├┐(save, rem. v1) │◄─────────────────────────────┘  │
    position 3    │└───────────────►│                                 │
                  │                 │ State:                          │
                  │                 │ v2_A: "hello"                   │
                  │                 │ v2_B: "helo world"              │
                  │    (update) ┌───┤                                 │
                  │◄────────────┘   ├──┐ (update)                     │
                  │                 │  └─────────────────────────────►│
                  │                 │                                 │ Merged state: "hello world"
                  │                 │                                 │
                  │                 │     (save, remove v2_A,v2_B) ┌──┤ Insert "!" at position 11
                  │                 │◄─────────────────────────────┘  │
                  │                 │ State:                          │
                  │                 │ v3_B: "hello world!"            │
                  │                 │                                 │
                  ▼                 ▼                                 ▼

[youknowriad]

Thanks for jumping in this in the project. I really appreciate the help here.

So first let me clarify (and introduce some jargon) to make sure that all the WP contributors are aligned and understand here. When you say save document, for WordPress that corresponds to the different "entities" that can be saved through the REST API using the editor. Here's a non-exhaustive list of document "types" or entities:

• Site object (settings)
• Post types (post, page, custom post types)
• Internal post types (template, template part, style variation, pattern)
• Taxonomies (categories, tags, ...)
• Media
• Users

Most of these entities are defined here in the frontend

gutenberg/packages/core-data/src/entities.js

Lines 23 to 232 in 3a310c4

	export const rootEntitiesConfig = [
	{
	label: __( 'Base' ),
	kind: 'root',
	name: '__unstableBase',
	baseURL: '/',
	baseURLParams: {
	_fields: [
	'description',
	'gmt_offset',
	'home',
	'name',
	'site_icon',
	'site_icon_url',
	'site_logo',
	'timezone_string',
	'url',
	].join( ',' ),
	},
	// The entity doesn't support selecting multiple records.
	// The property is maintained for backward compatibility.
	plural: '__unstableBases',
	syncConfig: {
	fetch: async () => {
	return apiFetch( { path: '/' } );
	},
	applyChangesToDoc: ( doc, changes ) => {
	const document = doc.getMap( 'document' );
	Object.entries( changes ).forEach( ( [ key, value ] ) => {
	if ( document.get( key ) !== value ) {
	document.set( key, value );
	}
	} );
	},
	fromCRDTDoc: ( doc ) => {
	return doc.getMap( 'document' ).toJSON();
	},
	},
	syncObjectType: 'root/base',
	getSyncObjectId: () => 'index',
	},
	{
	label: __( 'Post Type' ),
	name: 'postType',
	kind: 'root',
	key: 'slug',
	baseURL: '/wp/v2/types',
	baseURLParams: { context: 'edit' },
	plural: 'postTypes',
	syncConfig: {
	fetch: async ( id ) => {
	return apiFetch( {
	path: `/wp/v2/types/${ id }?context=edit`,
	} );
	},
	applyChangesToDoc: ( doc, changes ) => {
	const document = doc.getMap( 'document' );
	Object.entries( changes ).forEach( ( [ key, value ] ) => {
	if ( document.get( key ) !== value ) {
	document.set( key, value );
	}
	} );
	},
	fromCRDTDoc: ( doc ) => {
	return doc.getMap( 'document' ).toJSON();
	},
	},
	syncObjectType: 'root/postType',
	getSyncObjectId: ( id ) => id,
	},
	{
	name: 'media',
	kind: 'root',
	baseURL: '/wp/v2/media',
	baseURLParams: { context: 'edit' },
	plural: 'mediaItems',
	label: __( 'Media' ),
	rawAttributes: [ 'caption', 'title', 'description' ],
	supportsPagination: true,
	},
	{
	name: 'taxonomy',
	kind: 'root',
	key: 'slug',
	baseURL: '/wp/v2/taxonomies',
	baseURLParams: { context: 'edit' },
	plural: 'taxonomies',
	label: __( 'Taxonomy' ),
	},
	{
	name: 'sidebar',
	kind: 'root',
	baseURL: '/wp/v2/sidebars',
	baseURLParams: { context: 'edit' },
	plural: 'sidebars',
	transientEdits: { blocks: true },
	label: __( 'Widget areas' ),
	},
	{
	name: 'widget',
	kind: 'root',
	baseURL: '/wp/v2/widgets',
	baseURLParams: { context: 'edit' },
	plural: 'widgets',
	transientEdits: { blocks: true },
	label: __( 'Widgets' ),
	},
	{
	name: 'widgetType',
	kind: 'root',
	baseURL: '/wp/v2/widget-types',
	baseURLParams: { context: 'edit' },
	plural: 'widgetTypes',
	label: __( 'Widget types' ),
	},
	{
	label: __( 'User' ),
	name: 'user',
	kind: 'root',
	baseURL: '/wp/v2/users',
	baseURLParams: { context: 'edit' },
	plural: 'users',
	},
	{
	name: 'comment',
	kind: 'root',
	baseURL: '/wp/v2/comments',
	baseURLParams: { context: 'edit' },
	plural: 'comments',
	label: __( 'Comment' ),
	},
	{
	name: 'menu',
	kind: 'root',
	baseURL: '/wp/v2/menus',
	baseURLParams: { context: 'edit' },
	plural: 'menus',
	label: __( 'Menu' ),
	},
	{
	name: 'menuItem',
	kind: 'root',
	baseURL: '/wp/v2/menu-items',
	baseURLParams: { context: 'edit' },
	plural: 'menuItems',
	label: __( 'Menu Item' ),
	rawAttributes: [ 'title' ],
	},
	{
	name: 'menuLocation',
	kind: 'root',
	baseURL: '/wp/v2/menu-locations',
	baseURLParams: { context: 'edit' },
	plural: 'menuLocations',
	label: __( 'Menu Location' ),
	key: 'name',
	},
	{
	label: __( 'Global Styles' ),
	name: 'globalStyles',
	kind: 'root',
	baseURL: '/wp/v2/global-styles',
	baseURLParams: { context: 'edit' },
	plural: 'globalStylesVariations', // Should be different from name.
	getTitle: ( record ) => record?.title?.rendered || record?.title,
	getRevisionsUrl: ( parentId, revisionId ) =>
	`/wp/v2/global-styles/${ parentId }/revisions${
	revisionId ? '/' + revisionId : ''
	}`,
	supportsPagination: true,
	},
	{
	label: __( 'Themes' ),
	name: 'theme',
	kind: 'root',
	baseURL: '/wp/v2/themes',
	baseURLParams: { context: 'edit' },
	plural: 'themes',
	key: 'stylesheet',
	},
	{
	label: __( 'Plugins' ),
	name: 'plugin',
	kind: 'root',
	baseURL: '/wp/v2/plugins',
	baseURLParams: { context: 'edit' },
	plural: 'plugins',
	key: 'plugin',
	},
	{
	label: __( 'Status' ),
	name: 'status',
	kind: 'root',
	baseURL: '/wp/v2/statuses',
	baseURLParams: { context: 'edit' },
	plural: 'statuses',
	key: 'slug',
	},
	];
	export const additionalEntityConfigLoaders = [
	{ kind: 'postType', loadEntities: loadPostTypeEntities },
	{ kind: 'taxonomy', loadEntities: loadTaxonomyEntities },
	{
	kind: 'root',
	name: 'site',
	plural: 'sites',
	loadEntities: loadSiteEntity,
	},
	];

Please correct me if I'm wrong but my understanding here is that for each one of these entities, the proposal above requires

1- updating the database, to attach a list of YJS document to it.
2- updating the REST API (save and autosave endpoints) to support the protocol suggested above: (forwarding YJS documents to clients for merge)
3- Update the @wordpress/core-data package and sync engine (the client side abstraction that talks to the server to do fetch and save entities) to support the merging algorithm and to also send the YJS document attached with the entity for each save request.

Some of the questions that needs answering are:

• What happens when some third-party code tries to perform a save to a "collaborative" entity without providing the YJS document (Non-collaborative mode, existing API need to say to support backward compatibility). I'm guessing that we have two options here: either reset the whole YJS document to an empty state or reject the save request? Obviously, our goal would be to update wp-admin to prevent this case for Core code but It's definitely something to consider regardless.
• One editor session in WordPress can touch, edit and save multiple documents/entities at the same time. In order to introduce the collaborative editing support iteratively, I'm wondering if we can introduce it entity per entity. For instance, we start with "posts" add the YJS documents to the server and update the REST API... but leave the other entities untouched. Can we ensure that the editor UI is updated to be able to save "posts" using the collaborative engine but just regular behavior for the rest of the entities?

I would love as much feedback and discussion here as possible in order for us to make the right decisions as it seems that it's going to be an invasive changes regardless of the approach. cc @WordPress/gutenberg-core

It seems that the work here involves a lot of backend work. It would be ideal if @dmonad can pair with an experienced WP contributor to help with all the service side changes. Anyone available to help there?

[adamziel]

I wonder if there's any overlap with the general WordPress-to-WordPress sync.

On the first sight, the goals of the two projects are different:

• Real time Gutenberg collaboration is about quickly exchanging atomic, potentially conflicting state updates between clients. Undo and versioning matters.
• The Site Transfer Protocol is about streaming sometimes large chunks of WordPress data between servers. Re-entrancy and staying synchronized over months matters. Undo and versioning doesn't matter as much.

At the same time, both projects involve primitives for synchronizing WordPress data. With WordPress Playground we could reuse PHP code at the client side. We can't run JavaScript on the server yet, but there are some WASM-in-PHP explorations that may enable it one day. With a portable enough code, we could also synchronize non-WordPress native apps using headless WP as a CMS backend.

Maybe there's a common core here, or maybe the goals are too different. I'd like to at least discuss and explore the possibility.

cc @dmsnell @dawidurbanski

[dmonad]

@youknowriad

Thanks for your insights!

What happens when some third-party code tries to perform a save to a "collaborative" entity without providing the YJS document (Non-collaborative mode, existing API need to say to support backward compatibility). I'm guessing that we have two options here: either reset the whole YJS document to an empty state or reject the save request? Obviously, our goal would be to update wp-admin to prevent this case for Core code but It's definitely something to consider regardless.

This is where the "collaborative mode" conflicts with the "single-user mode", where every peer assumes it has the latest content.

To me, the "collaborative mode" reflects the real-world much better. Peers & plugins should not assume that they have the latest content.

If we assume that peers are currently working on a "collaborative entity", then it might not make sense to clear the Yjs state. For now, I would probably recommend rejecting changes to collaborative documents. Alternatively, we could reset the entity to a non-collaborative entity.

But I dislike the idea of having two different "modes" for all entities. The user would have to enable collaborative editing manually, when a reset happens.

One editor session in WordPress can touch, edit and save multiple documents/entities at the same time. In order to introduce the collaborative editing support iteratively, I'm wondering if we can introduce it entity per entity. For instance, we start with "posts" add the YJS documents to the server and update the REST API... but leave the other entities untouched. Can we ensure that the editor UI is updated to be able to save "posts" using the collaborative engine but just regular behavior for the rest of the entities?

I think that is a great idea! We should definitely explore that.

@adamziel
Looking forward maybe 5 years from now: All entity types in WordPress are associated with a Yjs document. We created a PHP port of Yjs, as we already did for several other languages, and we can easily merge & update Yjs docs in Wordpress without calling JavaScript. It would be very interesting to explore whether we could sync WordPress instances incrementally without downtime through the sync engine while resolving conflicts (I assume that is a problem). Updates from the old instance would eventually be synced to the new instance without merge conflicts.

But I want to limit your expectations a bit. WordPress is a huge project with a lot tools around it. I believe the key to success is incremental improvements. We most definitely won't make everything collaborative, because - that's my understanding - we want to stay compatible with the current API. For now, we are just going to make the editor collaborative.

It might not make sense to make more things collaborative. AFAIK we can't run WASM / Native Yrs yet on all WordPress instances. This would give us more options to make WordPress more collaborative.

[adamziel]

I wonder how much overlap is there between the live sync and RTC, though. Both will likely have to reconcile offline changes from last week with the current post. I wonder what other use-cases overlap.

To expand on live sync. If real-time collaboration is Google Docs, then live site sync is git. It's less about synchronizing block trees, and more about database rows, streaming large files, and retrying. In RTC, we don't want to overwrite your changes with mine. In live sync, a pinch of "the latest change wins" might be the right choice. See the vector clock idea by @dmsnell and the writeup in the next comment.

Looking forward maybe 5 years from now: All entity types in WordPress are associated with a Yjs document.

Would plugins be supported out of the box? Or would adoption require a non-trivial amount of effort from the plugin authors? Because if it's the latter, most WordPress sites may not be able to use it – pretty much everyone has 10 or 20 different plugins installed.

[mtias]

Welcome, @dmonad! Excited to see where this goes.

Undo/redo: It'd be great to improve this regardless of real-time collaboration. We have some long-standing issues about undo management, its connection to block attributes and third-party blocks — #8119.

Revisions: we have a whole roadmap for post revisions that would be great to align architecturally — #61161

Making Yjs a first-class citizen in the editor makes a lot of sense. I'm a lot more worried about server modifications. We've gone to great lengths to introduce blocks in WordPress without modifying the preexisting server representation of data. There are backwards and forwards-compatibility reasons for that, as well as ecosystem integration, long-term survival of the content, data sync issues with duplication, db size concerns, etc. There's a myriad ways in which WP content is manipulated server side by hooks and filters, sometimes overwriting source data in the process.

Our own explorations have shown that pure client indeed doesn't seem very feasible, even with the server orchestrating webrtc handshakes, but I'm curious how we can reduce the server responsibility to the minimum necessary to avoid bloating the representations we need to deal with and to ensure the foundation can be widely deployed. It stands to reason that a websockets configuration, for servers capable of offering it, would provide the best concurrent experience on top of it.

Also for context, some of the "sync engine" work traced here — #52593

[tyxla]

Thank you for working on this one @dmonad!

I'm a bit concerned about all the potentially breaking changes that need to happen in order to make this work.

Supporting multiple authors per revision (and therefore per entity) may be a breaking change for WordPress, and we'll need to devise a good plan to approach it. Will it be possible for the entities to continue having 1 author but multiple collaborators? In other words, it may make sense to introduce the concept of collaborator as a separate abstraction from the entity author. Would that make sense in the Yjs world? Also, what happens with the linear revision history once we have multiple collaborators?

I'm also concerned about the backend implementation. The Heartbeat API and post locking were introduced in WordPress 3.6 because many data loss problems were caused by multiple users editing the same post simultaneously. This has historically been a limitation due to the technology used (PHP and flavors of SQL) that don't support concurrent editing. Autosaving is also reliant on these limitations, FWIW. Considering that we may still need to have a lightweight version of concurrent editing (potentially without a dedicated WebRTC server), I'm concerned about how it will work and what limitations from the user's perspective we might have to impose to account for those pre-existing limitations, and if those won't hinder the overall collaboration experience. Do you have any experience with such limitations before? Is it possible that the most basic Apache shared server with just PHP / MySQL will be limited and unable to use any collaborative features?

My biggest concern is that from this proposal, it feels like Yjs will become the centerpiece, and WordPress will have to bend its database, structure, and flow around the Yjs as a single source of truth. Backward compatibility is a significant limitation - fundamental architectural change in WordPress can be challenging to achieve in a 20+-year-old ecosystem with hundreds of thousands of plugins, themes, custom setups, etc. I'm worried about the number of architectural changes we might have to make to WordPress to support Yjs as a first-class citizen and the high chances of breaking backward compatibility in the meantime. What are your thoughts or prior experience with integrating Yjs into legacy technologies where backward compatibility was required?

[dmonad]

Thanks @tyxla, I appreciate your feedback!

I have huge respect for the fact that WP runs anywhere, even on the most basic LAMP stack. Collaborative editing support shouldn't change that.

On the other hand, new users really want real-time collaborative features like you find them in Google Docs. But, as you said, *AMP is not well suited for real-time collaborative features. LAMP doesn't have a communication channel for real-time updates. LAMP doesn't have a database well suited for storing incremental updates / every single keystroke, as Google Docs does.

That said, we can teach an old dog new tricks.

My proposal is to add as little information to the database schema as possible to make basic collaborative editing work through the existing APIs. If we store the Yjs document instead of the JSON representation, we can merge documents when autosaving documents. Hence, we can also remove the locking feature from the autosave API. Clients can accept conflicting saves and merge documents on the client-side, without teaching PHP how to actually merge Yjs documents.

Basic collaborative editing (through the autosave API) will work even on the most basic LAMP stack. But without a real-time communication channel, updates will propagate slowly. Users will receive updates from other users every X seconds, which is not ideal but a huge improvement from the status quo.

We may optionally use WebRTC or WebSockets to improve the collaborative editing experience in the future. If we would add an (optional) WebSocket server, we can achieve real-time updates, like in Google Docs. This server could connect to the proposed API to persist documents.

I'd actually love to chat with someone who has experience with the WP backend / database schema to learn in my proposal is feasible. Please PM me if anyone could guide me through the database! :)

Supporting multiple authors per revision (and therefore per entity) may be a breaking change for WordPress, and we'll need to devise a good plan to approach it. Will it be possible for the entities to continue having 1 author but multiple collaborators?

Sure, that does make sense to me. We could store the identity that stores the entity ("author") and a list of "collaborators". In the UI, we should probably render this as a single list.

Is it possible that the most basic Apache shared server with just PHP / MySQL will be limited and unable to use any collaborative features?

My proposal above would allow anyone to have basic collaborative editing (long update periods). Real-time updates (immediate update propagation) can only be achieved through WebRTC or WebSockets, which we may add on-top of my proposal.

My biggest concern is that from this proposal, it feels like Yjs will become the centerpiece, and WordPress will have to bend its database, structure, and flow around the Yjs as a single source of truth.

I hear you. We need some - potentially breaking - changes to the API to enable collaborative editing. We want to make the APIs ready for "conflicting updates", which can happen when multiple users are working on the same thing. Yjs is just the tool that enables WordPress to merge updates without conflicts.

What are your thoughts or prior experience with integrating Yjs into legacy technologies where backward compatibility was required?

A few years ago, I worked on the Jupyter project. This might be an interesting read: https://blog.jupyter.org/how-we-made-jupyter-notebooks-collaborative-with-yjs-b8dff6a9d8af

We polyfilled most of the existing APIs and used Yjs underneath without breaking the whole plugin ecosystem. But that was a hard one..

[adamziel]

We need some - potentially breaking - changes to the API to enable collaborative editing.

@dmonad Instead of asking how can we make WordPress compatible with Y.js I would take a step back and change the question to how can we make Y.js compatible with WordPress? A good compatibility benchmark for not breaking WordPress core would be: Can the real-time collaboration feature be shipped as a WordPress plugin?

I don't believe WordPress can have any breaking changes in the way posts work. There's just too many people relying on the existing model.

WordPress powers 42% of all websites. It got here by maintaining backwards compatibility even when it was really inconvenient. A backwards compatibility breakage in a piece as central as the content model would prevent thousands, if not millions of websites from updating WordPress. It would likely also break a large percentage of plugins. A lot of people would withdraw from the WordPress ecosystem and that would be the beginning of the end of WordPress like it was for many open-source projects.

We can't migrate the existing posts either. There are sites out there with 1TB of wp_posts data or more. A WordPress migration running a global UPDATE query will time out at best, and exhaust all the resources plus lock all the rows at worst.

The string-based post_content must likely remain the source of truth. I may be wrong here, but I don't currently see any way of changing what's a source of truth without a) breaking the internet, and b) making plugin development significantly harder. So many people got into programming by building simple WordPress plugins. They were only able to do that because of the simple data model WordPress has.

Here's one challenge @youknowriad touched on earlier. We're likely to see server-side UPDATE queries like this one running in the middle of the collaboration session:

UPDATE wp_posts SET post_content = CONCAT(post_content, '<p>Another paragraph of text</p>');

Whenever that happens, the collaborative editing session will need to incorporate the new information. Since post_content is a text blob, text diffs will likely be involved. This means we'll need multiple synchronization mechanisms and not just a single CRDT/OT strategy.

It would be easy to dismiss that and say "most people will collaborate for 15 to 60 minutes and it should be fine to discard any small updates during that time window". However:

• That small update may be critically important. Imagine a newspaper website where a front-page story was updated via an API call while someone else was editing. Or imagine an e-commerce store where the sale starts at midnight via a textual post_content update. We can't miss out on those.
• I may make important updates on my computer today, but only get online in three weeks to sync them. I've done a lot of work and I really don't want to lose it or reconcile it manually. If WordPress messes my changes, I'll (rightfully) start publishing angry comments about WordPress on Twitter. Too many of these, and the community will start losing trust in WordPress.

Having founded a real-time collaboration product myself, I know retaining BC and the existing data model may seem impossible at first. And perhaps they are. But today I'm convinced it can be done and we can have a great collaborative editing experience without breaking changes. It will likely require innovation, more effort than the alternative, and adjustments in Y.js, but it will be worth it in the long run.

[adamziel]

I'd actually love to chat with someone who has experience with the WP backend / database schema to learn in my proposal is feasible. Please PM me if anyone could guide me through the database! :)

I'd be happy to! I just pinged you on wp.org slack

[adamziel]

Here's a few more disruptive events that may happen during a collaboration session:

• WordPress version update
• Plugin installation or update
• Block attribute migration kicks in
• WP-CLI commands like search&replace
• Content import
• Old revision restored
• A related database row gets updated

[dmonad]

Thanks, @adamziel. I used your input to rework my approach. I really didn't think that this was possible, but I think I came up with an approach that will work with the constraints mentioned by you. #65012 (comment)

[dmsnell]

Hi Kevin, glad to have you on board, welcome! I've been stewing on this post for a while because I'm not sure exactly how best to respond or express what I want. Collaboration is an obvious need, and it's nice to see someone with your background working on the problem.

1. I want to offer my input if you have anything to discuss related to this technology.

Thank you! This is the best place to discuss it, though we might also want to consider opening a Trac ticket (at https://core.trac.wordpress.org) at some point to host more discussion with a broader audience.

When I read these posts, and I've read many discussions about real-time collaboration in Gutenberg, going back many years, I constantly feel like it would help to differentiate different kinds of collaboration, needs, and timelines. What are the collaborative flows that people managing content in WordPress are interested in?

• I'm helping my customer setup their e-commerce store and we both have the product description template open. I can make edits and they can see the things I'm changing as I change them. I can see their cursor and they can see mine. If I make a change, they can undo it; maybe they can undo exclusively their own changes unless they use a modified undo. When either of us is done with proposed changes, either one of us hits save. Our documents "update," but since they were already syncronized, nothing visibly changes.

• Several of us are working to polish a press release that's about to go out. While I have the document open in the editor someone sends an API call as part of a mass update to replace Wordpress with WordPress. Meanwhile a batch migration on the server runs and inserts an ad-placement block in the middle of the post. None of us are aware of each other, but we expect our changes to merge as they don't conflict. I see a notice in my editor that the post I am working on has been updated externally.

• I'm on a plane working on a review for an upcoming pocket-computer that I've had a preview of. While I don't expect anyone else to modify it, someone made a mistake and edited the first paragraph while I was without WiFi and before they realized I was supposed to be in the same document. When I get back down to the airport and hit save, I expect to be presented with a dialog informing me that the document has changed, and showing me how, and if those changes conflicted with my edits.

• After opening a document and before it fully loads in the editor, some plugin manually updates the wp_posts table and modifies the document I'm about to edit. This didn't feed through wp_insert_post(), maybe it did through $wpdb, or maybe it was performed by an external script connecting directly to the database outside of WordPress.

It's my understanding that Yjs can probably handle all of these situations as long as everyone does everything properly all the time, including knowing that they need to learn about the new Yjs primitives and how to interact with them. Each of these, and probably more, have distinct timelines and UI needs, I think. Any project attempting to comprehensively introduce collaboration will probably benefit by clarifying above all what modes are supported, what flows are going to look like, and what scopes will be cut or limited, as well as how the inevitable failure will be resolved when interrupted.

This is where I start, and as with all things WordPress, how can we build best-effort systems when we know that we don't have control over how other code will interact. This is where I've been very tempted by a differential-synchronization model, explored in the already-linked Core Trac issue.

1. It is unclear which client should initialize content. This leads to content duplication.
2. It is unclear which client should "save" content to the backend. This leads to loss of data.

It feels like it might make sense to be to draw a boundary between real-time collaboration within the editor, conflict management from independent agents, and long-term collaborative work. I know that I'm oversimplifying here, but it does seem like revisions have a role to play, where an editor could "start" from a given revision and be developing a diff of its changes against that revision. Upon save, the server can attempt to reconcile the changes.

And maybe this would still provide the opportunity for simultaneously-opened editors to find each other and share app state more than they are sharing a distributed document. I would think that whoever hits "save" can be the legitimate author of that change.

4. Not all clients can connect to each other directly. Some need a "TURN Server" which the user would need to set up (there is no PHP implementation).
   LAMP doesn't have a communication channel for real-time updates.

LAMP doesn't by default, but I have long encouraged folks to strive for pseudo-realtime updates via HTTP long polling, or even simply via normal HTTP polling. There may be novel ways to harness stream_select() in PHP and a database queue to provide a two-way synchronous channel for connected clients that doesn't require WebSockets or WebRTC. This idea depends on how costly it would be to hold open a server process for active edit sessions, but given that those are usually countable in terms of one, two, or a handful of concurrent sessions instead of thousands, I would imagine early implementations to be fine. Optimizations can come later.

What I love about communicating over HTTP first is that it kind of forces some helpful designs that are easy to gloss over with sockets. Beyond that, they have valuable properties for long-lived sessions, routing, and backend updates. I'm guessing you're familiar with this, but for the sake of the discussion I wanted to toss it out there.

Effectively I'm asking if there's a chance we could employ something like Yjs for the highly-interactive-but-also-rarer collaborative sessions in the editor while harnessing simpler systems for longer-term conflict management. Simplenote is pretty effective with the second approach, leaning on revisions and human intervention when conflicts do arise (for example, the server cannot merge the changes, so the client must decide if it wants to modify its contents or overwrite what's been updated on the server - "lost" content remains in the version history).

When a user "saves" a document (through autosave), it will send the Yjs document alongside the JSON representation to the backend. In case of a conflict, we might end up with multiple "conflicting" Yjs documents on the backend. The client will be able to merge these Yjs documents.

This reminds me of something I shared internally (p5j4vm-2J3#comment-4930) in 2019, which is that "the more perfect our conflict-resolution is the worse our user-experience becomes." A system with zero latency and perfect merging forms eventually-consistent documents, but in the context of multiple editors with inherent and intentional conflicts the result might be what neither author prefers. For example, changing dog to cat in one case and rabbit in another. We don't want crabbaitt in the end document. So I think the point is that the most important aspect here for conflict resolution is how we expect someone inside an edit session to be alerted to and resolve conflicts. Sometimes worse technically can be better semantically, so how can we inform someone in a conflict of the choices and make it easy for that person to resolve them?

While it may not be viable to add a simple version to every document, I wonder how far a base version number plus a content hash of the content currently-known to exist at that version would go. For example, this complicated case:

• Editor checks out revision 5, at sha1:4eddde9e6ebc2184b5df9aa9fd0af5886350df0a.
• External code updated revision 5 from outside of WordPress' control.
• Editor tells server, "here is the new copy of the document, built from revision 5, which was sha1:4eddde9e6ebc2184b5df9aa9fd0af5886350df0a at the time."
• Server responds that "content was changed in the meantime. here is the current version of revision 5."
• Editor determines if it can rebase its changes after diffing the new mutated revision 5 against the one it started with.
  • if it can rebase, it applies its changes and resubmits to the server, "this time still revision 5, but sha1:anotherhash."
  • if it can't rebase, it can ask the editor what to do. "something was changed on the backend, compare against what you changed and fix."

In most cases the server could update and acknowledge, saying "glad to see you writing; new revision version is 7 at sha1:24e5fe758d2c7e65a57379ff8deea078c1beed74".

Now if there's a polling HTTP connection with the server, the server can also announce "Hey connected edit session: someone posted revision 6 with the following diff against revision 5 at sha1:4eddde9e6ebc2184b5df9aa9fd0af5886350df0a. New hash is sha1:someotherhash.

This is all very broad and hand-wavy, and not meant to be a roadmap or plan for collaboration, but I wonder what your thoughts are about such a broad system design. How can we get more out of what we already have without adding more, or adding the least amount possible? And how can we handle the fact that know we can't control everything?

Coincidentally, I just saw the paper for Eg-walker and it looks interesting from some of these angles. Maybe there's more to learn at the algorithmic level.

Thanks @adamziel and @danluu for your review and ideas!

[dmonad]

Thanks for your input @dmsnell!

You make a good case for the need for different merging approaches in collaborative editing.

1. real-time conflict resolution in a live session. Usually conflicts are resolved automatically.
2. conflict resolution in offline mode (e.g., older edits, created on a plane, merge into an updated document)
3. conflict resolution when different agents interact with each other (e.g., a plugin, api calls that replace all words)

Conflicts can happen automatically (e.g., Google Docs). But for some scenarios, we might want to resolve conflicts manually, as you would do in Git (e.g., 2 or 3).

I want to add that there are other features we might want to add for an enhanced editing experience.

4. Suggestions (you may suggest changes, editors may accept these changes)
5. Revision history + associating changes with their authors (Alice's changes have a different background color than Bob's)
6. "Git blame"-like: There is a special mode that shows you "who" created which content.
7. Forking documents, while giving you the option to merge them later (using manual conflict resolution features to sync site data).
8. Comments - being able to mark content and add text to it.

My goal is to give WordPress the primitives to implement all of these features. I know that all these features have been discussed extensively in the WordPress community as well.

In my mind, these features deeply interact with each other. If you, for example, make a change in a collaborative editing session, but another plugin adds a comment in the range X-Y to a document, the commenting feature should ensure that the range updates when live edits happen. Yjs has a feature "relative positions" that is invaluable to achieve this. In any case, the commenting feature needs to know how to interact with the live editing session.

We probably don't want too different algorithms interacting with each other (e.g., Yjs for live-changes, three-way merges for manual merges). Everything is much simpler if we can use one algorithm to track changes. Adding either one of them is complex enough.

A user interface, for manual conflict resolution, will inevitably interact with the real-time editing feature. Users might be in a live-collaboration session when a user merges offline-changes.

An API call for updating edits will inevitably also interact with the real-time feature (but this one does so on the server), as we don't want to give preference to any of those changes. My choice would be to do automatic merging here, if possible, because normal humans usually don't want to deal with merging interfaces (it's really just programmers that like them..). And conflicts will be quite common once we have real-time collaboration.

Taking a step back, we should think of a way forward. One step at a time. Let's try to handle real-time updates first. If we somehow find a way to do this, we can build the other features on-top of Yjs while avoiding additional algorithmic complexities. I hope to convince you that Yjs really is a nice baseline for building collab apps.

Effectively I'm asking if there's a chance we could employ something like Yjs for the highly-interactive-but-also-rarer collaborative sessions in the editor while harnessing simpler systems for longer-term conflict management. Simplenote is pretty effective with the second approach, leaning on revisions and human intervention when conflicts do arise (for example, the server cannot merge the changes, so the client must decide if it wants to modify its contents or overwrite what's been updated on the server - "lost" content remains in the version history).

I love collaborative editing, even though I rarely interact with others in live sessions. Google Docs was the first mainstream product that completely obsoleted the "save button" and I have been trying to replicate that in other products ever since. The real innovation is that we can trust that every single change is persisted to the server. I never fear loss of content with Google Docs. It is a model that makes sense for the web. I can easily open two tabs, without worrying which tab has the latest content.

Personally, I don't like that revisions can overwrite each other in simplenotes. If you forget to double-check the contributions of your peers, it will easily happen that you overwrite content. I prefer a model where I don't have to find and resolve conflicts manually.

Unless there is a good reason to do so, we don't have to settle for an inferior approach. Let's try to bring Google Docs-style collaboration to WordPress!

This reminds me of something I shared internally (p5j4vm-2J3#comment-4930) in 2019, which is that "the more perfect our conflict-resolution is the worse our user-experience becomes." A system with zero latency and perfect merging forms eventually-consistent documents, but in the context of multiple editors with inherent and intentional conflicts the result might be what neither author prefers. For example, changing dog to cat in one case and rabbit in another. We don't want crabbaitt in the end document. So I think the point is that the most important aspect here for conflict resolution is how we expect someone inside an edit session to be alerted to and resolve conflicts. Sometimes worse technically can be better semantically, so how can we inform someone in a conflict of the choices and make it easy for that person to resolve them?

I see your point here. However, these kinds of conflicts never happen in real-time editing experiences. People would immediately notice that someone else made edits. However, I agree that offline edits should be reviewed, even though no collab app I know implements it like this (except git, of course).

Worse than a few scrambled characters is, in my opinion!, loss of content by accidental saves. Users should never be able to overwrite each-other's changes. That deeply violates the trust that we have in our editing experience. Most users, so my assumption, won't double-check the revision history to ensure that all their edits survived.

This is all very broad and hand-wavy, and not meant to be a roadmap or plan for collaboration, but I wonder what your thoughts are about such a broad system design. How can we get more out of what we already have without adding more, or adding the least amount possible? And how can we handle the fact that know we can't control everything?

You might find research around three-way merges quite interesting. I dabbled with it for a while, but it's really complex to implement in a production system. It requires diffs for every change. Furthermore, you need to teach the client AND the server how to resolve conflicts on the HTML documents (while never breaking syntax!). Keeping track of the different revisions for performing correct three-way merges is incredibly complex and requires a complicated communication protocol as well. I developed Yjs because the conflict resolution algorithm is much simpler, while providing more advantages.

[dmsnell]

thanks for the response, @dmonad.

I hope to convince you that Yjs really is a nice baseline for building collab apps.

there's no need to convince me. I'm sure it's great at what it does. I'm more concerned about the question if it's also the best fit within the WordPress ecosystem. if we depend on having every edit to a resource go through a parallel Yjs document, I would be hard-pressed to imagine anything but repeated failures and the kind of concurrent failures that Yjs is supposed to prevent.

in this space we pretty much have a guarantee that third-party actors and code will modify the underlying resources in the database without going through any official means, and unless we create brand new content types in WordPress that bypass every form of existing extensibility, I don't see a great way to build a new system requiring that. even if we did, I would only predict the kind of new-but-never-used effect that often happens when creating "better" systems than ones that have become known and relied upon for decades.

I don't like that revisions can overwrite each other in simplenotes

Most users, so my assumption, won't double-check the revision history to ensure that all their edits survived.

normal humans usually don't want to deal with merging interfaces (it's really just programmers that like them..). And conflicts will be quite common once we have real-time collaboration.

based on these statements I don't think I communicated properly what I meant to. a quick note on Simplenote though is that the overwriting revisions is a choice the clients make. clients make that call simply because it's easy and simple to users, as you state. within certain limits, most people seems to have preferred that as a basic solution to a complex problem. but it doesn't have to be that way, and in WordPress it probably wouldn't.

what I was trying to communicate is that given that we know we don't have control over the full data pipeline, focusing on communicating to the author that something has changed and providing a means to resolve that could go far without requiring a complete overhaul of WordPress' established architecture (no matter how valiant such an effort might be).

simultaneous edit sessions seem easy, and if things are in sync I have my doubts about how important it would be that you or I was listed as the one who changed the color to orange. that was my point about whoever saves is the one ascribed to the changes.

right now WordPress does the bare minimum to say "it's possible someone else is editing this document" and then on save it does nothing to prevent conflicts. using the existing revisions, and few additional bits of data that already exist, we could dramatically improve the current state and introduce real-time collaborative edit sessions.

I want everything and I want it yesterday 😄 but a huge challenge in maintaining WordPress and not breaking the web is trying to figure out how to adapt modern and mature ideas into a rich and historied project.

so the biggest question is still this: how do you propose that Yjs will work when someone edits the raw HTML of post_content via a database update but doesn't edit the corresponding Yjs document? keep in mind there are multiple existing filters run on posts during save (such as wp_insert_post_data).

Suggestions (you may suggest changes, editors may accept these changes)

this sounds great also. though I wonder if it couldn't also fall within one of the existing paradigms. specifically, if we had a record of which revision it was based on and could form a diff of the changes, then it would look a bit like any other revision, just one requiring a manual marge authorization.

[dawidurbanski]

One step at a time. Let's try to handle real-time updates first. If we somehow find a way to do this, we can build the other features on-top of Yjs

I have nothing to back this up, but my gut feeling tells me it should be the other way around. First making sure the offline synchronization capabilities are rock solid and only then to pursue real-time on top of it.

[dmonad]

Thanks for your feedback @dmsnell! I hope that I addressed your concerns in #65012 (comment)

[dmonad]

An update on my progress.

TLDR: just watch the last video below.

I talked to several people and I think the consense is that we can't restrict the POST API. Thanks for all your input!! I'm new to WP & PHP & Gutenberg, so there was a lot to catch up on. If I understand correctly, the argument is that there are existing plugins that manipulate posts after they are submitted via the REST API (using filters).

A short overview of the problems that we need to deal with:

We are basically trying to keep different sources of truth in-sync: 1) The backend representing posts in a database. 2) The clients representing posts in Yjs. We want real-time collaboration. But PHP can't really do WebSockets right now. Additionally, WP can't speak Yjs, unless we port Yjs to PHP **(there are various language ports of Yjs, but they are all based on our C/Rust implementation of Yjs which we can't use in PHP). Then we have the additional complexity that any plugin could change post-content at any point in time without letting other clients know what exactly they changed. Any write to the database is a complete overwrite, not keeping any history, which is a requirement for automatic conflict resolution. Phew…

A simpler statement of the problem:
When we "save" a post (using a POST request overwriting the HTML), we simply replace the whole content. Filters might get applied that change the content once again. Now the collaborative editor is out-of-sync with the saved document. The Yjs history is currently not stored in WP at all.

Since I last posted, I've been investigating how we could make WP collaborative without disrupting the existing APIs.

I'm currently working on a prototype that keeps the Yjs document in-sync with manipulations of the post content. The Yjs document and some additional data is stored in a <!-- ydoc .. --> HTML comment in the post content. The client will regularly pull the history (currently using the existing autosave feature) and check if they need to apply changes from remote. Normally, this wouldn't be possible, because we can't have several clients all applying the changes from WP. This would lead to content duplication. But if the editing history in WP is linear, we can simulate a "system user" that applies the changes from remote to the Yjs document in a consistent manner (for readers familiar with Yjs: it's really complicated to do this correctly - I don't recommend doing it unless you know what you are doing). I did that and it works great!

We can't trust that all clients can communicate via WebRTC (or that a WebSocket server is set up). So our "baseline" must work only with REST requests, which all WP instances support.

I prepared a "baseline" demo that shows that we can sync using only the HTTP APIs. It even syncs if content in the database is manipulated directly. The demo runs only on HTTP requests, but I set the autosave interval to 3 seconds. This demo will work on all WP installations.

please ignore the warnings, they are expected

autosave.webm

To show that we get actual conflict resolution when three different peers manipulate content concurrently (two browser windows, one direct db manipulation), I disabled autosave so that we only get conflict resolution when saving a post. Most importantly, this demo shows that collab editing prevents content loss through concurrent save. Instead, you will actually merge the changes. You can still use the revision history to jump to a previous revision.

actual-conflict-resolution.webm

Now, we can add y-webrtc into the mix to get realtime collaboration for users that are able to establish a p2p WebRTC connection (it's a progressive enhancement). For this demo, I set the autosave interval to a reasonable 10 seconds.

collab-using-webrtc.webm

The encoded Yjs document - which we should retain for future sessions - is stored in a comment tag in the post content <!-- y:doc state=[base64] .. -->. I'm not fixed on keeping the document in a "comment", but there are several reasons that speak for it. I.e. It would enable users to "fork" a document (similar to a git fork) and merge it later with the original document (possibly using suggestions). Also, it would ensure that the WP revisions have a linear history of changes that we can replay, even in the case of a concurrent save.

While this will definitely add some additional overhead to the stored post size because it now contains the encoded Yjs document (we can get the collab overhead down to roughly the size of the HTML content), I believe that this would be a nice non-disrupting solution for adding collaborative editing to WP. There are several details that I need to work out before making a full proposal.

In the meantime, I'm looking for feedback.

[youknowriad]

The videos worked for me on Chrome but not Safari.

[bbertucc]

The videos worked for me on Chrome but not Safari.

Thanks. Chrome works. The GitHub app for iPhone does not.

[adamziel]

It looks promising @dmonad 🎉 ! I'd love to read through the code – is there a public branch or repository I could review?

We are basically trying to keep different sources of truth in-sync: 1) The backend representing posts in a database. 2) The clients representing posts in Yjs.

💯 lovely! That's exactly the constraint we're working with in WordPress.

The Yjs document and some additional data is stored in a HTML comment in the post content.

This is a really cool idea! I'm intrigued and interested to see what we discover about the advantages and limitations of this approach.

One thing I'm worried about is creating a gajillion post revisions during one short collaborative editing session. I'm not sure how to solve that yet. One thing I thought about was periodically pruning the list of revisions, but I don't entirely like that idea. Or perhaps it won't be that big of a deal with lower synchronization intervals.

The client will regularly pull the history (currently using the existing autosave feature) and check if they need to apply changes from remote.

Nice! Sounds fantastic.

But if the editing history in WP is linear, we can simulate a "system user" that applies the changes from remote to the Yjs document in a consistent manner

The history is indeed linear. The "system user" approach sounds very reasonable.

We can't trust that all clients can communicate via WebRTC (or that a WebSocket server is set up).

I'm thinking it might be possible to build a WebSocket server built in userland PHP code using socket utilities such as stream_select.

It might or might not be viable. Perhaps holding up multiple idle PHP workers will be a problem on some hosts – or perhaps it will be cheaper than running fetch() every 3 seconds. Similarly, perhaps the default 30 seconds script execution time limit will get in a way – or perhaps not. Either way, it's an avenue worth exploring. It might enable assuming WebSockets, not REST calls, are the baseline.

---

Overall I think it's a solid update in the right direction. ❤️

[alansmodic]

This is very cool to see! I know it's still early going, but I'm curious how the sidebar plays into this. The videos showcase the post editing experience, but have we given thought on whether others would be able to manipulate metadata and other bits within the sidebar concurrently?

[sinanisler]

seems like webrtc is way to go or maybe combination of both methods