Inversed relationship field

[donstephan]

Hey all,

I've been messing with Payload a few days here and just wanted to share a feature I implemented as a plugin and see if it can be a good use case for a payload field. The plugin creates a new field that is an inversed relationship field. It's a virtual field that returns relation data in the opposite manner. Here's an example of how it can be used and what it looks like:

Say you have two collections, a users collection and a roles collection. In this context, a user can have many roles for many different organizations. Roles have a relationship to users on the userId field. It would messy to have to maintain an array of roleIds on the user object (however that is very possible and easy to do with hooks). Instead you can choose to add an inversed relationship field in the users collection. This field looks like so:

  {
      name: "roles",
      type: "inversedRelationship",
      inversedBy: {
        collection: "roles",
        field: "userId"
      }
    }

It describes the collection (by slug) that it is inversed by and which field the relation is on. My plugin currently then swaps that field type for an array type, adds in the field schema of the inversed collection and makes it so the admin UI is disabled for this array. This allows it to pass GraphQL schema generation and show the data in the admin UI. Lastly it sets up an afterRead hook on that field which performs the query to get the documents in question. So now when I perform a query like so:

{
  Users {
    docs {
      name
      roles {
      	 role
         organizationId
         isGlobalRole
      }
    }
  }
}

I get back:

{
  "data": {
    "Users": {
      "docs": [
        {
          "name": "Jane Doe",
          "roles": [
            {
              "organizationId: "",
              "role": "super-admin",
              "isGlobalRole": true
            },
           {
              "organizationId: "6185260f8876d00088c1b3e1",
              "role": "owner",
              "isGlobalRole": false
            },
          ]
        }
      ]
    }
  }
}

Happy to chat about the use cases for this. Again it works fantastic in a plugin context and happy to leave it in that realm but figured I'd share and see if anyone is looking for a feature like this.

[jmikrut]

Hey @donstephan —

This is great! We have been looking forward to someone sharing code to do this. I think this would be great to offer as a plugin. It's possible that we may support bi-directional relationships internally in the future but with a setup like yours, and the fact that it's completely possible currently, I can't say that the likeliness is high.

Sounds like you have a great handle on how plugins work already even down to setting the new fields to readOnly. Good stuff.

I did notice that through how you are specifying a field's type as inversedRelationship—that might give TypeScript users some issues as field configs are strongly typed and inversedRelationship is not a valid field type.

Instead, the plugin's API could be adjusted to just accept options when the plugin is added to a user's config, and then append a field into the end of each relevant collection, using the specs that have been passed. There could be a few other TypeScript-friendly ways to do this too.

I would also say that instead of injecting the full field config of the referenced relationship, you might also just inject a relationship field itself, which would be dynamically populated. That would allow developers to retain full control over things like depth, locale, and access control while accessing data.

To take this a step further, for performance reasons, you might want to write a series of afterChange hooks that actually does save the data to the database bi-directionally as dynamically populated and saved relationship fields. So, instead of populating data in an afterRead hook, you'd straight up save each side of the relationship when one of the referenced sides changes. Might be some interesting ways to accomplish that in an elegant fashion.

Great work!

[donstephan]

Thanks for the input!

I like it. Yeah swapping the relationship field in makes more sense as a user can append the relevant fields in. Luckily it uses the Payload local API on the afterChange hook so it does preserves access control functionality.

I have it swapping out the inversedRelationship field type explicitly to get around typed field configs. Curious, is there a way for a plugin to add in new types there? I can do it at the plugin args level, but having it live right in the schema feels like a good experience.

Lastly I think there's an arguments on both sides whether to denormalize data or not. I think an option to denormalize the relationship should be possible, but maybe as an option? Say you have two collections, one describes organizations and another that describes users. If an organization updates their name or lastAccessedAt field you'd potentially have to go update tons of users which isn't ideal. However if we're dealing with an inversed link to a collection like files that describes an avatar image this would be a really good use case to denomalize since it's specific to a singular user document and probably doesn't change all that much (very similar to how relationships for Uploads currently exist in Payload).

I believe the ideal solution for inversed relationship queries would be that it only performs a single query at each layer in the process. Say you query 100 users that are part of 8 different organizations, that should only create 2 queries. One query for the usersand then another that is produced at theinversedRelationshiplayer to query all theorganizations` in question. Indexes here are also super important.

Anyways, I do agree to leave this in a plugin context as it's completely doable in that realm. Feel free to resolve this and happy to share the plugin here when it's all finished up.

[AlessioGr]

I'd definitely need such a feature.

I have a "User" and "Author" object. Posts on my website are created by multiple author. However, not every author needs to be an admin user which can log in to the admin panel.

So, I made an Author object with Name, Avatar, bio etc. and a relationship to User (with auth=true). Now, I only have to fill in that relationship if the author is a user which can log-in.

Only issue: the User field does not have access to the Name/author/bio fields of Author. Would be great if that bidirectional relationship/polymorphism could work great and be displayed nicely in the user interface and API.

[AlessioGr]

Maybe it's a good idea to "make the relation field better" and automate those inversed relations, similarly to how Strapi does it, by offering those 6 different kinds of relations?:

Would make it really easy to use it that way.

EDIT:
Could be structured like this, replacing the hasMany with relationType:

[DanRibbens]

Check out this repo from the discussion in #204. Specifically the two files in this directory: https://github.com/RobertAlbus/kto-cms/tree/payload-poc/payload-kto/src/collections/hierarchical show off updating related collections in a hook.

At some point we'll be having more conversations about native support and what the best way to handle this is. I'm extremely cautious about replacing the hasMany property since I like the simplicity and would be a breaking change.

[AlessioGr]

Check out this repo from the discussion in #204. Specifically the two files in this directory: https://github.com/RobertAlbus/kto-cms/tree/payload-poc/payload-kto/src/collections/hierarchical show off updating related collections in a hook.

At some point we'll be having more conversations about native support and what the best way to handle this is. I'm extremely cautious about replacing the hasMany property since I like the simplicity and would be a breaking change.

@DanRibbens I'm trying to implement something like that, but I'm unsure how to create such a AfterChange Field hook which should update the field of the other collection. In the field you linked, they have used the "AfterChangeHook" type. However, it gives me type errors.

In the docs, you use type ExampleFieldHook = FieldHook<ExampleDocumentType, string, SiblingDataType>; however, I don't know what to put as ExampleDocumentType and SiblingDataType.

Could you make a concrete example for such a field hook (and maybe add it to the docs)?

[AlessioGr]

Current progress of my InversedRelationship field:
InversedFieldHooks.ts:

import payload from "payload";

import { FieldHook } from 'payload/types';




//Remove all relations to parent from the child. This does not work yet
export function handleParentBeforeChange(parentCollectionName: string, parentFieldName: string, childFieldName: string): FieldHook {
    async function hook({data, findMany, operation, originalDoc, req, siblingData, value}) {
        console.log("handleParentBeforeChange")


        console.log("data", data);
        console.log("findMany", findMany);
        console.log("operation", operation);
        console.log("value", value);

        if(!data[parentFieldName]) {
            console.log("data.parentFieldName " + parentFieldName + " not found");
            return;
        }

        for(const relation of data[parentFieldName]){
            const relationCollection = getRelation(relation);
            const childData = await relationCollection.document;
            console.log("relCollection doc", childData);

            if(!childData[childFieldName]) {
                console.log("childData.childFieldName " + childFieldName + " not found");
                return;
            }

            console.log("Updating childData.childFieldName " + childFieldName);
            console.log("relationCollection.type", relationCollection.type);
            console.log("relationCollection.id", relationCollection.id);
            console.log("value", value);
            console.log("originalDoc", originalDoc);


            //remove existing relations

            const ii = payload.update({
                collection: relationCollection.type,
                id: relationCollection.id,
                data: {
                    [childFieldName]: [
                    ],
                },
            });

            console.log("ii", ii);
        }
    }

    return hook;
}

// Parent has changed inversed relation value => Update the respective hierarchical field of the child, if it exists. First find the child, then go through ALL possible parents and make the chields field correct
export function handleParentAfterChange(parentCollectionName: string, parentFieldName: string, childFieldName: string): FieldHook {
    async function hook({data, findMany, operation, originalDoc, req, siblingData, value}) {
        console.log("handleParentAfterChange")


        console.log("data", data);
        console.log("operation", operation);
        console.log("value", value);

        if(!data[parentFieldName]) {
            console.log("data.parentFieldName " + parentFieldName + " not found");
            return;
        }

        for(const relation of data[parentFieldName]){
            const relationCollection = getRelation(relation);
            const childData = await relationCollection.document;
            console.log("relCollection doc", childData);

            if(!childData[childFieldName]) {
                console.log("childData.childFieldName " + childFieldName + " not found");
                return;
            }

            console.log("Updating childData.childFieldName " + childFieldName);
            console.log("relationCollection.type", relationCollection.type);
            console.log("relationCollection.id", relationCollection.id);
            console.log("value", value);
            console.log("originalDoc", originalDoc);


            //Get all relations then add them to child
            const relationFieldValueOfChild: {relationTo: string, value: string}[] = [];

            relationFieldValueOfChild.push(
                {relationTo: parentCollectionName, value: originalDoc.id}
            )

            //Now fill from other possible parents
            const otherParents = await payload.find({
                collection: parentCollectionName, // required
                depth: 2,
                page: 1,
                limit: 10000,
                where: {},
            })
            console.log("otherParents", otherParents);

            for(const otherParent of otherParents.docs){
                if(otherParent.id === originalDoc.id || !otherParents[parentFieldName]) {
                    continue;
                }
                for(const otherParentChild of otherParent[parentFieldName]){
                    if(otherParentChild.relationTo === relationCollection.type && otherParentChild.value === relationCollection.id){
                        relationFieldValueOfChild.push(
                            {relationTo: parentCollectionName, value: otherParent.id}
                        )
                    }
                }
            }

            const updateResult = await payload.update({
                collection: relationCollection.type,
                id: relationCollection.id,
                data: {
                    [childFieldName]: relationFieldValueOfChild,
                },
            });

            console.log("update result", updateResult);
        }
    }

    return hook;
}

function getRelation(obj: any): { type: string, id: string, document: Promise<any> } {
    let type: string = obj["relationTo"];
    let id: string = obj["value"];

    return {
        type: type,
        id: id,
        document: payload.findByID({
            collection: type,
            id: id,
            depth: 1
        }) as Promise<any>
    }
}

InversedFieldTest.ts

import { Field } from 'payload/types';
import { handleParentBeforeChange, handleParentAfterChange } from "./InversedFieldHooks";

export function HierarchicalFields(args: {
    childOf?: string[],
    children?: string[],
    hasMany?: boolean,
    childFieldName?: string,
    parentFieldName?: string,
    collectionName?: string,
}): Field[]  {

    let fields: Field[] = [];

    //Add to the parent field => Child has changed
    if (!!args.childOf?.length) {
        fields.push({
            name: args.childFieldName,
            type: 'relationship',
            hasMany: args.hasMany,
            relationTo: args.childOf,
            admin: {
                readOnly: true, //TODO: Allow editing of parent field with own hooks in the future
            },
            hooks: {
                /*afterChange: [
                    handleParentChange(args.childFieldName ? args.childFieldName : "parent"), //TODO: Add (see readOnly: true comment above)
                ],*/
            },
        });
    }

    //Add to the child field => parent has changed
    if (!!args.children?.length) {
        fields.push({
            name: args.parentFieldName,
            type: 'relationship',
            hasMany: args.hasMany,
            relationTo: args.children,
            hooks: {
                beforeChange: [
                    handleParentBeforeChange(args.collectionName, args.parentFieldName, args.childFieldName),
                ],
                afterChange: [
                    handleParentAfterChange(args.collectionName, args.parentFieldName, args.childFieldName),
                ],
            },
        });
    }
    return fields;
}

Example Usage of field:
Parent:

...HierarchicalFields({
            childOf: [],
            children: ['gear-component'],
            childFieldName: 'gearBundles', //of child (gear-component)
            parentFieldName: 'gearComponents', //of this
            hasMany: true,
            collectionName: 'gear-bundle',
        }),

Child:

...HierarchicalFields({
            childOf: ['gear-bundle'],
            children: [],
            hasMany: true,
            childFieldName: 'gearBundles', //of this
            parentFieldName: 'gearComponents', //of parent (gear-bundle)
            collectionName: 'gear-component',
        }),

Current issues:

• Relationship only updates if the parent is changed - not if the child is changed
• Deleting the parent or removing the relationships from the parent completely does not update the child
• I don't think it works that well if there's multiple parents having a relationship one child? (Might be wrong there)
• I still have to add the field to both parent & child, even though adding it to only one should optimally be enough

[AlessioGr]

I've created a repository for this here where I explain my current challenges in more detail as well: https://github.com/AlessioGr/payload-inversed-relationship

[lkostrowski]

Hey

+1 to enforncing bi-directional relationships. I think this is very common in building every type of application (at least CRUD type). I think using Mongo is problematic here since relations are not "real" - I think in this case framework (CMS) should abstract this away and simulate strong bi-directional relationship 👍

Till then, maybe its worth to document this as some tutorial/guide how to achieve it with hooks? I think people will try to handle this on frontend which is dangerous due to transactions

[niallobrien]

So you're suggesting to simply expose this functionality to the API at the application level without actually making changes to the database? I like this idea, though I'd imagine it wouldn't be as performant as a dedicated solution, it'd at least solve the issue for now until the core team decides how best to approach it, as it's not currently on their roadmap.

[lkostrowski]

No.

At the moment this is a leaky abstraction, Payload allows me to construct relations, but the database under the hood is not. So I need to write custom hooks to fix it.

IMO, if Payload implements relations it should handle the database beneath it to actually contain relation (can be optional/configurable, whatever).

[niallobrien]

Yeah, I see what you mean. Quite a chunk of work I imagine.

[tyteen4a03]

Hope this can be a first-class citizen of Payload soon - Relationships are only half as useful without it.

[fr3fou]

Any updates on whether this will come in 3.0?

[DanRibbens]

We added the join field in v3 https://payloadcms.com/docs/fields/join for bi-directional relationship support.