Trying to figure out how/if to use delegateToSchema for a tricky join

[ztolley]

First of all, first time poster, long time user :) I appreciate the unique toolset that graphql-tools provides and I'm definitely thankful that it understands that schema stitching is still important a and we can't all just adopt federation.

I'm building a GraphQL API gateway to join 2 remote API's. The first backend (FDP) manages a list of trucks and stores information about them including how they use burn fuel. The 2nd backend (PLAN) is used for storing planning data and part of this involves defining which trucks take part in a plan. The two API's are maintained by remote teams and cannot be changed.

The aim is to create a gateway to act as a single GraphQL endpoint and stitch the related data together to give a single seamless API. As part of this the gateway will need to use a truck id from the plan and join that with the truck in the other datasource

The FDP API uses base types and then extends them for different types of data requested. A typical request to get a trucks name and fuel information would look like this:

query TruckBurnRates {
  dataSources(dataType: Fbr) {
    ... on FBRDataSource {
      fbrTruck(truckId: "a6276400-9e2b-48c7-af57-7b799e83b4c1") {
        uuid
        truckName
        burnRates {
          speed
          mpg
        }
      }
    }
  }
}

To get information about a plan you would use:

{
  getPlanById(id: "b6276400-9e2b-48c7-af57-7b799e83b4c1") {
    id
    name
    trucks {
      id
      truckParticipations {
        joinDate
        leaveDate
      }
    }
  }
}

So one proposed way to get all the data back in a singe query would be to use either:

{
  getPlanById(id: "b6276400-9e2b-48c7-af57-7b799e83b4c1") {
    id
    name
    trucks {
      fbrTruck {
        truckName
        burnRates {
          speed
          mpg
        }
      }
      truckParticipations {
        joinDate
        leaveDate
      }
    }
  }
}

Or, though may cause issues as schemas grows

{
  getPlanById(id: "b6276400-9e2b-48c7-af57-7b799e83b4c1") {
    id
    name
    trucks {
      truckName
      burnRates {
        speed
        mpg
      }
      truckParticipations {
        joinDate
        leaveDate
      }
    }
  }
}

I've setup a server at https://stitching-gateway.herokuapp.com/ that runs a copy of the code.

So the question?

All the documentation and examples for delegateToSchema describe its use in simple scenarios in which you specify a field in the Query object and a single field in the response. In this case though fetching the data does not follow that, it needs a more specific query and parameters passed down into that, plus the data requried is a few layers deep.

• Is it possible to join these two API's with delegateToSchema?
• Is there some form of transformation that could make it possible?
• Is there some form of use of fragments that could help?
• Does the API gate have to generate the query to use with the little information it has within the stitching resolver?

[yaacovCR]

Not sure what you mean, truck id is right there under trucks from plan schema, you can stitch from that. I may be misunderstanding the problem with your specific use case.

In general with schema stitching now the first question is whether you want to use extension on the gateway or type merging. Extension on the gateway is more tried and true but type merging may be closer to what you want...

[ztolley]

yeah I have the truck id from the plan schema but the query to get that truck from the other schema is not straightforward

query TruckBurnRates {
  dataSources(dataType: Fbr) {
    ... on FBRDataSource {
      fbrTruck(truckId: "a6276400-9e2b-48c7-af57-7b799e83b4c1") {
        uuid
        truckName
        burnRates {
          speed
          mpg
        }
      }
    }
  }
}

How would I configure delegateToSchema to that?

delegateToSchema({
  schema: fdpSchema,
  option: 'query',
  fieldName: 'FBRTruck',
  args: {
    ...args,
    id: truck.id,
  },
  context,
  info,
})

Doesn't work and if I used fieldName: 'FBRTruck' I don't know how that gets the truck back and picks the right thing. I tried both values for fieldName and threw a debug statement into their resolvers and they never got called.

How would type merging work here?

[gmac]

FWIW, the docs on passing arguments between delegated resolvers just got published on the live site today, see https://www.graphql-tools.com/docs/schema-stitching#passing-arguments-between-resolvers

[ztolley]

Interesting

[yaacovCR]

Oh, I see, you need to place the argument on a nested field, you can stitch to the inner field and then use WrapQuery or newer TransformQuery to wrap, TransformQuery I think may better support fragments (non inline)

[ztolley]

Ok, I think. Not used them before but I’ll head to the documentation and get my head around it. Thanks for the pointer

[ztolley]

Do you know where I can read about TransformQuery or any examples?

[yaacovCR]

Actually, I think WrapQuery and TransformQuery wrap the selectionSet and you want to wrap the root fields. You may need to create your own transform...

[ztolley]

That was one thing I wondered too, so I’d transform the fdp schema to make it appear like it has a root query field called truck that accepts an Id and returns the fields at the top level. Not sure where to start on that but will give it a go

[yaacovCR]

How did this go for you?

I realize I never addressed how type merging would help. Type merging merges together types with the same name, so it would work only if the fbrTruck field and the trucks list field could be renamed to refer to the same underlying named type, perhaps Truck.

In that case, the gateway will automatically query the other subschema for remaining fields. By default, you get all the root fields from both subschemas automatically merged.

In order to make the magic happen, each subschema has to provide a way to get at that type. If it is simple, you can just specify the root field name and arguments -- if it is complex, you can write your own merged resolver.

See: https://www.graphql-tools.com/docs/schema-stitching/#merging-types

Your merged type resolver for the FDP schema might be something like this:

mergedTypeConfig.resolve = (originalResult, context, info, schema, selectionSet) =>
  delegateToSchema({
    schema,
    operation: 'query',
    fieldName: 'dataSources',
    returnType: getNamedType(info.returnType),
    args: { dataType: 'Fbr' },
    selectionSet: {
      kind: Kind.SELECTION_SET,
      selections: [{
        kind: Kind.INLINE_FRAGMENT,
        typeCondition: {
          kind: Kind.NAMED_TYPE,
          name: {
            kind: Kind.NAME,
            value: 'FBRDataSource',
          }
        },
        selectionSet: {
          kind: Kind.SELECTION_SET,
          selections: [{
            kind: Kind.FIELD,
            name: {
              kind: Kind.NAME,
              value: 'fbrTruck',
            },
            arguments: [{
              kind: Kind.ARGUMENT,
              name: {
                kind: Kind.NAME,
                value: 'truckId',
              }
              value: {
                kind: Kind.STRING
                value: originalResult.id,
              },
            }],
            selectionSet,
          }]
        }
      }]
    },
    context,
    info,
    skipTypeMerging: true,
  });

You don't actually indicate above how to get a truck directly from the PLAN subschema -- so the merging would work from a FDP root field.

I believe the above would work even though the key fields seem to differ by name between the two schema (id vs uuid) because there are only two two subschemas, so you would always know which field within the original result you would need, but your other options are to rename the fields using transforms prior to merging to get a common key, or to use some logic to extract the key from the originalResult (something like originalResult.id ?? originalResult.uuid).

I hope that's helpful!

[ztolley]

Ok starting to get my head around this but it's unclear in the documentation how you actually implement it.

So in my code I'd change the Type in my plan schema I'd change the Truck type to be FBRTruck and access them as one type, which tbh is nicer. And I'd get rid of the resolver that tries to use delegate to schema in the plan to fetch the details from fdp. BUT

Reading through the docs and looking at this I don't see how you implement this piece of code. Simply put, where does it go?

[ztolley]

https://github.com/ztolley/stitching_gateway/blob/master/src/graphql/index.js

I'm so close it hurts. The schema produced it exactly what I want and I can get the truck.id BUT if I try to get any of the fields that come from fdpschema then it comes back with null. If I add some console commands to the fdpschema resolvers they aren't even called. I Can start to see what you are trying to do, I'm just at a loss as to what I'm doing wrong

[yaacovCR]

Instead of:

async function getSchema() {
  return stitchSchemas({
    subschemas: [
      { schema: fdpSchema },
      {
        schema: planSchema,
        merge: {
          FBRTruck: (originalResult, context, info, schema, selectionSet) =>
            delegateToSchema({

Try:

async function getSchema() {
  return stitchSchemas({
    subschemas: [
      { schema: fdpSchema },
      {
        schema: planSchema,
        merge: {
          FBRTruck: {
            selectionSet: `{ id }`,
            resolve: (originalResult, context, info, schema, selectionSet) =>
              delegateToSchema({
          }

you may need mergeTypes: true after all, this is not required when specifying a merging resolver, but so far you are only trying a one-way merge.

And a reminder that the key you are referencing in selectionSet has to be in the original result, usually this should have the same name in both schemas, but in your example you have id and uuid, this could be renamed using transforms prior to merging using RenameObjectFields.

You may be able to get away with it because you are using a one-way merge, and I think based on the direction the selection set hint should be id not uuid.

This hint is for what you need to be available under originalResult, i.e originalResult.id.

[ztolley]

I DO have control over one of the schemas at the back, the planning one, so I could change the field from id to uuid, would that help?

[ztolley]

Still no joy but if I throw some console logs in there I can see that the resolve function in that merge.FBRTruck property isn't even being called

[yaacovCR]

You can try console logging the stitches schema extensions to see what is happening, if you think there is a bug, you can submit a failing PR with a more minimal example.

Looking at this on my phone, I can't see the problem either... :(

Not sure when I can take a longer look, unfortunately.

[ztolley]

I appreciate the help, though I'm getting to the stage I would actually pay someone to solve the problem and provide code that works for the project I'm on

[yaacovCR]

When you get to that stage, you can message me offline, might be able to free up some time. Either way I will try to look into this more deeply when I can.

[yaacovCR]

Probably to avoid confusion.

But whatever fields you have control over, now you have to consider why not just putting those types and fields in gateway itself with extend as necessary...

Generally, delegation should be used only when necessary...

[ztolley]

In the real application (this is a mock up) the two graphql api's are on remote servers and I only have control over the planning one. The aim is to provide a single combined GraphQL API to the client side aplications

[yaacovCR]

Ok, type merging is tricky here because:

1. various superficial issues (uuid vs id not completely standardized, placement of the merge resolver under the wrong subschema, see https://github.com/ztolley/stitching_gateway/pull/2)
2. a hard requirement by stitchSchemas for a root field to both types in ALL schemas (no one way merging)
3. the fact that there are actually no root fields to the type in either schema. missing entirely is a way to get a specific truck from the plan schema, and the FBR schema actually returns a list of trucks rather than a single truck for each uuid.
4. a bug! see fix(createRequest): selectionSet overrides fieldNodes #1820, thanks for helping surface this!

Could you explain more what's going on with third point above? Why is that a list? You may need to flesh out more about the schema for those not so familiar with your use case. :) Apologies if this has been explained already.

[yaacovCR]

Have to try it and find out though!

[ztolley]

out tomorrow but will try it tomorrow evening. Not sure how to convert the list to object, or is that a 'it just works tm' :)

[yaacovCR]

In the merge resolve code, you should get an array as result from list (or an error, of course!).

So if it's an array, it should just be a single item array and just return the first item.

You also need to tweak returnType to be a list of whatever the type is now...

[yaacovCR]

https://github.com/ztolley/stitching_gateway/pull/3

Not so simple just to return the first item, as it turns out there is a nested object, and that requires re-annotating the nested data with the appropriate errors. New helper functions should be created to make this possible to do properly.

Linked solution is a hack that works but will cause errors to be dropped.

[yaacovCR]

Actually, we already export all the helper methods we need, PR is complete and ready to go.

[gmac]

Just tried setting up type-merging for all my tricky cases today including a cross-service interface and a model in one app returning paginated results from another app... and it all kinda “just works”. It’s really remarkable how well this works Yaacov (kudos!!), and it’s both simpler in service design than federation and feels more extensible. Plus I can start incrementally rolling it out in my existing API without a system-wide migration and cutover. I’ve still got to demo it all to my team, but there is a very strong chance that we’ll be tabling our intentions of migrating to federation later this year in light of these new capabilities. Awesome work!

…

On Mon, Jul 27, 2020 at 4:38 PM Yaacov Rydzinski ***@***.***> wrote: Have to try it and find out though! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1608 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFRROG2AXZWKJXUWVZDKZ3R5XQTNANCNFSM4NXR7YDA> .