Data sources

Requirements

• You should already have a basic understanding of the architecture of the API.
• You need to understand the package io-ts. We refer to those objects as decoders.
• You need to understand our cache system.
• When you want to add new endpoints you need to know how to work with node-fetch in order to make HTTP requests. See also the Fetch API for an introduction.

What is a data source?

api.serlo.org uses several services. Access to those services is provided by a data source object per service. These objects are implemented in ~/model. Normally each service provides one to many REST endpoints and the models provide a function for each of the used service endpoints:

All requests are implemented with the Fetch API implemented by node-fetch for implementing the requests.

An example:

Access to the database layer is implemented by the serlo model defined in ~/model/serlo. It provides the endpoint dataSources.serlo.getAlias({ ... }) for accessing information about an alias (an alias is a specification of a Serlo resource given by a subdomain / instance like "en" or "de" and the path name or the URL). When this function is called the following request is performed in the background:

~$: curl -X POST \
    -H "Content-Type: application/json"  \
    --data '{ \
        "type": "AliasQuery", \
        "payload": { "instance": "de", "path": "/mathe" } \
    }' https://<IP of database-layer>/

The result is something like which is returned as an result:

{
  "id": 19676,
  "instance": "de",
  "alias": "/mathe"
}

Using a data source

Via the context argument (= 3rd argument) each resolver function has access to all data sources via the dataSources property:

resolverFunction(parent, args, { dataSources }) {
  // use dataSources here to make requests to services

  // An example: request the alias
  const uuid = await dataSources.serlo.getAlias({ instance: "...", path: "..." })
}

Have a look in the source code of each data source to see which functions they have defined. Currently we have the following data sources:

• serlo: gives access the database layer and thus to the Serlo database
• googleSpreadsheetApi: gives access to the Google Spreadsheet API and thus to certain Google spreadsheets

Payload types

The return type of a data source function is called a payload type. Let's recall the returned object of the function dataSources.serlo.getAlias({ ... }):

{
  "id": 19676,
  "instance": "de",
  "alias": "/mathe"
}

In this example the alias payload is:

interface AliasPayload {
  id: number
  instance: Instance
  alias: string
}

enum Instance {
  De = "de"
  En = "en"
  ...
}

In order to reference a payload type in tests or helper functions you can use the Payload<..., ...> helper from ~/internals/model. This helper takes two arguments: the name of the service model and the name of the model function for the endpoint. To refer to the alias endpoint you can write Payload<"serlo", "getAlias">.

Note: Normally the payload and the model type are the same and thus there is no need to to use Payload<...> in tests or resolver function. The usage of Model<...> is recommended. However some payloads / endpoints are only used internally and their results are never exposed in a GraphQL type (the alias payload is an example for this since the alias endpoint is only used to fetch the uuid of an resource defined by an alias). In those circumstances it is okay to use the payload type.

Sidenote: Behind the scenes Payload<...> is the same as the return type of the endpoint function (unpromisfied version and without null). In this example we have:

import { A } from 'ts-toolbelt'

type Payload<"serlo", "getAlias"> = NonNullable<
  A.Await<
    ReturnType<typeof dataSources.serlo.getAlias>
  >
>

How to add new endpoints to data sources

Service endpoints can be divided into two different types:

• query: Query requests ask for some data. They are "read" operations. Normally they occur a lot and thus their responses are cached by api.serlo.org so that following similar requests can be performed fast.
• nonecached requests: In some circumstances there are "read" operations which shall never be cached. We refer to those requests just as requests.
• mutation: Mutation requests ask to change / update some data or to create new objects. They are "write" operations. Thus they are not cached. However after a mutation often the cache need to be updated.

To create such endpoints we have the following helper functions (see the documentation in the source code to see how to use those functions):

• createQuery()
• createRequest()
• createMutation()

We also have a detailed example how you can add data source endpoints:

• How to add a mutation

How to add new data sources

You can just copy & paste the implementation of other data sources. Note that you need to register the new data source at https://github.com/serlo/api.serlo.org/blob/main/src/model/index.ts in the dictionary modelFactories.