API Discovery

[georgkrause]

The problem

We want to allow different software to implement the API. Looking at the server software there might be the case that they already have a API path pattern. If we now define global paths that need to be available for that API, the software might be forced to move away from its schema or applying dirty hacks.

If we for example take the Funkwhale API, all paths start with /api/v1. If we now declare endpoints like /openpodcastapi/subscriptions/..., it might break the pattern.

Solution

In order to solve this I'd like to allow the server to either

• declare a custom prefix
• Move each endpoint wherever they want to.

If the client now wants to call endpoints for a specific server, it needs to discover those endpoints. Therefore we could place a json file inside .well-kown containing the prefix or each endpoints URL. (See https://en.wikipedia.org/wiki/Well-known_URI). This would allow the client to fetch those information from a static location using the information to actually talk to the API.

Example

Lets say we go the prefix way. The open podcast api specifies the following endpoints (just examples):

• /subscriptions
• /episodes

Now a random server software implements the API. It places a file /.well-known/openpodcastapi.json with the following content:

{
  "prefix": "/api/openpodcast/"
}

The client can now assemble the actual endpoint URLs:

SCHEMA://HOSTNAME/PREFIX/ENDPOINT

In our example to fetch the subscriptions:

https://some-server.org/api/openpodcast/subscriptions

Open Questions

• How much flexibility do we need? Is the prefix sufficient or do we need to have flexible Endpoint URLs?

[keunes]

With my limited technical understanding, I can only call on others.
@ByteHamster Any thoughts?
@MartinMouritzen, if you were potentially to implement synchronisation support into Podfriend, would you then need flexible Endpoint URLs in order to reuse some of the internal APIs you undoubtedly already have?

[ByteHamster]

Having it in .well-known sounds reasonable

[MartinMouritzen]

For me, it would be nice with a sane default, for simplicity. If you then want to change it, that's totally awesome that you can :)

[JonOfUs]

Using .well-known for the discovery sounds like a good approach.
I have a few thoughts on this:

• It is always good to version API specs, so we should think of this when building the discovery as well. The full endpoints could then look like this: SCHEMA://HOSTNAME/PREFIX/VERSION/ENDPOINT. Since we won't be able to version the discovery, we'll need to be especially careful while building it so that the spec remains backwards compatible.
  We'll then also need an endpoint about supported versions (e.g. a json array that contains all supported versions). matrix does this after the discovery, it has an endpoint https://$host/$prefix/versions. This might be our way-to-go as well, since this way it wouldn't even be necessary to put these info inside the .well-known file. That way we could take care of that endpoint at a later time.

• I never heard of a .well-known that has a file type extension, maybe /.well-known/openpodcastapi is enough?

• It should maybe also be optional to have the prefix in .well-known (e.g. if someone can't get .well-known to work). We could have a priority list of prefixes to use like:

  1. User inputs specific hostname+prefix (e.g. https://example.com/opapi)
  2. Prefix from .well-known
  3. Default prefix (e.g. https://example.com/openpodcastapi or https://example.com/_openpodcastapi)

• I don't think we'll need completely flexible endpoints. If we have to change endpoints, this should probably be done in a different version, so backwards compability is given. So a prefix should be enough in my opinion.

[keunes]

We'll then also need an endpoint about supported versions (e.g. a json array that contains all supported versions). matrix does this after the discovery, it has an endpoint https://$host/$prefix/versions. This might be our way-to-go as well, since this way it wouldn't even be necessary to put these info inside the .well-known file. That way we could take care of that endpoint at a later time.

Should we create a new Discussion for this so we remember later on? Or is it so straight-forward that it's not needed?

[JonOfUs]

Probably a good idea, since we'll need to define the json format of that endpoint as well (and maybe someone has a different opinion or more ideas).

[keunes]

Done. See https://github.com/orgs/OpenPodcastAPI/discussions/11

[georgkrause]

It is always good to version API specs, so we should think of this when building the discovery as well. The full endpoints could then look like this: SCHEMA://HOSTNAME/PREFIX/VERSION/ENDPOINT. Since we won't be able to version the discovery, we'll need to be especially careful while building it so that the spec remains backwards compatible.
We'll then also need an endpoint about supported versions (e.g. a json array that contains all supported versions). matrix does this after the discovery, it has an endpoint https://$host/$prefix/versions. This might be our way-to-go as well, since this way it wouldn't even be necessary to put these info inside the .well-known file. That way we could take care of that endpoint at a later time.

I think versioning is out of scope here, it will happen after the discovery as you already stated. In my simplified example the versioning would be part of the ENDPOINT. Lets move this to another discussion.

I never heard of a .well-known that has a file type extension, maybe /.well-known/openpodcastapi is enough?

Well, there is security.txt at least. Omitting the file type extension introduces some ambiguity I'd like to avoid, I am not settled on json though.

It should maybe also be optional to have the prefix in .well-known (e.g. if someone can't get .well-known to work). We could have a priority list of prefixes to use like:

Thats a pretty good one, I'll add this to my draft. What do you think about having some kind of DNS-based discovery as well?

I don't think we'll need completely flexible endpoints. If we have to change endpoints, this should probably be done in a different version, so backwards compability is given. So a prefix should be enough in my opinion.

Well, its not about changing the endpoint for us, its about giving server implementations the option to change it while still complying to the spec. This might be needed when there are path clashes or restrictions (for example for Nextcloud Plugins: As far as I am aware the prefix is given). However, this can be fixed with a prefix already and flexible endpoints probably introduce unneeded complexity.

[JonOfUs]

I think versioning is out of scope here, it will happen after the discovery as you already stated. In my simplified example the versioning would be part of the ENDPOINT. Lets move this to another discussion.

Agreed.

Well, there is security.txt at least. Omitting the file type extension introduces some ambiguity I'd like to avoid, I am not settled on json though.

Well, then /.well-known/openpodcastapi.json sounds good.
Since we want to use json as the data exchange format for the spec, it sounds good to also use json in the discovery. Less parsing libraries for clients.

What do you think about having some kind of DNS-based discovery as well?

You mean RFC 6763? I never heard of it, actually. A disadvantage could be that clients would need to implement this as well. But if it has great advantages...

However, this can be fixed with a prefix already and flexible endpoints probably introduce unneeded complexity.

Yes, that's what I thought too. I can't think of a use case where a prefix couldn't be enough, except for endpoints used by different services. But if they even share the same exchange format, then it'll also be possible for them to share the endpoint structure, I think.

[georgkrause]

You mean RFC 6763? I never heard of it, actually. A disadvantage could be that clients would need to implement this as well. But if it has great advantages...

I didn't even knew there is a spec for it, I just thought if we already have a priority list with different options, reading the prefix from a DNS TXT entry wouldn't hurt.

[JonOfUs]

I don't know how much implementation effort this is, if we write it into the spec, all clients would have to implement it. And having too much possibilities for discovery could lead to clients not implementing some of them. Maybe /.well-known is enough for discovery.

[georgkrause]

I agree. I was just considering it because people might not be able to place a .well-known in some cases. For example for Nextcloud this might be impossible for a plugin, but that is just a guess. But the admin might be able to place a DNS record.

[JonOfUs]

What do you think of the possibility of absolute addresses alongside relative ones for the prefix? So that all of the following examples would become possible:

• "prefix": "https://openpodcast.example.com/api/"
• "prefix": "openpodcast.example.com/api"
• "prefix": "/openpodcastapi/"

(maybe alongside a different name for the field)

This could be useful if a service uses a subdomain for the api. It would then only need to share its primary domain and put the /.well-known/ on it.

[georgkrause]

I think its a good idea to support the first and the third option. The second one seems to have some issues:

• What protocol should be used?
• We cannot reliable decide if its a path or a TLD (unless the /at the beginning would be the differentiation, but //openpodcast.example.com is a valid URL, too, so it is getting complex)
• It doesn't really add advantages over the absolut URL