Should Registration be strictly tied to a server endpoint ?

[sbernard31]

With new Transport Layer Abstraction, a sever can have several endpoints (opened socket address) that client can use to connect to it.
A new field lastEndpointUsed was added to the Registration to know which server endpoint is used by the client.

When I created this field, I had in mind that maybe client could connect to another server endpoint and just do a registration update.
But finally, I think that this could lead to some issues (e.g. for observation) and I'm not even sure if this is allowed by LWM2M specification.

Maybe it would be better to strictly tied the registration to only 1 server endpoint. In that case, if client want to use another server endpoint, it must create a new registration.

Do you have any opinion on this ? (or knowledge about this part of the specification)

If we agree that registration must be tied to only one server endpoint, so we probably need a better name ?
endpointUsed, serverEndpoindUsed any idea ?

[jvermillard]

I agree with your statement. I don't see a use case for a device to move its registration from one endpoint to another without registering (especially if the two endpoints are of very different natures, UDP vs non-IP).

As a side note: do we need to use the URI class to identify the endpoint?
https://github.com/openjdk/jdk/blob/master/src/java.base/share/classes/java/net/URI.java#L519-L543
could we replace it with something lighter like a good old String

[sbernard31]

I agree with your statement. I don't see a use case for a device to move its registration from one endpoint to another without registering (especially if the two endpoints are of very different natures, UDP vs non-IP).

👍

As a side note: do we need to use the URI class to identify the endpoint?

For the M11, I think clearly it's too late, for the M12 maybe.
I decide to use URI everywhere to identify endpoint because this allow to easily access to protocol scheme or host + port with getter of this class and this also valid that String is a Valid URI.
What are the main drawback about using URI ?

[jvermillard]

The memory usage:

    // Components of all URIs: [<scheme>:]<scheme-specific-part>[#<fragment>]
    private transient String scheme;            // null ==> relative URI
    private transient String fragment;

    // Hierarchical URI components: [//<authority>]<path>[?<query>]
    private transient String authority;         // Registry or server

    // Server-based authority: [<userInfo>@]<host>[:<port>]
    private transient String userInfo;
    private transient String host;              // null ==> registry-based
    private transient int port = -1;            // -1 ==> undefined

    // Remaining components of hierarchical URIs
    private transient String path;              // null ==> opaque
    private transient String query;

    // The remaining fields may be computed on demand, which is safe even in
    // the face of multiple threads racing to initialize them
    private transient String schemeSpecificPart;
    private transient int hash;        // Zero ==> undefined

    private transient String decodedUserInfo;
    private transient String decodedAuthority;
    private transient String decodedPath;
    private transient String decodedQuery;
    private transient String decodedFragment;
    private transient String decodedSchemeSpecificPart;

[sbernard31]

When I was coding this feature, I change several times from String to URI / URI to String and finally stopped on URI.

Maybe URI is overkill. After the M11, I can try to use a String again and if we need some method to extract data maybe we can create a dedicated class ?

[sbernard31]

If we agree that registration must be tied to only one server endpoint, so we probably need a better name ?
endpointUsed, serverEndpoindUsed any idea ?

Any ideas ?

[vonWojtas]

Good morning from Germany!
I would like to join this discussion because we are at a state right now, where we would like to run multiple instances of our Leshan server, but make this transparent for the client.

I wonder why we need to store with what endpoint the client has spoken to?

In addition maybe you could help with your opinion.
We actually think it's sufficient for us to use i.e. hazelcast to distribute the registration store and whenever we need to e.g. push data, we grab it from the distributed store.
From a CoAP point of view it's fine, because we just need IP and Port.
The client has registered with us, sees our IP (which is masked away anyway :D)

What do you do to scale horizontally your server applications?

[sbernard31]

Good morning from France 😉

I would like to join this discussion because we are at a state right now, where we would like to run multiple instances of our Leshan server, but make this transparent for the client.

Unfortunately there is no easy way to solve this. You could have a look at : https://github.com/eclipse-leshan/leshan/wiki/Using-Leshan-server-in-a-cluster

I wonder why we need to store with what endpoint the client has spoken to?

To know which endpoint need to be used to send request.

In addition maybe you could help with your opinion.
We actually think it's sufficient for us to use i.e. hazelcast to distribute the registration store and whenever we need to e.g. push data, we grab it from the distributed store.
From a CoAP point of view it's fine, because we just need IP and Port.
The client has registered with us, sees our IP (which is masked away anyway :D)

I didn't know hazelcast and I'm not sure I get your point.

What do you do to scale horizontally your server applications?

Already answer above ☝️
If you want to discuss more about scaling Leshan you should maybe create a dedicated issue because I feel this is not really the topic here.

[vonWojtas]

Hey, thank you for the quick answer!

To know which endpoint need to be used to send request.

But why actually? In the end the CoAP message just contains a IP and Port, correct? Who has send it, is not interesting for the client (at least, this is what I think :D )

hazelcast is a distributed store in Java. Kind of similar to Redis, just without an extra storage and transparent java-map semantics.

Thank you very much for pointing us into the right direction already!
When necessary I will create a new issue!

[sbernard31]

But why actually? In the end the CoAP message just contains a IP and Port, correct? Who has send it, is not interesting for the client (at least, this is what I think :D )

From a client point of view :

Generally who sent data is very important for the client.

There is often constraint about this in RFC/specification of different protocol,e.g. for CoAP :

The exact rules for matching a response to a request are as follows:

   1.  The source endpoint of the response MUST be the same as the
       destination endpoint of the original request."_

In production, this is really rare that you don't care especially for security reason. (Of course for CoAP without DTLS this is maybe less true but this is also rarely used in production)

From a server point of view :

A Leshan server can be created with different endpoints.
Currently we have 2 kind of endpoint provider for coap, 1 for coaps and we experiment coap+tcp.

If a client talks to server using a coap endpoint we should not answer with coap+tcp or coaps endpoint.

Even if you have only coap endpoints, maybe you don't configure it in the same way. Maybe you set different ACK_TIMEOUT ACK_RANDOM_FACTOR MAX_RETRANSMIT value and so you need to use the right endpoint.

[gcx-seb]

One more question regarding this… You say

Maybe it would be better to strictly tied the registration to only 1 server endpoint. In that case, if client want to use another server endpoint, it must create a new registration.

How does that play together with the concept of distributed registration stores (like the RedisRegistrationStore for example)? Because the “find endpoint to use” logic in DefaultDownlinkRequestSender doesn't work when trying to send something from an instance other than the one that received the registration (results in a NPE here).

[sbernard31]

How does that play together with the concept of distributed registration stores (like the RedisRegistrationStore for example)?

Distributed registrations store can mean a lot of different thing. 🤔
But until now, I understand that means having several leshan server instances sharing same registration stores where all the instances have exactly same endpoints available. (if more use case need to be supported we need to discuss more)

Example if server1 has endpoints :

• coap://0.0.0.0:5683
• coap+tcp://0.0.0.0:5683
• coaps://0.0.0.0:5684

Then server2 has those endpoints too.

And so this should work like before we add "Transport Abstraction Layer".

Because the “find endpoint to use” logic in DefaultDownlinkRequestSender doesn't work when trying to send something from an instance other than the one that received the registration (results in a NPE here).

If the endpoint doesn't exist, I'm not sure we can find a working solution.
But maybe the code could be adapted to something better than raising an NPE.
We need to define which behavior we want, any idea ?

[gcx-seb]

Ah, thanks for the clarification, that brings some light I guess 🙂

that means having several leshan server instances sharing same registration stores

Yes, this is what I meant.

server2 has those endpoints too

That's probably the crucial point here… because this is not the case in my test setup. The instances do not use the same listen port (cause they're located on the same machine). But now as I understand this it is a requirement for multiple Leshan server instances to share the same listen port…

[sbernard31]

The instances do not use the same listen port

Maybe it's possible to make this use case work but I never thought about it. 🤔

Maybe given an identifier to endpoint at creation ?

[gcx-seb]

This wasn't meant as feature request 😉 I guess it's fine to use the same listen port when deployed on separate machines…

[sbernard31]

(About replace usage of java.net.URI class, I created a dedicated issue :#1582)

[sbernard31]

This is implemented by :

• Ensure that client use same endpoint for a given registration #1655
• Strongly tie a observation to a server endpoint #1656

Please, if you are interested about this look at both PR description to know consequences of those changes.