Conform Zenoh spec to 1.5.8 #103

evshary · 2024-04-03T07:51:54Z

The PR is to deal with

The mapping of UUri and Zenoh Key: How UUri maps to Zenoh key #100
Using short UUri for Zenoh Key: Should we Bring Back Short Form URI? #83

Still need to wait for #92 before merging (I need to add some changes)

up-l1/zenoh.adoc

evshary · 2024-04-08T03:52:40Z

I've also updated the workflow and added the requirement of arguments in constructors.
Let me implement it in up-client-zenoh-rust first before merging it.

up-l1/zenoh.adoc

evshary · 2024-04-11T13:37:22Z

@gregmedd Thanks for your review. Feel free to advise me on how to make the spec clearer. Sometimes I might ignore some premise while writing down the spec.

stevenhartley

see comments

up-l1/zenoh.adoc

PLeVasseur · 2024-04-24T21:41:17Z

Hey @evshary -- today in the MQTT meeting when discussing @sophokles73's proposal for MQTT, I thought this could also serve a nice purpose for Zenoh as well.

The idea is to include both the source and the destination UAuthorities s.t. we can easily determine when a message is a publish vs notification message and bring the message through.

I think we'd also modify s.t. when we say we want to "subscribe to all", what we really mean is we'd like to subscribe to all Request, Response, and Notification messages, and none of Publish messages.

Publish messages will be handled on a topic-by-topic basis based on what uSubscription tells us to forward, so we do not have a need to get "all" Publish messages.

Essentially, I think that if we modify the Zenoh KeyExpr from:
up://origin-authority/origin-entity/origin-version/origin-resource.instance
to include the destination authority as well:
up://origin-authority/dest-authority/origin-entity/origin-version/origin-resource.instance
then I think we can disambiguate.

So based on the issue I raised in this PR, I have a proposal:

Setup

A uEntity (origin-authority/origin-entity) creates a message with

source: up://origin-authority/origin-entity/origin-version/origin-resource.instance#message
sink: -

Subscribing uE

Registers a listener on origin UUri

up://origin-authority/origin-entity/origin-version/origin-resource.instance#message

which UTransport.registerListener maps to Zenoh key

up/ + origin-authority-id/ + empty/ + origin-entity-id/origin-version/origin-resource-id

The MQTT proposal would have the destination as /, but I understand that // is not valid in a Zenoh KeyExpr. We'd need empty to be something reasonable and consistent.

Subscribing uStreamer

Registers a listener on special origin-authority UUri to catch out-going messages coming from Zenoh, intended for SOME/IP

up://origin-authority/*/*/*

which UTransport.registerListener maps to Zenoh key

up/ + */ + origin-authority-id/ + */*/*

uStreamer receives messages and forwards

uStreamer catches the following message on SOME/IP UTransport:

up://origin-authority/origin-entity/origin-version/origin-resource.instance#message

uStreamer then uses UTransport.send which maps to Zenoh Key:

up/ + origin-authority-id/ + empty/ + origin-entity-id/origin-version/origin-resource-id

which would now not match for the uStreamer's registerListener() because we have discriminated between Publish vs Notification messages via the two slots:

first the source authority
second the sink authority

Overall WDYT?

sophokles73 · 2024-04-25T06:52:16Z

Subscribing uStreamer
Registers a listener on special origin-authority UUri to catch out-going messages coming from > Zenoh, intended for SOME/IP

up://origin-authority/*/*/*
which UTransport.registerListener maps to Zenoh key

up/ + */ + origin-authority-id/ + */*/*

@PLeVasseur
If the origin-authority-id represents the source of the messages, then I do not see how this Zenoh Key could possibly match any messages that are intended for SOME/IP, because the third segment in the key represents the destination (sink) of the message, not the source.

I guess it would help if, in your example, you would use terms other than origin-authority but instead clearly distinguish between the non-SOME/IP authority vs the SOME/IP authority that you want to route between ...

evshary · 2024-04-25T13:27:46Z

@PLeVasseur I guess I missed some context. I still can't understand why we need to have both the source and sink authority in the Zenoh key. If we don't use this way to discriminate between Publish and Notification, will we face any problems in uStreamer?

Another question is that if we use up://origin-authority/dest-authority/origin-entity/origin-version/origin-resource.instance in Notification, it means that the one receiving notifications can only receive data from origin-authority in this Zenoh key. If I want to receive notifications that possibly come from A, B, or C, I need to subscribe to three Zenoh keys separately.

up://A/dest-authority/origin-entity/origin-version/origin-resource.instance
up://B/dest-authority/origin-entity/origin-version/origin-resource.instance
up://C/dest-authority/origin-entity/origin-version/origin-resource.instance

Or maybe I mistake something?

---Edit---
Maybe I got your point. Is this your problem?
eclipse-uprotocol/up-transport-zenoh-rust#31 (comment)

I'm thinking of whether using allowed_origin in subscriber can solve the problem or not.
https://github.com/eclipse-zenoh/zenoh/blob/0.10.1-rc/zenoh/src/subscriber.rs#L474

sophokles73 · 2024-04-25T14:58:58Z

Another question is that if we use up://origin-authority/dest-authority/origin-entity/origin-version/origin-resource.instance in Notification,

Not sure if you are talking about a UUri here or the Zenoh key created from a UUri?

In any case, a Notification message would contain a source UUri like
up://origin-authority/origin-entity/origin-version/origin-resource.instance
and a sink UUri like
up://sink-authority/sink-entity/sink-version/sink-resource.instance

The Zenoh uTransport might turn that into Zenoh key
up/origin-authority/sink-authority/sink-entity/sink-version/sink-resource.instance

On the receiving side, a uEntity would subscribe to up://sink-authority/sink-entity/sink-version/sink-resource.instance which the Zenoh uTransport would turn into key
up/*/sink-authority/sink-entity/sink-version/sink-resource.instance so that it can receive Notifications from all origins.

sophokles73 · 2024-04-25T15:21:09Z

The problem with all of this is that UTransport.register_listener currently does not support indicating if I want to register a listener for Publish or other types of messages. This, however, would be necessary in order for the transport impl to be able to create the proper Zenoh key to match on, i.e. whether to fill the authority into the source-authority or the sink-authority segment of the key.

We had this discussion weeks ago already but never really came to a conclusion. Maybe it is time to create a corresponding PR in up-rust to add that flexibility to UTransport.

WDYT? @PLeVasseur @evshary

PLeVasseur · 2024-04-25T16:03:41Z

We had this discussion weeks ago already but never really came to a conclusion. Maybe it is time to create a corresponding PR in up-rust to add that flexibility to UTransport.

WDYT? @PLeVasseur @evshary

Yeah I like the idea of being action-oriented here in some feature branches:

defining an updated register_listener() proposal for discriminating between Publish and Notification in up-rust
reflecting usage of new register_listener() API into up-client-zenoh-rust
seeing how we all feel about it
based on our thoughts and feelings, bringing a change to up-spec
merge up-rust & up-client-zenoh-rust changes from steps 1. and 2. (with any feedbacks from 3. & 4.)

WDYT @sophokles73, @evshary?

If sounds good, who would like to take 1. and do some API design?

PLeVasseur · 2024-04-25T16:06:44Z

@evshary

Maybe I got your point. Is this your problem?
eclipse-uprotocol/up-transport-zenoh-rust#31 (comment)

Yes, exactly.

I'm thinking of whether using allowed_origin in subscriber can solve the problem or not.
https://github.com/eclipse-zenoh/zenoh/blob/0.10.1-rc/zenoh/src/subscriber.rs#L474

There may be a Zenoh-specific way to solve this, but AFAIK this will also pose an issue to MQTT5, will it not @sophokles73?

And if so, it feels cleaner to be explicit in the implementation of register_listener() within up-rust (and more generally in the spec) on how to handle Notification vs Publish.

evshary · 2024-04-26T05:33:24Z

On the receiving side, a uEntity would subscribe to up://sink-authority/sink-entity/sink-version/sink-resource.instance which the Zenoh uTransport would turn into key
up/*/sink-authority/sink-entity/sink-version/sink-resource.instance so that it can receive Notifications from all origins.

OK, it makes sense to use * to listen to all notifications.

The problem with all of this is that UTransport.register_listener currently does not support indicating if I want to register a listener for Publish or other types of messages. This, however, would be necessary in order for the transport impl to be able to create the proper Zenoh key to match on, i.e. whether to fill the authority into the source-authority or the sink-authority segment of the key.

I always support the idea of introducing type while calling register_listener, just as we discussed it in the infamous #73
But, since we are also discussing the change of UUri now, how about introducing the type concept in UUri?
We can discriminate the requests and responses from others (publish & notification) based on the fact that they have special format in UResource.
Now we have:

request: name=rpc, instance=xxxx
response: name=rpc, instance=response

We can also have something like

publish: name=pub, instance=xxxx
notification: name=notify, instance=xxxx

WDYT, @sophokles73 @PLeVasseur

There may be a Zenoh-specific way to solve this, but AFAIK this will also pose an issue to MQTT5, will it not @sophokles73?

I know little about MQTT, but it seems possible.
https://stackoverflow.com/questions/66785306/paho-mqtt-client-how-to-ignore-messages-published-by-myself

sophokles73 · 2024-04-26T06:17:41Z

@evshary we are currently discussing to get rid of the long form completely and use numeric identifiers for the resource only. So, using such a naming convention would not really work. I also do not see, why we would need to introduce the message type into the UUri, given that a message already has the type in the UAttributes. I'd rather extend the UTransport interface to support specifying the message type. I'll give it a shot and create a corresponding PR in up-rust.

sophokles73 · 2024-04-29T14:21:33Z

I have created eclipse-uprotocol/up-rust#92 in the hope that this would allow the Zenoh client to use the source and sink in the Zenoh key ...

evshary · 2024-05-16T13:22:13Z

Hi @PLeVasseur @gregmedd
As a user and developer of up-client-zenoh, perhaps you can take a look at the changes and see whether it makes sense or not. I tried to align with the latest up-spec changes for 1.5.8. In the meantime, I'll implement it in up-client-zenoh-rust to see whether there is anything I missed.

Hi @sophokles73 To confirm some questions:

Will we decide to use unserved characters for authority name? If not, I'll add how to escape special characters for Zenoh key in the spec.
Is there any reserved word for authority name? I mean when we're using Publish, we need to keep sink authority empty and I'm not sure if this should be defined in uri spec or every protocol spec.

sophokles73 · 2024-05-16T14:49:54Z

Will we decide to use unserved characters for authority name? If not, I'll add how to escape special characters for Zenoh key in the spec.

Ah, I hadn't considered that yet. Let me check again ...

sophokles73 · 2024-05-16T14:50:44Z

Is there any reserved word for authority name? I mean when we're using Publish, we need to keep sink authority empty and I'm not sure if this should be defined in uri spec or every protocol spec.

Sorry, I do not understand your point. Can you elaborate?

PLeVasseur · 2024-05-16T14:58:30Z

Is there any reserved word for authority name? I mean when we're using Publish, we need to keep sink authority empty and I'm not sure if this should be defined in uri spec or every protocol spec.

Sorry, I do not understand your point. Can you elaborate?

@evshary is asking this because it is disallowed in a Zenoh key expression to have two slashes following one another, i.e. if it were a publish and in say, MQTT // could be used to represent that in the topic name, the same is not possible in Zenoh.

So I think the ask is -- is there some reserved or disallowed character or word in uProtocol that we could use instead? If say $ were disallowed in a uProtocol UAuthority, then a publish message when converted to a Zenoh key expression may instead contain /$/ to note that the sink UUri is empty.

sophokles73 · 2024-05-16T15:18:30Z

So I think the ask is -- is there some reserved or disallowed character or word in uProtocol that we could use instead? If say $ were disallowed in a uProtocol UAuthority, then a publish message when converted to a Zenoh key expression may instead contain /$/ to note that the sink UUri is empty.

Ok, the question is: how does the zenoh key look like for subscribing to Publish messages from source-authority, right?

Based on the proposal from a previous comment on this thread, it would be something like:
up/source-authority/{empty}/*/*/*

But {empty} cannot actually be empty, because Zenoh does not allow // in key expressions, right?

PLeVasseur · 2024-05-16T16:15:47Z

Yes, exactly. Thanks for wording it much better 😅

up-l1/zenoh.adoc

evshary · 2024-05-17T02:41:59Z

@evshary is asking this because it is disallowed in a Zenoh key expression to have two slashes following one another, i.e. if it were a publish and in say, MQTT // could be used to represent that in the topic name, the same is not possible in Zenoh.

@PLeVasseur Thank you for the clarification. This is exactly what I'm asking 😆
I didn't know that MQTT doesn't have this limitation before. What I can think of is that I can choose the special character which is not included in sub-delims to achieve the same effect. Or maybe I need to use escape character to indicate empty. I guess it's fine for me to define it in Zenoh spec, but we need to confirm the scope of the legal character in authority name first.

sub-delims    = "!" / "$" / "&" / "'" / "(" / ")"
                 / "*" / "+" / "," / ";" / "="

sophokles73 · 2024-05-17T12:34:13Z

We only allow the authority to be a host

host          = IP-literal / IPv4address / reg-name
reg-name      = *( unreserved / pct-encoded / sub-delims )

which means that you should be able to use any character from gen-delims

gen-delims    = ":" / "/" / "?" / "#" / "[" / "]" / "@"

What about #?

evshary · 2024-05-17T12:57:46Z

These are unavailable characters in Zenoh key

I'm thinking of using [] to escape the characters.
For example,

[] means empty
[question] means ?

etc.

sophokles73 · 2024-05-17T13:51:12Z

My understanding is that [] will overlap with the

IP-literal    = "[" ( IPv6address / IPvFuture  ) "]"

production and might be mistaken for a malformed IPv6 address?

However, the : seems to be unambiguous. Maybe you could use ::?

evshary · 2024-05-17T14:01:58Z

Oops, you're right. But IPv6 also uses :, for example ::1 means localhost.
Maybe using {} or \ might be better.

sophokles73 · 2024-05-17T15:21:02Z

Oops, you're right. But IPv6 also uses :, for example ::1 means localhost. Maybe using {} or \ might be better.

A corresponding IPv6 address would still look like [::1] but I guess {} is the best candidate yet as these characters are not allowed anywhere in URIs 👍

evshary · 2024-05-20T03:19:03Z

At first sight, I don't see any reason why we shouldn't further restrict the reg-name production used to contain

I'm fine with that. It's also easier to me.

A corresponding IPv6 address would still look like [::1] but I guess {} is the best candidate yet as these characters are not allowed anywhere in URIs 👍

Great! I'll use {} to indicate empty.

Here is the initial proposal on how to escape special characters. I'll update it if we decide to restrict the reg-name.
https://github.com/eclipse-uprotocol/up-spec/pull/103/files#diff-540fb00cfbe38ece62de18b3a95419f71516df5fb08cc89402bf33235411ae7fR140

sophokles73

In general, I wonder if you could simply use standard URL/Percent-Encoding for special characters?

up-l1/zenoh.adoc

sophokles73 · 2024-05-21T13:56:28Z

up-l1/zenoh.adoc

+The TTL only takes effect while sending RPC in the Zenoh implementation.
+TTL **MUST** be mapped to the timeout configuration in Zenoh query.


what about other messages that have timed out already on arrival? Would you still deliver those to registered listeners? Or send out messages in send() that have already timed out?

Maybe I missed something, but is there any way for us to know when the message is sent? How does the receiver side know the message is timed out only with TTL?

The UUIDv8 defined in the specifications includes the timestamp from when the request message was generated.

OK, I missed that.

what about other messages that have timed out already on arrival? Would you still deliver those to registered listeners? Or send out messages in send() that have already timed out?

Then we should check whether messages time out or not on arrival in the transport implementation. Let me update it.

Hmm... do you think we should drop the expired messages on the transport layer (Zenoh, MQTT..), or just passed to users and leave the decisions to them?

PLeVasseur

Thanks for continuing to pull this together @evshary!

Want to hear your feedback on a few points 🙂

up-l1/zenoh.adoc

PLeVasseur · 2024-05-21T20:23:13Z

up-l1/zenoh.adoc

@@ -253,7 +240,8 @@ The way to transfer Response message is different from others, because Zenoh can
 ==== Message routing

 1. A streamer that is interested in all incoming messages for dest-authority registers a listener for
-    * `up://dest-authority////`
+    * source: `-`


Can't leave the comment high up enough. I'm referring to this line.

With the separation coming between uP-L1 and uP-L2, I have a question: would the up-client-foo-rust libraries be responsible for / be expected to implement uP-L2? Or would they optionally implement uP-L2, but allow for the posibility of plugging in the UTransport implementation they made into e.g. the RpcClient "default" implementation we'd have in up-rust?

Tagging @sophokles73 in for this one as well to hear his thoughts.

PLeVasseur · 2024-05-21T20:42:23Z

up-l1/zenoh.adoc


 4. which `UTransport.registerListener` maps to Zenoh key
-    * local (w/o dest-authority): `upl/` + `dest-resource-id + dest-entity-id + dest-version`
-    * remote (with dest-authority): `upr/` + `dest-authority-id/` + `dest-resource-id + dest-entity-id + dest-version`
+    * `up/` + `+++*+++/dest-authority/dest-entity/dest-version/dest-resource`


I feel like this has been settled at this point, I just don't recall the conversation...

So for Notification messages, the latter parts of the Zenoh key (entity, version, resource) are pulled from the destination, not the source?

Would we want the ability to discriminate at the registerListener level as to which source the Notification is coming from? If that were the case, I suppose we'd have to put the latter parts (entity, version, resource) from the source and modify the registerListener example to show us using the source field.

So for Notification messages, the latter parts of the Zenoh key (entity, version, resource) are pulled from the destination, not the source?

Only Publish messages use source as the latter part of Zenoh key. Notification and Request only care about the destination.

Would we want the ability to discriminate at the registerListener level as to which source the Notification is coming from? If that were the case, I suppose we'd have to put the latter parts (entity, version, resource) from the source and modify the registerListener example to show us using the source field.

I don't quite get your point. Maybe retrieving the source information from uAttributes is enough?

stevenhartley · 2024-05-22T22:06:53Z

@evshary today on the MQTT call we concluded that we need to construct the MQTT topic including the full source and sink URI information so that we can register for notifications that require sink and source. We register a listener for notifications passing the source UUri and the destination UUri (sink) so that the message is routed directly to the uE that consumes the notification message.

Here is more information:

Based on our live discussion just now, we have concluded that in order to support notifications correctly (that will have source and sink addresses) we need to expand and update the MQTT5 topic to be the following:

{client_identifier}/{source.authority_name}/{source.ue_id}/{source.ue_version_major}/{source.resource_id}/{sink.authority_name}/{sink.ue_id}/{sink.ue_version_major}/{sink.resource_id}

Below are some examples using this updated notation:

Specific Publish Topic: d/{source.authority_name}/{source.ue_id}/{source.ue_version_major}/{source.resource_id}/////
All (incoming) requests to a specific Method (what a service would want to do): d/+/+/+/+/{sink.authority_name}/{sink.ue_id}/{sink.ue_version_major}/{sink.resource_id}
Streamer registering to receive any sent notifications, requests, or responses destine to its device:
d/+/+/+/+/{sink.authority_name}/+/+/0
Specific Notification topic: d/{source.authority_name}/{source.ue_id}/{source.ue_version_major}/{source.resource_id}/{sink.authority_name}/{sink.ue_id}/{sink.ue_version_major}/0

Signed-off-by: ChenYing Kuo <evshary@gmail.com>

evshary · 2024-05-23T07:24:32Z

@stevenhartley Updated. TBH, I'm not in favor of the complexity of Zenoh key or MQTT topic, but the majority rules.

I also tidied up the examples of Publish, Notification, and Request UUri mapping after referring to the MQTT spec.
@sophokles73 @PLeVasseur I would like to hear your comments. Is it clear enough to developers?

stevenhartley

LGTM

evshary mentioned this pull request Apr 3, 2024

MQTT5 Protocol spec #75

Merged

stevenhartley added this to the v1.0.0-alpha.1 milestone Apr 3, 2024

evshary mentioned this pull request Apr 3, 2024

Update up-client-zenoh-rust with up-spec 1.5.7 eclipse-uprotocol/up-transport-zenoh-rust#21

Merged

PLeVasseur reviewed Apr 3, 2024

View reviewed changes

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

PLeVasseur reviewed Apr 3, 2024

View reviewed changes

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

stevenhartley mentioned this pull request Apr 3, 2024

Update Zenoh Spec with work flow #92

Merged

stevenhartley assigned evshary Apr 4, 2024

stevenhartley added documentation Improvements or additions to documentation enhancement New feature or request labels Apr 4, 2024

evshary mentioned this pull request Apr 9, 2024

Convert Zenoh Client to Micro URIs eclipse-uprotocol/up-transport-zenoh-cpp#23

Merged

gregmedd requested changes Apr 11, 2024

View reviewed changes

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

evshary mentioned this pull request Apr 23, 2024

How to include Zenoh configuration into UPClientZenoh? #85

Closed

stevenhartley requested changes Apr 23, 2024

View reviewed changes

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

evshary commented Apr 24, 2024

View reviewed changes

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

stevenhartley added the v1.5.8 label May 16, 2024

evshary marked this pull request as ready for review May 16, 2024 12:06

evshary force-pushed the zenoh_uuri_mapping branch from 3d0ef34 to 388a24d Compare May 16, 2024 12:22

stevenhartley requested changes May 16, 2024

View reviewed changes

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

sophokles73 reviewed May 21, 2024

View reviewed changes

up-l1/zenoh.adoc Outdated Show resolved Hide resolved

sophokles73 reviewed May 21, 2024

View reviewed changes

PLeVasseur reviewed May 21, 2024

View reviewed changes

Update Zenoh spec to conform with up-spec 1.5.8

5c8f685

Signed-off-by: ChenYing Kuo <evshary@gmail.com>

evshary force-pushed the zenoh_uuri_mapping branch from b40b30d to 5c8f685 Compare May 23, 2024 03:45

evshary changed the title ~~Update how UUri maps to Zenoh key.~~ Conform Zenoh spec to 1.5.8 May 23, 2024

Include all source and sink information in Zenoh Key.

1a439b0

Signed-off-by: ChenYing Kuo <evshary@gmail.com>

stevenhartley approved these changes May 23, 2024

View reviewed changes

gregmedd approved these changes May 23, 2024

View reviewed changes

stevenhartley merged commit 05fdd5d into eclipse-uprotocol:main May 23, 2024
1 check passed

evshary deleted the zenoh_uuri_mapping branch May 24, 2024 02:02

		The TTL only takes effect while sending RPC in the Zenoh implementation.
		TTL MUST be mapped to the timeout configuration in Zenoh query.

Conform Zenoh spec to 1.5.8 #103

Conform Zenoh spec to 1.5.8 #103

Conversation

evshary commented Apr 3, 2024

evshary commented Apr 8, 2024

evshary commented Apr 11, 2024

stevenhartley left a comment

Choose a reason for hiding this comment

PLeVasseur commented Apr 24, 2024 • edited Loading

Setup

Subscribing uE

Subscribing uStreamer

uStreamer receives messages and forwards

sophokles73 commented Apr 25, 2024 • edited Loading

evshary commented Apr 25, 2024 • edited Loading

sophokles73 commented Apr 25, 2024 • edited Loading

sophokles73 commented Apr 25, 2024

PLeVasseur commented Apr 25, 2024

PLeVasseur commented Apr 25, 2024

evshary commented Apr 26, 2024

sophokles73 commented Apr 26, 2024

sophokles73 commented Apr 29, 2024

evshary commented May 16, 2024

sophokles73 commented May 16, 2024

sophokles73 commented May 16, 2024

PLeVasseur commented May 16, 2024 • edited Loading

sophokles73 commented May 16, 2024 • edited Loading

PLeVasseur commented May 16, 2024

evshary commented May 17, 2024

sophokles73 commented May 17, 2024

evshary commented May 17, 2024

sophokles73 commented May 17, 2024 • edited Loading

evshary commented May 17, 2024

sophokles73 commented May 17, 2024 • edited Loading

evshary commented May 20, 2024

sophokles73 left a comment

Choose a reason for hiding this comment

sophokles73 May 21, 2024 • edited Loading

Choose a reason for hiding this comment

evshary May 21, 2024

Choose a reason for hiding this comment

stevenhartley May 21, 2024

Choose a reason for hiding this comment

evshary May 22, 2024

Choose a reason for hiding this comment

evshary May 22, 2024

Choose a reason for hiding this comment

PLeVasseur left a comment

Choose a reason for hiding this comment

PLeVasseur May 21, 2024

Choose a reason for hiding this comment

PLeVasseur May 21, 2024

Choose a reason for hiding this comment

evshary May 22, 2024

Choose a reason for hiding this comment

stevenhartley commented May 22, 2024

evshary commented May 23, 2024 • edited Loading

stevenhartley left a comment

Choose a reason for hiding this comment

PLeVasseur commented Apr 24, 2024 •

edited

Loading

sophokles73 commented Apr 25, 2024 •

edited

Loading

evshary commented Apr 25, 2024 •

edited

Loading

sophokles73 commented Apr 25, 2024 •

edited

Loading

PLeVasseur commented May 16, 2024 •

edited

Loading

sophokles73 commented May 16, 2024 •

edited

Loading

sophokles73 commented May 17, 2024 •

edited

Loading

sophokles73 commented May 17, 2024 •

edited

Loading

sophokles73 May 21, 2024 •

edited

Loading

evshary commented May 23, 2024 •

edited

Loading