Centrifugo v2 #221
Comments
|
Hey! Great news! |
|
@arrowcircle hi! Just wrote a chapter in new docs about API. In short - it's just a POST request with JSON body to I think most of the things are pretty final though after some feedback still can change a bit. |
|
So just to give some info about v2 status - at moment I am trying to solve two questions:
|
|
I removed GRPC bidirectional streaming client transport because:
It's still possible to put it back in future if we find its advantages in some scenarios. Note that GRPC for server API is still here. Also improved message recovery - new docs here https://centrifugal.github.io/centrifugo/server/recover/ |
|
Hello, Great work. After browsing through the code I have some thoughts (I have never used centrifugo before, I'm just know checking the project to see if it fits my use case). I see that the Engine is no longer pluggable.
My use case:I need to write a special presence handling. In v1 I could write a custom engine that has an embedded RedisEngine struct with the presence methods overriden. Now with the Engine interface methods being unexported it's no longer possible. An another note:It would be really nice to have a mute client in channel feature in the server API, resulting in that client not getting the messages. My use case:Free clients join a channel. One of them starts paying. This one client will receive slightly different notifications on that channel. I know it's possible to leave that channel and join another for the paying session, but it would need to be initiated on frontend, and it would be really nice to be able to solve this on backend only. An alternate solution could be to be able to add "except_clients" id list to publish/broadcast messages. |
|
Hello @masterada ! Thanks for a great feedback!
Yes, the reason here is that I don't know any other Engine implementations and their requirements so decided to approach this with caution. I.e. my final goal is to make engine interface fully exported and pluggable in Centrifuge lib - but I don't want to export things right now to not break public API later. So if someone interested in having engine exported we can find a proper way and moment to export it. Also see below.
As Centrifugo will now use Centrifuge lib it's not that difficult to plug whatever developer wants. From my point this makes library much more manageable and easier to maintain. Regarding Centrifugo server implementation: adding new plugin using code from
I also noticed this and I actually have secret gist regarding to this. The problem with 3 parts is that it generally looks cleaner and more flexible but not justified by reality where we only have 2 main Engines where everything done in memory or in Redis and this separation can be a bit overkill. If you look to my gist you will see that Btw, this topic about correct engine separation is one of the reasons I don't want to export Engine interface right now.
Could you elaborate more about this - why not using 2 different channels for this? Actually I thought many times about having server-side |
|
I'll try to elaborate more on my points above as some of my thoughts were pretty chaotic. As far as I understand you are suggesting to do sth like this: type Engine interface{
Broker
HistoryKeeper
PresenceKeeper
}Both Memory and Engine will implement all methods of In Node publish we can call: Instead of If you look at Publish method of Redis engine you will see that it publishes to channel and saves history in one RTT to Redis. This is a property I want to keep for Redis engine. First idea is making Regarding to muting and Regarding to server-side subscribe. It's possible to subscribe on server-side but client will not have callback handler set to process messages coming from channel. Also there is a question about message recovery - can't imagine how to fit it into this model - looks like this must be a task for application code in this case. |
|
Thanks for the detailed explanation. I completely understand you reasons for not wanting to sacrifice performance for a feature that's might not even needed (separating broker from history). About subscribing clients on server sideLet's assume a js client subscribed to a On the other hand, it might make sense to subscribe the client to a Message taggingDuring publishing a message, there is an optional So instead of:
You could do:
Or:
It would solve my use case as well:
It's not the same as subscribing the client from backend, but I think this could be easier to use than using multiple subscription from the client to get public and access restricted messages of the same type. Regarding Of course all this is just a suggestion. If you like it, I can help with the implementation. If you don't, it will still be a great project :D |
Possible solutionsStill not sure I understand your difficulties right.
TagsAt moment Centrifugo assumes identical data for each channel - this is especially important in terms of history/recovery. Though we can still keep full channel history but do server-side filtering based on tags before sending message or history data to client. Also Centrifugo designed in a way that all state required to subscribe comes from client side - so server can be just in-memory message proxy. Do I understand right that when you say about I suppose yes because If you mean server-side integration of Centrifugo and backend via hooks then this idea fails quickly (for example restart of node with Memory engine - we can afford losing message history but loosing tag information is critical). So the only way to keep tags is always pass them from client on subscription request. This is actually an interesting idea that theoretically can allow to do some interesting channel configurations. Adding more frontend-backend integration points (a-la private channel authorization now) will be hard to maintain in client libraries. So maybe this can be included into private channel subscription workflow? At moment we have channels starting with Channel remains the same, One caveat: this requires resubscription in case when user subscription tags must be changed (i.e. user becomes paid user and have to receive paid events). At this point this looks similar to point 2 from possible solutions section above. Though at least you can publish new events into one channel (though for me it seems not too bad to publish into 2 different channels). It's difficult to tell at moment that there are no other caveats that can be found when we try to add this in code - it's hard to keep everything in the head. So proof of concept required and also this should not affect performance when tags not used. And of course I need an understanding that tags solve a problem that is hard (or not performant) to solve without them. Or maybe find some other applications for this feature. Summary 😀It's pretty hard to discuss this on Github, because I have feeling that I still don't understand your use case right and suggesting unviable solutions:) What is your thoughts on my points here? If you feel that I don't understand you right then maybe we could discuss this in chat on Gitter. |
|
You are right, I didn't think about the issue of persisting tag information. I will try to clarify my circumstances:
The 2. means the only viable workflow is the following:
(This is very similar to your suggestions. ) In order to keep this simple, everything more than a one time subscription to 1 or more channels is unexceptable. I can of course solve this issue by providing my own library that wraps the centrifuge js client and adds the above mentioned functionality, hiding these details from the users of our platform (and by users i mean frontend developers). So to summarize:
With tags the following workflow could be implemented in the centrifuge server and client:
Could even hide tags feature from client library, by sending client an updated subscription data + sign (with tag info that the client doesn't need to know about), which it used to upgrade it's existing subscription (and for later reconnect if needed). But this complex client side logic might not be worth the feature. It's always a tough call to draw the line between features and simplicity :) One more thing that popped into my mind during writing this: have you considered using JWT? It's a standardized solution that encapsulates data and it's signature, basically used for the same purpose as you use the signs for. |
|
A quick question - from your post I did not understand - is one subscription to private channel is acceptable for you? I. e. 1 Subscription to channel that needs requesting backend for sign and possibly tags (the request to backend will be sent every time user subscribes) ? |
|
Yes, if it's a one time thing (eg: during site load on traditional websites, or opening a page in a single page application). What's not ok is handling the free/paid status change in the 3rd party code in any way. In other words: if the developer who uses our platform needs to write any code that reacts to the paid/free status change, that's not ok. I want to completly hide the fact from the platform's users that there is even a free/paid status. I want them to subscribe to one channel and keep processing the messages without caring about whether they are free or paid. If in the backend it's solved by 2 channels I don't care, I just want to hide this implementation detail completly. |
|
Yep, thanks! I considered JWT before - but it seemed hard to support it across languages. Actually Centrifugo was born before JWT gained its popularity. Now looks like there are tons of libs implementing RFC spec, so this looks reasonable. Though still needs a bit investigation as all libs has its own API to generate tokens - hopefully resulting string is spec compliant and Go server can verify and decode it despite of language that was used to generate it :) It seems also that using JWT will allow to simplify integration with Centrifugo where we don't have helper libraries and be more flexible when we want to add features to Centrifugo-specific data (like Back to tags. Adding more stuff to protocol like updating subscription state seems a very complex solution. It's possible to implement but you are right that it makes things more difficult and hard to debug. Sure there could be a better way. Some ideas:
Both approaches never guarantee delivery (as Centrifugo is at most once delivery transport) but should work in practice in normal circumstances. And actually your suggested approach updating subscription state has the same guarantees. Does this make sense for you? |
|
BTW this all can be paired with connection check mechanism to ensure valid client state. Update: no, this is wrong as connection check does not operate with subscriptions. |
|
I investigated JWT a bit - looks like it suits pretty well. Generated token in Python: jwt.encode({"user": "42", "exp": 121010101010, "tags": ["a", "b"]}, key="secret")Then decoded in Go: package main
import (
"fmt"
"github.com/dgrijalva/jwt-go"
)
type ConnClaims struct {
User string `json:"user"`
Info string `json:"info"`
Tags []string `json:"tags"`
jwt.StandardClaims
}
func main() {
s := "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyIjoiNDIiLCJleHAiOjEyMTAxMDEwMTAxMCwidGFncyI6WyJhIiwiYiJdfQ.fUoNhGoYgXwJd9D9K_hloFo0MkwUgQyIrDQJDN0Akp8"
token, err := jwt.ParseWithClaims(s, &ConnClaims{}, func(token *jwt.Token) (interface{}, error) {
return []byte("secret"), nil
})
if claims, ok := token.Claims.(*ConnClaims); ok && token.Valid {
fmt.Printf("%v %v %v", claims.User, claims.Tags, claims.StandardClaims.ExpiresAt)
} else {
fmt.Println(err)
}
}An interesting idea here is adding tags to user connection itself instead of subscription. This will allow to set tags on connect and filter publications based on user tags and not on subscription tags. This is less flexible in general but will allow to not use private subscriptions. The only problem here is updating tags on the fly. This is easy to do during connection check request. But to change tags immediately after they changed on backend some sort of signal required - maybe new API |
|
I read the centrifuge js lib code, and had the exact same tought - using refresh to update the tags, and an option to force client refresh from server api. I don't think it's an issue to have user scoped tags instead of subscription tags - it's still possible to prefix the tag name with the channel name if needed. It might be a good idea though to make guest tags configurable (a static list of tags that apply to guests). If you are looking at jwt, I suggest you check out go-jose instead. It implements all of jws, jwe and jwt (go-jwt only implements jws + jwt), even thought you will probably not need the jwe part. I also found it a bit easier to use. Here is an example of usage (parse+validate). We use jwt in php, go and nodejs, so far the only difficulty we ran into is that some libraries accept the key in base64 format (eg: php), while others use it as-is (eg: go). It caused us some headache :) |
|
@masterada I've created pull request to https://github.com/centrifugal/centrifuge (#6) with JWT support. I had time to think more about Imagine situation where From this perspective having tags information in private subscription token is more robust as private subscription token is asked every time client resubscribes. This means that on Centrifugo node restart there will be lots of private subscription requests after every client reconnect. But this is a reasonable compromise that we already had before, people use this and not everyone actually using private subscriptions. But to update tags on the fly some sort of signal required (disconnect/resubscribe maybe) and looks like subscription token refreshing on expiration is also a good idea. But this requires quite a lot work - not sure I can spend time for this at moment. But seems like it's possible to add at any moment later. So I am not sure about best way to add this feature yet. |
In Centrifuge case we have to handle token expiration in special way to support refresh workflow. I looked at go-jose and have not found a straighforward way to check that the only problem with token is that it's expired. |
|
I see your point about tags and refresh. Still, tags could be a private channel only feature. I decided I will use the centrifuge library in a new project, because I will need to change private channel subscriptions very often, and I think the short refresh interval I would need to do this with token style is more of a performance overhead than using backend webhooks from centrifuge server for authentication. I will try to include the minimal code to be able to support tags with the library, and create a pull request. But before that I need to dig in some more :) |
|
@masterada ok, feel free to ask any questions on Gitter and via personal messages if you prefer. As you can see I was able to implement subscription expiration - implementation is not ideal but I think it's pretty sufficient for this moment. |
|
Centrifugo v2.0.0-alpha.2 just released - this is first public pre-release, hope someone will give it a try and share feedback. |
|
Centrifugo v2.0.0-beta.1 released |
|
So Centrifugo v2 released - release notes are here. Thanks a lot to everyone who helped during development: @masterada @mogol @Inpassor @furdarius @wlredeye and others. There are still lots of things to do in transition to v2 - update remaining libraries, several examples still use v1, fixing bugs (sure there are some). But the important step just made:) |
Hello dear Centrifugers.
The work on Centrifugo 2 started in the end of 2017 and it's now almost done. It will serve the same purpose as Centrifugo v1 but won't be backwards compatible – migration to it will require adapting both backend and frontend sides of your application (of course if you decide to migrate). Changes are not too difficult. I will try to write more information later. For now you can look at post describing some of v2 aspects and reasons lead to some decisions. It's not fully actual at moment but the main ideas are the same.
Several highlights of v2:
GRPC client transport (not for browser)(see below)recoveredflag false positivesSome things were removed from Centrifugo in v2 release:
Some things you can help with as it's really hard to do everything myself:
All these tasks require you already familiar with Centrifugo or want to dive deeper as you need to understand how things work internally.
During next days I am planning to work on docs - most of them must be written from scratch so I don't know how much time it will take. Docs prototype located here. Centrifugo v2 itself is in c2 branch.
At moment I am looking forward for developers who are using Centrifugo and want to review Centrifugo v2 at its alpha and beta state. If you ever wanted something backwards incompatible to be added into Centrifugo core - this is the right moment to say. Please contact me here, over email or Gitter.
The text was updated successfully, but these errors were encountered: