wslink grew out of several needs and pressures, and I've collected information and notes about the background and design here.
ParaViewWeb uses autobahn WAMP protocol for RPC and publish/subscribe messages as of May 2017. Due to changes to autobahn's WAMP implementation, we can't upgrade autobahn and continue using WAMP. We require:
- RPC - a remote procedure call that can be fired by the client and return sometime later with a response from the server, possibly an error.
- Publish/subscribe - client can subscribe to a topic provided by the server, possibly with a filter on the parts of interest. When the topic has updated results, the server publishes them to the client, without further action on the client's part.
We would also like:
- Real binary messages - WebSockets support binary messages, and one of our major use cases is publishing images (rendered frames). WAMP only supports base64-encoded binary objects.
- Other webservers - WAMP has been implemented elsewhere, but in JavaScript the only implementations require autobahn's router, crossbar.io. It would be great to support Tornado and CherryPy as alternative webservers to Twisted/autobahn.
jsonrpc has a well-defined, simple, and easily implemented specification for using JSON to do RPC. It is transport agnostic, and language independent, so we can use it between javascript and python. There are many implementations, but we are free to use them or ignore them. JSON can represent four primitive types (Strings, Numbers, Booleans, and Null) and two structured types (Objects and Arrays). We want to extend our messages to handle binary objects, inserted into JSON dict/object, discussed below.
Websockets allow bi-directional communication between the server and client. Authentication needs to be addressed.
Because we wish to support pub/sub, we make the client and server more symmetric than in the jsonrpc spec - the server can publish to the client after a subscription is made, so in jsonrpc terms, the server is making an RPC call to the client. So our server and client must both be able to make and receive jsonrpc calls.
Small examples have proven websocket and binary message support in twisted, tornado, and cherrypy, so if we can abstract the webserver sufficiently, we may be able to support all of these.
Existing ParaViewWeb applications use these code patterns:
- @exportRPC decorator in Python server code to register a method as being remotely callable
- session.call("method.uri", [args]) in the JavaScript client to make an RPC call. Usually wrapped as an object method so it appears to be a normal class method.
- session.subscribe("method.uri", callback) in JS client to initiate a pub/sub relationship.
- server calls self.publish("method.uri", result) to push info back to the client
We don't support introspection or initial handshake about which methods are supported - the client and server must be in sync. Could the server supply the client with a list of RPC methods it supplies? Yes, but then the client couldn't operate unless it's connected to a server - it would call undefined methods, rather than calling those methods and getting a 'not connected' error. Maybe not a concern.
The 'session' object is provided by Autobahn|JS WAMP, so we need to replace it.
An old wiki page lists implementations - most are transport/server specific.
For Python, Tinyrpc has a parser for messages that will do validity checking - seems useful.
For Javascript, jrpc extends jsonrpc to be bi-directional, and includes a handshake to upgrade from standard jsonrpc. RaptorRPC might be interesting.
Message format:
{
const request = {
wslink: 1.0,
id: `rpc:${clientId}:${count}`,
method: 'render.window.image',
args: [],
kwargs: { w: 512, h: 512 }
};
const response = {
wslink: 1.0,
id: `rpc:${clientId}:${count}`,
result: {}, // either result or error, not both
error: {}
};
// types used as prefix for id.
const types = ['rpc', 'publish', 'system'];
}// add a binary attachment
def getImage(self):
return {
"size": [512, 512],
"blob": session.addAttachment(memoryview(dataArray)),
"mtime": dataArray.getMTime()
}We would like to support kwargs, like wamp, which violates pure jsonrpc. We can extend it by simply adding a 'kwargs' param, as above. Therefore we'll use 'wslink' as our version string, instead of 'jsonrpc'. We also change from 'params' to 'args'.
AutobahnJS session.call() returns a Promise, except that IE doesn't support
promises, so it uses an alternative if needed. node_modules/autobahn/lib/session.js uses self._defer() to retrieve it. connection.js has the factory.
ParaViewWeb uses babel-polyfill, which includes a Promise implementation.
We can use the same defer() pattern, where we store the resolve, reject functions so we can call them when the message response is received.
session.addAttachment() takes binary data and stores it, returning a string key that will be associated with the attachment. When a message is sent that uses the attachment key, a binary message is sent beforehand with the attachment. The client can then substitute the binary buffer for the string key when it receives the final message.
Now sending a text header message for each binary message to associate it with a key.
The client needs to know about subscriptions - the server can blindly send out messages for any data it produces which might be subscribed to. This is not very efficient - if the client notifies the server of a subscription, it can send the data only when someone is listening.
When the client initially connects, it can authenticate with the server, so the server knows this client can handle the messages it sends, and the server can provide the client with a unique client ID - which the client can embed in the rpc "id" field of it's messages to the server.
- The first message client sends should be hello, with the secret key provided by it's launcher.
- Server authenicates the key, responds with the client ID.
- If the client doesn't send a key, the server can choose to serve an un-authenticated client, or respond with an authentication error message.