XYZ in a nutshell
The initiative idea of node XYZ was originally derived from a simple presentation regarding Microservice software architecture. Immediately after the speech I realized "I basically described a system with properties that are very coherent with what Node has to offer. This is the opportunity that I've been waiting for, writing a module/framework for Node". Henceforth, I started to think about how this specific architecture (microservices)can be implemented in Node, which tools are required and what needs to be done.
It is worth mentioning that at the time I was working on an actor-based system using Akka. Thus, I tried to embed some of the useful aspects of Actor based systems in my initial design.
The following sections describes some of the philosophy behind XYZ, what are its aims and how it (would) achieve them.
XYZ should be able to wrap any Node Process, without having any effect on the ongoing main process, giving it the ability to communicate with other Node processes having the same wrapper. This is the minimal idealogical concept behind XYZ. That is to say, an individual service in a microservice based system would probably have several incoming open ports, allowing foreign services (not clients necessarily) to request data/computation, but, it would be totally independent of that process's main function. a process should be able to run a standalone backend/API, meanwhile being connected to the global xyz system offering an entirely different functionality (although this is not preferred according to the microservice architecture)
Technically, Any Node process will wrap XYZ module by importing it. For some reasons unknown to me, I prefer referring to it the other way around.
Let's step back for a moment and see how this would look like in a bigger scenario.
Indeed, a moderated system to manage all of the nodes in a large scale system is something worth having. An important note is that XYZ does not intend to do anything regarding the user, end-user to be more specific. Our users are developers and XYZ serves developers. In other words, APIs exposed by XYZ infrastructure are designed for communication between servers, not clients and servers.
Now, let's dive into one of these red circles (aka XYZ-core module) and see what it has to offer.
XYZ-core is designed in a layer based architecture, very similar to how the network that you are using to view this wiki page is functioning.
The Service Repository layer is responsible (as the name suggests) for exposing local node's services, and keeping track of services that other nodes in the system are offering. Obviously, this also includes executing a local service and passing the response to the caller, and executing a foreign service and passing the result to the local caller (which is executed by making a call to the Transport Layer).
unlike the previous sections, the term Node was used in the last paragraph to define a host (one of those small boxes in the big picture). To avoid confusion, it's useful to define some terms in the XYZ ecosystem:
A function exposed by a node process using (using xyz-core module's registerService()) is called a Service. A Service can have a response for the caller, which will force the caller to wait asynchronously, or it can have no response and close the incoming message's connection immediately.
A service is identified by a Path. A Path is a sequence of names (letters/numbers) concatenated with a slash, just like any other path in a system. An important note is that these paths are to facilitate better grouping of services, nonetheless, they do not compel a hierarchy for services, they can be used optionally with any manner according to the application's need.
Any Node Process composing a xyz-core instance is referred to as a Node (Due to the potential confusion between Node=a Node in a distributed system, and Node=NodeJS Programming language, this name might change to xNode in the near future).
A Node is identified by one major identifier:
- An IP address and a port. Note that this is the incoming HTTP/TCP port of the Node, the one that other Nodes can connect to and send messages to. A minor additional outgoing port is also used to send messages to other Nodes (which at the time we do not force any specific port for it, because of technical difficulties, and let the OS choose it), therefore Nodes need to introduce themselves to other Nodes when sending a message by carrying their primary IP:PORT signature.
and two minor ones used mostly for logging and monitoring purposes:
- The name of the service (filename, module name etc.)
- A unique id
Overall, a Node (xNode) would look something like :
[ServiceName]~[uuid] @ [IP] : [PORT]/
assuming that we have exposed two services inside a Node by RegisterService() :
xyz.registerService('/math/add', () { ... })
xyz.registerService('/math/mul', () { ... })
we have :
[ServiceName]~[uuid] @ [IP] : [PORT]/math/mul
[ServiceName]~[uuid] @ [IP] : [PORT]/math/add
The Transport Layer exists to serve as a communication infrastructure to Service Repository layer. After dealing with Service related logic, The service repository passes all the required information to the Transport Layer (including any callback function) and goes back to doing its main task: Dealing with Services. All of dirty, event-driven code and functions required for HTTP calls and callbacks are left for the Transport layer.
Ideally, the Transport layer should work on both HTTP and pure TCP and expose the same functions, in a way that the transport technology would be transparent to the Service Repository.
As mentioned above, XYZ is aiming to be as configurable as possible. One of the main features that embodies this philosophy is the Middleware Stack.
The most common scenario in XYZ ecosystem is Node communication, one node sending a message to another node and waiting for the response. Many operations in a Service call are performed using middleware stacks. This enables us to build our source code in a modular way, meanwhile give the developer the configurability.
Assume that Node A wants to send a message to Node B. The following stacks are used.
- Service Level CallDispatchMiddlewareStack : Receives a call command from service repository and after applying any preprocessing on, passes it to the transport layer. Naturally the last index in this middleware must be a function determining the destination Node. A default approach would be to use a first-find approach. Other possibilities would be send-to-all etc. Note that this stack might be replaced with single runtime-configurable function.
- The Transport Layer will pass the request object, with respective payloads and parameters to a Transport Layer CallDispatchMiddlewareStack. This middleware can be used to apply a modification to all requests sent to other Nodes (like adding an authentication signature/header). Again, naturally, the last index of this middleware stack must be the actual transmission of the request aka (
call.dispatch.export.middleware) - On The receiving side, the Transport Layer will receive the request, perform basic operations, like identifying its type, and, if the type was determined to be a foreign Node call, passes it to the Transport Layer CallReceiveMiddleware. This middleware can act as an additional post-processor for request, rejecting some of them or manipulating them before reaching the Service Repository Layer. The last index of this middleware stack is
call.receive.event.middleware, which emits a call receive signal on the Transport layer.
All of these events happen separately with each Ping request, which is not visible to the developer.
Summery:
