11 Zinx Decoder

1. LengthField Frame Decoding

Zinx provides encoding and decoding capabilities for binary data frames during the data transmission process. With Zinx's TCP stream transmission, developers don't need to worry about packet fragmentation and packet reassembly issues because Zinx handles them internally.

Zinx's binary encoding and decoding algorithm adopts the LengthField algorithm. Developers need to understand the basic parameters of LengthField and can configure the parameters based on their own data frame format in their specific business scenario. The following are the parameter explanations for LengthField:

The code for Zinx's LengthField frame decoding algorithm can be found in this link. The FrameDecoder is a decoder that dynamically splits received binary data frames based on the value of the length field in the message. It is particularly useful when you decode binary messages with an integer header field representing the message body or the entire message length. ziface.LengthField has many configurable parameters, allowing it to decode messages with length fields in any format, which is commonly seen in proprietary client-server protocols.

LengthField is a general-purpose message encoding and decoding protocol that encodes the length of a message as part of the message header during the encoding process. During the decoding process, it first reads the message header, extracts the message length from it, and then reads the subsequent message content based on the length. In the LengthField protocol, four key parameters need to be configured:

• LengthFieldOffset: Represents the offset of the length field in the message header, i.e., the number of bytes between the length field and the start of the message header. Typically, the length field is placed before the message header, so this value is usually a non-negative integer.

• LengthFieldLength: Represents the number of bytes occupied by the length field. Usually, this value is a positive integer less than or equal to 8, as real-world messages typically don't exceed 8 bytes.

• LengthAdjustment: Represents the adjustment needed for the message length. After reading the length field, if any adjustment needs to be made to the length, it can be done through this parameter. For example, if the message length includes the length of the message header, then LengthAdjustment should be set to a negative value so that the actual length of the message can be correctly calculated when reading the message body.

• InitialBytesToStrip: Represents the number of bytes to skip during decoding, i.e., the number of bytes to skip the message header. Typically, this value equals the length of the message header because during decoding, it's necessary to first read the message header and parse the length field, and then skip the message header to read the message body.

• MaxFrameLength: Specifies the maximum frame length that the decoder can handle. When using a frame decoder such as LengthFieldBasedFrameDecoder, if the received frame exceeds MaxFrameLength, the decoder will throw an exception and close the connection. The purpose of this parameter is to limit the processing capability of the decoder to avoid risks such as memory overflow or denial of service attacks. This parameter should be set appropriately based on the specific application's protocol and data transmission method to ensure that the decoder can handle all valid frames without being affected by malicious or erroneous data. It's important to note that the value of MaxFrameLength should be determined according to the specific application's protocol and data transmission method. For example, if large files are being transmitted, a larger value for MaxFrameLength is needed to allow larger frames. On the other hand, if small text messages are being transmitted, a smaller value for MaxFrameLength can be set to improve security and reliability.

1.1 LengthField Configuration Example 1

"Using a 2-byte length field at offset 0, without stripping the message header."

In this example, the value of the length field is 12 (0x0C), which represents the length of the message "HELLO, WORLD". By default, the decoder assumes that the length field represents the number of bytes following the length field. Therefore, a simple parameter configuration can be used for decoding.

LengthFieldOffset = 0
LengthFieldLength = 2
LengthAdjustment = 0
InitialBytesToStrip = 0 (not stripping the message header)

The format of the message before and after decoding is as follows:

Before Decoding (14 bytes)         After Decoding (14 bytes)
+--------+----------------+      +--------+----------------+
| Length | Actual Content |----->| Length | Actual Content |
| 0x000C | "HELLO, WORLD" |      | 0x000C | "HELLO, WORLD" |
+--------+----------------+      +--------+----------------+

1.2 LengthField Configuration Example 2

"Using a 2-byte length field at offset 0, and stripping the message header."

You may want to strip the length field by specifying the "InitialBytesToStrip" parameter. In this example, we specify "2," which is the same as the length of the length field, to remove the first two bytes.

LengthFieldOffset = 0
LengthFieldLength = 2
LengthAdjustment = 0
InitialBytesToStrip = 2 (equal to the length of the Length field)

The format of the message before and after decoding is as follows:

Before Decoding (14 bytes)         After Decoding (12 bytes)
+--------+----------------+      +----------------+
| Length | Actual Content |----->| Actual Content |
| 0x000C | "HELLO, WORLD" |      | "HELLO, WORLD" |
+--------+----------------+      +----------------+

1.3 LengthField Configuration Example 3

"Using a 2-byte length field at offset 0, without stripping the message header, where the length field represents the length of the entire message."

In most cases, the length field only represents the length of the message body, as shown in the previous examples. However, in some protocols, the length field represents the length of the entire message, including the message header. In such cases, we need to specify a non-zero LengthAdjustment. Since in this example the length value in the message is always 2 bytes longer than the length of the message body, we set LengthAdjustment to -2 to compensate.

LengthFieldOffset = 0
LengthFieldLength = 2
LengthAdjustment = -2 (length of the Length field)
InitialBytesToStrip = 0

The format of the message before and after decoding is as follows:

Before Decoding (14 bytes)         After Decoding (14 bytes)
+--------+----------------+      +--------+----------------+
| Length | Actual Content |----->| Length | Actual Content |
| 0x000E | "HELLO, WORLD" |      | 0x000E | "HELLO, WORLD" |
+--------+----------------+      +--------+----------------+

1.4 LengthField Configuration Example 4

"5-byte header containing a 3-byte length field, without stripping the header."

This message is a simple variation of the first example, with an additional header value added before the message. The LengthAdjustment is once again set to 0 because the decoder always considers the length of the pre-defined data when calculating the frame length.

LengthFieldOffset = 2 (equal to the length of Header 1)
LengthFieldLength = 3
LengthAdjustment = 0
InitialBytesToStrip = 0

The format of the message before and after decoding is as follows:

Before Decoding (17 bytes)         After Decoding (17 bytes)
+----------+----------+----------------+      +----------+----------+----------------+
| Header 1 | Length   | Actual Content |----->| Header 1 | Length   | Actual Content |
| 0xCAFE   | 0x00000C | "HELLO, WORLD" |      | 0xCAFE   | 0x00000C | "HELLO, WORLD" |
+----------+----------+----------------+      +----------+----------+----------------+

1.5 LengthField Configuration Example 5

"5-byte header containing a 3-byte length field, without stripping the header."

This is an advanced example that demonstrates the scenario where there is additional header information between the length field and the message body. You need to specify a positive LengthAdjustment to allow the decoder to account for the extra header in the frame length calculation.

LengthFieldOffset = 0
LengthFieldLength = 3
LengthAdjustment = 2 (equal to the length of Header 1)
InitialBytesToStrip = 0

The format of the message before and after decoding is as follows:

Before Decoding (17 bytes)         After Decoding (17 bytes)
+----------+----------+----------------+      +----------+----------+----------------+
| Length   | Header 1 | Actual Content |----->| Length   | Header 1 | Actual Content |
| 0x00000C | 0xCAFE   | "HELLO, WORLD" |      | 0x00000C | 0xCAFE   | "HELLO, WORLD" |
+----------+----------+----------------+      +----------+----------+----------------+

1.6 LengthField Configuration Example 6

"4-byte header with a 2-byte length field at offset 1, stripping the first header field and length field."

This example combines all the previous examples. There is a pre-defined header before the length field and an additional header after the length field. The pre-defined header affects the LengthFieldOffset, while the additional header affects the LengthAdjustment. We also specify a non-zero InitialBytesToStrip to strip the length field and pre-defined header from the frame. If you don't want to strip the pre-defined header, you can specify an InitialBytesToSkip of 0.

LengthFieldOffset = 1 (length of HDR1)
LengthFieldLength = 2
LengthAdjustment = 1 (length of HDR2)
InitialBytesToStrip = 3 (length of HDR1 + length field)

The format of the message before and after decoding is as follows:

Before Decoding (16 bytes)                    After Decoding (13 bytes)
+------+--------+------+----------------+    +------+----------------+
| HDR1 | Length | HDR2 | Actual Content |--> | HDR2 | Actual Content |
| 0xCA | 0x000C | 0xFE | "HELLO, WORLD" |    | 0xFE | "HELLO, WORLD" |
+------+--------+------+----------------+    +------+----------------+

1.7 LengthField Configuration Example 7

"2-byte length field at offset 1 in a 4-byte header, stripping the first header field and length field, where the length field represents the entire message length."

Let's make some modifications to the previous example. The only difference from the previous example is that the length field represents the entire message length, similar to the third example. We need to account for the lengths of HDR1 and Length in the LengthAdjustment. Note that we don't need to consider the length of HDR2 because the length field already includes the length of the entire header.

LengthFieldOffset = 1
LengthFieldLength = 2
LengthAdjustment = -3 (length of HDR1 + length field, negative value)
InitialBytesToStrip = 3

The format of the message before and after decoding is as follows:

Before Decoding (16 bytes)                    After Decoding (13 bytes)
+------+--------+------+----------------+    +------+----------------+
| HDR1 | Length | HDR2 | Actual Content |--> | HDR2 | Actual Content |
| 0xCA | 0x0010 | 0xFE | "HELLO, WORLD" |    | 0xFE | "HELLO, WORLD" |
+------+--------+------+----------------+    +------+----------------+