costum request and response - response not fully received

[jikov]

Good morning,
my question is - do you have problem with receiving of the messages with more bytes?

Because I have found out this strange behaviour:
The custom HW (slave) doesnt support standard Modbus functions (like read coils etc.) so I use only "modbus_send_raw_request" and "modbus_receive_confirmation" functions. (Is this correct? Or shall I use different functions?)

The request (from the master = PC) is sent correctly but the response from the slave is not fully received. I am able to receive only 5 useful bytes: ID (1B) - function code (1B) - data bytes quantity (1B) - data bytes (2B). Together 5 bytes (there are printed out in the terminal).
But I need to receive more than 2 data bytes, of course :-)

I checked this behaviour with ModbusPoll SW (http://www.modbustools.com/modbus_poll.asp) and the response is received correctly in this case.
So I went through the source code little bit (I am not fully familiar with libmodbus source code) and the problem could be in function "_modbus_receive_msg".
Probably one of the most important variable here is "length_to_read".
Here is the part of the source code:

/* We need to analyse the message step by step. At the first step, we want
* to reach the function code because all packets contain this
* information. */
step = _STEP_FUNCTION;
length_to_read = ctx->backend->header_length + 1;

The variable "ctx->backend->header_length" still remains "1" and of course, I always get "2" in the variable "length_to_read". And it does not depend on lenght of the received message. Is this correct behaviour? Is this the reason why I can only read 2 data bytes?

But if I hardcoded the variable "length_to_read" to value e.g. 20 (length_to_read = 20;), because I want to receive 20 data bytes, all 20 data bytes are received correctly. That is strange.

Does anybody know what is wrong? Did I miss anything? Wrong settings etc...?

Thanks for the response.
Best regards,
Jiri

My HW and SW configuration:
PC - linux mint 17 Cinnamon 64b
usb/RS485 converter connected on /dev/ttyUSB0 at 38400, no parity, 8 data bits, 1 stop bit
libmodbus v 3.1.1,
RTU backend
debug on.

[jikov]

I also tried the version 3.0.6 with same results...

[stephane]

Message are read in several chunks and analyzed at each step (function, meta and data).
The header length depends on the backend and it used to reach function code information (first step), you must follow the code step by step to understand why your message isn't fully read (may be not valid).

[jikov]

Hi, thanks for the answer. I have already understood what happened.
The problem is that I use only user defined codes and the function "compute_data_length_after_meta" can only distinguish the standard function codes. My functions codes doesnt fit to the condition (that is the reason why I could read only 2 bytes).

I would suggest to make a small modification of this function (to be more universal).

_The original function:_*

static int compute_data_length_after_meta(modbus_t *ctx, uint8_t *msg,
msg_type_t msg_type)
{
int function = msg[ctx->backend->header_length];
int length;

if (msg_type == MSG_INDICATION) {
    switch (function) {
    case MODBUS_FC_WRITE_MULTIPLE_COILS:
    case MODBUS_FC_WRITE_MULTIPLE_REGISTERS:
        length = msg[ctx->backend->header_length + 5];
        break;
    case MODBUS_FC_WRITE_AND_READ_REGISTERS:
        length = msg[ctx->backend->header_length + 9];
        break;
    default:
        length = 0;
    }
} else {
    /* MSG_CONFIRMATION */
    if (function <= MODBUS_FC_READ_INPUT_REGISTERS ||
        function == MODBUS_FC_REPORT_SLAVE_ID ||
        function == MODBUS_FC_WRITE_AND_READ_REGISTERS) {
        length = msg[ctx->backend->header_length + 1];
    } else {
        length = 0;
    }
}

length += ctx->backend->checksum_length;

return length;

}

My modification:*************
As you can see below I only add the next condition to handle user defined codes. I have tried the several length of the messages and it works well.
But user messages must respect the modbus response format (address - function_conde - data_byte_quantity - data_byte_1 - data_byte_2 - data_byte_3...etc).

file modbus.h:
#define MODBUS_FC_USER_DEFINED_CODES_SECTION_1_LOW 0x41
#define MODBUS_FC_USER_DEFINED_CODES_SECTION_1_HIGH 0x48
#define MODBUS_FC_USER_DEFINED_CODES_SECTION_2_LOW 0x64
#define MODBUS_FC_USER_DEFINED_CODES_SECTION_2_HIGH 0x6E

file modbus.c

static int compute_data_length_after_meta(modbus_t *ctx, uint8_t *msg,
msg_type_t msg_type)
{
int function = msg[ctx->backend->header_length];
int length;

if (msg_type == MSG_INDICATION) {       
    switch (function) {
    case MODBUS_FC_WRITE_MULTIPLE_COILS:
    case MODBUS_FC_WRITE_MULTIPLE_REGISTERS:
        length = msg[ctx->backend->header_length + 5];
        break;
    case MODBUS_FC_WRITE_AND_READ_REGISTERS:
        length = msg[ctx->backend->header_length + 9];
        break;
    default:
        length = 0;
    }
} else {
    /* MSG_CONFIRMATION */          
    if (function <= MODBUS_FC_READ_INPUT_REGISTERS ||
        function == MODBUS_FC_REPORT_SLAVE_ID ||
        function == MODBUS_FC_WRITE_AND_READ_REGISTERS) {
        length = msg[ctx->backend->header_length + 1];          
    } else 
    {
        /* user defined function codes */
        if ((function >= MODBUS_FC_USER_DEFINED_CODES_SECTION_1_LOW && 
        function <= MODBUS_FC_USER_DEFINED_CODES_SECTION_1_HIGH) || 
            (function >= MODBUS_FC_USER_DEFINED_CODES_SECTION_2_LOW && 
        function <= MODBUS_FC_USER_DEFINED_CODES_SECTION_2_HIGH)) 
        {
            length = msg[ctx->backend->header_length + 1];
        }
        else
        {
            length = 0;
        }
    }
}

length += ctx->backend->checksum_length;

return length;

}

Please consider it for the next release.
Best regards,
Jiri Kovanda

[karlp]

Just ran into this myself. 👍 While I don't see anything in the spec that requires user functions to have the first byte be the remaining length, it seems a very reasonable thing to require. Without something like this, it's actually ~impossible to implement custom functions using libmodbus with any helpers. Doing everything via send_raw_request/receive_raw is tedious

If simply assuming that user functions have that byte as length, (I don't see any requirement in the spec for user functions to have any bytes after the function code) Would it be possible to register either functioncode/length mappings, or function code/byteoffset_containing_length mappings?

[steffen86]

I also ran into the same problem and after reading the code and some research I realized that this is something that is fixed in the ModbusTCP specification as it has 2 bytes reserved in the header for the size of the data. This information is just not used in libmodbus.

From https://en.wikipedia.org/wiki/Modbus:

Name 				Length (bytes) 	Function
Transaction identifier 		2 		For synchronization between messages of server and client
Protocol identifier 		2 		0 for Modbus/TCP
Length field 			2 		Number of remaining bytes in this frame
Unit identifier 		1 		Slave address (255 if not used)
Function code 			1 		Function codes as in other variants
Data bytes 			n 		Data as response or commands

[karlp]

@steffen86 that field is unfortuantely only in the modbus/tcp header, but the problem applies equally to modbus/rtu

[steffen86]

@karlp jep saw that as well. But at least one could fix it for tcp...

What would be a proper solution for that Problem?

• Extend the modbus_receive_confirmation() to have an extra parameter containing the expected reply length. -> Inteface will change
• Allow the programmer to parse what was received so far and have a new function like modbus_receive_confirmation_continue() to continue reading. -> This will probably collide with the crc check for rtu as it is performed at the end of modbus_receive_confirmation() and thus this function will return with a crc error even it hasn't received the complete answer.

[biergaizi]

I was hit by the bug today. The inability to receive an arbitrary response is a serious problem, worse, libmodbus does not expose any low-level API to interact with the serial port. In other words, currently it's totally impossible to send and receive a custom message via libmodbus, unless I abuse the internal API, or patch the library.

Patching the library is not an option for me. I am implementing a device driver for Network UPS Tools, which uses libmodbus as the official library to interact with Modbus devices, which means I'm only able to use libmodbus as it is. Abusing the internal API is not a choice either, such code will definitely be rejected by the maintainer due to safety problems. Currently, my workaround is to access the raw serial port, construct raw modbus messages, close the serial port (the custom message is only needed during initialization), and reopen it in libmodbus...

It's unfortunate that no solution has been introduced 6 years after the initial report. Although there has been a proposal to add callbacks to solve the general problem, but it's clear that such a solution will not be implemented in the foreseeable future. Thus, I suggest the following solution as a stop-gap measure.

1. Change ssize_t (*recv) (modbus_t *ctx, uint8_t *rsp, int rsp_length); to a public API, so the developer can receive raw data from the underlying device, bypassing libmodbus when it's necessary. Also, size_t (*send) (modbus_t *ctx, const uint8_t *req, int req_length); should also be a public API, so that the developer can also send an arbitrary message without CRC, just in case when it's needed.

2. Also make _modbus_rtu_check_integrity() a public API, so the user can invoke the function to do CRC checks manually without duplicating any code.

If this is an acceptable solution, I'm willing to contribute a patch.