External Converters not loading with 1.36.1 and Home Assistant 2024.4.0

[MrEcosse]

What happened?

None of my external converters to support new devices are working with this combination of Z2M and HA. These were working with the previous versions. Sorry, I don't know whether this happened as a result of the Z2M upgrade or the new April version of HA released today.

What did you expect to happen?

To use the external converters specified in the config file as it did before.

How to reproduce it (minimal and precise)

Add an unsupported device to the network
Create an external converter with correct fingerprint
Add the path to the .js file in the configuration.yaml
Restart HA
Note that the device is listed as unsupported still

Zigbee2MQTT version

1.36.1

Adapter firmware version

20230707

Adapter

zStack3x0

Setup

Add-on in Home Assistant on Intel NUC

Debug log

No response

[MrEcosse]

Here is the error in the log:

Zigbee2MQTT:error 2024-04-04 14:49:38: Failed to load external converter file 'ZSSQYGLUENGas.js' (Cannot find module 'zigbee-herdsman-converters/lib/extend'
Require stack:

• /app/dist/util/utils.js
• /app/dist/util/settings.js
• /app/index.js)
  Zigbee2MQTT:error 2024-04-04 14:49:38: Probably there is a syntax error in the file or the external converter is not compatible with the current Zigbee2MQTT version
  Zigbee2MQTT:error 2024-04-04 14:49:38: Note that external converters are not meant for long term usage, it's meant for local testing after which a pull request should be created to add out-of-the-box support for the device

[MrEcosse]

Have added the line
const {} = require('zigbee-herdsman-converters/lib/modernExtend');
To my external converter file but still seeing the error:

error 2024-04-04 15:05:28: Failed to load external converter file 'DINRelay.js' (Cannot find module 'zigbee-herdsman-converters/lib/extend'
Require stack:

• /app/dist/util/utils.js
• /app/dist/util/settings.js
• /app/index.js)

• Note this tried on a different device that I really need to get working again!

[MrEcosse]

I think that the documentation may be incorrect.

It appears that the line

const extend = require('zigbee-herdsman-converters/lib/extend');

should be removed from the external definitions and the line

const {} = require('zigbee-herdsman-converters/lib/modernExtend');

added in its place.

If so the example from the documentation at https://www.zigbee2mqtt.io/advanced/support-new-devices/01_support_new_devices.html#_2-1-extending-the-external-definition is incorrect.

[Great-Chart]

I don't have any external converters myself but I did note others have had such issues.
You might find some clues in #22076 with a similar error message and OP's solution(s) seemed to resolve things.

[MrEcosse]

Thanks for the reference - I will try this out. However if this solves the issue the documentation is definitely incorrect.

[aurelmarius]

Not sure is the same problem but, running latest dev build 1.36.1-dev commit: 3e91349.
I had 2 IKEA Symfonisk Gen2 remotes paired and noticed play/pause button not publishing state to mqtt
I tried to re-pair but clusters doesn't bind to coordinator anymore
Failed to configure '0x540f57fffe449664', attempt 3 (Error: Device 0x540f57fffe449664 has no output cluster genPollCtrl at getEndpointsWithCluster (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts:28:15) at configure (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts:126:31) at Object.configure (/app/node_modules/zigbee-herdsman-converters/src/index.ts:164:27) at Configure.configure (/app/lib/extension/configure.ts:122:37) at /app/lib/extension/configure.ts:87:62 at EventEmitter.wrappedCallback (/app/lib/eventBus.ts:174:23) at EventEmitter.emit (node:events:529:35) at EventBus.emitLastSeenChanged (/app/lib/eventBus.ts:51:22) at Controller. (/app/lib/zigbee.ts:74:27) at Controller.emit (node:events:517:28))

[mrskycriper]

Thanks for the reference - I will try this out. However if this solves the issue the documentation is definitely incorrect.

Yeah, it wasn't updated in time. Legacy extend was removed in zhc 19.0

[mrskycriper]

Not sure is the same problem but, running latest dev build 1.36.1-dev commit: 3e91349. I had 2 IKEA Symfonisk Gen2 remotes paired and noticed play/pause button not publishing state to mqtt I tried to re-pair but clusters doesn't bind to coordinator anymore Failed to configure '0x540f57fffe449664', attempt 3 (Error: Device 0x540f57fffe449664 has no output cluster genPollCtrl at getEndpointsWithCluster (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts:28:15) at configure (/app/node_modules/zigbee-herdsman-converters/src/lib/modernExtend.ts:126:31) at Object.configure (/app/node_modules/zigbee-herdsman-converters/src/index.ts:164:27) at Configure.configure (/app/lib/extension/configure.ts:122:37) at /app/lib/extension/configure.ts:87:62 at EventEmitter.wrappedCallback (/app/lib/eventBus.ts:174:23) at EventEmitter.emit (node:events:529:35) at EventBus.emitLastSeenChanged (/app/lib/eventBus.ts:51:22) at Controller. (/app/lib/zigbee.ts:74:27) at Controller.emit (node:events:517:28))

I'll investigate that, please create an issue. It might be connected to the recent IKEA refactoring in the dev branch

[Murada99]

Same problem, seems to be caused by upgrading Z2M as I didn't upgrade HA (2024.3.3)

[kloodhu]

same problem here.

[elmaswebon]

Yep, same here since the last two updates (1.36.1-0 and 1.36.1-1) with 2 different external files. Nothing to do with IKEA.

Edit: THIS solved the problem, thank you very much @MrEcosse

I think that the documentation may be incorrect.

It appears that the line

const extend = require('zigbee-herdsman-converters/lib/extend');

should be removed from the external definitions and the line

const {} = require('zigbee-herdsman-converters/lib/modernExtend');

added in its place.

If so the example from the documentation at https://www.zigbee2mqtt.io/advanced/support-new-devices/01_support_new_devices.html#_2-1-extending-the-external-definition is incorrect.

[Chartreusito]

Isn't that what the latest release Notes are about ? https://github.com/Koenkk/zigbee2mqtt/releases/tag/1.36.1

[MrEcosse]

I think this has morphed into the necessity to update the documentation which does not match the release notes

[Jacksoldano]

Legend thanks for this was tearing my hair out trying to get an External converter working last night stumbled on this post and this resolved my .js issue!

[ayybe]

Config path with Proxmox Debian LXC:
/opt/zigbee2mqtt/data/configuration.yaml

Alpine:
nano /etc/zigbee2mqtt/configuration.yaml

1. add this to z2m configuration.yaml (include empty line)

external_converters:
- _TZE200_libht6ua.js

2. touch _TZE200_libht6ua.js then nano _TZE200_libht6ua.js in the same dir as configuration.yaml with this code:

`const {} = require('zigbee-herdsman-converters/lib/modernExtend');

const zigbeeHerdsmanConverters = require('zigbee-herdsman-converters');
const zigbeeHerdsmanUtils = require('zigbee-herdsman-converters/lib/utils');

const fz = require('zigbee-herdsman-converters/converters/fromZigbee');
const tz = require('zigbee-herdsman-converters/converters/toZigbee');
const exposes = require('zigbee-herdsman-converters/lib/exposes');
const reporting = require('zigbee-herdsman-converters/lib/reporting');
const utils = require('zigbee-herdsman-converters/lib/utils');
const globalStore = require('zigbee-herdsman-converters/lib/store');
const e = exposes.presets;
const ea = exposes.access;
const tuya = require('zigbee-herdsman-converters/lib/tuya');

const definition = {
// Since a lot of TuYa devices use the same modelID, but use different datapoints
// it's necessary to provide a fingerprint instead of a zigbeeModel
fingerprint: [
{
// The model ID from: Device with modelID 'TS0601' is not supported
// You may need to add \u0000 at the end of the name in some cases
modelID: 'TS0601',
// The manufacturer name from: Device with modelID 'TS0601' is not supported.
manufacturerName: '_TZE200_libht6ua',
},
],
model: 'TS0601_cover',
vendor: 'TuYa',
description: 'Cover motor',
fromZigbee: [tuya.fz.datapoints],
toZigbee: [tuya.tz.datapoints],
onEvent: tuya.onEventSetTime, // Add this if you are getting no converter for 'commandMcuSyncTime'
configure: tuya.configureMagicPacket,
exposes: [
e.cover_position().setAccess('position', ea.STATE_SET),e.battery(),e.binary('motor_fault', ea.STATE, true, false),
e.enum('motor_direction', ea.STATE_SET, ['left', 'right']).withDescription('Motor side'),
e.enum('set_upper_limit', ea.STATE_SET, ['start', 'stop']).withDescription('Learning')
],
// Here you should put all functionality that your device exposes

meta: {
    // All datapoints go in here
    tuyaDatapoints: [
        [1, 'state', tuya.valueConverterBasic.lookup({'OPEN': tuya.enum(2), 'STOP': tuya.enum(1), 'CLOSE': tuya.enum(0)})],
        [2, 'position', tuya.valueConverter.coverPosition],
        [3, 'position', tuya.valueConverter.raw],
        [12, 'motor_fault', tuya.valueConverter.trueFalse1],
        [13, 'battery', tuya.valueConverter.raw],
        [101, 'motor_direction', tuya.valueConverterBasic.lookup({ 'left': tuya.enum(0), 'right': tuya.enum(1) })],
        [102, 'set_upper_limit', tuya.valueConverterBasic.lookup({ 'start': tuya.enum(0), 'stop': tuya.enum(1) })],
                            
                                
         //The datapoints below expose values but I can't detect why, I don't find them useful.
        [104], //when opening or closing and charging or not, expose value: 10
        [105], //when opening or closing and charging or not, expose value: 50 
        [7], //when opening or closing and charging or not, expose value: 2,
        [103], //maybe temperature, expose value: 22, 23, 28, 30, 50.. 
        [106], //when opening or closing and charging or not, expose value: 100,
        [108], //when opening or closing and charging or not, not expose value: "nothing",
    ],
},

};

module.exports = definition;`