Implement new zigpy network state #445

puddly · 2021-12-12T22:33:38Z

At the moment this PR is very WIP and I'm not very familiar with EZSP so there are likely mistakes with the hashed TCLK stuff. No unit tests pass and I've only tested it with zigpy-cli, though it does generate a valid backup for me.

Test coverage is 100% 🎉

SylvainPer · 2022-03-03T09:31:21Z

@puddly in the __init __() of ControllerApplication could you call SCHEMA(config) in the super().__init __() like other libs ?

zigpy/bellows#445 (comment)

puddly · 2022-03-06T17:25:52Z

This has been fixed inside of zigpy itself: puddly/zigpy@4c78afc

SylvainPer · 2022-03-08T13:33:39Z

@puddly I tried the new load_network_info function and I don't know what to think about the result:

all getChildData requests answer are in EMBER_NOT_JOINED state
all getAddressTableRemoteNodeId return 'ffff'
all getAddressTableRemoteEui64 return 'ffffffffffffffff' for the first one and then '0000000000000000'

do you have the same with your controller ?

puddly · 2022-03-08T20:03:06Z

That mostly matches what I see with my coordinator but since I don't normally use a SiLabs coordinator, I'm not sure if it's "normal" or not. None of that information is critical, it was just included to make the ZNP backup complete.

SylvainPer · 2022-03-09T09:13:50Z

It's not critical but it's something we use in the initialization part to check if the couple IEEE/shortId is still up-to-date.

~~Maybe the networkInit() reset those values, I saw that you check the network state before calling it, I'll give a try.~~
Edit: Misread your modification

pipiche38 · 2022-03-09T10:27:47Z

@puddly

That mostly matches what I see with my coordinator but since I don't normally use a SiLabs coordinator, I'm not sure if it's "normal" or not. None of that information is critical, it was just included to make the ZNP backup complete.

It is not critical, but it is quiet important. The main reason is that if you have your coordinator working, but you have your application/box/plugin off ( maintenance, upgrade ....), when restarting the plugin it is a really great things to be able to load Ieee/Nwkid and check against the last plugin state and eventually update it. In zigpy I believe device database is updated accordingly.
Lacking this part, will make receiving zigbee messages from unknown device (if that one has changed during the time the plugin/library was off), and will required some effort to retreive the IEEE/NwkId.

SylvainPer · 2022-03-09T13:13:53Z

Just made some test live:

even after driving a plug, nothing change in the tables
after a pairing, the device appears in getAddressTableRemoteNodeId and getAddressTableRemoteEui64 (index 16), nothing in getChildData
after a restart (startup), the device disappeared from the tables

SylvainPer · 2022-03-14T16:17:35Z

@puddly could you take #452 into your radio-api branch please ?

zigpy/bellows#445 (comment)

When a RSTACK message is processed right after the UART has been opened, it causes EZSP.enter_failed_state() getting called at a point where the application callbacks are not registered yet. In that case the UART will get closed and it won't get opened again. Bellows is stuck with a closed transport. Avoid this issue by not closing the port in case there is no application callback registered yet. Typically, it is unlikely that a RSTACK message arrives right when the port gets opened (the race window is very narrow). However, with hardware flow control opening the port leads to RTS signal to get asserted which causes the radio to send pending messages, e.g. resets caused by EmberZNet watchdog. Note: With hardware flow control this is only the case if the tty "hupcl" option is set. The option is set by default, but cleared by tools like GNU screen. This option makes sure that the RTS signal is deasserted while the port is closed. Pyserial/bellows does not change the state of that option.

…-api

pipiche38 · 2022-06-20T20:14:57Z

@puddly , I'm trying to test the bellows new-radio-API ,and I'm current getting issue when importing.

I'm using your branch puddly:puddly/zigpy-radio-api

Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:     from Classes.ZigpyTransport.AppBellows import App_bellows
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:   File "/home/domoticz/domoticz/plugins/Domoticz-Zigbee/Classes/ZigpyTransport/AppBellows.py", line 15, in <module>
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:     import bellows.ezsp as ezsp
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:   File "/home/domoticz/domoticz/plugins/Domoticz-Zigbee/bellows/ezsp/__init__.py", line 21, in <module>
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:     import bellows.types as t
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:   File "/home/domoticz/domoticz/plugins/Domoticz-Zigbee/bellows/types/__init__.py", line 3, in <module>
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:     from .struct import *  # noqa: F401,F403
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:   File "/home/domoticz/domoticz/plugins/Domoticz-Zigbee/bellows/types/struct.py", line 258, in <module>
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:     class EmberNetworkParameters(EzspStruct):
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:   File "/home/domoticz/domoticz/plugins/Domoticz-Zigbee/bellows/types/struct.py", line 286, in EmberNetworkParameters
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab:     def zigpy_network_information(self) -> app_state.NetworkInformation:
Jun 20 21:09:20 pi4 domoticz[674]: zigpy-Silicon Lab: AttributeError: module 'zigpy.state' has no attribute 'NetworkInformation'

puddly · 2022-06-20T20:16:34Z

I don't think you're using the correct branch. NetworkInformation does not appear in the source code.

zigbeefordomoticz · 2022-06-20T20:42:51Z

Ok. I’ll wait for the merge Envoyé de mon iPhone

…

Le 20 juin 2022 à 22:16, puddly ***@***.***> a écrit : I don't think you're using the correct branch. NetworkInformation does not appear in the source code. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

zigbeefordomoticz · 2022-06-22T07:24:18Z

would it be possible to get this PR approved and merged ? We are waiting for it in order to integrate the full new-radio-API ( zigpy , zigpy-znp, zigpy-deconz ) have already been merged

thanks in advance and thanks to @puddly for making it, this really make the usage across radio lib simpler

pipiche38 · 2022-06-27T08:38:21Z

gentlemen @Hedda, @puddly , @Adminiuga any chance to get this approve and merged ? Thanks in advance

puddly mentioned this pull request Dec 12, 2021

Implement new radio API zigpy/zigpy-cli#2

Merged

6 tasks

Hedda mentioned this pull request Jan 20, 2022

Weird error - EZSP_BACKUP mdeweerd/zha-toolkit#1

Closed

puddly mentioned this pull request Feb 12, 2022

Implement the new zigpy radio API zigpy/zigpy-zigate#117

Merged

Hedda mentioned this pull request Feb 16, 2022

[SUGGESTION] Use Nordic nRF52840 chip ... zigpy/zigpy#394

Open

puddly added a commit to puddly/zigpy that referenced this pull request Mar 6, 2022

Apply the config schema within ControllerApplication.__init__

4c78afc

zigpy/bellows#445 (comment)

puddly added a commit to puddly/zigpy that referenced this pull request Apr 14, 2022

Apply the config schema within ControllerApplication.__init__

d61bd55

zigpy/bellows#445 (comment)

puddly added 14 commits May 4, 2022 21:57

[WIP] Implement new zigpy network state

2bcfa9a

[WIP] Implement writing network settings

d2c1129

Set NWK and APS frame counters

b160490

Change key table size only if the new keys don't fit

8b8b313

Style cleanup

0d486e4

Directly create EZSPCoordinator instead of using a quirk

f51152b

Use zigpy types when creating network state objects

dfa5791

Allow load_network_info to be called before/after start_network

539f2b6

Allow node_info.ieee to be None

c6223c4

Use t.EUI64.UNKNOWN instead of None to represent an unset IEEE addr

71aab0f

Do not leak EZSP types into zigpy

bbd71fd

Fix network formation when a hashed TCLK is generated

1bd1c8e

Joins were not being properly permitting after fcf0b49

febed7a

Use the new zigpy add_endpoint method

1d267a4

puddly and others added 15 commits May 5, 2022 13:28

Add unit test for when getMfgToken is missing

eadf4ab

Unit test _ensure_network_running

76831bb

Fix startup delay caused by slow coordinator initialization

cb013f1

Key table size can't be adjusted

f897655

Unit test getChildData for EZSP v6

90a6b48

Unit test leaveNetwork failing with INVALID_CALL

2925708

Add the coordinator to the zigpy device dictionary before initialization

10e7ef4

Do not back up the address tables on EZSPv4, it is unstable

46cfa92

Do not set frame counters when using EZSPv4

e049ddc

Unit test EZSP v4 changes

74165f2

Fix tests

c59b92f

Merge branch 'fix-uart-close-race-at-startup' into puddly/zigpy-radio…

90223f2

…-api

Include radio library metadata in network info

c4c8d3b

Fix unit tests broken by new metadata key

8741573

Hedda mentioned this pull request May 16, 2022

[RFC] Unified radio command line interface and standard network backup format zigpy/zigpy#557

Closed

puddly added 5 commits May 20, 2022 12:28

Merge remote-tracking branch 'zigpy/dev' into puddly/zigpy-radio-api

2f073ec

Do not set the HAVE_TRUST_CENTER_EUI64 bit when forming a network

549632a

Merge remote-tracking branch 'zigpy/dev' into puddly/zigpy-radio-api

ad2c318

Bump minimum required zigpy version to 0.47.0

080e7b4

Fix unit tests for Python 3.7

aa865b3

puddly marked this pull request as ready for review June 16, 2022 18:53

puddly requested a review from Adminiuga June 19, 2022 21:48

puddly merged commit 766a6d6 into zigpy:dev Jun 27, 2022

Hedda mentioned this pull request May 16, 2023

ZBOSS Zigbee network backup and network restore + radio migration via zigpy radio API? kardia-as/zigpy-zboss#11

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement new zigpy network state #445

Implement new zigpy network state #445

puddly commented Dec 12, 2021 •

edited

Loading

SylvainPer commented Mar 3, 2022

puddly commented Mar 6, 2022

SylvainPer commented Mar 8, 2022

puddly commented Mar 8, 2022

SylvainPer commented Mar 9, 2022 •

edited

Loading

pipiche38 commented Mar 9, 2022

SylvainPer commented Mar 9, 2022

SylvainPer commented Mar 14, 2022

pipiche38 commented Jun 20, 2022

puddly commented Jun 20, 2022

zigbeefordomoticz commented Jun 20, 2022 via email

zigbeefordomoticz commented Jun 22, 2022

pipiche38 commented Jun 27, 2022

Implement new zigpy network state #445

Implement new zigpy network state #445

Conversation

puddly commented Dec 12, 2021 • edited Loading

SylvainPer commented Mar 3, 2022

puddly commented Mar 6, 2022

SylvainPer commented Mar 8, 2022

puddly commented Mar 8, 2022

SylvainPer commented Mar 9, 2022 • edited Loading

pipiche38 commented Mar 9, 2022

SylvainPer commented Mar 9, 2022

SylvainPer commented Mar 14, 2022

pipiche38 commented Jun 20, 2022

puddly commented Jun 20, 2022

zigbeefordomoticz commented Jun 20, 2022 via email

zigbeefordomoticz commented Jun 22, 2022

pipiche38 commented Jun 27, 2022

puddly commented Dec 12, 2021 •

edited

Loading

SylvainPer commented Mar 9, 2022 •

edited

Loading