Error "Invalid header value char" with a new HA version (2023.7.3)

[jhatala-myth]

I just installed a new HA, totally fresh installation and somehow integration is not working anymore but the same configuration and code is working fine on old HA

looks like with a new HA "login" part is not working, based on debug output "Step 1", "Step 2" never been performed or executed with an error
moreover, in previous HA no such message appears

message='Invalid header value char...

I guess its not exactly an issue with zyxel component but maybe you have an idea how it could be fixed

--- debug NON WORKING (Home Assistant 2023.7.3/ Supervisor 2023.08.1 / Operating System 10.3)

2023-08-10 20:31:27.844 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration zyxel_poe which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-08-10 20:32:26.643 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2023-08-10 20:32:28.394 DEBUG (MainThread) [custom_components.zyxel_poe.switch] No credential info stored, performing a login
2023-08-10 20:32:28.397 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logging in to: http://192.168.48.133/cgi-bin/dispatcher.cgi
2023-08-10 20:32:28.874 WARNING (MainThread) [homeassistant.components.switch] Platform zyxel_poe not ready yet: Connection error while connecting to http://192.168.48.133/cgi-bin/dispatcher.cgi: 400, message='Invalid header value char:\n\n b"Content-Security-Policy: default-src 'self';script-src 'self' 'unsafe-inline' 'unsafe-eval';style-src 'self' 'unsafe-inline'; base-uri 'self'"\n ^', url=URL('http://192.168.48.133/cgi-bin/dispatcher.cgi'); Retrying in background in 30 seconds
2023-08-10 20:32:56.433 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2023-08-10 20:32:56.473 DEBUG (MainThread) [custom_components.zyxel_poe.switch] No credential info stored, performing a login
2023-08-10 20:32:56.482 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logging in to: http://192.168.48.133/cgi-bin/dispatcher.cgi
2023-08-10 20:33:57.091 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2023-08-10 20:33:57.097 DEBUG (MainThread) [custom_components.zyxel_poe.switch] No credential info stored, performing a login
2023-08-10 20:33:57.100 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logging in to: http://192.168.48.133/cgi-bin/dispatcher.cgi
2023-08-10 20:35:27.386 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2023-08-10 20:35:27.393 DEBUG (MainThread) [custom_components.zyxel_poe.switch] No credential info stored, performing a login
2023-08-10 20:35:27.396 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logging in to: http://192.168.48.133/cgi-bin/dispatcher.cgi

--- debug WORKING ONE (Home Assistant 2023.4.6 / Supervisor 2023.08.1 / Operating System 10.1)

Home Assistant Core
2023-08-10 20:46:30.111 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration zyxel_poe which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-08-10 20:46:56.882 INFO (MainThread) [homeassistant.components.switch] Setting up switch.zyxel_poe
2023-08-10 20:46:59.489 DEBUG (MainThread) [custom_components.zyxel_poe.switch] No credential info stored, performing a login
2023-08-10 20:46:59.489 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logging in to: http://192.168.48.133/cgi-bin/dispatcher.cgi
2023-08-10 20:47:02.973 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Step 1 returned status code: 200, text:
2023-08-10 20:47:04.134 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Step 2 returned status code: 200, text: OK,
2023-08-10 20:47:04.135 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Logged in successfully
2023-08-10 20:47:04.661 DEBUG (MainThread) [custom_components.zyxel_poe.switch] CMD 773 returned status code: 200
2023-08-10 20:47:04.678 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 15 elements
2023-08-10 20:47:04.678 DEBUG (MainThread) [custom_components.zyxel_poe.switch] Found table with 1 elements

[cguedel]

Same here, did you find a solution yet?

[alchimista6]

I too have the same problem, since I updated home assistant to version 2023.8.1, I get the same error. Could it be a problem with the new aiohttp version (see: aio-libs/aiohttp#7494)? Not being a python programmer, I don't know how to solve this problem

[Dreamsorcerer]

It means that the server's response was not a valid HTTP message. If it relates to not using CRLF, then there's a chance it may work again with the next aiohttp release, but it would be better to fix it to send valid HTTP messages, if possible (I guess that'll be difficult if it's from the firmware of the switch though...).

[cguedel]

I think fixing firmware on a device which doesn't offer this basic functionality via a proper protocol like SNMP is never going to happen...

[Dreamsorcerer]

llhttp have added a new option that will hopefully make this work again in the next release.

[ghost]

Any solution on that issue? Did somebody tested the plugin with the newest version of HA?

[Dreamsorcerer]

There's a regression on llhttp blocking the next aiohttp release currently. Just downgrade your version of aiohttp until 3.8.6 is released.

[ghost]

could you pleaase subscribe how to do that? it seems there is no pip or pip3?

[Dreamsorcerer]

I'm not familiar with how HA OS works, that'll be a question for someone else.

[quizzical86]

I'm facing the same issue. If I understand correctly the info kindly provided by @Dreamsorcerer, those of us using this inside home assistant won't have this fixed before:
1: aiohttp 3.8.6 is released which I guess could be half a year.
2: Home Assistant uses this latest version of aiohttp (no clue how much more delay that causes).

@eyk87 @jhatala-myth @cguedel Have any of you succeeded in downgrading aiohttp inside home assistant or found another way of fixing the issue? If so could you please explain how you accomplished this? @lukas-hetzenecker I guess you are facing the same problem?

Many thanks!

[ghost]

It seems like it is only possible to change the package if you use a docker image or a supervised OS. I personaly use HassOS and it is not possible to change the used package.

BR
eyk

[quizzical86]

thanks @eyk87. I am also using HassOS. So you're also "stuck" with this issue andhoping aiohttp 3.8.6 will be released sooner rather than later?

[quizzical86]

I see aiohttp 3.8.6 was released and was in fact included in Home Assistant 2023.10.2, but then removed again from 2023.10.3 because it caused issues: home-assistant/core#101913
Not sure what the problem is, but I guess we are close to having a solution for us who are wanting to be able to switch our Zyxel POE switch from Home Assistant again?

[quizzical86]

The new version of Home Assistant that was released just now comes with a new version of aiohttp, so I wanted to test it out immediately.
But I seem to suffer from a new issue with my Zyxel Poe at the moment (it's web interface is unreachable) so can't see whether this aiohttp works until I've fixed that other issue.

[quizzical86]

Unfortunately the error is still there:

Platform zyxel_poe not ready yet: Connection error while connecting to http://192.168.0.3/cgi-bin/dispatcher.cgi: 400, message='Invalid header value char:\n\n b"Content-Security-Policy: default-src \'self\';script-src \'self\' \'unsafe-inline\' \'unsafe-eval\';style-src \'self\' \'unsafe-inline\'; base-uri \'self\'"\n ^', url=URL('http://192.168.0.3/cgi-bin/dispatcher.cgi'); Retrying in background in 30 seconds

@Dreamsorcerer where did you have the info from that the newer aiohttp releases might fix the issue?
Any clue what could be done to get it working again?
Thanks!

[Dreamsorcerer]

Pretty sure they reverted the version of aiohttp due to a regression, so you're probably not using the latest version. We'll probably have another release momentarily, and I suspect HA will end up with a new release shortly after (so, by next week).

[quizzical86]

Ah ok that would be good, fingers crossed that that's the explanation.
In that case the changelog is incorrect. It says they are using a beta version: 3.9.0b0

[Dreamsorcerer]

It also says 'Revert aiohttp to 3.8.5 for Python 3.11'. They clearly provide a list of commits, rather than a tailored list of changes.

[quizzical86]

Ah well spotted, thanks for clarifying.
Very glad that I can still hope for it to be fixed soon then :)

[quizzical86]

This seems to be the pull request to keep an eye on:
home-assistant/core#104176

[cguedel]

Can confirm this works again with HA 2023.12

[quizzical86]

Indeed it is back up and running. Hurray :-)

A bit of an offtopic question: what firmware version are you all using for your switches?
I'm using V2.70(AAHI.1) which dates back to November 2022, and I see there have been quite a few new firmware releases since then. The latest being 2.80(AAHI.0)C0 from October 2023.

Some of the releases introduced new features I would like to use (to keep track of who'se been logging into my switch as it is showing some strange behaviour), but also to fix some critical vulnerabilities!

But I know from experience upgrading the Zyxel firmware could mean this home assistant integration stops working...

[cguedel]

I'm running V2.70(AAHO.5) on a GS1900-48HP

[quizzical86]

Thanks! So you are also behind a couple of releases.
I took the plunge and upgraded to 2.80(AAHI.0)C0 from October 2023 and can happily report that the integration with Home Assistant still works!