Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support libbi export to grid #442

Closed
plord12 opened this issue Oct 26, 2023 · 97 comments
Closed

Support libbi export to grid #442

plord12 opened this issue Oct 26, 2023 · 97 comments

Comments

@plord12
Copy link

plord12 commented Oct 26, 2023

Is your feature request related to a problem? Please describe.

Since libbi 5.408 ( https://support.myenergi.com/hc/en-gb/articles/19276846239377-libbi-Firmware-Version-5-408 ) libbi supports export to grid. This is useful if you wanted to use your libbi to charge by cheap electricity (or solar) and export when export prices are high.

Describe the solution you'd like

HA entity to turn on and off export to grid.

Describe alternatives you've considered

none

Additional context

Also raised CJNE/pymyenergi#18 for the underlying library

@videojedi
Copy link
Contributor

Looks like the endpoints are...

to turn on
https://s18.myenergi.net/cgi-libbi-mode-L24047041-5

and to turn off
https://s18.myenergi.net/cgi-libbi-mode-L24047041-1

It's not a very consistent API is it!

@trizmark
Copy link
Contributor

I've got it working locally; will be posting it here soon.

The API is sort of consistent. Looking at the app code there are a number of modes that are defined, but not used and cannot be set. (capture, charge, match)
The new mode (drain) was simply added to the list and that's why it became 5.

@trizmark
Copy link
Contributor

Charge target is implemented as well, but it's super-broken in the libbi firmware. 😞

@plord12
Copy link
Author

plord12 commented Oct 29, 2023

Charge target is implemented as well, but it's super-broken in the libbi firmware. 😞

Nice :-) I'll try it with the app tonight.

BTW do you have 5.414 installed on your libbi ? ( see https://myenergi.info/libbi-no-longer-fully-charging-with-cheap-rate-ove-t11328.html )

@trizmark
Copy link
Contributor

I had 5.408 on until this morning. Overnight, instead of my set 2.5h charge, the libbi only charged for an hour and then it sort of held the SoC.
I installed 5.414, which now completely broke the charge from grid. Grrrr!

@trizmark
Copy link
Contributor

For those who are foolish and reckless.... I present you the latest bleeding edge version of the integration.
New operation mode for your libbi: 'Export'. Let me know if you have any issues with it.
myenergi-20231029.tgz

I am adding the service to set the charge target. New version probably tomorrow (depending on my schedule).

@videojedi
Copy link
Contributor

videojedi commented Oct 29, 2023

Great work. I'm running 5.414.

Can confirm export works!

@trizmark
Copy link
Contributor

My charge target remains the same with toggling the charge from grid. Does yours change?

@videojedi
Copy link
Contributor

videojedi commented Oct 29, 2023

Yes mine seems to default to zero... I reset it in the app to 100%, and toggle it again in home assistant, but it returns to zero

@videojedi
Copy link
Contributor

Same behaviour in the app too...... toggle it and it returns to zero.

@videojedi
Copy link
Contributor

aha.... this is another shared account issue. If I login into app with primary account details, then it remembers the chargetarget between toggles...

@trizmark
Copy link
Contributor

If it's the same issue in the app, then it's a myenergi issue.

@trizmark
Copy link
Contributor

trizmark commented Oct 30, 2023

I have tested it with my secondary account and it is indeed a myenergi backend issue. 🤦

@plord12
Copy link
Author

plord12 commented Oct 30, 2023

For those who are foolish and reckless.... I present you the latest bleeding edge version of the integration.
New operation mode for your libbi: 'Export'. Let me know if you have any issues with it.
myenergi-20231029.tgz

FYI, didn't work for me, plugin failed to startup. I see in the logs -

2023-10-30 13:09:29.625 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry Home for myenergi
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 399, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/__init__.py", line 53, in async_setup_entry
    conn = await hass.async_add_executor_job(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/pymyenergi/connection.py", line 40, in __init__
    self.oauth.authenticate(password=self.app_password)
  File "/usr/local/lib/python3.11/site-packages/pycognito/__init__.py", line 478, in authenticate
    tokens = aws.authenticate_user(client_metadata=client_metadata)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pycognito/aws_srp.py", line 269, in authenticate_user
    response = boto_client.initiate_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 535, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 936, in _make_api_call
    request_dict = self._convert_to_request_dict(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 1007, in _convert_to_request_dict
    request_dict = self._serializer.serialize_to_request(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/botocore/validate.py", line 381, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter AuthParameters.USERNAME, value: None, type: <class 'NoneType'>, valid types: <class 'str'>

Rolled back to official version and it started up.

@trizmark
Copy link
Contributor

As this version adds the OAuth endpoints you need to remove your integration and re-add it as it requires both API key/password and App email/app password parameters.

@plord12
Copy link
Author

plord12 commented Oct 30, 2023

As this version adds the OAuth endpoints you need to remove your integration and re-add it as it requires both API key/password and App email/app password parameters.

Ah, got it, thanks. After re-adding ( and entering username/password ) I was able to enable export :-)

My grid charge percentage stayed at 60% FYI.

@trizmark
Copy link
Contributor

Great to hear!
The charge target is only an issue if you're using a secondary account to access your main account.
People who signed up to a myenergi account using google/apple/facebook need a secondary account as currently we only support OAuth with email/password combo.

I am nearly ready with the charge target setting functionality, but as this is the first time I'm implementing a HA service, it is a lot of trial and error.

@whittyone
Copy link

Thanks for making this version. Is there a plan for getting a 0.0.25 made with this in?
(Sorry, I'm not sure how that side is managed)
Thanks

@trizmark
Copy link
Contributor

trizmark commented Jan 9, 2024

@whittyone Yes, there is. I am updating the pymyenergi library currently. Once that's done and out, I can push my modifications to the HA component.

@plord12
Copy link
Author

plord12 commented Feb 19, 2024

I see this didn't make 0.0.25

@trizmark
Copy link
Contributor

@plord12 The 0.1.0 version of pymyenergi (released yesterday) has all the necessary features in there, so once I rebase my local dev copy to 0.0.25, this will be in 0.0.26.

@plord12
Copy link
Author

plord12 commented Feb 19, 2024

@plord12 The 0.1.0 version of pymyenergi (released yesterday) has all the necessary features in there, so once I rebase my local dev copy to 0.0.25, this will be in 0.0.26.

Brilliant! Many thanks.

@MaximumFish
Copy link

I'm making really good use of this to dump excess power in the evenings and during Octopus saving sessions, so good to know that I shouldn't upgrade just yet. Thanks!

@MaximumFish
Copy link

@plord12 The 0.1.0 version of pymyenergi (released yesterday) has all the necessary features in there, so once I rebase my local dev copy to 0.0.25, this will be in 0.0.26.

Did this get added to 0.0.26 or 0.0.27? I don't see it in the changelogs.

@trizmark
Copy link
Contributor

I'm still working on getting the PR ready. Unfortunately work has been eating into my spare time, so no ETA at the moment.

@MaximumFish
Copy link

No worries. 0.0.25 is still working fine for me so I'm in no rush to update. 😄

@plord12
Copy link
Author

plord12 commented Apr 14, 2024

I hate to pester, given so many give their time for projects like this, but any news ?

Once this gets in I hope to look at the libbi dual tariff issue (eg sync libbi schedule from octopus) ... but I don't really want to get going until this is in.

Cheers :-)

@trizmark
Copy link
Contributor

I can see the light at the end of the tunnel!
I've been playing catch-up with the changes that happened in this repo vs what I've added. Hoping to submit the PR this weekend. 🤞

@MalcolmSpencer
Copy link

@trizmark would you be able to look at issue #516 when you do your update? Currently, Libbi operating mode is not working in 0.0.27 but the solution is described.

Thanks,

@trizmark
Copy link
Contributor

I found that issue when I brought my fork up to 0.0.27 and it's already fixed.

@trizmark
Copy link
Contributor

It's been a long time coming, but finally it's here!
#553

Apologies for the delay (new job, family stuff)

@githubjonny
Copy link

@trizmark great job 👍🏼 👏🏼

I'm still getting 0.0.27 in HACS. Can't see if this has been pushed through. Anyway to jump to the latest build via HA?

@MaximumFish
Copy link

Also eagerly waiting for the new release, though I note that the page for #553 says that some checks have failed?

I'm wondering if this new version will have a way to actually check and confirm the state of the "charge from grid" toggle? My automation completely failed a couple of nights ago because the myenergi API was down at 23:30 when it tried to turn on grid charging, and the "until" loop that was supposed to safeguard against failed commands was just checking the state of this toggle. It was only when I woke up to an empty battery that I realised it's just a dumb toggle and not actually read back from anywhere!

I've fixed that now by instead checking the state of sensor.myenergi_libbi_xxxxxxxx_status to make sure it's switched to "Target Charge", but it doesn't work for turning grid charging off in the morning due to the state changing overnight to "Full" when it's full. Turning off grid charging doesn't change this state at all (I think it needs to drop below 95% first?) and there doesn't seem to be anything else to check against.

For now I'm just going to have to toggle it and hope that the API is up and running, but it would be great to have something reliable to check against.

@trizmark
Copy link
Contributor

The failed checks are for linting and testing. The linting issue is a conflict between black8 and reorder-imports. The test failure is due to me making app_password and email optional, while the test expects them to be mandatory.

The charge_from_grid toggle is read and updated every minute. If you change it from the app, the state will be updated within a minute. If there's an issue with the ME API, the control will become unavailable and any turn_on or turn_off will fail.
My solution for this was to check the availability of the control at the beginning of my automation. If the control is available, then just go ahead and change its state. If it's unavailable, start a 2 minute timer and execute the same automation once the timer expires. It's a very simple way of handling this kind of failure.

@MaximumFish
Copy link

MaximumFish commented Jul 25, 2024

Hmm, it didn't work like that for me. I noticed that the API was being screwy when I went to bed at 23:00. The SoC sensor hadn't updated for a while and the ME app was showing all yellow warning triangles on everything, but I assumed my automation would take care of things when it came back up.

As it is though, HA saw a brief blip of "unavailable" that morning, but when the automation fired it was happy enough. The "until" loop thought that it had turned on successfully and the state didn't change at any point throughout the night, even after the API had come back at around 1:00. Essentially, HA thought that it was on but the Libbi did not.

Of course the version I'm using is a few revisions behind the current one now, so maybe it'll work better in 0.0.28? 🤔

image
Note: The name of the automation firing at 23:30 is misleading. It now starts the grid charging, but it's a repurposed automation that would stop the Libbi entirely during non-off-peak intelligent dispatch windows and the internal name didn't change along with the display name I guess.

@plord12
Copy link
Author

plord12 commented Jul 30, 2024

Hmm, it didn't work like that for me. I noticed that the API was being screwy when I went to bed at 23:00. The SoC sensor hadn't updated for a while and the ME app was showing all yellow warning triangles on everything, but I assumed my automation would take care of things when it came back up.

We need a local API.

I've raised support requests to myenergi to this effect. The more of us who do the same the better.

@trizmark
Copy link
Contributor

@MaximumFish I'll do some more testing to see how the integration behaves during an API outage.
@plord12 Fully agree with you, local API all the way! In fact, I implemented local monitoring by tapping into the Modbus interface. It doesn't give me control, but I can see everything real-time and have access to data that's not exposed via the API.

@MaximumFish
Copy link

That would be awesome, thank you.

It's not a huge issue right now as the only times I've ever had issues is overnight when switching grid charging on and I can check that's worked via the "Target Charge" state. In the mornings I'm currently just checking the app manually to make sure it's turned off properly. Though I wonder if I can do something funky with watching the grid draw to figure out if it's switched or not...

@plord12 if I were to raise the local API issue too, what would I be asking for exactly? I don't think I'd get away with just saying "Local API plz" :)

@plord12
Copy link
Author

plord12 commented Jul 30, 2024

@plord12 if I were to raise the local API issue too, what would I be asking for exactly? I don't think I'd get away with just saying "Local API plz" :)

I would say something like -

"There have been several times recently that I've been unable to control my myenergi devices .. a couple of cases where my internet connection was unreliable and a couple where the myenergi server didn't seem to respond. Regardless of the cause, its vital I can control the myenergi devices at all times. I've read that most manufacturers provide a "local API" to make this work.

I see companies like Octopus are giving out such devices for free.

Can you raise an enhacement request to provide a (documentend) local API and update the app to use it when possible.

Of course, once this exists, 3rdaprty tools such as home assistant can also make use of it"

@trizmark
Copy link
Contributor

trizmark commented Jul 30, 2024

You can also highlight the fact that if integrations switch to local API, then they could save on AWS costs!

@MaximumFish
Copy link

Cool, I'll send something in to add to the pile!

@githubjonny
Copy link

Hi @CJNE, do we have an ETA for the update? Be fantastic to take advantage of the charge feature for the libbi (opens up some massive savings)

@githubjonny
Copy link

githubjonny commented Aug 19, 2024

Can anyone give me a rough guide on upgrading before the latest version is officially released? Got caught out with a scheduled charge today (left over from yesterday's free electric 1-2pm grrr). So looking to update for charge/discharge full control

@trizmark
Copy link
Contributor

@githubjonny Here's the version that has been merged yesterday. You simply extract it in the custom_components folder of your HA install. (I'd suggest removing the existing myenergi directory from there.)
Once done, restart HA, find the integration on the config screen and select 'Configure' - this is where you can add the app email/password. Without that some of the libbi controls will be 'unavailable'.
myenergi-0.0.28-pre.tgz

@githubjonny
Copy link

githubjonny commented Aug 20, 2024

@trizmark Thank you. I just went to do this now and saw that the update was available in HA :) Ran the update and all went well. However, I can't seem to get the libbi to charge using the "on" toggle on switch.myenergi_libbi_XXXXXX_charge_from_grid (removed id). Clicking doesn't seem to do anything.

When I view logbook I see:
20 August 2024
became unknown
19:34:57 - 19 minutes ago
became unavailable
19:34:55 - 19 minutes ago

I then re-read the release notes and saw :

"In order to access the new functionality, the integration requires the app email and password (to access the OAuth controlled API calls). This can simply be done by 'Reconfiguring' the integration. The app email and password is optional and only required if someone owns a libbi."

I think this may be an issue for me as I use a google login for my myenergi account. Any ideas?
I can change the status of the libbi at the moment (export, normal, stopped) so there is some control there.

@trizmark
Copy link
Contributor

@githubjonny We found a workaround for that! So, you need to create a new ME account with email and pass and invite this new account to manage your house. The component already caters for this (there's an invitation ID in the API that needs to be saved/passed on) and it works fine.

@githubjonny
Copy link

@trizmark thanks for your help with this. I'll do that 1st thing tomorrow. It's interesting I can control the libbi discharge without a standard login though. Wonder if there's a way to control charging with just the API etc.

@ragg987
Copy link

ragg987 commented Aug 20, 2024 via email

@githubjonny
Copy link

see this is why I love open source software. Within a few minutes had two replies both offered solutions.

I just created a new account and can confirm what @ragg987 said works. As soon as I created the account and logged in it had all my details.

Thanks again everyone.

@githubjonny
Copy link

githubjonny commented Aug 20, 2024

Just an update, @trizmark I now can control the charge from the grid toggle :) thanks for your hard work on that one 👍🏼

Just wondered what everyone's doing regarding half hour time slots? I'm assuming two tier pricing with all half hours on the cheap tariff?

Although, now I've said that 'out loud', I guess just one tariff is only needed (assuming ME allows charging on a single tariff).

Nope, 1 tariff = no ability to charge

@trizmark
Copy link
Contributor

trizmark commented Aug 20, 2024

@githubjonny Yeah, two tier pricing and all the slots on the cheap tarriff. That way you can toggle the charge from grid on and off to control when the libbi charges.

Wonder if there's a way to control charging with just the API etc.

The ME API uses two different authentication methods. Most of the old sutff uses digest (based on API key/secret) and some of the new stuff uses OAuth (based on username/password). The ME engineers hinted on the forums that they're planning to move everything over to OAuth, but it looks like that'll take a long time. The mode selection API call is based on old auth (which worked without app email/pass), but charging from grid toggle and charge target setting API calls are based on new auth (which didn't work).

@githubjonny
Copy link

githubjonny commented Sep 7, 2024

Temperature is available on the historic data API https://s18.myenergi.net/cgi-jday-Lxxxxx-yyyy-mm-dd but not the real-time API used by the HA integration. On the historic API there is

Inverter temp
Battery temp
Battery radiator temp
Ambient temp
Controller temp

@MalcolmSpencer , Any guides on how to achieve this anywhere online?

@Future-Surfer
Copy link

Now that we can change operating mode to 'export', do we think there is any scope for setting/limiting charge/discharge rates?

@githubjonny
Copy link

That would be handy if possible. Currently I export my battery before the cheap session starts at a low amp rate (to help battery life). However would prefer to dump battery much quicker during octopus saving session without having to alter settings manually

@trizmark
Copy link
Contributor

trizmark commented Sep 8, 2024

Limiting charge/discharge rates is a no-go at the moment. While technically it would be possible, the feature is not implemented by the controller and it's not exposed via the API.
I can clearly see the modbus messages from the libbi controller that instruct the inverter as to what rate to charge/discharge with, but that's not useful at the moment.

@trizmark
Copy link
Contributor

trizmark commented Sep 8, 2024

Temperature is available on the historic data API https://s18.myenergi.net/cgi-jday-Lxxxxx-yyyy-mm-dd but not the real-time API used by the HA integration. On the historic API there is
Inverter temp
Battery temp
Battery radiator temp
Ambient temp
Controller temp

@MalcolmSpencer , Any guides on how to achieve this anywhere online?

Are you hoping to have this data in HA or just in general? I could add this to pymyenergi, but I don't think there's a way to add historical data to HA (well, not without touching the DB directly).

@Future-Surfer
Copy link

While technically it would be possible, the feature is not implemented by the controller and it's not exposed via the API.

That makes sense. Would be able to tell whether this sort of thing (limiting charge/discharge) is within the realm of possibility for myenergi to expose via the API at some point? Just want to have an idea of where to set my expectations :)

@trizmark
Copy link
Contributor

Possibly at some point, but I don't see that being in the next 12 months. I know they're reworking the libbi controller firmware. Not sure how big the controller FW is and how much space there is for extra features. After all, it's only an ESP32 doing an awful lot of things.

@G6EJD
Copy link

G6EJD commented Sep 29, 2024

Why hasn’t this ’issue’ been closed? It’s become a rambling set of comments.

@plord12 plord12 closed this as completed Sep 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

11 participants