Working on new HTTP + JSON Powermeter fetcher - inputs welcome

[berni2288]

Some people said that they would like to have support for HTTP + Json Powermeter reading, because they can't use MQTT.

Examples:

• Volkszähler / Energy meter + Tasmota
• Other Power meters that provide the current power consumption via HTTP and Json

I just started implementing this, but I need 2 things for this:

Wanted: HTTP request client library

• Should support HTTPS/SSL because this is state of the art nowadays and not supporting SSL seems weird to me
• If possible it should support async requests, because the ESP32 has a watchdog and could probably restart device if the request is blocking too long.
• Should support ESP32
• The Arduino HTTPClient class does not support async it seems

Possible candiates:

• https://github.com/khoih-prog/AsyncHTTPRequest_Generic - can do both things, but for some reason this repo is archived, so I would not use this.
• AsyncWebSocket.h from me-no-dev / ESPAsyncWebServer - suggested by @helgeerbe gonna have a look at this.
• esp_http_client - From espressif - looks promising

Does anyone know something good?

Wanted: An ESP32 Json library with JsonPath support

• Should support JsonPath or some similar feature where you can select a single value in a Json document by providing some kind of path in one string.**
• ArduinoJson can't do this
• I've found the project https://github.com/mobizt/FirebaseJson which can fetch a value from a path provided by a string

Any other inputs/ideas are also welcome!

Current Webview design:

[helgeerbe]

Why not using the AsyncWebSocketClient from AsyncWebsocketServer?

[berni2288]

Does it support HTTPS/SSL?

I also just found this: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/protocols/esp_http_client.html
this also looks promising.

[berni2288]

But a websocket is not quite what I want. web sockets are a subset of the HTTP protocol.

[helgeerbe]

You are right. If I remember right, Tasmota does not support web sockets. So the only thing what you can do ist to poll in main loop Tasmota. There is no asynchronus connection and if I remember right, Tasmota is not supporting ssl by default. It is possible to compile Tasmota with ssl support. But I doubt, that is really used.

[berni2288]

It's not just about Tasmota, it should be a generic solution for other power meters too. Maybe some support https or are cloud based? who knows.
SSL connections are state of the art today and if not necessary I would not want to send my credentials unencrypted over a network, you never know who is listening.
I could live without asynchronous requests.
Thanks, I'll just evaluate all the available options.

[Adminius]

Some Ideas and information (in german):
lumapu/ahoy#606 (comment)
lumapu/ahoy#606 (comment)

I mean, we need 3 URLs and not only one.
Some power meters have splitted values for export and input power and both are positive but only one is >0 simultaneously

[berni2288]

I decided to go with the arduino-esp32/HTTPClient. It seems to be a common way to do HTTP requests, It supports http and https, but there is no support for async mode.

I actually tried the esp_http_client first and would have preferred it because it seems quite major, but there were too many problems with it: async mode is supported for HTTPS only, there were conflicts with the HTTP_ enum with AsyncWebServer that could only be fixed by doing a lot of changes in the AsyncWebServer library and there were crashes during reading the HTTP response. The HTTP_ enum conflict is not only esp_http_client's fault.
Note to the makers of esp_http_client & ESPAsyncWebServer: polluting the global namespace is a really terrible idea!!!11

Since there will be no async requests, I'l implement a default timeout of 1 second, that can be customized. I think this should do it and safe the ESP32 from reboots when the Powermeter is not reachable.

For Json parsing by path, I'll try out https://github.com/mobizt/FirebaseJson .

[berni2288]

PR is open now #153
reviews are welcome.