fix(parsers): handle missing storage values for BE

[VIKTORVAV99]

Issue

BE don't report data properly so it's now getting flagged and estimated by the new validation rules in the parser.

Description

The code changes in parsers/ENTSOE.py address the issue of missing production values for the BE (Belgium) zone. Previously, if the number of production values received for a specific datetime was one less than the expected length, a warning was logged and the datapoint was discarded. However, for the BE zone, this discrepancy is expected due to the absence of 0 values for storage. Therefore, the code now includes a specific check for the BE zone and allows for the expected length to be one less than the actual number of values.

Note

I will also open a ticked with ENTSO-E and see if they can fix it on their end.

This is the issue:

Those blank spaces should be filled with values, probably 0.

Double check

• I have tested my parser changes locally with poetry run test_parser "zone_key"
• I have run pnpx prettier@2 --write . and poetry run format in the top level directory to format my changes.

[madsnedergaard]

Hmm, this overall makes sense, but there is a risk that an important mode might be missing and thus this would falsely parse. But up to you to decide if that is enough of a problem/risk to do more work here :)

[VIKTORVAV99]

Agreed but the alternative would be to set up a validation that checks all expected modes per zone. Possibly something we can and should do but I think this is fine for now as it's just limited to BE and there is a secondary level of validation that is happening in the backend as well.

[madsnedergaard]

Approving so you can continue, but see comment

[VIKTORVAV99]

To follow up on this:
I reached out to ENTSO-E which got back to me with the following from Elia:

"Values published by Elia are either production either consumption. When flow goes into one direction, we don’t send 0 for the opposite direction, and this is for us a normal behavior.
This being said, I can understand your concern in relation with API structure, and I will provide your valuable feedback to my colleagues in business and IT, in order to fix this. I cannot give your at this stage neither an engagement from our side, nor a delay for adaptation."

Hopefully this means they will start sending 0 values and we can remove this again in the future.