Add endogenous transport option

[s8au]

• select with endogenous_transport in config file
• it is possible to select transport shares for part of the fuel types
• creates ICE-vehicle-fleet for baseyear if myopic is selected and no transport share is defined
• adds constraints for endogenously chosen EVs in solve_network

relies on PR in technology-data repository: BertoGBG:master
Closes #507.

Changes proposed in this Pull Request

Checklist

• I tested my contribution locally and it seems to work fine.
• Code and workflow changes are sufficiently documented.
• Changed dependencies are added to envs/environment.yaml.
• Changes in configuration options are added in all of config.default.yaml.
• Changes in configuration options are also documented in doc/configtables/*.csv.
• A release note doc/release_notes.rst is added.

[martavp]

Many thanks @s8au , this looks great!

I still need to run the new code and I'll do it soon, but in the meantime see some suggestions based on reading the changes.

I think you can check "Changed dependencies are added to envs/environment.yaml." since no changes are required.

Try also to add a release note to doc/release_notes.rst

[martavp]

Why do lines 572 to 668 appear as new additions? They are not related to the endogenous transport

[martavp]

The current definition of p_nom for the link "land transport EV" needs to be changed.
Currently, it is based on the maximum value of the transport demand

n.links.loc[
(n.links.carrier == "land transport EV")
& (n.links.bus1.str.contains(place)),
"p_nom",
] = electric_share * max(p_nom[place])

This represents the minimum number of cars needed to supply that demand, but in reality, cars are most of the time parked and we should use the total number of cars, as it was done before

pypsa-eur/scripts/prepare_sector_network.py

Line 1501 in 7069429

p_nom = number_cars * options.get("bev_charge_rate", 0.011) * electric_share

This is important to avoid problems when calculating the cost associated with cars.

[martavp]

There is a bug when adding the storage for EVs exogenous. The following code

n.stores.loc[
(n.stores.carrier == "EV battery storage")
& (n.links.bus1.str.contains(place)),
"e_nom",
] = (
number_cars[place]
* options.get("bev_energy", 0.05)
* options["bev_availability"]
* electric_share
)

Should be:

n.stores.loc[
(n.stores.carrier == "EV battery storage")
& (n.stores.bus.str.contains(place)),
"e_nom",
] = (
number_cars[place]
* options.get("bev_energy", 0.05)
* options["bev_availability"]
* electric_share
)

[martavp]

The constraint relating the BEV charge rate and the storage capacity of the batteries, needs to take into account that in the config file it can be selected that only a percentage of the BEV provides flexibility

lhs2 = (
n.model.variables["Store-e_nom"].loc[ev_store]
/ n.config["sector"]["bev_energy"]
)

should be:

lhs2 = (
n.model.variables["Store-e_nom"].loc[ev_store]
/ (n.config["sector"]["bev_energy"]*n.config["sector"]["bev_availability"])
)

[martavp]

EDITED on 20/09/2023 at 8:35

I found a bug in the definition of p_max_pu and p_min_pu of the link "land transport oil"
The following

n.links_t.p_max_pu[
n.links[
(n.links.carrier == "land transport oil")
& (n.links.bus1.str.contains(place))
].index.values[0]
] = ((p_nom[place]) / max(p_nom[place]) * ice_efficiency / p_set[place])

should be:

n.links_t.p_max_pu[
n.links[
(n.links.carrier == "land transport oil")
& (n.links.bus1.str.contains(place))
].index.values[0]
] = p_nom[place]/ (max(p_nom[place]* ice_efficiency)

otherwise, since p_nom and p_set are defined to be equal on a previous line, p_max_pu becomes a constant number and the flow of energy through that link is not able to supply the land transport demand.

The definition of the efficiency in the link should also be modified:

the current definition:

efficiency=1.0
/ options["transport_internal_combustion_efficiency"]
* p_set[nodes],

should be:

efficiency=options["transport_internal_combustion_efficiency"]

IMPORTANT: This should be changed for p_max_pu and p_min pu of both links 'land transport oil' and 'land transport fuel cell'

[lisazeyen]

@martavp @s8au thanks a lot for all your work on this!

Tom and I just had a look at the code since we would like to have the endogenous land transport in the main repository quite soon. Two main points came to our mind:

1.) As a first step it made make sense to fix the number of cars and not allow switching between cars by assuming a p_min_pu=p_max_pu=profile transport demand for all three links (EV, H2 Fuel Cell and ICEs). This should also speed up the solving.
2.) Further the transport demand profile p_set should not be shifted since this was only done for smoothing the EV charging. We can first just leave out the shifting completely when defining p_set . In a second step one could think about adding a time-dependent efficiency to the EV links to account for different charging times.

@s8au would you have time to include it in the next two weeks or should we give it a try? Did you have time to check also for newer cost assumptions for EVs?

[s8au]

@lisazeyen

Tom and I just had a look at the code since we would like to have the endogenous land transport in the main repository quite soon. Two main points came to our mind:

1.) As a first step it made make sense to fix the number of cars and not allow switching between cars by assuming a p_min_pu=p_max_pu=profile transport demand for all three links (EV, H2 Fuel Cell and ICEs). This should also speed up the solving. 2.) Further the transport demand profile p_set should not be shifted since this was only done for smoothing the EV charging. We can first just leave out the shifting completely when defining p_set . In a second step one could think about adding a time-dependent efficiency to the EV links to account for different charging times.

@s8au would you have time to include it in the next two weeks or should we give it a try? Did you have time to check also for newer cost assumptions for EVs?

Thank you both for looking over it and your suggestions. I will try to include the p_min_pu/p_max_pu constraint and remove the smoothing next week.
I looked a bit around but so far I only found a datasheet for commercial freight and passenger transport . Did you have something specific in mind?

[lisazeyen]

@lisazeyen

Tom and I just had a look at the code since we would like to have the endogenous land transport in the main repository quite soon. Two main points came to our mind:
1.) As a first step it made make sense to fix the number of cars and not allow switching between cars by assuming a p_min_pu=p_max_pu=profile transport demand for all three links (EV, H2 Fuel Cell and ICEs). This should also speed up the solving. 2.) Further the transport demand profile p_set should not be shifted since this was only done for smoothing the EV charging. We can first just leave out the shifting completely when defining p_set . In a second step one could think about adding a time-dependent efficiency to the EV links to account for different charging times.
@s8au would you have time to include it in the next two weeks or should we give it a try? Did you have time to check also for newer cost assumptions for EVs?

Thank you both for looking over it and your suggestions. I will try to include the p_min_pu/p_max_pu constraint and remove the smoothing next week. I looked a bit around but so far I only found a datasheet for commercial freight and passenger transport . Did you have something specific in mind?

Great! I think the technology data is a good source for the heavy duty vehicles. For the light vehicles I guess there is a Bloomberg report for at least EVs. But I think every source for light vehicles which is a bit more up to date (like after 2020) should be fine!

[martavp]

Great! I think the technology data is a good source for the heavy duty vehicles. For the light vehicles I guess there is a Bloomberg report for at least EVs. But I think every source for light vehicles which is a bit more up to date (like after 2020) should be fine!

BNEF include cost assumption for trucks in Figure 4 in
https://assets.bbhub.io/professional/sites/24/2431510_BNEFElectricVehicleOutlook2023_ExecSummary.pdf
(here, we agree that using DEA is better)

and cost assumptions for EVs until 2030 in Figure 28. https://www.transportenvironment.org/discover/hitting-the-ev-inflection-point/
(because the data only covers until 2030 and the report includes a disclaimer saying that we should not use the data, I suggest we keep using Fraunhofer data), unless @s8au and I have missed some additional BNEF database?

But you are totally right @lisazeyen that is too pessimistic about EV costs. See below the cost comparison for EVs between our current assumptions and BNEF (red dots). We will implement a cost sensitivity analysis soon so that we can learn about the impacts.

[lisazeyen]

Great! I think the technology data is a good source for the heavy duty vehicles. For the light vehicles I guess there is a Bloomberg report for at least EVs. But I think every source for light vehicles which is a bit more up to date (like after 2020) should be fine!

BNEF include cost assumption for trucks in Figure 4 in https://assets.bbhub.io/professional/sites/24/2431510_BNEFElectricVehicleOutlook2023_ExecSummary.pdf (here, we agree that using DEA is better)

and cost assumptions for EVs until 2030 in Figure 28. https://www.transportenvironment.org/discover/hitting-the-ev-inflection-point/ (because the data only covers until 2030 and the report includes a disclaimer saying that we should not use the data, I suggest we keep using Fraunhofer data), unless @s8au and I have missed some additional BNEF database?

But you are totally right @lisazeyen that is too pessimistic about EV costs. See below the cost comparison for EVs between our current assumptions and BNEF (red dots). We will implement a cost sensitivity analysis soon so that we can learn about the impacts.

Yes, I guess a cost sensitivity is the best way!

[s8au]

The current definition of p_nom for the link "land transport EV" needs to be changed. Currently, it is based on the maximum value of the transport demand

n.links.loc[
(n.links.carrier == "land transport EV")
& (n.links.bus1.str.contains(place)),
"p_nom",
] = electric_share * max(p_nom[place])

This represents the minimum number of cars needed to supply that demand, but in reality, cars are most of the time parked and we should use the total number of cars, as it was done before

pypsa-eur/scripts/prepare_sector_network.py

Line 1501 in 7069429

p_nom = number_cars * options.get("bev_charge_rate", 0.011) * electric_share

This is important to avoid problems when calculating the cost associated with cars.

I updated it for the exogenous land transport EV
I also indicate with the 🚀 that I fixed the problems in the other comments

[martavp]

This link represents the "capacity" of EVs to supply transport demand.
However, the current definition represents the capacity for charging the cars
since options.get("bev_charge_rate", 0.011) is the capacity to charge a car.
@s8au can you change this? (similarly to the exogenous capacity definition for H2 and ICE cars)?

[martavp]

Hi @s8au ,

I mentioned to @lisazeyen yesterday that she can start reviewing this.
To facilitate the review, Can you update the PR to the last version of the master branch and check that everything run smoothly? This facilitates the revision (currently endogenous-transport-related and non-related changes are shown and make the revision cumbersome)

There are two additional small issues:

[ ] 1. I think the link representing the "capacity" of EVs to supply transport demand is not correct.
The current definition represents the capacity for charging the cars
since options.get("bev_charge_rate", 0.011) is the capacity to charge a car.
Can you please correct this? (similarly to the exogenous capacity definition for H2 and ICE cars)?

[] 2. I understand how you estimated the capacity of the "land transport oil." In essence, you calculate the minimum capacity needed to supply the demand for land transport today and then decommission a percentage of it in every future planning horizon. The approach is fine for me.

However, in reality, we have many more cars than "needed". Can you calculate the ratio between the existing number of cars and the cars that you calculate that are needed to supply the demand and write it as a log message?
Something like:
"Although, existing ICE vehicles in 2020 account for XXXX, the model assumes only the minimum number of ICE vehicles to supply the transport demand which represents XXX"

[s8au]

@martavp @lisazeyen I updated the branch with the master branch and I ran a test case for the exogenous and the endogenous transport each (resulting in again being behind by 4 commits). Since these are only minor changes I would prefer to update them with additional changes to my code.

[lisazeyen]

this shift of demand was done to reflect that EV charging is smoothed out and not done exactly when the car is driven but it is not necessary e.g. for internal combustion cars. I would suggest in a first step to leave the cycling_shift out of the transport demand. In a second step one could think about adding a time-dependent efficiency to the BEV link to account for different charging times

[lisazeyen]

I would suggest adding a p_max_pu and p_min_pu which are equal and both follow the demand profile from p_set. The same should be done for the links of the ICE and the H2 fuel cell cars to keep the share constant. I know we discussed that people are also switching cars (e.g. they have an EV for short distances and an ICE for longer distances) but as a first step I think that is the easiest way and might speed up the optimisation as well since there are less decision variables.

[lisazeyen]

The build_transport_demand script did not change in your PR, so shouldn't it stay as before efficiency=options.get("bev_charge_efficiency", 0.9) ?

[s8au]

I updated the PR with the newest version, so we can discuss the changes, but I still have to check if the added code in the add_brownfield works because the Gurobi license on our cluster has expired.
@lisazeyen
I had to move the calculation of the profiles for the p_min_pu/p_max_pu constraints after the links have their new temporal resolution because the efficiencies are now time dependent with the temperature corrections. Due to this, taking the mean of the profiles gives wrong values.
Gurobi has some trouble with setting these profiles and then solving the land transport when the links p_nom_extendable is set to True. I am currently trying to only constrain two out of three links directly (which works fine) and then constrain the third link for the myopic pathway in the add_brownfield script (WIP).
The exogenous pathway should work fine, it runs - depending on the CO2 budget - until 2040/2050 before becoming infeasible. But that was also what I experienced with the unmodified code from the repository.

[s8au]

@martavp I added a commit with the fixed bug, so it should run now

[lisazeyen]

@s8au @martavp I am currently planning to integrate as the land transport demand as one single node and then the links for the different car types. I made a suggestion, which you can see the PR here #957.