Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Onboard Iris wind model #146

Merged
merged 6 commits into from
Sep 17, 2023
Merged

Onboard Iris wind model #146

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a367585
Wind becomes accute; tidy vulnerability models.
joemoorhouse Sep 3, 2023
637366a
Update test
joemoorhouse Sep 5, 2023
a5803da
Filter out NaNs in hazard requests (for wind data)
joemoorhouse Sep 16, 2023
0d66a7a
Merge branch 'main' into iris
joemoorhouse Sep 17, 2023
ed5acdd
Remove partial test
joemoorhouse Sep 17, 2023
d313d09
Add description
joemoorhouse Sep 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ pip install physrisk-lib
Access to hazard event data requires setting of environment variables specifying the S3 Bucket, for example:

```
OSC_S3_BUCKET=redhat-osc-physical-landing-647521352890
OSC_S3_BUCKET=physrisk-hazard-indicators
OSC_S3_ACCESS_KEY=**********6I
OSC_S3_SECRET_KEY=**********mS
```
Expand Down
2 changes: 1 addition & 1 deletion docs/handbook/onboarding.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Specific types of model also exist for common modelling approaches. In particula
On-boarding a model based on a damage/disruption curve
------------------------------------------------------

A vulnerability distribition can be inferred directly from a damage/disruption curve:
A vulnerability distribution can be inferred directly from a damage/disruption curve:

.. image:: onboarding/disruption_curve.png
:width: 500
Expand Down
193 changes: 174 additions & 19 deletions ...U JRC global flood depth-damage functions/EU JRC global flood depth-damage functions.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,47 +256,142 @@
{
"asset_type": "Buildings/Commercial",
"event_type": "RiverineInundation",
"impact_mean": [],
"impact_mean": [
0.0,
0.15,
0.3,
0.45,
0.55,
0.75,
0.9,
1.0,
1.0
],
"impact_std": [],
"impact_type": "Damage",
"intensity": [],
"intensity": [
0.0,
0.5,
1.0,
1.5,
2.0,
3.0,
4.0,
5.0,
6.0
],
"intensity_units": "m",
"location": "Europe"
},
{
"asset_type": "Buildings/Commercial",
"event_type": "RiverineInundation",
"impact_mean": [
0.0
],
"impact_std": [
0.0
0.0,
0.018404908,
0.239263804,
0.374233129,
0.466257669,
0.552147239,
0.687116564,
0.82208589,
0.90797546,
1.0
],
"impact_std": [],
"impact_type": "Damage",
"intensity": [
0.0,
0.01
0.01,
0.5,
1.0,
1.5,
2.0,
3.0,
4.0,
5.0,
6.0
],
"intensity_units": "m",
"location": "North America"
},
{
"asset_type": "Buildings/Commercial",
"event_type": "RiverineInundation",
"impact_mean": [],
"impact_std": [],
"impact_mean": [
0.0,
0.611477587,
0.839531094,
0.923588457,
0.991972477,
1.0,
1.0,
1.0,
1.0
],
"impact_std": [
0.0,
0.077023435,
0.035924027,
0.026876525,
0.016055046,
0.0,
0.0,
0.0,
0.0
],
"impact_type": "Damage",
"intensity": [],
"intensity": [
0.0,
0.5,
1.0,
1.5,
2.0,
3.0,
4.0,
5.0,
6.0
],
"intensity_units": "m",
"location": "South America"
},
{
"asset_type": "Buildings/Commercial",
"event_type": "RiverineInundation",
"impact_mean": [],
"impact_std": [],
"impact_mean": [
0.0,
0.376789623,
0.537681619,
0.659336684,
0.762845232,
0.883348656,
0.941854895,
0.98075938,
1.0
],
"impact_std": [
0.0,
0.240462285,
0.240596279,
0.243605156,
0.250253511,
0.171703625,
0.11240992,
0.052781064,
0.0
],
"impact_type": "Damage",
"intensity": [],
"intensity": [
0.0,
0.5,
1.0,
1.5,
2.0,
3.0,
4.0,
5.0,
6.0
],
"intensity_units": "m",
"location": "Asia"
},
Expand All @@ -306,27 +401,87 @@
"impact_mean": [],
"impact_std": [],
"impact_type": "Damage",
"intensity": [],
"intensity": [
0.0,
0.5,
1.0,
1.5,
2.0,
3.0,
4.0,
5.0,
6.0
],
"intensity_units": "m",
"location": "Africa"
},
{
"asset_type": "Buildings/Commercial",
"event_type": "RiverineInundation",
"impact_mean": [],
"impact_std": [],
"impact_mean": [
0.0,
0.238953575,
0.481199682,
0.673795091,
0.864583333,
1.0,
1.0,
1.0,
1.0
],
"impact_std": [
0.0,
0.142878204,
0.204113206,
0.190903594,
0.178000078,
0.0,
0.0,
0.0,
0.0
],
"impact_type": "Damage",
"intensity": [],
"intensity": [
0.0,
0.5,
1.0,
1.5,
2.0,
3.0,
4.0,
5.0,
6.0
],
"intensity_units": "m",
"location": "Oceania"
},
{
"asset_type": "Buildings/Commercial",
"event_type": "RiverineInundation",
"impact_mean": [],
"impact_mean": [
0.0,
0.323296918,
0.506529105,
0.63459558,
0.744309656,
0.864093044,
0.932788157,
0.977746968,
1.0
],
"impact_std": [],
"impact_type": "Damage",
"intensity": [],
"intensity": [
0.0,
0.5,
1.0,
1.5,
2.0,
3.0,
4.0,
5.0,
6.0
],
"intensity_units": "m",
"location": "Global"
},
Expand Down
6 changes: 3 additions & 3 deletions notebooks/vulnerability_onboarding/EU JRC global flood depth-damage functions/onboard.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -31,7 +31,7 @@
"# consistent with physrisk continent definition\n",
"location_mappings = { \"Europe\": \"Europe\", \"North America\": \"North America\", \"Central & South America\": \"South America\", \"Asia\": \"Asia\", \"Africa\": \"Africa\", \"Oceania\": \"Oceania\", \"Global\": \"Global\" }\n",
"type_mappings = { \"Residential buildings\": \"Buildings/Residential\",\n",
" \"Commerical buildings\": \"Buildings/Commercial\",\n",
" \"Commercial buildings\": \"Buildings/Commercial\",\n",
" \"Industrial buildings\": \"Buildings/Industrial\"\n",
"}\n",
"\n",
Expand Down Expand Up @@ -89,7 +89,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
"version": "3.8.13"
},
"orig_nbformat": 4
},
Expand Down
22 changes: 18 additions & 4 deletions src/physrisk/data/hazard_data_provider.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,14 @@ def __init__(
super().__init__(get_source_path, store=store, zarr_reader=zarr_reader, interpolation=interpolation)

def get_intensity_curves(
self, longitudes: List[float], latitudes: List[float], *, indicator_id: str, scenario: str, year: int
self,
longitudes: List[float],
latitudes: List[float],
*,
indicator_id: str,
scenario: str,
year: int,
hint: Optional[HazardDataHint] = None,
):
"""Get intensity curve for each latitude and longitude coordinate pair.

Expand All @@ -83,7 +90,7 @@ def get_intensity_curves(
return_periods: return periods in years.
"""

path = self._get_source_path(indicator_id=indicator_id, scenario=scenario, year=year)
path = self._get_source_path(indicator_id=indicator_id, scenario=scenario, year=year, hint=hint)
curves, return_periods = self._reader.get_curves(
path, longitudes, latitudes, self._interpolation
) # type: ignore
Expand All @@ -104,7 +111,14 @@ def __init__(
super().__init__(get_source_path, store=store, zarr_reader=zarr_reader, interpolation=interpolation)

def get_parameters(
self, longitudes: List[float], latitudes: List[float], *, indicator_id: str, scenario: str, year: int
self,
longitudes: List[float],
latitudes: List[float],
*,
indicator_id: str,
scenario: str,
year: int,
hint: Optional[HazardDataHint] = None,
):
"""Get hazard parameters for each latitude and longitude coordinate pair.

Expand All @@ -119,6 +133,6 @@ def get_parameters(
parameters: numpy array of parameters
"""

path = self._get_source_path(indicator_id=indicator_id, scenario=scenario, year=year)
path = self._get_source_path(indicator_id=indicator_id, scenario=scenario, year=year, hint=hint)
parameters, _ = self._reader.get_curves(path, longitudes, latitudes, self._interpolation)
return parameters[:, 0]
2 changes: 1 addition & 1 deletion src/physrisk/data/image_creator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ def _to_image(
# (from zarr 2.16.0 we can also use block indexing)
data = data[index, 256 * tile.y : 256 * (tile.y + 1), 256 * tile.x : 256 * (tile.x + 1)]

if any(dim > 1500 for dim in data.shape):
if any(dim > 4000 for dim in data.shape):
raise Exception("dimension too large (over 1500).")
map_defn = colormap_provider.colormap(colormap)

Expand Down
20 changes: 12 additions & 8 deletions src/physrisk/data/pregenerated_hazard_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
from collections import defaultdict
from typing import Dict, List, Mapping, MutableMapping, Optional, cast

import numpy as np

from physrisk.data.zarr_reader import ZarrReader
from physrisk.kernel.hazards import Hazard, HazardKind

Expand Down Expand Up @@ -40,24 +42,26 @@ def get_hazard_events(self, requests: List[HazardDataRequest]) -> Mapping[Hazard
responses: MutableMapping[HazardDataRequest, HazardDataResponse] = {}
for key in batches.keys():
batch: List[HazardDataRequest] = batches[key]
event_type, indicator_id, scenario, year = (
hazard_type, indicator_id, scenario, year, hint = (
batch[0].hazard_type,
batch[0].indicator_id,
batch[0].scenario,
batch[0].year,
batch[0].hint,
)
longitudes = [req.longitude for req in batch]
latitudes = [req.latitude for req in batch]
if event_type.kind == HazardKind.acute: # type: ignore
intensities, return_periods = self.acute_hazard_data_providers[event_type].get_intensity_curves(
longitudes, latitudes, indicator_id=indicator_id, scenario=scenario, year=year
if hazard_type.kind == HazardKind.acute: # type: ignore
intensities, return_periods = self.acute_hazard_data_providers[hazard_type].get_intensity_curves(
longitudes, latitudes, indicator_id=indicator_id, scenario=scenario, year=year, hint=hint
)

for i, req in enumerate(batch):
responses[req] = HazardEventDataResponse(return_periods, intensities[i, :])
elif event_type.kind == HazardKind.chronic: # type: ignore
parameters = self.chronic_hazard_data_providers[event_type].get_parameters(
longitudes, latitudes, indicator_id=indicator_id, scenario=scenario, year=year
valid = ~np.isnan(intensities[i, :])
responses[req] = HazardEventDataResponse(return_periods[valid], intensities[i, :][valid])
elif hazard_type.kind == HazardKind.chronic: # type: ignore
parameters = self.chronic_hazard_data_providers[hazard_type].get_parameters(
longitudes, latitudes, indicator_id=indicator_id, scenario=scenario, year=year, hint=hint
)

for i, req in enumerate(batch):
Expand Down
Loading