Add secondary object type (#413)

* add secondary object type if not in model * add secondary Object Type * add hazard clipping * bugfix osm setup * remove extra columns in assets if more than 'geometry' column * allow multiple user input for meters and feet * add unit conversion and units for gfh and ground elevation * fix OSM download exposure * update OSM download * update untis class * fix spatialjoins.toml aggregation * change writing lovation * update untis throughout repo * change primary_ to Primary * remove buggy code * fix units and remove hazard clip exposure * update unit to length_unit * bugfix drop index * replace "" string in primary object with np.nan * remove country as exposurevector input
Deltares · Nov 20, 2024 · ed42cf1 · ed42cf1
1 parent da10708
commit ed42cf1
Show file tree

Hide file tree

Showing 36 changed files with 199 additions and 165 deletions.
diff --git a/docs/data/hazus.rst b/docs/data/hazus.rst
@@ -14,11 +14,11 @@ by providing the following values in the configuration file::
     [setup_vulnerability]
     vulnerability_fn = "default_vulnerability_curves"
     vulnerability_identifiers_and_linking_fn = "default_hazus_iwr_linking"
-    unit = "ft"
+    unit = "feet"
 
     [setup_exposure_buildings]
     max_potential_damage = "hazus_max_potential_damages"
-    unit = "m"
+    length_unit = "meters"
 
 
 The HAZUS flood depth-damage functions and replacement values are processed into an easy-to-use format

diff --git a/docs/data/jrc.rst b/docs/data/jrc.rst
@@ -12,11 +12,11 @@ by providing the following values in the configuration file::
     [setup_vulnerability]
     vulnerability_fn = "jrc_vulnerability_curves"
     vulnerability_identifiers_and_linking_fn = "jrc_vulnerability_curves_linking"
-    unit = "m"
+    unit = "meters"
 
     [setup_exposure_buildings]
     max_potential_damage = "jrc_damage_values"
-    unit = "m"
+    length_unit = "meters"
 
 
 The JRC Global flood depth-damage functions and replacement values are processed into an easy-to-use format

diff --git a/docs/user_guide/user_guide_overview.rst b/docs/user_guide/user_guide_overview.rst
@@ -31,7 +31,7 @@ Exposure data can be build from a data_catalog_, a data API key, or by supplying
 to local data on the user's machine. The `asset_locations`, `occupancy_type`, and `max_potential_damage`
 data should be provided as a vector file (e.g. *.shp* or *.gpkg*). The `ground_floor_height` can currently
 only be set to a single value (this will be updated soon!). The `damage_types` should be provided as a
-list of strings (e.g. ["structure", "content"]). The `unit` should be provided as a string (e.g. "m").
+list of strings (e.g. ["structure", "content"]). The `length_unit` should be provided as a string (e.g. "meters").
 See below how the `setup_exposure_buildings` method can be used to build or update the exposure data::
 
    [setup_exposure_buildings]
@@ -40,7 +40,7 @@ See below how the `setup_exposure_buildings` method can be used to build or upda
    damage_types = <List of damage types, e.g. ["structure", "content"]>
    max_potential_damage = <Key in the Data Catalog, data API key, or path to local data>
    ground_floor_height = <For now only a number can be entered here for uniform ground floor heights>
-   unit = <The unit of the values in the exposure data inputs, e.g. if the ground floor height in meters, "m">
+   length_unit = <The unit of the values in the exposure data inputs, e.g. if the ground floor height in meters, "meters">
 
 The following method is used to build or update the **exposure** data:
 
@@ -110,7 +110,7 @@ building footprint or over the whole length of the linestring). The user can use
 a default by setting it to "default". If required, the user can provide the names of the damage functions
 as a list to `functions_mean` or `functions_max` to respectively use the mean or max hazard value for all
 assets that use that damage function. The `damage_types` should be provided as a
-list of strings (e.g. ["structure", "content"]). The `unit` should be provided as a string (e.g. "m").
+list of strings (e.g. ["structure", "content"]). The `unit` should be provided as a string (e.g. "meters").
 The `step_size` should be provided as a float (e.g. 0.01).
 See below how the `setup_vulnerability` method can be used to build or update the exposure data::
 

diff --git a/examples/add_roadnetwork.ipynb b/examples/add_roadnetwork.ipynb
@@ -1135,7 +1135,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.13"
+   "version": "3.10.14"
   }
  },
  "nbformat": 4,

diff --git a/examples/aggregation_zones_example.ipynb b/examples/aggregation_zones_example.ipynb
@@ -448,7 +448,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "fiat_dev",
+   "display_name": "hydromt-fiat-dev",
    "language": "python",
    "name": "python3"
   },
@@ -462,7 +462,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
+   "version": "3.10.14"
   },
   "orig_nbformat": 4
  },

diff --git a/examples/data/aggregation_zones/fiat_model/output/aggregation_zones_test1/settings.toml b/examples/data/aggregation_zones/fiat_model/output/aggregation_zones_test1/settings.toml
@@ -18,5 +18,5 @@ step_size = 0.1
 [exposure.geom]
 csv = "./exposure/exposure.csv"
 crs = "EPSG:4326"
-unit = "ft"
+length_unit = "feet"
 file1 = "./exposure/buildings.gpkg"
diff --git a/examples/data/aggregation_zones/fiat_model/output/aggregation_zones_test2/settings.toml b/examples/data/aggregation_zones/fiat_model/output/aggregation_zones_test2/settings.toml
@@ -18,5 +18,5 @@ step_size = 0.1
 [exposure.geom]
 csv = "./exposure/exposure.csv"
 crs = "EPSG:4326"
-unit = "ft"
+length_unit = "feet"
 file1 = "./exposure/buildings.gpkg"
diff --git a/examples/data/aggregation_zones/fiat_model/settings.toml b/examples/data/aggregation_zones/fiat_model/settings.toml
@@ -20,5 +20,5 @@ file = "./exposure/exposure.csv"
 
 [exposure.geom]
 crs = "EPSG:4326"
-unit = "ft"
+length_unit = "feet"
 file1 = "./exposure/buildings.gpkg"
diff --git a/examples/data/aggregation_zones/fiat_model_aggregation1/settings.toml b/examples/data/aggregation_zones/fiat_model_aggregation1/settings.toml
@@ -20,5 +20,5 @@ file = "./exposure/exposure.csv"
 
 [exposure.geom]
 crs = "EPSG:4326"
-unit = "ft"
+length_unit = "feet"
 file1 = "./exposure/buildings.gpkg"
diff --git a/examples/data/aggregation_zones/fiat_model_aggregation2/settings.toml b/examples/data/aggregation_zones/fiat_model_aggregation2/settings.toml
@@ -20,5 +20,5 @@ file = "./exposure/exposure.csv"
 
 [exposure.geom]
 crs = "EPSG:4326"
-unit = "ft"
+length_unit = "feet"
 file1 = "./exposure/buildings.gpkg"
diff --git a/examples/data/road_network/configuration.yaml b/examples/data/road_network/configuration.yaml
@@ -8,4 +8,4 @@ setup_exposure_roads:
   roads_fn: OSM
   road_types: ["motorway", "primary", "secondary", "tertiary"]
   road_damage: default_road_max_potential_damages
-  unit: meter
+  unit: meters
diff --git a/examples/data/road_network/hydromt_fiat_catalog_USA.yml b/examples/data/road_network/hydromt_fiat_catalog_USA.yml
@@ -13,7 +13,7 @@ default_vulnerability_curves:
   data_type: DataFrame
   driver: csv
   meta:
-    unit: ft
+    unit: feet
     category: vulnerability
     source: HAZUS SQL database, USACE-IWR and FEMA expert elicitation curves. The source of these curves is the Draft Report Nonresidential Flood Depth-Damage Functions Derived from Expert Elicitation, April 2009, Revised 2013. FEMA Contract Number HSFEHQ-06-D-0162. Task Order HSFEHQ-08-J-0014. Not for distribution, but data shared by USACE.
 
@@ -22,15 +22,15 @@ jrc_vulnerability_curves:
   data_type: DataFrame
   driver: xlsx
   meta:
-    unit: m
+    unit: meters
     category: vulnerability
 
 hazus_max_potential_damages:
   path: max_potential_damages/damage_values_fema_hazus-inventory-technical-manual-4.2.3.xlsx
   data_type: DataFrame
   driver: xlsx
   meta:
-    unit: ft
+    unit: feet
     category: vulnerability
 
 social_vulnerability:
@@ -52,6 +52,6 @@ default_road_max_potential_damages:
   data_type: DataFrame
   driver: csv
   meta:
-    unit: ft
+    unit: feet
     category: exposure
     source: Bouwer, Laurens & Haasnoot, Marjolijn & Wagenaar, Dennis & Roscoe, Kathryn. (2018). Assessment of alternative flood mitigation strategies for the C-7 Basin in Miami, Florida.
diff --git a/examples/data/setup_exposure_buildings/test_vulnerability_and_exposure_NSI/settings.toml b/examples/data/setup_exposure_buildings/test_vulnerability_and_exposure_NSI/settings.toml
@@ -20,5 +20,5 @@ file = "./exposure/exposure.csv"
 
 [exposure.geom]
 crs = "EPSG:4326"
-unit = "ft"
+length_unit = "feet"
 file1 = "./exposure/buildings.gpkg"
diff --git a/.../setup_exposure_buildings_with_dem_data/test_vulnerability_and_exposure_NSI/settings.toml b/.../setup_exposure_buildings_with_dem_data/test_vulnerability_and_exposure_NSI/settings.toml
@@ -20,5 +20,5 @@ file = "./exposure/exposure.csv"
 
 [exposure.geom]
 crs = "EPSG:4326"
-unit = "ft"
+length_unit = "feet"
 file1 = "./exposure/buildings.gpkg"
diff --git a/examples/global_OSM_JRC.ipynb b/examples/global_OSM_JRC.ipynb
@@ -203,15 +203,15 @@
     "### Setup vulnerability parameters ###\n",
     "vulnerability_fn = \"jrc_vulnerability_curves\"\n",
     "vulnerability_identifiers_and_linking_fn = \"jrc_vulnerability_curves_linking\"\n",
-    "unit = \"m\"\n",
+    "unit = \"meters\"\n",
     "\n",
     "### Setup exposure parameters ###\n",
     "asset_locations = \"OSM\"\n",
     "occupancy_type = \"OSM\"\n",
     "max_potential_damage = \"jrc_damage_values\"\n",
     "ground_floor_height = 0\n",
     "damage_types = [\"total\"]\n",
-    "unit = \"m\"\n",
+    "length_unit = \"meters\"\n",
     "\n",
     "### Setup output parameters ###\n",
     "output_dir = \"output\"\n",
@@ -251,7 +251,7 @@
     "        \"occupancy_type\": occupancy_type,\n",
     "        \"max_potential_damage\": max_potential_damage,\n",
     "        \"ground_floor_height\": ground_floor_height,\n",
-    "        \"unit\": unit,\n",
+    "        \"length_unit\": unit,\n",
     "        \"damage_types\": damage_types,\n",
     "        \"country\": country,\n",
     "    },\n",
@@ -438,7 +438,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "hydromt-fiat-dev",
    "language": "python",
    "name": "python3"
   },
@@ -452,12 +452,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.12"
-  },
-  "vscode": {
-   "interpreter": {
-    "hash": "be81aace73da0095c79977b505c53aa75bc0c4e17d113b3604023cbe32f7e068"
-   }
+   "version": "3.10.14"
   }
  },
  "nbformat": 4,

diff --git a/hydromt_fiat/api/data_types.py b/hydromt_fiat/api/data_types.py
@@ -13,7 +13,6 @@ class Units(str, Enum):
     meters = "meters"
     feet = "feet"
 
-
 class Conversion(float, Enum):
     meters_to_feet = 3.28084
     feet_to_meters = 0.3048
@@ -81,25 +80,28 @@ class ExposureBuildingsSettings(BaseModel):
     occupancy_type: str
     max_potential_damage: str
     ground_floor_height: Union[str, float]
-    unit: Units
+    gfh_unit: Units = None
+    length_unit: Units
     extraction_method: ExtractionMethod
     damage_types: Optional[List[str]] = None
     damage_unit: str
     country: str = None
     bf_conversion: bool = False
     keep_unclassified: bool = True
+    grnd_elev_unit: Units = None
 
 
 class ExposureSetupGroundFloorHeight(BaseModel):
     source: str
     attribute_name: Optional[Union[str, List[str]]] = None
     gfh_method: Optional[Union[str, List[str]]] = None
     max_dist: Optional[Union[float, int]] = None
+    gfh_unit: Units = None
 
 
 class ExposureSetupGroundElevation(BaseModel):
     source: Union[int, float, None, str]
-    unit: Union[str, Units]
+    grnd_elev_unit: Units = None
 
 
 class ExposureSetupDamages(BaseModel):

diff --git a/hydromt_fiat/api/exposure_vm.py b/hydromt_fiat/api/exposure_vm.py
@@ -66,7 +66,7 @@ def set_asset_locations_source_and_get_data(
         max_potential_damage: str = None,
         fiat_key_maps: Optional[Dict[str, str]] = None,
         crs: Union[str, int] = None,
-        ground_elevation_unit: str = None,
+        grnd_elev_unit: Units = None,
         bf_conversion: bool = False,
         keep_unclassified: bool = True,
     ):
@@ -81,10 +81,9 @@ def set_asset_locations_source_and_get_data(
                 logger=self.logger,
                 region=region,
                 crs=crs,
-                unit=Units.feet.value,
-                country="United States",
+                length_unit=Units.feet.value,
             )
-
+        
             self.exposure.setup_buildings_from_single_source(
                 source,
                 ground_floor_height,
@@ -121,9 +120,9 @@ def set_asset_locations_source_and_get_data(
                 logger=self.logger,
                 region=region,
                 crs=crs,
-                unit=Units.meters.value,
-                country=country,
-            )
+                length_unit=Units.meters.value,
+                )
+
             self.exposure.setup_buildings_from_multiple_sources(
                 asset_locations=source,
                 occupancy_source=source,
@@ -132,7 +131,7 @@ def set_asset_locations_source_and_get_data(
                 extraction_method="centroid",
                 damage_types=["structure", "content"],
                 country=country,
-                ground_elevation_unit=ground_elevation_unit,
+                grnd_elev_unit=grnd_elev_unit,
                 bf_conversion=bf_conversion,
                 keep_unclassified=keep_unclassified,
             )
@@ -168,7 +167,7 @@ def set_asset_locations_source(
                 occupancy_type=source,
                 max_potential_damage=source,
                 ground_floor_height=ground_floor_height,
-                unit=Units.feet.value,  # TODO: make flexible
+                length_unit=Units.feet.value, 
                 extraction_method=ExtractionMethod.centroid.value,
                 damage_types=["structure", "content"],
                 damage_unit=Currency.dollar.value,
@@ -199,7 +198,7 @@ def set_asset_locations_source(
                 keep_unclassified=keep_unclassified,
                 max_potential_damage=max_potential_damage,
                 ground_floor_height=ground_floor_height,
-                unit=Units.meters.value,  # TODO: make flexible
+                length_unit=Units.meters.value,
                 extraction_method=ExtractionMethod.centroid.value,
                 damage_types=["structure", "content"],
                 damage_unit=Currency.euro.value,
@@ -283,10 +282,10 @@ def set_ground_elevation(
             None,
             str,
         ],
-        unit: Union[str, Units],
+        grnd_elev_unit: Units,
     ):
         self.exposure_ground_elevation_model = ExposureSetupGroundElevation(
-            source=source, unit=unit
+            source=source, grnd_elev_unit=grnd_elev_unit
         )
 
     def set_roads_settings(

diff --git a/hydromt_fiat/api/hydromt_fiat_vm.py b/hydromt_fiat/api/hydromt_fiat_vm.py
@@ -252,8 +252,8 @@ def new_additional_attributes(self, config_yaml):
 
     def new_ground_elevation(self, config_yaml):
         source = config_yaml.model_extra["update_ground_elevation"].source
-        unit = config_yaml.model_extra["update_ground_elevation"].unit
-        self.fiat_model.exposure.setup_ground_elevation(source, unit)
+        grnd_elev_unit = config_yaml.model_extra["update_ground_elevation"].grnd_elev_unit
+        self.fiat_model.exposure.setup_ground_elevation(source, grnd_elev_unit)
 
     def new_max_potential_damages(self, config_yaml):
         source = config_yaml.model_extra["update_max_potential_damage"].source

diff --git a/hydromt_fiat/data/hydromt_fiat_catalog_USA.yml b/hydromt_fiat/data/hydromt_fiat_catalog_USA.yml
@@ -14,7 +14,7 @@ default_vulnerability_curves:
   data_type: DataFrame
   driver: csv
   meta:
-    unit: ft
+    unit: feet
     category: vulnerability
     source: HAZUS SQL database, USACE-IWR and FEMA expert elicitation curves. The source of these curves is the Draft Report Nonresidential Flood Depth-Damage Functions Derived from Expert Elicitation, April 2009, Revised 2013. FEMA Contract Number HSFEHQ-06-D-0162. Task Order HSFEHQ-08-J-0014. Not for distribution, but data shared by USACE.
 
@@ -23,15 +23,15 @@ jrc_vulnerability_curves:
   data_type: DataFrame
   driver: xlsx
   meta:
-    unit: m
+    unit: meters
     category: vulnerability
 
 hazus_max_potential_damages:
   path: max_potential_damages/damage_values_fema_hazus-inventory-technical-manual-4.2.3.xlsx
   data_type: DataFrame
   driver: xlsx
   meta:
-    unit: ft
+    unit: feet
     category: vulnerability
 
 social_vulnerability:
@@ -60,7 +60,7 @@ default_road_max_potential_damages:
   data_type: DataFrame
   driver: csv
   meta:
-    unit: ft
+    unit: feet
     category: exposure
     source: Bouwer, Laurens & Haasnoot, Marjolijn & Wagenaar, Dennis & Roscoe, Kathryn. (2018). Assessment of alternative flood mitigation strategies for the C-7 Basin in Miami, Florida.