Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fiona <-> OGR field mappings take 2 (refactor) #1366

Merged
merged 14 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 19 additions & 7 deletions CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,23 @@ All issue numbers are relative to https://github.com/Toblerity/Fiona/issues.

Deprecations:

The Python style of rio-filter expressions introduced in version 1.0 are
deprecated. Only the parenthesized list type of expression will be supported by
version 2.0.
- The FIELD_TYPES, FIELD_TYPES_MAP, and FIELD_TYPES_MAP_REV attributes
of fiona.schema are no longer used by the project and will be removed
in version 2.0 (#1366).
- The Python style of rio-filter expressions introduced in version 1.0 are
deprecated. Only the parenthesized list type of expression will be supported
by version 2.0.

New features:

The filter, map, and reduce CLI commands from the public domain version 1.1.0
of fio-planet have been incorporated into Fiona's core set of commands (#1362).
These commands are only available if pyparsing and shapely (each of these are
declared in the "calc" set of extra requirements) are installed.
- All supported Fiona field types are now represented by classes in
fiona.schema. These classes are mapped in FIELD_TYPES_MAP2 and
FIELD_TYPES_MAP2_REV to OGR field type and field subtype pairs
(#1366).
- The filter, map, and reduce CLI commands from the public domain version 1.1.0
of fio-planet have been incorporated into Fiona's core set of commands
(#1362). These commands are only available if pyparsing and shapely (each of
these are declared in the "calc" set of extra requirements) are installed.

Bug fixes:

Expand All @@ -33,6 +40,11 @@ Bug fixes:
- Openers are now registered only by urlpath. The mode is no longer considered
as OGR drivers may use a mix of modes when creating a new dataset.

Other changes:

- Feature builder and field getter/setter instances are reused when
reading and writing features (#1366).

1.10a1 (2024-03-01)
-------------------

Expand Down
14 changes: 4 additions & 10 deletions fiona/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
get_gdal_version_tuple,
)
from fiona._env import driver_count
from fiona._path import _ParsedPath, _UnparsedPath, _parse_path, _vsi_path
from fiona._show_versions import show_versions
from fiona._vsiopener import _opener_registration
from fiona.collection import BytesCollection, Collection
Expand All @@ -54,15 +55,8 @@
from fiona.errors import FionaDeprecationWarning
from fiona.io import MemoryFile
from fiona.model import Feature, Geometry, Properties
from fiona.ogrext import (
FIELD_TYPES_MAP,
_bounds,
_listdir,
_listlayers,
_remove,
_remove_layer,
)
from fiona._path import _ParsedPath, _UnparsedPath, _parse_path, _vsi_path
from fiona.ogrext import _bounds, _listdir, _listlayers, _remove, _remove_layer
from fiona.schema import FIELD_TYPES_MAP, NAMED_FIELD_TYPES
from fiona.vfs import parse_paths as vfs_parse_paths

# These modules are imported by fiona.ogrext, but are also import here to
Expand Down Expand Up @@ -534,7 +528,7 @@ def prop_type(text):

"""
key = text.split(':')[0]
return FIELD_TYPES_MAP[key]
return NAMED_FIELD_TYPES[key].type


def drivers(*args, **kwargs):
Expand Down
10 changes: 6 additions & 4 deletions fiona/fio/load.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
import fiona
from fiona.fio import options, with_context_env
from fiona.model import Feature, Geometry
from fiona.schema import FIELD_TYPES_MAP_REV
from fiona.transform import transform_geom


Expand Down Expand Up @@ -87,11 +86,14 @@ def feature_gen():
except TypeError:
raise click.ClickException("Invalid input.")

# print(first, first.geometry)
# TODO: this inference of a property's type from its value needs some
# work. It works reliably only for the basic JSON serializable types.
# The fio-load command does require JSON input but that may change
# someday.
schema = {"geometry": first.geometry.type}
schema["properties"] = {
k: FIELD_TYPES_MAP_REV.get(type(v)) or "str"
for k, v in first.properties.items()
k: type(v if v is not None else "").__name__
for k, v in first.properties.items()
}

if append:
Expand Down
5 changes: 4 additions & 1 deletion fiona/gdal.pxi
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,9 @@ cdef extern from "ogr_core.h" nogil:
cdef int OFSTBoolean = 1
cdef int OFSTInt16 = 2
cdef int OFSTFloat32 = 3
cdef int OFSTMaxSubType = 3
cdef int OFSTJSON = 4
cdef int OFSTUUID = 5
cdef int OFSTMaxSubType = 5

ctypedef struct OGREnvelope:
double MinX
Expand Down Expand Up @@ -553,6 +555,7 @@ cdef extern from "ogr_api.h" nogil:
void OGR_F_SetFieldDouble(OGRFeatureH feature, int n, double value)
void OGR_F_SetFieldInteger(OGRFeatureH feature, int n, int value)
void OGR_F_SetFieldString(OGRFeatureH feature, int n, const char *value)
void OGR_F_SetFieldStringList(OGRFeatureH feature, int n, const char **value)
int OGR_F_SetGeometryDirectly(OGRFeatureH feature, OGRGeometryH geometry)
OGRFeatureDefnH OGR_FD_Create(const char *name)
int OGR_FD_GetFieldCount(OGRFeatureDefnH featuredefn)
Expand Down
Loading
Loading