Improve reflection mechanism and alembic interactions (#374)

* Fix reflection for PostgreSQL and SpatiaLite columns

* Set _column_flag to False for spatial indexes

* Fix Alembic helpers and related tests

* Move load_spatialite and improve the doc for Alembic

* Check that spatialite is loaded before reflecting spatial indexes

.coveragerc

@@ -1,5 +1,3 @@
 [run]
 source = geoalchemy2
 relative_files = True
-omit =
-    geoalchemy2/alembic_helpers.py

.github/workflows/test_and_publish.yml

@@ -20,10 +20,10 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        python-version: [3.6, 3.7, 3.8, 3.9, pypy3]
+        python-version: ["3.7", "3.8", "3.9", "3.10", "pypy-3.8"]
 
     # The type of runner that the job will run on
-    runs-on: ubuntu-18.04
+    runs-on: ubuntu-22.04
 
     services:
       postgres:
@@ -48,7 +48,7 @@ jobs:
 
     # Setup Python
     - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v2
+      uses: actions/setup-python@v3
       with:
         python-version: ${{ matrix.python-version }}
         architecture: x64
@@ -123,10 +123,10 @@ jobs:
 
     - uses: actions/checkout@v2
 
-    - name: Set up Python 3.6
+    - name: Set up Python 3.7
       uses: actions/setup-python@v2
       with:
-        python-version: 3.6
+        python-version: 3.7
 
     # Build distribution and deploy to Pypi
     - name: Build and deploy package

.gitignore

@@ -29,9 +29,12 @@ pip-delete-this-directory.txt
 # Unit test / coverage reports
 .tox/
 .coverage
+.coverage-*
 .cache
 nosetests.xml
 coverage.xml
+.pytest_cache/
+reports/
 
 # Translations
 *.mo
@@ -51,3 +54,9 @@ coverage.xml
 # Sphinx documentation
 doc/_build/
 doc/gallery/
+
+# Miscellaneous
+*.csv
+*.sh
+*.txt
+issues

doc/alembic.rst

@@ -41,18 +41,18 @@ following migration script:
 
     """Create new table
 
-    Revision ID: 01b69e67a408
-    Revises: 2371af7aed3f
-    Create Date: 2022-01-27 15:53:05.268929
+    Revision ID: <rev_id>
+    Revises: <down_rev_id>
+    Create Date: <date>
 
     """
     from alembic import op
     import sqlalchemy as sa
 
 
     # revision identifiers, used by Alembic.
-    revision = "01b69e67a408"
-    down_revision = "2371af7aed3f"
+    revision = "<rev_id>"
+    down_revision = "<down_rev_id>"
     branch_labels = None
     depends_on = None
 
@@ -122,8 +122,8 @@ file used by Alembic, like in the following example:
         # ...
         context.configure(
             # ...
-            render_item=render_item,
-            include_object=include_object,
+            process_revision_directives=alembic_helpers.writer,
+            render_item=alembic_helpers.render_item,
         )
         # ...
 
@@ -132,12 +132,12 @@ file used by Alembic, like in the following example:
         # ...
         context.configure(
             # ...
-            render_item=render_item,
-            include_object=include_object,
+            process_revision_directives=alembic_helpers.writer,
+            render_item=alembic_helpers.render_item,
         )
         # ...
 
-After running the ``alembic`` command, the migration script will be properly generated and should
+After running the ``alembic`` command, the migration script should be properly generated and should
 not need to be manually edited.
 
 
@@ -173,7 +173,7 @@ in ``my_package.custom_types``, you just have to edit the ``env.py`` file like t
         if spatial_type:
             return spatial_type
 
-        # For the cumstom type
+        # For the custom type
         if obj_type == 'type' and isinstance(obj, TheCustomType):
             import_name = obj.__class__.__name__
             autogen_context.imports.add(f"from my_package.custom_types import {import_name}")
@@ -188,8 +188,8 @@ in ``my_package.custom_types``, you just have to edit the ``env.py`` file like t
         # ...
         context.configure(
             # ...
+            process_revision_directives=alembic_helpers.writer,
             render_item=render_item,
-            include_object=include_object,
         )
         # ...
 
@@ -198,69 +198,34 @@ in ``my_package.custom_types``, you just have to edit the ``env.py`` file like t
         # ...
         context.configure(
             # ...
+            process_revision_directives=alembic_helpers.writer,
             render_item=render_item,
-            include_object=include_object,
         )
         # ...
 
 Then the proper imports will be automatically added in the migration scripts.
 
 
-Add / Drop columns
-------------------
+Dialects
+--------
 
-Some dialects (like SQLite) require some specific management to alter columns of a table. In this
+Some dialects (like SQLite) require some specific management to alter columns or tables. In this
 case, other dedicated helpers are provided to handle this. For example, if one wants to add and drop
-columns in a SQLite database, the ``env.py`` file should look like the following:
+columns in a SQLite database, the SpatiaLite extension should be loaded when the engine connects,
+thus the ``env.py`` file should look like the following:
 
 .. code-block:: python
 
-    from alembic.autogenerate import rewriter
-
-    writer = rewriter.Rewriter()
-
-
-    @writer.rewrites(ops.AddColumnOp)
-    def add_geo_column(context, revision, op):
-    """This function replaces the default AddColumnOp by a geospatial-specific one."""
-        col_type = op.column.type
-        if isinstance(col_type, TypeDecorator):
-            dialect = context.bind().dialect
-            col_type = col_type.load_dialect_impl(dialect)
-        if isinstance(col_type, (Geometry, Geography, Raster)):
-            new_op = AddGeospatialColumn(op.table_name, op.column, op.schema)
-        else:
-            new_op = op
-        return new_op
-
-
-    @writer.rewrites(ops.DropColumnOp)
-    def drop_geo_column(context, revision, op):
-    """This function replaces the default DropColumnOp by a geospatial-specific one."""
-        col_type = op.to_column().type
-        if isinstance(col_type, TypeDecorator):
-            dialect = context.bind.dialect
-            col_type = col_type.load_dialect_impl(dialect)
-        if isinstance(col_type, (Geometry, Geography, Raster)):
-            new_op = DropGeospatialColumn(op.table_name, op.column_name, op.schema)
-        else:
-            new_op = op
-        return new_op
-
-
-    def load_spatialite(dbapi_conn, connection_record):
-        """Load SpatiaLite extension in SQLite DB."""
-        dbapi_conn.enable_load_extension(True)
-        dbapi_conn.load_extension(os.environ['SPATIALITE_LIBRARY_PATH'])
-        dbapi_conn.enable_load_extension(False)
-        dbapi_conn.execute('SELECT InitSpatialMetaData()')
+    from geoalchemy2.alembic_helpers import load_spatialite
+    from geoalchemy2.alembic_helpers import writer
 
 
     def run_migrations_offline():
         # ...
         context.configure(
             # ...
             process_revision_directives=writer,
+            render_item=alembic_helpers.render_item,
         )
         # ...
 
@@ -276,5 +241,6 @@ columns in a SQLite database, the ``env.py`` file should look like the following
             context.configure(
                 # ...
                 process_revision_directives=writer,
+                render_item=alembic_helpers.render_item,
             )
             # ...

doc/index.rst

@@ -6,11 +6,12 @@ GeoAlchemy 2 Documentation
 GeoAlchemy 2 provides extensions to `SQLAlchemy <http://sqlalchemy.org>`_ for
 working with spatial databases.
 
-GeoAlchemy 2 focuses on `PostGIS <http://postgis.net/>`_. PostGIS 1.5 and
-PostGIS 2 are supported.
+GeoAlchemy 2 focuses on `PostGIS <http://postgis.net/>`_. PostGIS 1.5,
+PostGIS 2 and PostGIS 3 are supported.
 
 SpatiaLite is also supported, but using GeoAlchemy 2 with SpatiaLite requires some specific
-configuration on the application side. GeoAlchemy 2 works with SpatiaLite 4.3.0 and higher.
+configuration on the application side. GeoAlchemy 2 works with SpatiaLite 4.3.0 and higher
+(except for alembic helpers that need SpatiaLite >= 5).
 
 GeoAlchemy 2 aims to be simpler than its predecessor, `GeoAlchemy
 <https://pypi.python.org/pypi/GeoAlchemy>`_. Simpler to use, and simpler
@@ -22,8 +23,7 @@ The current version of this documentation applies to the version |version| of Ge
 Requirements
 ------------
 
-GeoAlchemy 2 requires SQLAlchemy 0.8. GeoAlchemy 2 does not work with
-SQLAlchemy 0.7 and lower.
+GeoAlchemy 2 requires SQLAlchemy 1.4.
 
 Installation
 ------------