diff --git a/CHANGELOG.md b/CHANGELOG.md index 32d3ba003..bc81f6fe3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,12 +10,14 @@ ### Migrations and checks - [#154](https://github.com/jirik/layman/issues/154) All workspaces are checked, that their name did not end with '_wms'. With any of the workspaces ended with the suffix, startup process is stopped with error code 45. In that case, please downgrade to the previous minor release version and contact Layman contributors. - [#154](https://github.com/jirik/layman/issues/154) All layers are copied into [dedicated WMS GeoServer workspace](doc/data-storage.md#geoserver). Styles are also moved into that workspace. +- [#154](https://github.com/jirik/layman/issues/154) Maps with URLs pointing to any layer stored in GeoServer are rewritten to dedicated [WMS workspace](doc/data-storage.md#geoserver). ### Changes - [#154](https://github.com/jirik/layman/issues/154) [WMS](doc/endpoints.md#web-map-service) is available in dedicated [GeoServer workspace](doc/data-storage.md#geoserver) whose name is composed from Layman's [workspace](doc/models.md#workspace) name and suffix `_wms`. [WFS](doc/endpoints.md#web-feature-service) remains in GeoServer workspace whose name is equal to Layman's workspace name. - [#154](https://github.com/jirik/layman/issues/154) SLD style published in dedicated WMS GeoServer workspace. - [#99](https://github.com/jirik/layman/issues/99) New endpoint [`/rest/about/version'](doc/rest.md#get-version). Also available in Layman Test Client. - [#154](https://github.com/jirik/layman/issues/154) Workspace name can not end with '_wms'. In such case, error with code 45 is raised. - New environment variables [LAYMAN_QGIS_HOST](doc/env-settings.md#LAYMAN_QGIS_HOST), [LAYMAN_QGIS_PORT](doc/env-settings.md#LAYMAN_QGIS_PORT), and [LAYMAN_QGIS_PATH](doc/env-settings.md#LAYMAN_QGIS_PATH). +- [#154](https://github.com/jirik/layman/issues/154) For endpoints [POST Layers](doc/rest.md#post-layers) and [PATCH Layer](doc/rest.md#patch-layer), parameter *sld* is replaced by the new parameter *style* and marked as deprecated. In response to endpoints [GET Layer](doc/rest.md#get-layer) and [PATCH Layer](doc/rest.md#patch-layer), *sld* is replaced by the new *style* item and marked as deprecated. ## v1.9.1 2021-01-18 diff --git a/doc/async-file-upload.md b/doc/async-file-upload.md index f30365fdc..7a494c431 100644 --- a/doc/async-file-upload.md +++ b/doc/async-file-upload.md @@ -21,8 +21,8 @@ You need some HTML form for user to choose files he wants to publish and fill so CRS: - SLD style: - + Style file: + diff --git a/doc/publish-map.md b/doc/publish-map.md index f6fe6987f..af9bff81c 100644 --- a/doc/publish-map.md +++ b/doc/publish-map.md @@ -37,7 +37,7 @@ Remember that Layman supports only `EPSG:4326` and `EPSG:3857` projections by de In QGIS, you need to implement following steps. -First, publish each layer whose data source is local ShapeFile or GeoJSON as WMS layer using [POST Layers](rest.md#post-layers) endpoint. Do not forget to respect supported projection (see `crs` input parameter). Also set `sld` parameter to layer style, otherwise the data file will be displayed with default GeoServer style. +First, publish each layer whose data source is local ShapeFile or GeoJSON as WMS layer using [POST Layers](rest.md#post-layers) endpoint. Do not forget to respect supported projection (see `crs` input parameter). Also set `style` parameter to layer style, otherwise the data file will be displayed with default GeoServer style. In response of [POST Layers](rest.md#post-layers) you will obtain - `name` of the layer unique within all layers in used [workspace](models.md#workspace) @@ -45,4 +45,4 @@ In response of [POST Layers](rest.md#post-layers) you will obtain In response of [GET Layer](rest.md#get-layer) you will obtain among others URL of WMS endpoint of the layer (`wms/url`). Together with `name` of the layer you have now enough information to represent the original local vector file as WMS layer. -Continue with the same steps as in [previous example](#maps-composed-from-wms-layers). \ No newline at end of file +Continue with the same steps as in [previous example](#maps-composed-from-wms-layers). diff --git a/doc/rest.md b/doc/rest.md index e09f80f97..fe7a12ddf 100644 --- a/doc/rest.md +++ b/doc/rest.md @@ -57,7 +57,7 @@ Processing chain consists of few steps: If workspace directory, database schema, GeoServer's workspaces, or GeoServer's datastores does not exist yet, it is created on demand. -Response to this request may be returned sooner than the processing chain is finished to enable asynchronous processing. Status of processing chain can be seen using [GET Layer](#get-layer) and **status** properties of layer sources (wms, wfs, thumbnail, db_table, file, sld, metadata). +Response to this request may be returned sooner than the processing chain is finished to enable asynchronous processing. Status of processing chain can be seen using [GET Layer](#get-layer) and **status** properties of layer sources (wms, wfs, thumbnail, db_table, file, style, metadata). It is possible to upload data files asynchronously, which is suitable for large files. This can be done in three steps: 1. Send POST Layers request with **file** parameter filled by file names that you want to upload @@ -90,8 +90,9 @@ Body parameters: - *crs*, string `EPSG:3857` or `EPSG:4326` - CRS of the file - by default it is read/guessed from input file -- *sld*, SLD file +- *style*, style file - by default default SLD style of GeoServer is used + - SLD file or QGIS style file - uploading of additional style files, e.g. point-symbol images or fonts is not supported - *access_rights.read*, string - comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [read access](./security.md#publication-access-rights) to this publication @@ -99,6 +100,9 @@ Body parameters: - *access_rights.write*, string - comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [write access](./security.md#publication-access-rights) to this publication - default value is current authenticated user, or EVERYONE if published by anonymous +- ~~sld~~, SLD file + - **deprecated parameter** + - alias for *style* parameter #### Response Content-Type: `application/json` @@ -176,10 +180,13 @@ JSON object with following structure: - *name*: String. DB table name within PostgreSQL workspace schema. This table is used as GeoServer source of layer. - *status*: Status information about DB import and availability of the table. See [GET Layer](#get-layer) **wms** property for meaning. - *error*: If status is FAILURE, this may contain error object. -- **sld** +- **style** - *url*: String. URL of layer default style. It points to [GET Layer Style](#get-layer-style). - - *status*: Status information about publishing SLD. See [GET Layer](#get-layer) **wms** property for meaning. + - *status*: Status information about publishing style. See [GET Layer](#get-layer) **wms** property for meaning. - *error*: If status is FAILURE, this may contain error object. +- **~~style~~** + - **Deprecated** + - Replaced by **style**, contains same info - *metadata* - *identifier*: String. Identifier of metadata record in CSW instance. - *record_url*: String. URL of metadata record accessible by web browser, probably with some editing capabilities. @@ -216,13 +223,16 @@ Body parameters: - *description* - *crs*, string `EPSG:3857` or `EPSG:4326` - Taken into account only if `file` is provided. -- *sld*, SLD file +- *style*, style file + - SLD or QGIS style file - If provided, current layer thumbnail will be temporarily deleted and created again using the new style. - *access_rights.read*, string - comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [read access](./security.md#publication-access-rights) to this publication - *access_rights.write*, string - comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [write access](./security.md#publication-access-rights) to this publication - +- ~~sld~~, SLD file + - **deprecated parameter** + - alias for *style* parameter #### Response Content-Type: `application/json` diff --git a/src/layman/layer/geoserver/sld.py b/src/layman/layer/geoserver/sld.py index 66711b02f..55c9b9af2 100644 --- a/src/layman/layer/geoserver/sld.py +++ b/src/layman/layer/geoserver/sld.py @@ -37,7 +37,7 @@ def delete_layer(workspace, layername): return {} else: return { - 'sld': { + 'style': { 'file': sld_stream, } } @@ -48,7 +48,7 @@ def get_layer_info(username, layername): if r.status_code == 200: url = url_for('rest_layer_style.get', username=username, layername=layername) info = { - 'sld': { + 'style': { 'url': url }, } diff --git a/src/layman/layer/rest_layer.py b/src/layman/layer/rest_layer.py index fb523efa9..8e8198f6d 100644 --- a/src/layman/layer/rest_layer.py +++ b/src/layman/layer/rest_layer.py @@ -81,12 +81,14 @@ def patch(username, layername): update_info = True # SLD - sld_file = None - if 'sld' in request.files and not request.files['sld'].filename == '': - sld_file = request.files['sld'] + style_file = None + if 'style' in request.files and not request.files['style'].filename == '': + style_file = request.files['style'] + elif 'sld' in request.files and not request.files['sld'].filename == '': + style_file = request.files['sld'] delete_from = None - if sld_file is not None: + if style_file is not None: delete_from = 'layman.layer.geoserver.sld' if len(files) > 0: delete_from = 'layman.layer.filesystem.input_file' @@ -107,13 +109,13 @@ def patch(username, layername): if delete_from is not None: deleted = util.delete_layer(username, layername, source=delete_from, http_method='patch') - if sld_file is None: + if style_file is None: try: - sld_file = deleted['sld']['file'] + style_file = deleted['style']['file'] except KeyError: pass - if sld_file is not None: - input_sld.save_layer_file(username, layername, sld_file) + if style_file is not None: + input_sld.save_layer_file(username, layername, style_file) kwargs.update({ 'crs_id': crs_id, diff --git a/src/layman/layer/rest_layer_test.py b/src/layman/layer/rest_layer_test.py index 291de2f86..7268cf177 100644 --- a/src/layman/layer/rest_layer_test.py +++ b/src/layman/layer/rest_layer_test.py @@ -11,30 +11,30 @@ @pytest.mark.usefixtures('ensure_layman') -def test_sld_value(): - username = 'test_layer_sld_user' - layername = 'test_layer_sld_layer' +def test_style_value(): + username = 'test_style_value_user' + layername = 'test_style_value_layer' process_client.publish_layer(username, layername) with app.app_context(): layer_url = url_for('rest_layer.get', username=username, layername=layername) - style_url = url_for('rest_layer_style.get', username=username, layername=layername) + expected_style_url = url_for('rest_layer_style.get', username=username, layername=layername) r = requests.get(layer_url) assert r.status_code == 200, r.text resp_json = json.loads(r.text) - assert "sld" in resp_json, r.text - assert "url" in resp_json["sld"], r.text - assert "status" not in resp_json["sld"], r.text + assert 'style' in resp_json, r.text + assert 'url' in resp_json['style'], r.text + assert 'status' not in resp_json['style'], r.text - sld_url = resp_json["sld"]["url"] - assert sld_url == style_url, (r.text, sld_url) + style_url = resp_json['style']['url'] + assert style_url == expected_style_url, (r.text, style_url) - r_get = requests.get(sld_url) - assert r_get.status_code == 200, (r_get.text, sld_url) + r_get = requests.get(style_url) + assert r_get.status_code == 200, (r_get.text, style_url) - r_del = requests.delete(sld_url) - assert r_del.status_code >= 400, (r_del.text, sld_url) + r_del = requests.delete(style_url) + assert r_del.status_code >= 400, (r_del.text, style_url) process_client.delete_layer(username, layername) diff --git a/src/layman/layer/rest_layers.py b/src/layman/layer/rest_layers.py index ff8a3cc96..3324db1ae 100644 --- a/src/layman/layer/rest_layers.py +++ b/src/layman/layer/rest_layers.py @@ -10,7 +10,6 @@ from layman.authn import authenticate from layman.authz import authorize_publications_decorator from layman.common import redis as redis_util -from layman import util as layman_util bp = Blueprint('rest_layers', __name__) @@ -95,10 +94,12 @@ def post(username): # DESCRIPTION description = request.form.get('description', '') - # SLD - sld_file = None - if 'sld' in request.files and not request.files['sld'].filename == '': - sld_file = request.files['sld'] + # Style + style_file = None + if 'style' in request.files and not request.files['style'].filename == '': + style_file = request.files['style'] + elif 'sld' in request.files and not request.files['sld'].filename == '': + style_file = request.files['sld'] actor_name = authn.get_authn_username() @@ -142,7 +143,7 @@ def post(username): task_options.update({'uuid': uuid_str, }) # save files - input_sld.save_layer_file(username, layername, sld_file) + input_sld.save_layer_file(username, layername, style_file) if use_chunk_upload: files_to_upload = input_chunk.save_layer_files_str( username, layername, files, check_crs) diff --git a/src/layman/layer/rest_test.py b/src/layman/layer/rest_test.py index 3464ea13f..6f96459be 100644 --- a/src/layman/layer/rest_test.py +++ b/src/layman/layer/rest_test.py @@ -539,7 +539,7 @@ def test_post_layers_complex(client): 'name': 'countries', 'title': 'staty', 'description': 'popis států', - 'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)), + 'style': (open(sld_path, 'rb'), os.path.basename(sld_path)), }) assert rv.status_code == 200 resp_json = rv.get_json() @@ -638,7 +638,7 @@ def test_uppercase_attr(client): rv = client.post(rest_path, data={ 'file': files, 'name': layername, - 'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)), + 'style': (open(sld_path, 'rb'), os.path.basename(sld_path)), }) assert rv.status_code == 200 resp_json = rv.get_json() @@ -784,7 +784,7 @@ def test_patch_layer_style(client): sld_path = 'sample/style/generic-blue.xml' assert os.path.isfile(sld_path) rv = client.patch(rest_path, data={ - 'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)), + 'style': (open(sld_path, 'rb'), os.path.basename(sld_path)), 'title': 'countries in blue' }) assert rv.status_code == 200 @@ -854,7 +854,7 @@ def test_post_layers_sld_1_1_0(client): rv = client.post(rest_path, data={ 'file': files, 'name': layername, - 'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)), + 'style': (open(sld_path, 'rb'), os.path.basename(sld_path)), }) assert rv.status_code == 200 resp_json = rv.get_json() @@ -866,7 +866,7 @@ def test_post_layers_sld_1_1_0(client): layer_info = util.get_layer_info(username, layername) while ('status' in layer_info['wms'] and layer_info['wms']['status'] in ['PENDING', 'STARTED'])\ - or ('status' in layer_info['sld'] and layer_info['sld']['status'] in ['PENDING', 'STARTED']): + or ('status' in layer_info['style'] and layer_info['style']['status'] in ['PENDING', 'STARTED']): time.sleep(0.1) layer_info = util.get_layer_info(username, layername) diff --git a/src/layman/layer/util.py b/src/layman/layer/util.py index 191349c81..e3bf26a18 100644 --- a/src/layman/layer/util.py +++ b/src/layman/layer/util.py @@ -143,7 +143,7 @@ def get_complete_layer_info(username=None, layername=None, cached=False): 'metadata': { 'status': 'NOT_AVAILABLE' }, - 'sld': { + 'style': { 'status': 'NOT_AVAILABLE' }, } @@ -152,6 +152,7 @@ def get_complete_layer_info(username=None, layername=None, cached=False): clear_publication_info(complete_info) + complete_info['sld'] = complete_info['style'] return complete_info @@ -192,8 +193,9 @@ def patch_layer(username, layername, task_options, stop_sync_at, start_async_at) TASKS_TO_LAYER_INFO_KEYS = { 'layman.layer.filesystem.input_chunk.refresh': ['file'], 'layman.layer.db.table.refresh': ['db_table'], - 'layman.layer.geoserver.wfs.refresh': ['wms', 'wfs'], - 'layman.layer.geoserver.sld.refresh': ['sld'], + 'layman.layer.geoserver.wfs.refresh': ['wfs'], + 'layman.layer.geoserver.wms.refresh': ['wms'], + 'layman.layer.geoserver.sld.refresh': ['style'], 'layman.layer.filesystem.thumbnail.refresh': ['thumbnail'], 'layman.layer.micka.soap.refresh': ['metadata'], } diff --git a/src/layman/upgrade/upgrade_v1_10_test.py b/src/layman/upgrade/upgrade_v1_10_test.py index 56072bc74..ad8bfbcd1 100644 --- a/src/layman/upgrade/upgrade_v1_10_test.py +++ b/src/layman/upgrade/upgrade_v1_10_test.py @@ -91,7 +91,7 @@ def test_migrate_layers_to_wms_workspace(ensure_layer): layer_info = process_client.get_layer(workspace, layer) assert layer_info['wms']['url'] == f'http://localhost:8000/geoserver/{wms_workspace}/ows' assert layer_info['wfs']['url'] == f'http://localhost:8000/geoserver/{workspace}/wfs' - assert layer_info['sld']['url'] == f'http://layman_test_run_1:8000/rest/{workspace}/layers/{layer}/style' + assert layer_info['style']['url'] == f'http://layman_test_run_1:8000/rest/{workspace}/layers/{layer}/style' all_workspaces = gs_common.get_all_workspaces(settings.LAYMAN_GS_AUTH) assert workspace in all_workspaces diff --git a/test/process_client.py b/test/process_client.py index 8cfac33f2..42e2dee69 100644 --- a/test/process_client.py +++ b/test/process_client.py @@ -205,7 +205,7 @@ def publish_publication(publication_type, if access_rights and access_rights.get('write'): data["access_rights.write"] = access_rights['write'] if style_file: - files.append(('sld', (os.path.basename(style_file), open(style_file, 'rb')))) + files.append(('style', (os.path.basename(style_file), open(style_file, 'rb')))) if description: data['description'] = description r = requests.post(r_url,