Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use 'style' instead of deprecated 'sld' parameter #241

Merged
merged 3 commits into from
Feb 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,14 @@
### Migrations and checks
- [#154](https://github.com/jirik/layman/issues/154) All workspaces are checked, that their name did not end with '_wms'. With any of the workspaces ended with the suffix, startup process is stopped with error code 45. In that case, please downgrade to the previous minor release version and contact Layman contributors.
- [#154](https://github.com/jirik/layman/issues/154) All layers are copied into [dedicated WMS GeoServer workspace](doc/data-storage.md#geoserver). Styles are also moved into that workspace.
- [#154](https://github.com/jirik/layman/issues/154) Maps with URLs pointing to any layer stored in GeoServer are rewritten to dedicated [WMS workspace](doc/data-storage.md#geoserver).
### Changes
- [#154](https://github.com/jirik/layman/issues/154) [WMS](doc/endpoints.md#web-map-service) is available in dedicated [GeoServer workspace](doc/data-storage.md#geoserver) whose name is composed from Layman's [workspace](doc/models.md#workspace) name and suffix `_wms`. [WFS](doc/endpoints.md#web-feature-service) remains in GeoServer workspace whose name is equal to Layman's workspace name.
- [#154](https://github.com/jirik/layman/issues/154) SLD style published in dedicated WMS GeoServer workspace.
- [#99](https://github.com/jirik/layman/issues/99) New endpoint [`/rest/about/version'](doc/rest.md#get-version). Also available in Layman Test Client.
- [#154](https://github.com/jirik/layman/issues/154) Workspace name can not end with '_wms'. In such case, error with code 45 is raised.
- New environment variables [LAYMAN_QGIS_HOST](doc/env-settings.md#LAYMAN_QGIS_HOST), [LAYMAN_QGIS_PORT](doc/env-settings.md#LAYMAN_QGIS_PORT), and [LAYMAN_QGIS_PATH](doc/env-settings.md#LAYMAN_QGIS_PATH).
- [#154](https://github.com/jirik/layman/issues/154) For endpoints [POST Layers](doc/rest.md#post-layers) and [PATCH Layer](doc/rest.md#patch-layer), parameter *sld* is replaced by the new parameter *style* and marked as deprecated. In response to endpoints [GET Layer](doc/rest.md#get-layer) and [PATCH Layer](doc/rest.md#patch-layer), *sld* is replaced by the new *style* item and marked as deprecated.

## v1.9.1
2021-01-18
Expand Down
4 changes: 2 additions & 2 deletions doc/async-file-upload.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ You need some HTML form for user to choose files he wants to publish and fill so
CRS:
<input name="crs" type="text" />

SLD style:
<input name="sld" type="file" />
Style file:
<input name="style" type="file" />

<button type="submit">Submit</button>
</form>
Expand Down
4 changes: 2 additions & 2 deletions doc/publish-map.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,12 +37,12 @@ Remember that Layman supports only `EPSG:4326` and `EPSG:3857` projections by de

In QGIS, you need to implement following steps.

First, publish each layer whose data source is local ShapeFile or GeoJSON as WMS layer using [POST Layers](rest.md#post-layers) endpoint. Do not forget to respect supported projection (see `crs` input parameter). Also set `sld` parameter to layer style, otherwise the data file will be displayed with default GeoServer style.
First, publish each layer whose data source is local ShapeFile or GeoJSON as WMS layer using [POST Layers](rest.md#post-layers) endpoint. Do not forget to respect supported projection (see `crs` input parameter). Also set `style` parameter to layer style, otherwise the data file will be displayed with default GeoServer style.

In response of [POST Layers](rest.md#post-layers) you will obtain
- `name` of the layer unique within all layers in used [workspace](models.md#workspace)
- `url` of the layer pointing to [GET Layer](rest.md#get-layer)

In response of [GET Layer](rest.md#get-layer) you will obtain among others URL of WMS endpoint of the layer (`wms/url`). Together with `name` of the layer you have now enough information to represent the original local vector file as WMS layer.

Continue with the same steps as in [previous example](#maps-composed-from-wms-layers).
Continue with the same steps as in [previous example](#maps-composed-from-wms-layers).
22 changes: 16 additions & 6 deletions doc/rest.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Processing chain consists of few steps:

If workspace directory, database schema, GeoServer's workspaces, or GeoServer's datastores does not exist yet, it is created on demand.

Response to this request may be returned sooner than the processing chain is finished to enable asynchronous processing. Status of processing chain can be seen using [GET Layer](#get-layer) and **status** properties of layer sources (wms, wfs, thumbnail, db_table, file, sld, metadata).
Response to this request may be returned sooner than the processing chain is finished to enable asynchronous processing. Status of processing chain can be seen using [GET Layer](#get-layer) and **status** properties of layer sources (wms, wfs, thumbnail, db_table, file, style, metadata).

It is possible to upload data files asynchronously, which is suitable for large files. This can be done in three steps:
1. Send POST Layers request with **file** parameter filled by file names that you want to upload
Expand Down Expand Up @@ -90,15 +90,19 @@ Body parameters:
- *crs*, string `EPSG:3857` or `EPSG:4326`
- CRS of the file
- by default it is read/guessed from input file
- *sld*, SLD file
- *style*, style file
- by default default SLD style of GeoServer is used
- SLD file or QGIS style file
- uploading of additional style files, e.g. point-symbol images or fonts is not supported
- *access_rights.read*, string
- comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [read access](./security.md#publication-access-rights) to this publication
- default value is current authenticated user, or EVERYONE if published by anonymous
- *access_rights.write*, string
- comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [write access](./security.md#publication-access-rights) to this publication
- default value is current authenticated user, or EVERYONE if published by anonymous
- ~~sld~~, SLD file
- **deprecated parameter**
- alias for *style* parameter

#### Response
Content-Type: `application/json`
Expand Down Expand Up @@ -176,10 +180,13 @@ JSON object with following structure:
- *name*: String. DB table name within PostgreSQL workspace schema. This table is used as GeoServer source of layer.
- *status*: Status information about DB import and availability of the table. See [GET Layer](#get-layer) **wms** property for meaning.
- *error*: If status is FAILURE, this may contain error object.
- **sld**
- **style**
jirik marked this conversation as resolved.
Show resolved Hide resolved
- *url*: String. URL of layer default style. It points to [GET Layer Style](#get-layer-style).
- *status*: Status information about publishing SLD. See [GET Layer](#get-layer) **wms** property for meaning.
- *status*: Status information about publishing style. See [GET Layer](#get-layer) **wms** property for meaning.
- *error*: If status is FAILURE, this may contain error object.
- **~~style~~**
- **Deprecated**
- Replaced by **style**, contains same info
- *metadata*
- *identifier*: String. Identifier of metadata record in CSW instance.
- *record_url*: String. URL of metadata record accessible by web browser, probably with some editing capabilities.
Expand Down Expand Up @@ -216,13 +223,16 @@ Body parameters:
- *description*
- *crs*, string `EPSG:3857` or `EPSG:4326`
- Taken into account only if `file` is provided.
- *sld*, SLD file
- *style*, style file
- SLD or QGIS style file
- If provided, current layer thumbnail will be temporarily deleted and created again using the new style.
- *access_rights.read*, string
- comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [read access](./security.md#publication-access-rights) to this publication
- *access_rights.write*, string
- comma-separated names of [users](./models.md#user) and [roles](./models.md#role) who will get [write access](./security.md#publication-access-rights) to this publication

- ~~sld~~, SLD file
- **deprecated parameter**
- alias for *style* parameter
#### Response
Content-Type: `application/json`

Expand Down
4 changes: 2 additions & 2 deletions src/layman/layer/geoserver/sld.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def delete_layer(workspace, layername):
return {}
else:
return {
'sld': {
'style': {
'file': sld_stream,
}
}
Expand All @@ -48,7 +48,7 @@ def get_layer_info(username, layername):
if r.status_code == 200:
url = url_for('rest_layer_style.get', username=username, layername=layername)
info = {
'sld': {
'style': {
'url': url
},
}
Expand Down
18 changes: 10 additions & 8 deletions src/layman/layer/rest_layer.py
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,14 @@ def patch(username, layername):
update_info = True

# SLD
sld_file = None
if 'sld' in request.files and not request.files['sld'].filename == '':
sld_file = request.files['sld']
style_file = None
if 'style' in request.files and not request.files['style'].filename == '':
style_file = request.files['style']
elif 'sld' in request.files and not request.files['sld'].filename == '':
style_file = request.files['sld']

delete_from = None
if sld_file is not None:
if style_file is not None:
delete_from = 'layman.layer.geoserver.sld'
if len(files) > 0:
delete_from = 'layman.layer.filesystem.input_file'
Expand All @@ -107,13 +109,13 @@ def patch(username, layername):

if delete_from is not None:
deleted = util.delete_layer(username, layername, source=delete_from, http_method='patch')
if sld_file is None:
if style_file is None:
try:
sld_file = deleted['sld']['file']
style_file = deleted['style']['file']
except KeyError:
pass
if sld_file is not None:
input_sld.save_layer_file(username, layername, sld_file)
if style_file is not None:
input_sld.save_layer_file(username, layername, style_file)

kwargs.update({
'crs_id': crs_id,
Expand Down
26 changes: 13 additions & 13 deletions src/layman/layer/rest_layer_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,30 +11,30 @@


@pytest.mark.usefixtures('ensure_layman')
def test_sld_value():
username = 'test_layer_sld_user'
layername = 'test_layer_sld_layer'
def test_style_value():
username = 'test_style_value_user'
layername = 'test_style_value_layer'

process_client.publish_layer(username, layername)

with app.app_context():
layer_url = url_for('rest_layer.get', username=username, layername=layername)
style_url = url_for('rest_layer_style.get', username=username, layername=layername)
expected_style_url = url_for('rest_layer_style.get', username=username, layername=layername)
r = requests.get(layer_url)
assert r.status_code == 200, r.text
resp_json = json.loads(r.text)

assert "sld" in resp_json, r.text
assert "url" in resp_json["sld"], r.text
assert "status" not in resp_json["sld"], r.text
assert 'style' in resp_json, r.text
assert 'url' in resp_json['style'], r.text
assert 'status' not in resp_json['style'], r.text

sld_url = resp_json["sld"]["url"]
assert sld_url == style_url, (r.text, sld_url)
style_url = resp_json['style']['url']
assert style_url == expected_style_url, (r.text, style_url)

r_get = requests.get(sld_url)
assert r_get.status_code == 200, (r_get.text, sld_url)
r_get = requests.get(style_url)
assert r_get.status_code == 200, (r_get.text, style_url)

r_del = requests.delete(sld_url)
assert r_del.status_code >= 400, (r_del.text, sld_url)
r_del = requests.delete(style_url)
assert r_del.status_code >= 400, (r_del.text, style_url)

process_client.delete_layer(username, layername)
13 changes: 7 additions & 6 deletions src/layman/layer/rest_layers.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
from layman.authn import authenticate
from layman.authz import authorize_publications_decorator
from layman.common import redis as redis_util
from layman import util as layman_util

bp = Blueprint('rest_layers', __name__)

Expand Down Expand Up @@ -95,10 +94,12 @@ def post(username):
# DESCRIPTION
description = request.form.get('description', '')

# SLD
sld_file = None
if 'sld' in request.files and not request.files['sld'].filename == '':
sld_file = request.files['sld']
# Style
style_file = None
if 'style' in request.files and not request.files['style'].filename == '':
style_file = request.files['style']
elif 'sld' in request.files and not request.files['sld'].filename == '':
style_file = request.files['sld']

actor_name = authn.get_authn_username()

Expand Down Expand Up @@ -142,7 +143,7 @@ def post(username):
task_options.update({'uuid': uuid_str, })

# save files
input_sld.save_layer_file(username, layername, sld_file)
input_sld.save_layer_file(username, layername, style_file)
if use_chunk_upload:
files_to_upload = input_chunk.save_layer_files_str(
username, layername, files, check_crs)
Expand Down
10 changes: 5 additions & 5 deletions src/layman/layer/rest_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ def test_post_layers_complex(client):
'name': 'countries',
'title': 'staty',
'description': 'popis států',
'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)),
'style': (open(sld_path, 'rb'), os.path.basename(sld_path)),
})
assert rv.status_code == 200
resp_json = rv.get_json()
Expand Down Expand Up @@ -638,7 +638,7 @@ def test_uppercase_attr(client):
rv = client.post(rest_path, data={
'file': files,
'name': layername,
'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)),
'style': (open(sld_path, 'rb'), os.path.basename(sld_path)),
})
assert rv.status_code == 200
resp_json = rv.get_json()
Expand Down Expand Up @@ -784,7 +784,7 @@ def test_patch_layer_style(client):
sld_path = 'sample/style/generic-blue.xml'
assert os.path.isfile(sld_path)
rv = client.patch(rest_path, data={
'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)),
'style': (open(sld_path, 'rb'), os.path.basename(sld_path)),
'title': 'countries in blue'
})
assert rv.status_code == 200
Expand Down Expand Up @@ -854,7 +854,7 @@ def test_post_layers_sld_1_1_0(client):
rv = client.post(rest_path, data={
'file': files,
'name': layername,
'sld': (open(sld_path, 'rb'), os.path.basename(sld_path)),
'style': (open(sld_path, 'rb'), os.path.basename(sld_path)),
})
assert rv.status_code == 200
resp_json = rv.get_json()
Expand All @@ -866,7 +866,7 @@ def test_post_layers_sld_1_1_0(client):

layer_info = util.get_layer_info(username, layername)
while ('status' in layer_info['wms'] and layer_info['wms']['status'] in ['PENDING', 'STARTED'])\
or ('status' in layer_info['sld'] and layer_info['sld']['status'] in ['PENDING', 'STARTED']):
or ('status' in layer_info['style'] and layer_info['style']['status'] in ['PENDING', 'STARTED']):
time.sleep(0.1)
layer_info = util.get_layer_info(username, layername)

Expand Down
8 changes: 5 additions & 3 deletions src/layman/layer/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ def get_complete_layer_info(username=None, layername=None, cached=False):
'metadata': {
'status': 'NOT_AVAILABLE'
},
'sld': {
'style': {
'status': 'NOT_AVAILABLE'
},
}
Expand All @@ -152,6 +152,7 @@ def get_complete_layer_info(username=None, layername=None, cached=False):

clear_publication_info(complete_info)

complete_info['sld'] = complete_info['style']
return complete_info


Expand Down Expand Up @@ -192,8 +193,9 @@ def patch_layer(username, layername, task_options, stop_sync_at, start_async_at)
TASKS_TO_LAYER_INFO_KEYS = {
'layman.layer.filesystem.input_chunk.refresh': ['file'],
'layman.layer.db.table.refresh': ['db_table'],
'layman.layer.geoserver.wfs.refresh': ['wms', 'wfs'],
'layman.layer.geoserver.sld.refresh': ['sld'],
'layman.layer.geoserver.wfs.refresh': ['wfs'],
'layman.layer.geoserver.wms.refresh': ['wms'],
'layman.layer.geoserver.sld.refresh': ['style'],
'layman.layer.filesystem.thumbnail.refresh': ['thumbnail'],
'layman.layer.micka.soap.refresh': ['metadata'],
}
Expand Down
2 changes: 1 addition & 1 deletion src/layman/upgrade/upgrade_v1_10_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ def test_migrate_layers_to_wms_workspace(ensure_layer):
layer_info = process_client.get_layer(workspace, layer)
assert layer_info['wms']['url'] == f'http://localhost:8000/geoserver/{wms_workspace}/ows'
assert layer_info['wfs']['url'] == f'http://localhost:8000/geoserver/{workspace}/wfs'
assert layer_info['sld']['url'] == f'http://layman_test_run_1:8000/rest/{workspace}/layers/{layer}/style'
assert layer_info['style']['url'] == f'http://layman_test_run_1:8000/rest/{workspace}/layers/{layer}/style'

all_workspaces = gs_common.get_all_workspaces(settings.LAYMAN_GS_AUTH)
assert workspace in all_workspaces
Expand Down
2 changes: 1 addition & 1 deletion test/process_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ def publish_publication(publication_type,
if access_rights and access_rights.get('write'):
data["access_rights.write"] = access_rights['write']
if style_file:
files.append(('sld', (os.path.basename(style_file), open(style_file, 'rb'))))
files.append(('style', (os.path.basename(style_file), open(style_file, 'rb'))))
if description:
data['description'] = description
r = requests.post(r_url,
Expand Down