868 get_workspace_map_file

.gitignore

@@ -17,3 +17,5 @@
 !/.env.dev
 !/.env.test
 /deps/**/venv
+
+**/geckodriver.log

.remarkrc

@@ -8,6 +8,7 @@
     "lint-no-dead-urls": {
       "skipLocalhost": true,
       "skipUrlPatterns": [
+        "https://security.stackexchange.com",
         "https://help.liferay.com",
         "https://github.com/LayerManager/layman/issues/"
       ]

CHANGELOG.md

@@ -15,24 +15,35 @@
 - [#765](https://github.com/LayerManager/layman/issues/765) Remove `authn.txt` files from workspace directories. The same information as in `authn.txt` files is saved in prime DB schema.
 - [#868](https://github.com/LayerManager/layman/issues/868) Fill table `map_layer` with relations between maps and [internal layers](doc/models.md#internal-map-layer) (layers published on this Layman instance). Relations to [external layers](doc/models.md#internal-map-layer) (layers of other servers) are not imported into the table.
 ### Changes
-- [#868](https://github.com/LayerManager/layman/issues/868) Endpoints [GET Publications](doc/rest.md#get-publications), [GET Layers](doc/rest.md#get-layers), [GET Workspace Layers](doc/rest.md#get-workspace-layers), [GET Maps](doc/rest.md#get-maps), [GET Workspace Maps](doc/rest.md#get-workspace-maps), [GET Workspace Layer](doc/rest.md#get-workspace-layer), [GET Workspace Map](doc/rest.md#get-workspace-map), [POST Workspace Layers](doc/rest.md#post-workspace-layers), [DELETE Workspace Layer](doc/rest.md#delete-workspace-layer), [DELETE Workspace Layers](doc/rest.md#delete-workspace-layers), [DELETE Workspace Map](doc/rest.md#delete-workspace-map), [DELETE Workspace Maps](doc/rest.md#delete-workspace-maps), [POST Workspace Maps](doc/rest.md#post-workspace-maps), [PATCH Workspace Layer](doc/rest.md#patch-workspace-layer), [PATCH Workspace Map](doc/rest.md#patch-workspace-map) and [WMS/WFS endpoints](doc/endpoints.md) respects [HTTP header `X-Forwarded-Prefix`](doc/client-proxy.md#x-forwarded-prefix-http-header) of the request in the response.
-  - Header does not affect responses of [GET Workspace Layer Metadata Comparison](doc/rest.md#get-workspace-layer-metadata-comparison) and [GET Workspace Map Metadata Comparison](doc/rest.md#get-workspace-map-metadata-comparison).
+- [#868](https://github.com/LayerManager/layman/issues/868) Responses to many requests respects [HTTP header `X-Forwarded-Prefix`](doc/client-proxy.md#x-forwarded-prefix-http-header) of the request. Those requests are:
+  - GET [Publications](doc/rest.md#get-publications), [Layers](doc/rest.md#get-layers), [Workspace Layers](doc/rest.md#get-workspace-layers), [Maps](doc/rest.md#get-maps), and [Workspace Maps](doc/rest.md#get-workspace-maps)
+  - [GET](doc/rest.md#get-workspace-layer), [PATCH](doc/rest.md#patch-workspace-layer), and [DELETE](doc/rest.md#delete-workspace-layer) Workspace Layer
+  - [GET](doc/rest.md#get-workspace-map), [PATCH](doc/rest.md#patch-workspace-map), and [DELETE](doc/rest.md#delete-workspace-map) Workspace Map
+  - [GET Workspace Map File](doc/rest.md#get-workspace-map-file)
+  - [POST](doc/rest.md#post-workspace-layers) and [DELETE](doc/rest.md#delete-workspace-layers) Workspace Layers
+  - [POST](doc/rest.md#post-workspace-maps) and [DELETE](doc/rest.md#delete-workspace-maps) Workspace Maps
+  - requests to [WMS](doc/endpoints.md#web-map-service) and [WFS](doc/endpoints.md#web-feature-service) endpoints
+- [#868](https://github.com/LayerManager/layman/issues/868) Responses to [GET Workspace Layer Metadata Comparison](doc/rest.md#get-workspace-layer-metadata-comparison) and [GET Workspace Map Metadata Comparison](doc/rest.md#get-workspace-map-metadata-comparison) do not respect [HTTP header `X-Forwarded-Prefix`](doc/client-proxy.md#x-forwarded-prefix-http-header) of the request intentionally, in order to keep URLs in canonical form.
 - [#868](https://github.com/LayerManager/layman/issues/868) Relations between map and [internal layers](doc/models.md#internal-map-layer) are updated in `map_layer` table when calling [POST Workspace Maps](doc/rest.md#post-workspace-maps), [PATCH Workspace Map](doc/rest.md#patch-workspace-map), [DELETE Workspace Map](doc/rest.md#delete-workspace-map), and [DELETE Workspace Maps](doc/rest.md#delete-workspace-maps).
 - [#880](https://github.com/LayerManager/layman/issues/880) Use Docker Compose v2 (`docker compose`) in Makefile without `compatibility` flag and remove `Makefile_docker-compose_v1` file. Docker containers are named according to Docker Compose v2 and may have different name after upgrade.
 - [#765](https://github.com/LayerManager/layman/issues/765) Stop saving OAuth2 claims in filesystem, use prime DB schema only.
 - [#893](https://github.com/LayerManager/layman/issues/893) It is possible to specify logging level by new environment variable [LAYMAN_LOGLEVEL](doc/env-settings.md#LAYMAN_LOGLEVEL). Default level is `INFO`.
+- Add new test Python dependency:
+  - jsonpath-ng 1.6.0
 - Upgrade Python dependencies
   - certifi 2023.5.7 -> 2023.7.22 (suggested by dependabot)
+  - tornado 6.3.2 -> 6.3.3 (suggested by dependabot)
+  - flask 2.3.2 -> 2.3.3
   - jsonschema 4.17.3 -> 4.19.0
   - lxml 4.9.2 -> 4.9.3
   - owslib 0.28.1 -> 0.29.2
   - psycopg2-binary 2.9.5 -> 2.9.7
-  - redis 4.5.5 -> 4.6.0
-  - autopep8 2.0.1 -> 2.0.2
+  - redis 4.5.5 -> 5.0.0
+  - autopep8 2.0.1 -> 2.0.4
   - flake8 6.0.0 -> 6.1.0
   - pillow 9.3.0 -> 10.0.0
   - pycodestyle 2.10.0 -> 2.11.0 (to be consistent with GitHub Actions)
-  - pytest 7.2.0 -> 7.4.0
+  - pytest 7.2.0 -> 7.4.2
   - pytest-rerunfailures 10.3 -> 12.0
   - watchdog 2.2.0 -> 3.0.0
 

doc/client-proxy.md

@@ -63,6 +63,13 @@ Currently, value of `X-Forwarded-Prefix` affects following URLs:
   * `file`.`url` key
   * `thumbnail`.`url` key
   * `metadata`.`comparison_url` key
+* [GET Workspace Map File](rest.md#get-workspace-map-file)
+  * some URLs of each [internal layer](models.md#internal-map-layer):
+    * `url` key
+    * `protocol`.`url` key
+    * each `legends` key if its HTTP protocol and netloc corresponds with `url` or `protocol`.`url`
+    * `style` key if its HTTP protocol and netloc corresponds with `url` or `protocol`.`url`
+  * NOTE: If client proxy prefix was used in URLs in uploaded file, then such prefix is also replaced with prefix according to `X-Forwarded-Prefix` header value. Such prefix is removed for requests without `X-Forwarded-Prefix` header.
 * [POST Workspace Layers](rest.md#post-workspace-layers)
   * `url` key
 * [DELETE Workspace Layer](rest.md#delete-workspace-layer)

doc/dependencies.md

@@ -56,6 +56,7 @@
 | [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) | MPL | Pipfile | test | bin | to automatically rerun flaky tests |
 | [pytest-timeout](https://pypi.org/project/pytest-timeout/) | MIT | Pipfile | test | bin | to automatically stop tests after given timeout |
 | [pillow](https://github.com/python-pillow/Pillow) | HPND | Pipfile | test | bin | to ensure similarity of images |
+| [jsonpath-ng](https://github.com/h2non/jsonpath-ng) | HPND | Pipfile | test | bin | to query JSON files |
 
 ### Node.js dependencies
 | name | license | used by | env | bin or src | purpose |

doc/oauth2/client-recommendations.md

@@ -10,7 +10,7 @@ Both **Authorization Code** and **Authorization Code with PKCE** grant flows bet
 ## Storing Tokens on a Client
 An important decision when implementing OAuth2 *client* is where to store access tokens and refresh tokens. The recommendations differ based on [*client profile*](https://tools.ietf.org/html/rfc6749#section-2.1).
 
-[*Web Applications*](https://tools.ietf.org/html/rfc6749#section-2.1) are able to store tokens either on server side or client side. There exists quite straightforward recommendations (see e.g. [auth0](https://auth0.com/docs/security/store-tokens#regular-web-apps), [DZone](https://dzone.com/articles/security-best-practices-for-managing-api-access-to) or [StackOverflow](https://security.stackexchange.com/a/209388)). Generally three options are available:
+[*Web Applications*](https://tools.ietf.org/html/rfc6749#section-2.1) are able to store tokens either on server side or client side. There exists quite straightforward recommendations (see e.g. [auth0](https://auth0.com/docs/security/store-tokens#regular-web-apps), [DZone](https://dzone.com/articles/security-best-practices-for-managing-api-access-to) or [StackExchange](https://security.stackexchange.com/a/209388)). Generally three options are available:
 1. Store access tokens in browser memory. It would mean to provide new authorization request against authorization server and manual end-user authorization consent on every page open or page reload.
 2. Store access tokens in both secure and HTTP-only cookies. This option requires to have server side to set access tokens in cookies, and to refresh them using refresh tokens. Also refresh tokens need to be saved on server side.
 3. Store access token on server side in any well-protected database.

doc/rest.md

@@ -578,6 +578,7 @@ Content-Type: `multipart/form-data`
 Body parameters:
 - **file**, JSON file
    - must be valid against [map-composition schema](https://github.com/hslayers/hslayers-ng/wiki/Composition-schema)
+   - URLs of [internal layers](models.md#internal-map-layer) may contain [client-proxy prefix](client-proxy.md)
 - *name*, string
    - computer-friendly identifier of the map
    - must be unique among all maps of one workspace
@@ -738,6 +739,7 @@ Notice that some JSON properties are automatically updated by layman, so file ob
    - **email** set to email of the owner, or empty string if not known
    - other properties will be deleted
 - **groups** are removed
+- [some layer URLs](client-proxy.md#x-forwarded-prefix-http-header) according to `X-Forwarded-Prefix` header
 
 #### Request
 No action parameters.

docker/Pipfile

@@ -13,6 +13,7 @@ autopep8 = "*"
 pytest-rerunfailures = "*"
 pytest-timeout = "*"
 pillow = "*"
+jsonpath-ng = "*"
 
 [packages]
 celery = {extras = ["redis"],version = "<5.3.0"}