Skip to content

Commit

Permalink
Refacto: split profiles sync job (#432)
Browse files Browse the repository at this point in the history
This PR is part of #343 and of a whole refactoring.

In this PR:

- deprecating `qprofiles-manager`
- update related documentation and JSON schemas
- remove some outdated source code

## Migration guide

Before:

```yaml
[...]
steps:
  - name: Download profiles from remote git repository and synchronize with installed profiles
    uses: qprofiles-manager
    with:
      source: https://github.com/Guts/qgis-deployment-cli.git
      protocol: git_remote
      sync_mode: overwrite
      branch: main
[...]
```

After:

```yaml
[...]
steps:
  - name: Download profiles from remote git repository
    uses: qprofiles-downloader
    with:
      source: https://github.com/Guts/qgis-deployment-cli.git
      protocol: git_remote
      branch: main

  - name: Synchronize downloaded profiles with installed ones
    uses: qprofiles-synchronizer
    with:
      sync_mode: overwrite
[...]
```
  • Loading branch information
Guts authored Feb 22, 2024
2 parents f71a6d8 + b9e9d53 commit 4bd43c0
Show file tree
Hide file tree
Showing 30 changed files with 514 additions and 153 deletions.
3 changes: 2 additions & 1 deletion docs/jobs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ glob:
environment_variables.md
plugins_downloader.md
plugins_synchronizer.md
profiles_manager.md
profiles_downloader.md
profiles_synchronizer.md
shortcuts_manager.md
splash_screen_manager.md
*
Expand Down
86 changes: 86 additions & 0 deletions docs/jobs/profiles_downloader.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Profiles Downloader

This job downloads remote profiles through different protocols to the local QDT working folder.

----

## Use it

Sample job configurations.

### **Remote** HTTP repository

```yaml
- name: Download profiles from remote HTTP server
uses: qprofiles-downloader
with:
branch: main
protocol: http
source: https://organization.intra/qgis/qdt/
```
:::{note}
If you use the HTTP procotol, a `qdt-files.json` must be downloadable at the URL source. Typically: `https://organization.intra/qgis/qdt/qdt-files.json`.

See this guide on [how to generate the qdt-files.json](../usage/profile.md#generate-the-qdt-filesjson-index-file).
:::

### Public **remote** git repository

```yaml
- name: Download profiles from remote Git server
uses: qprofiles-downloader
with:
branch: main
protocol: git_remote
source: https://github.com/geotribu/profils-qgis.git
```

### **Local** git repository

```yaml
- name: Download profiles from local Git repository
uses: qprofiles-downloader
with:
branch: main
protocol: git_local
source: file:///home/jmo/Git/Geotribu/profils-qgis
```

----

## Vocabulary

### Profiles states

- `remote`: a profile stored outside the end-user computer, on a git repository, an HTTP server or a LAN drive. Typically: `https://gitlab.com/Oslandia/qgis/profils_qgis_fr_2022.git`.
- `downloaded`: a profile downloaded into the QDT local working folder. Typically: `~/.cache/qgis-deployment-toolbelt/Oslandia/`.
- `installed`: a profile's folder located into the QGIS profiles folder and so accessible to the end-user through the QGIS interface. Typically: `~/.local/share/QGIS/QGIS3/profiles/default` or `%APPDATA%/QGIS/QGIS3/profiles/default`

----

## Options

### branch

Name of the branch to use when working with a git repository.

### protocol

Set which protocol to use.

Possible_values:

- `git_local`: use git to clone or pull changes from a repository accessible through filesystem, on the same computer or a shared drive on local network. `source` must end with `.git` and `branch` should also be set.
- `git_remote` (_default_): use git to clone or pull changes from a remote repository accessible through underlying HTTP protocol. `source` must end with `.git` and `branch` should also be set.
- `http`: use HTTP to download remote profiles. Source must start with `http`.

### source

Location of profiles to use as reference.

Must start with:

- `file://`: for local disk or network
- `git://` (_recomended_): for git repositories
- `https://`: for profiles stored into git repositories accessible through HTTP or profiles downloadable through an HTTP server
9 changes: 9 additions & 0 deletions docs/jobs/profiles_manager.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Profiles manager

```{error}
Since version 0.31.0, this job has been split into 2 jobs:
- [profiles-downloader](./profiles_downloader.md)
- [profiles-synchronizer](./profiles_synchronizer.md)
Please update your scenario files.
```

This job synchronize local profiles from remote storage (git for now).

----
Expand Down
53 changes: 53 additions & 0 deletions docs/jobs/profiles_synchronizer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Profiles Synchronizer

This job synchronizes installed profiles (those stored in QGIS profiles folder) from the downloaded ones (those stored in QDT local folder).

----

## Use it

Sample job configurations.

### Update or install profiles only with a newer version number

```yaml
- name: Synchronize installed profiles from downloaded ones
uses: qprofiles-synchronizer
with:
sync_mode: only_new_version
```
### Systematically overwrite installed profile with downloaded one
```yaml
- name: Synchronize installed profiles from downloaded ones
uses: qprofiles-synchronizer
with:
sync_mode: overwrite
```
----
## Vocabulary
### Profiles states
- `remote`: a profile stored outside the end-user computer, on a git repository, an HTTP server or a LAN drive. Typically: `https://gitlab.com/Oslandia/qgis/profils_qgis_fr_2022.git`.
- `downloaded`: a profile downloaded into the QDT local working folder. Typically: `~/.cache/qgis-deployment-toolbelt/Oslandia/`.
- `installed`: a profile's folder located into the QGIS profiles folder and so accessible to the end-user through the QGIS interface. Typically: `~/.local/share/QGIS/QGIS3/profiles/default` or `%APPDATA%/QGIS/QGIS3/profiles/default`

----

## Options

### sync_mode

Synchronization mode to apply with profiles.

Possible_values:

- `only_missing` (_default_): only install profiles that does not exist locally
- `only_different_version`: only install profiles that does not exist locally and update those with a different version number (lesser or upper)
- `only_new_version`: only install profiles that does not exist locally and update those with a lesser version number
- `overwrite`: systematically overwrite local profiles
4 changes: 2 additions & 2 deletions docs/schemas/profile/qgis_plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/qgis_plugin.json",
"$comment": "A QGIS profile described in a JSON file.",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/main/docs/schemas/profile/qgis_plugin.json",
"$comment": "A QGIS plugin described in a JSON file.",
"type": "object",
"properties": {
"folder_name": {
Expand Down
2 changes: 1 addition & 1 deletion docs/schemas/profile/qgis_profile.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/qgis_profile.json",
"$id": "https://github.com/Guts/qgis-deployment-cli/raw/main/docs/schemas/profile/qgis_profile.json",
"$comment": "A QGIS profile described in a JSON file.",
"type": "object",
"properties": {
Expand Down
7 changes: 6 additions & 1 deletion docs/schemas/scenario/jobs/generic.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/jobs/generic.json",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/main/docs/schemas/scenario/jobs/generic.json",
"description": "Definition of a job, i.e. a logic execution which can be ran during a step.",
"title": "Steps",
"type": "array",
Expand All @@ -26,9 +26,14 @@
"manage-env-vars",
"qplugins-downloader",
"qplugins-synchronizer",
"qprofiles-downloader",
"qprofiles-synchronizer",
"qprofiles-manager",
"shortcuts-manager",
"splash-screen-manager"
],
"deprecated": [
"qprofiles-manager"
]
},
"with": {
Expand Down
32 changes: 25 additions & 7 deletions docs/schemas/scenario/jobs/manage-env-vars.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/jobs/manage-env-vars.json",
"$id": "https://github.com/Guts/qgis-deployment-cli/raw/main/docs/schemas/scenario/jobs/manage-env-vars.json",
"description": "Job in charge of setting/updating/removing environment variables on target computer.",
"title": "Environment variables manager.",
"type": "array",
Expand All @@ -10,7 +10,10 @@
"action": {
"default": "add",
"description": "Tell the job what to do with the environment variable.",
"enum": ["add", "remove"],
"enum": [
"add",
"remove"
],
"type": "string"
},
"name": {
Expand All @@ -20,16 +23,27 @@
"scope": {
"default": "user",
"description": "Level of the environment variable.",
"enum": ["user", "system"],
"enum": [
"user",
"system"
],
"type": "string"
},
"value": {
"description": "Value to set to the environment variable.",
"type": ["boolean", "string"]
"type": [
"boolean",
"string"
]
},
"value_type": {
"description": "Value type to avoid ambiguity when interpreting it.",
"enum": ["bool", "path", "str", "url"],
"enum": [
"bool",
"path",
"str",
"url"
],
"type": "string"
}
},
Expand All @@ -44,10 +58,14 @@
}
},
"then": {
"required": ["value"]
"required": [
"value"
]
}
}
],
"required": ["name"]
"required": [
"name"
]
}
}
2 changes: 1 addition & 1 deletion docs/schemas/scenario/jobs/qplugins-downloader.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/jobs/qplugins-downloader.json",
"$id": "https://github.com/Guts/qgis-deployment-cli/raw/main/docs/schemas/scenario/jobs/qplugins-downloader.json",
"description": "Download plugins into QDT local folder.",
"title": "QPlugins Downloader.",
"type": "object",
Expand Down
2 changes: 1 addition & 1 deletion docs/schemas/scenario/jobs/qplugins-synchronizer.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/jobs/qplugins-synchronizer.json",
"$id": "https://github.com/Guts/qgis-deployment-cli/raw/main/docs/schemas/scenario/jobs/qplugins-synchronizer.json",
"description": "synchronize plugins between those stored locally (typically downloaded by the Plugins Downloader job) and the installed plugins.",
"title": "QPlugins Synchronizer.",
"type": "object",
Expand Down
60 changes: 60 additions & 0 deletions docs/schemas/scenario/jobs/qprofiles-downloader.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/main/docs/schemas/scenario/jobs/qprofiles-downloader.json",
"description": "Job to download remote profiles to local QDT working folder.",
"title": "QProfiles Downloader",
"type": "object",
"properties": {
"branch": {
"default": "master",
"description": "Name of the branch to use when working with a git repository.",
"type": "string"
},
"protocol": {
"description": "Set which protocol to use for downloading profiles.",
"enum": [
"http",
"git_local",
"git_remote"
],
"deprecated": [
"git"
],
"type": "string"
},
"source": {
"description": "Location of profiles. Typically: 'https://github.com/Guts/qgis-deployment-cli.git' or 'https://raw.githubusercontent.com/Guts/qgis-deployment-cli/examples/'",
"type": "string"
}
},
"allOf": [
{
"if": {
"properties": {
"protocol": {
"const": "git_local"
}
}
},
"then": {
"required": [
"branch"
]
}
},
{
"if": {
"properties": {
"protocol": {
"const": "git_remote"
}
}
},
"then": {
"required": [
"branch"
]
}
}
]
}
8 changes: 4 additions & 4 deletions docs/schemas/scenario/jobs/qprofiles-manager.json
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/feature/scenario-pseudo-ci/docs/schemas/jobs/qprofiles-manager.json",
"description": "QDT job synchronizing local working directory with remote profiles.",
"title": "QProfiles Synchronizer.",
"$id": "https://raw.githubusercontent.com/Guts/qgis-deployment-cli/main/docs/schemas/scenario/jobs/qprofiles-manager.json",
"description": "!! DEPRECATED !! Split into 2 different jobs since version 0.31. Former description: job synchronizing local working directory with remote profiles.",
"title": "QProfiles Manager.",
"type": "object",
"deprecated": true,
"properties": {
"action": {
"default": "download",
Expand Down Expand Up @@ -42,7 +43,6 @@
"only_new_version",
"overwrite"
],

"type": "string"
}
},
Expand Down
Loading

0 comments on commit 4bd43c0

Please sign in to comment.