Skip to content

Commit

Permalink
Initial qgis test plan (#1691)
Browse files Browse the repository at this point in the history
An initial QGIS test plan for the plugin. There are still some tests to
be written.
I have also performed all these tests and created quite some new
reports.
There is also a known issues page for all of Ribasim, as we have some
things that simply don't work as expected.

New issues:

* #1678 
* #1679 
* #1681 
* #1682 
* #1683 
* #1684 
* #1685 
* #1688 
* #1689 
* #1690 

Fixes #320

---------

Co-authored-by: Martijn Visser <mgvisser@gmail.com>
  • Loading branch information
deltamarnix and visr committed Aug 27, 2024
1 parent ea1d83f commit 3e70543
Show file tree
Hide file tree
Showing 16 changed files with 382 additions and 88 deletions.
3 changes: 2 additions & 1 deletion .vscode/extensions.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
"quarto.quarto",
"tamasfe.even-better-toml",
"samuelcolvin.jinjahtml",
"yy0931.vscode-sqlite3-editor"
"yy0931.vscode-sqlite3-editor",
"streetsidesoftware.code-spell-checker"
]
}
54 changes: 53 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,57 @@
".pixi"
],
"julia.lint.run": true,
"files.insertFinalNewline": true
"files.insertFinalNewline": true,
"cSpell.enabledLanguageIds": [
"asciidoc",
"c",
"cpp",
"csharp",
"css",
"elixir",
"erlang",
"git-commit",
"go",
"graphql",
"handlebars",
"haskell",
"html",
"jade",
"java",
"javascript",
"javascriptreact",
"json",
"jsonc",
"jupyter",
"latex",
"less",
"markdown",
"php",
"plaintext",
"python",
"pug",
"restructuredtext",
"rust",
"scala",
"scss",
"scminput",
"swift",
"text",
"typescript",
"typescriptreact",
"vue",
"yaml",
"yml",
"quarto",
"julia"
],
"cSpell.words": [
"gpkg",
"ipynb",
"pixi",
"pkgdown",
"qgis",
"quartodoc",
"Ribasim"
]
}
22 changes: 9 additions & 13 deletions docs/_quarto.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,10 @@ website:
left:
- text: "Overview"
file: index.qmd
- text: "Tutorials"
file: tutorial/index.qmd
- text: "How-to guides"
file: guide/index.qmd
file: guide/examples.ipynb
- text: "Concepts"
file: concept/index.qmd
file: concept/concept.qmd
- text: "Reference"
file: reference/index.qmd
right:
Expand All @@ -31,24 +29,19 @@ website:
- index.qmd
- install.qmd
- changelog.qmd

- title: "Tutorials"
contents:
- tutorial/index.qmd
- tutorial/qgis.qmd
- known_issues.qmd

- title: "How-to guides"
contents:
- guide/index.qmd
- guide/examples.ipynb
- guide/qgis.qmd
- guide/coupling.qmd
- section: "Coupling guides"
contents:
- guide/delwaq.ipynb

- title: "Concepts"
contents:
- concept/index.qmd
- concept/concept.qmd
- section: "Numerics"
contents:
Expand Down Expand Up @@ -85,18 +78,21 @@ website:
- reference/node/continuous-control.qmd
- reference/node/pid-control.qmd

- title: "Dev docs"
- title: "Contributing"
contents:
- dev/index.qmd
- dev/core.qmd
- dev/benchmark.qmd
- dev/addnode.qmd
- dev/python.qmd
- dev/qgis.qmd
- dev/allocation.qmd
- dev/bmi.qmd
- dev/ci.qmd
- dev/release.qmd
- section: QGIS
contents:
- dev/qgis.qmd
- dev/qgis_test_plan.qmd

format:
html:
Expand Down
3 changes: 0 additions & 3 deletions docs/concept/index.qmd

This file was deleted.

2 changes: 1 addition & 1 deletion docs/dev/index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ There is developer documentation for the [Julia core](/dev/core.qmd), the [Basic
A guide on how to add a new node type to both is written in [adding node types](/dev/addnode.qmd).
[Release process](/dev/release.qmd) describes the steps to follow when creating a new Ribasim release.

# Set up the developer environment
# Setting up the developer environment

## Clone Ribasim

Expand Down
227 changes: 227 additions & 0 deletions docs/dev/qgis_test_plan.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,227 @@
---
title: "QGIS plugin manual test plan"
---

This document describes how to perform a full manual test on the Ribasim QGIS plugin.
Known shortcomings and issues can be documented [here](/known_issues.qmd).
Bugs can be reported on [GitHub](https://github.com/Deltares/Ribasim/issues).

# Clean slate tests
Before starting with data, perform the following tests to see if the plugin doesn't result in any errors.

## Enable and disable

- Open QGIS and navigate to "Plugins > Manage and Install Plugins...": _The plugin management window opens_.
- Navigate to "Installed": _Ribasim plugin is in the list (enabled)_.
- Disable the Ribasim plugin: _Ribasim plugin panel hides if it was open, Ribasim button hides from navigation toolbar_.
- Enable the Ribasim plugin: _Ribasim button shows on the navigation toolbar_.

## Open and close

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the Ribasim button on the QGIS toolbar: _Ribasim panel hides_.

## Button states on startup

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- _In the Model tab, the "Add to QGIS" and "Remove from Dataset" buttons are disabled_.
- _In the Nodes tab, all buttons are disabled_.

[Failing](https://github.com/Deltares/Ribasim/issues/1678)

# New model tests

## New and cancel

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name.
- Press "Cancel": _Ensure that no files are created in that location_.

## New single model

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- _The layers tab shows a group with a "Node" and "Edge" layer, the canvas is empty._
- _The Ribasim panel shows a Node and Edge layer, the file path is set in the text field, all nodes buttons are enabled_.
- Cleanup: Delete the created files from disk.

## New model in same folder

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test2").
- Press OK: _An error is given that the database.gpkg already exists, test2.toml is not written_.
- Cleanup: Delete the created files from disk.

[Failing](https://github.com/Deltares/Ribasim/issues/1679)

## New model with same name

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Open test1.toml in a text editor and write "EXTRA_LINE=true"
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _Shows a popup window to ask if you want to overwrite_.
- Press Yes: _An error is given that the database.gpkg already exists, test1.toml is not overwritten (EXTRA_LINE=true) is still in test1.toml_.
- Cleanup: Delete the created files from disk.

Unsure if this is wanted behavior, as we said we wanted to overwrite the TOML file, and therefore also the database file.
Perhaps we should create folders instead of TOML files only.

[Failing](https://github.com/Deltares/Ribasim/issues/1681)

## New model and clean new project cleans Ribasim plugin

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Press "Project > New": _Popup asks to save the project_.
- Press Discard: _Layers tab is emptied, Ribasim panel is emptied_.

[Failing](https://github.com/Deltares/Ribasim/issues/1682)

# Model tab button interaction tests

## Open model twice

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Press the "Open" button in the Model tab: _file navigation window pops up_.
- Open test1.toml: _A new layer group is added to the layers panel_.
- In the Nodes tab press "Basin / profile": _A Basin / profile table layer is added to the last layer group only_.

Intended behavior: The same model is loaded twice, but there is only a connection on the last loaded model when interacting with the plugin.

## Add to QGIS button

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Press "Add to QGIS": _Nothing happens_.
- Press the "Edge" layer in the Model tab: _"Add to QGIS" button is disabled_.

[Failing](https://github.com/Deltares/Ribasim/issues/1683)

## Remove from Dataset button

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Press the "Edge" layer in the Model tab, press "Remove from Dataset": _Popup shows "Deleting: Edge"_.
- Press "Yes": "Edge" layer is removed from Layers panel and from Model tab.

Unsure if this is wanted behavior, it is now impossible to load the model afterwards because it no longer contains an Edge layer.
Perhaps better if we only allow removal of the optional tables. See this [bug report](https://github.com/Deltares/Ribasim/issues/1684).

# Nodes tab button interaction tests

## Add all tables to new model

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Go to the Nodes tab: _All buttons are enabled_.
- From left to right, top to bottom, press every button to add tables to the layers: _The layer group contains extra layers for every button clicked. Basin / area is a MultiPolygon layer_.
- Navigate to Model tab: _All layers are added to the panel_.
- Press the "Open" button to open the same model once again: _The layers panel adds the model containing all of its layers, the Model tab was refreshed_.

## Add table twice to new model

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Go to the Nodes tab: _All buttons are enabled_.
- Press the Basin / time button: _A layer is added to the layers panel with that name, the button becomes disabled_.

[Failing](https://github.com/Deltares/Ribasim/issues/1685)

# Map interaction tests

## Add points to Node layer

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Select the Node layer in the Layers tab: _edit buttons in the toolbar become enabled_.
- Edit the layer by pressing the pencil button: _Add Point Feature button becomes enabled_.
- Click in the canvas: _Popup appears with Feature Attributes to fill in_.
- Press OK: _The first Node appears on the map_.
- Click in the canvas again: _Popup appears with Feature Attributes to fill in_.
- Press OK: _The second Node appears on the map_.

Unexpected behavior: The default type of the nodes is NULL, and therefore undefined.
Should be enforced and validated.
See [issue](https://github.com/Deltares/Ribasim/issues/1688#issuecomment-2265315907).

## Add edges to Edge layer

- Open QGIS and ensure that the Ribasim plugin is installed and enabled.
- Open the application via the Ribasim button on the QGIS toolbar: _Ribasim panel opens_.
- Press the "New" button in the Model tab: _file navigation window pops up_.
- Fill in a name ("test1").
- Press OK: _A TOML and database file are created on the given location_.
- Select the Node layer in the Layers tab: _edit buttons in the toolbar become enabled_.
- Edit the layer by pressing the pencil button: _Add Point Feature button becomes enabled_.
- Click in the canvas: _Popup appears with Feature Attributes to fill in_.
- Add a node id of 1, press OK: _The first Node appears on the map_.
- Click in the canvas again: _Popup appears with Feature Attributes to fill in_.
- Add a node id of 2, press OK: _The second Node appears on the map_.
- Select the Edge layer in the Layers tab: _edit buttons in the toolbar become enabled_.
- Enable snapping under View > Toolbars > Snapping Toolbars: _Magnet button is enabled and active_.
- Press the Add Line Feature button: _Mouse becomes a crosshair_.
- Snap a line between the two nodes, click the two nodes and then right click to finish: _Popup shows with input, all is set to NULL_.
- Press OK: _Line appears on screen between the two nodes_.
- Save the layer's edits: _The line becomes blue_.
- Open the attribute table: _The information shows the from_node_id, to_node_id. This information matches the information from the Node table._

# Tables tests

## Edit tables with data
TODO

## Edit the type of a node
TODO

## Edit information in optional tables
TODO

# Result inspection tests

## Run a model and check the time series
TODO

# Tutorial tests

## Perform tutorial in documentation

Go through the tutorial as described in the [How-to guide](/guide/qgis.qmd).
6 changes: 3 additions & 3 deletions docs/dev/release.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,10 @@ In order to publish Ribasim Python or Ribasim API follow the following steps:

2) Open a terminal and run `pixi run publish-ribasim-api`

## Do manual checks
## QGIS manual testing

Our continuous integration (CI) should have caught most issues.
A current weak spot in our testing is the QGIS plugin, so it is a good idea to do some manual checks to see if it works properly.
A current weak spot in our testing is the QGIS plugin, so a manual test plan is in place.
Start with running the automated task to see if it can be correctly installed.

```sh
Expand All @@ -96,7 +96,7 @@ Start with running the automated task to see if it can be correctly installed.
pixi run test-ribasim-qgis-ui
```

It is a good idea to load new test models if there are any, or test any other changed functionality.
Then follow the instructions as described in the [QGIS manual test plan](/dev/qgis_test_plan.qmd).

## Announce release

Expand Down
Loading

0 comments on commit 3e70543

Please sign in to comment.