Gillan suggestions #73
Conversation
There was a problem hiding this comment.
Thanks @jeffgillan , super helpful and a major improvement! I answer your questions and make a few additional suggestions here.
| ``` | ||
| docker pull ghcr.io/open-forest-observatory/automate-metashape:latest | ||
| ``` | ||
| The `automate-metashape` docker image contains the python libraries needed to run the script, while you (the user) need to provide at minimum the **1.** aerial images; **2** a configuration file specifying your choices for processing; **3.** a license to use Metashape; and optionally **4.** [ground control points (GCPs)](https://github.com/jeffgillan/automate-metashape/tree/main?tab=readme-ov-file#preparing-ground-control-points-gcps). |
There was a problem hiding this comment.
I'd just change "libraries needed to run the script" to "libraries needed to run the workflow", and your link to the GCPs section should point to the open-forest-observatory copy of this repo, not the jeffgillan clone (though maybe github takes care of that when the PR is merged?).
There was a problem hiding this comment.
Good suggestion on the first one and good catch on the second one.
| #### Image Directory | ||
| On your local machine please create a directory that includes all of the aerial images (e.g., `/home/aerial_images`). This directory can be called anything you want and be located anywhere on your computer. | ||
|
|
||
| **????The input data, config file, and output data can each be any number of levels below the data directory (meaning you do not need a separate data directory for each project to be run).????** |
There was a problem hiding this comment.
Here is a hopefully clearer way to say this:
To provide the input data to Metashape, you need to specify a folder from your computer to be mirrored ("mounted") within the Docker container. The files needed to run Metashape (the folder of aerial images, the configuration file, and optionally the GCPs file) must all be in the folder that you mount. The files can be located in any arbitrary directory beneath this folder. For example, if the folder you mount is
~/drone_dataon your local computer, the images could be located at~/drone_data/projects/project_10/images/and the config file could be located at~/drone_data/configs/project_10/config.yml. The folder from your local machine is mounted into the Docker container at the path/data/, so for the example above, the images would be found inside the docker container at the path/data/projects/project_10/images/.
What do you think?
There was a problem hiding this comment.
Good explanation!
| <br/> | ||
|
|
||
| #### Configuration File | ||
| Please include the configuration file within the same directory as the images. An example configuration file is provided in this repository at `config/config-example.yml`. Please download this file to your local machine and rename it `config.yml`. Within the `config.yml` you will need to edit some of the project level parameters to specify where to find input images and where to put output products within the container. Within this config file, all paths will be relative to the file structure of the docker container. Please edit the `photo_path` to read "/data", the `output_path` to read "/data/output", and the `project_path` to read "/data/project". You have flexibility to change these paths as long as they begin with "/data/". | ||
|
|
||
| By default, the container expects the config YAML file describing the Metashape workflow parameters to be located at `/data/config.yaml`, but this can be overridden by passing a different location as a final (optional) command line argument to the `docker run` command. This would be useful if you have multiple imagery datasets/projects (and config files) nested below your mounted `/data` directory and/or if you are taking advantage of the `automate-metashape` feature to name output files based on the name of the config file. |
There was a problem hiding this comment.
Can add after "to be located at /data/config.yaml":
(meaning, in the example above, on your local computer it would be at
~/drone_data/config.yaml)
| <br/> | ||
|
|
||
| #### Configuration File | ||
| Please include the configuration file within the same directory as the images. An example configuration file is provided in this repository at `config/config-example.yml`. Please download this file to your local machine and rename it `config.yml`. Within the `config.yml` you will need to edit some of the project level parameters to specify where to find input images and where to put output products within the container. Within this config file, all paths will be relative to the file structure of the docker container. Please edit the `photo_path` to read "/data", the `output_path` to read "/data/output", and the `project_path` to read "/data/project". You have flexibility to change these paths as long as they begin with "/data/". |
There was a problem hiding this comment.
If you like my suggestion just above, then you can delete the first sentence here. And possibly the header #### Image Directory should be broadened to something like #### Input files or #### Input imagery and processing configuration file.
Also for the part starting "Please edit the photo path...", I think you can get more generic, like:
In the config.yml, at a minimum the following entries should be updated:
- The value for
photo_pathshould be updated to/data/{path_to_images_folder_within_mounted_folder} - The value for
output_pathshould be updated to/data/{path_to_desired_ouputs_folder_within_mounted_folder}(can be any location you want; will be created if it does not exist) - The value for
project_pathshould be updated similarly as foroutput_path.
There was a problem hiding this comment.
Here's my suggestion:
User Input
To provide the input data to Metashape, you need to specify a folder from your computer to be mirrored ("mounted") within the Docker container. The files needed to run Metashape (the folder of aerial images, the configuration file, and optionally the GCPs file) must all be in the folder that you mount. The files can be located in any arbitrary directory beneath this folder. For example, if the folder you mount is ~/drone_data on your local computer, the images could be located at ~/drone_data/projects/project_10/images/ and the config file could be located at ~/drone_data/configs/project_10/config.yml The folder from your local machine is mounted into the Docker container at the path /data/, so for the example above, the images would be found inside the docker container at the path /data/projects/project_10/images/.
An example configuration file is provided in this repository at config/config-example.yml. Please download this file to your local machine and rename it config.yml. By default, the container expects the config YAML file describing the Metashape workflow parameters to be located at /data/config.yaml, but this can be overridden by passing a different location as a final (optional) command line argument to the docker run command (see below).
Within the config.yml you will need to edit some of the project level parameters to specify where to find input images and where to put output products within the container. Within this config file, all paths will be relative to the file structure of the docker container. In the config.yml, at a minimum the following entries should be updated:
- The value for photo_path should be updated to /data/{path_to_images_folder_within_mounted_folder}
- The value for output_path should be updated to /data/{path_to_desired_ouputs_folder_within_mounted_folder} (can be any location you want; will be created if it does not exist)
- The value for project_path should be updated similarly as for output_path.
|
|
||
| #### Enable GPUs for Accelerated Processing | ||
|
|
||
| The use of graphical processing units (GPUs) can greatly increase the speed of photogrammetry processing. If your machine has GPU hardware, you will need extra software so docker can find and use your GPUs. For linux users please see [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). For Windows users please see [this documentation](https://docs.docker.com/desktop/features/gpu/). For macOS user, it may not be possible to use your local GPU (Apple Silicon) through Docker. |
There was a problem hiding this comment.
For linux, the only thing people need to know about nvidia-container-toolkit is that it must be installed, via sudo apt install nvidia-container-toolkit, so maybe we could just say that here?
|
|
||
| `ghcr.io/open-forest-observatory/automate-metashape` This is the docker image that has the software to run the `automate-metashape` script. It is located in the Github container registry. When you execute the `docker run...` command, it will download the container image to your local machine and start the script to process imagery using Metashape. | ||
|
|
||
| **???How do I change the location of config.yml or run multiple configs???** |
There was a problem hiding this comment.
If your config.yaml is located anywhere other than /data/config.yaml (or the file is named differently), you can specify its location as one additional command line argument at the end of the docker run command. For example, if it is located at /data/configs/project_10/config.yml (meanining, in the example above, it is located on your local computer at ~/drone_data/configs/project_10/config.yml), just append /data/configs/project_10/config.yml to the docker run command above.
There was a problem hiding this comment.
Can you please write out what the full docker run command would look like if you append the new config.yml path? I'm not understanding exactly.
There was a problem hiding this comment.
Yes, for this case it would be:
docker run -v </host/data/dir>:/data -e AGISOFT_FLS=$AGISOFT_FLS --gpus all ghcr.io/open-forest-observatory/automate-metashape /data/configs/project_10/config.yml
| <br/> | ||
|
|
||
| #### Outputs | ||
| When the script is done running, the completed imagery products will be deposited within the same directory on your local machine that contains the images and config file (e.g., '/home/aerial_images'). |
There was a problem hiding this comment.
Not exactly, at least not always. I suggest changing to the following:
As the processing runs, the completed imagery products will be deposited in the folder you specified for the
output_pathparameter in the config.yaml. In the example above, if your config.yaml specifies theoutput_pathas/data/{path_to_desired_ouputs_folder_within_mounted_folder}, the outputs will be saved on your local computer at~/drone_data/{path_to_desired_ouputs_folder_within_mounted_folder}.
| The following command puts it all together. This runs `automate_metashape` on the config file named `config.yml` within the mounted `/data` directory and writes the results back out to the same folder | ||
| ``` | ||
| docker run -v </host/data/dir>:/data -e AGISOFT_FLS=$AGISOFT_FLS --gpus all ghcr.io/open-forest-observatory/automate-metashape | ||
| ``` | ||
| Note that the owner of the output data will be the `root` user. To set the ownership to your user account, you can run `sudo chown <username>:<username> <file name>` or `sudo chown <username>:<username> -R <folder name>`. |
There was a problem hiding this comment.
Add "on Linux or Mac" run sudo...
Hi OFO, I re-organized the instructions for using Docker in the readme.md. Please review and accept what you feel is appropriate. I have some questions about 1. the organization of the imagery folder, and 2. how to alter the docker run command to change the default location of the config.yml. I would be happy to discuss these topics further so we can make the readme very clear to outside users.