[Enhance] Enhance vis-pipeline tool. (#604)

* enhance vis-pipeline add intermediate imgs * enhance vis-pipeline add intermediate imgs * improve code of vi-pipeline * modify docs for vis-pipeline * Use `mmcv.utils.digit_version` instead of `distutils` * add size info in the bottom * preform adaptive-resize in before concat * add warning info * fix docs * fix lint * fix comment * fix docs Co-authored-by: mzr1996 <mzr1996@163.com>
open-mmlab · Mar 4, 2022 · d08c2a1 · d08c2a1
1 parent 779a062
commit d08c2a1
Show file tree

Hide file tree

Showing 10 changed files with 241 additions and 144 deletions.
diff --git a/docs/en/_static/image/tools/visualization/pipeline-concat.jpg b/docs/en/_static/image/tools/visualization/pipeline-concat.jpg
diff --git a/docs/en/_static/image/tools/visualization/pipeline-original.jpg b/docs/en/_static/image/tools/visualization/pipeline-original.jpg
diff --git a/docs/en/_static/image/tools/visualization/pipeline-pipeline.jpg b/docs/en/_static/image/tools/visualization/pipeline-pipeline.jpg
diff --git a/docs/en/tools/miscellaneous.md b/docs/en/tools/miscellaneous.md
@@ -1,4 +1,4 @@
-# MISCELLANEOUS
+# Miscellaneous
 
 <!-- TOC -->
 

diff --git a/docs/en/tools/visualization.md b/docs/en/tools/visualization.md
@@ -2,78 +2,87 @@
 
 <!-- TOC -->
 
-- [Visualization](#visualization)
-  - [Pipeline Visualization](#pipeline-visualization)
-  - [Learning Rate Schedule Visualization](#learning-rate-schedule-visualization)
-  - [Class Activation Map Visualization](#class-activation-map-visualization)
-  - [FAQs](#faqs)
+- [Pipeline Visualization](#pipeline-visualization)
+- [Learning Rate Schedule Visualization](#learning-rate-schedule-visualization)
+- [Class Activation Map Visualization](#class-activation-map-visualization)
+- [FAQs](#faqs)
 
 <!-- TOC -->
 ## Pipeline Visualization
 
 ```bash
 python tools/visualizations/vis_pipeline.py \
     ${CONFIG_FILE} \
-    --output-dir ${OUTPUT_DIR} \
-    --phase ${DATASET_PHASE} \
-    --number ${BUNBER_IMAGES_DISPLAY} \
-    --skip-type ${SKIP_TRANSFORM_TYPE}
-    --mode ${DISPLAY_MODE} \
-    --show \
-    --adaptive \
-    --min-edge-length ${MIN_EDGE_LENGTH} \
-    --max-edge-length ${MAX_EDGE_LENGTH} \
-    --bgr2rgb \
-    --window-size ${WINDOW_SIZE}
+    [--output-dir ${OUTPUT_DIR}] \
+    [--phase ${DATASET_PHASE}] \
+    [--number ${BUNBER_IMAGES_DISPLAY}] \
+    [--skip-type ${SKIP_TRANSFORM_TYPE}] \
+    [--mode ${DISPLAY_MODE}] \
+    [--show] \
+    [--adaptive] \
+    [--min-edge-length ${MIN_EDGE_LENGTH}] \
+    [--max-edge-length ${MAX_EDGE_LENGTH}] \
+    [--bgr2rgb] \
+    [--window-size ${WINDOW_SIZE}] \
+    [--cfg-options ${CFG_OPTIONS}]
 ```
 
 **Description of all arguments**：
 
 - `config` : The path of a model config file.
 - `--output-dir`: The output path for visualized images. If not specified, it will be set to `''`, which means not to save.
 - `--phase`: Phase of visualizing dataset，must be one of `[train, val, test]`. If not specified, it will be set to `train`.
-- `--number`: The number of samples to visualize. If not specified, display all images in the dataset.
+- `--number`: The number of samples to visualized. If not specified, display all images in the dataset.
 - `--skip-type`: The pipelines to be skipped. If not specified, it will be set to `['ToTensor', 'Normalize', 'ImageToTensor', 'Collect']`.
 - `--mode`: The display mode, can be one of `[original, pipeline, concat]`. If not specified, it will be set to `concat`.
 - `--show`: If set, display pictures in pop-up windows.
-- `--adaptive`: If set, automatically adjust the size of the visualization images.
+- `--adaptive`: If set, adaptively resize images for better visualization.
 - `--min-edge-length`: The minimum edge length, used when `--adaptive` is set. When any side of the picture is smaller than `${MIN_EDGE_LENGTH}`, the picture will be enlarged while keeping the aspect ratio unchanged, and the short side will be aligned to `${MIN_EDGE_LENGTH}`. If not specified, it will be set to 200.
 - `--max-edge-length`: The maximum edge length, used when `--adaptive` is set. When any side of the picture is larger than `${MAX_EDGE_LENGTH}`, the picture will be reduced while keeping the aspect ratio unchanged, and the long side will be aligned to `${MAX_EDGE_LENGTH}`. If not specified, it will be set to 1000.
 - `--bgr2rgb`: If set, flip the color channel order of images.
 - `--window-size`: The shape of the display window. If not specified, it will be set to `12*7`. If used, it must be in the format `'W*H'`.
+- `--cfg-options` : Modifications to the configuration file, refer to [Tutorial 1: Learn about Configs](https://mmclassification.readthedocs.io/en/latest/tutorials/config.html).
 
 ```{note}
 
-1. If the `--mode` is not specified, it will be set to `concat` as default, get the pictures stitched together by original pictures and transformed pictures; if the `--mode` is set to `original`, get the original pictures; if the `--mode` is set to `pipeline`, get the transformed pictures.
+1. If the `--mode` is not specified, it will be set to `concat` as default, get the pictures stitched together by original pictures and transformed pictures; if the `--mode` is set to `original`, get the original pictures; if the `--mode` is set to `transformed`, get the transformed pictures; if the `--mode` is set to `pipeline`, get all the intermediate images through the pipeline.
 
 2. When `--adaptive` option is set, images that are too large or too small will be automatically adjusted, you can use `--min-edge-length` and `--max-edge-length` to set the adjust size.
 ```
 
 **Examples**：
 
-1. Visualize all the transformed pictures of the `ImageNet` training set and display them in pop-up windows：
+1. In **'original'** mode, visualize 100 original pictures in the `CIFAR100` validation set, then display and save them in the `./tmp` folder：
 
-   ```shell
-   python ./tools/visualizations/vis_pipeline.py ./configs/resnet/resnet50_8xb32_in1k.py --show --mode pipeline
-   ```
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py configs/resnet/resnet50_8xb16_cifar100.py --phase val --output-dir tmp --mode original --number 100  --show --adaptive --bgr2rgb
+  ```
 
-   <div align=center><img src="../_static/image/tools/visualization/pipeline-pipeline.jpg" style=" width: auto; height: 40%; "></div>
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146117528-1ec2d918-57f8-4ae4-8ca3-a8d31b602f64.jpg" style=" width: auto; height: 40%; "></div>
 
-2. Visualize 10 comparison pictures in the `ImageNet` train set and save them in the `./tmp` folder：
+2. In **'transformed'** mode, visualize all the transformed pictures of the `ImageNet` training set and display them in pop-up windows：
 
-   ```shell
-   python ./tools/visualizations/vis_pipeline.py configs/swin_transformer/swin_base_224_b16x64_300e_imagenet.py --phase train --output-dir tmp --number 10 --adaptive
-   ```
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py ./configs/resnet/resnet50_8xb32_in1k.py --show --mode transformed
+  ```
 
-   <div align=center><img src="../_static/image/tools/visualization/pipeline-concat.jpg" style=" width: auto; height: 40%; "></div>
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146117553-8006a4ba-e2fa-4f53-99bc-42a4b06e413f.jpg" style=" width: auto; height: 40%; "></div>
 
-3. Visualize 100 original pictures in the `CIFAR100` validation set, then display and save them in the `./tmp` folder：
+3. In **'concat'** mode, visualize 10 pairs of origin and transformed images for comparison in the `ImageNet` train set and save them in the `./tmp` folder：
 
-   ```shell
-   python ./tools/visualizations/vis_pipeline.py configs/resnet/resnet50_8xb16_cifar100.py --phase val --output-dir tmp --mode original --number 100  --show --adaptive --bgr2rgb
-   ```
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py configs/swin_transformer/swin_base_224_b16x64_300e_imagenet.py --phase train --output-dir tmp --number 10 --adaptive
+  ```
+
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146128259-0a369991-7716-411d-8c27-c6863e6d76ea.JPEG" style=" width: auto; height: 40%; "></div>
+
+4. In **'pipeline'** mode, visualize all the intermediate pictures in the `ImageNet` train set through the pipeline：
+
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py configs/swin_transformer/swin_base_224_b16x64_300e_imagenet.py --phase train --adaptive --mode pipeline --show
+  ```
 
-   <div align=center><img src="../_static/image/tools/visualization/pipeline-original.jpg" style=" width: auto; height: 40%; "></div>
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146128201-eb97c2aa-a615-4a81-a649-38db1c315d0e.JPEG" style=" width: auto; height: 40%; "></div>
 
 ## Learning Rate Schedule Visualization
 

diff --git a/docs/zh_CN/_static/image/tools/visualization/pipeline-concat.jpg b/docs/zh_CN/_static/image/tools/visualization/pipeline-concat.jpg
diff --git a/docs/zh_CN/_static/image/tools/visualization/pipeline-original.jpg b/docs/zh_CN/_static/image/tools/visualization/pipeline-original.jpg
diff --git a/docs/zh_CN/_static/image/tools/visualization/pipeline-pipeline.jpg b/docs/zh_CN/_static/image/tools/visualization/pipeline-pipeline.jpg
diff --git a/docs/zh_CN/tools/visualization.md b/docs/zh_CN/tools/visualization.md
@@ -2,11 +2,10 @@
 
 <!-- TOC -->
 
-- [可视化](#可视化)
-  - [数据流水线可视化](#数据流水线可视化)
-  - [学习率策略可视化](#学习率策略可视化)
-  - [类别激活图可视化](#类别激活图可视化)
-  - [常见问题](#常见问题)
+- [数据流水线可视化](#数据流水线可视化)
+- [学习率策略可视化](#学习率策略可视化)
+- [类别激活图可视化](#类别激活图可视化)
+- [常见问题](#常见问题)
 
 <!-- TOC -->
 
@@ -15,17 +14,18 @@
 ```bash
 python tools/visualizations/vis_pipeline.py \
     ${CONFIG_FILE} \
-    --output-dir ${OUTPUT_DIR} \
-    --phase ${DATASET_PHASE} \
-    --number ${BUNBER_IMAGES_DISPLAY} \
-    --skip-type ${SKIP_TRANSFORM_TYPE} \
-    --mode ${DISPLAY_MODE} \
-    --show \
-    --adaptive \
-    --min-edge-length ${MIN_EDGE_LENGTH} \
-    --max-edge-length ${MAX_EDGE_LENGTH} \
-    --bgr2rgb \
-    --window-size ${WINDOW_SIZE}
+    [--output-dir ${OUTPUT_DIR}] \
+    [--phase ${DATASET_PHASE}] \
+    [--number ${BUNBER_IMAGES_DISPLAY}] \
+    [--skip-type ${SKIP_TRANSFORM_TYPE}] \
+    [--mode ${DISPLAY_MODE}] \
+    [--show] \
+    [--adaptive] \
+    [--min-edge-length ${MIN_EDGE_LENGTH}] \
+    [--max-edge-length ${MAX_EDGE_LENGTH}] \
+    [--bgr2rgb] \
+    [--window-size ${WINDOW_SIZE}] \
+    [--cfg-options ${CFG_OPTIONS}]
 ```
 
 **所有参数的说明**：
@@ -35,71 +35,80 @@ python tools/visualizations/vis_pipeline.py \
 - `--phase`: 可视化数据集的阶段，只能为 `[train, val, test]` 之一，默认为 `train`。
 - `--number`: 可视化样本数量。如果没有指定，默认展示数据集的所有图片。
 - `--skip-type`: 预设跳过的数据流水线过程。如果没有指定，默认为 `['ToTensor', 'Normalize', 'ImageToTensor', 'Collect']`。
-- `--mode`: 可视化的模式，只能为 `[original, pipeline, concat]` 之一，如果没有指定，默认为 `concat`。
+- `--mode`: 可视化的模式，只能为 `[original, transformed, concat, pipeline]` 之一，如果没有指定，默认为 `concat`。
 - `--show`: 将可视化图片以弹窗形式展示。
 - `--adaptive`: 自动调节可视化图片的大小。
 - `--min-edge-length`: 最短边长度，当使用了 `--adaptive` 时有效。 当图片任意边小于 `${MIN_EDGE_LENGTH}` 时，会保持长宽比不变放大图片，短边对齐至 `${MIN_EDGE_LENGTH}`，默认为200。
 - `--max-edge-length`: 最长边长度，当使用了 `--adaptive` 时有效。 当图片任意边大于 `${MAX_EDGE_LENGTH}` 时，会保持长宽比不变缩小图片，短边对齐至 `${MAX_EDGE_LENGTH}`，默认为1000。
 - `--bgr2rgb`: 将图片的颜色通道翻转。
 - `--window-size`: 可视化窗口大小，如果没有指定，默认为 `12*7`。如果需要指定，按照格式 `'W*H'`。
+- `--cfg-options` : 对配置文件的修改，参考[教程 1：如何编写配置文件](https://mmclassification.readthedocs.io/zh_CN/latest/tutorials/config.html)。
 
 ```{note}
 
-1. 如果不指定 `--mode`，默认设置为 `concat`，获取原始图片和预处理后图片拼接的图片；如果 `--mode` 设置为 `original`，则获取原始图片； 如果  `--mode` 设置为 `pipeline`，则获取预处理后的图片。
+1. 如果不指定 `--mode`，默认设置为 `concat`，获取原始图片和预处理后图片拼接的图片；如果 `--mode` 设置为 `original`，则获取原始图片；如果 `--mode` 设置为 `transformed`，则获取预处理后的图片；如果 `--mode` 设置为 `pipeline`，则获得数据流水线所有中间过程图片。
 
 2. 当指定了 `--adaptive` 选项时，会自动的调整尺寸过大和过小的图片，你可以通过设定 `--min-edge-length` 与 `--max-edge-length` 来指定自动调整的图片尺寸。
 ```
 
 **示例**：
 
-1. 可视化 `ImageNet` 训练集的所有经过预处理的图片，并以弹窗形式显示：
+1. **'original'** 模式，可视化 `CIFAR100` 验证集中的100张原始图片，显示并保存在 `./tmp` 文件夹下：
 
-   ```shell
-   python ./tools/visualizations/vis_pipeline.py ./configs/resnet/resnet50_8xb32_in1k.py --show --mode pipeline
-   ```
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py configs/resnet/resnet50_8xb16_cifar100.py --phase val --output-dir tmp --mode original --number 100 --show --adaptive --bgr2rgb
+  ```
 
-   <div align=center><img src="../_static/image/tools/visualization/pipeline-pipeline.jpg" style=" width: auto; height: 40%; "></div>
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146117528-1ec2d918-57f8-4ae4-8ca3-a8d31b602f64.jpg" style=" width: auto; height: 40%; "></div>
 
-2. 可视化 `ImageNet` 训练集的10张原始图片与预处理后图片对比图，保存在 `./tmp` 文件夹下：
+2. **'transformed'** 模式，可视化 `ImageNet` 训练集的所有经过预处理的图片，并以弹窗形式显示：
 
-   ```shell
-   python ./tools/visualizations/vis_pipeline.py configs/swin_transformer/swin_base_224_b16x64_300e_imagenet.py --phase train --output-dir tmp --number 10 --adaptive
-   ```
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py ./configs/resnet/resnet50_8xb32_in1k.py --show --mode transformed
+  ```
 
-   <div align=center><img src="../_static/image/tools/visualization/pipeline-concat.jpg" style=" width: auto; height: 40%; "></div>
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146117553-8006a4ba-e2fa-4f53-99bc-42a4b06e413f.jpg" style=" width: auto; height: 40%; "></div>
 
-3. 可视化 `CIFAR100` 验证集中的100张原始图片，显示并保存在 `./tmp` 文件夹下：
+3. **'concat'** 模式，可视化 `ImageNet` 训练集的10张原始图片与预处理后图片对比图，保存在 `./tmp` 文件夹下：
 
-   ```shell
-   python ./tools/visualizations/vis_pipeline.py configs/resnet/resnet50_8xb16_cifar100.py --phase val --output-dir tmp --mode original --number 100 --show --adaptive --bgr2rgb
-   ```
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py configs/swin_transformer/swin_base_224_b16x64_300e_imagenet.py --phase train --output-dir tmp --number 10 --adaptive
+  ```
 
-   <div align=center><img src="../_static/image/tools/visualization/pipeline-original.jpg" style=" width: auto; height: 40%; "></div>
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146128259-0a369991-7716-411d-8c27-c6863e6d76ea.JPEG" style=" width: auto; height: 40%; "></div>
+
+4. **'pipeline'** 模式，可视化 `ImageNet` 训练集经过数据流水线的过程图像：
+
+  ```shell
+  python ./tools/visualizations/vis_pipeline.py configs/swin_transformer/swin_base_224_b16x64_300e_imagenet.py --phase train --adaptive --mode pipeline --show
+  ```
+
+  <div align=center><img src="https://user-images.githubusercontent.com/18586273/146128201-eb97c2aa-a615-4a81-a649-38db1c315d0e.JPEG" style=" width: auto; height: 40%; "></div>
 
 ## 学习率策略可视化
 
 ```bash
 python tools/visualizations/vis_lr.py \
     ${CONFIG_FILE} \
-    --dataset-size ${Dataset_Size} \
-    --ngpus ${NUM_GPUs}
-    --save-path ${SAVE_PATH} \
-    --title ${TITLE} \
-    --style ${STYLE} \
-    --window-size ${WINDOW_SIZE}
-    --cfg-options
+    [--dataset-size ${Dataset_Size}] \
+    [--ngpus ${NUM_GPUs}] \
+    [--save-path ${SAVE_PATH}] \
+    [--title ${TITLE}] \
+    [--style ${STYLE}] \
+    [--window-size ${WINDOW_SIZE}] \
+    [--cfg-options ${CFG_OPTIONS}] \
 ```
 
 **所有参数的说明**：
 
 - `config` : 模型配置文件的路径。
-- `dataset-size` : 数据集的大小。如果指定，`build_dataset` 将被跳过并使用这个大小作为数据集大小，默认使用 `build_dataset` 所得数据集的大小。
-- `ngpus` : 使用 GPU 的数量。
-- `save-path` : 保存的可视化图片的路径，默认不保存。
-- `title` : 可视化图片的标题，默认为配置文件名。
-- `style` : 可视化图片的风格，默认为 `whitegrid`。
-- `window-size`: 可视化窗口大小，如果没有指定，默认为 `12*7`。如果需要指定，按照格式 `'W*H'`。
-- `cfg-options` : 对配置文件的修改，参考[教程 1：如何编写配置文件](https://mmclassification.readthedocs.io/zh_CN/latest/tutorials/config.html)。
+- `--dataset-size` : 数据集的大小。如果指定，`build_dataset` 将被跳过并使用这个大小作为数据集大小，默认使用 `build_dataset` 所得数据集的大小。
+- `--ngpus` : 使用 GPU 的数量。
+- `--save-path` : 保存的可视化图片的路径，默认不保存。
+- `--title` : 可视化图片的标题，默认为配置文件名。
+- `--style` : 可视化图片的风格，默认为 `whitegrid`。
+- `--window-size`: 可视化窗口大小，如果没有指定，默认为 `12*7`。如果需要指定，按照格式 `'W*H'`。
+- `--cfg-options` : 对配置文件的修改，参考[教程 1：如何编写配置文件](https://mmclassification.readthedocs.io/zh_CN/latest/tutorials/config.html)。
 
 ```{note}