Skip to content

Commit

Permalink
add lovasz loss, mixed loss, road extration dataset (#825)
Browse files Browse the repository at this point in the history
  • Loading branch information
LutaoChu authored Feb 22, 2021
1 parent df178c2 commit 038cde8
Show file tree
Hide file tree
Showing 18 changed files with 683 additions and 1 deletion.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ python train.py --config configs/quick_start/bisenet_optic_disc_512x512_1k.yml
* [API Tutorial](https://aistudio.baidu.com/aistudio/projectdetail/1339458)
* [Data Preparation](./docs/data_prepare.md)
* [Training Configuration](./configs/)
* [Loss Usage](./docs/loss_usage.md)
* [API References](./docs/apis)
* [Add New Components](./docs/add_new_model.md)
* [Model Compression](./slim)
Expand Down
1 change: 1 addition & 0 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ python train.py --config configs/quick_start/bisenet_optic_disc_512x512_1k.yml
* [API使用教程](https://aistudio.baidu.com/aistudio/projectdetail/1339458)
* [数据集准备](./docs/data_prepare.md)
* [配置项](./configs/)
* [Loss使用](./docs/loss_usage.md)
* [API参考](./docs/apis)
* [添加新组件](./docs/add_new_model.md)
* [模型压缩](./slim)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
_base_: '../_base_/cityscapes.yml'

batch_size: 2
iters: 160000

model:
type: OCRNet
backbone:
type: HRNet_W18
pretrained: https://bj.bcebos.com/paddleseg/dygraph/hrnet_w18_ssld.tar.gz
backbone_indices: [0]

optimizer:
type: sgd

learning_rate:
value: 0.01
decay:
type: poly
power: 0.9


loss:
types:
- type: MixedLoss
losses:
- type: CrossEntropyLoss
- type: LovaszSoftmaxLoss
coef: [0.8, 0.2]
- type: MixedLoss
losses:
- type: CrossEntropyLoss
- type: LovaszSoftmaxLoss
coef: [0.8, 0.2]
coef: [1, 0.4]
48 changes: 48 additions & 0 deletions configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@


batch_size: 4
iters: 15000

train_dataset:
type: MiniDeepGlobeRoadExtraction
dataset_root: data/MiniDeepGlobeRoadExtraction
transforms:
- type: ResizeStepScaling
min_scale_factor: 0.5
max_scale_factor: 2.0
scale_step_size: 0.25
- type: RandomPaddingCrop
crop_size: [768, 768]
- type: RandomHorizontalFlip
- type: Normalize
mode: train

val_dataset:
type: MiniDeepGlobeRoadExtraction
dataset_root: data/MiniDeepGlobeRoadExtraction
transforms:
- type: Normalize
mode: val

model:
type: OCRNet
backbone:
type: HRNet_W18
pretrained: https://bj.bcebos.com/paddleseg/dygraph/hrnet_w18_ssld.tar.gz
backbone_indices: [0]

optimizer:
type: sgd

learning_rate:
value: 0.01
decay:
type: poly
power: 0.9


loss:
types:
- type: CrossEntropyLoss
- type: CrossEntropyLoss
coef: [1, 0.4]
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@


batch_size: 4
iters: 15000

train_dataset:
type: MiniDeepGlobeRoadExtraction
dataset_root: data/MiniDeepGlobeRoadExtraction
transforms:
- type: ResizeStepScaling
min_scale_factor: 0.5
max_scale_factor: 2.0
scale_step_size: 0.25
- type: RandomPaddingCrop
crop_size: [768, 768]
- type: RandomHorizontalFlip
- type: Normalize
mode: train

val_dataset:
type: MiniDeepGlobeRoadExtraction
dataset_root: data/MiniDeepGlobeRoadExtraction
transforms:
- type: Normalize
mode: val

model:
type: OCRNet
backbone:
type: HRNet_W18
pretrained: https://bj.bcebos.com/paddleseg/dygraph/hrnet_w18_ssld.tar.gz
backbone_indices: [0]

optimizer:
type: sgd

learning_rate:
value: 0.01
decay:
type: poly
power: 0.9

loss:
types:
- type: MixedLoss
losses:
- type: CrossEntropyLoss
- type: LovaszHingeLoss
coef: [1, 0.01]
- type: MixedLoss
losses:
- type: CrossEntropyLoss
- type: LovaszHingeLoss
coef: [1, 0.01]
coef: [1, 0.4]
1 change: 1 addition & 0 deletions docs/apis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@
* [paddleseg.datasets](./datasets.md)
* [paddleseg.models](./models.md)
* [paddleseg.models.backbones](./backbones.md)
* [paddleseg.models.losses](./losses.md)
* [paddleseg.transforms](./transforms.md)
33 changes: 33 additions & 0 deletions docs/apis/losses.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# [paddleseg.models.losses](../../paddleseg/models/losses)

## LovaszSoftmaxLoss
> CLASS paddleseg.models.losses.LovaszSoftmaxLoss(ignore_index=255, classes='present')
Multi-class Lovasz-Softmax loss.

> > Args
> > > - **ignore_index** (int64): Specifies a target value that is ignored and does not contribute to the input gradient. Default ``255``.
> > > - **classes** (str|list): 'all' for all, 'present' for classes present in labels, or a list of classes to average.

## LovaszHingeLoss
> CLASS paddleseg.models.losses.LovaszHingeLoss(ignore_index=255)
Binary Lovasz hinge loss.

> > Args
> > > - **ignore_index** (int64): Specifies a target value that is ignored and does not contribute to the input gradient. Default ``255``.

## MixedLoss
> CLASS paddleseg.models.losses.MixedLoss(losses, coef)
Weighted computations for multiple Loss.
The advantage is that mixed loss training can be achieved without changing the networking code.

> > Args
> > > - **losses** (list of nn.Layer): A list consisting of multiple loss classes
> > > - **coef** (float|int): Weighting coefficient of multiple loss
> > Returns
> > > - A callable object of MixedLoss.
Binary file added docs/images/Lovasz_Hinge_Evaluate_mIoU.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/Lovasz_Softmax_Evaluate_mIoU.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/deepglobe.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 4 additions & 0 deletions docs/loss_usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Loss usage

- [Lovasz loss](lovasz_loss.md)
- To be continued
125 changes: 125 additions & 0 deletions docs/lovasz_loss.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
# Lovasz loss
对于图像分割任务中,经常出现类别分布不均匀的情况,例如:工业产品的瑕疵检测、道路提取及病变区域提取等。我们可使用lovasz loss解决这个问题。

Lovasz loss基于子模损失(submodular losses)的凸Lovasz扩展,对神经网络的mean IoU损失进行优化。Lovasz loss根据分割目标的类别数量可分为两种:lovasz hinge loss和lovasz softmax loss. 其中lovasz hinge loss适用于二分类问题,lovasz softmax loss适用于多分类问题。该工作发表在CVPR 2018上,可点击[参考文献](#参考文献)查看具体原理。


## Lovasz loss使用指南
接下来介绍如何使用lovasz loss进行训练。需要注意的是,通常的直接训练方式并一定管用,我们推荐另外2种训练方式:
- (1)与cross entropy loss或bce loss(binary cross-entropy loss)加权结合使用。
- (2)先使用cross entropy loss或bce loss进行训练,再使用lovasz softmax loss或lovasz hinge loss进行finetuning.

以方式(1)为例,通过`MixedLoss`类选择训练时的损失函数, 通过`coef`参数对不同loss进行权重配比,从而灵活地进行训练调参。如下所示:

```yaml
loss:
types:
- type: MixedLoss
losses:
- type: CrossEntropyLoss
- type: LovaszSoftmaxLoss
coef: [0.8, 0.2]
```
```yaml
loss:
types:
- type: MixedLoss
losses:
- type: CrossEntropyLoss
- type: LovaszHingeLoss
coef: [1, 0.02]
```
## Lovasz softmax loss实验对比
接下来以经典的[Cityscapes](https://www.cityscapes-dataset.com/)数据集为例应用lovasz softmax loss. Cityscapes数据集共有19类目标,其中的类别并不均衡,例如类别`road`、`building`很常见,`fence`、`motocycle`、`wall`则较为罕见。我们将lovasz softmax loss与softmax loss进行了实验对比。这里使用OCRNet模型,backbone为HRNet w18.


* 数据准备

见[数据集准备教程](data_prepare.md)

* Lovasz loss训练
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
--config configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k_lovasz_softmax.yml \
--use_vdl --num_workers 3 --do_eval
```

* Cross entropy loss训练
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
--config configs/ocrnet/ocrnet_hrnetw18_cityscapes_1024x512_160k.yml \
--use_vdl --num_workers 3 --do_eval
```

* 结果比较

实验mIoU曲线如下图所示。
<p align="center">
<img src="./images/Lovasz_Softmax_Evaluate_mIoU.png" hspace='10' /> <br />
</p>


|Loss|best mIoU|
|-|-|
|cross entropy loss|80.46%|
|lovasz softmax loss + cross entropy loss|81.53%|

图中蓝色曲线代表lovasz softmax loss + cross entropy loss,绿色曲线代表cross entropy loss,相比提升1个百分点。

可看出使用lovasz softmax loss后,精度曲线基本都高于原来的精度。

## Lovasz hinge loss实验对比

我们以道路提取任务为例应用lovasz hinge loss.
基于MiniDeepGlobeRoadExtraction数据集与cross entropy loss进行了实验对比。
该数据集来源于[DeepGlobe CVPR2018挑战赛](http://deepglobe.org/)的Road Extraction单项,训练数据道路占比为 4.5%. 道路在整张图片中的比例很小,是典型的类别不均衡场景。图片样例如下:
<p align="center">
<img src="./images/deepglobe.png" hspace='10'/> <br />
</p>

这里使用OCRNet模型,backbone为HRNet w18.

* 数据集
我们从DeepGlobe比赛的Road Extraction的训练集中随机抽取了800张图片作为训练集,200张图片作为验证集,
制作了一个小型的道路提取数据集[MiniDeepGlobeRoadExtraction](https://paddleseg.bj.bcebos.com/dataset/MiniDeepGlobeRoadExtraction.zip)。
运行训练脚本将自动下载该数据集。

* Lovasz loss训练
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
--config configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k_lovasz_hinge.yml \
--use_vdl --num_workers 3 --do_eval
```

* Cross entropy loss训练
```shell
CUDA_VISIBLE_DEVICES=0,1,2,3 python -u -m paddle.distributed.launch train.py \
--config configs/ocrnet/ocrnet_hrnetw18_road_extraction_768x768_15k.yml \
--use_vdl --num_workers 3 --do_eval
```

* 结果比较

实验mIoU曲线如下图所示。
<p align="center">
<img src="./images/Lovasz_Hinge_Evaluate_mIoU.png" hspace='10'/> <br />
</p>


|Loss|best mIoU|
|-|-|
|cross entropy loss|78.69%|
|lovasz softmax loss + cross entropy loss|79.18%|

图中紫色曲线为lovasz hinge loss + cross entropy loss,蓝色曲线为cross entropy loss,相比提升0.5个百分点。

可看出使用lovasz hinge loss后,精度曲线全面高于原来的精度。



## 参考文献
[Berman M, Rannen Triki A, Blaschko M B. The lovász-softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 4413-4421.](http://openaccess.thecvf.com/content_cvpr_2018/html/Berman_The_LovaSz-Softmax_Loss_CVPR_2018_paper.html)
4 changes: 3 additions & 1 deletion paddleseg/cvlibs/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -211,7 +211,9 @@ def loss(self) -> dict:
if key == 'types':
self._losses['types'] = []
for item in args['types']:
item['ignore_index'] = self.train_dataset.ignore_index
if item['type'] != 'MixedLoss':
item['ignore_index'] = \
self.train_dataset.ignore_index
self._losses['types'].append(self._load_object(item))
else:
self._losses[key] = val
Expand Down
1 change: 1 addition & 0 deletions paddleseg/datasets/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,4 @@
from .ade import ADE20K
from .optic_disc_seg import OpticDiscSeg
from .pascal_context import PascalContext
from .mini_deep_globe_road_extraction import MiniDeepGlobeRoadExtraction
Loading

0 comments on commit 038cde8

Please sign in to comment.