Original classes baseline #23

lucamarini22 · 2023-06-13T15:21:59Z

I created a branch to train a model on the original 11 classes used in MARIDA: A benchmark for Marine Debris detection from Sentinel-2 remote sensing data, and also with the original 15 classes of the dataset. Now it is possible to train with:

15 classes if the arg --aggregate_classes = CategoryAggregation.ALL
11 classes if the arg --aggregate_classes = CategoryAggregation.ELEVEN
5 classes if the arg --aggregate_classes = CategoryAggregation.MULTI
2 classes if the arg --aggregate_classes = CategoryAggregation.BINARY

The classes that correspond to each of the above configuration ca be seen in the assets.py file

…asses used in the marida paper

gomezzz

some smaller comments

evaluation.py

marineanomalydetection/dataset/aggregator.py

marineanomalydetection/dataset/marineanomalydataset.py

marineanomalydetection/dataset/update_class_distribution.py

marineanomalydetection/parse_args_train.py

marineanomalydetection/utils/assets.py

gomezzz · 2023-06-16T12:08:09Z

Also, probably you will want to merge this into main not the other branch, I suspect? 🤔

gomezzz

I presume it's ok but I can't really give feedback because I don't know the intent of the changes. Would advocate for shorter-lived, smaller PRs with clear relationship between

<what_was_the_goal> <-> <what_changed_in_the_code>

(only looked at changes since last review given size ✌️)

marineanomalydetection/dataset/update_class_distribution.py

…n evaluating on test set

…hange intensity only as first augmentation (to avoid changing intensity of padding pixels). Remove the application of augmentations that change intensity of pixels to pseudo labels.

…ing sets setting. Still todo for one training set setting

… training as padding pixles

…works with 0.0 because on the logits of the strongly aug-img I ignore I ignore the padding pixels based on the logits of the weakly aug-img, which has not cutout applied

…Focal loss in the unsup component of the ssl loss. Update descriptions.

…ility of using Cross Entropy in the unsupervised component of the ssl loss

… the loss when using semi-supervised learning

Fixes

Grad lu fix

Softmax and augmentations fixes

Small fixes

Channel importance

Log val miou

lucamarini22

review is ok

…aseline

lucamarini22

ok

lucamarini22 added 5 commits June 13, 2023 12:15

add method to aggregate classes (in segmentation maps) into the 11 cl…

b6660c0

…asses used in the marida paper

add category aggregation enum with 11 classes

4c9c659

add option to evaluate with 11 classes aggregation

60ac41b

add 11 classes (original from marida paper) option

b3a3cac

add 11 classes labels, mapping, and count of labeled pixels

b8e6a4a

lucamarini22 assigned gomezzz Jun 13, 2023

lucamarini22 added the enhancement New feature or request label Jun 13, 2023

gomezzz removed their assignment Jun 16, 2023

gomezzz self-requested a review June 16, 2023 11:13

gomezzz requested changes Jun 16, 2023

View reviewed changes

lucamarini22 changed the base branch from ssl-and-fully-sup-same-batches to main July 16, 2023 07:46

add comment and create labels_11 from labels

0c2a78c

lucamarini22 mentioned this pull request Jul 16, 2023

One thing to think about in general maybe if you want your code at all to be usable with other datasets. Because it seems to me you are writing it in a way that will make it extremely difficult to ever use any other dataset? 🤔 #28

Open

lucamarini22 added 3 commits July 16, 2023 09:53

add category aggregation to train with all 15 classes of MARIDA

65015a1

remove param

47acf0b

remove

6d8ec0d

lucamarini22 requested a review from gomezzz July 16, 2023 10:52

gomezzz reviewed Jul 17, 2023

View reviewed changes

marineanomalydetection/dataset/update_class_distribution.py Outdated Show resolved Hide resolved

lucamarini22 added 9 commits July 24, 2023 20:18

log max val miou and the epoch at which it was reached

82d0670

create functions to avoid repetition of code

265a2dd

add config for one single train set config

7094c2b

add default config file for fully-sup setting

c9815c5

add default config for two train set setting

3effca9

add the option to test channel importance by zero-ing one channel whe…

fb07d41

…n evaluating on test set

use is instead of ==

f5a390e

add method to convert tif img store in array to rgb

0d11126

remove unused file io

2ec3ef4

lucamarini22 and others added 24 commits August 18, 2023 10:08

update

785ea2f

add notebook to show and save a MARIDA patch and its sem seg map

5cfc392

update notebook

36fbf07

fix parsing of "mode"

1cad442

add init

7e5572d

fix channel idx where softmax is computed. Apply augmentations that c…

d405542

…hange intensity only as first augmentation (to avoid changing intensity of padding pixels). Remove the application of augmentations that change intensity of pixels to pseudo labels.

update

4be0af2

remove detach of logits_u_s to keep grad consistent. Did for tw train…

5c56127

…ing sets setting. Still todo for one training set setting

wrap loss in tensor when it's equal to zero

f113391

set CutOut val = 0.1 instead of 0.0, otherwise it gets ignored during…

40c9b79

… training as padding pixles

update

72135a0

add code to visually debug images + re-set CutOut to 0.0. Indeed it …

bd64971

…works with 0.0 because on the logits of the strongly aug-img I ignore I ignore the padding pixels based on the logits of the weakly aug-img, which has not cutout applied

fix grad Lu also for the one training set setting

f5ca2f1

add flag "use_ce_in_unsup_component" to use Cross Entropy instead of …

bf0ca27

…Focal loss in the unsup component of the ssl loss. Update descriptions.

add drop_last param in all dataloaders and set it to True

6f708e1

add function to visually debug images and predictions, and add possib…

a2b73e8

…ility of using Cross Entropy in the unsupervised component of the ssl loss

update

4518c23

add only_supervised flag to train only on the supervised component of…

bf091a6

… the loss when using semi-supervised learning

Merge pull request #29 from lucamarini22/fixes

b7ede57

Fixes

Merge pull request #30 from lucamarini22/grad-Lu-fix

ce39cb8

Grad lu fix

Merge pull request #31 from lucamarini22/softmax-and-augmentations-fixes

e0c6288

Softmax and augmentations fixes

Merge pull request #32 from lucamarini22/small-fixes

27ab78d

Small fixes

Merge pull request #33 from lucamarini22/channel-importance

a766534

Channel importance

Merge pull request #34 from lucamarini22/log_val_miou

0740be2

Log val miou

lucamarini22 changed the base branch from main to ssl-and-fully-sup-same-batches October 4, 2023 15:21

lucamarini22 commented Oct 4, 2023

View reviewed changes

Merge branch 'ssl-and-fully-sup-same-batches' into original-classes-b…

c24745e

…aseline

lucamarini22 commented Oct 4, 2023

View reviewed changes

lucamarini22 merged commit 22be673 into ssl-and-fully-sup-same-batches Oct 4, 2023

lucamarini22 deleted the original-classes-baseline branch October 4, 2023 15:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Original classes baseline #23

Original classes baseline #23

lucamarini22 commented Jun 13, 2023 •

edited

Loading

gomezzz left a comment

gomezzz commented Jun 16, 2023

gomezzz left a comment

lucamarini22 left a comment

lucamarini22 left a comment

Original classes baseline #23

Original classes baseline #23

Conversation

lucamarini22 commented Jun 13, 2023 • edited Loading

gomezzz left a comment

Choose a reason for hiding this comment

gomezzz commented Jun 16, 2023

gomezzz left a comment

Choose a reason for hiding this comment

lucamarini22 left a comment

Choose a reason for hiding this comment

lucamarini22 left a comment

Choose a reason for hiding this comment

lucamarini22 commented Jun 13, 2023 •

edited

Loading