Adding doc page for the object detection task #20874

MKhalusova · 2022-12-22T15:08:37Z

This is a PR for the #20805 issue.
The guide has content and working code examples for:

Introduction
Loading CPPE-5 dataset from Hub
Preprocessing both images and annotations. Images are augmented, annotations are reformatted to be in the format DETR expects
Training with Trainer
Evaluation
Inference

HuggingFaceDocBuilderDev · 2022-12-22T15:25:59Z

The documentation is not available anymore as the PR was closed or merged.

MKhalusova · 2022-12-28T13:58:12Z

If I missed someone who has to be invited as a reviewer, please feel free to add them.

sayakpaul · 2022-12-28T14:21:03Z

@MKhalusova thanks for doing this! I will take a look tomorrow my time.

I think you can follow the instructions noted here to resolve the quality bug in the CI. Let me know if anything's unclear.

* Fix incorrect import for the base optimizer for AdamWeightDecay * Fix incorrect import for the base optimizer for AdamWeightDecay

remove unused use_cache in config classes Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

…gface#20279) * Add support for binary segmentation * Fix loss calculation and add test * Remove space * use fstring Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local> Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>

* clarify use_fast parameter * make style * remove check frameworks, apply review

* [S2T, Whisper] Add copied from statements * rebase and fix-copies

* embed circle packing chart * trim whitespace from bottom * explain bubble sizes

* add new model like * add v1 * v1 * v1 * vision encoder logits match * v2 * fix * add docstring * CI tests pass * fix tests * make fixup * add to `toctree` * fix processors * fix processors * fix doc * fill title * add content doc * remove from tokenization auto * fix config * change order * add `# Copied from` * few fixes - add correct license on modeling text - remove dummy argument * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * replace name * refactor a bit * more refactor * remove unused arg * make fixup + remove some `# Adapted from ...` * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * more `# Copied from` * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * now `generate` supports no prefix * remove `FeatureExtractor` * fix path * correct dependency * fix tests * few fixes * add integration tests * add correct conversion script * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> * add `blip` to tokenization auto * fix docstrings * fix test + add image * remove processor from uncorrect place * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * clean up a bit * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * clean pixel mask * clean pixel mask * fix `F` * Update src/transformers/models/blip/modeling_blip.py Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * fix output * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * fix pad token id * remove `token_type_ids` * make fixup * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * make fixup * Apply suggestions from code review Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * add comments * Update src/transformers/models/blip/modeling_blip.py Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * remove `token_type_ids` * make fixup * better name * replace with `image_attention_mask` * refactor * make fixup * better docstring * replace `answer_xx` * remove ununsed args * add `labels` * add `labels` * fix processing tests * make fixup * make fixup * put correct repo * remove `pad` * remove `crop` and `center_crop` * Update src/transformers/models/blip/image_processing_blip.py Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * fix * remove `size_divisor` * fix weights `init` * remove unneeded functions * add suggestions * minor changes - change slow test output for PT 1.13 - docstring order * replace `feature_extractor` by `image_processor` * fix doctests * fix weight init order + add fp16 slow test * add `blip` to doctest * add correct repo name and fix test * Update src/transformers/models/blip/processing_blip.py Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> * fix tests * use `convert_to_rgb` from `image_transforms` * make fixup * fix large loading issue Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Fix doc tests * Use Auto API * Apply suggestion * Revert "Apply suggestion" This reverts commit cd9507a. Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local> Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>

Update big table Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>

) Use config.num_channels in CLIP-like modeling files Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

…on (huggingface#20621) * fix past_key_values in GPTNeoXForCausalLM.prepare_inputs_for_generation * fix formatting

Adds visual_prompt argument to CLIPSegProcessor to enable image-guided segmentation

…books (huggingface#20850) Adding `evaluate` to the list of libraries to be installed for every generated notebook in transformers

…ace#20859) * Fix past CI * Fix past CI Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

* fix typo `onnx` * fix test

poolformer onnx Co-authored-by: syed <syed.abdul@sandlogic.com>

* Supporting `fp16` for asr pipeline * Adding test. * Style. * Oops. * Flake8 update ? * Fixing flake8 ? * Revert "Flake8 update ?" This reverts commit 0b917fc. * Style (acctidentally deleted flake8 F401.) * Move to a bigger test (no small whisper model, and s2t doesn't seem to accept torch_dtype=fp16). Also we need to use a GPU to actually compute on fp16. * Using BatchFeature capability.

…uggingface#20801) * Add script to convert T5X T5 (v1.0 and v1.1) checkpoints to PyTorch * Remove unnecessary check and update docstring * Format docstring * Fix whitespace in docstring

* add japanese translation of template * fix japanese translation - fix special cases - fix typos - manually translate special cases Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com> Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

get correct checkpoint

* fix fp16 loading issue * add backward compatibility * better refactor * better readability - remove `force_upcast_dtype` as it is used once - use `inspect` - add `TODO`

Update flan-t5.mdx

fix Jupyter typos

* torch.jit._state * Fix past CI * Fix for perceiver * Fix REALM * Fix for Bloom * Fix for SwinMode * Fix for TrajectoryTransformerModel * Fix for test_wav2vec2_with_lm * make style Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

…ggingface#20837) * Avoid collisions in writing metrics via 2 APIs - azureml + mlflow MLflow tracking API is enabled by default in AzureML and HF MLflow integration is more fully featured. I'd remove the AzureML integration but leaving the current behavior for backwards compatibility (though it should really be removed) * Trigger CI

correctly detect default max length

MKhalusova · 2022-12-28T14:47:35Z

Did a rebase in an attempt to fix the CI issue. Accidentally added a whole bunch of unrelated commits to the PR. Figuring out how to remove them.

sayakpaul · 2022-12-28T15:06:23Z

Did a rebase in an attempt to fix the CI issue. Accidentally added a whole bunch of unrelated commits to the PR. Figuring out how to remove them.

You might want to revert to the previous commit. This thread might be helpful in that regard. And then from there:

Create a separate Python virtual environment.
Make sure you're in the virtual environment you just created and then from the transformers directory root run pip install -e .[quality].
Now, once the dependencies have been installed to the new virtual environment, run make style.

This should likely fix it. Since the code quality errors were previously coming from a doc page (c.f. https://app.circleci.com/pipelines/github/huggingface/transformers/54549/workflows/18b53122-1cc3-4c87-ad12-486853427500/jobs/657389), I suspect this to be stemming from the task page we're adding in this PR.

Let me know if anything is unclear.

This reverts commit ee2ad42.

This reverts commit d278bff.

MKhalusova · 2022-12-28T15:55:10Z

Closing this due to messed up rebase. The new PR is now here #20925

MKhalusova added 2 commits December 20, 2022 14:42

initial commit to add the how-to guide for object detection

aff8aae

Loading data, preprocessing and training sections completed

6a4bc61

MKhalusova mentioned this pull request Dec 22, 2022

Add Object Detection task tutorial to the transformers documentation #20805

Closed

MKhalusova added 4 commits December 22, 2022 13:28

Minor test polishing

3498af7

Added the inference section.

4a332fe

Fixed formatting in a code example

3d4e04e

Added the evaluation section

8d67be0

MKhalusova changed the title ~~[WIP] Adding doc page for the object detection task~~ Adding doc page for the object detection task Dec 28, 2022

MKhalusova requested review from NielsRogge, sgugger, stevhliu and sayakpaul December 28, 2022 13:56

MKhalusova and others added 15 commits December 28, 2022 09:24

Fixed a typo

ee2ad42

TF AdamWeightDecay fix for 2.11 (huggingface#20848)

2d917fe

* Fix incorrect import for the base optimizer for AdamWeightDecay * Fix incorrect import for the base optimizer for AdamWeightDecay

remove unused use_cache in config classes (huggingface#20844)

15d0c8e

remove unused use_cache in config classes Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

Clarify use_fast parameter in docstring (huggingface#20840)

a48e9b5

* clarify use_fast parameter * make style * remove check frameworks, apply review

[S2T, Whisper] Add copied from statements (huggingface#20787)

a9b8fd8

* [S2T, Whisper] Add copied from statements * rebase and fix-copies

Embed circle packing chart for model summary (huggingface#20791)

6af8488

* embed circle packing chart * trim whitespace from bottom * explain bubble sizes

[Swin2SR] Add doc tests (huggingface#20829)

30ca838

* Fix doc tests * Use Auto API * Apply suggestion * Revert "Apply suggestion" This reverts commit cd9507a. Co-authored-by: Niels Rogge <nielsrogge@Nielss-MacBook-Pro.local> Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>

[Examples] Update big table (huggingface#20845)

c1e3785

Update big table Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>

Use config.num_channels in CLIP-like modeling files (huggingface#20857

a3fc8cb

) Use config.num_channels in CLIP-like modeling files Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

fix past_key_values in GPTNeoXForCausalLM.prepare_inputs_for_generati…

6cb64a8

…on (huggingface#20621) * fix past_key_values in GPTNeoXForCausalLM.prepare_inputs_for_generation * fix formatting

Add visual prompt to processor of CLIPSeg model (huggingface#20816)

83aad6d

Adds visual_prompt argument to CLIPSegProcessor to enable image-guided segmentation

Adding evaluate to the list of libraries required in generated note…

aa69d64

…books (huggingface#20850) Adding `evaluate` to the list of libraries to be installed for every generated notebook in transformers

Fix past CI by skipping LevitModelTest.test_problem_types (huggingf…

a756683

…ace#20859) * Fix past CI * Fix past CI Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>

younesbelkada and others added 16 commits December 28, 2022 09:37

[MobileNet-v2] Fix ONNX typo (huggingface#20860)

f8a1f11

* fix typo `onnx` * fix test

having new model entries in Hindi for Hindi README (huggingface#20869)

35f33de

Add Onnx Config for PoolFormer (huggingface#20868)

2566674

poolformer onnx Co-authored-by: syed <syed.abdul@sandlogic.com>

Add script to convert T5X T5 (v1.0 and v1.1) checkpoints to PyTorch (h…

91a1a9d

…uggingface#20801) * Add script to convert T5X T5 (v1.0 and v1.1) checkpoints to PyTorch * Remove unnecessary check and update docstring * Format docstring * Fix whitespace in docstring

[RobertaPreLayernom] Fixes the CI daily test (huggingface#20886)

d8137f2

get correct checkpoint

Fixes typo in the help text for --max_length (huggingface#20883)

9a4205e

typo fix (huggingface#20891)

6a88b27

[ T5] fix fp16 loading issue (huggingface#20878)

ca4dd73

* fix fp16 loading issue * add backward compatibility * better refactor * better readability - remove `force_upcast_dtype` as it is used once - use `inspect` - add `TODO`

Update flan-t5 original model link (huggingface#20897)

354cde5

Update flan-t5.mdx

fix docs typos in "add_new_model" (huggingface#20900)

7e7cb40

fix Jupyter typos

Generate: correctly detect default max length (huggingface#20911)

ec6ba67

correctly detect default max length

Rebase and minor updates

d278bff

MKhalusova added 2 commits December 28, 2022 10:17

Revert "Fixed a typo"

05f2e8a

This reverts commit ee2ad42.

Revert "Rebase and minor updates"

560b643

This reverts commit d278bff.

MKhalusova removed request for sayakpaul, sgugger, NielsRogge and stevhliu December 28, 2022 15:32

anton-l changed the base branch from main to add_deformable_detr December 28, 2022 15:52

anton-l changed the base branch from add_deformable_detr to main December 28, 2022 15:52

MKhalusova mentioned this pull request Dec 28, 2022

Add: doc page for the object detection task #20925

Merged

MKhalusova closed this Dec 28, 2022

MKhalusova deleted the add-doc-task-obj-detection branch January 17, 2023 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding doc page for the object detection task #20874

Adding doc page for the object detection task #20874

MKhalusova commented Dec 22, 2022 •

edited

Loading

HuggingFaceDocBuilderDev commented Dec 22, 2022 •

edited

Loading

MKhalusova commented Dec 28, 2022

sayakpaul commented Dec 28, 2022 •

edited

Loading

MKhalusova commented Dec 28, 2022

sayakpaul commented Dec 28, 2022

MKhalusova commented Dec 28, 2022

Adding doc page for the object detection task #20874

Adding doc page for the object detection task #20874

Conversation

MKhalusova commented Dec 22, 2022 • edited Loading

HuggingFaceDocBuilderDev commented Dec 22, 2022 • edited Loading

MKhalusova commented Dec 28, 2022

sayakpaul commented Dec 28, 2022 • edited Loading

MKhalusova commented Dec 28, 2022

sayakpaul commented Dec 28, 2022

MKhalusova commented Dec 28, 2022

MKhalusova commented Dec 22, 2022 •

edited

Loading

HuggingFaceDocBuilderDev commented Dec 22, 2022 •

edited

Loading

sayakpaul commented Dec 28, 2022 •

edited

Loading