Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Py torch lightning master #2

Merged
merged 80 commits into from
Aug 23, 2021
Merged

Conversation

KonstantinKorotaev
Copy link
Owner

What does this PR do?

Fixes # (issue)

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

akihironitta and others added 30 commits July 8, 2021 09:59
* Fix pytorch link

* typo

* Fix section title

* typo

* typo

* Ignore data dir

* Update docs

* Update docs/source/general/data.rst
* Update swagger UI image

* Move assets to S3
* Fix metric computation

* Update CHANGELOG.md

* Fixes

* Fixes
* Initial commit

* Add support for from_csv and from_data_frame to ImageClassificationData

* Update CHANGELOG.md

* Fixes

* Clean
* Try fix

* Temp enable CI

* Try fix

* Fixes

* Try fix

* Try fixes

* Fixes

* Update

* Update

* Updates

* Revert temp enable CI
remove redefined object - `STYLE_TRANSFER_BACKBONES`
* Factored out ROGUE and BLEU metrics

* Changed docs references

* Updates

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
* Created QuestionAnsweringData and QuestionAnsweringPreprocess

* Added tests for the QuestionAnsweringData class

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
* restructured pretrained weights flag for ImageClassifier

* changelog

* changelog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* updated PR

* rebase

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* formatting

* Format code with autopep8

* formatting

* formatting

* removed temp code from example

* removed temp code from example

* removed temp code from example

* tests

* Format code with autopep8

* tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
…ghtning-Universe#567)

* Adding QuestionAnsweringTask class to the question answering task

* Small changes based on pep8 guidelines

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
…#574)

* added TabularClassificationData,TabularRegressionData extending TabularData

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update flash/tabular/regression/data.py

Co-authored-by: thomas chaton <thomas@grid.ai>

* Update flash/tabular/classification/data.py

Co-authored-by: thomas chaton <thomas@grid.ai>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added TabularClassificationData,TabularRegressionData extending TabularData

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* PEP8 fix

* modified tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas chaton <thomas@grid.ai>
* init smp integration 🚀

* fix backbone & head

* backbone/heads backward compatibility

* update

* update

* move ENCODERS to bottom

* update self.encoder

* remove lrsapp

* update tests ✅

* fix model tests

* update

* Fixes

* Update CHANGELOG.md

Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
* Add flash to docs

* Add flash.core.serve to docs

* Add todo extension

* Fix format

* Add /data to .gitignore

* Move tutorials out of API Reference

* Add noindex

* Exclude flash.* members from data API docs

* Update flash.core

* Add docstring to FlashCallback

* Remove callback note

* Remove flash.core.data_module.DataModule in favor of flash.DataModule

* Add missing docstring to Trainer's methods

* Update flash api ref

* Update flash.core api ref

* Don't remove docs/source/api

* api/{flash,core,data,image}

* api/{image,text,video,tabular,serve}

* udpate

* make clean before make docs

* Remove :ref: for now

* resolve bad-looking table

* Split classes and functions

* Shorten paths to class/func

* Update docs/source/_static/main.css

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* Fix docs build

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
* Add option for nested tasks

* Update CHANGELOG.md

* Update CHANGELOG.md

* Updates

* Add grandparent test
* restructured pretrained weights flag for ImageClassifier

* changelog

* changelog

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* updated PR

* rebase

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* formatting

* Format code with autopep8

* formatting

* formatting

* removed temp code from example

* removed temp code from example

* removed temp code from example

* tests

* Format code with autopep8

* tests

* fixed loading state dict to models for pretrained flag

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
* Create datatype_audio.txt

* Update setup.py

* Update imports.py

* Update imports.py

* Update README.md

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: thomas chaton <thomas@grid.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…s fully optional (Lightning-Universe#583)

* Initial commit

* Move out matplotlib and fiftyone

* Updates

* Update tests

* Try fix

* Fixes

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix
* Initial structure of GraphClassification model.py

* Improvement of model.py. Still need to debug etc

* BasicDataset Implemented

* Create __init__.py

* Implemented dataset and DataModule as for image processing

Lacking Pipeline and it is possible that division in raw and processed folders might be needed.

* Pipeline taken from images.

I'm unsure how to adapt

* Initial structure of GraphClassification model.py

* Improvement of model.py. Still need to debug etc

* BasicDataset Implemented

* Implemented dataset and DataModule as for image processing

Lacking Pipeline and it is possible that division in raw and processed folders might be needed.

* Pipeline taken from images.

I'm unsure how to adapt

* Choice of model implemented (you can pass a model to GraphClassifier)

The class BasicGraphDataset in graphClassification/data.py is probably unneded

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Initial readaptation of the structure

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minimal structure of how to structure data.py files

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor corrections

* update

* i

* update

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Added auto_dataset.num_features

* Deleted manually included num_features so that it is extracted from GraphDatasetSource()

* Test for GraphClassification implemented

* Documentation for GraphClassification included

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Creation of from_pygdatasequence method in DataModule and GraphSequenceDataSource()

* Update graph_classification.py

* Update datatype_graph.txt

* Tests and docs for the from_pygdatasequence method

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Graph requirements

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update CHANGELOG.md

* Update requirements with pytorch geometric libraries

* Simplified, version with only the DataSource

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor tweaks

* Update the flash_example to reflect the new template

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Delete IMDB-BINARY_A.txt

* Delete IMDB-BINARY_graph_indicator.txt

* Delete IMDB-BINARY_graph_labels.txt

* Class method from_pygdatasequence from flash/core/data/data_module.py

* Update docs

* fix imports.py

* remove unused imports

* clean init.py

* updates

* Updates

* Updates

* Updates

* Updates

* Update docs

* Update docs

* Update docs

* fix tests

* fix tests

* Add API reference

* Try fix

* Try fix

* Try fix

* Update flash/core/data/auto_dataset.py

* Update docstring

Co-authored-by: pablo <pablo@MacBook-Pro-de-pablo-2.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: tchaton <thomas@grid.ai>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
* update

* wip

* update

* update

* update

* resolve issues

* update

* update

* add doc

* update

* add tests

* update

* update tests

* update on comments

* update

* update

* resolve some bugs

* remove breakpoint

* Update docs/source/api/pointcloud.rst

* update

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
* add weights path

* add available weights

* remove weight path

* add tests ✅

* fix

* update

* add str pretrained

* add test ✅

* fix

* Update flash/image/segmentation/heads.py

* Update CHANGELOG.md

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Lightning-Universe#585)

* Added field parameter to the from_json method with other required changes.

* Updating field parameter type and CHANGELOG

* Added docs for the new parameter

* Add some tests

* Update flash/core/data/data_module.py

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
* Fixed a bug where drop_last=True for testing

* Fix a test
Co-authored-by: thomas chaton <thomas@grid.ai>
* Fix point cloud tests

* Add backbone tests

* Updates

* Add test for datasets

* Updates
Replaced available_models in docs/source/general/registry.rst with available_keys
* wip

* wip

* wip

* add tests

* add docs

* update changelog

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* Update tests/pointcloud/detection/test_data.py

* Apply suggestions from code review

* Update tests/pointcloud/detection/test_data.py

* Update tests/pointcloud/detection/test_data.py

* Update tests/pointcloud/detection/test_data.py

* Update tests/pointcloud/detection/test_data.py

* resolve test

* Update tests/pointcloud/detection/test_data.py

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
…erse#594)

* added audio spectrogram classification data, transforms and tests based on image classification

* added audio spectrogram classification data, transforms and tests based on image classification

* added audio spectrogram classification example and notebook

* fixed formatting issues about newlines and longlines

* updated docs to include audio classification task

* removed empty `model` package

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updates

* Update CHANGELOG.md

* Updates

* Updates

* Try fix

* Updates

* Updates

* Updates

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
* Fix docs

* Fixes

* Fixes

* Fixes
* Base files for wav2vec integration

* Format code with autopep8

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Closer to working

* Format code with autopep8

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Refactors

* Refactors

* Cleanups

* Refactor to allow files

* Get predictions working

* Add licence

* Fix loads

* Add check

* Fix imports

* Cleanups

* Add backbone API

* Cleanups

* Fix

* Add tests

* Docs, requirements

* topic thing

* Doc fix

* test

* Add serve

* Fix path

* Swap to audio available

* Small fix

* Some fixes

* Small fix

* Small fix

* Fix

* Updates

* Fix docs

* Remove duplicate

* Add check for audio

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
ethanwharris and others added 29 commits August 13, 2021 22:42
* Initial commit

* Fixes

* Drop asteroid

* Drop asteroid

* Try fix

* Speed improvements

* Updates

* Fixes

* Updates

* Updates

* Updates

* Updates

* Updates

* Fixes

* Debug

* Debug

* Fixes

* Fixes

* Docstrings

* Fixes

* Fixes

* CHANGELOG.md
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Try to debug docs build

* Try something

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try fix

* Try something

* Try something

* Try something

* Try fix

* Try fix

* Try fix

* Try fix

* Updates

* Updates

* Drop

* Revert
* , gpus=-1

* torch.cuda.device_count()

* .

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* .

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Initial commit

* Add instance segmentation and keypoint detection tasks

* Updates

* Updates

* Updates

* Add docs

* Update API reference

* Fix some tests

* Small fix

* Drop failing JIT test

* Updates

* Updates

* Fix a test

* Initial credits support

* Credit -> provider

* Update available backbones

* Add adapter

* Fix a test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Updates

* Fixes

* Refactor

* Refactor

* Refactor

* minor changes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* 0.5.0dev

* pl

* imports

* Update adapter.py

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update adapter.py

* Updates

* Add transforms to and from icevision records

* Fix tests

* Try fix

* Update CHANGELOG.md

* Fix tests

* Fix a test

* Try fix

* Try fix

* Add some docs

* Add API reference

* Small updates

* pep fix

* Fixes

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
…#666)

Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
* codeowners update

* .

* Update CODEOWNERS

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
…ng-Universe#665)

* fix prints, remove embedding_dim

* undo example since print is fine there

Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
* read_image to default_loader

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* imports

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix drop last for predicting and testing

* Update CHANGELOG.md

* Update CHANGELOG.md

* Fixes
…rse#667)

* Add torch ORT support, move transformer Tasks to use general task class

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix import

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update transformers version

* Revert

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Revert

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add tests

* Add tests

* fix

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add docs for text classification and translation

* Add note

* Add CHANGELOG.md

* Address code review

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply suggestions from code review

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
…ning-Universe#672)

* Add instance segmentation and keypoint detection to flash zero

* Add instance segmentation and keypoint detection to flash zero

* Add docs

* Uodate CHANGELOG.md

* Fixes
* updated mock object

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added property to cls variable

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor PR

* merge

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add in_chans arg to resnet

* Update CHANGELOG.md
* added lars, lamb, warmup+decay

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added exports

* tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pep8

* test for scheduler

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pep8

* added types

* .

* chaged tests format

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add some providers

* Updates

* Updates

* Add speech recognition

* Updates

* Add providers list to docs

* Add sorting
* added lars, lamb, warmup+decay

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* added exports

* tests

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pep8

* test for scheduler

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* pep8

* added types

* .

* chaged tests format

* added API reference

* added API reference

* docs

* docs

* added tests for optimizers

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Make tests run

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ethan Harris <ethanwharris@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
@KonstantinKorotaev KonstantinKorotaev merged commit 9a0d919 into master Aug 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.