-
-
Notifications
You must be signed in to change notification settings - Fork 609
GSoC 2022 project
Google Summer of Code 2022 project
There few basic requirements for new contributors. They have to accomplish the following steps:
- read project's contributing guide and setup development environment
- learn the concepts of the project
- run some of our examples
To be considered as a GSoC student, it is necessary to have at least 3 accepted PRs to the project. They can be something small like a doc fix or a simple bug fix. Please, see help wanted issues.
- Improve Metrics module (175 hours)
- Examples and integrations (175 hours)
Metrics module in PyTorch-Ignite proposes very unique features to evaluate a trained PyTorch model in an online manner and for any type of computation settings (single process or distributed configuration). Namely, the module provides essential metrics for classification/segmentation tasks and various regression metrics.
The plan for the improvements contains the following topics:
Note: task complexity is indicated as easy, medium, hard.
Coding tasks:
-
(medium) All metrics should work in distributed configuration (#1284)
- Remains: Improve
EpochMetric
to correctly gather the output (see #1758) and all related metrics: PrecisionRecallCurve, AveragePrecision. - Explore asymmetric distributed metrics computation: PyTorch uneven distributed input support
- Remains: Improve
-
(medium) Provide configurable distributed metrics reduce/gather methods (#1288 #1242)
-
(hard) Provide new metrics for object detection task: implement mean Average Precision metric
- Implement the metric and its tests
-
(easy) Add minor improvements:
- better support of sklearn metrics
- classification metrics with micro/macro options
-
(medium) Work on stateful handlers (#966)
- Implement a base class for stateful handlers
- Improve
RunningAverage
metric to be able to restore internal values from a checkpoint - Improve existing handlers that require a state
Redesign tasks:
-
(medium) Work on enabling label-wise metrics (Accuracy etc.) for multi-label problems (#513)
- Prototype new API to add label-wise option for multi-label metrics
- Implement chosen API and implement the tests
-
(medium) Redesign metric classes to fix issues:
- TODO: explain what to do.
RunningAverage
andFrequency
metrics #1405
- TODO: explain what to do.
-
(medium) Output type (e.g. logits or probas) for mertrics input
- Related issue and PR #1354
We would expect at least 5 medium items from the above list to be done for this project. In addition, it would be nice to have a short blog post communicating about the work done.
- Intermediate level for Python and Git
- Intermediate knowledge of PyTorch
- Beginner knowledge of PyTorch-Ignite codebase
- Fluent with Python and Git
- Already trained neural networks with PyTorch
- Willing to maintain AI-related open-source project
- Curiosity and motivation to learn new technical things
4 / 5 or medium
175 hours
According to our Roadmap, we would like to provide more integrations with other tools to simplify Machine/Deep Learning end-to-end applications.
In this project we would like to provide more examples of PyTorch-Ignite using other tools bringing new features or insights, e.g. federated learning, interpretability, hyper-parameter tuning, etc.
-
(medium) Code an example for federated learning with PySift or any other equivalent tool (openfl, ...). #1378
-
(medium) Code an example of hyper-parameters tuning using Ray tune, #1735
-
(medium) Code an example using ffcv : https://ffcv.io/
-
(medium) Code an example using zenml or a similar tool
-
(hard) Port Detectron2 training script and use ignite instead of custom trainer.
-
(hard) Define a scope and code an example using pytorch Distributed RPC framework.
We can discuss and add other integrations during the GSoC program if needed.
We may decide later about example format: a tutorial or plain script or a template to our code-generator. Writing tutorials may be more time expensive but better for visibilty. Minimal requirement is a python script. Providing a code-generator template is desirable.
We would expect at least 4 medium items (hard item == 2 medium) from the above list to be done for this project. In addition, it would be nice to have a short blog post communicating about the work done.
- Intermediate level for Python and Git
- Intermediate knowledge of PyTorch
- Beginner knowledge of PyTorch-Ignite codebase
- Fluent with Python
- Already trained neural networks with PyTorch
- Willing to maintain AI-related open-source project
- Curiosity and motivation to learn new technical things
4 / 5 or medium
175 hours
PyTorch-Ignite presented to you with love by PyTorch community