Skip to content

Commit

Permalink
Added Gsoc Expcted Outcome
Browse files Browse the repository at this point in the history
Signed-off-by: Ayush Gupta <guptaayush9040@gmail.com>
  • Loading branch information
Ayush9026 committed Feb 21, 2024
1 parent 6a26e5f commit bf763f8
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions content/en/events/gsoc-2024.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ You would be part of the larger effort, and involved in one or more code deliver
* See Kubeflow Notebooks 2.0 GitHub proposal: [https://github.com/kubeflow/kubeflow/issues/7156](https://github.com/kubeflow/kubeflow/issues/7156)
* See Kubeflow Notebooks 2.0 design document: [https://docs.google.com/document/d/1_zk06zebbaTBdJ8TdU07Ibky25hqHGARXjVcsp2qEnU/edit](https://docs.google.com/document/d/1_zk06zebbaTBdJ8TdU07Ibky25hqHGARXjVcsp2qEnU/edit)


**Expected Outcome:** The expected outcome of this project is to overhaul Kubeflow Notebooks to allow for easier configuration and management of workspace environments, enhancing user experience and administrative control within Kubernetes clusters.

**Skills required:** Kubernetes Controllers (Golang - Kubebuilder) AND/OR Web Development (JS - Angular, Python - Flask)

**Difficulty:** medium/high
Expand All @@ -65,6 +68,9 @@ You would be involved in testing and investigating the impacts of these changes,

See the proposal for more information: [https://github.com/kubeflow/manifests/blob/master/proposals/20200913-rootlessKubeflow.md](https://github.com/kubeflow/manifests/blob/master/proposals/20200913-rootlessKubeflow.md)


**Expected Outcome:** The project aims to integrate rootless Kubeflow container images with Istio Ambient Mesh, reducing reliance on privileged containers for network permissions and enhancing security in Kubeflow deployments.

**Skills required:** Istio, Kubernetes, YAML

**Difficulty:** medium
Expand All @@ -90,6 +96,9 @@ Specifically, your goal would be to:
* [https://github.com/kubeflow/manifests/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22](https://github.com/kubeflow/manifests/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)
* Review which PRs are likely safe to merge (especially dependabot ones)


**Expected Outcome:** The project aims to streamline Kubeflow GitHub issue and pull request management by triaging, categorizing, and prioritizing, while facilitating new contributor engagement through "good first issue" identification and ensuring the safe merging of pull requests, especially dependabot updates.

**Skills required:** GitHub, Kubernetes, YAML, Python, GO, JS

**Difficulty:** medium
Expand All @@ -107,6 +116,8 @@ Recently, we implemented [a new `train` Python SDK API](https://github.com/kubef

To continue our roadmap around LLMOps in Kubeflow, we want to give user functionality to tune HyperParameters of LLMs using simple Python SDK APIs. It requires making appropriate changes to the Katib Python SDK which allows users to set model, dataset, and HyperParameters that they want to optimize for LLM.

**Expected Outcome:** The expected outcome is the implementation of a Python SDK API within Katib, enabling users to fine-tune HyperParameters for Large Language Models (LLMs) seamlessly. This enhancement will empower users to optimize LLM performance efficiently by specifying model configurations, datasets, and desired HyperParameters through simplified Python SDK interactions.


**Skills required:** Kubernetes, YAML, Python

Expand All @@ -125,6 +136,8 @@ Open issue: [https://github.com/kubeflow/training-operator/issues/1619](https://

We want to integrate Jax in Training Operator to run distributed training and fine-tuning jobs on Kubernetes using the Jax ML framework. We need to create a new Kubernetes Custom Resource for Jax (e.g. JaxJob) and update the Training Operator controller to support it. Potentially, we can integrate Jax with the Training Operator Python SDK to give Data Scientists simple APIs to create JaxJob on Kubernetes.

**Expected Outcome:**The project aims to enable distributed training and fine-tuning using the Jax ML framework within the Training Operator on Kubernetes, facilitated by the creation of a new JaxJob Custom Resource and updates to the Training Operator controller, potentially supplemented with simplified Python SDK APIs for user convenience.

**Skills required:** Kubernetes, Go, YAML, Python

**Difficulty:** medium
Expand All @@ -144,6 +157,7 @@ Katib implements Metrics Collector as a sidecar container to collect training me

Sometimes the container sidecar approach might not work for users. For example, if their Trial resources executor doesn’t support sidecar containers. For such use-cases, we want to implement a new API to the Katib Python SDK to allow users to push metrics directly from their training scripts to the Katib DB.

**Expected Outcome:** The project aims to develop a new API for the Katib Python SDK, enabling direct metric push from training scripts to the Katib database, offering users an alternative to the sidecar container method, thus enhancing flexibility and compatibility in metrics collection.

**Skills required:** Kubernetes, Go, YAML, Python

Expand All @@ -162,6 +176,8 @@ Open issue: [https://github.com/kubeflow/katib/issues/2081](https://github.com/k

Training Operator and Katib SDKs have [a valid docstring](https://github.com/kubeflow/training-operator/blob/0b6a30cd348e101506b53a1a176e4a7aec6e9f09/sdk/python/kubeflow/training/api/training_client.py#L50-L74) for each public API that users are running. We want to automatically generate documentation for Kubeflow users from these docstrings, so users don’t need to read source code to understand APIs parameters.

**Expected Outcome:** The project aims to automate the generation of documentation for Kubeflow Python SDKs by extracting information from existing docstrings in Training Operator and Katib SDKs, streamlining API parameter understanding for users without the need to inspect source code.

**Skills required:** Python

**Difficulty**: medium
Expand All @@ -179,6 +195,8 @@ Open issue: [https://github.com/kubeflow/katib/pull/2059](https://github.com/kub

We need to enhance Katib Experiment APIs to support various parameter distributions like uniform, log-uniform, qlog-uniform to make Katib more native to other HyperParameter tuning frameworks like Hyperopt. Currently, Katib supports only uniform distribution of integer, float, and categorical HyperParameters.

**Expected Outcome:** This project seeks to extend Katib Experiment APIs to support a range of parameter distributions including uniform, log-uniform, and qlog-uniform, aligning Katib with other HyperParameter tuning frameworks like Hyperopt for enhanced versatility and efficacy in model optimization.

**Skills required:** Kubernetes, Python, Go, YAML

**Difficulty:** medium
Expand All @@ -198,6 +216,8 @@ Kubeflow Pipelines must store information about pipelines, experiments, runs, an

We plan to support PostgreSQL as an alternative to MySQL/MariaDB so users will be able to reuse existing databases, and PostgreSQL will be a good use case for supporting multiple databases.

**Expected Outcome:** The project endeavors to incorporate PostgreSQL compatibility into Kubeflow Pipelines, offering users greater database choice and enabling seamless integration with existing infrastructure, thereby bolstering scalability and adaptability across varied environments.

**Skills required:** Kubernetes, Python, Go, YAML

**Difficulty:** medium
Expand All @@ -213,6 +233,8 @@ We plan to support PostgreSQL as an alternative to MySQL/MariaDB so users will b

We aim to extend the capabilities of the KF Model Registry Python client by enabling smooth imports from various machine learning registries. While import from HuggingFace is already implemented (and can be used as a basis) we seek to integrate support for MLFlow, and other popular registry formats.

**Expected Outcome:** This project seeks to enrich the functionality of the KF Model Registry Python client, enabling effortless ML imports from a range of registries beyond HuggingFace. By incorporating support for formats like MLFlow and others, it aims to enhance the versatility of Kubeflow's Model Registry, fostering greater ease and flexibility in ML model management.

**Skills required:** Python, ML model serialization formats, YAML, Kubernetes/Kubeflow as a plus

**Difficulty:** medium
Expand Down

0 comments on commit bf763f8

Please sign in to comment.