Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restructuring Kubeflow docs proposal #440

Closed
wants to merge 33 commits into from
Closed

Restructuring Kubeflow docs proposal #440

Changes from 3 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
2521041
[WIP] - Mapping Kubeflow website nav
rui-vas Oct 29, 2020
d7a757c
Finishing docs map
rui-vas Oct 29, 2020
1b91456
Adding link to discussion
rui-vas Oct 29, 2020
d906fdc
Adding Events Calendar (missed from map)
rui-vas Nov 16, 2020
c9d348b
About section refactor
rui-vas Nov 25, 2020
0dbe37f
Moving About Working Groups
rui-vas Dec 5, 2020
d6f4524
Remove "of Kubeflow"
rui-vas Dec 6, 2020
c221cbb
Move `Jupyter Notebooks` under `Components`
rui-vas Dec 6, 2020
54c4db8
Add "Kubeflow" to `Pipelines`
rui-vas Dec 6, 2020
d9ff2f6
Resources folder name to just "Resources"
rui-vas Dec 6, 2020
0ed158a
Split "Further Setup and Troubleshooting" into 2
rui-vas Dec 6, 2020
0e62402
fix - Google Summer of Code
rui-vas Dec 6, 2020
65dcbdb
Update restructure-KF-docs-proposal.md
rui-vas Dec 6, 2020
3e417d3
Move OpenShift up
rui-vas Dec 6, 2020
483ecf0
Add spacing
rui-vas Dec 7, 2020
ae12a53
Add "Deployment section"
rui-vas Dec 7, 2020
4f459dd
Moved all installation under `Deployment`
rui-vas Dec 7, 2020
38dff43
Update MiniKF naming as per request from @cspavlou
rui-vas Dec 10, 2020
2a03bc8
Move KF Pipelines to under Components
rui-vas Dec 10, 2020
99cb9a4
Update Katib docs to reflect v1.2 (current docs)
rui-vas Dec 10, 2020
a9d4905
Moving `Community` down
rui-vas Dec 10, 2020
eb3bd71
Rename Jupyter Notebooks to Notebook Servers
rui-vas Dec 14, 2020
feec189
Remove "Miscellaneous" with only 1 thing
rui-vas Dec 14, 2020
22d01b6
Rename "Setups" to "Config"
rui-vas Dec 14, 2020
737e8d9
Adding **Home**
rui-vas Dec 14, 2020
0cf3a60
Reordered Components in list + move multi-tenancy
rui-vas Dec 14, 2020
c87229a
Sections reorder
rui-vas Dec 14, 2020
a45552e
Create the overarching concept of **Tasks**
rui-vas Jan 13, 2021
bc3e48a
Convert from "tasks" to "platforms" & "methods"
rui-vas Jan 19, 2021
417b713
Small Fix
rui-vas Jan 19, 2021
bf9fbab
Update restructure-KF-docs-proposal.md
rui-vas Jan 21, 2021
40ce95d
Remove "Methods &"
rui-vas Jan 28, 2021
99b8663
Distributions -> Methods & Distributions
rui-vas Feb 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
226 changes: 226 additions & 0 deletions proposals/restructure-KF-docs-proposal.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
This is an initial proposal, to be iterated on, following the discussion in [#2293](https://github.com/kubeflow/website/issues/2293).
rui-vas marked this conversation as resolved.
Show resolved Hide resolved

### Kubeflow documentation structure

- **About**
rui-vas marked this conversation as resolved.
Show resolved Hide resolved
- Kubeflow
- Use Cases
- Contributing to Kubeflow
- Community
- Google Summer of Code
- Docs
- Style Guide for Kubeflow docs
rui-vas marked this conversation as resolved.
Show resolved Hide resolved

- **Getting Started**
- Kubeflow Overview
- Installing Kubeflow
- Cloud Installation
- AWS
- Azure
- Google Cloud
- IBM Cloud
- Kubernetes installation
- Overview of Deployment on Existing Clusters
- Kubeflow Deployment with kfctl_k8s_istio
- Multi-user, auth-enabled Kubeflow with kfctl_existing_arrikto
- Multi-user, auth-enabled Kubeflow with kfctl_istio_dex
- Workstation Installation
- Kubeflow on Linux
- Kubeflow on macOS
- Kubeflow on Windows
- MiniKF
- Deploy using MiniKF on GCP
- Deploying with minikube on a single node
- Kubeflow on MicroK8s
- **Components of Kubeflow**
- Central Dashboard
- Central Dashboard
- Registration Flow
- Metadata
- Jupyter Notebooks
- Fairing
- Overview of Kubeflow Fairing
- Install Kubeflow Fairing
- Configure Kubeflow Fairing
- Fairing on Azure
- Fairing on GCP
- Tutorials
- Reference
- Feature Store
- Introduction to Feast
- Getting started with Feast
- Frameworks for Training
- Chainer Training
- MPI Training
- MXNet Training
- PyTorch Training
- TensorFlow Training (TFJob)
- Hyperparameter Tuning
rui-vas marked this conversation as resolved.
Show resolved Hide resolved
- Introduction to Katib
- Getting started with Katib
- Running an experiment
- Katib Configuration Overview
- Environment Variables for Katib Components
- Pipelines
- Pipelines
- Tools for Serving
- Overview
- KFServing
- Seldon Core Serving
- BentoML
- NVIDIA Triton Inference Server
- TensorFlow Serving
- TensorFlow Batch Prediction
- Multi-Tenancy in Kubeflow
rui-vas marked this conversation as resolved.
Show resolved Hide resolved
- Introduction to Multi-user Isolation
- Design for Multi-user Isolation
- Getting Started with Multi-user Isolation
- Miscellaneous
- Nuclio functions
- **Jupyter Notebooks**
- Overview of Jupyter Notebooks in Kubeflow
- Set Up Your Notebooks
- Create a Custom Jupyter Image
- Submit Kubernetes Resources
- Build a Docker Image on GCP
- Troubleshooting Guide
- **Pipelines**
rui-vas marked this conversation as resolved.
Show resolved Hide resolved
- Pipelines Quickstart
- Installing Pipelines
- Installation Options for Kubeflow Pipelines
- Kubeflow Pipelines Standalone Deployment
- Deploying Kubeflow Pipelines on a local cluster
- Understanding Pipelines
- Overview of Kubeflow Pipelines
- Introduction to the Pipelines Interfaces
- Concepts
- Building Pipelines with the SDK
- Introduction to the Pipelines SDK
- Install the Kubeflow Pipelines SDK
- Build Components and Pipelines
- Create Reusable Components
- Building Python function-based components
- Best Practices for Designing Components
- Pipeline Parameters
- Python Based Visualizations
- Visualize Results in the Pipelines UI
- Pipeline Metrics
- DSL Static Type Checking
- DSL Recursion
- Using environment variables in pipelines
- GCP-specific Uses of the SDK
- Manipulate Kubernetes Resources as Part of a Pipeline
- Multi-user Isolation for Pipelines
- Caching
- Upgrading
- Samples and Tutorials
- Using the Kubeflow Pipelines Benchmark Scripts
- Using the Kubeflow Pipelines SDK
- Experiment with the Kubeflow Pipelines API
- Experiment with the Pipelines Samples
- Run a Cloud-specific Pipelines Tutorial
- Troubleshooting
- Reference
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pipelines has their own Reference.
Does it make sense to move Pipelines reference under /docs/reference where other projects are located ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that Pipelines and Fairing are the only components that have references outside of reference. Might make sense to merge everything.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comes back to the question of splitting component applications onto separate sites. In that case, each component application's documentation will need to be self-contained.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the conversations in the community meetings that I have attended, it sounds like we are moving towards separate sites for each component. Would it make sense to add a separate file for modeling out what that restructure would look like if we split the component docs onto other sites? Or, should we model that in this doc?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for Reference, if we move to the separate domains method, I would argue that we should keep a copy of reference on both, in the same manner as we currently do with manifests in the github org.

@joeliedtke happy with either. I guess none of this is set in stone. With this document I aim at identifying the quickest structural changes we can make to make the website more clear. We still need to rely on WG-leads to make sure their lower-level docs structure makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like I can't edit files in this PR, so here is a quick sketch of what I have been thinking about for information architecture of Kubeflow.org after the component docs move to their own site:

  • Kubeflow Home Page
  • Getting started (same as current proposal, consider renaming to About Kubeflow)
  • Community (same as current proposal)
  • Components
    • Jupyter Notebooks (Single page that describes what this component is and where to learn more)
    • Central Dashboard (Single page that describes what this component is and where to learn more)
    • Metadata (Single page that describes what this component is and where to learn more)
    • Fairing (Single page that describes what this component is and where to learn more)
    • Feature Store (Single page that describes what this component is and where to learn more)
    • Frameworks for training (Single page that describes what this component is and where to learn more)
    • Hyperparameter Tuning (Single page that describes what this component is and where to learn more)
    • Kubeflow Pipelines (Single page that describes what this component is and where to learn more)
    • Jupyter Notebooks (Single page that describes what this component is and where to learn more)
    • Tools for Serving (Single page that describes what this component is and where to learn more)
    • Multi-Tenancy (Single page that describes what this component is and where to learn more)
    • Nuclio functions (Single page that describes what this component is and where to learn more)
  • Docs
    • Deployment (same as current proposal)
    • Configuring Kubeflow (same as Setups in the current proposal)
    • Resources (Review content, if it is general to Kubeflow then keep this section)
    • Troubleshooting
    • Reference (Review content, if it is general to Kubeflow then keep this section)

The thought is to focus the Kubeflow.org site on describing the Kubeflow project, components, and deployment options. All of the docs for the component applications will be hosted on other sites.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @joeliedtke, thank you for the feedback!

Regarding whether docs are hosted on other sites, I think the downsides might be consistency, driving people away from Kubeflow.org and harder to make sure the websites are high quality. But despite that, I'm happy with what the community decides.

As per your suggestions:

  • I like the Kubeflow Home Page on the sidebar.
  • I think Docs is not a good word there as all are docs, but I see the point of merging those 5 things under an umbrella.
  • I also like Configuring instead of Setups

- Component Specification
- Pipelines API Reference
- Pipelines SDK Reference
- **Kubeflow on AWS**
- Deployment
- Install Kubeflow
- Uninstall Kubeflow
- Customizing Kubeflow on AWS
- AWS IAM Role for Service Account
- Logging
- Private Access
- Authentication and Authorization
- Authentication using OIDC
- Configure Kubeflow Pipelines on AWS
- Custom Domain
- Optimized Jupyter Notebooks on AWS
- Storage Options
- Configure External Database Using Amazon RDS
- Troubleshooting Deployments on Amazon EKS
- Kubeflow on AWS Features
- End-to-end Kubeflow on AWS
- **Kubeflow on Azure**
- Deployment
- Install Kubeflow
- Initial cluster setup for existing cluster
- Uninstall Kubeflow
- Authentication using OIDC in Azure
- End-to-End Pipeline Example on Azure
- Access Control for Azure Deployment
- Troubleshooting Deployments on Azure AKS
- **Kubeflow on GCP**
- Deploying Kubeflow
- Set up a Google Cloud Project
- Set up OAuth for Cloud IAP
- Management cluster setup
- Deploy using kubectl and kpt
- Monitor Cloud IAP Setup
- Delete using CLI
- Features of Kubeflow on GCP
- Deploy using UI
- Kubeflow On-premises on Anthos GKE
- Pipelines on GCP
- Connecting to Kubeflow Pipelines on Google Cloud using the SDK
- Authenticating Pipelines to GCP
- Upgrading
- Enabling GPU and TPU
- Using Preemptible VMs and GPUs on GCP
- Customizing Kubeflow on GKE
- Using Your Own Domain
- Authenticating Kubeflow to GCP
- Using Cloud Filestore
- Securing Your Clusters
- Troubleshooting Deployments on GKE
- Tutorial: End-to-end Kubeflow on GCP
- Logging and monitoring
- **Kubeflow on IBM Cloud**
- Deployment
- Install Kubeflow
- Securing the Kubeflow authentication with HTTPS
- Uninstall Kubeflow
- Create an IBM Cloud cluster
- Initial cluster setup for existing cluster
- Using IBM Cloud Container Registry (ICR)
- Pipelines on IBM Cloud Kubernetes Service (IKS)
- End-to-end Kubeflow on IBM Cloud
- **Kubeflow Operator**
Copy link
Member

@andreyvelich andreyvelich Oct 30, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubeflow operator is also part of Kubeflow components ?
Or we would say that kfctl is tool to deploy, monitor and manage the lifecycle of Kubeflow ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubeflow operator seems to be deployment tooling, loosely connected to kfctl, though under the same WG umbrella. It could fall into a "Lifecycle management" section or under "Getting started", as in effect it is an alternative deployment method for OpenShift.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with that @RFMVasconcelos.
My personal thought is that all Lifecycle management tools should be under the same website section.

- Introduction
- Installing Kubeflow Operator
- Installing Kubeflow
- Uninstalling Kubeflow
- Uninstalling Kubeflow Operator
- Troubleshooting
- **Kubeflow on OpenShift**
- Install Kubeflow on OpenShift
- Uninstall Kubeflow
- **Tutorials, Samples, and Shared Resources**
rui-vas marked this conversation as resolved.
Show resolved Hide resolved
- Kubeflow Samples
- Codelabs, Workshops, and Tutorials
- Blog Posts
- Videos
- Shared Resources and Components
- **Further Setup and Troubleshooting**
- Configuring Kubeflow with kfctl and kustomize
- Kubeflow On-prem in a Multi-node Kubernetes Cluster
- Usage Reporting
- Istio Usage in Kubeflow
- Job Scheduling
- Troubleshooting
- Frequently Asked Questions
- Support
- Integrations
- **Upgrading Kubeflow** (to be removed)
- **Reference**
- Reference Overview
- Kubeflow Versioning Policies
- Dockerfile Locations
- Katib Reference
- MPIJob Reference
- Notebook CRD Reference
- PyTorchJob Reference
- TFJob Reference