[HOPSWORKS.AI-DOC] add and remove workers (#159)

docs/hopsworksai/adding_removing_workers.md

@@ -0,0 +1,84 @@
+# Adding and removing workers
+Once you have started a hopsworks cluster you can add and remove workers from the cluster to accommodate your workload.
+
+## Adding workers
+If the computation you are running is using all the resources of your Hopsworks cluster you can add workers to your cluster.
+To add workers to a cluster, go to the *Details* tab of this cluster and click on *Add workers*.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/add-worker.png">
+      <img src="../../../assets/images/hopsworksai/add-worker.png" alt="Add worker">
+    </a>
+    <figcaption>Add worker</figcaption>
+  </figure>
+</p>
+
+Select the number of workers you want to add (1). Select the type of instance you want the workers to run on (2). Select the local storage size for the workers (3). Click on *Next*.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/add-workers-config.png">
+      <img src="../../../assets/images/hopsworksai/add-workers-config.png" alt="Add workers config">
+    </a>
+    <figcaption>Add workers</figcaption>
+  </figure>
+</p>
+
+Review your request and click *Add*.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/add-workers-review.png">
+      <img src="../../../assets/images/hopsworksai/add-workers-review.png" alt="Add workers review">
+    </a>
+    <figcaption>Add workers</figcaption>
+  </figure>
+</p>
+
+Hopsworks.ai will start the new workers and you will be able to use them in your cluster as soon as they have finished starting.
+
+## Removing workers
+
+If the load on your Hopsworks cluster is low, you can decide to remove worker nodes from your cluster.
+
+!!! warning
+    When removing workers Hopsworks.ai will try to select workers that can be removed while interfering as little as possible with any ongoing computation. It will also wait for the workers to be done with their computation before stopping them. But, if this computation lasts too long, the worker may get stopped before the computation properly finish. This could interfere with your ongoing computation.
+
+!!! note 
+    You can remove all the workers of your cluster. If you do so the cluster will be able to store data but not run any computations. This may affect feature store functionality.
+
+To remove workers from a cluster, go to the *Details* tab of this cluster and click on *Remove workers*
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/remove-worker.png">
+      <img src="../../../assets/images/hopsworksai/remove-worker.png" alt="Remove worker">
+    </a>
+    <figcaption>Remove workers</figcaption>
+  </figure>
+</p>
+
+For each of the types of instances existing in your cluster select the number of workers you want to remove and click on *Next*.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/remove-worker-config.png">
+      <img src="../../../assets/images/hopsworksai/remove-worker-config.png" alt="Remove workers config">
+    </a>
+    <figcaption>Remove workers</figcaption>
+  </figure>
+</p>
+
+Review your request and click *Remove*.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/remove-workers-review.png">
+      <img src="../../../assets/images/hopsworksai/remove-workers-review.png" alt="Remove workers review">
+    </a>
+    <figcaption>Remove workers</figcaption>
+  </figure>
+</p>
+
+Hopsworks.ai will select the workers corresponding to your criteria which can be stopped with as little interferences as possible with any ongoing computation. It will set them to decommission and stop them when they have finished decommissioning.

docs/hopsworksai/gpu_support.md

@@ -0,0 +1,52 @@
+# GPU support
+Hopsworks can harness the power of GPUs to speed up machine learning processes. You can take advantage of this feature in Hopsworks.ai by adding GPU equipped workers to your cluster. This can be done in two way: creating a cluster with GPU equipped workers or adding GPU equipped workers to an existing cluster.
+
+!!! warning
+    This feature is not supported yet in Azure but will be added shortly.
+
+## Creating a cluster with GPU equipped workers
+When selecting the [workers' instance type](aws/cluster_creation.md#step-2-setting-the-general-information) during the cluster creation, you can select an instance type equipped with GPUs. The cluster will then be created and Hopsworks will automatically detect the GPU resource.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/create-gpu.png">
+      <img src="../../../assets/images/hopsworksai/create-gpu.png" alt="Create cluster with GPUs">
+    </a>
+    <figcaption>Create cluster with GPUs</figcaption>
+  </figure>
+</p>
+
+## Adding GPU equipped workers to an existing cluster.
+When [adding workers](adding_removing_workers.md#adding-workers) to a cluster, you can select an instance type equipped with GPUs. The workers will then be added to the cluster and Hopsworks will automatically detect the new GPU resource.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/add-gpu.png">
+      <img src="../../../assets/images/hopsworksai/add-gpu.png" alt="Add GPUs to cluster">
+    </a>
+    <figcaption>Add GPUs to cluster</figcaption>
+  </figure>
+</p>
+
+## Using the GPUs
+Once workers with GPUs have been added to your cluster you can use them by allocating GPUs to JupyterLab or Jobs.
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/gpu-jupyter.png">
+      <img src="../../../assets/images/hopsworksai/gpu-jupyter.png" alt="Using GPUs in JupyterLab">
+    </a>
+    <figcaption>Using GPUs in JupyterLab</figcaption>
+  </figure>
+</p>
+
+<p align="center">
+  <figure>
+    <a  href="../../../assets/images/hopsworksai/gpu-jobs.png">
+      <img src="../../../assets/images/hopsworksai/gpu-jobs.png" alt="Using GPUs in jobs">
+    </a>
+    <figcaption>Using GPUs in jobs</figcaption>
+  </figure>
+</p>
+
+For more information about using GPUs in Hopsworks you can consult Hopsworks [Experiments documentation](https://hopsworks.readthedocs.io/en/stable/hopsml/experiment.html).

mkdocs.yml

@@ -22,6 +22,8 @@ nav:
       - AWS: hopsworksai/aws/cluster_creation.md
       - Integration with EKS and ECR: hopsworksai/aws/eks_ecr_integration.md
       - Azure: hopsworksai/azure/cluster_creation.md
+    - Adding and Removing workers: hopsworksai/adding_removing_workers.md
+    - GPU support: hopsworksai/gpu_support.md
     # - Databricks Quickstart:
     #  - AWS: hopsworksai/aws/databricks_quickstart.md
     #  - Azure: hopsworksai/azure/databricks_quickstart.md