From 9273cc568627472ce5b6411d00f3f121dd15b6f2 Mon Sep 17 00:00:00 2001 From: adrianc Date: Mon, 8 May 2023 20:24:55 +0300 Subject: [PATCH] Add Documentation - Update README - Add CONTRIBUTING doc Signed-off-by: adrianc --- CONTRIBUTING.md | 67 +++++++++++++ README.md | 250 +++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 303 insertions(+), 14 deletions(-) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..64ea15e --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,67 @@ +# NVIDIA IPAM Plugin + +## How to Contribute + +NVIDIA IPAM plugin is [Apache 2.0 licensed](LICENSE) and accepts contributions via GitHub pull requests. +This document outlines some of the conventions on development workflow, commit message formatting, +contact points and other resources to make it easier to get your contribution accepted. + +## Coding Style + +Please follows the standard formatting recommendations and language idioms set out in [Effective Go](https://golang.org/doc/effective_go.html) and in the [Go Code Review Comments wiki](https://github.com/golang/go/wiki/CodeReviewComments). + +## Format of the patch + +Each patch is expected to comply with the following format: + +```text +Change summary + +More detailed explanation of your changes: Why and how. +Wrap it to 72 characters. +See [here] (http://chris.beams.io/posts/git-commit/) +for some more good advices. + +[Fixes #NUMBER (or URL to the issue)] +``` + +For example: + +```text +Fix poorly named identifiers + +One identifier, fnname, in func.go was poorly named. It has been renamed +to fnName. Another identifier retval was not needed and has been removed +entirely. + +Fixes #1 +``` + +## Contributing Code + +* Make sure to create an [Issue](https://github.com/Mellanox/nvidia-k8s-ipam/issues) for bug fix or the feature request. +* **For bugs**: For the bug fixes, please follow the issue template format while creating a issue. If you have already found a fix, feel free to submit a Pull Request referencing the Issue you created. Include the `Fixes #` syntax to link it to the issue you're addressing. +* **For feature requests**, For the feature requests, please follow the issue template format while creating a feature requests. + * Please make sure each PR are compiling and CI checks are passing. + * In order to ensure your PR can be reviewed in a timely manner, please keep PRs small. + +Once you're ready to contribute code back to this repo, start with these steps: + +* Fork the project. +* Clone the fork to your machine: + +```shell +git clone https://github.com/Mellanox/nvidia-k8s-ipam.git +``` + +* Create a topic branch with prefix `dev/` for your change and checkout that branch: + +```shell +git checkout -b dev/some-topic-branch +``` + +* Make your changes to the code and add tests to cover contributed code. +* Run `make build && make test` to validate it builds and will not break current functionality. +* Commit your changes and push them to your fork. +* Open a pull request for the appropriate project. +* Maintainers will review your pull request, suggest changes, run tests and eventually merge or close the request. diff --git a/README.md b/README.md index 897d7da..5fa7ec7 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,153 @@ -# nvidia-k8s-ipam -NVIDIA IPAM plugin +[![Coverage Status](https://coveralls.io/repos/github/Mellanox/nvidia-k8s-ipam/badge.svg)](https://coveralls.io/github/Mellanox/nvidia-k8s-ipam) -## Usage +# NVIDIA IPAM Plugin -### Deploy IPAM +An IP Address Management (IPAM) CNI plugin designed to operate in a Kubernetes environment. +This Plugins allows to assign IP addresses dynamically across the cluster while keeping speed +and performance in mind. -_NOTE: This command will deploy latest dev build_ +NVIDIA IPAM plugin currently support only IP allocation for K8s Secondary Networks. +e.g Additional networks provided by [multus CNI plugin](https://github.com/k8snetworkplumbingwg/multus-cni). +This repository is in its first steps of development, APIs may change in a non backward compatible manner. + +## Introduction + +NVIDIA IPAM plugin consists of 3 main components: + +1. controller ([ipam-controller](#ipam-controller)) +2. node agent ([ipam-node](#ipam-node)) +3. IPAM CNI plugin ([nv-ipam](#nv-ipam)) + +### ipam-controller + +A Kubernetes(K8s) controller that Watches on a predefined K8s ConfigMap for defined IP pools. +It then proceeds in reconciling K8s Node objects by assiging each node via `ipam.nvidia.com/ip-blocks` +annotation a cluster unique range of IPs of the defined pools. + +### ipam-node + +A node agent that performs initial setup and installation of nv-ipam CNI plugin. + +### nv-ipam + +An IPAM CNI plugin that allocates IPs for a given interface out of the defined IP pool as provided +via CNI configuration. + +IPs are allocated out of the provided IP Block assigned by ipam-controller for the node. +To determine the cluster unique IP block for the defined IP pool, nv-ipam CNI queries K8s API +for the Node object and extracts IP block infromation from node annotation. + +nv-ipam plugin currently leverages [host-local](https://www.cni.dev/plugins/current/ipam/host-local/) +IPAM to allocate IPs from the given range. + +### IP allocation flow + +1. User (cluster administrator) defines a set of named IP Pools to be used for IP allocation +of container interfaces via Kubernetes ConfigMap (more information in [Configuration](#configuration) section) + +_Example_: + +```json +{ + "pools": { + "my-pool": {"subnet": "192.188.0.0/16", "perNodeBlockSize": 24, "gateway": “192.168.0.1”}, + }, + "nodeSelector": { + "kubernetes.io/os": "linux" + } +} ``` -kubectl apply -f https://raw.githubusercontent.com/Mellanox/nvidia-k8s-ipam/main/deploy/nv-ipam.yaml + +2. ipam-controller calculates and assigns unique IP blocks for each Node via annotation + +_Example_: + +```yaml +annotations: + ipam.nvidia.com/ip-blocks: '{ + "my-pool": {"startIP": "192.168.0.2", "endIP": "192.168.0.25", "gateway": "192.168.0.1", "subnet": "192.168.0.0/16"} + }' ``` -### Create config +3. User specifies nv-ipam as IPAM plugin in CNI configuration + +_Example macvlan CNI configuration_: + +```json +{ + "type": "macvlan", + "cniVersion": "0.3.1", + "master": "enp3s0f0np0", + "mode": "bridge", + "ipam": { + "type": "nv-ipam", + "poolName": "my-pool" + } +} ``` -cat < of node selector labels, only nodes that match the provided labels will get assigned IP blocks for the defined pools + + +### ipam-node +ipam-node accepts configuration via command line flags + +```text + --cni-bin-dir string CNI binary directory (default "/host/opt/cni/bin") + --cni-conf-dir string CNI config directory (default "/host/etc/cni/net.d") + -h, --help show help message and quit + --host-local-bin-file string host-local binary file path (default "/host-local") + --nv-ipam-bin-file string nv-ipam binary file path (default "/nv-ipam") + --nv-ipam-cni-data-dir string nv-ipam CNI data directory (default "/host/var/lib/cni/nv-ipam") + --nv-ipam-cni-data-dir-host string nv-ipam CNI data directory on host (default "/var/lib/cni/nv-ipam") + --nv-ipam-kubeconfig-file-host string kubeconfig for nv-ipam (default "/etc/cni/net.d/nv-ipam.d/nv-ipam.kubeconfig") + --nv-ipam-log-file string nv-ipam log file (default "/var/log/nv-ipam-cni.log") + --nv-ipam-log-level string nv-ipam log level (default "info") + --skip-host-local-binary-copy skip host-loca binary file copy ``` -## CNI config +### nv-ipam (CNI configuration) -Example config for bridge CNI +nv-ipam accepts the following CNI configuration: +```json +{ + "type": "nv-ipam", + "poolName": "my-pool", + "kubeconfig": "/etc/cni/net.d/nv-ipam.d/nv-ipam.kubeconfig", + "dataDir": "/var/lib/cni/nv-ipam", + "confDir": "/etc/cni/net.d/nv-ipam.d", + "logFile": "/var/log/nv-ipam-cni.log", + "logLevel": "info" +} ``` + +* `type` (string, required): CNI plugin name, MUST be `"nv-ipam"` +* `poolName` (string, optional): name of the IP Pool to be used for IP allocation. (default: network name as provided in CNI call) +* `kubeconfig` (string, optional): path to kubeconfig file. (default: `"/etc/cni/net.d/nv-ipam.d/nv-ipam.kubeconfig"`) +* `dataDir` (string, optional): path to data dir. (default: `/var/lib/cni/nv-ipam`) +* `confDir` (string, optional): path to configuration dir. (default: `"/etc/cni/net.d/nv-ipam.d"`) +* `logFile` (string, optional): log file path. (default: `"/var/log/nv-ipam-cni.log"`) +* `logLevel` (string, optional): logging level. one of: `["verbose", "debug", "info", "warning", "error", "panic"]`. (default: `"info"`) + +## Deployment + +### Deploy IPAM plugin + +> _NOTE:_ This command will deploy latest dev build with default configuration + +```shell +kubectl apply -f https://raw.githubusercontent.com/Mellanox/nvidia-k8s-ipam/main/deploy/nv-ipam.yaml +``` + +### Create ipam-controller config + +```shell +cat < /etc/cni/net.d/10-mynet.conf < /etc/cni/net.d/10-mynet.conf <