Skip to content

Commit

Permalink
ACL flag with reworked SMI/Service handling
Browse files Browse the repository at this point in the history
  • Loading branch information
jspdown authored Apr 15, 2020
1 parent d7b6546 commit e6ae31d
Show file tree
Hide file tree
Showing 109 changed files with 6,247 additions and 5,911 deletions.
1 change: 1 addition & 0 deletions .golangci.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@
"funlen",
"bodyclose", # Too many false positives
"godox",
"goconst", # Too many false positives
]

[issues]
Expand Down
8 changes: 4 additions & 4 deletions .semaphore/semaphore.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,12 @@ blocks:
- cache restore maesh-img-$SEMAPHORE_GIT_BRANCH-$SEMAPHORE_WORKFLOW_ID
- docker load < maesh-img.tar
jobs:
- name: SMI Integration Suite
- name: ACL Enabled Suite
commands:
- "make test-integration-nobuild TESTFLAGS=\"-check.f SMISuite\""
- name: Kubernetes Suite
- "make test-integration TESTFLAGS=\"-check.f ACLEnabledSuite\""
- name: ACL Disabled Suite
commands:
- "make test-integration-nobuild TESTFLAGS=\"-check.f KubernetesSuite\""
- "make test-integration TESTFLAGS=\"-check.f ACLDisabledSuite\""
- name: CoreDNS Suite
commands:
- "make test-integration-nobuild TESTFLAGS=\"-check.f CoreDNSSuite\""
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ which means that your existing services are unaffected until you decide to add t
Maesh does not use any sidecar container but handles routing through proxy endpoints running on each node.
The mesh controller runs in a dedicated pod and handles all the configuration parsing and deployment to the proxy nodes.
Maesh supports multiple configuration options: annotations on user service objects, and SMI objects.
Not using sidecars means that Maesh does not modify your kubernetes objects, and does not modify your traffic without your knowledge.
Not using sidecars means that Maesh does not modify your Kubernetes objects, and does not modify your traffic without your knowledge.
Using the Maesh endpoints is all that is required.

<p align="center">
Expand Down
21 changes: 12 additions & 9 deletions cmd/configuration.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,16 @@ type MaeshConfiguration struct {
ConfigFile string `description:"Configuration file to use. If specified all other flags are ignored." export:"true"`
KubeConfig string `description:"Path to a kubeconfig. Only required if out-of-cluster." export:"true"`
MasterURL string `description:"The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster." export:"true"`
Debug bool `description:"Debug mode" export:"true"`
SMI bool `description:"Enable SMI operation" export:"true"`
DefaultMode string `description:"Default mode for mesh services" export:"true"`
Debug bool `description:"Debug mode." export:"true"`
ACL bool `description:"Enable ACL mode." export:"true"`
SMI bool `description:"Enable SMI operation, deprecated, use --acl instead." export:"true"`
DefaultMode string `description:"Default mode for mesh services." export:"true"`
Namespace string `description:"The namespace that maesh is installed in." export:"true"`
IgnoreNamespaces []string `description:"The namespace that maesh should be ignoring." export:"true"`
APIPort int32 `description:"API port for the controller" export:"true"`
APIHost string `description:"API host for the controller to bind to" export:"true"`
LimitTCPPort int32 `description:"Number of TCP ports allocated" export:"true"`
LimitHTTPPort int32 `description:"Number of HTTP ports allocated" export:"true"`
APIPort int32 `description:"API port for the controller." export:"true"`
APIHost string `description:"API host for the controller to bind to." export:"true"`
LimitTCPPort int32 `description:"Number of TCP ports allocated." export:"true"`
LimitHTTPPort int32 `description:"Number of HTTP ports allocated." export:"true"`
}

// NewMaeshConfiguration creates a MaeshConfiguration with default values.
Expand All @@ -25,6 +26,7 @@ func NewMaeshConfiguration() *MaeshConfiguration {
ConfigFile: "",
KubeConfig: os.Getenv("KUBECONFIG"),
Debug: false,
ACL: false,
SMI: false,
DefaultMode: "http",
Namespace: "maesh",
Expand All @@ -39,10 +41,11 @@ func NewMaeshConfiguration() *MaeshConfiguration {
type PrepareConfig struct {
KubeConfig string `description:"Path to a kubeconfig. Only required if out-of-cluster." export:"true"`
MasterURL string `description:"The address of the Kubernetes API server. Overrides any value in kubeconfig. Only required if out-of-cluster." export:"true"`
Debug bool `description:"Debug mode" export:"true"`
Debug bool `description:"Debug mode." export:"true"`
Namespace string `description:"The namespace that maesh is installed in." export:"true"`
ClusterDomain string `description:"Your internal K8s cluster domain." export:"true"`
SMI bool `description:"Enable SMI operation" export:"true"`
SMI bool `description:"Enable SMI operation, deprecated, use --acl instead." export:"true"`
ACL bool `description:"Enable ACL mode." export:"true"`
}

// NewPrepareConfig creates PrepareConfig.
Expand Down
15 changes: 11 additions & 4 deletions cmd/maesh/maesh.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,19 @@ func maeshCommand(iConfig *cmd.MaeshConfiguration) error {
minHTTPPort := int32(5000)
minTCPPort := int32(10000)

if iConfig.SMI {
log.Warnf("SMI mode is deprecated, please consider using --acl instead")
}

aclEnabled := iConfig.ACL || iConfig.SMI

log.Debugf("ACL mode enabled: %t", aclEnabled)

// Create a new stop Channel
stopCh := signals.SetupSignalHandler()
// Create a new ctr.
ctr, err := controller.NewMeshController(clients, controller.MeshControllerConfig{
Log: log,
SMIEnabled: iConfig.SMI,
ctr, err := controller.NewMeshController(clients, controller.Config{
ACLEnabled: aclEnabled,
DefaultMode: iConfig.DefaultMode,
Namespace: iConfig.Namespace,
IgnoreNamespaces: iConfig.IgnoreNamespaces,
Expand All @@ -93,7 +100,7 @@ func maeshCommand(iConfig *cmd.MaeshConfiguration) error {
MaxTCPPort: minTCPPort + iConfig.LimitTCPPort,
MinHTTPPort: minHTTPPort,
MaxHTTPPort: minHTTPPort + iConfig.LimitHTTPPort,
})
}, log)
if err != nil {
return fmt.Errorf("unable to create controller: %w", err)
}
Expand Down
10 changes: 9 additions & 1 deletion cmd/prepare/prepare.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,15 @@ func prepareCommand(pConfig *cmd.PrepareConfig) error {
return fmt.Errorf("error during cluster check: %v", err)
}

if err = p.StartInformers(pConfig.SMI); err != nil {
if pConfig.SMI {
log.Warnf("SMI mode is deprecated, please consider using --acl instead")
}

aclEnabled := pConfig.ACL || pConfig.SMI

log.Debugf("ACL mode enabled: %t", aclEnabled)

if err = p.StartInformers(aclEnabled); err != nil {
return fmt.Errorf("error during informer check: %v, this can be caused by pre-existing objects in your cluster that do not conform to the spec", err)
}

Expand Down
36 changes: 23 additions & 13 deletions docs/content/configuration.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Configuration

The configuration for Maesh is broken down into two parts: the static configuration, and the dynamic configuration.
The static configuration is configured when the Maesh service mesh is installed, and is configured via the values.yaml file in the helm install.
The static configuration is configured when the service mesh is installed and is configured via the `values.yaml` file in the Helm install.

## Static configuration

Expand All @@ -14,22 +14,32 @@ The static configuration is configured when the Maesh service mesh is installed,

- Tracing can be enabled.

- Service Mesh Interface (SMI) mode can be enabled.
This configures Maesh to run in SMI mode, where access and routes are explicitly enabled.
Note: By default, all routes and access are denied.
Please see the [SMI Specification](https://github.com/servicemeshinterface/smi-spec) for more information.
- Access-Control List (ACL) mode can be enabled.
This configures Maesh to run in ACL mode, where all traffic is forbidden unless explicitly allowed via
an SMI [TrafficTarget](https://github.com/servicemeshinterface/smi-spec/blob/master/traffic-access-control.md).
Please see the [SMI Specification](https://github.com/servicemeshinterface/smi-spec/blob/master/traffic-access-control.md) for more information.

## Dynamic configuration

Dynamic configuration can be provided to Maesh using either annotations on kubernetes services (default mode) or SMI resources if Maesh is installed with [SMI enabled](./install.md#service-mesh-interface).
Dynamic configuration can be provided to Maesh using annotations on Kubernetes services and via SMI objects.

### With Kubernetes Services
| Features | ACL disabled | ACL enabled |
|-----------------------|--------------|-------------|
| Traffic-Type |||
| Scheme |||
| Retry |||
| Circuit-Breaker |||
| Rate-Limit |||
| Traffic-Split (SMI) |||
| Traffic-Target (SMI) |||

#### Traffic type
### Kubernetes Service Annotations

Annotations on services give the ability to configure how Maesh interprets them.

Annotations on services are the main way to configure Maesh behavior.
#### Traffic type

The service mode can be enabled by using the following annotation:
The traffic type can be configured by using the following annotation:

```yaml
maesh.containo.us/traffic-type: "http"
Expand Down Expand Up @@ -91,7 +101,7 @@ Please note that this value is a string, and needs to be quoted.

Further details about the rate limiting can be found [here](https://docs.traefik.io/v2.0/middlewares/ratelimit/#configuration-options).

### With Service Mesh Interface
### Service Mesh Interface

#### Access Control

Expand Down Expand Up @@ -160,11 +170,11 @@ More information can be found [in the SMI specification](https://github.com/serv
SMI defines the `TrafficSplit` resource which allows to direct subsets of the traffic to different services.

```yaml
apiVersion: split.smi-spec.io/v1alpha1
apiVersion: split.smi-spec.io/v1alpha2
kind: TrafficSplit
metadata:
name: server-split
namespace server
namespace: server
spec:
service: server
backends:
Expand Down
15 changes: 9 additions & 6 deletions docs/content/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ Here are some examples on how to easily deploy Maesh on your cluster.
Deploy those two yaml files on your Kubernetes cluster in order to add a simple backend example, available through HTTP and TCP.

```yaml tab="namespace.yaml"
---
apiVersion: v1
kind: Namespace
metadata:
Expand Down Expand Up @@ -184,7 +183,6 @@ Now, in order to configure Maesh for your `whoami` service, you just need to upd
The HTTP service needs to have `maesh.containo.us/traffic-type: "http"` and the TCP service, `maesh.containo.us/traffic-type: "tcp"`.

```yaml
---
apiVersion: v1
kind: Service
metadata:
Expand Down Expand Up @@ -243,12 +241,16 @@ Accept: */*
X-Forwarded-For: 3.4.5.6
```

## SMI Example
## ACL Example

The [ACL mode](install.md#access-control-list) can be enabled when installing Maesh. Once activated, all traffic is forbidden unless explicitly authorized
using the SMI [TrafficTarget](https://github.com/servicemeshinterface/smi-spec/blob/master/traffic-access-control.md#traffictarget-v1alpha1) resource. This example will present the configuration required to allow the client
pod to send traffic to the HTTP and TCP services defined in the previous example.

In order to expose the HTTP Services from above with SMI enabled, please create the following resources:
Each `TrafficTarget` defines that a set of source `ServiceAccount` is capable of sending traffic to a destination `ServiceAccount`. To authorize the `whoami-client` pod to send traffic to `whoami.whoami.maesh`, we need to
explicitly allow it to hit the pods exposed by the `whoami` service.

```yaml
---
apiVersion: specs.smi-spec.io/v1alpha1
kind: HTTPRouteGroup
metadata:
Expand Down Expand Up @@ -280,7 +282,8 @@ sources:
namespace: whoami
```

For TCP mode, you only need a TCPRoute and a TrafficTarget:

Incoming traffic on a TCP service can also be authorized using a `TrafficTarget` and a `TCPRoute`.

```yaml
kind: TrafficTarget
Expand Down
2 changes: 1 addition & 1 deletion docs/content/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Moreover, Maesh is opt-in by default, which means that your existing services ar
Maesh does not use any sidecar container but handles routing through proxy endpoints running on each node.
The mesh controller runs in a dedicated pod and handles all the configuration parsing and deployment to the proxy nodes.
Maesh supports multiple configuration options: annotations on user service objects, and SMI objects.
Not using sidecars means that Maesh does not modify your kubernetes objects, and does not modify your traffic without your knowledge.
Not using sidecars means that Maesh does not modify your Kubernetes objects and does not modify your traffic without your knowledge.
Using the Maesh endpoints is all that is required.

<p align="center">
Expand Down
40 changes: 28 additions & 12 deletions docs/content/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ helm repo add maesh https://containous.github.io/maesh/charts
helm repo update
```

Install Maesh helm chart:
Install Maesh Helm Chart:

```bash
helm install maesh maesh/maesh
Expand All @@ -28,12 +28,12 @@ make

You will then be able to use the tagged image as your image in your `values.yaml` file.

### Deploy helm chart
### Deploy Helm Chart

??? Note "Helm V3"
Please keep in mind, that our current Helm Chart (v1.1.0) is v3 compatible only.

To deploy the helm chart, run:
To deploy the Helm Chart, run:

```shell
helm install maesh helm/chart/maesh --set controller.image.pullPolicy=IfNotPresent --set controller.image.tag=latest
Expand All @@ -58,29 +58,45 @@ If you use a cluster domain other than `cluster.local` set it by using the `clus
helm install maesh maesh/maesh --set clusterDomain=my.custom.domain.com
```

## Service Mesh Interface
## Access Control List

Maesh supports the [SMI specification](https://smi-spec.io/) which defines a set of custom resources
to provide a fine-grained control over instrumentation, routing and access control of east-west communications.
By default, Maesh does not restrict traffic between pods and services. However, some scenarios require more control
over the rules for internal communication. The Access Control List mode (ACL) requires a set of rules to explicitly allow
traffic between different resources.

To enable SMI, install maesh in SMI mode by setting the `smi.enable` helm chart option to true.
To enable ACL, install Maesh in ACL mode by setting the `acl` Helm Chart option to `true`.

```bash
helm install maesh --namespace=maesh maesh/maesh --set smi.enable=true`
helm install maesh --namespace=maesh maesh/maesh --set acl=true
```

- The `smi.enable` option makes Maesh process SMI resources.
Maesh supports the [SMI specification](https://smi-spec.io/) which defines a set of custom resources
to provide a fine-grained control over instrumentation, routing and access control of east-west communications.

!!! Note CRDs
Helm v3 automatically will install the CRDs in the `/crds` directory.
If you are re-installing into a cluster with the CRDs already present, helm may give you a warning.
If you are re-installing into a cluster with the CRDs already present, Helm may print a warning.
If you do not want to install them, or want to avoid the warning during a re-install,
please use the new `--skip-crds` flag.
More information can be found on the [helm documentation](https://helm.sh/docs/chart_best_practices/custom_resource_definitions/#method-1-let-helm-do-it-for-you).

## SMI Mode (Deprecated)

Maesh supports the SMI specification which defines a set of custom resources to provide a fine-grained control over instrumentation, routing and access control of east-west communications.

To enable SMI, install maesh in SMI mode by setting the smi.enable helm chart option to true.

```bash
helm install maesh --namespace=maesh maesh/maesh --set smi.enable=true`
```

The smi.enable option makes Maesh process SMI resources.

!!! Note This option is deprecated. Please consider using `--acl` instead.

## Platform recommendations

Maesh will work on pretty much any kubernetes environment that conforms to the global kubernetes specification.
Maesh works on Kubernetes environments that conforms to the global Kubernetes specification.
That being said, we have had users encounter issues when using variants such as minikube, microk8s,
and other development distributions.

Expand All @@ -89,7 +105,7 @@ If you want to run Maesh in development, we would recommend using [k3s](https://
We use k3s in Maesh's integration tests, so you can be sure that it works properly.
If you encounter issues on variants such as minikube or microk8s, please try and reproduce the issue on k3s.
If you are unable to reproduce, it may be an issue with the distribution behaving differently than official kubernetes.
If you are unable to reproduce, it may be an issue with the distribution behaving differently than official Kubernetes.
## Verify your installation
Expand Down
4 changes: 2 additions & 2 deletions docs/content/quickstart.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Quickstart

Maesh can be installed in your cluster without affecting any running services.
It will safely install itself via the helm chart, and will be ready for use immediately after.
It can safely be installed using the [Helm Chart](https://helm.sh/docs/intro/using_helm/#helm-install-installing-a-package).

## Prerequisites

Expand Down Expand Up @@ -159,7 +159,7 @@ Take note of the client app pod name (here it's `client-7446fdf848-x96fq`) and o
kubectl -n maesh-test exec -ti client-7446fdf848-x96fq ash
```

From inside the client container make sure you are able to reach your server using kubernetes DNS service discovery.
From inside the client container, make sure your server is reachable using the Kubernetes DNS service discovery.

```bash tab="Command"
curl server.maesh-test.svc.cluster.local
Expand Down
4 changes: 2 additions & 2 deletions helm/chart/maesh/Guidelines.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Maesh Helm Chart Guidelines

This document outlines the guidelines for developing, managing and extending the maesh helm chart.
This document outlines the guidelines for developing, managing and extending the Maesh Helm Chart.

## Optionality

All non-critical features (Features not mandatory to starting a cluster) in the helm chart must be optional.
All non-critical features (features not mandatory to starting a cluster) in the Helm Chart must be optional.
All non-critical features should be disabled (commented out) in the `values.yaml` file.
All optional non-critical features should be disabled (commented out) in the `values.yaml` file, and have a comment `# (Optional)` in the line above.
This allows minimal configuration, and ease of extension.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ spec:
{{- if .Values.mesh }}
- "--defaultMode={{ .Values.mesh.defaultMode }}"
{{- end }}
{{- if .Values.acl }}
- "--acl"
{{- end }}
{{- if .Values.smi.enable }}
- "--smi"
{{- end }}
Expand Down Expand Up @@ -120,6 +123,9 @@ spec:
{{- if .Values.smi.enable }}
- "--smi"
{{- end }}
{{- if .Values.acl }}
- "--acl"
{{- end }}
- "--clusterdomain"
- {{ default "cluster.local" .Values.clusterDomain | quote }}
- "--namespace=$(POD_NAMESPACE)"
Expand Down
Loading

0 comments on commit e6ae31d

Please sign in to comment.