First version of Auto Scaler

README.md

@@ -12,16 +12,18 @@ Routes Minecraft client connections to backend servers based upon the requested
 ```text
   -api-binding host:port
     	The host:port bound for servicing API requests (env API_BINDING)
+  -auto-scale-up
+      Increase Kubernetes StatefulSet Replicas (only) from 0 to 1 on respective backend servers when accessed (env AUTO_SCALE_UP)
   -connection-rate-limit int
     	Max number of connections to allow per second (env CONNECTION_RATE_LIMIT) (default 1)
   -cpu-profile string
     	Enables CPU profiling and writes to given path (env CPU_PROFILE)
   -debug
     	Enable debug logs (env DEBUG)
   -in-kube-cluster
-    	Use in-cluster kubernetes config (env IN_KUBE_CLUSTER)
+    	Use in-cluster Kubernetes config (env IN_KUBE_CLUSTER)
   -kube-config string
-    	The path to a kubernetes configuration file (env KUBE_CONFIG)
+    	The path to a Kubernetes configuration file (env KUBE_CONFIG)
   -mapping string
     	Comma-separated mappings of externalHostname=host:port (env MAPPING)
   -metrics-backend string
@@ -61,9 +63,9 @@ Routes Minecraft client connections to backend servers based upon the requested
     "backend": "HOST:PORT"
   }
   ```
-  
+
 * `POST /defaultRoute` (with `Content-Type: application/json`)
-  
+
   Registers a default route to the given backend. JSON body is structured as:
   ```json
   {
@@ -74,7 +76,7 @@ Routes Minecraft client connections to backend servers based upon the requested
 * `DELETE /routes/{serverAddress}`
 
   Deletes an existing route for the given `serverAddress`
-  
+
 # Docker Multi-Architecture Image
 
 The [multi-architecture image published at Docker Hub](https://hub.docker.com/repository/docker/itzg/mc-router) supports amd64, arm64, and arm32v6 (i.e. RaspberryPi).
@@ -86,25 +88,25 @@ configures two Minecraft server services named `vanilla` and `forge`, which also
 network aliases. _Notice those services don't need their ports exposed since the internal
 networking allows for the inter-container access._
 
-The `router` service is only one of the services that needs to exposed on the external 
+The `router` service is only one of the services that needs to exposed on the external
 network. The `--mapping` declares how the hostname users will enter into their Minecraft client
 will map to the internal services.
 
 ![](docs/compose-diagram.png)
-  
+
 To test out this example, I added these two entries to my "hosts" file:
 
 ```
 127.0.0.1 vanilla.example.com
 127.0.0.1 forge.example.com
 ```
-  
+
 # Kubernetes Usage
 
-## Using kubernetes service auto-discovery
+## Using Kubernetes Service auto-discovery
 
-When running `mc-router` as a kubernetes pod and you pass the `--in-kube-cluster` command-line argument, then
-it will automatically watch for any services annotated with 
+When running `mc-router` as a Kubernetes Pod and you pass the `--in-kube-cluster` command-line argument, then
+it will automatically watch for any services annotated with
 - `mc-router.itzg.me/externalServerName` : The value of the annotation will be registered as the external hostname Minecraft clients would used to connect to the
    routed service. The service's clusterIP and target port are used as the routed backend. You can use more hostnames by splitting them with comma.
 - `mc-router.itzg.me/defaultServer` : The service's clusterIP and target port are used as the default if
@@ -140,13 +142,13 @@ metadata:
     "mc-router.itzg.me/externalServerName": "external.host.name,other.host.name"
 ```
 
-## Example kubernetes deployment
+## Example Kubernetes deployment
 
-[This example deployment](docs/k8s-example-auto.yaml) 
+[This example deployment](docs/k8s-example-auto.yaml)
 * Declares an `mc-router` service that exposes a node port 25565
 * Declares a service account with access to watch and list services
 * Declares `--in-kube-cluster` in the `mc-router` container arguments
-* Two "backend" Minecraft servers are declared each with an 
+* Two "backend" Minecraft servers are declared each with an
   `"mc-router.itzg.me/externalServerName"` annotation that declares their external server name(s)
 
 ```bash
@@ -157,8 +159,31 @@ kubectl apply -f https://raw.githubusercontent.com/itzg/mc-router/master/docs/k8
 
 #### Notes
 * This deployment assumes two persistent volume claims: `mc-stable` and `mc-snapshot`
-* I extended the allowed node port range by adding `--service-node-port-range=25000-32767` 
-  to `/etc/kubernetes/manifests/kube-apiserver.yaml` 
+* I extended the allowed node port range by adding `--service-node-port-range=25000-32767`
+  to `/etc/kubernetes/manifests/kube-apiserver.yaml`
+
+#### Auto Scale Up
+
+The `-auto-scale-up` flag argument makes the router "wake up" any stopped backend servers, by changing `replicas: 0` to `replicas: 1`.
+
+This requires using `kind: StatefulSet` instead of `kind: Service` for the Minecraft backend servers.
+
+It also requires the `ClusterRole` to permit `get` + `update` for `statefulsets` & `statefulsets/scale`,
+e.g. like this (or some equivalent more fine-grained one to only watch/list services+statefulsets, and only get+update scale):
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  name: services-watcher
+rules:
+- apiGroups: [""]
+  resources: ["services"]
+  verbs: ["watch","list"]
+- apiGroups: ["apps"]
+  resources: ["statefulsets", "statefulsets/scale"]
+  verbs: ["watch","list","get","update"]
+```
 
 # Development
 
@@ -168,16 +193,26 @@ kubectl apply -f https://raw.githubusercontent.com/itzg/mc-router/master/docs/k8
 docker build -t mc-router .
 ```
 
+## Build locally without Docker
+
+After [installing Go](https://go.dev/doc/install) and doing a `go mod download` to install all required prerequisites, just like the [Dockerfile](Dockerfile) does, you can:
+
+```bash
+make test # go test -v ./...
+go build ./cmd/mc-router/
+```
+
 ## Skaffold
 
 For "in-cluster development" it's convenient to use https://skaffold.dev. Any changes to Go source code
 will trigger a go build, new container image pushed to registry with a new tag, and refresh in Kubernetes
-with the image tag used in the deployment transparently updated to the new tag and thus new pod created pulling new images:
+with the image tag used in the deployment transparently updated to the new tag and thus new pod created pulling new images,
+as configured by [skaffold.yaml](skaffold.yaml):
 
     skaffold dev
 
 When using Google Cloud (GCP), first create a _Docker Artifact Registry_,
-then add the _Artifact Registry Reader_ Role to the _Compute Engine default service account_ of your GKE clusterService Account_ (to avoid error like "container mc-router is waiting to start: ...-docker.pkg.dev/... can't be pulled"),
+then add the _Artifact Registry Reader_ Role to the _Compute Engine default service account_ of your _GKE `clusterService` Account_ (to avoid error like "container mc-router is waiting to start: ...-docker.pkg.dev/... can't be pulled"),
 then use e.g. `gcloud auth configure-docker europe-docker.pkg.dev` or equivalent one time (to create a `~/.docker/config.json`),
 and then use e.g. `--default-repo=europe-docker.pkg.dev/YOUR-PROJECT/YOUR-ARTIFACT-REGISTRY` option for `skaffold dev`.
 

cmd/mc-router/main.go

@@ -37,8 +37,9 @@ type Config struct {
 	CpuProfile           string   `usage:"Enables CPU profiling and writes to given path"`
 	Debug                bool     `usage:"Enable debug logs"`
 	ConnectionRateLimit  int      `default:"1" usage:"Max number of connections to allow per second"`
-	InKubeCluster        bool     `usage:"Use in-cluster kubernetes config"`
-	KubeConfig           string   `usage:"The path to a kubernetes configuration file"`
+	InKubeCluster        bool     `usage:"Use in-cluster Kubernetes config"`
+	KubeConfig           string   `usage:"The path to a Kubernetes configuration file"`
+	AutoScaleUp          bool     `usage:"Increase Kubernetes StatefulSet Replicas (only) from 0 to 1 on respective backend servers when accessed"`
 	MetricsBackend       string   `default:"discard" usage:"Backend to use for metrics exposure/publishing: discard,expvar,influxdb"`
 	UseProxyProtocol     bool     `default:"false" usage:"Send PROXY protocol to backend servers"`
 	MetricsBackendConfig MetricsBackendConfig
@@ -113,14 +114,14 @@ func main() {
 	}
 
 	if config.InKubeCluster {
-		err = server.K8sWatcher.StartInCluster()
+		err = server.K8sWatcher.StartInCluster(config.AutoScaleUp)
 		if err != nil {
 			logrus.WithError(err).Fatal("Unable to start k8s integration")
 		} else {
 			defer server.K8sWatcher.Stop()
 		}
 	} else if config.KubeConfig != "" {
-		err := server.K8sWatcher.StartWithConfig(config.KubeConfig)
+		err := server.K8sWatcher.StartWithConfig(config.KubeConfig, config.AutoScaleUp)
 		if err != nil {
 			logrus.WithError(err).Fatal("Unable to start k8s integration")
 		} else {

docs/k8s-deployment.yaml

@@ -20,6 +20,7 @@ spec:
       containers:
       - image: itzg/mc-router:latest
         name: mc-router
+        # Add "--auto-scale-up" here for https://github.com/itzg/mc-router/#auto-scale-up
         args: ["--api-binding", ":8080", "--in-kube-cluster"]
         ports:
         - name: proxy

server/connector.go

@@ -167,7 +167,15 @@ func (c *connectorImpl) HandleConnection(ctx context.Context, frontendConn net.C
 func (c *connectorImpl) findAndConnectBackend(ctx context.Context, frontendConn net.Conn,
 	clientAddr net.Addr, preReadContent io.Reader, serverAddress string) {
 
-	backendHostPort, resolvedHost := Routes.FindBackendForServerAddress(serverAddress)
+	backendHostPort, resolvedHost, waker := Routes.FindBackendForServerAddress(ctx, serverAddress)
+	if waker != nil {
+		if err := waker(ctx); err != nil {
+			logrus.WithFields(logrus.Fields{"serverAddress": serverAddress}).WithError(err).Error("failed to wake up backend")
+			c.metrics.Errors.With("type", "wakeup_failed").Add(1)
+			return
+		}
+	}
+
 	if backendHostPort == "" {
 		logrus.WithField("serverAddress", serverAddress).Warn("Unable to find registered backend")
 		c.metrics.Errors.With("type", "missing_backend").Add(1)