Spring Cloud Kubernetes

Why do you need Spring Cloud Kubernetes?

Spring Cloud Kubernetes provide Spring Cloud common interfaces implementations to consume Kubernetes native services. The main objective of the projects provided in this repository is to facilitate the integration of Spring Cloud/Spring Boot applications running inside Kubernetes.

DiscoveryClient for Kubernetes

This project provides an implementation of Discovery Client for Kubernetes. This allows you to query Kubernetes endpoints (see services) by name. A service is typically exposed by the Kubernetes API server as a collection of endpoints which represent http, https addresses that a client can access from a Spring Boot application running as a pod. This discovery feature is also used by the Spring Cloud Kubernetes Ribbon project to fetch the list of the endpoints defined for an application to be load balanced.

This is something that you get for free just by adding the following dependency inside your project:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-kubernetes</artifactId>
    <version>${latest.version}</version>
</dependency>

To enable loading of the DiscoveryClient, add @EnableDiscoveryClient to the according configuration or application class like this:

@SpringBootApplication
@EnableDiscoveryClient
public class Application {
  public static void main(String[] args) {
    SpringApplication.run(Application.class, args);
  }
}

Then you can inject the client in your code simply by:

@Autowired
private DiscoveryClient discoveryClient;

If for any reason you need to disable the DiscoveryClient you can simply set the following property in application.properties:

spring.cloud.kubernetes.discovery.enabled=false

Some Spring Cloud components use the DiscoveryClient in order to obtain info about the local service instance. For this to work you need to align the Kubernetes service name with the spring.application.name property.

Kubernetes PropertySource implementations

The most common approach to configure your Spring Boot application is to create an application.properties|yaml or an application-profile.properties|yaml file containing key-value pairs providing customization values to your application or Spring Boot starters. Users may override these properties by specifying system properties or environment variables.

ConfigMap PropertySource

Kubernetes provides a resource named ConfigMap to externalize the parameters to pass to your application in the form of key-value pairs or embedded application.properties|yaml files. The Spring Cloud Kubernetes Config project makes Kubernetes `ConfigMap`s available during application bootstrapping and triggers hot reloading of beans or Spring context when changes are detected on observed `ConfigMap`s.

The default behavior is to create a ConfigMapPropertySource based on a Kubernetes ConfigMap which has metadata.name of either the name of your Spring application (as defined by its spring.application.name property) or a custom name defined within the bootstrap.properties file under the following key spring.cloud.kubernetes.config.name.

However, more advanced configuration are possible where multiple ConfigMaps can be used This is made possible by the spring.cloud.kubernetes.config.sources list. For example one could define the following ConfigMaps

spring:
  application:
    name: cloud-k8s-app
  cloud:
    kubernetes:
      config:
        name: default-name
        namespace: default-namespace
        sources:
         # Spring Cloud Kubernetes will lookup a ConfigMap named c1 in namespace default-namespace
         - name: c1
         # Spring Cloud Kubernetes will lookup a ConfigMap named default-name in whatever namespace n2
         - namespace: n2
         # Spring Cloud Kubernetes will lookup a ConfigMap named c3 in namespace n3
         - namespace: n3
           name: c3

In the example above, it spring.cloud.kubernetes.config.namespace had not been set, then the ConfigMap named c1 would be looked up in the namespace that the application runs

Any matching ConfigMap that is found, will be processed as follows:

• apply individual configuration properties.

• apply as yaml the content of any property named application.yaml

• apply as properties file the content of any property named application.properties

The single exception to the aforementioned flow is when the ConfigMap contains a single key that indicates the file is a YAML or Properties file. In that case the name of the key does NOT have to be application.yaml or application.properties (it can be anything) and the value of the property will be treated correctly. This features facilitates the use case where the ConfigMap was created using something like:

kubectl create configmap game-config --from-file=/path/to/app-config.yaml

Example:

Let’s assume that we have a Spring Boot application named demo that uses properties to read its thread pool configuration.

• pool.size.core

• pool.size.maximum

This can be externalized to config map in yaml format:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  pool.size.core: 1
  pool.size.max: 16

Individual properties work fine for most cases but sometimes embedded yaml is more convenient. In this case we will use a single property named application.yaml to embed our yaml:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  application.yaml: |-
    pool:
      size:
        core: 1
        max:16

The following also works:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  custom-name.yaml: |-
    pool:
      size:
        core: 1
        max:16

Spring Boot applications can also be configured differently depending on active profiles which will be merged together when the ConfigMap is read. It is possible to provide different property values for different profiles using an application.properties|yaml property, specifying profile-specific values each in their own document (indicated by the --- sequence) as follows:

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo
data:
  application.yml: |-
    greeting:
      message: Say Hello to the World
    farewell:
      message: Say Goodbye
    ---
    spring:
      profiles: development
    greeting:
      message: Say Hello to the Developers
    farewell:
      message: Say Goodbye to the Developers
    ---
    spring:
      profiles: production
    greeting:
      message: Say Hello to the Ops

In the above case, the configuration loaded into your Spring Application with the development profile will be:

  greeting:
    message: Say Hello to the Developers
  farewell:
    message: Say Goodbye to the Developers

whereas if the production profile is active, the configuration will be:

  greeting:
    message: Say Hello to the Ops
  farewell:
    message: Say Goodbye

If both profiles are active, the property which appears last within the configmap will overwrite preceding values.

To tell to Spring Boot which profile should be enabled at bootstrap, a system property can be passed to the Java command launching your Spring Boot application using an env variable that you will define with the OpenShift DeploymentConfig or Kubernetes ReplicationConfig resource file as follows:

apiVersion: v1
kind: DeploymentConfig
spec:
  replicas: 1
  ...
    spec:
      containers:
      - env:
        - name: JAVA_APP_DIR
          value: /deployments
        - name: JAVA_OPTIONS
          value: -Dspring.profiles.active=developer

Notes: - check the security configuration section, to access config maps from inside a pod you need to have the correct Kubernetes service accounts, roles and role bindings.

Table 1. Properties:

Name	Type	Default	Description
spring.cloud.kubernetes.config.enabled	Boolean	true	Enable Secrets PropertySource
spring.cloud.kubernetes.config.name	String	${spring.application.name}	Sets the name of ConfigMap to lookup
spring.cloud.kubernetes.config.namespace	String	Client namespace	Sets the Kubernetes namespace where to lookup
spring.cloud.kubernetes.config.paths	List	null	Sets the paths where ConfigMaps are mounted
spring.cloud.kubernetes.config.enableApi	Boolean	true	Enable/Disable consuming ConfigMaps via APIs

Secrets PropertySource

Kubernetes has the notion of [Secrets](https://kubernetes.io/docs/concepts/configuration/secret/) for storing sensitive data such as password, OAuth tokens, etc. This project provides integration with Secrets to make secrets accessible by Spring Boot applications. This feature can be explicitly enabled/disabled using the spring.cloud.kubernetes.secrets.enabled property.

The SecretsPropertySource when enabled will lookup Kubernetes for Secrets from the following sources: 1. reading recursively from secrets mounts 2. named after the application (as defined by spring.application.name) 3. matching some labels

Please note that by default, consuming Secrets via API (points 2 and 3 above) is not enabled for security reasons and it is recommend that containers share secrets via mounted volumes.

If the secrets are found their data is made available to the application.

Example:

Let’s assume that we have a spring boot application named demo that uses properties to read its database configuration. We can create a Kubernetes secret using the following command:

oc create secret generic db-secret --from-literal=username=user --from-literal=password=p455w0rd

This would create the following secret (shown using oc get secrets db-secret -o yaml):

apiVersion: v1
data:
  password: cDQ1NXcwcmQ=
  username: dXNlcg==
kind: Secret
metadata:
  creationTimestamp: 2017-07-04T09:15:57Z
  name: db-secret
  namespace: default
  resourceVersion: "357496"
  selfLink: /api/v1/namespaces/default/secrets/db-secret
  uid: 63c89263-6099-11e7-b3da-76d6186905a8
type: Opaque

Note that the data contains Base64-encoded versions of the literal provided by the create command.

This secret can then be used by your application for example by exporting the secret’s value as environment variables:

apiVersion: v1
kind: Deployment
metadata:
  name: ${project.artifactId}
spec:
   template:
     spec:
       containers:
         - env:
            - name: DB_USERNAME
              valueFrom:
                 secretKeyRef:
                   name: db-secret
                   key: username
            - name: DB_PASSWORD
              valueFrom:
                 secretKeyRef:
                   name: db-secret
                   key: password

You can select the Secrets to consume in a number of ways:

1. By listing the directories where secrets are mapped: ` -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/db-secret,etc/secrets/postgresql `

   If you have all the secrets mapped to a common root, you can set them like:

   ```
   -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets
   ```

2. By setting a named secret: ` -Dspring.cloud.kubernetes.secrets.name=db-secret `

3. By defining a list of labels: ` -Dspring.cloud.kubernetes.secrets.labels.broker=activemq -Dspring.cloud.kubernetes.secrets.labels.db=postgresql `

Table 2. Properties:

Name	Type	Default	Description
spring.cloud.kubernetes.secrets.enabled	Boolean	true	Enable Secrets PropertySource
spring.cloud.kubernetes.secrets.name	String	${spring.application.name}	Sets the name of the secret to lookup
spring.cloud.kubernetes.secrets.namespace	String	Client namespace	Sets the Kubernetes namespace where to lookup
spring.cloud.kubernetes.secrets.labels	Map	null	Sets the labels used to lookup secrets
spring.cloud.kubernetes.secrets.paths	List	null	Sets the paths where secrets are mounted (example 1)
spring.cloud.kubernetes.secrets.enableApi	Boolean	false	Enable/Disable consuming secrets via APIs (examples 2 and 3)

Notes: - The property spring.cloud.kubernetes.secrets.labels behaves as defined by Map-based binding. - The property spring.cloud.kubernetes.secrets.paths behaves as defined by Collection-based binding. - Access to secrets via API may be restricted for security reasons, the preferred way is to mount secret to the POD.

Example of application using secrets (though it hasn’t been updated to use the new spring-cloud-kubernetes project): spring-boot-camel-config

PropertySource Reload

Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration. The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap or Secret changes.

This feature is disabled by default and can be enabled using the configuration property spring.cloud.kubernetes.reload.enabled=true (eg. in the application.properties file).

The following levels of reload are supported (property spring.cloud.kubernetes.reload.strategy): - refresh (default): only configuration beans annotated with @ConfigurationProperties or @RefreshScope are reloaded. This reload level leverages the refresh feature of Spring Cloud Context. - restart_context: the whole Spring ApplicationContext is gracefully restarted. Beans are recreated with the new configuration. - shutdown: the Spring ApplicationContext is shut down to activate a restart of the container. When using this level, make sure that the lifecycle of all non-daemon threads is bound to the ApplicationContext and that a replication controller or replica set is configured to restart the pod.

Example:

Assuming that the reload feature is enabled with default settings (refresh mode), the following bean will be refreshed when the config map changes:

@Configuration
@ConfigurationProperties(prefix = "bean")
public class MyConfig {

    private String message = "a message that can be changed live";

    // getter and setters

}

A way to see that changes effectively happen is creating another bean that prints the message periodically.

@Component
public class MyBean {

    @Autowired
    private MyConfig config;

    @Scheduled(fixedDelay = 5000)
    public void hello() {
        System.out.println("The message is: " + config.getMessage());
    }
}

The message printed by the application can be changed using a ConfigMap as follows:

apiVersion: v1
kind: ConfigMap
metadata:
  name: reload-example
data:
  application.properties: |-
    bean.message=Hello World!

Any change to the property named bean.message in the ConfigMap associated to the pod will be reflected in the output. More generally speaking, changes associated to properties prefixed with the value defined by the prefix field of the @ConfigurationProperties annotation will be detected and reflected in the application. [Associating a ConfigMap to a pod](#configmap-propertysource) is explained above.

The full example is available in [spring-cloud-kubernetes-reload-example](spring-cloud-kubernetes-examples/kubernetes-reload-example).

The reload feature supports two operating modes: - event (default): watches for changes in config maps or secrets using the Kubernetes API (web socket). Any event will produce a re-check on the configuration and a reload in case of changes. The view role on the service account is required in order to listen for config map changes. A higher level role (eg. edit) is required for secrets (secrets are not monitored by default). - polling: re-creates the configuration periodically from config maps and secrets to see if it has changed. The polling period can be configured using the property spring.cloud.kubernetes.reload.period and defaults to 15 seconds. It requires the same role as the monitored property source. This means, for example, that using polling on file mounted secret sources does not require particular privileges.

Table 3. Properties:

Name	Type	Default	Description
spring.cloud.kubernetes.reload.enabled	Boolean	false	Enables monitoring of property sources and configuration reload
spring.cloud.kubernetes.reload.monitoring-config-maps	Boolean	true	Allow monitoring changes in config maps
spring.cloud.kubernetes.reload.monitoring-secrets	Boolean	false	Allow monitoring changes in secrets
spring.cloud.kubernetes.reload.strategy	Enum	refresh	The strategy to use when firing a reload (refresh, restart_context, shutdown)
spring.cloud.kubernetes.reload.mode	Enum	event	Specifies how to listen for changes in property sources (event, polling)
spring.cloud.kubernetes.reload.period	Long	15000	The period in milliseconds for verifying changes when using the polling strategy

Notes: - Properties under spring.cloud.kubernetes.reload. should not be used in config maps or secrets: changing such properties at runtime may lead to unexpected results; - Deleting a property or the whole config map does not restore the original state of the beans when using the refresh level.

Ribbon discovery in Kubernetes

Spring Cloud client applications calling a microservice should be interested on relying on a client load-balancing feature in order to automatically discover at which endpoint(s) it can reach a given service. This mechanism has been implemented within the [spring-cloud-kubernetes-ribbon](spring-cloud-kubernetes-ribbon/pom.xml) project where a Kubernetes client will populate a Ribbon ServerList containing information about such endpoints.

The implementation is part of the following starter that you can use by adding its dependency to your pom file:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-kubernetes-netflix</artifactId>
    <version>${latest.version}</version>
</dependency>

When the list of the endpoints is populated, the Kubernetes client will search the registered endpoints living in the current namespace/project matching the service name defined using the Ribbon Client annotation:

@RibbonClient(name = "name-service")

You can configure Ribbon’s behavior by providing properties in your application.properties (via your application’s dedicated ConfigMap) using the following format: <name of your service>.ribbon.<Ribbon configuration key> where:

• <name of your service> corresponds to the service name you’re accessing over Ribbon, as configured using the @RibbonClient annotation (e.g. name-service in the example above)

• <Ribbon configuration key> is one of the Ribbon configuration key defined by Ribbon’s CommonClientConfigKey class

Additionally, the spring-cloud-kubernetes-ribbon project defines two additional configuration keys to further control how Ribbon interacts with Kubernetes. In particular, if an endpoint defines multiple ports, the default behavior is to use the first one found. To select more specifically which port to use, in a multi-port service, use the PortName key. If you want to specify in which Kubernetes' namespace the target service should be looked up, use the KubernetesNamespace key, remembering in both instances to prefix these keys with your service name and ribbon prefix as specified above.

Examples that are using this module for ribbon discovery are:

• Spring Cloud Circuitbreaker and Ribbon

• fabric8-quickstarts - Spring Boot - Ribbon

• Kubeflix - LoanBroker - Bank

Note: The Ribbon discovery client can be disabled by setting this key within the application properties file spring.cloud.kubernetes.ribbon.enabled=false.

Kubernetes Awareness

All of the features described above will work equally well regardless of whether your application is running inside Kubernetes or not. This is really helpful for development and troubleshooting. From a development point of view, this is really helpful as you can start your Spring Boot application and debug one of the modules part of this project. It is not required to deploy it in Kubernetes as the code of the project relies on the [Fabric8 Kubernetes Java client](https://github.com/fabric8io/kubernetes-client) which is a fluent DSL able to communicate using http protocol to the REST API of Kubernetes Server.

Kubernetes Profile Autoconfiguration

When the application runs as a pod inside Kubernetes a Spring profile named kubernetes will automatically get activated. This allows the developer to customize the configuration, to define beans that will be applied when the Spring Boot application is deployed within the Kubernetes platform (e.g. different dev and prod configuration).

Pod Health Indicator

Spring Boot uses HealthIndicator to expose info about the health of an application. That makes it really useful for exposing health related information to the user and are also a good fit for use as readiness probes.

The Kubernetes health indicator which is part of the core module exposes the following info:

• pod name, ip address, namespace, service account, node name and its ip address

• flag that indicates if the Spring Boot application is internal or external to Kubernetes

Leader Election

<TBD>

Security Configurations inside Kubernetes

Namespace

Most of the components provided in this project need to know the namespace. For Kubernetes (1.3+) the namespace is made available to pod as part of the service account secret and automatically detected by the client. For earlier version it needs to be specified as an env var to the pod. A quick way to do this is:

      env:
      - name: "KUBERNETES_NAMESPACE"
        valueFrom:
          fieldRef:
            fieldPath: "metadata.namespace"

Service Account

For distros of Kubernetes that support more fine-grained role-based access within the cluster, you need to make sure a pod that runs with spring-cloud-kubernetes has access to the Kubernetes API. For any service accounts you assign to a deployment/pod, you need to make sure it has the correct roles. For example, you can add cluster-reader permissions to your default service account depending on the project you’re in:

Examples

List of examples using these projects:

<TBD>

Other Resources

Building

Basic Compile and Test

To build the source you will need to install JDK 1.7.

Spring Cloud uses Maven for most build-related activities, and you should be able to get off the ground quite quickly by cloning the project you are interested in and typing

$ ./mvnw install

Note	You can also install Maven (>=3.3.3) yourself and run the mvn command in place of ./mvnw in the examples below. If you do that you also might need to add -P spring if your local Maven settings do not contain repository declarations for spring pre-release artifacts.

Note

Be aware that you might need to increase the amount of memory available to Maven by setting a MAVEN_OPTS environment variable with a value like -Xmx512m -XX:MaxPermSize=128m. We try to cover this in the .mvn configuration, so if you find you have to do it to make a build succeed, please raise a ticket to get the settings added to source control.

For hints on how to build the project look in .travis.yml if there is one. There should be a "script" and maybe "install" command. Also look at the "services" section to see if any services need to be running locally (e.g. mongo or rabbit). Ignore the git-related bits that you might find in "before_install" since they’re related to setting git credentials and you already have those.

The projects that require middleware generally include a docker-compose.yml, so consider using Docker Compose to run the middeware servers in Docker containers. See the README in the scripts demo repository for specific instructions about the common cases of mongo, rabbit and redis.

Note	If all else fails, build with the command from .travis.yml (usually ./mvnw install).

Documentation

The spring-cloud-build module has a "docs" profile, and if you switch that on it will try to build asciidoc sources from src/main/asciidoc. As part of that process it will look for a README.adoc and process it by loading all the includes, but not parsing or rendering it, just copying it to ${main.basedir} (defaults to ${basedir}, i.e. the root of the project). If there are any changes in the README it will then show up after a Maven build as a modified file in the correct place. Just commit it and push the change.

Working with the code

If you don’t have an IDE preference we would recommend that you use Spring Tools Suite or Eclipse when working with the code. We use the m2eclipse eclipse plugin for maven support. Other IDEs and tools should also work without issue as long as they use Maven 3.3.3 or better.

Importing into eclipse with m2eclipse

We recommend the m2eclipse eclipse plugin when working with eclipse. If you don’t already have m2eclipse installed it is available from the "eclipse marketplace".

Note

Older versions of m2e do not support Maven 3.3, so once the projects are imported into Eclipse you will also need to tell m2eclipse to use the right profile for the projects. If you see many different errors related to the POMs in the projects, check that you have an up to date installation. If you can’t upgrade m2e, add the "spring" profile to your settings.xml. Alternatively you can copy the repository settings from the "spring" profile of the parent pom into your settings.xml.

Importing into eclipse without m2eclipse

If you prefer not to use m2eclipse you can generate eclipse project metadata using the following command:

$ ./mvnw eclipse:eclipse

The generated eclipse projects can be imported by selecting import existing projects from the file menu.

Contributing

Spring Cloud is released under the non-restrictive Apache 2.0 license, and follows a very standard Github development process, using Github tracker for issues and merging pull requests into master. If you want to contribute even something trivial please do not hesitate, but follow the guidelines below.

Sign the Contributor License Agreement

Before we accept a non-trivial patch or pull request we will need you to sign the Contributor License Agreement. Signing the contributor’s agreement does not grant anyone commit rights to the main repository, but it does mean that we can accept your contributions, and you will get an author credit if we do. Active contributors might be asked to join the core team, and given the ability to merge pull requests.

Code of Conduct

This project adheres to the Contributor Covenant code of conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to spring-code-of-conduct@pivotal.io.

Code Conventions and Housekeeping

None of these is essential for a pull request, but they will all help. They can also be added after the original pull request but before a merge.

• Use the Spring Framework code format conventions. If you use Eclipse you can import formatter settings using the eclipse-code-formatter.xml file from the Spring Cloud Build project. If using IntelliJ, you can use the Eclipse Code Formatter Plugin to import the same file.

• Make sure all new .java files to have a simple Javadoc class comment with at least an @author tag identifying you, and preferably at least a paragraph on what the class is for.

• Add the ASF license header comment to all new .java files (copy from existing files in the project)

• Add yourself as an @author to the .java files that you modify substantially (more than cosmetic changes).

• Add some Javadocs and, if you change the namespace, some XSD doc elements.

• A few unit tests would help a lot as well — someone has to do it.

• If no-one else is using your branch, please rebase it against the current master (or other target branch in the main project).

• When writing a commit message please follow these conventions, if you are fixing an existing issue please add Fixes gh-XXXX at the end of the commit message (where XXXX is the issue number).