delineate.io

portray or describe (something) precisely.

Microservices Dynamic Configuration

Demonstrates of centralised dynamic configuration of microservices without downtime.

Report Bug · Request Feature

• About The Project
  • Problem with Env Variables
• Built With
• Getting Started
  • Prerequisites
  • Initialisation
• Usage
  • Customers Service Implementation
  • Running Natively
  • Running in Docker
  • Testing
  • Full Make Targets
• Roadmap
• Contributing
• License
• Acknowledgements

About The Project

This repo provides an example of using Hashicorp Consul KV store for centralised and dynamic service configuration that enables zero downtime reconfiguration of existing running microservices.

Problem with Env Variables

Config is one of the 12 factors specified building for 12 Factor Apps. Most guidance is to use env variables to achieve externalisation of configuration, such as this.

However there are some common challenges with using env variables:

• In highly configurable apps this can result in env variable sprawl
• Env variables to not have any native way to group or hierarchically structure config
• Updates to env variables do not support atomic transactions
• Most hosting platforms require redeployment to update env variables

Built With

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

The following should be installed locally.

• Homebrew
• Docker

Initialisation

Note is is assumed most developers are using MacOS, if not then some dependencies will need to be installed manually.

To initialise the local development environment run make init, this will:

• Install brew dependencies using brew bundle
• Generates a local certificate authority and certificates for consul
• Creates a python virtual environment and install PyPi dependencies
• Sets up pre-commit hooks

You can inspect exactly what gets installed by brew and pip using make list.

Usage

Customers Service Implementation

An example service written in go has been provided. The customers service leverages a set of modules for developing production services:

Go Module	Purpose
gin	Web framework used for the APIs.
gin swagger	Used to expose swagger documentation for the API.
viper	Module for configuration management.
consul structure	Specific library which exposes an interface to interact with Consul.
zap	High performance logging framework from Uber.
gorm	Native go Object Relational Mapper (ORM).
retry-go	Simple retry library from Avast.

The example service API starts on port 1102 by default. The details of the API endpoints can be explored by browsing the swagger documentation where the service is running here.

Running Natively

To run, test and debug the go service natively (not inside docker) it is necessary to have the following background services running in advance:

• consul running on port 8500
• postgres running on port 5432

Runningmake services will stand up the required backend services.

When the services are running it is possible to browse the Consul UI. You can subsequently use make ps to show the running containers.

In addition psql is installed then it's possible to connect to the database using psql -U postgres -h localhost. You will need to source .env file to avoid being prompted for the password each time.

Running in Docker

One of the key advantages of using centralised and dynamic configuration is that config is updated across multiple services. To demonstrate this a make target has been provided that instantiates three instances of the customers service in Docker behind an nginx load balancer.

# builds the image, scales to 3 by default
make build=true up

# doesn't rebuild the image, scales up 5
make scale=5 up

# make scales back down to 2
make scale=2 up

The customers service is packaged using Build Packs so there is no explicit Dockerfile.

Testing

Once the service is running either natively or deployed in Docker a set of tests have been provided. These tests use BDD and are implemented by extending behave. These tests can be run using make tests.

Full Make Targets

This section lists

Target	Purpose
make list	Lists the brew and pip dependencies that will be installed.
make init	Full initialises the local development environment.
make graph	Writes a file called graph.txt which contains the go dependency tree for inspection.
make build	Builds the application binary for the application in ./build.
make services	Creates the backend services to enable the application to be run natively.
make up	Deploys the application into Docker and scales to three instances.
make ps	Lists the primary info and status of the docker containers.
make tests	Runs the test feature files that have been provided in ./tests.
make rename	Only required in exceptional cased to rename the repo and update readme badges.

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

If you would like to contribute to any Capco Digital OSS projects please read:

• Code of Conduct
• Contributing Guidelines

License

Distributed under the MIT License. See LICENSE for more information.

Acknowledgements

• Best README Template
• Markdown Badges