In this lecture we will examine Continuous Integration (CI), how it allows us to automate our build processes, and how it can provide feedback. The aim is to explore why CI is useful, and the process of undertaking CI. Most of these ideas are built into Travis CI, which makes our life easy. Here, we explore the best practice methods in more detail.
- Define Continuous Integration.
- Explain why Continuous Integration is useful.
- Describe the Continuous Integration workflow.
- Describe risk in software development and how this is managed via Continuous Integration.
Build Software at Every Change.
-Continuous Integration.
Martin Fowler has defined Continuous Integration as a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. This allows teams to reduce integration problems and develop cohesive software faster.
CI is therefore the practice of performing integration upon every commit. So that raises the question, what is integration?
Integration is the process of combining the software components of a system together: integrating them. The combination of these components results in an integration build. We perform such a build for two purposes:
- To perform integration tests.
- To deploy the software.
This is one of the key ideas of this lecture. An integration test is one that is performed when the system is integrated. From Wikipedia:
Integration testing (sometimes called integration and testing, abbreviated I&T) is the phase in software testing in which individual software modules are combined and tested as a group. It occurs after unit testing and before validation testing.
So integration tests are tests performed on combined software modules rather than unit tests which target single methods and classes.
CI provides feedback, which is the Second Way of DevOps:
Continuous Integration increases your opportunities for feedback.
-Continuous Integration
CI provides the following value:
- It reduces risk.
- It reduces the number of repetitive manual processes.
- It allows us to create deployable software at any point - or be potentially shippable.
- It makes the project output more visible.
- It creates greater confidence in the product being developed.
Risk we will cover in more detail later. A key addition is automating processes. It is often the case that software engineers create software to automate other users' processes, but not their own. CI is about automating some of our processes. In particular, we want the delivery of software to be as simple as possible:
The development of software may be complex, but the delivery of software must be a push-button affair.
-Continuous Integration.
Automated build scripts reduce the manual, repetitive and error-prone processes development and operations have to perform. We have mainly been using Maven and Travis for this. We also want our builds to be fast: we want to builds to fail fast so we get the rapid feedback to resolve problems.
And finally, we enable information sharing. Our CI system can provide metrics that we visualise or share with the necessary team members to better understand the complexity of our work. However, we must remember to not spam information everywhere. If everyone gets all the feedback it just means everyone will ignore it.
The practice of CI is more than just a tool such as Travis CI. It requires dedication from the development team to undertake certain practices such as frequent commits, immediately fixing broken builds, and having an integration system. All of this is automated as far as possible. The general principles of CI are:
- Developers commit their code to a centralised version control at least once per day.
- A separate build machine performs integration builds several times per day.
- 100% of tests must pass for every build.
- A deployable product is created for functional testing.
- If the build breaks then the highest priority is to fix it.
- Developers review reports generated by the build system such as coding standards and dependency analysis looking for improvement.
The point is as developers you have to live these practices. Saying that CI is good and undertaking the practice of CI are different. However, although developers agree that CI and other practices (such as code reviews and testing) are good, they do not practice these principles often.
So as a software engineer your role is to practice CI. There are seven general practices that work well:
- Commit code frequently.
- Don’t commit broken code.
- Fix broken builds immediately.
- Write automated developer tests.
- All tests and inspections must pass.
- Run private builds.
- Avoid getting broken code.
Get everything you need into source control and get it so that you can build the whole system with a single command.
-Martin Fowler
Continuous Integration starts when a developer commits code to the repository. A general workflow is:
Source: www.pepgotesting.com.
The steps in a CI scenario will typically go something like this:
- The developer commits changes to source control.
- The CI server polls for and receives the changes.
- The CI server builds the software.
- The CI server runs automated tests on the build.
- The build and tests either succeed or fail.
- The team are notified about the status of the build and tests.
We have already built this workflow via Travis CI with very little effort. The key aspect of CI is again the practice: developers must commit their code frequently.
Remember that our CI system provides feedback on the status of our builds and tests. We want to know as soon as possible if there is a problem with our build and prioritise fixing the build if there is a problem. By running our automated build and test whenever we commit a change we can answer the following questions (taken from Continuous Integration: Improving Software Quality and Reducing Risk):
- Do all the software components work together?
- What is my code complexity?
- Is the team adhering to the established coding standards?
- How much code is covered by automated tests?
- Were all the tests successful after the latest change?
- Does my application still meet the performance requirements?
- Were there any problems with the last deployment?
This is beyond a test for the software being successfully built. From our CI system we can know how well our software was built. We can spot defects early. Continuous building ensures our feedback is quick and happens at every change of the system.
Some CI systems are visualised in the real-world. For example, a build-light indicator might exist to provide clear global signalling of the build status:
By Dushan Hanuska - https://www.flickr.com/photos/hanuska/5931613961/in/photostream/, CC BY-SA 2.0, Link
CI is not just a technical implementation; it is also an organizational and cultural implementation.
-Continuous Integration.
As stated, CI requires developers to change how they work. Developers must commit code frequently, prioritise broken builds, create automated build scripts with tests that always pass, and not commit or fetch broken code to/from version control.
Incremental development of the CI system is important. In a large codebase, going from no CI to full CI in one step will fail. Rather, choose a lower occurrence build, such as a nightly build. As the team gets comfortable with the practice, we can increase the number of builds until we are fully undertaking CI.
A CI system only requires four features (from Continuous Integration: Improving Software Quality and Reducing Risk):
- A connection to a version control repository.
- A build script (e.g., we use Maven and Travis).
- Some sort of feedback mechanism (such as e-mail).
- A process for integrating the source code changes (manual or CI server).
Once the team have started to adopt CI practices, and the CI system is setup, the following steps are used to ensure we maintain a good CI workflow:
- Identify a process that requires automation (e.g., compilation, test, inspection, deployment, database integration, etc.).
- Build a script to support the automation making it repeatable and consistent.
- Share via version control so others can use the scripts.
- Make it continuous by running the script with every change applied.
We can further improve our CI by asking the following questions:
- How much code coverage do you have with your tests?
- How long does it take to run your builds?
- What is your average code complexity?
- How much code duplication do you have?
- Are you labelling your builds in your version control repository?
- Where do you store your deployed software?
These questions allow us to continuously improve our CI method. We are asking the questions about the quality of our software that we are building.
A build acts as the process for putting source code together and verifying that the software works as a cohesive unit.
-Continuous Integration
Building software is the first stage of our CI system after it has fetched changes from version control. But what is a software build? Is it just compiling software? Is compiling and testing? What other automated processes should be used? The answer is up to the team. A key point is that we want builds to be fast so that we can get the feedback.
The aim is to have a build script that contains all of the processes we have decided to automate. The script also needs to be separate from any IDE. These steps ensure that:
- Developers can use different IDEs.
- The CI server can run the script without intervention.
A basic build script should perform the following steps:
- Clean.
- Compile source code.
- Integrate database.
- Run tests.
- Run inspections.
- Deploy software.
As discussed, developers run separate builds on their own machines before committing their code. This prevents broken builds being pushed to source control. The steps for running a private build are:
- Check out code from version control.
- Make changes to that code.
- Get the latest changes from version control.
- Build, including unit tests.
- Commit changes to version control.
Our GitFlow workflow mimics this idea:
- Create a new
featurebranch fromdevelop. - Implement the feature.
- Merge any changes in
developintofeature. - Build, including unit tests.
- Merge
featureintodevelop.
Once code is pushed to version control and fetched by the CI server, an integration build occurs. This includes all the components in the project, such as other software components, databases, etc. The aim is to ensure that all the components work together.
For deployable software we need to create a release build. A release build can happen at the end of milestone, can include performance (nonfunctional) tests, and must include any acceptance tests. The aim is to create installation media and prepare for full QA testing. When we examine Continuous Delivery (Lecture 16) we will cover more detail of this practice.
As stated, CI reduces risk by continuously building and integrating our software. Risk is the potential for a problem to occur. When a risk occurs it becomes a problem. We focus on the high-priority risks (most damaging) that are most likely to occur.
Agile itself reduces risk as we are always delivering value to the customer and receiving feedback. In traditional approaches, our risk builds over time as we do not know if we are delivering the right product. Resources have been committed without knowledge if we are delivering what the customer wants. In Agile, small iterations mean we reduce risk. The below image illustrates:
By Christopher Little - Own work, CC BY 3.0, Link
There are four key risks addressed by Continuous Integration:
- Lack of deployable software.
- Late discovery of defects.
- Lack of project visibility.
- Low-quality software.
Let us look at each of these risks in turn.
Our goal is to always have potentially shippable software. Therefore, having no deployable software is a risk to our development process. We do so by:
- Decoupling the IDE from the build process.
- Placing all database artefacts in version control.
- Rebuilding the database from the build script.
- Testing and inspecting the database.
- Automating the build process to reduce wasted time and effort.
We automate regression testing to find defects quickly. We do so by:
- Writing test code for all source code.
- Running tests in the build script.
- Running tests continuously at any change committed to version control
Once tests are written, we check code coverage to ensure as much code as possible is tested. Local unit tests and integration tests should cover most, if not all, our code.
Remember we want to visualise both the work that is happening and the state of our system. A CI system can provide useful telemetry about the state of our build. Charts can be produced by the CI system that provides us with confidence about our software.
Complex code, code that does not follow the architectural design, and code duplication all lead to defects in the software. Rather than writing a report about the quality of the software, we can run a tool as part of the build. These automated inspection tools can provide vital metrics on our software quality as we perform CI.
To summarise, CI overcomes main software risks as follows:
| Risk | Mitigation |
|---|---|
| Lack of deployable software | CI system builds software at any time via repeatable process from version control. |
| Late discovery of defects | Run tests at every change. |
| Lack of project visibility | Software health known at all times via continuous builds. |
| Low-quality software | Tests and inspections run at each change to maintain high standards. |
In this lecture we have covered the fundamentals of CI. Specifically, we:
- Defined Continuous Integration as the process of performing integration building and testing on each change committed to version control.
- Explained why Continuous Integration is useful, focusing on risk reduction and workflow.
- Described the Continuous Integration workflow, looking at the steps and CI system setup.
- Described risk in software development and how this is managed via Continuous Integration, focusing on lack of deployable software, late discovery of defects, lack of project visibility, and low-quality software.
Continuous Integration: Improving Software Quality and Reducing Risk by Paul M. Duvall is the go to book on Continuous Integration practice.
