Top DZone Article of 2011: The 7 Levels of Continuous Integration
I’ve noticed that when other developers talk about “continuous integration” they do not always mean the same thing.
The following is an outline of the seven stages of continuous integration based solely on my own experience:
- No Server, no automation - no problems
- Have nightly build that run each day – at least we know that the project compiles
- Started writing “unit” tests – now running as part of the nightly build
- Created a build that runs on each commits – who broke the build?!
- Added static code analysis and style checking to the build
- Server Build, deploy & run the tests on several environments also used to deploy on testing (and production) environments\
- Everything is automated – Even Gui tests runs automatically on the latest code, you even got as far as testing the installer. Code is analyzed and results are sent by email. Nothing is done manually unless it is done only once…
These are mine stages as I experienced them in several jobs – what are yours?
Happy coding…
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)
Tags:
Comments
Neil Shannon replied on Fri, 2011/05/20 - 7:21am
Jordan Lee replied on Fri, 2011/05/20 - 8:40am
Jon Davis replied on Fri, 2011/05/20 - 2:54pm
Balázs Bessenyei replied on Fri, 2011/05/20 - 4:33pm
Between 6 and 7. (The test FW and the environments is too unstable, so human verification is needed for the failed GUI tests.)
However we have a few additions.
Level 5: newly introduced serious static code analysis errors or failed unit tests also causes build failure. Email based notification for the code comiters (and SCRUM Masters, just in case the responsible has left the building before successful build, thus violating the check in process). These checks are also extended to JS and configuration files as well, not just Java.
After successful Level 5 version control Labels are updated to point to the last successful build.
This is true for Level 6 as well, after the deploy have been verified to be successful and basic functionality is working.
Patroklos Papapetrou replied on Sat, 2011/05/21 - 11:51am
We
have full automatied builds (trunk for every commit, nightly,
integration tests etc. ) , unit tests, database integration tests (
dbunit ), static analysis ( quite easy that part with Sonar ) and now
we are trying to automate some gui tests using Selenium.
You can add maybe between stage two and stage three the adoption of a developers guidelines to ensure tha continuous integration is part of the every-day development lifecycle
Edwin Quita replied on Sun, 2011/05/22 - 10:10pm
David Updike replied on Thu, 2011/05/26 - 1:01pm
9 Stages of Continuous Integration with each step building on the prior step.
1. No Server, no automation - no problems (ignorance is bliss)
2. Daily or Nightly builds – know that the project compiles
3. Builds include Automated Unit Tests (AUTs)
(AUTs run against only local/mock data)
4. Multiple Builds per Day with AUTs.
Stop-the-Line approach for failed builds.
5. Builds with AUTs on each commit.
6. Build on each Commit with AUTs, static code analysis, style checking, etc.
7. Build on each Commit with AUTs, static code analysis, style checking, etc. and select important Functional Tests (Functional Tests run against real external data)
8. Builds with AUTs, static code analysis, style checking, etc. and multiple Functional Unit Tests Suites to run at different times/environments on Commit, Nightly, Weekend, etc.
9. Everything is automated – GUI, Unit Tests, Acceptance Tests, Functional Tests, all deploys, etc. Nothing is done manually unless it is done only once.
Marko Taipale replied on Sun, 2011/05/29 - 1:47am
I think the list is missing the continuous deployment and testing the properties:
10: 9 + security, performance, scalability and failovers tested automatically (in different cadence)
11: 10 + continuous deployment (on commit / on feature push / daily).. optionally to the immune cluster
12: 11 + automated alerts and real time monitoring
13: 12 + automated rollback mechanism
Often the database and automated data migrations are not part of the CI, so that could also been taken into account as a variable in all the levels.
Eric Minick replied on Mon, 2011/05/30 - 7:31pm
in response to:
Marko Taipale
Back at the 2009 CITCON, we looked at the maturity as spanning several categories (build, deploy, test, report)- you could be more advanced on the build side than the deployment side - or deploying to production but be doing lots of manual testing.
Jeffery Fredrick and I wrote that up and converted it into a wall-chart. There's a matching white paper too (reg required for the wp).
You might be starting too advanced even. Since the first challenge is integrating the code (before testing that integration and releasing it), I always include a nod to reasonable use of source control. It's amazing, but there are still a decent number of shops whose SCM practices and tools are pretty much non-existant.
Emma Watson replied on Fri, 2012/03/30 - 6:12am
Carla Brian replied on Sun, 2012/05/27 - 9:41am