Dror Helper 
Top DZone Article of 2011: The 7 Levels of Continuous Integration
Email
The DevOps Zone is
presented by DZone with partners including ThoughtWorks Studios and UrbanCode to bring you the most interesting and
relevant content on the DevOps movement. See today's top DevOps content and be sure to check out ThoughtWorks Studio's Continuous Delivery Whitepapers and UrbanCode's Webinars.
I’ve noticed that when other developers talk about “continuous integration” they do not always mean the same thing.
The following is an outline of the seven stages of continuous integration based solely on my own experience:
- No Server, no automation - no problems
- Have nightly build that run each day – at least we know that the project compiles
- Started writing “unit” tests – now running as part of the nightly build
- Created a build that runs on each commits – who broke the build?!
- Added static code analysis and style checking to the build
- Server Build, deploy & run the tests on several environments also used to deploy on testing (and production) environments\
- Everything is automated – Even Gui tests runs automatically on the latest code, you even got as far as testing the installer. Code is analyzed and results are sent by email. Nothing is done manually unless it is done only once…
These are mine stages as I experienced them in several jobs – what are yours?
Happy coding…
Tags:
Published at DZone with permission of Dror Helper, author and DZone MVB. (source)(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)
ThoughtWorks Studios and UrbanCode, the sponsors of the DevOps Zone, are champions of the DevOps movement. Their deployment tooling solutions focus on the entire software development lifecycle, involving all parts of an organization, which helps facilitate a migration to the DevOps philosophy.
Comments
Neil Shannon replied on Fri, 2011/05/20 - 7:21am
Jordan Lee replied on Fri, 2011/05/20 - 8:40am
Jon Davis replied on Fri, 2011/05/20 - 2:54pm
Balázs Bessenyei replied on Fri, 2011/05/20 - 4:33pm
Between 6 and 7. (The test FW and the environments is too unstable, so human verification is needed for the failed GUI tests.)
However we have a few additions.
Level 5: newly introduced serious static code analysis errors or failed unit tests also causes build failure. Email based notification for the code comiters (and SCRUM Masters, just in case the responsible has left the building before successful build, thus violating the check in process). These checks are also extended to JS and configuration files as well, not just Java.
After successful Level 5 version control Labels are updated to point to the last successful build.
This is true for Level 6 as well, after the deploy have been verified to be successful and basic functionality is working.
Patroklos Papapetrou replied on Sat, 2011/05/21 - 11:51am
We
have full automatied builds (trunk for every commit, nightly,
integration tests etc. ) , unit tests, database integration tests (
dbunit ), static analysis ( quite easy that part with Sonar ) and now
we are trying to automate some gui tests using Selenium.
You can add maybe between stage two and stage three the adoption of a developers guidelines to ensure tha continuous integration is part of the every-day development lifecycle
Edwin Quita replied on Sun, 2011/05/22 - 10:10pm
David Updike replied on Thu, 2011/05/26 - 1:01pm
9 Stages of Continuous Integration with each step building on the prior step.
1. No Server, no automation - no problems (ignorance is bliss)
2. Daily or Nightly builds – know that the project compiles
3. Builds include Automated Unit Tests (AUTs)
(AUTs run against only local/mock data)
4. Multiple Builds per Day with AUTs.
Stop-the-Line approach for failed builds.
5. Builds with AUTs on each commit.
6. Build on each Commit with AUTs, static code analysis, style checking, etc.
7. Build on each Commit with AUTs, static code analysis, style checking, etc. and select important Functional Tests (Functional Tests run against real external data)
8. Builds with AUTs, static code analysis, style checking, etc. and multiple Functional Unit Tests Suites to run at different times/environments on Commit, Nightly, Weekend, etc.
9. Everything is automated – GUI, Unit Tests, Acceptance Tests, Functional Tests, all deploys, etc. Nothing is done manually unless it is done only once.
Marko Taipale replied on Sun, 2011/05/29 - 1:47am
I think the list is missing the continuous deployment and testing the properties:
10: 9 + security, performance, scalability and failovers tested automatically (in different cadence)
11: 10 + continuous deployment (on commit / on feature push / daily).. optionally to the immune cluster
12: 11 + automated alerts and real time monitoring
13: 12 + automated rollback mechanism
Often the database and automated data migrations are not part of the CI, so that could also been taken into account as a variable in all the levels.
Eric Minick replied on Mon, 2011/05/30 - 7:31pm
in response to: mt67534
Back at the 2009 CITCON, we looked at the maturity as spanning several categories (build, deploy, test, report)- you could be more advanced on the build side than the deployment side - or deploying to production but be doing lots of manual testing.
Jeffery Fredrick and I wrote that up and converted it into a wall-chart. There's a matching white paper too (reg required for the wp).
You might be starting too advanced even. Since the first challenge is integrating the code (before testing that integration and releasing it), I always include a nod to reasonable use of source control. It's amazing, but there are still a decent number of shops whose SCM practices and tools are pretty much non-existant.
Emma Watson replied on Fri, 2012/03/30 - 6:12am