John is an experienced consultant specialising in Enterprise Java, Web Development, and Open Source technologies, currently based in Sydney, Australia. Well known in the Java community for his many published articles, and as author of Java Power Tools and Jenkins: The Definitive Guide, and founder of the open source Thucydides Automated Acceptance Test Library project, John helps organisations to optimize their Java development processes and infrastructures and provides training and mentoring in agile development, automated testing practices, continuous integration and delivery, and open source technologies in general. John is the CEO of Wakaleo Consulting, and runs several Training Courses on open source Java development tools and best practices. John is a DZone MVB and is not an employee of DZone and has posted 125 posts at DZone. You can read more from them at their website. View Full User Profile

Build Pipelines with Jenkins/Hudson

  • submit to reddit

This article is an extract from the upcoming book Jenkins: The Definitive Guide, to be published in the coming months with O'Reilly..

One of the more interesting plugins to emerge over the last few months is the Build Pipeline plugin, written by the folks at Centrum Systems. The Build Pipelines plugin takes the idea of build promotion further, and helps you design and monitor deployment pipelines. A deployment pipeline is a way of orchestrating your build through a series of quality gates, with automated or manual approval processes at each stage, culminating with deployment into production.

The Build Pipeline plugin provides an alternative way to define downstream build jobs. A build pipeline, unlike conventional downstream dependencies, is considered to be a linear process, a series of build jobs executed in sequence.

To use this plugin, start by configuring the downstream build jobs for each build job in the pipeline, using the 'Build other projects' field just as you would normally do. The Build Pipelines plugin uses the standard upstream and downstream build configurations, and for automatic steps this is all you need to do. However the Build Pipeline plugin also supports manual build steps, where a user has to manually approve the next step. For manual steps, you also need to configure In the Post-build Actions of your upstream build job: just tick the 'Build Pipeline Plugin -> Specify Downstream Project', select the next step in your project, and tick the 'Require manual build executor' option, as shown here.

Configuring a manual step in the build pipeline

>Once you have set up your build process to your satisfaction, you can configure the build pipeline view. You can create this view just like any other view.

Creating a Build Pipeline view

There is a trick when it comes to configuring the view, however. At the time of writing, there is no menu option or button that lets you configure the view directly. In fact, you need to enter the URL manually. Fortunately, this is not difficult: just add "/configure" to the end of the URL shown when you are displaying this view. For example, if you have named your view "phoenix-build-pipeline", as shown here, the URL to configure this view would be "http://my_hudson_server/view/phoenix-build-pipeline".

Configuring a Build Pipeline view

The most important thing to configure in this screen is the initial job. This marks the starting point of your build pipeline. You can define multiple build pipeline views, each with a different starting job. You can also configure the maximum number of build sequences to appear on the screen at once.

Once you have configured the starting point, you can return to the view to see the current state of your build pipeline. Jenkins displays the successive related build jobs horizontally, using a color to indicate the outcome of each build. There is a column for each build job in the pipeline. Whenever the initial build job kicks off, a new row appears on this page. As the build progresses through the successive build jobs in the pipeline, Jenkins will add a colored box in the successive columns, indicating the outcome of each stage. You can click on the box to drill down into a particular build result for more details. Finally, if a manual execution is required, a button will be displayed where the user can trigger the job.

Configuring a Build Pipeline view

This plugin is still relatively new, and does not integrate with all of the other plugins we have seen here. In particular, it is really designed for a linear build pipeline, and does not cope well with branches or parallel build jobs. Nevertheless, it does give an excellent global vision of a build pipeline.



Published at DZone with permission of John Ferguson Smart, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Robert Greathouse replied on Fri, 2011/03/11 - 10:35am

Is there a way for these pipelines to share (or move) artifacts from one workspace to the other? I would like the .ear file that is generated in the phoenix-web-tests to be the exact same .ear file that moves to phoenix-test-deploy, phoenix-uat-deploy and ultimately phoenix-production-deploy. Right now, each project must download the code and rebuild the artifacts.

Geoff Bullen replied on Fri, 2011/03/11 - 8:33pm

Hi Robert, There is some discussion on this going on in the comments to our blog which rather than repeat, I'll point you at -

Eirik Maus replied on Sat, 2011/03/12 - 6:33pm in response to: Robert Greathouse



I guess the URL-change build trigger (-plugin?) was made for that kind of duct-tape-plumbing: It monitors the last-modified-time of any file or http url. You could for instance monitor the maven-metadata.xml-file for a dependency build by a different hudson instance. Or you could monitor the upload directory for a new batch of test data or anything like that. I think pull-based triggering mechanisms for build steps feel less coupled and create less mishaps in transitive build step dependencies. 

John Ferguson Smart replied on Sun, 2011/03/13 - 3:37am in response to: Robert Greathouse

You can copy artifacts from one build to another quite easily with the Copy Artfacts. I discuss this technique in some detail in the next chapter of 'Jenkins: The Definitive Guide', which should be available shortly.

Guillaume Jeudy replied on Mon, 2011/03/14 - 3:30pm

Thanks for your post. I was excited to try out this plugin. Unfortunately it seems to have some bugs. All the images had broken links. Moreover it does not seem to lend itself well when each module are top-level maven project that need to be built independently. Ideally I would group all the module builds in the same stage. Today it seems like each module build corresponds to a separate stage.

Geoff Bullen replied on Thu, 2011/04/14 - 1:35am

Version 1.1 has been released. This fixes some major defects, but also supports forked (but not joined pipelines). The GUI has had a major overhaul as well. Please try it out! Gives us feedback as blog comments, or raise issues for bugs and new features... Release notes are here:

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.