Agile Zone is brought to you in partnership with:

Julian writes by night as The Build Doctor, a poorly disguised non de plume. By day he is a freelance build engineer, systems administrator and all round useful guy. Julian lives near London with his fiance and children. He enjoys balancing a love of real ale and Indian food with cycling. Julian has posted 16 posts at DZone. View Full User Profile

Supporting Multiple Environments – Part 4

04.21.2010
| 3468 views |
  • submit to reddit

In the final installment, I’m going to talk about how to share configuration between developer level environments on through to clustered or “stack” type environments.

Recycling Configuration

Ok – so now your configuration is its own standalone Maven module with its own series of branches and build process.  But what if there are shared items between the generic configuration for both development boxes and your deployment environments?  This is where my little Maven plugin comes in.  With the configuration stored in property files and built and treated as a very simply library and deployed to your repository manager, you can then put a dependency on this configuration jar.

My simple plugin reads in the development configuration as ordered and outlined above and jams the resulting property set into the Maven project’s property set.  I bound my configuration plugin to the validate phase and this allowed the “process-resources” phase to leverage these settings.

One thing I haven’t had a chance to implement is the actual recycling of the code that does the layering.  Ideally, the plugin that does the layering would live close to (or be part of) the actual configuration project so when a new layer is added (or one removed), you could address this in a single location versus updating N number of deploy scripts.

Caveats

There are a few problems this overall concept brings with it.  Here are a few of the main ones I’m aware of (that haven’t already been addressed):

- Confusion on overlay procedure – Of course you’ll have to explain how various layers of configuration get compressed into a single flat configuration.  The  plugin as written allows the user to print out what the final (local) configuration will look like.  The deploy process similarly generates a flattened property file that is processed in isolation and gives people who deploy an idea of the flattened landscape.  A well annotated set of source files as well as detailed “mvn site” output is also strongly suggested (you are publishing site documentation, aren’t you?  Hudson makes it really easy, just do it!).

- Who’s allowed to touch which layer  - I mentioned this above, but pushing each deployment specific bundle of property files into a single directory within the configuration project allows (in Perforce at least) your SCM admins to restrict access by group to various bundles.  We block QA from touching any configuration outside of their own stacks but let them have read only access to everything else for example.

- Preventing duplication and unnecessary overrides – A few times during testing, we noticed a property set at a low level (in the general bucket) and at an another deeper level.  Same property, same value.  Ideally, we’d have some set of unit testing to go through various combinations of properties and verify that this doesn’t happen.  Also, in a perfect world, there would be a report tied to generating the config jar that would say how many times a particular value comes up across various stacks.  This would highlight things such as a port number that was initially introduced in testing on a particular deployment environment and rather than pushed higher in the config tree, it was simply replicated.

- Migrating from settings.xml or profiles to property based configuration – This was one of the larger challenges, how to take everything that has been established and move it to an entirely new way of thinking and construction?  The solution for us was a one-off Maven plugin that could look at profiles (and a combination of profiles) and generate the various files listed above.

If you’ve followed me this far, you’ll see very little configuration the further you get out in the tree (if you find you need all the layers at all).  You can also imagine breaking the configuration into smaller more specific chunks.

This may also this give you a chance to also revisit what things deserve a property and what doesn’t (there’s a tendency to make everything under the sun configurable).

Hopefully this gets your gears turning and helps you find an efficient solution for managing the configuration for your product.
________

Related posts:

  1. Supporting Multiple Environments – Part 3 Supporting Multiple Environments – Part 3 (part one and...
  2. Supporting Multiple Environments – Part 2 (see part one of this series here) In this...
  3. Supporting Multiple Environments One of the longest running debates I’ve watched in...
  4. The hidden cost of building Thanks to EJ Ciramella for this thought provoking post....
  5. Clover plugin for Hudson The Atlassian guys emailed me a couple of days...

Related posts brought to you by Yet Another Related Posts Plugin.

 

References
Published at DZone with permission of its author, Julian Simpson. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Sindy Loreal replied on Sat, 2012/02/25 - 8:51am

but what if you need to build a single 200 mb deployable unit? I'm not building that 30 – 40 times activating profiles for each possible deployment. This is one of the approaches that works for small companies, but not large ones with tons of deployment configurations…

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.