DevOps Zone is brought to you in partnership with:

By day I'm a build and release engineer in London, but by night I'm a normal person! If anyone ever asks me what I do, I usually generarlise and say "I'm in I.T." and then check to see if they've already stopped listening. When I'm not working or blogging I can be found playing rugby or cycling around the countryside on my bike, in an attempt to keep fit and fool myself into thinking I'm still young. James is a DZone MVB and is not an employee of DZone and has posted 53 posts at DZone. You can read more from them at their website. View Full User Profile

Automate Configuration Management Using Tokens!

12.18.2011
| 6240 views |
  • submit to reddit

Here’s the problem:

Your application has numerous config files, and the values in these config files differ on every server or every environment. You hate manually updating the values every time you deploy your applications to a new environment, because that takes up too much of your time and inevitably leads to costly mistakes.

Here’s the solution:

Automate it.

And here’s one way of doing it:

  • Use “master” config files that have ALL environmental details replaced with tokens
  • Move copies of these files to folders denoting the environments they’ll be deployed to
  • Use a token replacement operation to replace the tokens
  • Deploy over the top of your code deployments, in doing so replacing the default config files

All the above can be automated very easily, and here’s how:
First off, make tokenised copies of your config files, so that environmental values are replaced with tokens, e.g.
change things like:

<add key=”DB:Connection” value=”Server=TestServer;Initial Catalog=TestDB;User id=Adminuser;password=pa55w0rd”/ >
to

<add key=”DB:Connection” value=”Server=%DB_SERVER%;Initial Catalog=%DB_NAME%;User id=%DB_UID%;password=%DB_PWD%”/ >

Then save a copy of these tokens, and their associated values in a sed file. This sed file should contain values specific to one environment, so that you’ll end up with 1 sed file per environment. These files act as lookups for the tokens and their values.

The sybntax for these sed files is:

s/%TOKEN%/TokenValue/i

So here’s the contents of a test environmemt sed file (testing.sed)

s/%DB_SERVER%/TestServer/i

s/%DB_NAME%/TestDB/i

s/%DB_UID%/Adminuser/i

s/%DB_PWD%/pa55w0rd/i

And here’s live.sed:

s/%DB_SERVER%/LiveServer/i

s/%DB_NAME%/LiveDB/i

s/%DB_UID%/Adminuser/i

s/%DB_PWD%/Livepa55w0rd/i

Next up, we want to have a section in our build script which renames the web_master.config files and copies them, and then runs the token replacement task….so here it is:

<target name=”moveconfigs” description=”renames configs, copies them to respective prep locations”>

<delete file=”${channel.dir}\web.config” verbose=”true” if=”${file::exists (webconfig)}” />

<move file=”${channel.dir}\web_Master.config” tofile=”${channel.dir}\web.config” if=”${file::exists (webMasterConfig)}” />

<delete file=”${channel.dir}\web.config” verbose=”true” if=”${file::exists (webconfig)}” />

<move file=”${channel.dir}\web_Master.config” tofile=”${channel.dir}\web.config” if=”${file::exists (webMasterConfig)}” />

<mkdir dir=”${build.ID.dir}\configs\TestArea” />

<mkdir dir=”${build.ID.dir}\configs\Live” />

<copy todir=”${build.ID.dir}\configs\TestArea\${channel.output.name}” >

<fileset basedir=”${channel.dir}” >

<include name=”**\*.config” />

<exclude name=”*.bak” />

</fileset>

</copy>

<copy todir=”${build.ID.dir}\configs\Live\${channel.output.name}” >

<fileset basedir=”${channel.dir}” >

<include name=”**\*.config” />

<exclude name=”*.bak” />

</fileset>

</copy>

</target>

<target name=”EditConfigs” description=”runs the token replacement by calling the sed script and passing the location of the tokenised configs as a parameter” >

<exec program=”D:\compiled\call_testarea.cmd” commandline=”${build.ID.dir}” />

<exec program=”D:\compiled\call_Live.cmd” commandline=”${build.ID.dir}” />

</target>

As you can see, the last target calls a couple of cmd files, the first of which looks like this:
xfind “%*\TestArea” -iname *.* xargs sed -i -f “D:\compiled\config\testing.sed”

xfind “%*\TestArea” -iname *.* xargs sed -i s/$/\r/

This is the sed command to read the config file, pipe the contents to sed and run the script file against it, and edit it in place. the second line handles Line Feeds so that the file ends up in a readable state. Essentially we’re telling sed to recursively read through the config file, and replace the tokens with the relevant value.

The advantage that this method has over using Nant’s “replacetokens” is that we can call the script for any number of files in any number of subdirectories using just one call, and the fact that the tokens and values are extracted from the build script. Also, the syntax means that the sed files are a lot smaller than a similar functioning Nant script would be.

Of course, you could make this whole thing even more elegant by putting the token/value pairs in a database instead of in a sed file, simply pull them out of the db at build/deploy time and then do the substitution.

People sometimes say that this method doesn’t work well if there are a large number of config files; they don’t like the idea of maintaining a large number of “master” versions as well as standard code versions. So to get around this, you can just not use tokens, but instead have the sed/replace look for the xml node and then the element, and simply replace the value there. There are plenty of ways of doing this using xml xPath. Both approaches have their own advantages, I guess the decision of which one to go for could just be a matter of how numerous your config files are.

Source:  http://jamesbetteley.wordpress.com/2010/08/09/tokens/

Published at DZone with permission of James Betteley, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Jean-Baptiste Nizet replied on Sun, 2011/12/18 - 6:44am

And why not use the replace ant task, rather than executing sed? Or even use the copy ant task with a filterset?

Marc Swingler replied on Mon, 2011/12/19 - 1:41am in response to: Jean-Baptiste Nizet

Ant is a compile time tool. He's using a tool that's available to him at install time. The installer reads his master config file and either detects the configuration it should use from the config or asks the user. In an enivronment where you have a limited number of hosts running your application, this makes a lot of sense. (e.g.: 100 hosts running a webapp)

Roman Kramer replied on Mon, 2011/12/19 - 3:40am

Indeed an example of bad advice. Why use sed instead of ANT's Replace? you will have to have sed installed in system and calling external program is not a very robust solution.

Marc Swingler replied on Mon, 2011/12/19 - 10:32pm in response to: Roman Kramer

Ant is an external program at install-time as much as sed is. Moreover a small binary version of sed could be incuded in an installer. Personally I use VBScript and XPath embeded in an installer to deploy to hundreds of servers. I've never had a problem.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.