Alex Collins is a enterprise Java developer and currently works in the UK. He has worked in the insurance, publishing, telecoms and supply chain. In his spare time he codes in Python, Clojure and Scala (but not at the same time) and enjoys a multitude of sports. His catchphrase is K.I.S.S! Alex has posted 20 posts at DZone. You can read more from them at their website. View Full User Profile

Elements of Build Tools

08.13.2010
| 4495 views |
  • submit to reddit

Why bother with having standards for builds? Do we need generic conventions for compiling, packaging and versioning? No need. Should you roll your own? What do we need from a build tool? What functions does it provide? Should it be all encompassing big brother or a transparent minion waiting at our beck and call?

 This is the first in a multi-part series on how and why we use build tools and the methodologies behind the various approaches. The intention is to highlight the positives and negatives behind each approach as well as realising how advanced we might be in the Java world.

In the beginning

In a non-commercial environment - think a dark basement with a genius teenager learning to hack together a kernel or the next best thing - you'd be sensible to assume that the project in development was not rife with best practices, and that maintenance and standards were not top of the TODO list.

For me a good project is one that's scientific. Something that's got requirements, clearly defined conventions, aims for all involved, sensible timescales and lengthly analysis behind each intended feature. Something of a wishlist perhaps; I'd bet on many a developer listing the same sorts of things they'd want from a project.

"Clearly defined conventions" is one element of a project that manifests itself in many different ways. From raising changes to requirements, to logging a bug, all the way to how the software is built, packaged and deployed. Convention provides a common ground for everyone involved in a project to work consistently, easily and to set expectations. Whether you're the project manager or a new developer, having conventions that everyone follows helps you to avoid pitfalls and the dreaded unknowns.

Now this isn't to say conventions should be everywhere - there should be a certain amount of 'organised chaos' and 'flow' for creativity - nor that it's a be-all-and-end-all to how you approach software development or even build processes; it's just that those tasks within a project that can be well defined, explicit and/or absolute can be documented and set in stone to allow for consistent approaches throughout your project. If a team of 15 developers all know that the project should be Interface driven by design, then you have a consistent, maintainable and testable code base from the start.

This polymorphic entity that is convention in software (a paradox then?) certainly translates to how a programmer should approach their work. One who writes code for any system needs to have a logical approach to what they're doing and this transcends to a scientific or almost religious tendency to be thorough and methodic about their work. They also need an environment that allows them to concentrate on what they do best: writing code. That means building, testing and even deploying should not be an added chore; it shouldn't distract. Getting logic right from programmer to bytes is crucial to the success of a project.

If we accept convention as a necessity in software development, and that it plays a big part in many different aspects of the role, does that mean our build processes and the tools that do it for us should be rigid and cleanly defined set of goals, or something that's fluid and dynamic, molding to your needs and wants easily and quickly?

Some projects might survive without the added pleasure (or pain) of an automated build system. I've seen a large website taking millions of hits a week built purely by a release manager once a week for a release; the developers each had their own build and it was anyone's guess as to whether the entire codebase would compile at all (in fact it frequently didn't). This can't be the only example of such a thing. Since moving away from that scenario and introducing tools such as Ant, Maven and Cruise Control, things are working better, releases have increased and there's certainly more confidence in the code, but can you really quantify any of that?

Inside the egg

At its simplest, a build tool is supposed to serve at the hands of the developer to assist in the mundane tasks of compilation and packaging of software. It exists to automate the tasks that a developer needs to perform on a regular basis that do not change - at least at a conceptual level - from start to finish. To compile, package, test or organise is not something a programmer should be worrying about on a regular basis. It's there to allow the programmer to write code.

In modern times this role - that is the role of a build tool - has grown to mean much more. Such a tool should now resolve dependencies, distribute versioned packages of the software, deploy, unit test,  and even clean up after us. There is little wonder then that this is now a complex and large subject - an array of books have been published throughout the years promising to solve the riddle.

Environment

Much of what we do as professional programmers will be dictated to by our superiors or the goals of the business. That's assuming we're talking about commercial/professional environments (which I am).  We therefore may not have a choice about how we approach building software. Furthermore, the type of approach that would be deamed appropriate depends on the requirements or the needs of the business, the teams involved in building (from developers to designers and testers) as well as budgets, time and skillset.

For the sake of this article I will define the following 'environment' for discussion:

  • A medium sized programmer team, of 15+ developers. New programmers come and go on a monthly basis
  • New programmer up-take needs to be high; should be up-and and running within hours
  • The software is a very large website, ficticious
  • The build process is entirely up to the development team
  • Rate of change is high; multiple branches

So given this environment, how do we approach our build process? What tools should we use? Do we have a conventionalised approach by using something like Maven, or do we go and roll our own and use Ant or Gradle?

A common ideology perhaps not unique to programming is to start out strong: have something concrete from the ground-up that you can build on easily and cleanly. Maven certainly satisfies this, but what about Ant for instance?

Ant on the other hand would mean extra (although simple) work from the word go. We'd have to define our targets, write them ourselves, but before any of that we'd need to think about what functionality we'd want. It's at this point we may realise it's a common goal we're solving and you could argue we shouldn't reinvent the wheel.

Why maven?

Computing and more specifically programming is a science. Science needs standards, rules, methodology and documentation. Maven certainly satifies the more rigid and conventionlised approach to development in our environment. It gives us predefined notions such as goals, snapshots, versioning and the like. Once you understand this you don't have to learn much else. 

Regardless of where you go, should the shop you end up at be using maven you can be up and running in minutes. You know how to compile, package and you don't have to think much about versioning or even deployment.

That said though, at what price does this come? If you know maven at all you'll have heard this a lot of times: it doesn't give kindly to a customised or a build setup that's not maven's doing. I'm not sure what weight this argument has these days. Maven's perfectly capable of being told what's what and where, and getting along fine without needing to dictate where resources or code is located. In fact I've seen examples of projects that go totally against the maven idea of src/main and resources in a similar structure and yet it still worked.The typical argument to this "rigidity" - which I maintain is a plus - is that you want that structure so you maintain consistency throughout your modules and therefore your whole codebase. A new developer only has to understand maven, not your entire codebase from the word go.

When starting new projects or modules for an existing one, you can get up and running within minutes by using maven's extensive archetype system that is for lack of a better phrase a templating system for project structure. It's simple yet powerful and can aid in consistency and expectations within an organisation; new starts know where they stand!

Maven's object oriented ethos lends to a modular approach to how you develop software, and this is arguably the best solution to large software projects. The code is broken up or orthoganol; its concerns are isolated and kept simple. Encouraging this ideology - which is what I believe Maven does - is certainly something that should remain in all software projects.

The dependency resolution and management aspects are also a superior feature of maven that cannot be ignored. Transient dependencies are resolved without intervention from the user. This powerful function allows a developer to declare just the dependency they want and not have to worry about hunting down the various sub-dependencies that they would need. Using Spring for instance would require a multitude of dependencies just to get started; maven finds these all for you without a problem. Combine maven with a repository proxy like Nexus and you have a powerful combination indeed.

Another plus of maven is its plugin architecture. For many aspects of the build process there's a plugin to cover it. You can propogate version numbers throughout your modules, package your software in a number of spec-defined ways; generate entire sites from documentation, and even generate project files for your favourite IDE. The list goes on. 

You of course cannot ignore the open nature of maven, which is a tool that's been round for many years and is an excellent example of open source software that works very well.

To conclude, out of the box maven gives us the following:

  • structured andconsistent approach to organising and managing software code
  • enforces best practics by always running unit tests
  • superior dependency resolution
  • quick uptake for newcomers
  • popular/industry recognised approach usable by anyone
  • open source

With something like maven comes a required set of skills; for large projects some investment in configuration, but its pros certainly outweigh the cons. If you need a well defined and conventionalised approach to development in your organisation then maven may be worth a shot.

In the next installment I'll cover alternative build methods and conclude with what I believe is the optimum solution to building scalable, maintainable and reliable software. 

Published at DZone with permission of its author, Alex Collins.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Jilles Van Gurp replied on Fri, 2010/08/13 - 1:16pm

The notion of a build process assumes that there is something to build and that that something is somehow complicated/tedious/difficult.

This is indeed true for Java but not inherently so. Maven is a workaround, not a solution here (and hardly the best workaround I would argue)

Why do we compile? Because JVMs consume byte code. What language is the compiler written in? Well, Java of course. So why not integrate the compiler with the jvm and just treat the output as something transient and conveniently cacheable? Really, there's no reason for a Java file to be different than a jsp file (which is actually compiled to Java source at run time before being fed into a Java compiler, at run time). Have you ever heard of people pre-compiling jsp files? It's possible but most would consider that a waste of time.

Enter the play framework for Java which reduces the edit/compile/test/package/integration-test/deploy/manual test cycle to : edit, F5 (browser refresh), and run the tests (unit, integration and/or manual) in the browser. It's Java. It's still statically compiled. It just happens to be automated the way it should be. It's not for anyone but it is a nice example of how much simpler our lives could be without maven. Integrating runtime and build tools reduces the build tools to a just in time optimization. If practical you may sometimes want to do that ahead of time but not all of the time.

90% of what maven does is just moving files around. That includes downloading them or creating them, copying them from A to B, archiving them, in some cases unarchiving them, and uploading them to servers. If you are developing that is technically a waste of time that comes out of your very expensive budget of time you have reserved for coding rather than watching the maven console.

Get rid of the stupid moving files around and you no longer need rediculously complicated build tools. We need to get rid of the need for build tools. Maven is part of the problem, not the solution.

Alex Collins replied on Sat, 2010/08/14 - 4:58am in response to: Jilles Van Gurp

Thanks for reading.

 The problem with what you're saying is;

  1. You're assuming JSPs are a problem and in fact that this is entirely on the web tier. It isn't.
  2. Pre-compiling JSPs is actually a very useful integration test to ensure changes don't break what's already working. You'd only notice this at runtime otherwise. Granted, few do it.
  3. Maven does a lot more than "moving files around" and without maven doing it seamlessly and unobtrusively for you you'd have to do it yourself. It's not a problem to need a dependency ("file" as you put it) and having to obtain that dependency is part of the development process.

Mladen Girazovski replied on Mon, 2010/08/16 - 2:37am in response to: Alex Collins

Pre-compiling JSPs is actually a very useful integration test to ensure changes don't break what's already working. You'd only notice this at runtime otherwise. Granted, few do it.

There is even a couple of Maven plugins that compile your JSPs, very useful if you want to find out if the JSPs are still compilatbly after refactoring or other changes. No need to include the compiled versions in the war, just compiling them is a valuable safety net that saves time compared with the usual manual testing.

 

Rick Siskey replied on Thu, 2011/10/27 - 12:13am

Wow, I can see there is conflicting opinion regarding certain issues. But I have to agree that both of them are geniuses on their own right. So keep on debating, who knows you might get something good from it. - Rick Siskey

Rick Siskey replied on Thu, 2011/10/27 - 12:15am

. - Rick Siskey

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.