Steven has posted 36 posts at DZone. View Full User Profile

What's Wrong with Build Systems in Java Today?

02.17.2008
| 14238 views |
  • submit to reddit

I think there's something fundamentally wrong with all build systems for Java that are available today.

The key point is not whether Ivy dependency management is better than Maven's. It's not whether Maven's plugins are good enough or not. And it's not that Maven's conventions can be mimicked by any other tool, even Ant.

The key points why build systems for Java are fundamentally broken are:

  • They're not as efficient as they should be
  • They're restrictive, especially for multi-project builds

Now I know that many people are happy with Ant. And I know many people are happy with Maven 2. And I'm happy for them. If you've found what you expect from a build system in Ant, Maven 2 or any other tool you should treasure it.

But there are many people who have not yet found what they need, especially people doing multi-project builds. For any serious software development effort the build system is a vital tool, a tool that can either restrict you or help you move ahead. Move ahead of your competitors for example.

If you're interested in build systems and multi-project builds then read this excellent post by Sherali Karimov on how Atlassian uses Maven 2. Although it's only the first post of four it gives you an idea of how important their build system and build configuration is to support their way of working. I'm sure it gives them an edge over their competitors. Or at least, if they would lose control over their build it would probably affect their progress, quality of their work, release management and with that their competitive advantage.

They're using Maven 2 and they seem to be doing okay. Yet we know from Don Brown's interview here on DZone - who also works for Atlassian - that they still lose time because of Maven 2.

The people at Atlassian have clearly made a dedicated commitment to get their build working the way they want it to. They're using Maven 2 but I'm sure that if they would have found a better tool they would be using that. In other words: they don't care which build tool they're using. They'll use the one that fits them best. It's today's reality that made them pick the tools that bothers them least.

How many people can you dedicate to set up your build tools according to your needs? I'm sure Atlassian would prefer to put these people on actual software development if a tool better than Maven 2 would require less effort. Every company or group of people serious about software development like Atlassian will indeed put in the effort it takes to get the build they need. It becomes wasteful if everybody serious about software development has to repeat these efforts.

What we need is a build tool that requires much less effort to configure it for anybody's needs. I know the limitations of Maven 2 and I know Atlassian would have a more flexible and easier to maintain build if they wouldn't be faced by those limitations. Yet they settle for what they have and I'm sure that after the time and efforts they've already invested they're happy with their build.

I'm also sure that if a new build tool would become available, say in the course of 2008, that has no limitations yet has similar or better conventions than Maven 2 they would certainly consider it. And if this new build tool would seamlessly integrate with their current Maven 2 build, maybe they would be even more eager to have a look. And if this new build tool would make their current Maven 2 build more powerful and versatile they may actually get excited.

What we need is a build system that marries Ant and Maven 2:

  • A dependency programming framework inspired by Ant and Rake
  • Maven 2's conventions by default, configurable if desired
  • Domain models for the build domain that are lightweight, extensible and replaceable
  • Re-use of Ant tasks where it makes sense
  • Tasks that are at least as easy to develop as Ant tasks
  • Plugins that are a magnitude easier to develop (and maintain) than Maven 2 plugins
  • Better-than-Maven 2 JAR and project dependency management
  • Integration of Ant, Maven 2, Gant, Buildr, Rake, Grails, ... projects in multi-project builds
  • No XML
  • Support for many dynamic languages + Java

Such a build system will become available in 2008 and this list is just a teaser. This build system will offer features most developers probably have never thought about.

2008 will be an interesting year for people interested in build systems. Check back to DZone to hear the news first.

Happy building!

Published at DZone with permission of its author, Steven Devijver.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags:

Comments

Fred Grott replied on Sun, 2008/02/17 - 7:14am

 I would say its lack of understanding of wonderfull ANT task components that could make the task easier. For example, how many people are aware that you can use Antlion ANT Taks to artifact both the build ouputs and the 3rd party libraries?

And it allows integration with Apache IVY and or Apache Maven repositories..

 

 

Fred Grott(aka shareme) sometimes a JavaZone(JavaLobby) and EclipseZone contributor. Top visited blog on Jroller.com at: http://www.jroller.com/shareme/

Thomas Mueller replied on Sun, 2008/02/17 - 9:33am

I would add:

  • Debuggable
  • Autocomplete support
  • Easy to integrate in Java
  • No new language to learn
That leaves only Java as the build language. The project definition would be written in Java. It wouldn't need to be fully procedural.

Zqudlyba Navis replied on Sun, 2008/02/17 - 9:46am

The problem with the build system today is XML HELL itself.

XML is verbose, hard to read, and very unplesant to program in.

When I write ANT scripts, I don't feel like I'm programming, but fighting with configuration.

Is there somebody out there building a build+dependency framework that allow you to feel like a "real programmer" by having :

  • static typing
  • IDE support
  • compile time checking and
  • refactoring

?

 

phil swenson replied on Sun, 2008/02/17 - 10:31am

Using Rake made me realize how badly ANT sucks.  I agree - XML is a horrible way to write build configurations.

Imperitive programming in XML is simply retarded.  Even the guy who started ANT (James Duncan Davidson) doesn't like it any more.  He's now a rake fan.

 Gant looks promising.

Chris Hubick replied on Sun, 2008/02/17 - 1:53pm

We need a build system that is *declarative* - one where you tell it about the *what* and not the *how*.

This is why Maven is heading in the right direction over Ant. It's like semantics on the web, where you write "h1" instead of "times 24 pt" in your HTML, and the browser figures out how best to display that in the given environment. With Maven, if I use their standard directory layout, it can build and package my software almost automatically.

But sadly, I am still forced to write a lot of stuff into my project descriptors that I shouldn't have to... like for generating an assembly or an RPM - these should just work out of the box. I just want to label my project as "webapp", "library", "user application" or whatever, tell it my dependencies, and have it all just work.

Steven Devijver replied on Sun, 2008/02/17 - 2:28pm in response to: Chris Hubick

But sadly, I am still forced to write a lot of stuff into my project descriptors that I shouldn't have to... like for generating an assembly or an RPM - these should just work out of the box. I just want to label my project as "webapp", "library", "user application" or whatever, tell it my dependencies, and have it all just work.

We want the same thing ;-)

Pether Sorling replied on Sun, 2008/02/17 - 4:12pm

Maven is an excellent tool to force common conventions regarding development environment, build & release management across team/projects.

Will it slow down developers, of course it will until they learn the new way as with any adoption of new technology.

ff aaa replied on Sun, 2008/02/17 - 4:26pm

Blowing your  groovy horn again?

Steven Devijver replied on Sun, 2008/02/17 - 4:26pm in response to: Pether Sorling

Maven is an excellent tool to force common conventions regarding development environment, build & release management across team/projects.

These conventions are valuable. There is however no value in enforcing them.

Will it slow down developers, of course it will until they learn the new way as with any adoption of new technology.

Criticism on Maven 2 is founded. The core Maven 2 build system and its core plugins is poorly implemented. Bug fix releases are too few. Maven 2 and its developer team just don't seem to have what it takes to take developers down the new road.

Blaming developers is too easy. One interesting thing to notice though is that as far as I know there is today no other build system that tries to mimic Maven 2's conventions. That's about to change.

phil swenson replied on Sun, 2008/02/17 - 8:06pm in response to: Chris Hubick

>We need a build system that is *declarative* - one where you tell it about the *what* and not the *how*

 

True, but once you get past very basic builds you need some imperative code too.   Seems like we need a mix of the imperative and the declarative.

Thomas Mueller replied on Mon, 2008/02/18 - 12:37am

> We need a build system that is *declarative*

You can write declarations in regular programming languages:

addDependencies(
  "javax/servlet/servlet-api/2.3/servlet-api.jar",
  "org/apache/lucene/lucene-core/2.2.0/lucene-core-2.2.0.jar"
);

 

 

Milos Kleint replied on Mon, 2008/02/18 - 2:37am

Such a build system will become available in 2008 and this list is just a teaser. This build system will offer features most developers probably have never thought about.

<cynical note>

Now all the Maven bashing in previous posts finally makes sense.

</cynical note>

 

Steven Devijver replied on Mon, 2008/02/18 - 3:02am in response to: Milos Kleint

Such a build system will become available in 2008 and this list is just a teaser. This build system will offer features most developers probably have never thought about.
Now all the Maven bashing in previous posts finally makes sense.

LOL

Hopefully this new build system makes sense. If Maven 2 works for you then we're happy about that. We just want to offer an alternative for people that are interested.

About the Maven bashing, I'm guilty as charged. However, I've been bashing Maven 2 long before I got involved in this new build system. Actually, I got involved because I think build systems can and should be better than Maven 2.

John Ferguson Smart replied on Mon, 2008/02/18 - 4:53am

Hmmm, I'll be watching this space with great interest...

cowwoc replied on Mon, 2008/02/18 - 9:09am

Please consider using Java for your future build systems.

As others have mentioned you can use a declarative syntax in any language, including Java. By letting developers specify a build file using normal Java code you leverage the existing Java tools and lower the learning barrier to entry.

The key is to spend a lot of time in the design phase, to ensure the API is very clean and intuitive. 

Steven Devijver replied on Mon, 2008/02/18 - 12:15pm in response to: cowwoc

Please consider using Java for your future build systems. As others have mentioned you can use a declarative syntax in any language, including Java. By letting developers specify a build file using normal Java code you leverage the existing Java tools and lower the learning barrier to entry. The key is to spend a lot of time in the design phase, to ensure the API is very clean and intuitive.

That's certainly something we're considering, although supporting Java code for writing build scripts come with a lot of technical hurdles. I think it can be done technical. Remains to be seen if writing this Java code will be easy and straightforward enough.

Guillermo Schwarz replied on Mon, 2008/02/18 - 5:31pm

The main problem I see with build systems is that actually you need one.

People were very happy with Visual Basic and Deplhi because they let programmers draw a screen, press a button and see it working. Why can't we do that with a web application?

Because we want to write operations in Java, therefore recompile, then redeploy, so that instad of "a = b + c;" now it says "a = b - c;". That is a waste of time.

The UI, the business rules and the database should be configured, not programmed. This way, as in Smalltalk, the application would be always running, you would just change the behavior on the fly by changing the UI, the business rules and the database, but not a single line of code should change.

Steven Devijver replied on Tue, 2008/02/19 - 1:08am in response to: Guillermo Schwarz

The main problem I see with build systems is that actually you need one. People were very happy with Visual Basic and Deplhi because they let programmers draw a screen, press a button and see it working. Why can't we do that with a web application? Because we want to write operations in Java, therefore recompile, then redeploy, so that instad of "a = b + c;" now it says "a = b - c;". That is a waste of time. The UI, the business rules and the database should be configured, not programmed. This way, as in Smalltalk, the application would be always running, you would just change the behavior on the fly by changing the UI, the business rules and the database, but not a single line of code should change.

There's a lot of room for innovation.

Brian Sayatovic replied on Tue, 2008/02/19 - 8:44am in response to: Chris Hubick

Expanding on that, I think part of the what needs to be a more formal way of tying JARs and Java packages together.  Today, there is no default restriction on this.  You can add package information to a JAR's manifest, and you can sign/seal JARs to prevent other JARs from adding to packages.

However, because Java code depends on classes in packages, but there is no string coupling between packages and JARs, a build system that treats JARs as black boxes is fundamentally flawed.  The what will never be able to succeed until we can permeate that barrier.

While Java doesn't have strong support for a Java-package tie, there is some support.  Does anyone know of build tolls that take advantage of this?

Thomas Mueller replied on Tue, 2008/02/19 - 11:28am

A build system should have good default settings. Of course. But trying to enforce one specific behavior or configuration is problematic. A build system should be flexible. Some projects need multiple targets (multiple .jar files, some signed and some not, .war files, .exe files). Some projects want to support multiple Java specifications or API versions, and multiple Javadoc targets (public and internal). Running test cases, benchmarks, and code coverage. Maybe you want to add web site generation, translation, spell checking, and so on. One size doesn't fit all.

> The main problem with build systems is that actually you need one.

You don't need one if you use a class loader that compiles source code on the fly. JSP works like this (or: can work like this).

> supporting Java code for writing build scripts come with a lot of technical hurdles

What are those hurdles? I don't see any technical problem.

> Remains to be seen if writing this Java code will be easy and straightforward enough.

I think people have accepted POJO.  

Steven Devijver replied on Tue, 2008/02/19 - 1:10pm in response to: Thomas Mueller

supporting Java code for writing build scripts come with a lot of technical hurdles
What are those hurdles? I don't see any technical problem.

Before running the Java code it needs to be compiled. And after compiling it it needs to be loaded by a ClassLoader. Only then can the Java code actually be executed. This brings up all kinds of questions, like which class name and package name to use. Also, this Java code won't be compiled before the build system runs which is not what Java developers are used to.

This needs to be looked in to. It probably works technically - at least on paper - but if it becomes too weird and cumbersome for developers to write Java build code it may not be worth the while.

Thomas Mueller replied on Wed, 2008/02/20 - 12:43am

> Before running the Java code it needs to be compiled.

Yes - just like ant build.xml and maven pom.xml. What you need is a batch file (jvn.bat / jvn.sh; similar to ant.bat and mvn.bat) that starts the boot application stored a jar file (jvn.jar), which compiles (if necessary) and then runs the Build.java file. Let's say you have core tasks, and optional tasks (plugins). To find out what plugins are required to compile the Build.java file, you need to partially parse the Build.java file. I know it sounds ugly, but it's not that bad: only the import section needs to be parsed. The plugins are downloaded if required, and then the Java compiler compiles and runs the Build.java

The Build.java only needs to contain what you would write in the build.xml and pom.xml.

Michael Parmeley replied on Wed, 2008/02/20 - 8:24am

[quote]

Because we want to write operations in Java, therefore recompile, then redeploy, so that instad of "a = b + c;" now it says "a = b - c;". That is a waste of time.

The UI, the business rules and the database should be configured, not programmed. This way, as in Smalltalk, the application would be always running, you would just change the behavior on the fly by changing the UI, the business rules and the database, but not a single line of code should change.[/quote]

 

Academic, pie-in-the-sky nonsense.

Xavier Hanin replied on Fri, 2008/02/22 - 1:21pm

I agree with the lack of a good out of the box build system today. That's why I started discussing EasyAnt on Ant developer list, which has been reported by Matt Raible over here.

 But there's already a lot of initiative toward new build systems for Java, until now none of them have reached the mass. I would be very interested in learning more about your upcoming build system. If this could save me some of the time I involve in build systems today, I'd be more than happy. But when will your magic build system be available? Will it be open source? How do you gather your requirements? Who is working on it? Is it possible to get involved, or at least share ideas?

michael replied on Wed, 2008/03/12 - 2:22am in response to: phil swenson

wasn't it the fact the introducing more imperative programming to and scripts made duncan start to dislike maven? I remember having read some interview with him where that was mentioned.

Aries McRae replied on Mon, 2009/04/06 - 4:19am

hmm...so you were referring to Gradle. Awesome !

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.