John is an experienced consultant specialising in Enterprise Java, Web Development, and Open Source technologies, currently based in Sydney, Australia. Well known in the Java community for his many published articles, and as author of Java Power Tools and Jenkins: The Definitive Guide, and founder of the open source Thucydides Automated Acceptance Test Library project, John helps organisations to optimize their Java development processes and infrastructures and provides training and mentoring in agile development, automated testing practices, continuous integration and delivery, and open source technologies in general. John is the CEO of Wakaleo Consulting, and runs several Training Courses on open source Java development tools and best practices. John is a DZone MVB and is not an employee of DZone and has posted 125 posts at DZone. You can read more from them at their website. View Full User Profile

Maven Mythbusters - Maven Automatically Updates for Every Build

01.07.2010
| 5507 views |
  • submit to reddit

Maven seems to be one of those topics that brings out passion in many developers. Apparently, some developers love it, and find it a highly valuable and time-saving tool, whereas others hate it with a passion. (Of course there are still others who just want to get on with the job, but those ones usually keep quiet on the blogosphere). Every once in a while someone comes out with a blog entry explaining in more or less detail what they dislike so much about Maven. Sometimes these articles contain constructive criticism that allows Maven to evolve in the right direction. That's great. Sometimes they contain inaccuracies or misunderstandings about how Maven works. Sometimes they are just downright wrong. But they nevertheless represent a perception of Maven in parts of the Java community. So in this series of articles, I want to take a look at some of the common myths and ideas that circulate about Maven, and see how they stand up to the light of scientific examination.

Myth Number 1: Maven automatically checks for, and downloads, updates at every build

The first myth we will tackle involves Maven's network habits. Indeed, there seems to be a widely spread idea about Maven that it slows down your build with excessive network access. Here is quote from a recent blog entry illustrating and propagating this idea:

"Maven is broken and wrong if it thinks nothing of slowing down every build by connecting to the network and checking every dependency for any updates, and automatically downloading them"

Interesting. Indeed, it would be a grievous fault! Let's see how this idea stands up to the facts.

Actually, this idea is flat out wrong. In fact, Maven only checks for SNAPSHOT updates once a day by default, and even this is configurable. Maven 3 takes things even further: automatic updates will be turned off by default - you will have to explicitly ask for updates (using the -U option).

Maven doesn't need to check for core plugin updates at all - since Maven 2.0.9, the versions of the core plugins are bound to the version of Maven you are using, unless you explicitly tell it otherwise. So no updates here either.

However, like snapshots, Maven will check for plugin updates once a day if you don't provide a version of your plugin. So provide versions for your plugins, or accept the overhead of some network network access once every 24 hours. But it is recommended practice (and common sense, really) to specify the version number of any plugins you use in your build. If you do have explicit plugin version numbers for non-lifecycle plugins, it won't need to check for plugin updates either.

So in theory, this myth should be well and truly busted. However, true to MythBuster tradition, we'll check this hypothesis with a few tests. First, let's try with a simple multi-module project. To make things interesting, I've deleted a few directories from my local repository to force Maven to download some stuff initially:

$ mvn clean verify
[INFO] Scanning for projects...
[INFO] Reactor build order: 
[INFO]   Tweeter
[INFO]   Tweeter domain model
[INFO]   Tweeter service layer
[INFO]   Tweeter web application
[INFO] ------------------------------------------------------------------------
[INFO] Building Tweeter
[INFO]    task-segment: [clean, verify]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
Downloading: <a target="_blank" href="http://localhost:8081/nexus/content/groups/public/org/easyb/easyb/" title="http://localhost:8081/nexus/content/groups/public/org/easyb/easyb/">http://localhost:8081/nexus/content/groups/public/org/easyb/easyb/</a>
0.9.6/easyb-0.9.6.pom
2K downloaded  (easyb-0.9.6.pom)
Downloading: <a target="_blank" href="http://localhost:8081/nexus/content/groups/public/org/easyb/easyb/" title="http://localhost:8081/nexus/content/groups/public/org/easyb/easyb/">http://localhost:8081/nexus/content/groups/public/org/easyb/easyb/</a>
0.9.6/easyb-0.9.6.jar
343K downloaded  (easyb-0.9.6.jar)
[INFO] [easyb:test {execution: default}]
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] Tweeter ............................................... SUCCESS [4.434s]
[INFO] Tweeter domain model .................................. SUCCESS [9.724s]
[INFO] Tweeter service layer ................................. SUCCESS [1.722s]
[INFO] Tweeter web application ............................... SUCCESS [30.865s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 47 seconds
[INFO] Finished at: Tue Jan 05 12:14:12 NZDT 2010
[INFO] Final Memory: 63M/123M
[INFO] ------------------------------------------------------------------------
2010-01-05 12:14:13.588::INFO:  Shutdown hook executing
2010-01-05 12:14:13.090:/tweeter-web:INFO:  Closing Spring root WebApplicationContext
2010-01-05 12:14:13.092::INFO:  Shutdown hook complete

That downloaded the easyb plugin because it didn't have it locally. Fair enough, that's what its supposed to do. To boot, it downloads dependencies in parallel when it can, so it's pretty fast. What Maven does downloads, it needs to build your application, and it does it only once. Not once per project, but once, period. But back to the subject in question: now let's rerun to see if it tries to download any updates.

$	mvn clean verify
[INFO] Scanning for projects...
[INFO] Reactor build order: 
[INFO]   Tweeter
[INFO]   Tweeter domain model
[INFO]   Tweeter service layer
[INFO]   Tweeter web application
[INFO] ------------------------------------------------------------------------
[INFO] Building Tweeter
[INFO]    task-segment: [clean, verify]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting directory /Users/johnsmart/Projects/wakaleo-training/tdd-traini
ng/lab-solutions/tweeter/target
[INFO] [easyb:test {execution: default}]
[INFO] /Users/johnsmart/Projects/wakaleo-training/tdd-training/lab-solutions/tw
eeter/src/test/easyb does not exists.  Skipping easyb testing
[INFO] [site:attach-descriptor {execution: default-attach-descriptor}]
[INFO] ------------------------------------------------------------------------
[INFO] Building Tweeter domain model
[INFO]    task-segment: [clean, verify]
[INFO] ------------------------------------------------------------------------
[INFO] [clean:clean {execution: default-clean}]
[INFO] Deleting directory /Users/johnsmart/Projects/wakaleo-training/tdd-traini
ng/lab-solutions/tweeter/tweeter-core/target
lures: 0, Errors: 0, Skipped: 0
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] Tweeter ............................................... SUCCESS [3.266s]
[INFO] Tweeter domain model .................................. SUCCESS [8.338s]
[INFO] Tweeter service layer ................................. SUCCESS [1.830s]
[INFO] Tweeter web application ............................... SUCCESS [20.185s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34 seconds
[INFO] Finished at: Tue Jan 05 12:21:04 NZDT 2010
[INFO] Final Memory: 63M/123M
[INFO] ------------------------------------------------------------------------
2010-01-05 12:21:04.559::INFO:  Shutdown hook executing
2010-01-05 12:21:05.062:/tweeter-web:INFO:  Closing Spring root WebApplicationC
ontext
2010-01-05 12:21:05.064::INFO:  Shutdown hook complete

Nope, no downloads there. That went pretty well, but the example might have been a bit simple. So let's check this with a medium-size (around 32000 lines of code) real-world project (an old legacy one, to boot!):

$ mvn verify
[INFO] Scanning for projects...
[INFO] Reactor build order: 
[INFO]   BNBGlobal parent module
[INFO]   BNBGlobal localization utilities
[INFO]   BNB Global Core Application
[INFO]   BNB Global Web Application
[INFO]   BNBGlobal aggregator module
[INFO] ------------------------------------------------------------------------
[INFO] Building BNBGlobal parent module
[INFO]    task-segment: [verify]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: enforce-java}]
[INFO] [site:attach-descriptor {execution: default-attach-descriptor}]
[INFO] Preparing source:jar
[WARNING] Removing: jar from forked lifecycle, to prevent recursive invocation.
[INFO] [enforcer:enforce {execution: enforce-java}]
[INFO] [source:jar {execution: default}]
[INFO] NOT adding sources to attached artifacts for packaging: 'pom'.
[INFO] ------------------------------------------------------------------------
[INFO] Building BNBGlobal localization utilities
[INFO]    task-segment: [verify]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: enforce-java}]
[INFO] [groovy:generateStubs {execution: default}]
[INFO]  No sources found for Java stub generation
[INFO] [resources:resources {execution: default-resources}]
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [groovy:compile {execution: default}]
[INFO]  No sources found to compile
[INFO] [groovy:generateTestStubs {execution: default}]
[INFO]  No sources found for Java stub generation
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Copying 24 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [groovy:testCompile {execution: default}]
[INFO]  No sources found to compile
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /Users/johnsmart/Projects/bnbglobal/bnbglobal
-l10n/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
....
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] ------------------------------------------------------------------------
[INFO] BNBGlobal parent module ............................... SUCCESS [3.378s]
[INFO] BNBGlobal localization utilities ...................... SUCCESS [15.495s]
[INFO] BNB Global Core Application ........................... SUCCESS [23.054s]
[INFO] BNB Global Web Application ............................ SUCCESS [26.336s]
[INFO] BNBGlobal aggregator module ........................... SUCCESS [0.112s]
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 minute 9 seconds
[INFO] Finished at: Wed Jan 06 16:22:07 NZDT 2010
[INFO] Final Memory: 92M/179M
[INFO] ------------------------------------------------------------------------

Hmmm, no updates in sight there either. Just to make sure that my local repository manager wasn't interfering with the process, somehow, I also ran the same tests with no settings.xml in my .m2 directory, with (wait for it!) exactly the same result! So it looks like this one is well and truly busted - Maven does not check for updates, for SNAPSHOT dependencies, for plugins or for anything else, for each build. At most it checks for SNAPSHOT updates once a day. It checks for unversioned plugins once a day, and not at all for versioned plugins (a best practice). And in Maven 3, it only checks for updates if you explicitly ask it to.

Next time we'll take a look at another Maven Myth: Maven requires an internet connection to delete a directory.

(The title and some of the images of this blog were of course shamelessly stolen from the great and highly scientific MythBusters TV series.)

From http://weblogs.java.net/blog/johnsmart

Published at DZone with permission of John Ferguson Smart, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Stefan De replied on Thu, 2010/01/07 - 4:38am

like any tool or framework, Maven also has its downsides, but overall it's a good tool. nice to see someone taking the time to tackle these myths.
i hope to read more of this.
thanks!

Jean-Baptiste Nizet replied on Thu, 2010/01/07 - 11:07am

Maven only checks for SNAPSHOT updates once a day by default.

This is new to me, I must confess. But this means that the Maven team still hasn't understood one of the most important constructive critics that have been made about Maven : builds must be repeatable. If one iteration of a build doesn't do the same thing as the previous ones (i.e. download a new version of a snapshot or not), all other things being equal, then the builds are not repeatable, and developers want to kill someone. It took them until version 2.0.9 to understand that it was insane to allow Maven downloading an alpha version of a core plugin. I sincerely hope they fix these kinds of things, since I'm forced to use it instead of Ant in some of my contracts.

Jilles Van Gurp replied on Thu, 2010/01/07 - 2:50pm

> Total time: 1 minute 9 seconds

 That's one minute you'll never get back. I've seen non toy projects where it's more like 5 minutes. That's why maven sucks. It does a lot of shit you don't need it to do at a moment where it is definitely amounting to wasting everybodies time. Downloading shit you already have is just one of the many things it shouldn't be doing to begin with.-o should be the default.

My Java builds were < 2 seconds in 1997. Something went seriously fucking wrong in between. It annoys the hell out of me that I have to tell it to compile to begin with. That's just a run-time optimization in my view.

Technically, the maven compilation step is a waste of time on a local desktop since your IDE (a decent one that is) does that for you, incrementally on save. It can do that because your dependencies are already there. So any attempt to update or download them is redundant. That leaves running your tests and launching your app server. The latter actually requires a shitload of configuration in Maven and the former you can do straight from your IDE. 

Seriously, maven is a time waster. It's measurable, it's about 20% of the time you pay developers to do actual work. Worse, it interrupts their flow, thus diluting the value of the remaining 80%. Solution, leave maven builds to you CI server. Don't bother with it on your desktop at all. Use ant and other scripting solutions to automate things that need to run fast locally. Like running your test suite or starting an application server or whatever. Any second that takes longer than strictly necessary is time that will really start accumulating when you have more important stuff to do.

Jason Marshall replied on Thu, 2010/01/07 - 3:59pm in response to: Jean-Baptiste Nizet

This is exactly my problem with Maven. 

It is designed to facilitate an activity that I do not want my (mature) application doing.  Whether it can be configured to not do that action is only barely relevant.  It can do it, and I have to remain vigilant that nobody is using it in that questionable capacity.

If I'm doing my job right, I should only have to tweak my build scripts a couple times per release.  That means that either I have to disallow anyone else from touching the build scripts, or keep an eye on them constantly.  If I'm not allowing anyone else to maintain the build scripts, then I might as well do the bits Maven does, manually.  I'm perfectly happy to do that, as it's already ingrained into my notion of software development lifecycle. 

What's more, everyone always seems to want to push for adding Maven to the build tools just when we're trying to mature the build process.  I have to wonder what their motivation is for wanting it, and now the guilt-by-association compromises my working relationship with them.  Now I'm wary of the intentions of otherwise perfectly responsible and productive team members (I say that non-hypothetically).

Fetching indeterminant libraries at unpredictable intervals is anything but mature behavior.  New library can mean a full regression cycle by the test team, even if you have great unit tests.  It always amazes me how many developers completely discount the very real and very serious costs of regression testing, both in manpower and calendar days.

In short, changing your project library versions SHOULD be hard, because it IS hard.  The book-keeping involved is only the merest thin slice of the complexity involved, and writing a tool that trivializes that is putting a dangerous weapon in untrained hands.


 

 

Patrick Angeles replied on Tue, 2010/01/12 - 12:31am in response to: Jean-Baptiste Nizet

Seriously, why is this a problem? If you don't want your dependencies to update, don't use SNAPSHOTs. A SNAPSHOT follows the HEAD of a branch. Use that last stable, released version. If you want something more recent than the last released version but not-quite trunk, then you should build the dependency yourself and publish to your internal repo.

Jeroen Wenting replied on Tue, 2010/01/12 - 2:38am

"Fetching indeterminant libraries at unpredictable intervals is anything but mature behavior. New library can mean a full regression cycle by the test team, even if you have great unit tests. It always amazes me how many developers completely discount the very real and very serious costs of regression testing, both in manpower and calendar days."

Well said, and (apart from the fact that maven project files are even harder to use than ANT build files and Maven dictates a project structure that's utterly unsuitable to anything I've ever seen in mature projects) the main reason maven is not a tool I tend to use or advocate.
And yes, I've tried it, I've read the books (you just about have to as the documentation is useless, another sure sign of an immature hobby project).
Maven has potential, but no more than that. It's as yet not mature enough for serious use in larger projects, and certainly not for retrofitting into such projects. And given the development history of Maven and the attitude of its core community, I doubt it ever will be mature enough.

Jason Marshall replied on Tue, 2010/01/12 - 2:13pm in response to: Patrick Angeles

I submit to you, as evidence, another apache project: Ant.  At many release milestones in the past, including virtually every 1.5 series release, the latest stable version of Ant is not backward nor forward compatible with code (scripts) written against the previous stable version.  Very often the semantics of operations have changed to the point that your code only works against a very narrow range of official builds.

This is neither unique to Ant, nor to Apache projects (although this and heavy coupling with co-projects seems to be some of the worst defining features of Apache projects in general). 

You do most of your development and testing against version 1.2.X of a library/tool, you better damn well not ship with version 1.2.X + 1 of that tool.  This almost always ends badly, especially for the development team.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.