Java Champion / JavaOne Rockstar Adam Bien (adam-bien.com) is a self-employed consultant, lecturer, software architect, developer, and author in the enterprise Java sector. He is also the author of several books and articles on Java and Java EE technology, as well as distributed Java programming. adam has posted 59 posts at DZone. View Full User Profile

How To Kill An OSGi Project - With 10 Questions

03.08.2010
| 21286 views |
  • submit to reddit

OSGi focusses on modularity and it is right now (future may change it) the only viable way to split your application into modules with well-defined dependencies. It solves, however, "only" the technical problem - which is actually relatively easy. Before you going to introduce OSGi into your project, answer the following questions:

  1. What is your versioning scheme for modules(bundles)? Do you care about minor versions, major versions etc?
  2. Whats your scm strategy - do you plan to open and maintain a branch for every version of a module? How many branches do you plan to maintain? (with svn? :-))
  3. How many versioned modules will be active at the same time in production?
  4. How the system is going to be tested? Each module and all the combination of modules. Every version will increase the complexity significantly.
  5. What is your release-management strategy? Do you really plan to provide customer-specific module combinations? What is your bug-fixing / patch strategy (trunk, branch)?
  6. Do you really want to replace modules in running system? If it is a serverside system - what happens with in-flight transactions?
  7. If it is an Eclipse-RCP application - are you even allowed to expose the plugins to the end-user? (in the majority of my projects, we had to disable the update manager in production :-))
  8. What is your software distribution system - many companies have already a software-distribution system in place. Often not only the application, but also the JVM are packaged into one binary file and entirely installed. Incremental updates are often impossible.
  9. What is exactly the contract between the modules? Only a Java-Interface? If so - what to do with direct relations between JPA-entities. If you disallow direct JPA-relations - you will probably see a huge "domain" package with all domain objects inside it. You will need to provide "friend" relations between modules as well.
  10. Is maven the canonical representation of modules, OSGi, or both? A single representation would be the best. Will maven module versions be reflected in the OSGi bundle versions?

The major OSGi-challenge is not the technology, rather than the governance of your modules and bundles. The problem is very similar to SOA. For most enterprise applications you will get only one variant of your business logic, or even UI - so you will end up with modules, which will probably never be replaced. Because of additional complexity - you will probably test and deliver the whole system at once. If you are building an IDE, platform or a server - governance, modularization and versioning are a major part of your problem domain / functional requirements. You will have to solve that anyway. It is far more likely, that someone will have to install a new driver to an application server, or even maintain different versions of the driver at the same time. IDEs without plugins are also extremely rare. Replacing an algorithm in a business application is far more uncommon - often even not allowed. Its just crazy to introduce a modularization solution and then not use it.

From http://www.adam-bien.com/roller/abien/entry/how_to_kill_an_osgi

Published at DZone with permission of its author, adam bien.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags:

Comments

Karim Elsayed replied on Mon, 2010/03/08 - 5:10am

+1

these are the words of someone with "dirty hands".. you know what I mean.. but you need to prepare  for some heat!

Neil Bartlett replied on Mon, 2010/03/08 - 5:35am

How disappointing that this article doesn't live up to its sensational title! Where's the info on how to kill an OSGi project? Come on Adam, you're holding something back... all I see are some quite sensible questions that should be asked irrespective of whether you intend to develop a modular or a monolithic application.

Just one point regarding question 6. If you don't want to dynamically replace modules then JUST DON'T DO IT. Nobody is holding a gun to your head and forcing you to update your bundles.

Karsten Silz replied on Mon, 2010/03/08 - 5:43am

I nominate this article for "Most stupid and attention-grabbing headline of the day".

Daniel Haynes replied on Mon, 2010/03/08 - 6:01am

+1 Sounds like a nightmare to me. My suspicion has always been: OSGI == over-engineering. Does anyone have an example of where is is categorically not true and OSGI delivers real benefits that could not be achieved in a simpler way?

adam bien replied on Mon, 2010/03/08 - 6:27am in response to: Neil Bartlett

The message is: before you are going to introduce OSGi, Jigsaw, Maven (for modularization sake), you should think about the conceptual requirements issues. Otherwise YAGNI :-).

adam bien replied on Mon, 2010/03/08 - 6:30am in response to: Karim Elsayed

Have no problem with some heat. I killed some projects with such questions already. You should actually start a project with such questions *before* even thinking about introducing a solution for probably not existing problem. Even as an independent consultant :-).

adam bien replied on Mon, 2010/03/08 - 6:43am in response to: Karsten Silz

Your comment is not very constructive either. FYI: this article was "intercepted" from my blog - almost without my knowledge :-). It wasn't written intentionally for dzone.
Btw. I killed some projects with such questions already. The problem was - there was no clear answers. In some projects java.util.ServiceLoader was just good enough.

adam bien replied on Mon, 2010/03/08 - 6:39am in response to: Daniel Haynes

Exactly. My approach is:
  1. Show a business case
  2. Search in JDK first :-)
  3. Then introduce a framework
and not vice versa. For building platforms, IDEs, servers - there is no discussion about the viability or the business case. In enterprise projects it is totally different story.

Christopher Brind replied on Mon, 2010/03/08 - 6:52am

"Because of additional complexity - you will probably test and deliver the whole system at once."

Initially maybe - but by using OSGi we've already been able to deliver patches to software running in production just by updating a bundle here or there and it is so quick there is none of the sweaty palms associated with trying to redeploy a J2EE application in production in the smallest amount of time possible.

Will that affect 'in flight transactions'? Possibly, but you have the same problem with J2EE - if not worse. At least with OSGi you have a chance of getting away with updating isolated parts of the application. With J2EE (assuming large scale app with clustering) you're talking about cycling each node to deploy the entire app, coordinating that somehow (maybe your app server provides that bloated management tool for you), if not taking the whole cluster down and showing a 'site down for maintenance' message.

Fabrizio Giudici replied on Mon, 2010/03/08 - 7:05am

I think that this post is good when properly contextualized. As OSGi and, generically speaking, component frameworks are spreading more and more, and getting known by more and more people, the risk is that they are perceived as a "silver bullet". It happened a lot of time in the past, and it will happen in future for other technologies. I understand Adam's post as the warning that OSGi is just a facilitator, but it must be introduced in a project where the best practices work. As it's powerful, OSGi is also more dangerous if not well understood.

Of course people with OSGi experience know these facts pretty well. But they aren't the intended target of this post, AFAIU.

adam bien replied on Mon, 2010/03/08 - 7:36am in response to: Christopher Brind

But: a load-balancer in the front would be the simplest possible solution in your case, isn't it?

adam bien replied on Mon, 2010/03/08 - 7:40am in response to: Fabrizio Giudici

Exactly. But even being an OSGi guru it is now always required to have versioning, dynamic loading capabilities for a typical business application. In that particular case you can still use OSGi, but the other guys in your team will have some trouble to not only understand your intention, but also what happens behind the scenes. Just KISS.

adam bien replied on Mon, 2010/03/08 - 7:50am in response to: Christopher Brind

"Will that affect 'in flight transactions'? Possibly, but you have the same problem with J2EE - if not worse."

If you are still using J2EE - you have other problems -> go with Java EE 5+ :-).
Instead of thinking about loading and unloading modules in running system, I would rather re-route the traffic to another server and reboot the whole JVM - if possible even the whole machine. Just KISS.

Neil Bartlett replied on Mon, 2010/03/08 - 8:47am

As I said, these are reasonable questions to ask, both for modular and for monolithic projects. The answers will differ for each project but here are some generic pointers to answers which may help you:

  1. If you want to think about versioning modules then it will be time well spent. This will allow you to evolve parts of your application or library at different rates. For example it is well known that APIs evolve more slowly than implementations; you don't want to force everybody using an API to update to a new version just because you fixed a bug in an implementation.

    On the other hand if you don't want to spend time thinking about versions just yet, then just make everything version zero (this is the default in OSGi if you do not specify a version). Then everything will be the same as in standard, non-versioned Java. The advantage of this is you can then gradually introduce versions to your modules as needed.

  2. SCM strategy is orthogonal to module development. Typical practice in OSGi projects is to have either a module or simple a folder for each module. There is no need to branch for every module, and there is no need to version your runtime modules the same as your SCM revision numbers.

    Version numbers should communicate something meaningful about the code, not just the SCM revision. I.e. an increment of the Minor digit should indicate a bugfix while an increment of the Major digit should indicate an incompatible change. Many OSGi projects use a build timestamp as the final part of their version (the Qualifier segment); for example, Eclipse does this. But the qualifier is very rarely used for specifying import ranges.

  3. Deciding on the number of modules in a modular system is of course important, and is generally a job for the senior architect on the team. Modules boundaries should be chosen so that modules have high coherence and low coupling.

    Due to its package-level dependencies, OSGi makes it very easy to refactor the contents of modules. So you can start with a few large modules and then gradually separate them as the boundaries become clearer and you gain more experience. In fact a good approach to modularising an existing code base is to start with just ONE huge module containing the whole application, and then gradually pull chunks out of it.

    Jigsaw unfortunately will support only named-module dependencies, so refactoring module granularity will be much, much harder.

  4. Testing is a big subject; we should consider only integration testing here, because unit testing should be done on individual units.

    Naturally, the more flexibility you allow in your module configuration, the more testing you have to do. The advantage of doing more thorough testing of modules is that they become much more reusable, e.g. in your next project or by other development teams. If on the other hand you want to lock down your module configurations and only test a fixed combination of modules, then that is an option, but you get what you pay for.

    It seems clear to me that the long-run savings from module reuse will outweigh the cost of some additional testing, especially when tools exist (e.g. PAX Exam) that can start up arbitrarily many module combinations and run a whole suite of tests against them. For a large project this takes a lot of processing power, but it is easily parallelisable... run it on the cloud!

  5. These concerns are orthogonal to modularity. If you plan to provide customer-specific module combinations, then OSGi supports that and standard Java does not. If you do NOT plan to provide custom combinations, well OSGi supports that also.

  6. In my experience almost all commercial software, aside from standalone consumer-grade software, requires customization for each customer, so OSGi makes this much more manageable. Even in-house software, or software built for exactly one customer, can benefit from customization as requirements change.

  7. OSGi enables hot-swapping of modules but does not require it. If you do not wish to take advantage of this feature, then do not use it. OSGi provides many compelling benefits besides this.

  8. Note that in-flight transactions are rarely a problem because even when a module is updated, the old version still exists in memory until the garbage collector cleans up the class loader for the module. The existence of an incomplete transaction will mean that some objects are still referenced, and therefore they will not be garbage collected until the transaction is completed.

  9. This seems to be a meaningless question... what does "expose the plugins to the end-user" mean? If you wish to hide the implementation details of your plugins then a standard Java obfuscation tool can be used. They are not perfect, but OSGi is no worse in this regard than standard Java.

  10. If you have a software distribution system already then OSGi will almost certainly work with it. If you do not then you will need to get one, whether you are using OSGi or not! OSGi enables more flexible deployment options than standard Java so your distribution system can be both simpler and more powerful.

  11. OSGi bundles can share implementation classes as well as interfaces, and "friend" modules are not necessary (and IMHO are a bad idea since they increase coupling by requiring a producer to know its consumers). Bear in mind that the JPA reference implementation is EclipseLink, it has been deeply integrated with OSGi, and Oracle is participating in the OSGi Enterprise Expert Group to define a standard approach to working with JPA in OSGi. The specification for this will be published at EclipseCon in a few weeks time.

  12. It is not necessary to use Maven to build OSGi projects, but if you do wish to use Maven then you will find lots of support for OSGi development. Generally OSGi metadata is generated from the Maven POM, ensuring that there is only one canonical source for dependency and version information.

If you have any further questions or concerns about aspects of OSGi then I encourage you to ask on the osgi-dev mailing list. The members of the list are very friendly and have lots of experience in dealing with the issues you described. Alternatively you can email me directly.

Jacek Furmankiewicz replied on Mon, 2010/03/08 - 10:00am

My experience with OSGi has been nothing but negative.

We had two main tasks to deliver:
a) provider a common platform for our company to develop apps on
b) be able to reuse common modules across different apps.

OSGi was chosen and it was excellent...at killing our daily productivity.

ClassCastException errors all around, classloader issues, weird errors I've never seen before. build process and unit testing became very complicated. In one case we were down for two weeks where no one could commit 'cause one architect realized he needed to re-design all the bundles and their dependencies.

In my new company, I had the same tasks to deliver. In the end, we just used Maven and some careful engineering:
a) a parent pom.xml gave us a common platform
b) defining a proper Maven structure with parent/child modules gave us reusabiility

Now our daily productivity is through the roof, unit testing and building is a breeze.

OSGi..I am sure it has some use cases where it's needed, but for the 2 common use cases above a good solid setup with Maven did the exact same thing. And with far less headaches.

 

 

 

 

Christopher Brind replied on Mon, 2010/03/08 - 11:40am in response to: adam bien

If you are still using J2EE - you have other problems -> go with Java EE 5+ :-).

Same rubbish, different name.

Instead of thinking about loading and unloading modules in running system, I would rather re-route the traffic to another server and reboot the whole JVM - if possible even the whole machine. Just KISS.

And you think that's the simple approach? The whole machine? Crazy.

adam bien replied on Mon, 2010/03/08 - 12:22pm in response to: Christopher Brind

The simplest approach. It could be a VM or e.g. a zone (this would take only few seconds). But it is a lot saver. You can just get rid off possible class-leaks, permgen issues etc.

You should try to be more constructive with "Same rubbish, different name." It is hard to answer that one. But: would you consider Tomcat, Caucho, Jetty be also a part of Java EE, or not?

adam bien replied on Mon, 2010/03/08 - 12:23pm in response to: Jacek Furmankiewicz

I'm an independent consultant - so my experience with OSGi in real world was very good - from the business perspective :-).

Jacek Furmankiewicz replied on Mon, 2010/03/08 - 1:09pm in response to: adam bien

Ah, yes...just like J2EE :-)

Reza Rahman replied on Mon, 2010/03/08 - 1:56pm

Adam,

It is good to see the post - brazen and to-the-point as usual. I think it is about time people started asking these questions vis-a-vis OSGi. I too have seen a lot of people waste a lot of time on OSGi before figuring out the pitfalls. Personally, I am very happy to see Java modularity not going the OSGi route and staying very Java centric. I had been very fearful we will see an entirely avoidable marketing-driven replay of EJB 1.x/EJB 2.x if Java SE and Java EE went the OSGi route and basically handed a big blow to the hard-earned advances in Java EE 5, Java EE 6 and the Java EE 7 effort that is now underway...

Cheers,

Reza

Neil Bartlett replied on Mon, 2010/03/08 - 2:15pm in response to: Jacek Furmankiewicz

Hello Jacek,

I have read your complaints about OSGi before but since you never go into much detail about the problems encountered, it's difficult to know where you went wrong. Fortunately such negative experiences seem to be the exception rather than the rule. It's a shame that it seems you did not come to the OSGi community (e.g. the mailing lists, IRC channels etc.) with your problems before abandoning it altogether.

One thing you said -- about the architect redesigning all the bundles and their dependencies -- rang a warning bell. As I said in my longer response to Adam, OSGi bundles are easy to refactor since they use package-level dependencies; that means that the consumers of a bundle are unaffected when that bundle is split or merged. Unfortunately OSGi also supports named-module dependencies (known as "Require-Bundle"), though they are deprecated, and in that case consumers ARE negatively affected when bundles are refactored.

Also in my experience most of the classloader problems which may arise in OSGi are a result of using Require-Bundle. The normal package-level dependencies (known as "Import-Package") are so simple that they can hardly go wrong: if you use a package you must import it, end of story. If you get a ClassNotFoundException, it means you missed an import.

So my suspicion is that you probably used Require-Bundle, and possibly made some other serious errors as well, which unfortunately led to a bad result for you. OSGi should probably help you more with tools that diagnose these problems or at least report them in understandable ways. Such tools are starting to appear (e.g. OSGi Inspector by Mirko Jahn)... but too late to save your project, alas!

PS: Jigsaw will support only Require-Bundle (named module) dependencies.

Reza Rahman replied on Mon, 2010/03/08 - 2:44pm in response to: Neil Bartlett

Neil,

It is always good to see folks in a particular community accepting non-positive feedback seriously and professionally. From a personal standpoint, I did explore OSGi with an open mind. To my dismay, what I found is a very complex, low-level specification that needs a lot of evolution/abstraction to be adapted to be digestible to most enterprise Java environments. It also seems quite overkill for most realistic enterprise needs. Jigsaw in comparison feels much more "clean room", Java-centric, compact and easy to understand. These qualities do make it very compelling for me personally. Without devolving into a flame war, could you give me some simple, layman points that I am missing? Alternatively, please do feel free to simply point me to a resource I can take a look at.

Thanks in advance,

Reza

Neil Bartlett replied on Mon, 2010/03/08 - 3:32pm in response to: Reza Rahman

Hi Reza, many thanks for your comment and question. I started to write a response but it's going way, way too long for a comment on DZone! I will post it on my blog in the next couple of days and put a link in the comment thread here when it is available.

Regards, Neil

Neil Bartlett replied on Mon, 2010/03/08 - 3:37pm in response to: Reza Rahman

Reza, in replying to your later comment I missed this one, I apologise. Just to correct the last point: there is absolutely no reason for the adoption of OSGi to be a "big blow" to the advances of Java EE 5 or 6. To prove this point, take a look at the application server you are using for your Java EE 6 development right now... I am willing to bet real money that it (a) uses OSGi, and (b) allows you to deploy OSGi bundles that can interoperate with your JEE applications.

OSGi is rising alongside JEE, not at its expense.

adam bien replied on Mon, 2010/03/08 - 3:55pm in response to: Jacek Furmankiewicz

O.k. you like J2EE but hate Java EE, right? :-)

Reza Rahman replied on Mon, 2010/03/08 - 4:35pm in response to: Neil Bartlett

Neil,

I look forward to your post. I hope you will consider posting it on it's own right on DZone or elsewhere instead of merely as a link. You are correct in that there is no inherent mismatch between Java EE (a pretty high level developement API on a platform) and a modularity systems be it JAR/EAR/WAR/Jigsaw/OSGi that deals with more-or-less orthogonal concerns. I guess that the point I was trying to get across is that in Java EE and Java SE, it is more critical than ever to keep usability and developer productivity in mind. As you know that has been the focus in Java EE 5 and Java EE 6. I can assure you there is no tribalism going on from my perspective, just a focus on keeping things simple where they can be. You are also correct in that I think most application servers will support both the OSGi and "Jigsaw+" (if it comes to that via broader vendor consensus) development models going forward so that it is the best solution that "wins" in the end.

I do hope you will make an attempt to keep your analysis technical but digestible from a "layman's" perspective and keep my/Adam's comments (and similar others that might be more pervasive than you think and are quite natural questions) in mind in your write-up.

Best regards,

Reza

Pascal Leclercq replied on Mon, 2010/03/08 - 5:14pm

The question behind OSGi and modularity in general is not about technical issues, It's all about Time to market. You cannot be fast If you work on you entire system all the time (unless you have a very small Information System). Now here is my answer on different points : For points 1, 2, 3, 4, 5 the answer is clear : You have to handle backward compatibility as any integrated system does. Please read this excellent article talking about Backward compatibilty and versionning policy :http://www.osgi.org/blog/2010/01/backward-compatibility.html. For point 7 and 8, my company found p2 better than their distribution system and took It for RCP deployments. It just works fine, For point 10. I highly recommend the use of maven Tycho which takes the best of both (dependencies relies on manifest files). Regards.

Neil Bartlett replied on Mon, 2010/03/08 - 7:58pm in response to: Reza Rahman

I think all application servers will support OSGi (all the major ones already do), but I'm sure none of them will support "Jigsaw+", whatever that might be!

Jigsaw was designed with exactly one use-case in mind: modularising the JDK itself, which is one of the world's worst-designed, tangled up and incoherent libraries due to the organic and haphazard way it has grown over a period of about 12 years. And with approximately 9M Java developers (Oracle's estimate), it is also one of the most heavily-depended-upon libraries in the world: you CANNOT just refactor it. So it requires extreme measures in order to modularise; these measures are dangerous and not appropriate for any library or application that does not suffer such extreme problems and constraints as the JDK. Frankly if you use them yourself, your code will likely end up as tangled and incoherent as the JDK.

For a long time I felt that Jigsaw was an abomination that should just die. My position has now changed: I accept that it may be a practical short-term approach to modularising the JDK, though do I believe it is still possible to use OSGi for the JDK with some effort. But if you are not writing a JDK then you should not consider using Jigsaw for your own code.

More details in my forthcoming post.

Reza Rahman replied on Mon, 2010/03/08 - 10:51pm

Neil,

Hmm...I hope your analysis focuses more on the kind questions about OSGi that people are asking about here and elsewhere rather than your opinion on how the JDK code is organized. Granted Sun-bashing as a "populist" tactic worked for some companies in the past, but I really don't think it is a real substitute for actual solid technical merit for any solution. For one thing, if the JDK is indeed a "mess" and Jigsaw solves the problem there, it stands to reason that Jigsaw would be much more effective for enterprise applications that are presumably much less messy :-).

Cheers,

Reza

Neil Bartlett replied on Tue, 2010/03/09 - 3:25am in response to: Reza Rahman

Reza, this is no Sun-bashing exercise! I criticise Jigsaw, not the company that produced it. In fact, they paid my wages, and Oracle do now, though as an independent consultant rather an employee.

The reason I talked about the messiness of the JDK is simply because Jigsaw's requirements are driven by the JDK. My argument is that because those requirements are so peculiar to the JDK, Jigsaw is not appropriate unless you are writing a JDK. Also note that Jigsaw does NOT "solve" the messiness of the JDK... it merely allows some parts of the JDK to be separated out, without reducing the mess one iota.

It is this desire to make superficial changes without really fixing anything that is Jigsaw's fatal flaw.

It's true that some enterprise applications can be as messy as the JDK -- though on a smaller scale -- but none of them are used by 9M people! That means that simple techniques such as getting rid of split packages, refactoring dependency cycles and replacing the use of Class.forName() with ClassLoader.loadClass() can make even the most messed up enterprise app amenable to modularisation through OSGi, which will result in a much cleaner and more modular design.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.