Andrew has posted 4 posts at DZone. View Full User Profile

Evolve - a powerful, UML-based alternative to dependency injection

10.04.2010
| 9472 views |
  • submit to reddit

Phew - I've just gone into public beta with my new UML-based tool after many years of work. It builds on the research that I did for my PhD at Imperial College. It also draws on my 20+ years of software experience to create a new way of connecting up Java classes to form a system. It is called Evolve and I'm proud to share it with you at Javalobby, the community where I've had many discussions about Java and its future.

At the heart of Evolve is a component model with full connectors. This makes it far more powerful and flexible than dependency injection approaches. Some of you may remember a previous article I wrote describing how DI was limited compared to full components: http://www.javalobby.org/articles/di-heritage/

Anyway, Evolve is the commercialization of that research work.

Evolve provides the following:

  • A sophisticated UML-based graphical editor for creating and wiring up plain vanilla classes
  • A runtime environment for executing these
  • The ability to turn an Evolve model into a set of Java classes
  • A bean import tool to import your existing libraries into Evolve

A central part of Evolve is that it provides version-control like facilities deep in the component language. In other words, you can evolve components inside the tool, using a graphical delta editor. Systems created with Evolve therefore have the remarkable property that they can always be extended. This capability was at the heart of my research work.

You can read more about Evolve, and download the software and manual, at:

http://www.intrinsarc.com/

Here's a very quick rundown on how Evolve allows you to create components...

The first step is to define leaf components. These are basically a fancy graphical view on plain-old JavaBeans, bringing them into the more sophisticated world of components.

 

 

Composite components are then created by connecting up instances of other components. These components are powerful enough to represent entire systems if required. It's a scalable model.

 

 

You can then evolve a component and amend it using deltas. This doesn't destroy the original structure, but like a version-control system, applies the deltas when required.

 

 

At all times, the code is kept synchronized with the diagrams. You literally see the system and code you are creating.

Please download the software and try it out. I'd love to hear your opinions.  I will start selling it at the end of the beta period, but I am keeping it affordable. Further, there is a community edition which is free but generates GPL'ed code.

Oh - the runtime engine of Evolve is called Backbone, and is open source under the Apache License. Any program you write in Evolve runs on Backbone, and does not need to be tied to Evolve.

Published at DZone with permission of its author, Andrew McVeigh.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Jacek Furmankiewicz replied on Mon, 2010/10/04 - 2:18pm

Sorry...I'd rather code in asssembler than replace this basic code

@Autowired private MyService svc;

with some gargantuan UML diagram.

Any company that forces their developers to use this must place very little value on their productivity.

Esko Luontola replied on Mon, 2010/10/04 - 2:35pm

I wouldn't trust any system which claims to replace DI but whose examples do not include tests for code which uses the system. I couldn't even see any code examples on the web site, and also the manual was scarce on code.

Andrew McVeigh replied on Mon, 2010/10/04 - 3:00pm in response to: Jacek Furmankiewicz

Hi Jacek,

The UML diagrams do not have to be gargantuan.  In fact, they tend to be rather concise in practice, particularly as the model is hierarchical. The examples in the diagrams above are complex with many connectors and ports.

And having one trivial annotation is one thing; having hundreds as we have seen in systems is another entirely as any example of size will demonstrate. We have also seen thousands of lines of XML code for Spring configurations which render systems completely opaque.

So, yes, you are clearly correct in that an example with a single annotation will be simpler in guice/spring. Large systems will not, particularly those with many aspects.

Andrew

Andrew McVeigh replied on Mon, 2010/10/04 - 2:50pm in response to: Esko Luontola

Hi Esko,

Could you clarify your statement "do not include tests for code which uses the system"?

The code examples are in the user manual, and in the download -- a fairly extensive set of tutorials which build up a small GWT/hibernate system and shows the concepts along the way. They are not on the web pages, but as you point out it might be a good idea to put them there.

The underlying Backbone runtime is open source (ASL2), and we are presently on track to move it to the google code site by early november. It has full unit tests.

Cheers,
Andrew

Jacek Furmankiewicz replied on Mon, 2010/10/04 - 3:28pm in response to: Andrew McVeigh

Well, I would say the solution to Spring XML files is not to convert them to UML but to convert them to annotations.

I still think you have a very bad solution for a real problem...when a very good alternative exist.

Andrew McVeigh replied on Mon, 2010/10/04 - 4:09pm in response to: Jacek Furmankiewicz

>Well, I would say the solution to Spring XML files is not to convert them to UML but to convert
>them to annotations. 

(To correct a possible misconception here, note that Evolve doesn't create Spring files. Underneath Evolve is a component model that has full connectors. It is demonstrably more powerful than DI approaches, and I'll touch on this in future posts)

Most real-world spring systems are actually a combination of XML and annotations. And as the systems get large, they get very difficult to understand. Great for small systems though, I use Spring myself on smaller projects in investment banking all the time.

>I still think you have a very bad solution for a real problem...when a very good alternative exist.

Well, at least we both agree there's a problem ;-)

I actually hear what you are saying. I'd have had a similar opinion several years ago, and I certainly take your point that UML can be verbose and awful. UML has also got a bad reputation, which is often deserved.

However, Evolve is simply using the standard presentation of UML for drawing components. Hierarchical components were added in UML2, and they really stands out as the high point. It is a bit like a circuit design metaphor for software - and noone complains about that in electronics - visual descriptions in that domain are a useful complement to textual descriptions.

The saving grace of Evolve though, and what stops it from descending into diagram-hell are that (a) it is hierarchical and (b) that the code is always synched up to the model. Hierarchical means that you can hide the inside of a component and shield yourself from that complexity. You can also view a system or create a system at any level of abstraction, a very powerful feature.  (b) means that when you draw diagrams you are actually moving around code. It makes the diagrams mean something concrete. Try it -- it's actually quite exciting and shocking at the same time.

And we haven't even gotten onto one of the major advantages of Evolve - systems created this way have the remarkable property of always being extensible.  In other words, even if you wire up your systems in a hardcoded way in Evolve, they can always be "evolved" to adjust them for new requirements.

(As an aside, the diagrams in Evolve are actually just visual representations of a concise textual notation of the Backbone language. To see the Backbone code for any component, right click on the package and choose "show backbone code")

Please consider giving it a try and using it for a small example and feeding back specifics. I'm not against the things you say, it's just that they don't apply to Evolve. I'd feel much more comfortable discussing the actual limitations and features of Evolve rather than the strawman you seem to be considering.

Cheers,
Andrew

p.s. Of course, there are some people who will never like a diagram-view of software.  That's ok too, but it doesn't mean that diagrams are necessarily bad . I'm sure you'll appreciate that different does not always mean evil.

 

Jacek Furmankiewicz replied on Mon, 2010/10/04 - 4:11pm in response to: Andrew McVeigh

Sure, we can agree to disagree :-)

I just had a chance to do a year-long MDD (Model Driven Development) EE project a while ago that was all about diagrams, wiring them together and then generating code. From that I have a deep dislike of anything that gets between me and the code.

Andrew McVeigh replied on Mon, 2010/10/04 - 4:28pm in response to: Jacek Furmankiewicz

i hear you, I've tried the MDD thing and it didn't work at all for me either.  i felt it took away almost all my freedom to code. In Evolve the diagrams are just of the relationships between things and the structure of the system - the skeletons.  people still need to write the code, and there's no MDD stuff in there at all.

thankyou for your comments - even though you disagree perhaps with the approach i find your point of view interesting and it will help me to steer Evolve away from the stereotype of "bad UML".

besides, I'm pretty sure I'll cop much worse on theserverside.com ;-)

Zhong Yang replied on Mon, 2010/10/04 - 11:21pm

Wow, congrats Andrew, great job. Finally I see some hope. It's a great idea (I haven't looked into the real product), but you have to realize whom you are marketing it to. You have people like Jacek, who is a hard core coder, they think you are wasting your time. Then you have people like me. I have done my share of coding, now I don't really care to read any code but I want to know how the whole thing works together. DI is good when you are the developer working on a small project. Try to reverse engineer a big project done by a few dozen developers from a few years back. And with all the documentation left behind by super productive "Agile" teams. If people don't realize the value of Modeling, you won't be able to convince them you have a potentially great product. Even for people who appreciates modeling it may take time for them to get use to a new tool and new way of building applications. You will get more supports among architects. The war between developers and architects goes on...............

Balázs Bessenyei replied on Tue, 2010/10/05 - 1:29am

Not completely seeing the light here. However a direct plug in bases integration into the popular IDEs should be considered. A constant export import cycle to/from IDE and the modeler will really hurt the productivity in the long run. 

 It is possible to manage large projects (60+ developers) with DI based solution it depends on the Technical Lead and the Technical Architect skills to ensure that the system architecture and design remain in a consistent state. An if they can prevent the "uber talented code/technology virtuous" developers to introduce "cool" designs, which are ultimately unmaintainable and only a few similarly talented can understood. Then it is possible for the project to survive even if the project is Agile. 

 The modularization idea is a good thing. We have a modularization solution in place, which more or less accomplishes the same thing at first glance. It only took 3 additional class to Spring to accomplish it. Although we don`t have a nice GUI for it.

 However this is still interesting so its worth a test drive.

 

Andrew McVeigh replied on Tue, 2010/10/05 - 2:24am in response to: Zhong Yang

Wow, congrats Andrew, great job. Finally I see some hope

Thanks Zhong ;-)

Try to reverse engineer a big project done by a few dozen developers
from a few years back. And with all the documentation left behind by super productive "Agile" teams.

Well, agile or not, i've seen a lot of bad systems unfortunately... I find it happens when people can't see the "wood from the trees".  They get lost in the details and subvert the structure of the system, eventually creating a ball of mud. Evolve is one way out of this, and can even be applied to systems very late in their lifecycle.

If people don't realize the value of Modeling...

I'm personally a coder, and I love coding. However, I also see great value in modeling. I believe that people currently don't like modeling for a good reason - it tends to be divorced from the code, so the boxes and lines to mean very little.

Evolve is my attempt to rectify that. It keeps the component diagrams and code completely in sync. When you draw components, you are creating classes and the relationships between them, but also adhering to a strong and proven component model. (In fact the underlying component model has a full formal specification)

So, i actually see very able people like Jacek (who makes the cool swingbuilder stuff) as part of the target market.  i.e. people who have not yet seen the value of modeling. It's a tough sell, but there's a lot of merit in the approach...

The war between developers and architects goes on...

My point is that the separation between the 2 roles is a false dichotomy. I work as an "architect" in an investment bank, but I make sure I code at least 50%. I believe that everyone in a project needs to contribute to the architecture, and fully understand it, as much as they contribute and understand the code. Evolve makes the 2 activities one and the same.

Thanks very much for the support.  Give Evolve a try, if you have time, and let me know how you find it.

Cheers,
Andrew

p.s. An interesting point of trivia - the component model underlying Evolve has its roots in the same academic research that influenced COM. Of course, COM lost a lot of power when it removed connectors, but that's a different story...

 

Andrew McVeigh replied on Tue, 2010/10/05 - 4:23am in response to: Balázs Bessenyei

 Hi Balázs, thanks for the comments.

Not completely seeing the light here.

It's a common reaction.  Part of the problem is that I am only presenting part of Evolve - the ability to do what DI does. In actuality, it's a more capable and expressive model that can do all that DI does and more. See a previous article I wrote for the difference: http://www.javalobby.org/articles/di-heritage

That article was checked for me at the time by Rod "Spring" J; the difference in power is real.

So, what is Evolve really?  Evolve provides, amongs other things:

  • An intuitive component approach that can express things that confound DI
  • A visual (and standard) modeling environment where the code and diagrams are kept completely in sync
  • Systems created using evolve are always extensible (evolvable), even without source code. No more need to factor in extension points == agile design. This remarkable ability has to be experienced to be believed -- it was my phd topic.
  • Evolve has full error checking which outlaws many bad practices
  • It's built on a rigorous, formally specified runtime, called Backbone == very robust
  • It can either run your model directly (via text configuration) or create a small Java program to wire everthing together

There's actually a lot more, and i hope to blog on the features over time.

However a direct plug in bases integration into the popular IDEs should be considered. A constant export import cycle to/from IDE and the modeler will really hurt the productivity in the long run.

It's a good point, and it's in our roadmap, but in practice I haven't found it necessary.  I'd certainly appreciate any feedback you have on the roundtripping if you get a chance to use it.

How does it work?: when using Evolve with Eclipse as my IDE, I create a model and press ctrl-G, which generates the classes to the file system.  I then move to Eclipse, press F5 and the classes appear.  I fill in any code, run the program.  I then go back to Evolve and repeat.  Evolve knows what to overwrite and what to leave, so there's no real import/export cycle.  I only need to reverse engineer when I've overwritten a relationship in code (and you can select which ones are managed), and that process has been heavily optimised also to a few keystrokes.

It is possible to manage large projects (60+ developers) with DI based solution it depends on the Technical Lead and the Technical Architect skills to ensure that the system architecture and design remain in a consistent state.

Yes, i've seen large-ish systems created in this way. Then again, I've seen people successfully systems using the awful EJB CMP stack, so anything is possible if the people are good enough and disciplined enough.

What I'm claiming with Evolve is that there's a better way. A way where the limitation of DI-type approaches are not there, and the architecture can be visualised at all times. These are real benefits.

An if they can prevent the "uber talented code/technology virtuous" developers to introduce "cool" designs, which are ultimately unmaintainable and only a few similarly talented can understood. Then it is possible for the project to survive even if the project is Agile.

;-) I guess we all want more than to just "survive".  We want to thrive, and create elegant and expressive designs that everyone in the team can understand and manipulate. Evolve is my "statement" in that direction.

The modularization idea is a good thing. We have a modularization solution in place, which more or less accomplishes the same thing at first glance. It only took 3 additional class to Spring to accomplish it. Although we don`t have a nice GUI for it.

The modularization side of Evolve is actually very deep. For instance, if you have multiple variants in a single model, Evolve checks the cross-product of all possibilities for conflicts. And if you get a conflict, you can always "evolve" it and fix the problem, without destroying the original definitions.

This part of the system is rooted in a detailed formal specification, which understands things like interface inheritance.

However this is still interesting so its worth a test drive.

Thanks.  That's all I could ask for ;-)  Let me know what you think, if you do get a chance to try it out.  Thanks again for your comments - the feedback is very helpful.

Cheers,
Andrew

Developer Dude replied on Tue, 2010/10/05 - 12:55pm

One of the values of some forms of DI (e.g., Spring and XML config) is that the code is not changed when it is reconfigured via DI - it is just run with a new dataset (configuration).

This allows for more reuse of code components than writing code that reuses components - both by design and in construction of an app/system. Reuse of components is a good thing (this should be obvious, but unfortunately, in the real world, it is all too often ignored)

QA/test sees typically sees the resulting system of components as a different configuration of existing code, instead of a change in the code itself, and typically requires less testing/QA of the new configuration. And rightly so (IMO as a dev who worked in QA/test for 7 years before going over to the 'dark side' - dev).

Many dev teams/orgs see this advantage of certain forms of DI as a very compelling reason to use DI.

I don't see any system that generates new code as having that major advantage of DI, no matter how clever the wiring is. The code changes, and therefore the resulting code will need to be retested to a greater extent than code that was merely reconfigured.

Beyond that, many devs have a strong aversion to generated code - myself included.

Now all of my comments may seem very basic compared to the underlying principles you are using in the project and referencing in your previous article - and they are basic, but ignore them at your peril because they too are underlying principles which drive many dev/QA teams and projects - and probably will for quite some time

Interesting project/product/approach, but I will stick with plain vanilla DI for now. Thanks for the article though.

Andrew McVeigh replied on Tue, 2010/10/05 - 2:53pm in response to: Developer Dude

Hi Lauren,

One of the values of some forms of DI (e.g., Spring and XML config) is that the code is not changed when it is reconfigured via DI - it is just run with a new dataset (configuration).

Sure, completely agree.  In keeping with this, Evolve operates at 2 levels: on one level, you simply change configurations. This is what you are doing when you change the wiring or configure up a composite component.  There is no Java code generated at that level, and if you want you can keep it completely that way. This is shown in pictures 2 & 3 of the above article.

The second level is effectively a bean designer, which can generate skeletons for code with a clever way of roundtripping. This part is completely optional; some people like it others don't. You can avoid any generation whatsoever if this works best for you. Evolve is agnostic in this way.

QA/test sees typically sees the resulting system of components as a different configuration of existing code

Yup, again, I am in complete agreement. Evolve supports this strongly, actually far more strongly than conventional DI by providing a simple way to swap out parts of a wired up component, using the "evolution" construct. Part of the motivation for this was that we found that people were having to duplicate and slightly change parts of large configurations for testing.

The code changes, and therefore the resulting code will need to be retested to a greater extent than code that was merely reconfigured.

Of course.  Evolve goes a step further - it prevents the need to rewrite or duplicate configurations. In effect, the composite components have a form of inheritance that lets you incrementally adjust the wiring only. This incremental change carries even less risk.

...but ignore them at your peril because they too are underlying principles which drive many dev/QA teams and projects

yes, you are completely correct.  i use spring this way on a daily basis. as mentioned before, evolve supports this type of wiring variation with very powerful facilities. It is not uncommon to have 10 different variants of a program in a single model, with various test cases etc.

Interesting project/product/approach, but I will stick with plain vanilla DI for now. Thanks for the article though

No worries, thanks very much for your comments, they are helpful. As you can probably tell, I'm still tuning the product message and this very much involves finding out how people perceive the product. This is slightly more frustrating than i figured ;-) I'm in violent agreement with most of the posters and their perceived limitations of the product, which is frustrating because it doesn't have these limitations at all. It's a problem with my communication, basically.

If you have the time, please give Evolve a go and test your preconceptions of it. The user manual is quite accessible (says he ;-). I will also feed in the helpful notion that I need to more clearly demarcate between the configuration/wiring part and the bean code gen parts.

cheers,

Andrew

Balázs Bessenyei replied on Tue, 2010/10/05 - 6:11pm in response to: Andrew McVeigh

It's a common reaction. Part of the problem is that I am only presenting part of Evolve - the ability to do what DI does. In actuality, it's a more capable and expressive model that can do all that DI does and more. See a previous article I wrote for the difference: http://www.javalobby.org/articles/di-heritage

That article was checked for me at the time by Rod "Spring" J; the difference in power is real.

 I have an idea what it is and how it can be useful. I just don`t see the revolution/evolution in it, yet.

Usability can be of a concern here, especially in already existing projects. Unless the reverse engineering capabilities are good. This can be though, due to the existing DI FWs out there.

Strong refactoring support probably a must.  Especially, if there are unit tests in existence.

I`m not convinced that this is the right solution for solving DI short comings. That is probably in between the two approaches. DI really needs some heavy modeling capabilities, which can be enforced during design and runtime.  

 

;-) I guess we all want more than to just "survive". We want to thrive, and create elegant and expressive designs that everyone in the team can understand and manipulate. Evolve is my "statement" in that direction.

Creating simple and easily understandable solution from both novice and senior level programmers point of view, is a rare talent. Usually a sign of a great Technical Lead/Architect.

I was originally meant to refer to the reckless use of solutions, which are used because they are "cool". Since they are using the greatest and latest technologies and frameworks or some rarely known and hard to comprehend technique as a center piece in the architecture. Like building an extremely complex machine just to hammer a nail into a piece of wood. Thriving is good, till the solution is simple and "easily" understood.  

(Actually we are migrating to the latest versions of the used components few months after they get released, depends on how big they are.)

Andrew McVeigh replied on Wed, 2010/10/06 - 8:04am

I have an idea what it is and how it can be useful. I just don`t see the revolution/evolution in it, yet.

Ah, a good point.  What is the essence of what makes Evolve special?

So, at its heart, Evolve gets its power by adding 2 things to a fuller component model: resemblance (i.e. structural inheritance between components) and evolution (the ability to replace a component in a scoped way) directly into the component model.  Two small things, but they have far reaching consequences.

Resemblance works by allowing you to express deltas against existing components. i.e. take that component, remove 2 connectors, add a new port, and replace an attribute. The graphical side of evolve, however, always shows you the full structure, and in picture 3 of the article above, you can see the "delta marks" against a resembled component.

Evolution works by substituting a new component for an existing one.  It build on resemblance, so you can incrementally evolve an existing component.

So, the big question is what do adding resemblance and evolution mean?  Amongst other things:

 1.  complete extensibility.

You can always extend a system created in evolve via an evolution of a component, which lets you incrementally adjust the wiring, in a way which does not destroy the original. Unlike other systems where you have to explicitly design in extension points, in evolve the extension points are naturally created. System creation and extension are completely aligned.

2.  a better design experience

You can use resemblance directly when you design to express commonality.  Multiple resembance is supported. You can use evolution to express variants of things over time.  Want 5 slightly different variants with different combinations of test components wired in? no problems.

3. ability to rectify conflicts

Evolve allows variants to be created, and later joined back together, and any conflicts can be rectified using the same 2 constructs. i.e. one person can change or fork the system and add to it, and another can do the same, and you can join them up again later in the same model.

And this is made more robust because under the covers, this works via UUIDs, even though the user only ever works with names. This means that a large amount of refactoring ability already exists in Evolve.

Can you do the same things with Spring/Guice/Unity with lots of planning and careful management of names and extension points? Possibly, although it would be a lot of work and it would not be anywhere near complete. At that point, using Spring for this type of thing feels a bit like doing OO programming in C rather than C++.

Can you use a version control system to manage evolution? Sure, but it won't understand the underlying component model so it doesn't let you merge/rectify conflicts at a component level, or let you have multiple variants in a single model. It also won't support resemblance/inheritance. And it certainly won't understand the interplay between resemblance and evolution.

With Evolve however, these facilities are all built in with first class support.

Cheers,
Andrew

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.