Mitch Pronschinske is a Senior Content Analyst at DZone. That means he writes and searches for the finest developer content in the land so that you don't have to. He often eats peanut butter and bananas, likes to make his own ringtones, enjoys card and board games, and is married to an underwear model. Mitch is a DZone Zone Leader and has posted 2573 posts at DZone. You can read more from them at their website. View Full User Profile

Op4j Creator Talks About His 'Developer Happiness Tool'

05.03.2010
| 5369 views |
  • submit to reddit
Last week DZone had the opportunity to interview Daniel Fernández, the lead developer of the op4j library.  Also called the "developer happiness tool," op4j is a powerful implementation of the Fluent Interface concept.  It helps you polish the quality and readability of Java code with specific emphasis on auxiliary code such as structure iteration, filtering, mapping, data conversion, and more.  op4j is also a large repository for hundreds of functions with over one thousand different parameterizations.  Version 1.0 of op4j was released two weeks ago.  Fernández had some really interesting things to say about op4j's origin, performance, and usage.

DZone:  What motivations led to your authoring of op4j?

Daniel Fernández:  Quick answer: accident.

I'll explain:

I started developing what now is op4j back in 2007 and under a different project name.  At first I just wanted to write an extensible library for type conversions, for which I even wrote a runtime representation of the Java type system (which is now the javaRuntype [http://www.javaruntype.org ] project).

But when I came to write converters for arrays-of-x, lists-of-x, etc. it really turned nasty, and I started to think that I needed something that allowed me to write conversion code in a more readable way, something maybe a little bit similar to functional languages.  And so the op4j "fluent" core was born.

Dozens of thousands of lines of code after that, when I saw the results I thought "Wow, this is something else. This makes code look great!".  And so I went much further than just type conversions and created a lot of other useful features, which is the concept you know today for op4j.

So, as you can see, there were no exact "motivations" for authoring op4j as it is today.  It just happened.

DZone:  What other tools inspired the technology behind op4j?

Daniel:  First: Java itself.  Java is a WONDERFUL development platform, but the language is so rigid, so "straitjacketed", that sometimes it just spoils all the fun that we should find in coding.  So Java was my first inspiration... for change.

Also jQuery, Ruby and functional languages in general.  I love jQuery, and I really like the flexibility of Ruby enumerables.  Many ideas came from there.  Some of my best friends are brilliant Ruby and ex-Java programmers and they helped me identify things that could be improved in the way we code in Java.  Of course I also looked at some other technologies like lambdaj and linq, which I found really interesting.

DZone:  Could you briefly describe the Fluent Interface style, and how it relates to op4j?

Daniel:  "Fluent interfaces" is a concept that has been around some time (some would say since the times of Smalltalk) which tries to produce more
readable code by making it look, in some ways, similar to phrases in written English.

How?  Mainly, by means of method chaining.  For example, in op4j you could write a line like:

"Op.on(myList).forEach().exec(myFunction).get()", which you could almost read like "Operate on myList and, for each element, execute myFunction and get the results".  That is what "code fluency" is about.

DZone:  What types of Java applications will benefit the most from op4j?

Daniel:  Any applications with an important amount of "auxiliary" code: type conversions, collection/map/array management, etc.  To be honest, I think a very big percent of applications being developed nowadays have a lot of this stuff and could benefit from using op4j.  Unreadable auxiliary code is just everywhere.

DZone:  This question comes from a DZone community member:  "Do you have any performance figures?  I recall when I looked at lambdaj, they suffered a 400-1200% performance penalty compared to regular Java code.  In the end that was the main reason we decided not to use it.  Does op4j have similar issues?"

Daniel:  From what I know about lambdaj, lambdaj and op4j cores are very different in concept.  For example, op4j does not use any kind of dynamic proxy creation, which gives lambdaj a very interesting typesafe way of calling object methods... but maybe could be affecting performance in the scenario this member explains (just maybe).

The code you "plan" with an op4j chained expression is translated quite straighforwardly to equivalent Java code by the op4j core, which has been heavily -and I really mean heavily- optimized to make it as quick as possible.  The problem is, the only way of executing the same bytecode instructions as with "plain" Java code (and consequently, be as fast) is effectively writing plain Java code instead of using any libraries (be it op4j or any other library in your stack).

Some figures:  we have a small "benchmark test" I used during development in order to be sure I optimized the op4j core to the maximum.  It is not any kind of scientifically-correct benchmark, but it can give an approximate idea: The test takes a List with 1000 Strings and iterates it, converting each element to upper case before finally converting the whole list into an array. And it does this 1000 times, both using op4j and equivalent non-op4j Java code.

Timing in my poor old laptop: Without op4j: 828ms. With op4j: 1125ms.

Anyway, I always say that if you aren't a bad programmer and have confidence in your code, unless you are generating random numbers or parsing patterns or large Strings (which are time-consuming operations) you should only care so much about the performance of your in-memory code.  Just go optimize your input/output, that is the key.  And if you are apocalyptically worried about performance, don't use any libraries.  And write native code ;-)

DZone:  How do I get started with op4j?  How is it used?

Daniel:  op4j is just a Java library.  It is used in the same way you would use Apache Commons Collections or similar: just add it to your classpath, either manually or by using maven or whichever build tool you fancy.  Once in your classpath, you are ready to code. No configuration needed.

As for how to use it, op4j was designed with the "content assist" feature of IDEs in mind, as it will only offer you the methods that make sense at each step of your expressions. For example, if you have a String[], op4j won't let you write "forEach().forEach()"; the API will never offer that second "forEach()" method to you.  So there is no point in trying to learn the entire API from the start.  Just let op4j show you your options step by step by pressing Ctrl+Space.

And finally, a good place to start for examples is the recipes blog, "Bending the Java spoon". http://www.bendingthejavaspoon.com

DZone:  What new features are you planning for the next version of op4j and beyond?

Daniel:  I think after releasing a 1.0 version of an open source project like op4j, one should be mainly oriented to listening to the user experience for a while.  After the intense months of effort the team has gone through in order to publish 1.0, we will spend some weeks reading reviews and listening to comments and critics.  In my experience, feedback received after releasing something new is certainly the most important -and desirable- influence for future work.

And then of course there will be more functions, more fine-tuning, more examples... all the usual stuff in project evolution.

DZone:  Is there anything else you'd like to mention?

Daniel:  Let's make Java coding a little better --and more enjoyable!

Comments

Vladimir Vilinski replied on Tue, 2010/05/04 - 1:25am

This framework highly depends on reflection, and in a way that fast refactorings are not possible anymore.

For example: 

Map<String,List<City>> citiesByCountry = Op.on(cities).zipAndGroupKeysBy(Get.attrOfString("country")).get();

If someone renames field country, this code will not work anymore.

Daniel Fernández replied on Tue, 2010/05/04 - 3:13am in response to: Vladimir Vilinski

Hi,

"This framework highly depends on reflection"

Well... that is not exact. This library includes some functions (like the one you mention, "Get") that use reflection (in this case "java.beans.Expression" objects, to be specific), but it is a "feature" and not a "dependency". No part of the op4j core uses reflection, and 95%+ of the several hundred bundled functions (in fact, all of them except two) do not use reflection at all.

Unfortunately, because of the way Java is built, reflection is a need if you want to specify calls on specific methods of an object without really calling them or creating dynamic proxies.

Thanks! Daniel.

Mario Fusco replied on Tue, 2010/05/04 - 5:30am

Hi Daniel,

I am Mario Fusco, the creator of the lambdaj library. I had a look to op4j a few months ago and I must say that both the readability of your DSL and the number of the features you packed in it are amazing. Sincere congratulations for the release 1.0.

That said I believe that Vladimir hits the point in his comment. In the statement of his example if the name of the "country" property of the City bean is refactored in "nation" that statement stops working and it is hard to figure out why. The dynamic proxies implemented by lambdaj allow both to overcome this problem and to leverage the autocompletion feature of your favorite IDE when you need to refer to a property of a bean.

As for the performance issue, I agree with you: in memory operations are almost never the bottleneck of a given application. I heard complains about lambdaj performances many times, but the biggest part of the time they come from people who make a huge use of libraries like Spring and Hibernate that in turns are heavily based on dynamic proxy creation through cglib, exactly as lambdaj does. Don't you think this is weird?

As you did for op4j, I just honestly pointed out which are the pros and cons of lambdaj in order to allow developers to decide which is, case by case, the best tool to use in a given situation. As it always happens in our work, doesn't exist any Holy Grail.

Regards

Mario

Daniel Fernández replied on Tue, 2010/05/04 - 6:18am in response to: Mario Fusco

Hi Mario,

"[..] but the biggest part of the time they come from people who make a huge use of libraries like Spring and Hibernate that in turns are heavily based on dynamic proxy creation through cglib, exactly as lambdaj does. Don't you think this is weird?"

Absolutely true.
I am sure some people might see a harmful difference in performance in their applications because of using dynamic proxies, but most times this difference should be so small that it should matter only in those critical cases in which the use of libraries itself should be discouraged.

Regards,
Daniel.

Vadim Berezniker replied on Wed, 2010/05/05 - 6:41am

It's possible to make it more safer by combining it with something like bindgen.

J Szy replied on Thu, 2010/05/06 - 6:33pm

Well, I wouldn't call this sort of thing increasing developer happiness, or code readability. OK, maybe little happiness, but it certainly will increase maintenance headaches and costs. Here's why:

- increase in readability is only good for show, but, unless we're taking trivial examples there's none. Even what is advertised on author's site is a good example of why it's good to stick with plain old Java:

Author compares this:

Calendar now = Calendar.getInstance();
SimpleDateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy");
Set<Calendar> set = new LinkedHashSet<Calendar>();
for (String element : list) {
if (element != null) {
try {
date = dateFormat1.parse(element);
} catch (ParseException e) {
throw new SomeException(e);
}
Calendar calendar = Calendar.getInstance();
calendar.setTimeInMillis(date.getTime());
if (!calendar.after(now)) {
set.add(calendar);
}
}
}

 to this:

 Calendar now = Calendar.getInstance();
Set<Calendar> set =
Op.on(list).toSet().map(FnString.toCalendar("dd/MM/yyyy")).removeAllNullOrTrue(FnCalendar.after(now)).get();

 Op4j code looks nicer at first, but: I don't know what it will do if the format specified is invalid. I hope it will throw something, but what? Nevertheless, there should be a try/catch here but it seems omitted for advertisement purposes :-)

Next: I have only just learnt that map() will leave nulls untouched. I think it is very dangerous and breaks life-saving rule of failing fast. Of course, it lets get rid of one call to removeAllNull() and score some points on the comparison, but the cost is huge: most of the time null on the list would mean that something already went wrong and the best choice is to fail ASAP. If the user expects nulls, they should handle them explicitly.

And that's pretty much it: almost all other points are scored on utility functions. Utility functions are good, and we all know that standard Java library is quite poor here, and that Calendar/Date API is especially bad. If we abstract parsing String to Calendar away from the plain Java example, it becomes almost as succint as op4j example, more readable, and with less opportunity for nasty surprises.

But hey, this example was fairly trivial. But we can see more:

  
herbs =
    Op.on(herbs).forEach().
        ifTrue(FnString.matches("\\*.*?\\*")).
            exec(FnString.matchAndExtract("\\*(.*?)\\*",1)).
            exec(FnOgnl.evalForString("#target + ' (sold out!)'")).
        endIf().exec(FnFunc.chain(FnString.toLowerCase(),FnString.capitalize())).get();

(from author's blog)

I must say that I'm just staring at this example trying hopelessly to make any sense out of it. Good it's indented or it would be even worse (and no, IDE won't help). But let's see some (possibly) equivalent Java code:

final Pattern pattern = Pattern.compile("\\*(.*?)\\*");
final List<String> newHerbs = new LinkedList<String>();

for (final String herb: herbs) {
  final Matcher matcher = pattern.matcher(herb);
  final String rewritten;
  if (matcher.matches()) {
   rewritten = matcher.group(1).concat(" (sold out!)");
  }
  else {
   rewritten = herb;
  }
  newHerbs.add(StringUtils.capitalize(rewritten.toLowerCase()));
}

Yes, that's 14 lines of code, but 4 of them are either empty or just a closing bracket, so it's 10 lines of well formatted familiar Java code compared to 5 lines of arcane op4j code, including 2 subtly different versions of the same regexp and some textual embedded pseudolanguage! Speak readability and maintainability.

Now, if operations depending on regexp match were complicated I'd leave it just as it is, but since they are fairly trivial, I might try to use the ternary operator instead of if/else:

final Pattern pattern = Pattern.compile("\\*(.*?)\\*");
final List<String> newHerbs = new LinkedList<String>();

for (final String herb: herbs) {
  final Matcher matcher = pattern.matcher(herb);
  final String rewritten =
matcher.matches() ? matcher.group(1).concat(" (sold out!)") : herb;
  newHerbs.add(StringUtils.capitalize(rewritten.toLowerCase()));
}

 

Looks like I'm down to 7 lines of readable code (plus one empty, one bracket) and all this without any fancy DSL.

But wait, there's more:

- expressions like those in op4j will tend to be written in place, not in separate methods and that's a recipe for missed refactoring opportunities and duplicating code. 

One might ask, what's the harm in duplication of such short code? None if and only if it's the only place where such a code (or similar) exists in the project, but it hardly ever happens, and nevertheless it should be extracted away to a method, and IF programmers start to extract it, they will also start noticing similarities to other extracted code elsewhere and refactor them out. Imperatively written utility method will be fairly easy to split into two so one part can be shared, but how do I split an op4j expression? How do I refactor?

- Java syntax was designed for imperative, object oriented programming. Op4j tries to mimic functional style. It just doesn't fit, and it's painfully visible even with the two above examples. It's doomed to include horrors such as ifTrue().doThis().doThat().endIf() and so on. 

- the only real advantage is in functions like capitalize(), which, by the way, seem written just to score some points in comparison to standard Java APIs, but that's not much. Apache Commons has many of them anyway, prepared to use in plain Java code and not this DSL nonsense. And if some are missing, they might be implemented in no time.

Erik Van Oosten replied on Mon, 2010/05/10 - 6:58am

 From the article:

Daniel:  I think after releasing a 1.0 version of an open source project like op4j, one should be mainly oriented to listening to the user experience for a while.

 

Please tel me how to give feedback. I could found no mailing list, the forums are not read nor replied to.

 

Mitch Pronschinske replied on Mon, 2010/05/10 - 8:09am in response to: Erik Van Oosten

I'd recommend writing directly to the team.  They're very nice guys.

Erik Van Oosten replied on Tue, 2010/05/11 - 6:44am in response to: Mitch Pronschinske

Thanks Mitchell, I did.

Erik Van Oosten replied on Tue, 2010/05/11 - 7:05am in response to: Erik Van Oosten

Turns out that notification settings on the forum were lost at sourceforge.

Daniel Fernández replied on Tue, 2010/05/11 - 7:27am

Yup. Sorry for that!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.