Mario has posted 14 posts at DZone. You can read more from them at their website. View Full User Profile

No more excuses to use null references in Java 8

10.08.2012
| 40697 views |
  • submit to reddit
Tony Hoare introduced null references in ALGOL W back in 1965 “simply because it was so easy to implement”. After many years he regretted his decision calling it "my billion dollar mistake".

Unfortunately the vast majority of the languages created in the last decades have been built with the same wrong design decision so language designers and software engineers started to look for workarounds to avoid the infamous NullPointerException.

Functional languages like Haskell or Scala structurally resolve this problem by wrapping the nullable values in an Option/Maybe monad. Other imperative languages like Groovy introduced a null-safe dereferencing operator (?. operator) to safely navigate values that could be potentially null. A similar feature has been proposed (and then discarded) as part of the project Coin in Java 7.

Honestly I don't miss a null safe dereferencing operator in Java even because I can imagine that the majority of developers would start abusing it "just in case". Moreover, since the upcoming Java 8 will have lambda expressions, it will be straightforward to implement an Option monad that, as I hope to show in the remaining part of the post, is a far more powerful and flexible construct. 

I don't want to delve in category theory and explain what a monad is, even because there are already tons of very good articles doing this. My purpose is to quickly implement an Option monad using the Java 8 lambda expression syntax and then show how to use it with a very practical example. In Scala, a monad M is any class having the following 3 methods:
def map[B](f: A => B): M[B]
def flatMap[B](f: A => M[B]): M[B]
def filter(p: A => Boolean): M[A] 
In particular you can think to an Option monad as a wrapper around a, possibly absent, value. So an Option of a generic type A could be define as it follows:
import java.util.functions.Predicate;
public abstract class Option<A> {

    public static final None NONE = new None();

    public abstract <B> Option<B> map(Func1<A, B> f);

    public abstract <B> Option<B> flatMap(Func1<A, Option<B>> f);

    public abstract Option<A> filter(Predicate<? super A> predicate);

    public abstract A getOrElse(A def);

    public static <A> Some<A> some(A value) {
        return new Some(value);
    }

    public static <A> None<A> none() {
        return NONE;
    }

    public static <A> Option<A> asOption(A value) {
        if (value == null) return none(); else return some(value);
    }
} 
I also added some convenient factory methods for the Some and None concrete implementations of Option that I will implement later. Here Predicate is a single method interface defined in the new java.util.functions package:
public interface Predicate<T> {
    boolean test(T t);
}
that is used to determine if the input object matches a given criteria, while Func1 is another single method interface:
public interface Func1<A1, R> {
    R apply(A1 arg1);
} 
that I defined to represent a more generic function of one argument of type A1 returning a result of type R.

The abstract class Option has then two concrete implementations, one representing the absence of a value (something that we are used to wrongly model with the infamous null reference):
public class None<A> extends Option<A> {

    None() { }

    @Override
    public <B> Option<B> map(Func1<A, B> f) {
        return NONE;
    }

    @Override
    public <B> Option<B> flatMap(Func1<A, Option<B>> f) {
        return NONE;
    }

    @Override
    public Option<A> filter(Predicate<? super A> predicate) {
        return NONE;
    }

    @Override
    public A getOrElse(A def) {
        return def;
    }
}
and the other wrapping an actually existing value:
public class Some<A> extends Option<A> {

    private final A value;

    Some(A value) {
        this.value = value;
    }

    @Override
    public <B> Option<B> map(Func1<A, B> f) {
        return some(f.apply(value));
    }

    @Override
    public <B> Option<B> flatMap(Func1<A, Option<B>> f) {
        return f.apply(value);
    }

    @Override
    public Option<A> filter(Predicate<? super A> predicate) {
        if (predicate.test(value)) return this; else return None.NONE;
    }

    @Override
    public A getOrElse(A def) {
        return value;
    }
} 
Now, to try to put the Option at work with a concrete example, let's suppose we have a Map<String, String> representing a set of named parameters with the corresponding values. We want to develop the method
int readPositiveIntParam(Map<String, String> params, String name) { // TODO ... }
that, if the value associated with a given key is a String representing a positive integer returns that integer, but returns zero in all other case. In other words we want the following test to pass:
@Test
public void testMap() {
    Map<String, String> param = new HashMap<String, String>();
    param.put("a", "5");
    param.put("b", "true");
    param.put("c", "-3");

    // the value of the key "a" is a String representing a positive int so return it
    assertEquals(5, readPositiveIntParam(param, "a"));

    // returns zero since the value of the key "b" is not an int
    assertEquals(0, readPositiveIntParam(param, "b"));

    // returns zero since the value of the key "c" is an int but it is negative
    assertEquals(0, readPositiveIntParam(param, "c"));

    // returns zero since there is no key "d" in the map
    assertEquals(0, readPositiveIntParam(param, "d"));
}
If we couldn't rely on our Option we should accomplish this task with something similar to this: 
int readPositiveIntParam(Map<String, String> params, String name) { 
	String value = params.get(name);
	if (value == null) return 0;
	int i = 0;
	try {
		i = Integer.parseInt(value);
	} catch (NumberFormatException nfe) { }
	if (i < 0) return 0;
	return i;
}
too many conditional branches and returning points, isn't it? Using the Option monad we can achieve the same result with a single fluent statement:
int readPositiveIntParam(Map<String, String> params, String name) {
    return asOption(params.get(name))
            .flatMap(FunctionUtils::stringToInt)
            .filter(i -> i > 0)
            .getOrElse(0);
}
where we used an helper static method FunctionUtils.stringToInt() as a function literal, with the :: syntax also introduced in Java 8, defined as:
import static Option.*;
public class FunctionUtils {
    public static Option<Integer> stringToInt(String s) {
        try {
            return some(Integer.parseInt(s));
        } catch (NumberFormatException nfe) {
            return none();
        }
    }
}
This methods tries to convert a String in an int and, if it can't, it returns the None Option. Note that we could also define this behavior inline, while invoking the flatMap() method, using an anonymous lambda expression, but my advice is to develop a small library of utility functions, as I started doing here, in order to leverage the grater reusability allowed by functional programming. I think the comparison of the two readPositiveIntParam methods I provided illustrates well how the extensive use of the Option monad can finally allow us to write completely NullPointerException free software and, more in general, how a bigger employment of functional programming can dramatically reduce its cyclomatic complexity.
Published at DZone with permission of its author, Mario Fusco.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Mario Gleichmann replied on Tue, 2013/01/01 - 6:03am

 @Loren

Your comparison of LOC between your and Marios solution lack one important aspect IMHO. While you only have to code Options inner working only once and be done with it, your solution needs to go the extra mile and check for null (tangled with the logic what to do then in one case or the other) over and over again within every new use case.

Moreover, having implemented and tested the correct behaviour of Option, your solution need to test for correct behaviour of correct null handling also within every new use case.

@Erwin

Maybe another point (or just another real world example) that may speak for Option is that it allows for more 'honest' APIs (as oxbow_lakes already mentioned).
Think of a DAO which allows you to search for a user by Id. Under 'normal' conditions you would come up with the following signature:

<code>public User findUserBy( Long id )</code>

Now what will happen if you call that service with an unknown Id? The method 'promises' to return something of type User, yet it can't deliver a User for an unknown Id. It may return null, in which case the signature of the method isn't honest to you (it lies to you, as Dr. Eric Meijer would say, since it doesn't always deliver an instance of User). It also may return a Null-Object for an unknown Id. You simply don't know how the method behaves in that scenario just by looking at the method signature. If there are no tests (or specifications in the sense of BDD) your only chance is to take a look into the implementation, which kind of breaks encapsulation (at least at the semantic level) or write some test cases for yourself (which then needs to be found by the next developer which may face the same question when using the given service).

In contrast, look at the following, adapted method signature:

<code>public Option<User> findUserBy( Long id )</code>

Now it's crystal clear (at the API level) that this method may not return a User under all circumstances - it's encoded within the type this time, which makes the signature honest about its behaviour.

Hope this brings another aspect for the usefulness of Option into the discussion :o)

Erwin Mueller replied on Tue, 2013/01/01 - 8:49am in response to: Mario Gleichmann

 Happy new year.
I disagree, again :)

In my opinion, if you have a method called findUserBy() then it's already implied that maybe you can't find a user by the id specified. "find" implies that it's either found or not. IMHO "null" is always valid for "I can't find what you are looking for".

Now, in your solution you will return an Option<User> object. Now you have to test result == NONE to verify that you found a user. It's just the same, you test for NONE or you test for null.

But at least if you are return null instead of Option.NONE you will get later an NPE if you have forget to test that findUserBy() actually found any user. With Option.NONE you don't get a NPE and in the worst case all operations are done on a None-User and are lost.

Think of returning null vs. returning Option: what if all file operations just return /dev/null on default? Your code will never encounter any errors because /dev/null is a valid file. But any data is lost forever.

Runtime-Exceptions are your friend. They are not an inconvenience or your foe. They are "Runtime-Errors", or "compiler errors that are encountered at runtime". If you ever have a RuntimeException you have a bug in your code that the compiler cannot catch because they are encountered at runtime.

Look at the RuntimeExceptions: NPE, AnnotationTypeMismatchException, ArithmeticException, ArrayStoreException, BufferOverflowException, ClassCastException, UnsupportedOperationException, IndexOutOfBoundsException, StackOverflowError, OutOfMemoryError, etc.

They are all bugs in your code! You don't try and work-around a compiler error, why are you trying to work around your runtime errors?

IMHO the only mistake Sun make with the whole unchecked exceptions/checked exceptions business is that they called "unchecked exceptions" "exceptions". Sun had had to call "unchecked exceptions" "errors" and "checked exceptions" "exceptions". Because unchecked exceptions are errors or bugs in your code! And "checked exceptions" are normal exceptions that you can recover from.

IMHO Option just makes sense if you are certain that you can have a meaningful default value. If you don't have any meaningful default value, then null is the best option.

By meaningful, I mean meaningful for the user. Not the programmer to skip dealing with NPE.

Now my counter-examples. This method needs either to return null or throw an exception. Anything else is just wrong.

public User getUserBy(Long id)

This just waits for a catastrophe to happen.

public Option<User> getUserBy(Long id)

This method should return the user identified by the ID or the default user if no such user was found.

public User getUserBy(Long id, User defaultUser)


Raging Infernoz replied on Tue, 2013/01/01 - 9:57am

This is a waste of time, it is over complication, thus mental masturbation.

Null is the equivalent of mathematical zero for untyped references and untyped values; it is not wrong.  If you want typing for numbers NaN already exists and throws up it's own problems,  which make code more complex; just as this will.  This is no different than testing for a zero divisor before attempting a division which would otherwise result in an infinity value.  IMO null is actually useful, even for typed values, because it says a value was never provided, which can be very useful because you can test for it, and do a cheap early escape rather than do pointless and possibly costly subsequent pipeline code.

If you really must kludge functional code in a non functional language, check operation results and use filter functions.

This will never be added to Java because it is obviously stupid and will break existing Java code.

I'm unsubscribing from this noise now.

Jed Wesley-Smith replied on Tue, 2013/01/01 - 6:14pm

Raging Infernoz said:

> This will never be added to Java

Actually:

http://hg.openjdk.java.net/lambda/lambda/jdk/file/858e2f478d62/src/share/classes/java/util/Optional.java

Mario Gleichmann replied on Wed, 2013/01/02 - 6:10pm in response to: Erwin Mueller

 Erwin,

first of all also a happy new year to you!

Thanks for your extensive answer.

> ... "find" implies that it's either found or not.

The use of Option isn't restricted to methods which starts with 'find' ... :o)
Of course you're right to imply some behaviour from a methods name, but this isn't the most safest way to put trust into an (maybe foreign) API. Too many times i've seen some setter methods which triggered some state mutation as a 'side effect' (to bring a very extreme example to the table, admitted), so what's in a name anyway? As said before i would put more trust into the type system than into a methods name.

> IMHO "null" is always valid for "I can't find what you are looking for"

Are you really sure about that? What about a method which let's you search for a bunch of users by name, where you might expect more than one User as a result? In most cases you would come up with something like the following signature:

<code>public List<User> findUserBy( String name )<code>

Now what will this method deliver if you seach for an entirely unknown name? I 'bet' (and this might be very natural; at least i saw this 'contract' or idiom in every project i worked for yet, even Springs Jdbc- or HibernateTemplate behaves this way) you would implement that method to not return null in case no User is found but to return an empty list, right?

In this case, all your given arguments ('With Option.NONE you don't get a NPE and in the worst case all operations are done on a None-User and are lost', ...) are suddenly depleted, because in this case you also carry on and work with an empty List instead of null ...

Now take a look at the return types: In the first case you return Option<User> and in the second List<User>. Do you see some similarities? The API-Statements (on type level!) here are almost the same: While Option<User> says that the method is able to deliver zero to one User, List<User> says that the method is able to deliver zero to n Users. If you abstract over the cardinality you can absolutely work with the return value of both methods in the absolutely same way, e.g. using map, flatMap or filter, as Mario pointed out in the original post. Why making any difference when returning one User at Max (null) in contrast to delivering more than one User at Max (empty List)?

> Runtime-Exceptions are your friend

You are absolutely right. As you said some lines later, they show bugs in your code ('They are all bugs in your code') in the sense of that i want to be 'informed' if there's any bug in my code rather than let my application go on with some corrupted state or data! But as you said it right, they show bugs in the code, produced by developers, not by users of the application!

Before i go on, i assume that you agree with me, that we want to omit bugs in the code (introduced by development), right? Now i ask you, where did an NPE happen? Is it on the call side (the one which delivers null) or is it on the callers (or receiver) side? It's on the receiver side, mostly because we 'forget' the famous null check. Beside the fact, that those null checks clutter your code (tangling with your business or application or presentation logic), you tend to check for null on each and every location in your code. At least we saw, that NPE appear at the receiver side in case we forget to check for null. Using Option for stating the possible absence of a value you simply can't face this kind of bug any longer, because the possible absence is directly encoded within your type, rather than to bet if a given value might be null at all and check for it / handle it in an explicit way 'polluting' your actual logic. This handling is all captured within Option in conjunction with map and flatMap.

Mario Fusco replied on Thu, 2013/01/03 - 6:03am in response to: Raging Infernoz

> Null is the equivalent of mathematical zero

I don't get the meaning of this equivalence, but I hope you didn't want to say that

Integer i = null;

and

Integer i = 0;

have the same semantic.

Kris Nuttycombe replied on Thu, 2013/01/03 - 5:30pm in response to: Mark Hammons

No, when using Option, you essentially never explicitly check for None! That's the point. The compiler forces you to handle the case, but most of the time you will use map and flatMap to pass on the handling to an outer layer. That outer layer, of course, will also be forced to handle the case, which it would not if you were using nulls in a similar fashion.

A simple case to consider is obtaining a value from a java.util.Map. You have to check for null, and have a few lines of handling, *every* single place where you call get. It's very nice for the compiler to help you make sure that you've done so.

Erwin Mueller replied on Fri, 2013/01/04 - 7:23am

 First of all, you don't check for null.
You check for null if "null" have a special meaning, like No-Element-Found. For example, in findUser(name) you would check for null one time only, to make sure that you actually found a user. After that you never check for null.

In my code I check for null only on a few places. Like above, if null have the meaning of "No-Element-Found" or similar. And on API boundaries I check that a parameter is not null, if I'm 100% sure that the parameter cannot be null.

So my code looks like:

user = findUser("aaaa")
if (user == null) {
    Show Dialog to inform that we can't find the user
    And ask the User to input a new name.
}

some_api_method(User user) {
    Precondition.notNull(user, "Need to specify a user.");
    // notNull() will throw an NPE, but with a nice message

    // do something with the user
}

I only test for null if a) I can ask the user to enter valid data; b) I can specify default value for the null and c) I'm 100% sure that I get an NPE anyway, so I rather give the user a meaningful message then just the NPE with a line number.

Besides that I will never test if the user is null. If somehow the user is null, I get an NPE and I know that my program is buggy. So I don't know where you need to "clutter your code" with null checks.

So the Option class is useful, if and only if you can specify a meaningful default value that can be used. If you can't do that, then a null is IMHO better.

Also you can't compare the Option with a List. An empty list is always valid, it's just the definition of a list. Like in mathematics an empty set is still a valid set. (Also with a empty list you will get a IndexOutOfBoundsException if you try and access any element). I would argue here to use a NotEmptyList#decorate(List) (a list that throws an IllegalArgumentException if the underlying list is empty).

How about a method: getCountOfUsers("family name"). It can return 0 for "No Users with the specified family name found". But 0 is still a valid number just like an empty list.

But Option.NONE is not a valid anything. You just fool the runtime because Option.NONE is a valid reference, but for your domain it's not a valid value.

Mario Fusco replied on Fri, 2013/01/04 - 8:10am in response to: Erwin Mueller

First of all, you don't check for null.
You check for null if "null" have a special meaning, like No-Element-Found. For example, in findUser(name) you would check for null one time only, to make sure that you actually found a user. After that you never check for null.

The problem with this approach is how can you (or even better the users of your API) know where you have to check for null and where you don't need it? Returning an Option is much more informative for the users of your API because the returned type tells them if they have to consider the possibility that the particular method could eventually return "no value".

This approach is also more uniform to what (I hope) you do when a method can return multiple values. For instance what do you return from the following method

List<User> findUsersByCountry(Country country);

when there isn't any user in a specific country? If you return an empty list, as I think, instead of a null you can easily see why I am saying that using an Option for single values is more uniform: just think to an Option as a List that can have at most only one value.

Martin Vilcans replied on Fri, 2013/01/04 - 4:53pm

Thanks for an interesting and somewhat amusing read. As always with people arguing on the internet, everybody except me is wrong, so here I come to the rescue to clarify.


The problem is that you're arguing about two different things:

  1. Whether it's a good idea to signal to the compiler and the code's readers that a value may be null.
  2. Whether to use functional style programming to work with "null" values.

Let's have a look at this example function and function call again:

/** @return null if user is not found */
public User findUser(int id);

User u = findUser(someUserId);
System.out.println("User name:" + u.getName());

This is a programmer error. You have to remember to check for null, or suffer from the embarassing NPE.

I think everyone agrees that you don't want runtime errors because of programmer mistakes. Therefore some people (including me) think it's a good idea to signal to the compiler and programmers whether a value can be null or not. I'm inventing some syntax here:

public notnull User findUser1(int id);
// obviously never returns null
public nullable User findUser2(int id);
// obviously returns null if user is not found
notnull User u = findUser1(someUserId);
System.out.println("User name:" + u.getName());
// Guaranteed to never throw an NPE.
nullable User u = findUser2(someUserId);
System.out.println("User name:" + u.getName());
// gives a compilation error because u can be null

Some people may think this bloats the code without gaining much. That's OK. Many people think that static typing adds too much bloat. But when you're programming in a statically typed language, why not take advantage of the staticness and let the compiler find errors for us? (IMO, "notnull" should be the default, so there you get rid of some bloat, but for this example I wanted to be explicit.)

This syntax I made up is of course not valid Java, but you can implement something similar using generics, which is exactly what the Option template does.

So if you agree that it's a good idea to have "nullable"/"not null" as a part of an expression's type, you'd have to agree that having something like Option is a good idea. But maybe this implementation of Option is not your cup of tea. Maybe the functional style is confusing? Maybe there is a better way to implement Option? That is another question. If you don't like Marco's implementation of Optional, don't throw the baby out with the bathwater. Maybe another Optional, without the functional "mental masturbation" would suit you better?

Martin Vilcans replied on Fri, 2013/01/04 - 5:09pm in response to: Mario Fusco

Empty vs. one element list is a good explanation of Option. Unfortunately IMO, I'd prefer an exception instead of nothing happening in the case of a User missing. This is one of Erwin Mueller's arguments, which you haven't addressed in your replies.

I'm for a "fail fast" way of handling runtime errors. Instead of hiding the error, let the application blow up as soon as possible as close to the error's source as possible. I'd often prefer an NPE over nothing signaling an error and getting unexpected behaviour.

Example: jQuery, where $('#foo').show() tries to show the element with ID "foo", but does nothing if it doesn't exist. I'd prefer it to tell me straight away.

Kris Nuttycombe replied on Fri, 2013/01/04 - 7:44pm in response to: Martin Vilcans

Option can fail as quickly or slowly as you need it to; at times, there is a perfectly reasonable code path to follow in the case that the user is missing. In fact, usually you wouldn't use Option in a case where you wanted to signal an error condition; instead you'd use Either[MyErrorType, User] or similar (and there are combinators that allow you to go back and forth between Option and Either. The point is that the compiler forces you to choose what to do each time you want to do something with the user value. Simply giving up and throwing an exception (particularly allowing something like an NPE to occur) is not a valid solution because it's a bomb waiting to go off in your production environment.

In fact, you can kind of consider the use of Either where the left side is an error condition a reasonable alternative to using checked exceptions; you get all of the compile-time safety, plus a bunch of useful combinators that allow you to recover from or propagate errors in much more interesting ways than a primitive try/catch.

So, Option should be used for what it says on the package: instances where missing data is okay, or where missing data is expected and the absence of such data doesn't automatically indicate that something is wrong, for example, the return value of Map#get. If the absence of an optional value from a source like this is an error in your domain, then transform it into a correct error type.

Erwin Mueller replied on Sat, 2013/01/05 - 5:29am

 This syntax I made up is of course not valid Java, but you can implement something similar using generics, which is exactly what the Option template does.

So if you agree that it's a good idea to have "nullable"/"not null" as a part of an expression's type, you'd have to agree that having something like Option is a good idea.

No it does not. You cannot implement a language feature in Java yourself. I think in Lisp you can do that.

Maybe a nullable/not-null expression can be useful, but a Option type is not a replacement. nullable/not-null expression will generate a compiler error, the Option.NONE will not generate anything.

That is the whole point of my argument. Option.NONE is just to trick the runtime into thinking we have a valid reference, but we have not. Option.NONE is not valid in your domain.

Option<User> u = findUser(someUserId);
System.out.println("User name:" + u.getName());
What should happen here? Should Option just "invent" a user? Should getName() return an empty string? Should getName() return null? Should getName() return Option?

There is no reasonable thing to do for Option, except to throw an exception.

Erwin Mueller replied on Sat, 2013/01/05 - 5:35am in response to: Kris Nuttycombe

 Either[MyErrorType, User]

So we are using return-values to indicate errors? So back to C-style programming?

If you are returning MyErrorType if the user can't be found, why are you not just throw MyErrorType?

Gleichmann Mario replied on Sat, 2013/01/05 - 9:45am in response to: Erwin Mueller

Erwin, thanks again for your time to answer, giving some of your thoughts to the discussion!

> First of all, you don't check for null. You check for null if "null" have a special meaning

That's the whole point of the discussion: null shouldn't have *any* special meaning, including the misuse as a representation for the absence of a sensible value. Showing the absence of a value with null, it's all to easy to forget to check for it (beside the fact that you have to handle it as a special case). It's exactly that case we want to avoid using Option!

As your examples show more or less some simple, direct handling of a single value which possibly might be null, let me focus your attention to a somewhat more 'complex' example: Think of extracting some values from different Maps, using the value from the first Map as the key to the second and the retrieved value of the second Map as the key to retrieve the final value from the third Map. Of course, in each step the current Map could return null as there might be no value for the given key at hand, right? For a better understanding of this example you may want to visit some slides on slideshare (for some reason my former posts got blocked, i think because of the URL i put within - so please google using the following keywords: slideshare functional scala 2 in practice) and take a closer look at pages 100 - 109.

The proponents for Option would now argue that returning null when calling Map.get( K key ) in case no value is found for the given key is a wrong API desicion, because you have to handle / check the possible absence of a value (represented as null) in each step (as shown on page 102). And there you have it - your code is cluttered with null checks all over the place, because you nedd to - let me say it in your words - 'check for null if "null" have a special meaning' ! Now if, you don't think of that fact while implementing your logic and forget to check for null, you'll receive your NPE. I think you'll agree that this isn't wanted and should be avoided!

Would the method return an instance of Option instead, the handling for the possible absence of a value is now all handled / encapsulated within Option. As you see at page 103 (or at page 109, using some syntactic sugar by leveraging so called for-comprehensions) there's no null handling at all. Your code consists only of the core logic, no clutter, no null handling at all - and it's safe, you simply can't forget to not checking against the absence of a value, because this is all handled for you by the Option instance - this time the type is your friend (in contrast to using null) !

> Also you can't compare the Option with a List

Of course i can! :o)
This is only natural within the given context of showing the presence or absence of a sensible value (on the opposite - as i tried to explain in my previous answer - you cant compare the empty list with null!). Both, Option and List can be seen as a 'Container' in this
context, holding some value(s) or not. And in addition to that, the 'Container-Methaper' isn't the only valid point of view where Option and List share similarities. If you abstract further, you'll see that both types conform to some behaviour (or laws) that qualify both as a so
called Functor or even as a Monad. I wanted to avoid those words, honestly! But as you question that you can in fact compare Option with List, i need to point you to those concepts. There are many good resources out there, so i highly recommend to get some deeper insights on those concepts when talking about the similarities between Option and
List. It's exactly those concepts that let you work with Options and Lists in exactly the same way using map or flatMap (with or without syntactic sugar) without any need to not forget any 'special cases' like the absence of a value(s) and without getting bitten by nasty NPEs.

> How about a method: getCountOfUsers("family name"). It can return 0 for
"No Users ... But Option.NONE is not a valid anything

I'm afraid you're confusing here different ideas! Option should be used for showing the potential absence of any sensible value on the type level. Having said that, Option.NONE is especially NOT used to represent the numerical value zero, as zero is a valid, sensible value (not only in your example) ! So your method isn't an example for a scenario in that
you want to state the possible absence of a value at all, since your method can
always provide a sensible value! So i could also use it the other way round and say within that scenario, again in your own words 'But null is not a valid anything ...'

Erwin Mueller replied on Sat, 2013/01/05 - 10:20am

 I think we agree to disagree. :)

I think null is a valid value for No-data-found. Like an empty list is a valid value for Empty. And 0 is a valid value for Zero.

First, to your example with the map:

PredicatedMap.decorate(new HashMap(), NotNullPredicate.INSTANCE, NotNullPredicate.INSTANCE)

See http://commons.apache.org/collections/api-release/org/apache/commons/collections/map/PredicatedMap.html

Now you can chain your maps and you already made sure that neither the key or the value can ever be null. No need to clutter your code with null-checks and no need for Option.

Strange that you say that 0 is a valid value, but null is not:

User foo = null;
foo.getName()

int number = 0
result = 10 / number

In both examples the runtime will throw an exception. But somehow you need to workaround if it's a NPE but an ArithmethicException is all fine and well.

I have a last requests. Can you please explain what the code here should do:

Option<User> u = findUser(someUserId);
System.out.println("User name:" + u.getName());

And if you come to the conclusion that that code can only throw an exception, then explain why we not just throw an NPE and be done with it?

Gleichmann Mario replied on Sat, 2013/01/05 - 1:04pm in response to: Erwin Mueller

> I think we agree to disagree. :)

Well, that's also my impression :o)
But i think it's mostly because we argue on different grounds yet. I think what's needed is to fix the context or prerequisites on which we elaborate, as i will show you ...

> ... you already made sure that neither the key or the value can ever be null

Please don't confuse the above mentioned context here. I thought we already agreed that we talk about scenarios where it's perfectly valid to face the possible absence of a sensible value. That's THE point of the whole discussion here - show the possible absence of a value (Dear caller, expect to get NONE) by using Option and let the type 'prove' that the caller / receiver works in such a scenario in a valid way (avoiding NPEs at all within such scenarios) !

Your example lacks the appreciation of that context at all, since your contract now says, there must be no absence of a value - we expect to get a value under all circumstances (by using PredicatedMap you kind of put a contract around the Map to meet your expectations that there must be no absense of any of your expected values - otherwise it's an implementation flaw on the 'producer' side). This is not what we are talking about here!

If it helps your imagination, think of the mentioned three Maps as of three different Services (or DAOs), where you can call some finder-Methods ...

- EmployeeDAO.findProjectFor( String employee)
- ProjectDAO.findCustomerFor( String projectName )
- CustomerDAO.findLocationFor( String customer )

... where each finder might not find a project / customer / location for a given input (note, it is EXPECTED to potentially receive NONE result). Now think of a compound service of an HR application on which you can ask at which location one of your saff member is currently working. Again, your three finder-methods need to work in conjunction to produce a possible answer, right?  In this scenario, you need to 'chain' the outcome of one finder to the use of the next one while taking into consideration that each single finder might deliver NONE result. I hope you agree with me, that doing this in a save way is the preferred way to go! 

Now, using Option in such a scenario provides two advantages:

1. You simply can't accidently forget to check for null before do any further processing between each step as it might happen if you would use null. Of course it's fine to receive a NPE if using null and fail fast in case you really forget to check - but explain that to your Boss if he needs to locate his most important Developer in the next five minutes and the application answers with an exceptional state in production ...

2.  The handling of the absence is all done for you by the Option-instance. Esp. in case you 'chain' some operations based upon an optional value, the actual execution of any chained action is controlled by the Option (in case you're using map and / or flatMap for chaining)

> In both examples the runtime will throw an exception. But somehow you need to workaround if it's a NPE but an ArithmethicException

Again, please don't confuse the context. We're all talking about Option to overcome NPEs upon Null-References - that's the whole topic of Mario's original post!

It's NOT about to damn the concept of failing fast or Exceptions at all (thus, Java's concept of checked Exception is a quite not ideal implementation of an effect system) - it's about some alternative Options to avoid nulls and NPEs which are considered a better way to handle the expected absense of a sensible value. In case of using Option, we choose to have the type system on our site at compile time, rather than NPEs at Runtime (and by the way, if the type systenm could help me to avoid ArithmethicExceptions, i would happily buy into that!)

>  Can you please explain what the code here should do ...

Again, you only use a very simple, direct way to work with the possible absence of a value. The true power of Option really come into the game when chaining some operations, as i tried to demonstrate you in the above example.

For your question, it depends on your intention what to do (it might that you don't want to throw an Exception at all if you expect to not find a User). If you would like to print a string in any case (even there might be no user found), you could use map:

foo map ( _.getName ) getOrElse( "n.a." )

As you see, no Exception necessary in case of the above mentioned intention ...

> I have a last requests ...

As you had a request on me, i have a last question on you - please be honest:

Have you seen / listen to the presentation 'Null references - the billion dollar mistake' of Sir Tony Hoare (THE inventor of Null-References) as it's linked within the original post of Mario?

If yes, Sir Hoare is saying (something around Minute 19) that 'when you introduce Null-Pointer you either have to check every reference or you risk disaster'

Can you think of what Sir Tony Hoare had in mind when he was saying that ... ?

Kris Nuttycombe replied on Sat, 2013/01/05 - 3:19pm in response to: Erwin Mueller

Returning Either is isomorphic to throwing a checked exception in Java, with the exception that you have a better combinator library which gives you more possibilities for how to deal with the error; for example, you can combine the errors of two independent calls which might both fail.

I think it's a well-established tenet of modern software engineering that exceptions should be used to signal exceptional conditions, and explicitly *not* used for control flow. To take the findUser example, I would expect an exception if the database were unavailable, or an I/O error occurred or some such. If, instead of using Either, you are using a checked exception to propagate information about the presence or absence of data up the stack, that's really just an example of using exceptions for control flow. At least, it is if you plan to recover from it. I too believe in failing fast; so much so that I believe all exceptions should be unchecked and allowed to propagate all the way to the root of the application.

C-style programming, as you call it, has much to recommend it, particularly if you have a nice combinator library that allows you to manipulate both successes and failures effectively (though I realize you intended this as a derogatory remark implying that such a style is inherently useless.)

If you're actually interested to learn some of the virtues of this model of error handling and not simply dismiss it out of hand, I'd recommend taking a look at the scalaz Validation type. A short introduction can be found at the bottom of http://debasishg.blogspot.com/2010/12/composable-domain-models-using-scalaz.html

Kris Nuttycombe replied on Sat, 2013/01/05 - 3:43pm in response to: Erwin Mueller

I think null is a valid value for No-data-found.

The reason that null is problematic is that the null value represents a hole in the type system. Null has the bottom type, meaning that it can be assigned to any reference type. This is a problem because if you propagate a null (say one that was supposed to represent an absent User) to another part of the system, you have no idea what that null was initially supposed to be. By using null, you discard type safety entirely. This is, in fact, the most pernicious thing about NPEs; it's frequently difficult to figure out exactly where in the history of your computation a null value was derived. If you can trust your reference types to never be null, this problem goes away entirely. You may have an api that returns None for a user at some point, but in order to use that user value for anything else you're forced to deal with the possibility that it may be absent. If you naively pass it on to another method, you've lost information and your NPE may have a call stack entirely divorced from the place that the null originated.

In practice, it's rare that a method takes an Option as an argument unless the method can operate correctly in the absence of that data. On the other hand, it's frequent for methods to *return* optional values, if they're unable to produce a sensible result given the inputs. 

With respect to your last request, I believe it to be something of a strawman because here you're returning information directly to the user; most of the time you'll be passing it to some other part of the system, manipulating it, and so forth. Nonetheless, since the user must expect their program not to crash, the only sensible solution is (written in Scala):

val userOpt: Option[User] = findUser(someUserId)
val message = userOpt map { u => "User name: " + u.getName } getOrElse { "No user found for id " + someUserId }
println(message)

Erwin Mueller replied on Sat, 2013/01/05 - 8:35pm


- EmployeeDAO.findProjectFor( String employee)
- ProjectDAO.findCustomerFor( String projectName )
- CustomerDAO.findLocationFor( String customer )

So what should that methods return if there is no employee found, no project found or no customer found? It can only throw an exception or return null. Should it just invent an employee? Or use an None-Employee?

The only "save" way I can think of is this way:

Project p = EmployeeDAO.findProjectFor(employeeName)
if (p==null) { Inform the user that no project was found.}
else {
  Customer c = ProjectDAO.findCustomerFor(p.getName())
  if (c==null) { Inform the user that no customer was found. }
  else {
    Location c = CustomerDAO.findLocationFor(c.getName())
    ...
  }
}

Of course you can return a Option.NONE and test for Option.NONE instead. But it will be just the same code.

val userOpt: Option[User] = findUser(someUserId)
val message = userOpt map { u => "User name: " + u.getName } getOrElse { "No user found for id " + someUserId }
println(message)

--->

User user = findUser(ID);
String userName = user == null ? "No user found" : user.getName()
println(userName);

Hm that was a no-brainer.

Can you think of what Sir Tony Hoare had in mind when he was saying that ... ?

Yes I was listening. I don't think an NPE is a "disaster". In a lower level language like C a null pointer can be a "disaster" but not in Java where the runtime is protecting you.

He is also offer the alternative: the not-null statement. Since it will be part of the language, the compiler can check it and return compiler errors. The Option type is not an alternative to that.

Jed Wesley-Smith replied on Sat, 2013/01/05 - 11:51pm in response to: Erwin Mueller

You continue to miss the point. Yes you can do null checks, but you cannot ensure the checks are performed (modulo external static analysis tools). The point is to solve the problem of nulls as opposed to fixing the bugs nulls cause.

Option<User> is a type that is unambiguous. There may or may not be a user object and you can only access it via the correct API and are otherwise forced to deal with the lack of it. There is no way to express in the type that something may or may not be nullable, so the only alternative are ad-hoc opt-in schemes using annotation, documentation and naming. These schemes are all sub-optimal as they cannot be enforced by the compiler. You mention an alternative "not-null statement" that may be compiler checked, but this is an as yet mythical feature – Option is here noew.

Furthermore, this is not a syntactic pissing contest. In Java the syntactic overhead of the Option type is often the equivalent of null, sure. But it is semantically superior for many reasons as already outlined. There are various techniques that make the syntax more tractable though, and using languages that support lambdas (including Java8) make this all the much better. 

Now, clearly you don't need this as you do not care so much about NPEs for some reason. That is fine, this is only offered as an alternative approach, not something you must use. Others though have used it already to very good effect.

Gleichmann Mario replied on Sun, 2013/01/06 - 2:05pm in response to: Erwin Mueller

> The only "save" way I can think of is this way: ...

Thanks for your example. It was exactly my impression that you can't think of another way for handling the absence of a value other than check for null between each step! You just gave a nice demonstration for what i mentioned more than once in the preceding posts: cluttering your code with null checks (hopefully you not forget to check between two checks) instead of let Option do that work for you.

In addition to that, it seems that you didn't visit my example on slideshare, otherwise you had seen the alternative way using Option without any check for absence at all within your logic (using map and flatMap). For the sake of clarification i will show you how it might look like using Java 8 with Lambda-Expressions, so you hopefully get an impression for another way you can think of handling the absence of values:

public Location whereIs( String employeeName ){
  EmployeeDAO.findProjectFor(employeeName)
    .flatMap( p -> ProjectDAO.findCustomerFor( p ) )
    .flatMap( c -> CustomerDAO.findLocationFor( c ) )
    .getOrElse( HomeBase.Location );
}

Since it seems that you kind of ignore the statements of my last post, let me repeat the two fundamental advantages:

1. No possibility to forget to check against the absence of a value, because of ...

2. No explicit testing for a possible absence of a value - no cluttering, just chaining the business-operations, while Option do the rest. In case you still can't see why, here's a possible scenario:

Suppose the call to EmployeeDAO.findProjectFor( String employee) delivers a value, therefore we receive Some<String> (for the sake of simplicity; in apps with a domain modell we would hope for Some<Project>). Now, calling flatMap() on Some<String> with a given instance of Func1 (please look at Marios original post) will call that Func1 with the found project name, which in turn is used to call ProjectDAO.findCustomerFor( projectName ). Now suppose we would'nt found any customer for the given project, hence the resulting value is of type None<String>. If we now pass the next Func1 instance to that None, it won't get executed at all (again, take a look at Marios original post), so we stay with None<String>! After we've chained all operations we either have a Some<Location> or a None<Location> at Hand. Calling now getOrElse we either get the Location for the given employee in case all finder delivered some value or we fall back to the HomeBase location of the Employer. 

 > Hm that was a no-brainer.

You again seem to miss the point here. While it's all to easy to forget the null check in your example (ok, in your case, you doubtlessly would not forget to check against null, but what about your non-sensitized team members?), you simply have no other chance for getting to your value (if there is any) without paying attention to the possible absence while using Option. You simply can't ignore that case of absence while you can with null, resulting in a NPE (what the whole article is all about).

Alexandru Repede replied on Fri, 2013/04/19 - 10:18am in response to: Erwin Mueller

the monads are not workarounds for business bugs, the problem is still there, just solved differently. the monads offer an alternative to solving a missing value, but the missing value is still there and needs fixing.

on the other hand, you make a good point with "Why didn't you just test that the user have entered a valid number at the GUI level and inform the user that he/she have to enter a number?", which says "i might want to actually display an error to the user". well, in this case, i can say that current implementation is not what is needed (the monad is not the right one; you need a monad that stores exception or exception messages, and a different readPositiveIntParam that does not catch and transforms the bad value into a zero, but just throws the Exception)

Alexandru Repede replied on Fri, 2013/04/19 - 10:34am

<replaced>

Alexandru Repede replied on Fri, 2013/04/19 - 10:33am

 your implementation of Some is not correct (well, not complete). at least for the map function.

what if the function f returns null? then, you do some(null), which pretty much annihilated the whole point of this monad.

i think some extra check is required there, and if the return of that function is null, then map should return none()

Kris Nuttycombe replied on Fri, 2013/04/19 - 12:36pm

Alexandru: in a well-formed application written in a functional style, returning null from any function is an error. That's the essence of Option; it is intended to be used in cases where the absence of a value is expected. And, this is the problem of null - having nullable references means that *every* reference is suspect.

Many, if not most, states of any given program require that essentially no variables hold null references. Those states where it is *expected* that a value be absent should make that information accessible to the type system, by including that semantic in the type of the value. This is the whole point of the Option type. The compositional properties that arise from Option having a monad are just gravy. Why have a static type system if you can't use it to encode the semantics of your program? The presence or absence of a value is a tremendously important bit of semantic information, every much as the difference between a single value and a collection of values. You wouldn't hesitate to encode the latter information in a way that the type system can see it; why would you hesitate with the former?

Luca Cavagnoli replied on Wed, 2013/04/24 - 10:58am in response to: Loren Kratzke

@Loren

It is better to have one null check or a try/catch block in a reusable util class, rather than 1000 times in top level code.

Peter Lawrey replied on Thu, 2013/12/26 - 7:14am in response to: Jed Wesley-Smith

>  It has been battle-tested and proven to be superior to the alternative you are peddling.

Its so new it is still in EA and poorly understood by most Java developers.  I would be interested in your proof that closures either make the code simpler or eliminate common programming pitfalls in Java.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.