Chas Emerick is the founder of Snowtide, a small software company specializing in document content extraction service sand libraries for Java and .NET, and a co-author of the forthcoming "Clojure Programming" book from O'Reilly Chas is a DZone MVB and is not an employee of DZone and has posted 17 posts at DZone. You can read more from them at their website. View Full User Profile

The placebo effect is what makes the software world go ’round

  • submit to reddit

I’ve been of the opinion for some time now that software development, regardless of the methodology followed or the tools used, is not an engineering discipline (unfortunately), but rather is a craft.  I recently laid out that opinion in some detail, which was quickly followed by many people (both in the tubes and in private communications) mostly disagreeing, some suggesting that I just don’t understand engineering particularly well.

In general, I’ll quickly concede that last point, but I keep running into people who ostensibly do understand engineering, and who also reject the notion of software development being a variety of it.  As an example, if I had known of Terence Parr‘s essay “Why writing software is not like engineering” before writing on the same topic with essentially the same premise, I would have simply tweeted a link or something.

A more pointed, and in my opinion, brilliant restatement of this thesis was delivered by Linda Rising in a talk at the Philly ETE 2010 conference in April of this year.  The focus of the talk was far, far broader than the topic of how to classify and characterize software development, and very much worth a listen (audio of the talk is available).  But for now, the key relevant quote comes at 25:15:

One of the things we can learn from psychology is that they do real experiments…whereas in our industry, we do not.  We cannot call ourselves “engineering”; [our work] is not based on science.  In fact, last year at the Agile conference I gave a talk1 that said “I think that mostly what runs this industry is what you’d call the ‘placebo effect’.”

Think about [what would happen] if drug testing were done the way we decide to do anything in software development.  We’d bring in a bunch of pills, and we’d say, “Oh, look at this one, it’s blue! I love blue! And it has a nice shape, oh my goodness, it’s hexagonal! Hexagonal things have always had a special place in my heart.  I think these hexagonal blue ones will be very powerful in solving this disease.  Let’s go with the hexagonals.

I’ll give this hexagonal blue one to all my friends.  I’ll tell them, “This is it, this hexagonal blue one will solve all your problems.  My team tried it, and we really liked it, so your team tried it, you’ll really like it too.”

And that’s how we run projects.  There’s certainly no double-blind controlled experiments.  Is agile any good?  Oh yeah, it is, there’s so much excitement, and  so much buzz, and everybody seems to think so!

Most if not all software developers will recognize the parallels between their profession and that parable.  Why did your team choose C++, or Rails, or F#, or Clojure?  There are often some tangible, technical considerations involved in such choices.  However, as Ms. Rising covers in the talk, human beings are generally not capable of making “rational” choices, but we are stellar when it comes to rationalizing the choices we have made.  So, even our most fundamental decisions – which tools and methods to use – are not made with the rigor that would be demanded of decisions made in an engineering setting.  Of course, this is entirely separate from how we go about actually building things using those tools, which can only be characterized as relative chaos.  How and why we are able to make such abundantly arbitrary choices is something I addressed in my last post on this topic.

And here is where I appeal to a somewhat more respected authority.  Aside from her expertise in the software methodology space, Ms. Rising’s credentials are fairly impeccable, apparently having had some significant involvement in the software development process attached to the design of the Boeing 777 airliner.  So, I’d say she’s in a fairly secure position to be making informed comparisons between engineering, science, and what we do when we build software systems.  Just one more data point, really, but a strong one.


1 A description of this talk is here, but I wasn’t able to find a recording of it.  However, there is an InfoQ interview with Ms. Rising on the topic.



Published at DZone with permission of Chas Emerick, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Mike Dzone replied on Mon, 2011/09/12 - 8:26am

I have to totally agree.  For years I have said what I do is more of an art form than engineering and people have laughed at me - software development, an art?  But saying "craft" is I think a much better word because that is exactly what we do.  As software developers we are basically given a problem and asked to craft a solution, and there is no discipline of engineering in this at all.  I remember back 10 years ago when software development being akin to drawing out blueprints was all the rage. We "should" be able to design everything, get the design correct, then just code it up. Of course this failed misserably.  Why?  Was it because we can't do it?  No, we certainly can do it, but think about it, suppose you want to build a new skyscraper or a new bridge.  How many years (or decades even) does it take to go from blueprints, proof of concepts, and testing to actually building the real thing?  Now just imagine whatever software project you're working on you telling your company it'll be 3 years just to design the thing before any development even starts!  That probably won't sit well with the company. 

Mikael Couzic replied on Mon, 2011/09/12 - 4:18pm

human beings are generally not capable of making “rational” choices, but we are stellar when it comes to rationalizing the choices we have made

I would even go further, saying that being too "rational" in our choices would actually be a bad idea, because no matter how much effort we put into analysis, we always have to make decisions based on incomplete information.
In such situations, the capacity of our brain to make a logical choice is completely overrun. It is a much better strategy to make a less "rational" choice.
I'm not saying that we shouldn't try to get as much information as possible before making a decision, I'm saying that once we have, we shouldn't try to reason out a crooked logical proof, but rather "go with the flow" and trust our intuition. Sometimes, being logical is just the fastest path to being dumb.

OrangeBook replied on Tue, 2011/09/13 - 2:09am in response to: Mikael Couzic

Exactly. The reason engineering techniques are not applied to software is that it is too complex. The possibilities domain is too big to apply a set of defined rules on it and come up with the best way. And thus we rely in "intuition", which is a word for mechanisms in out brain that we don't understand.

It is right the same than chess.. a chess player is not applying any logical procedure to decide his next move (at least not consciously), but still can do very well. He cannot tell you the rules for playing well (not in a formal, logical way) but still can play well. The difference is for chess we already have computers powerful enough to calculate lots of possibilities and make a good game. Good luck with doing that with programming.

Erin Garlock replied on Tue, 2011/09/13 - 7:42am

I'd have to say the real reason we don't do traditional engineering is most software projects don't require it.

To use a different analogy, think about all those disposable razors and razor blades - cheap, easily obtained, and for all intents and purposes total junk, but they get the job done - kind of like facebook games.

Most software won't kill someone if there is an error, or have any other sort of catostrophic effect.  Then again, software tyically doesn't come with a $260M pricetag (Boeing 777) and a $1.5B development upgrade to production facilities (Boeing 777).  And to top it all off, the software makes money - sometimes tons of money (unlike today's airlines), and that is the real reason businesses exist.  Airlines could do the same thing with inferior planes (they do in other countries), but there's all the lawsuits that occur if one goes down.  So you have to ask, what does the risk/reward equation look like?

As for the drug analogy, it's flawed.  Next time you're at the doctor's office ask to see a copy of the PDR (Physician's Desk Reference).  It's a comprehensive reference book detailing the composition and accepted applications of pharmaceuticals from major manufacturers, and there are some surprising revelations in it.  You'll find things that effectively say, "we have no idea how this works", "this only works sometimes", "you might have to dosomething else weird to get a positive effect", "research is incomplete", etc. - all very much the same things we hear in software developement.  Also, many of the items in the PDR have the same adverse side effect - may cause death! 

Personally, I'll take a program crash on a $100 piece of software over a trip to the morgue.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.