Jens Schauder is software developer since 1997. He loves software development for the constant challenges and constantly changing environment. A great chance to learn and teach. He is also blogger, author of various articles and speaker at conferences. Jens is a DZone MVB and is not an employee of DZone and has posted 86 posts at DZone. You can read more from them at their website. View Full User Profile

The Truth About Estimates in Software Development

03.07.2012
| 8573 views |
  • submit to reddit

One of the most difficult tasks in software development is estimating also known as guessing. Here are a couple of thoughts about the topic.

  1. Estimates aren’t Commitments. At least they shouldn’t. Lets assume you estimate you need 1 hour from your home to the airport, through all formalities and into the plane. Do you leave your house 1 hour before departure of the plane? Probably not. You’ll add all kinds of buffers and leave your house a lot earlier. Same should apply to any kind of project. If you think it takes 6 months it is a bad idea to promise a delivery date in 6 months. Promise 7, 9 or 12 months, depending how bad it is when you show up late.
  2. Estimates aren’t measurements. If you want to build a shelf, you can measure the space you have available, and if you measure properly you can cut the boards according to that measurement and they will fit. If they don’t it is probably your fault because you didn’t measure properly. With the tools in your house you might be able to measure size up to a couple meters with an error of about 1/1000 and if you want to measure more precise you just have to invest in better tools and measure more carefully.Not so with software. If you estimate a task to take a day you might be done after a day. Or maybe after 6 hours. Or you give up after 3 days. There is no way to be sure. You can’t just put more effort in estimating to make it more precise. Tools won’t help you much. The only thing reducing the margin of error is actually implementing it.
  3. Some basics in statistics. If you roll a dice once you will score approximately 3.5 points. With a margin of error of about 100%. If you roll the dice 1 million times the average value will be 3.5 with pretty good precision. This is no accident, but pretty strong math. The basic idea is that you can add up indipendent random events and that the relative uncertainty is smaller for the added up events than for each single event. This is good news since if you estimate a project you typically break it down into tasks and estimate each task. Then you add up everything and get your result. Which has a smaller range of error.
  4. The rule above applies does not apply. It holds only when the events (guesses/estimates) follow a normal distribution and are independent of each other. Neither is true for software estimates. Go figure.
  5. We learn through feedback
    It’s almost impossible to learn anything without getting feedback on how you are doing. Imagine learning chess without anybody ever telling you when you make a bad or even illegal move. You just have a book with the rules and play games against yourself. How likely do you think it is to ever become a respectable chess player? Nil? I think so too. But with estimating we are often in such a situation. When you do waterfall projects you often do estimates for 100s of tasks in one go and only a year later you get the feedback: The sum of your estimates was 20% of. It’s really hard to learn this way. The situation improves a lot when you follow agile practices, where you typically estimate smaller batches, closer to the time when they are actually performed. This enables you to actually learn how long tasks take.

 

Source: http://blog.schauderhaft.de/2012/03/04/about-estimating/

Published at DZone with permission of Jens Schauder, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags:

Comments

Chuck Dillon replied on Wed, 2012/03/07 - 2:07pm

Thanks for the thoughts Jens.  Here's a couple of mine...

Estimate at a relatively high level of granularity.  Typically estimate in units of weeks with a task breakdown containing work items at least a week.  I've seen people with minimal statistical insight assume that if they break it down finer and finer they get more measurements and so better precision.  It's doesn't work that way.  If you break it down too fine your error grows as a percentage of work estimated.  In effect your ruler isn't precise enough to measure the small quanta of work.  Also when looking it in too much detail you tend to introduce bias so that error accumulates instead of averaging out.  If you keep the work items course you reduce bias and error is more likely to average out.

Think time-boxing.  Include nice-to-have work that can be deferred if stuff happens.  Estimate this work along with must-haves but know that when the unexpected happens you have some room to adjust with built in acceptability to the customer.

BTW, your #4 is wrong.  You don't need a normal distribution.  A single dice role has a uniform distribution.  This method does apply as long as you don't break the work down finer than your ruler can handle.

Jim Karabatsos replied on Wed, 2012/03/07 - 11:49pm

 

Good article, Jens and ditto for your comments, Chuck.

One of the things that I find so infuriating is the continual comparisons being made between software development and construction.  I've ranted at greater length on my blog at jimako.com/blog/2010/12/07/pure-design-is-pure-garbage but the central gist is that we are REALLY, REALLY GOOD at estimating the time to construct software, and it is VERY, VERY quick, because construction is the compile/build process, which essentially costs zero time and dollars to within an acceptable margin of error on any real project.

What we actually spend our time on is detailed design, because writing programs is a DESIGN activity.  The code we write is specifying the business rules at the most detailed level of precision, a bit like doing the final, detailed technical drawing and calculations for a new widget to be manufactured.  The actual creation of the widget is the compile/build process.

My experience with the construction of physical world artefacts is that those trades do far worse at the construction than the software industry does.  When I had my house built over a decade ago, it was an off-the-shelf design with no modifications either before or during the build.  Yet it took the builder (one of the largest in Australia at the time) 2 years to complete the project that was originally estimated at six months. Even then, I had to physically take possession and force their hand to get the project finished.  And this really was a CONSTRUCTION activity -- they were building to the exact same plan that they had built many times before. Their excuse was that the tradesmen were all busy in Sydney building infrastructure for the 2000 Olympic games.

Try asking a design firm to give you a precise and accurate estimate to design something. Make it something novel, something that is uniquely focused on your requirements.  What you will get is either a quote for an hourly rate (and one that makes our standard developer rates look pitifully low) or you will get a time-boxed or otherwise constrained proposal along the lines of "three candidate designs with one completed to detailed level" or "three iterations of feedback and modification".  I don't blame them, because design is inherently not accurately estimable other than in a broad sense and based on prior experience with "similar" work, whatever that means.

I am not trying to make excuses for our profession here (or more accurately, our craft). We are ultimately at fault for playing this unsustainable game, based on our unfortunately all-too-often valid fear that if we are honest and say that we don't really know how long something will take to within an unrealistic level of precision, someone else will give customers the answers they want to hear, get the work and then be paid for the work they do, even when it grows way beyond the initial estimate.

I don't pretend to have the answers to this dilemma.  I suspect that until our "master craftsmen" get together as some sort of guild (note I am NOT talking union here) which puts forward a standard of behaviour in estimation, and that guild is given some recognition and value by the customer community, we are doomed to continually being painted as inept, because our projects never come in on time or on budget when the real culprit is that the time and dollar budgets are (almost) always imposed by external parties or factors.

The other thing we need to get right is the need to have project managers who actually understand the software development process, but that's a subject of another post.

 

Erin Garlock replied on Wed, 2012/03/14 - 10:00am

Jim,

So your house had estimates for length of work to completion, i.e. X number of manhours, computed with Y number of workers gives you a finish date 6 months into the future.  I'd be willing to bet that your contractor actually spent X number of manhours building your house.  Ergo, it was accurately estimated.

The failure in delivery on an expected date, had nothing to do with estimation.  The flaw was in risk management.  Did anyone ask, what if there is a shortage of workers?  Surely it was known that the Olympics were coming.  I'd also assume salary was higher building for the IOC than building your personal home, and shortage of workers willing to work for typical residential construction wages might be a problem.  So what was the risk mitigation strategy?  Pay higher wages?  Wait longer?  Accept a lesser house?  In agile terms this is called the iron-triangle = time, cost, and quality.  Historically the mantra has been, "pick two" - which were you willing to sacrifice?  Doesn't sound like you wanted to compromise on anything, and thus the project was viewed disfavorably.

The guild for "a standard of behaviour in estimation" already exists, several in fact (e.g. PMI, PERT, etc.).  Pick your flavor of poison, stick to it, and refine your estimation process.  BTW, there was a wonderful talk about estimation on TED recently (http://blog.ted.com/2012/02/29/of-oxes-and-the-wisdom-of-crowds-lior-zoref-at-ted2012/), specifically addressing the use of having multiple estimates from different people.  On our projects, all of the developers give estimates for the same tasks, and we are pretty close in the end.  We have blown it from time to time, but it almost always comes down to some risk we didn't forsee and were bitten by.  Rarely is it that we just failed to perform.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.