I've been a zone leader with DZone since 2008, and I'm crazy about community. Every day I get to work with the best that JavaScript, HTML5, Android and iOS has to offer, creating apps that truly make at difference, as principal front-end architect at Avego. James is a DZone Zone Leader and has posted 639 posts at DZone. You can read more from them at their website. View Full User Profile

The Dark Art Of Software Estimation: How Do You Estimate?

09.30.2010
| 10793 views |
  • submit to reddit
If there's one surefire way to make a developer uncomfortable it's to ask for an estimate for work to be done. No one really likes to do it, but it's a necessary evil. But how good is the quality of your estimate? A colleague of mine recently sent on an article about "Bees and the Art of Estimating". The idea of the article is to "write your estimate for the number of insects in an average hive of English honeybees".

If you're like me, you'll have put down some number with some level of reasoning, but without any real analysis. You probably didn't ask yourself the following questions that the article poses:

"Oh, you don't know anything about bees? Well, this is an estimating task. Estimating is about predicting in the face of uncertainty and incomplete knowledge, so that's no bar. We estimate for unfamiliar software projects every day. You don't bother? Oh! How ever do you prioritize your set of possible tasks (that's a rhetorical question)? What do I mean by "average"? Now, that's a good question. Do I mean the "mean population of bees in a hive over time, taking a typical annual cycle into account"? Or do I mean "the mean, calculated from the total population of bees in England at a specific time, divided by the number of hives in England"? And anyway, what do I mean by "insect"? Should you include critters such as mites, pillbugs, and wasps that live on, in, or among the bees? Hang on! Bees are hive animals; arguably the queen and her workers and drones constitute a "single genetic individual," so should you just count the number of queens? And if the hive is on the point of swarming (May or June maybe), should you count the queens that leave to find a new hive?"

So for a start, you probably didn't go deep enough into analysis to provide the estimate. And when you did, you produced a simple number, not even a range.

The article really made me question my estimation process. It made me feel a bit stupid about the whole thing - surely I should be asking more questions when I'm asked for an estimate.

So what type of estimator are you? And what type of process do you follow when estimating development tasks?

- The Optimistic Estimator
You always think things will get done quickly and that there will be no problems along the way. But you always find yourself caught for time at the end.

- The Pessimistic Estimator
You always assume the worst case, even if it doesn't turn out that way. When you find that you have more time than you really need, you manage to fill it up anyway. (Although I'll admit, sometimes negative estimates still leave you stuck for time)

- The Iterative Estimator
You've found the perfect balance - you keep refining your estimates along the way, ensuring they get brought into the project plan accordingly. As well as knowing you'll have the right amount of time, you ensure that all spare time is used wisely.

For the record, I still classify myself under optimistic, but hopefully I'll get the balance right sometime soon.

Comments

Casper Bang replied on Thu, 2010/09/30 - 9:32am

We play cards about it, 50% and 90% estimates.

Karl Peterbauer replied on Thu, 2010/09/30 - 10:07am

Simple empirical strategy: Keep records of initial estimations and effective efforts and calculate your personal estimation error factor. Multiply future estimates with this factor.

Stephane Vaucher replied on Thu, 2010/09/30 - 10:41am in response to: Karl Peterbauer

Similar to Karl. I keep track of how long it takes for me to do tasks I've done in the past to use it to forecast the time of future tasks. The more similar tasks I've done in the past, the more accurate my forecasts.

Lon Rankin replied on Thu, 2010/09/30 - 11:23am

FWIW, what most us fail to remember, primarily because our tools are inadequate for this process, is that any estimate is a probability distribution because few tasks are exactly like another and few teams stay exactly the same from project to project. Any attempt to generate a single value is a fool's errand. My approach for any estimation is similar to the original PERT approach, survey some "experts" to get a range of numbers, then modify the range based on my gut feel for the level of knowledge for this task or project. Then, if the project is big enough, I do a Monte Carlo simulation to develop a range of probable completion dates and quote the range and the most probable date. All the while I remember I am the bitch of the law of large numbers...

Karl Peterbauer replied on Thu, 2010/09/30 - 11:42am in response to: Lon Rankin

Any attempt to generate a single value is a fool's errand.

Well said. Unfortunately managers, controllers, sales people and customers are commonly not familiar with the basic principles of stochastics. 

Alessandro Santini replied on Thu, 2010/09/30 - 3:52pm in response to: Karl Peterbauer

And programmers too often are not familiar with the concepts of efficiency, contracts, schedules and project control.

Raw ThinkTank replied on Thu, 2010/09/30 - 10:06pm

Dont play poker with Software development.

 

 

Maxim Zakharenkov replied on Fri, 2010/10/01 - 2:59am

Guys, you don't take into account the implementation part. In case developer does estimate and implementation he should probably estimate it longer than it takes (a bit). Than he'll be able to fit into the schedule and that is really good point for the managers. In case he completes the task faster it doesn't mean he always tells about that. So he can be more relaxed, have time for reading dzone etc :)

In most cases developers have some internal knowledge about their project managers. E.g. how whould they believe  if the estimates are too pessimistic and how much pessimism can be accepted. So, experienced developers can adjust their estimates to that factor as well :)

Karl Peterbauer replied on Fri, 2010/10/01 - 4:36am in response to: Alessandro Santini

@Allessandro: Estimations are nothing else than mean values of probalistic distributions. It's a common phenomenon that the development team's carefully crafted estimation finally ends up as single number in the project's executive summary. Decisions, budgets, contracts, time schedules and HR planning are then made based upon this number, adding reserves here or there based on experience or simple gut instincts. This is the "Dark Art of Project Management". Finally somebody has to be blamed when the project is late, even if the correct answer is sometimes quite simple: It was an estimation, after all ;-)

Risk as function of probability of occurrence and size of loss, propagation of uncertainty etc. are well-defined concepts, but widely ignored in practical project management.

Alessandro Santini replied on Fri, 2010/10/01 - 4:51am in response to: Karl Peterbauer

Karl,

I totally appreciate your point but I am pretty sure you have had some  exposure to the technical sales facet of our profession. RFPs commonly require estimates and costs; the difference between winning and losing a contract lies in those raw, awful and pointless figures; unfortunately, those are the figures (among others) read by the customer when choosing a provider.

It is very unlikely that someone goes to the customer after - let's say - three months and says "we regret you owe us another 10,000,000 € because those were only estimates".

Karl Peterbauer replied on Fri, 2010/10/01 - 5:40am

Alessandro,

you are absolutely right, RFP's boil down to some raw figures, and claiming more money from the customer is generally not an option. But neglecting the inherent risk lying in these compressed figures is indeed a fool's errand. It's a question of responsibility: Who takes the risk? Who earns the benefit? Don't blame developers for carefully made estimates which a posteriori turn out to be overly optimistic. Blame the people in charge for inappropriate risk management ;-)

Brian Reich replied on Fri, 2010/10/01 - 8:15am

I'm definitely an optimistic estimator and constantly find myself in a pickle towards the end of the project.  Sometimes it's because of changing project requirements or clients that simply don't know what they want until you give it to them, but more often than not it's caused by a willingness on my part to give the client what they want (a great product, fast and cheap).  As they say, you can only choose two!

Adam Davis replied on Fri, 2010/10/01 - 12:13pm

As a former optimistic estimator, I found a technique (long ago) that helps me focus on how complex a given issue is/could be instead of focusing on how fast I think I can resolve it. My technique is based on an article by Arpad Elo, Jr. in the May 1994 issue of Software Development magazine. It's called "An Algorithm for Estimating Development Costs". Unfortunately I have not been able to find this article online.

The article includes formulas for estimating both the time needed to specify a system, and to construct and verify that system. I've only used the second formula:

    C = sum(X[i]) * P * L

Where C is the work days to construct and verify the system, X[i] represents the scores asigned to the complexity terms, P is the personnel proficiency factor, and L is the (computer) language factor.

Complexity terms are scored from 0 to 5 for five different facets of the system (where 0 is no work at all and 5 is extremely difficult). The article suggests the following facets: Data storage and retrieval, Numerical algorithms, Non-numeric algorithms, Conditional logic, and Interactions with other software. Personally, I lump the two algorithm facets together and add another facet for Screen/page design.

In the article, the personnel proficiency factor can vary from 1 (a guru with detailed knowledge of the application) to 8 (a trainee with no knowledge of the application) and the language factor from 0.5 to 8. Personally, I usually ignore the language factor.

Anyway, I have found that this helps me focus on analyzing how difficult the problem is instead of focusing on how fast I think I can solve it.

Paul W. Homer replied on Fri, 2010/10/01 - 12:46pm

I worked in a small startup were we lived (and nearly died) based on the estimations. At first it seems impossible, but after a while, if it matters enough, you get the hang of it. A few suggestions: Break the estimate into pieces. Always have a sense of the number of lines of code, pages of documentation, number of meetings, etc. The more concrete the small pieces, the more likely you'll be correct. Assume at least a couple of major hiccups. Don't use wild-estimates, write them in as *. After an estimate, pick a fixed amount of time for the work (which crystalized the *s), but balance that against having to drop some of the lessor requirements. Use ranges for all work. At the end you'll get something like 3-4 weeks of work, and we will do these five features and if we have time these other two (in this priority order).

Now customer's hate a range, and they hate the fact that somethings may get dropped. To get out of this plan a series of 'phases'. The long long term estimates, if your unlucky, will be wrong, but if you tie billing (or success) to only the first phase, then you can keep the customers under control. At each phase, re-estimate, whether it needs it or not. Use the failures from the earlier phases to help predict the latter ones. Be careful of any *s.

Estimates can work, but doing them isn't trivial (and requires its own set of skills and knowledge).

 

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.