Agile Zone is brought to you in partnership with:

Johanna Rothman helps managers and teams solve problems and deliver products. Her most recent book is Manage your Project Portfolio: Increase Your Capacity and Finish More Projects. You can read her blogs and other writings at jrothman.com Johanna is a DZone MVB and is not an employee of DZone and has posted 115 posts at DZone. You can read more from them at their website. View Full User Profile

Why Do We Estimate, Anyway?

10.29.2013
| 11649 views |
  • submit to reddit

I’ve been thinking about estimation these days. After the healthcare.gov site fiasco, and all the schedule games–many of which are estimation problems, I thought about why we estimate.

The larger the effort, the more we need to estimate. And, the more your estimate will be wrong. The more we estimate, the more we have schedule games. The smaller your effort, the easier it is to estimate.

That’s why I suggested you use agile approaches in this estimation series. You can break things down, and iterate. You get more information, and estimate smaller chunks. You are more likely to have accurate estimates.

Software is Not Construction; Software is Learning

Here’s one problem I have with estimation. Software is not construction. We can’t build software the same way we construct or manufacture anything. Software is all about learning as a team. We can timebox our learning. We can choose to stop doing something. We can put acceptance or release criteria around it and say, “We have done enough for now.”

But, we cannot say, “We can build this software for $xx per square foot.” We don’t know how to do that. Because we have not built exactly this software before. If we had built software like this before, we can estimate pretty darn close, because we either have historical data with good estimation quality, or we have small chunks of work we know about, or both.

When we estimate, other people think of our estimates the way they think of estimates in other fields, especially construction. Especially if you provide a single-point estimate. Even if you provide assumptions, which no one hears.

We estimate for these reasons:

  1. To provide an order-of-magnitude size/cost/date about the project, so we have a rough idea of the size/cost/date for planning purposes. An order-of-magnitude size means we want to invest just enough time in the estimate that we believe in the accuracy of it for planning purposes.
  2. We want to know when we will be done, because we are close.
  3. We need to allocate money or teams of people for some amount of time.
  4. Someone wants to know who to blame.

There is an alternative to estimating

Remember I said software is about learning? And, remember I said we never (okay, almost never) do the same software project twice?

Here’s something you can do. Make your features really small. Swarm (or as Woody Zuill says, mob) over every story every day. Always finish one or more stories every day.

If you always have deliverable software—this includes all tests, documentation, everything you need—you don’t need to estimate anything. You also gain the benefit of learning, so if someone asks, “How hard is thing to do?” the entire team can huddle together for a few minutes and say, “It’s this story and that story and this story, too.”

They then say, “We know it’s at least these three stories, and that’s off the top of our heads. Are those stories more important the ones at the top of our queue?”

What Do You Do? (Or, Prove It, JR!)

I don’t estimate my work. I work in chunks of work that take anywhere from 5 minutes to one hour. I rarely work in one-hour chunks. Most of my work takes less time than that.  I doubled my output this year by moving to smaller chunks of work.

I finished one book this year, and have another in beta. I have more books in progress. All while maintaining the same number of speaking and training engagements. I wrote roughly the same number of blog posts. I edited more agileconnection.com articles. Why can I do more?

Because my tasks are small. Because they are small I don’t have to estimate. I don’t have estimation time built into my work. I work in flow. That changes everything. I rank what’s most valuable at any time, and work on that.

Why Do You Estimate?

Why do you estimate? If you’ve estimated because you always have, think about it. If you estimate because your money people want to do once-a-year money allocation, well, you know that’s fiction. You can do it without detailed project estimation.

For money allocation, decide how valuable the project is to you. When does the project have to deliver the value? Now, tell the project team when the value has to be delivered. That’s all.

Remember, you hired these people because they were smart, responsible human beings. Stop with the phases and all that nonsense. Tell them what you want. Remember, the phases exist because management wanted to be able to cancel the project before it got too far along. You were supposed to show a deliverable and re-estimate at each phase. If don’t cancel, deliver, and re-estimate at each phase, your phases are not working for you.

Buy them a copy of Manage It! Your Guide to Modern, Pragmatic Project Management, which explains how to manage projects in any lifecycle. Give them a ranked backlog. Let them deliver. If they can’t deliver in the money or date frame, they will tell you. They are responsible humans.

If you need an order-of-magnitude estimation, fine. That doesn’t take days to determine. That takes hours. It will be precise-wrong and order-of-magnitude-right. Timebox it. It’s an order of magnitude. Don’t hold anyone to that estimate. (Quick, what’s the definition of estimate? “Guess.” That’s the definition. I kid you not.)

If you want to know when you’ll be done because you think you’re close to the end of the project, ask yourself this question: Is it worth the time to estimate vs. the time to finish? It might be. But know you are taking time away from finishing.

And, if you want to play the blame game remember that management is the one who needs to shoulder the most blame. Why? Because management set the constraints. Don’t believe me? Read the estimation series now.

I can sympathize with management’s need for estimates. I like order-of-magnitude estimates for many things. I even like specific estimates as we get closer. But creating software is not like driving someplace or like constructing a building. When I drive somewhere, I do want step-by-step instructions. When constructing a building, I do want an estimate. And even then, I am pretty sure the estimate is optimistic.

When creating software, I want to see working software as we create it, because with software, we learn. The learning is what’s most important. Because once we’ve learned enough, we can stop. That’s what’s most valuable. Not the estimate.



Published at DZone with permission of Johanna Rothman, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Fabien Bergeret replied on Tue, 2013/10/29 - 8:31am

Why estimating? Because of a difference way of selling projects: fixed-price projects. In my country (France), most of the software projects are externalized to software companies using this fixed-price paradigm: the client knows what he expects (the functional specs, the delay) and he knows how much he pays. The sub-contractor takes all the risks, and expects to improve its margin by putting more pressure on the project teams (more quality in less time).

The only way to win such a deal is to make the best estimation. If you estimate too high, you'll be too expensive and your competitors will win the deal. If you estimate too low, you lose all your margin.

Johanna Rothman replied on Tue, 2013/10/29 - 10:36am

 Hi Fabien,

Well, if you have a fixed-price contract, and you've estimated the best way you know how, it's not the best project management to put more pressure on the team.

The best project management is to say, "Here's what we thought it would take. Here's where we want to go. Let's do our best to achieve that." Then use small deliverables, rolling wave planning and steering the project (whether you use agile principles or not) to get there.

I wrote a 5-part series a couple of years ago on estimation. See http://www.jrothman.com/blog/mpd/2011/11/estimating-the-unknown-dates-or-budgets-part-5.html.

That doesn't help you finish your project. If you can't build trust with the customer first, can you build in slack or buffers for the inevitable problems that will occur? I wonder.

Good luck.

Johanna

Ian Mitchell replied on Thu, 2013/10/31 - 11:01am in response to: Fabien Bergeret

"... the client knows what he expects (the functional specs, the delay) and he knows how much he pays. The sub-contractor takes all the risks..."

In my experience the client *thinks* he knows what he expects and he *thinks* he knows how much he will pay. In other words the risk is the other way around. It's with the client and not the sub-contractor.

This is because of something called "The Curse of the Change Control Mechanism" and an experienced sub-contractor will be able to play this system like a violin. Any change to a specification, or even the hint of one, will result in a Change Request and an associated cost change. Even the time spent drafting a Change Request is potentially billable. A skilled supplier can double or even triple the projected revenue through Change Requests.

Having once worked for the Dark Side on such matters, I am not even a little bit surprised when I hear about how government projects go over budget. Frankly, I'm amazed that taxpayers aren't fleeced more than they are.

This is why it is so important to replace the Change Control Mechanism with a sensible contractual framework that is based on emergence, and to hire suppliers who can demonstrably work in such a way. That should rule out the "usual suspects" in big-project overspend.

Oleksandr Alesinskyy replied on Wed, 2013/11/06 - 6:53am

"But creating software is not like driving someplace or like constructing a building." - definitely, it is very much alike to constructing a (more-or-less unique) building or even driving someplace (off-road).

Soma Giri Parupalli replied on Wed, 2014/02/26 - 6:09am

 

“That’s why I suggested you use agile approaches in this estimation series. You can break things down, and iterate. You get more information, and estimate smaller chunks.” This is the best way to solve any problem Well said Johanna.

Dr. Dakshinamurthy kolluru explaind the same in the video tutorial given at

http://beyond.insofe.edu.in/category/essentialskillstoolkit/estimation/ listen to it and give us your opinion.

Johanna Rothman replied on Wed, 2014/02/26 - 9:10am in response to: Soma Giri Parupalli

 Soma, most of what those tutorials discuss have nothing to do with the kind of estimation I am discussing here.

I have since collected a number of my essays, and published a book on leanpub: Essays on Estimation . I hope you enjoy it.

Johanna Rothman replied on Wed, 2014/02/26 - 9:13am in response to: Oleksandr Alesinskyy

 Oleksandr, if by that you mean, some building we've never built before, or driving someplace we've never been before, where we have to incorporate learning? Then I agree.

With software, it's the incorporation of learning that is key.

Oleksandr Alesinskyy replied on Wed, 2014/02/26 - 10:15am in response to: Johanna Rothman

 Yes, you took my point 100% correctly.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.