Agile Zone is brought to you in partnership with:

Nicolas Frankel is an IT consultant with 10 years experience in Java / JEE environments. He likes his job so much he writes technical articles on his blog and reviews technical books in his spare time. He also tries to find other geeks like him in universities, as a part-time lecturer. Nicolas is a DZone MVB and is not an employee of DZone and has posted 232 posts at DZone. You can read more from them at their website. View Full User Profile

Can we Put an End to this 'Estimate' Game of Fools?

04.14.2014
| 4148 views |
  • submit to reddit

When I was a young software programmer, I had to develop features with estimates given by more senior programmers. If more time was required for the task, I had to explain the reasons – and I’d better be convincing about that. After some years, I became the one who had to provide feature estimates, but this did not mean it was easier: if the development team took more time to develop, I had to justify it to my management. Now, after even more years, I have to provide estimates for entire projects, not just fine-grained features.

But in essence, what do we need estimates for? For big enough companies, those are required by internal processes. Whatever the process, it goes somewhat along these lines: in order to start a project, one needs estimate in order to request a budget, then approved (or not) by management.

Guess what? It doesn’t work, it never did and I’m pretty sure it never will.

Note: some organizations are smart enough to realize this and couple estimates with a confidence factor. Too bad this factor has no place in an Excel sheets and that it is lost at some point during data aggregation :-(

Unfortunately, my experience is the following: estimates are nearly always undervalued! Most of the time, this has the following consequences, (that might not be exclusive):

  1. In general, the first option is to cancel all planned vacations of team members. The second step is to make members work longer hours, soon followed by cutting on week-ends so they work 6/7. While effective in the very short-term, it brings down the team productivity very soon afterwards. People need rest and spirit – and developers are people too!
  2. After pressure on the development team, it’s time to negotiate. In this phase, project management goes to the customer and try to strike a deal to remove parts of the project scope (if it was ever defined…). However, even if the scope is reduced, it generally is not enough to finish on budget and on time.
  3. The final and last step is the most painful: go back to the powers that be, and ask for more budget. It is painful for the management, because it’s acknowledging failure. At this point, forget the initial schedule.
  4. Meanwhile and afterwards, management will communicate everything is fine and goes according to the plan. Human nature…

In some cases (e.g. a bid), that’s even worse, as the project manager will frequently (always?) complain that those estimates are too high and pressuring you to get lower ones, despite the fact that lowering estimates never lowers workload.

You could question why estimates are always wrong. Well, this is not the point of this post but my favorite answer is that Civil Engineering is around 5,000 years old and civil engineers also rarely get their estimates right. Our profession is barely 50 years old and technology and methodologies keep changing all the time.

I’m not a methodologist, not a Project Manager, not a Scrum Master… only a Software Architect. I don’t know if Agile will save the world; however, I witnessed first-hand every upfront estimate attempt as a failure. I can only play this game of fools for so long, because we’re all doomed to lose by participating in it.


Published at DZone with permission of Nicolas Frankel, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Fabien Bergeret replied on Mon, 2014/04/14 - 3:37am

Wake up,Nicolas. The world in general and the IT world in particular is driven by budget, goals and scheduling.

An estimate is needed. Because it's the only way for the management to have a view on what's going on. No choice, no other option.

Now, give that strong statement, estimates have to be given. With a good level of precaution. I work for an IT services company, as a software architect. When a customer makes a request for proposal, the only way to make a good answer is to do a good estimate: too low, and you're over your budget and don't make any profit, too high and your competitors make better offer.

I have some templates, shared with my peers architect, I have a good technical knowledge of my customer's habits, and I have some functional experts around me.

With that, I assure you that without (most of the time) having to cancel anybody's vacations, project is delivered on time and on budget (let's say +/- 10%).

Few things I did for having accurate estimates:

  • Peer review for estimates: presenting the estimates to other architects (can provide you with some cheaper solution, and/or challenge your estimates)
  • Having the estimates validated by the development team: when a task is described to the dev team, ask the developer what workload it would require him. If it's lower that your estimates, the developer commits to faster delivery. If it's higher, then the discussion starts. If the developer's arguments are good, then his estimates are taken into account (and your ego attacked ... but architect shouldn't be too ego-sensitive, right ;-) )


Luca Cavagnoli replied on Tue, 2014/04/15 - 3:56am

Estimates are not applicable to creative and research activities, and software development is one of them. Ask a research lab for an estimate about time and budget for finding the cure for cancer; or, ask a drilling site's manager for an estimate on how long it will take them to find oil. It makes no sense. An estimate is just a guess dressed with a shirt.

Nicolas Frankel replied on Tue, 2014/04/15 - 4:48am in response to: Fabien Bergeret

 Wake up Fabien, it doesn't work...

Fabien Bergeret replied on Tue, 2014/04/15 - 8:48am in response to: Nicolas Frankel

That, of course, could explain why so many IT projects are killed before being released.

But how to you explain that IT services company that sell fixed price projects manage to make profits?

Nicolas Frankel replied on Tue, 2014/04/15 - 9:07am in response to: Fabien Bergeret

Easy enough: what they sell is "in the specifications". During all my career, I've never seen any unmoving specification during the course of the project (or even worse, full of holes). So the gap is oversold - and that's an understatement.

But I assume it's a trick question, isn't it?

Fabien Bergeret replied on Tue, 2014/04/15 - 10:06am in response to: Nicolas Frankel

So true for the unmoving specs :-)

But do you think that customers aren't smart enough to realize that specs change is oversold? Will the customer trust the same company? Is that the proper way to establish a trusting long-term relationship (and I strongly believe that without trust between the customer and the vendor, the project is doomed)?

The only way, at my level*, to have trust is to be transparent and coherent on the estimate, the budget and the schedule.


*meaning: for an architect. Sales guys have other possibilities ;-)

Javier Gonel replied on Fri, 2014/04/18 - 9:29am in response to: Fabien Bergeret

Estimates are not final dates, even if they have the same unit: time. You still cannot make operation with them as a source of absolute truth. An schedule in the other hand, is a date: 25th of April is 25th of April.

The company might need a feature before the season starts or <insert here any other reason for a specific date>, that won't change. It is your schedule. But estimates do not drive the schedule here.

Of course you want something finished for that date, and here is where prioritising comes into play. Depending on the value that each feature gives to the company. From that value you give priorities. Priorities go really deep into technical issues and small issues and mixed with deadlines, you do as much scope there as you can.

This requires "continuous" observation of what is going on, to be able to detect when new issues arise and be able to say "no, this is not a priority right now". It is a lot of work, it is much easier to believe that estimates are final dates and go with it.

Anyway:

  • Peer review: they must be blind estimates because each one of us is biased. "If the technical leaders says X... I'm junior I should say X too". That's the reasoning behind planning poker. 
  • Having the estimates validated from the people doing the job... this is how it should be in the first place. As far as knowledge of the work, they know more than the ones not doing the job. This applies everywhere and it's the base of the empowerment in lean manufacturing.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.