Mr. Lott has been involved in over 70 software development projects in a career that spans 30 years. He has worked in the capacity of internet strategist, software architect, project leader, DBA, programmer. Since 1993 he has been focused on data warehousing and the associated e-business architectures that make the right data available to the right people to support their business decision-making. Steven is a DZone MVB and is not an employee of DZone and has posted 144 posts at DZone. You can read more from them at their website. View Full User Profile

The Problem With Software Development is...

  • submit to reddit

Folks like to say that there's a "Software Crisis". We can't build software quickly enough, cheaply enough or well enough. I agree with EWD -- software is really very, very complex. See EWD 316 for more justification of this position. Is my conclusion is that the cost of software stems from complexity? Hardly news.

No, my conclusion is that the high cost of software comes from doing the wrong things to manage the high cost of software.

The Illusion of Control

Nothing gives a better illusion of control than a project plan. I think that software development project management tools -- MS Project specifically -- is the biggest mistake we can make.As evidence, I look at Agile methods. One key element of Agile methods is to reduce (or eliminate) the project management nonsense that accumulates around software development.

I think that software development projects are generally pretty complex and a big MPP file doesn't reduce the complexity or help anyone's understanding. I think that we should not make an effort to capture the complexity -- that's simply silly. If you find that you need a really complex document to capture a lot of really complex complexity, you're doing something wrong.

Hands in the Pocket Explanations

I think that user stories are great because they reduce the complexity down to something that we can articulate and remember. This gives us a fighting chance at understanding.

If the use case requires a big, complicated document, we're missing something essential. It should have a pithy, easy-to-remember, easy-to-write-on-a-sticky-note summary. It can have a detailed technical appendix. But it has to have a pithy, easy-to-articulate summary.

If you can't explain the use case with your hands in your pockets, it's too complex.


An architecture diagram is helpful. Architecture -- as a foundation -- has to be subject to considerable analysis to be sure it's right. You need to be absolutely confident that it works. And like any piece of mathematical analysis, you need diagrams and formulas, and you need to show your work.

A miraculous pronunciation that some architecture will work is a terrible thing. A few pithy formula (that we can remember) and some justification are a whole lot better.

The WBS Is The Problem

I find that projects with complicated WBS's have added a layer of complexity and management that aren't helpful. The cost of software is high, so lets add management to try and reduce our costs. On the surface, adding labor to reduce labor doesn't make sense.

Rather than waste time adding work, it would be better to introduce someone who can facilitate decision-making (i.e., a Scrum Master) and keep progress on track.

Incremental releases of partial solutions have more value than weekly status reports.Meetings with product owners have more value than a carefully-written schedule for doing the poorly-understood process of detailed design.


We can justify project management by saying that it somehow makes the software development process more efficient by eliminating "roadblocks" or "inefficiencies". I used to believe. I no longer buy this.

Let's look at some candidate roadblocks that a project management might smooth out.

  • User Involvement. Or rather, the lack of user involvement. I don't see how a PM does anything except nag the users. If the users aren't motivated to help with software development by answering questions or reviewing the results of a sprint, then the software isn't creating any value. Stop work now and find something the users really want.
  • Technical Resources. Coordinating technical resources (DBA's, sysadmins, independent testers, etc.) doesn't require a complex plan, status meetings or reports. It only requires some phone calls among the relevant folks. Directly.
  • Decision-Making. The PM isn't the product owner, nor are they a user, nor are they technically proficient enough to understand what's really at stake. Essentially, they only act as a facilitator in a conversation that don't fully understand. That's fine, as long as they stick to facilitating and don't take on responsibilities that aren't assigned to them.

At this point, I can find a use for a facilitator ("Scrum Master"). But I can't see why we have just an emphasis on IT project management. The Agile folks seem to have it right. Reduce cost and complexity by actually reducing the cost and complexity. Not by adding management.


Published at DZone with permission of Steven Lott, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Cosmin Mutu replied on Wed, 2009/12/16 - 3:03am

nice article

Moldskred replied on Wed, 2009/12/16 - 4:48am

  As for your view on complexity, I think the Jargon file put it best:

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.
"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-tac-toe" Sussman replied.
"Why is the net wired randomly?", asked Minsky.
"I do not want it to have any preconceptions of how to play", Sussman said.
Minsky then shut his eyes.
"Why do you close your eyes?", Sussman asked his teacher.
"So that the room will be empty."
At that moment, Sussman was enlightened.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.