Mr. Mahal has worked his way through all aspects of software development. He started as a software engineer and learned how to design and architect software systems in different industries. He has been a director of engineering, product management, and professional services; mid-career he worked for a company dedicated to training engineers in collecting requirements and understanding OO and conducted research into why software projects fail; and before starting Accelerated Development he rescued a technology startup in Vancouver as their COO. Accelerated Development is dedicated to the production of high quality software through implementing light weight engineering practices for all size organizations up to Fortune 500. Dalip is a DZone MVB and is not an employee of DZone and has posted 37 posts at DZone. You can read more from them at their website. View Full User Profile

No Experience Required!

03.26.2013
| 2625 views |
  • submit to reddit

The original study that found huge variations in individual programming productivity was conducted in the late 1960s by Sackman, Erikson, and Grant (1968).  This study has been repeated at least 8 times over 30 years and the results have not changed! (see below) 
Sackman et al studied professional programmers with an average of 7 years’ experience and found that:

  • the ratio of initial coding time was about 20 to 1
  • the ratio of debugging times over 25 to 1
  • program execution speed about 10 to 1
  • program size 5 to 1

They found no relationship between a programmer's number of years of experience and code quality or productivity.  That is there was NO correlation between experience and productivity (i.e. ability to produce code) and there was NO correlation between experience and quality (i.e. minimizing defects) .

That is the worst programmers and the best programmers made distinct groups and each group had people of low and high experience levels.  Whether training helps developers or not is not indicated by these findings, only that years of experience do not matter.  If training does makes a difference then there are very few developers getting training out there.

Results Have Been Confirmed for 30 Years!

There were flaws in the study, however, even after accounting for the flaws, their data still shows more than an order of magnitude difference between the best programmers and the worst, and that difference was not related to experience.  In years since the original study, the general finding that "There are order-of-magnitude differences among programmers" has been confirmed by many other studies of professional programmers (full references at the end of the article):

  • Curtis 1981
  • Mills 1983
  • DeMarco and Lister 1985
  • Curtis et al. 1986
  • Card 1987
  • Boehm and Papaccio 1988
  • Valett and McGarry 1989
  • Boehm et al 2000

Technology is More Sophisticated, Developers are not

You might  think that we know much more about software development today than we knew in 1968, after all today:

  • we have better computer languages
  • we have more sophisticated technology
  • we have better research on effective software patterns
  • we have formal software degrees available in university

It turns out that all these things are true, but we still have order of magnitude differences among programmers and the difference is not related to years of experience.  That means that there is some other x-factor that drives productive developers;  that x-factor is probably the ability to plan and make good decisions.

Developers face making decisions on how to structure their code every day.  There is always a choice when it comes to:

  • laying out code pathways
  • packaging functions into classes
  • packaging classes into packages/modules

Because developers face coding decisions, many of which are complex, the best developers will plan their work and make good decisions.  Bad developers just 'jump in'; they assume that they can always rewrite code or make up for bad decisions later. Bad developers are not even aware that their decision processes are poor and that they can become much better by planning their work.

Solution might be PSP and TSP

Watts Humphrey tried to get developers to understand the value of estimating, planning development, and making decisions in the Personal Software Process (PSP) for individuals and the Team Software Process (TSP) for teams, but only a handful of organizations have embraced it.  Capers Jones has done analysis of over 18,000 projects and discovered that1:

All of these findings should have a profound effect on the way that we build our teams.  Rather than having large teams of mediocre developers, it makes much more sense to have smaller teams of highly productive developers that know how to plan and make good decisions.

Conclusion

We inherently know this, we just don't do it.  If the senior management of organizations only knew about these papers, we could make sure that the productive people get paid what they are worth and the non-productive people could seek employment in some other field.  This would not only reduce the cost of building software but also increase the quality of the software that is produced.

Unfortunately, we are doomed to religious battles where people debate methodologies, languages, and technologies in the foreseeable future.  The way that most organizations develop code makes voodoo look like a science!



Other articles Bibliography

Boehm, Barry W., and Philip N. Papaccio. 1988. "Understanding and Controlling Software Costs." IEEE Transactions on Software Engineering SE-14, no. 10 (October): 1462-77.

Boehm, Barry, et al, 2000. Software Cost Estimation with Cocomo II, Boston, Mass.: Addison Wesley, 2000.

Card, David N. 1987. "A Software Technology Evaluation Program." Information and Software Technology 29, no. 6 (July/August): 291-300.

Curtis, Bill. 1981. "Substantiating Programmer Variability." Proceedings of the IEEE 69, no. 7: 846.

Curtis, Bill, et al. 1986. "Software Psychology: The Need for an Interdisciplinary Program." Proceedings of the IEEE 74, no. 8: 1092-1106.

DeMarco, Tom, and Timothy Lister. 1985. "Programmer Performance and the Effects of the Workplace." Proceedings of the 8th International Conference on Software Engineering.

Washington, D.C.: IEEE Computer Society Press, 268-72.

1Jones, Capers. SCORING AND EVALUATING SOFTWARE METHODS, PRACTICES, AND RESULTS. 2008.

Mills, Harlan D. 1983. Software Productivity. Boston, Mass.: Little, Brown.

Valett, J., and F. E. McGarry. 1989. "A Summary of Software Measurement Experiences in the Software Engineering Laboratory." Journal of Systems and Software 9, no. 2 (February): 137-48.

Published at DZone with permission of Dalip Mahal, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Lund Wolfe replied on Sun, 2013/03/31 - 3:05am

I think that confirms what we already know.  It doesn't mean that experience and acquired knowledge/skill does not dramatically increase productivity and quality, but it may mean that the increase applies equally to the weak as well as the strong.

It doesn't compare weak experienced developers to strong inexperienced developers either.  We only know, or assume, that the strong inexperienced developers will increase in productivity and quality much faster over time.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.