A couple of weeks ago I, Trygve Reenskaug (inventor of the MVC pattern) and Jim Coplien did a track at the Oredev conference about DCI (Data, Context, Interaction), which is a new way of thinking about object-oriented software development. Trygve's presentation did a good job at analyzing the progression from the procedural programming, where code could be read and understood, to todays object-oriented paradigm in which descriptions of interactions and algorithms are spread out in a multitude of so-called "objects", thereby making it impossible to get an understanding of how the overall system works, and what the interactions between objects are. DCI is an attempt to reconnect with the procedural programming success, where it was possible to read a procedure and figure out what the software does. By bringing this concept, that of systems and a sense holistic views, back into software development we might be able to avert the quality crisis that our industry is currently in.
After having done this presentation, I also realized that software development is not the only field of human activity which suffers from this general issue. Apart from studying software development I am also looking into management theory, healthcare, and psychopathological influences on societal levels (aka political ponerology). What I can see now, with software development as an example of the general pattern, is that all these fields, in their popular western form, suffer from the same issue: the inability to look at the problem from a systems point of view, and instead focusing on the individual parts, thereby introducing suboptimizations and harming flow that decrease quality and increases cost.
In management theory we have the notion of "economies of scale", where service is seen as a form of production where workers are given standards and highly detailed processes to be adhered to (and which were not created by the workers themselves). The idea is that by having these standards, and targets to measure success with, we can reach new levels of productivity and efficiency, which are impossible to reach otherwise. These notions are sometimes implicit in the management of a company, and sometimes they are explicitly expounded through implementation of things like ISO 9000. This idea has now been shown to be false, even fatally so, and is slowly being replaced with a more holistic view, that of systems thinking, which talks of "economies of flow" instead, and where knowledge and understanding replaces guessing and predicting. Here is one example outlining this.
In the healthcare sector we are currently seeing a failure on a monumental scale, as the western methods of understanding, diagnosing and treating disease based on the "fix the parts"-idea are not working. We are spending monumental amounts of money on healthcare and medicine, and which don't solve the problem, introduce new problems through sideeffects, and which ultimately only works to create an even sicker population, which benefit the "big pharma" that is supposed to help us deal with it in the first place. The healthcare issues are then further compounded by the use of flawed management theory, as outlined above, which makes it even more inefficient and useless. Those who work in hospitals are often trying to desperately workaround the system to help patients, but ultimately they are only the 5% of the puzzle, with 95% of the outcome being ruled by the system itself, flawed as it is.
In contrast, holistic medicinal practices such as TCM (Traditional Chinese Medicine) provides a way of looking at the human body as a system, understanding the parts as being related to each other, and has uninvasive methods of treatment, largely without side-effects, and at a cost that is only a fraction of what the western "modern" method would be. TCM is slowly being adapted by western institutions, but it has to be done cautiosly as it is still officially considered "alternative" and unproven. Whether TCM works or not isn't really the issue. The issue is that its very underlying idea, that of the human body being a system whose parts interact symbiotically, threatens the ideology that western medicine is based on, and as any weakly founded religion would do, such threats must be dealt with without remorse. Questioning dogma is simply not an option.
Finally, when it comes to understanding pathology of the mind, and the effect that it has on our society, the western research so far has focused on the diagnosis and treatment of individuals, without looking at the bigger picture. Thus the popular idea of psychopaths, and other pathological disorders, is still in most peoples mind limited to the extreme examples provided by Hollywood and flashing headlines, where extreme psychopaths go on killing sprees and whatnot. This, however, we now know is not an accurate reflection of the impact that people with pathological mental disorders have on society. Through the study of political ponerology one can get a glimpse of the fact that not only are these mental disorders more common than one would think, they also interact in such a way as to boost each other. A psychopath with a devious plan might manipulate a paranoid schizophrenic to further it, which in turn causes people with narcissistic wounding to become devotees and followers, thereby creating a chain of delusion which ultimately infects entire social groups, societies, and eventually nations. We have many examples of this in our contemporary history, and there is no reason to believe that this will not happen again, as the only vaccine is knowledge, and knowledge is sparse. By studying pathology and ponerology it also becomes possible to understand the impact that it has had on the previous two systems, that of management and healthcare, and as noted, healthcare in itself is afflicted by the delusions of current management thinking as well.
In short, not only do we have to start thinking in terms of systems with regard to individual aspects of life, such as healthcare, we must also consider how these systems interact, pathologically, on an even larger scale. Only then does it become possible to understand the monumental amount of waste and failure that we have created, and which is made possible by remaining in the dark as to the underlying causes.
Doom and gloom? Yes, indeed, the picture is pretty much pitch black as it is. But as John Seddon points out, there is no way to create change without first studying the system as it is, in its current incarnation. The good news is that the interconnectedness works both ways. If we collectively deal with the root cause of all these issues, which is the pathological influence of various mental disorders, then the other affected systems will benefit from it, and when you have systems interacting with systems in a positive way you get exponential improvement as a result: if management theory accepts and corrects itself based on an understanding of pathological thinking, it will help the healthcare system, which in turn will give them a way to get a more holistic way of looking at their work, which can improve the results of treatment, which will improve everyones lives, which... and so on, you get the idea. This should be the main concern of people wanting to make a real difference in todays world, as most of anything else will be to be cutting the leaves of a tree, rather than its roots. You are, as western medicine is focused on, treating a symptom rather than the true cause. Treating the true problem will clear out the symptoms that it caused, and in a way that is much more effective as you can be sure that the symptoms will not be coming back anytime soon.
Coming back to software development, which was the initial cause of this realization of mine, it is now clear to me that if we do not accept the conclusions that Trygve put forward, and which are reflected in so many other forms of human activity, there is no chance of creating software in a way that the code becomes readable, understandable and maintainable. We cannot continue to focus on the parts, the objects, without also taking into consideration the contexts and interactions that these objects have. We need a holistic way of thinking about systems.The question is: can we afford not to do this? Can we allow ourselves to continue using bad practices, which lead to more low-quality code and unmaintainable software?