In my previous blog, I provided a summarized view of what OSGi is and how it impacts Java developers. I got some interesting feedback. In this article, I would like to take a detour to examine modularity itself and its importance in creating robust systems that can scale in size and functionality. This post does not discuss any specific Java technologies or platform directly. However, it looks at how modular design has affected industries as diverse as manufacturing to software engineering.
In the Merriam-Webster dictionary, being modular is defined as "... composed of standardized, self-contained units for easy construction and flexible use ..." The notion of modularity is universal and has found usage in process (policy), manufacturing, business, finance, engineering, etc. Those who study the natural sciences have presented arguments about the modularity of nature.
Biological studies have shown that the evolution of complex organisms
are due in part to functional modularity (composition of smaller
independent components with clear bound & functions) and behavioral
modularity (having traits and attributes that can evolve
independently). To stay closer to the discernible realms, let's examine
how contemporary usage of modularity and has affected our modern lives.
Product manufacturers figured out the way to increase output, increase quality, and reduce cost pressures was in modularity. At the turn of last century, industrialist Henry Ford introduced the notion of modularity in his assembly line with standardized and interchangeable automobile parts. He was able to reduce his production cycles and costs to achieve mass production of his Model-T automobiles.
The adoption of modular architecture, not processor speed, has been argued to be the quintessential force behind rapid growth and adoption of computing technologies. Today's hardware, while ever decreasing in size, offers cheap information processing power that has permeated through all industries. Achieving such complex and powerful systems has been possible through the use of modularity. Manufacturers independently create standardized, interchangeable subsystems (microprocessor, memory chips, input/output devices, etc) that can function together to create complex devices. They have been able to build systems with high level of complexity by combining hardware modules (subsystems) into innumerable configurations.
In term of longevity, Unix is arguably one of the most successful operating system today. Created in 1969, Unix has been used in all conceivable computing environment from small embedded to research class supercomputers and everything in between. Unix's success is attributed to its flexibility and depth made possible by its simple, clean, and orthogonal design. As complex an operating system as Unix is, it is composed of simple, distinct, interacting modules (small programs) with clean interfaces that do one thing and does it well (and nothing more).
In the world of Java, there are numerous examples of successful modular designs. One of the earliest and most popular was the JBoss application server. The JBoss designers realized early on the power of modularity. They were one of the first to build a modular architecture on the top of the JMX (though JMX was designed for management, not a general purpose lightweight component server, its standardized API provided the tools needed to build a modular systems). Other examples of well-known, successful modular systems include the Eclipse and the NetBeans desktop platforms. Both frameworks let developers build complex desktop applications (including IDE's) using small pluggable modules that interact at runtime.
Modularity in Software
So, the question remains: how does one develop modular code? In disciplines such as engineering and manufacturing, for example, there is a natural affinity to design modularity along the physical attributes of materials. In software development, however, there are little or no affordances to provide guidance to shape code into standardized modules. As such, developers tend to take the path of least resistance and write software systems organically (i.e. monolithic) where the code produced has little or no clear boundaries. The next few paragraphs review provide some points on creating modular software. If you already create your software as modules great, otherwise, you can use the following text as guidelines for your next projects.
Generally in software, modularity can be categorized as logical or physical. Logical Modularity is associated with the internal organization of code into logically-related units. In modern high level languages, logical modularity usually starts with the class, the smallest code group that can be defined. In languages such as Java and C#, classes can be further aggregated into packages (namespace in C#) which allows developers to organize code into group of related classes (languages such as Ruby, Scala, Smalltak, Java7 support similar organizational structure with additional semantics for managed components) .
In the Art of Unix Programming, author Eric S. Raymond provides an iconic rule for modularity that states "... write simple parts connected by clean interfaces." Depending on the context, a module can be implemented as a single class, several classes in a package, or an entire API (a collection of packages). Regardless of the implementation scale of your module, you should be able to describe its functionality in a single sentence (i.e. this module calculates tax per zip code). Your module should expose its functionality as simple interfaces that shield callers from all implementation details. The functionality of a module should be accessible through a published interface that allows the module to expose its functionalities to the outside world while hiding its implementation details (it is way beyond the scope of this blog entry to provide a comprehensive treatments of design patterns and rules for modularity in software).
Physical Modularity, is probably the earliest form of modularity introduced in software creation. Physical modularity is comprised of two main components 1) a file that contains compiled code and other resources 2) an executing environment that understand how to execute the file. Developers build and assemble their modules into compiled assets that can be distributed as single or multiple files. In Java for instance, the jar file is the unit of physical modularity for code distribution (.Net has the assembly). The file and its associated meta data is designed to be loaded and executed by the run time environment that understands how run the compiled code.
Physical modularity can also be affected by the context and scale of abstraction. Within Java, for instance, the developer community has created and adopted several physical modularity strategies to address different aspects of enterprise development 1) WAR for web components 2) EJB for distributed enterprise components 3) EAR for enterprise application components 4) vendor specific modules such as JBoss Service Archive (SAR). These are usually a variation of the JAR file format with special meta data to target the intended runtime environment. The current trend of adoption seems to be pointing to OSGi as a generic physical module format (see previous post). OSGi provides the Java environment with additional functionalties that should allow developers to model their modules to scale from small emddeable to complex enterprise components (a lofty goal in deed).
Why Be Modular?
We have seen numerous disciplines have benefited by adopting modularity in process and development efforts. So, why should you invest time in implementing a modular architecture for your next project? After all, adopting a modular design introduces new overhead such as change in design practices, module files management as project grows, design- and build-time dependency management, and each group working on a module has a narrow knowledge of the code. The following is a list of benefits that you gain that outweigh the headaches you may encounter in modular design.
- Scalable Development: a modular design allows a project to be naturally subdivided along the lines of its modules. A developer (or groups of developers) can be assigned a module to implement independently which can produce an asynchronous project flow.
- Testable Code Unit: when your code is partition into functionally-related chunks, it facilitates the testing of each module independently. With the proper testing framework, developers can exercise each module (and its constituencies) without having to bring up the entire project.
- Build Robust System: in the monolithic software design, as your system grows in complexity so does its propensity to be brittle (changes in one section causes failure in another). Modularity lets you build complex system composed of smaller parts that can be independently managed and maintained. Fixes in one portion of the code does not necessarily affect the entire system.
- Easier Modification & Maintenance: post-production system maintenance is another crucial benefit of modular design. Developers have the ability to fix and make non-infrastructural changes to module without affecting other modules. The updated module can independently go through the build and release cycle without the need to re-build and redeploy the entire system.
- Functionally Scalable: depending on the level of sophistication of your modular design, it's possible to introduce new functionalities with little or no change to existing modules. This allows your software system to scale in functionality without becoming brittle and a burden on developers.
In this post, I wanted to provide a broad overview of modularity as a concept and its effect on software system. We have seen how modularity is intrinsically entrenched in non-computing disciplines and how they all benefit from the practice. Software is no different, complex systems from desktop applications to enterprise behemoth such as application servers benefit from modularity in design. It is not the intent of this entry to insinuate that modular design means an end to buggy code. On the contrary, developers will always write buggy code as long there's code to be written. However, it is the intent of this entry to show that modularity is a step toward making software development, deployment, and management easier to scale and maintain.
1. Merriam-Webster, http://www.merriam-webster.com/dictionary/Modular
2. Mark Dodgson, David Gann and Ammon Salter, Think, Play, Do: Innovation, Technology, and Organization
4. Carliss Young Baldwin and Kim B. Clark, Design Rules: the Power of Modularity
4. Werner Callebaut, Diego Rasskin-Gutman, Herbert A. Simon, Modularity: understanding the development and evolution of natural complex systems
5. Eric S. Raymond, The Art of Unix Programming, 1999
6. Mark Fleury, Juha Lindfors, Enabling Component Architectures with JMX, http://www.onjava.com/pub/a/onjava/2001/02/01/jmx.html, 2001
7. P. Jalote, An Integrated Approach to Software Engineering, 1997
8. Tim Boudreau, Jaroslav Tulach, Geertjan Wielenga Rich Client Programming: Plugin into the NetBeans Platform