August 3, 2019
In March 2008, Paul Graham of Y Combinator wrote an essay called “How to Disagree.” In the article he defines a hierarchy to gauge the quality of one’s disagreement with another person’s argument, a measure which ranges from name-calling at worst, to refutation of the central point at best.
I think a similar hierarchy can be defined to gauge the quality of architectural thinking and discourse of software engineers. I have noticed that most entry-level programmers focus their attention and criticisms on first-order issues such as code style, naming conventions, source code and repository layout, etc. They neglect or are unaware of higher-order concerns that can make a bigger impact on reducing the complexity of their systems. Even experienced programmers seem to suffer from this myopia.
If we’re to build the systems of the future, I think we must set our sights above first-order issues. To afford the time to do so, we may need to reconsider, or even abandon, some concerns and practices that currently constitute much of what we call software engineering. Here I propose a hierarchy of concerns, ordered by the degree to which they reduce system complexity from lowest to highest.
1. Fixations
I use the word “fixations” to denote those concerns that programmers often argue about which generate more heat than light. Some examples include code style, line length limits, repository layout, unit test coverage, avoidance of GOTO, and other “best practices,” but this list is merely suggestive. Fixations are self-imposed constraints that prematurely restrict the set of possible implementations (set A) to a subset of implementations that are comfortable to the engineers who impose them (set B). Rarely, if ever, is it demonstrated that the set of optimal solutions (set C) is a subset of B.
Fixations are often adopted to solve problems that the team does not have, and may never have. There are cases where a GOTO is the perfect choice – even Dijkstra admitted as much – but many teams will flatly reject code that includes even one GOTO because “GOTO can make the code hard to read.” Yes, we’ve all read Dijkstra’s paper and understand the problems of excess GOTOs, but the question is not whether a tool used in excess causes problems, but rather whether the use of the toolin this particular casehas in fact created a problem.
The worst thing about fixations is that they generate work that does nothing to reduce the complexity of the business problem. They become a continuing source of puzzles for engineers to solve. The complexity compounds when the satisfaction of one self-imposed constraint conflicts with that of another. For example, line-length limits often conflict with indentation rules, so programmers spend time searching for the least-ugly compromise. I have seen a mob programming group of 6 engineers spend 45 minutes fixing these exact problems to satisfy their linter. This is waste.
There are reasonable arguments to support the myriad views surrounding these kinds of concerns, but that’s precisely the problem. If you can go either way on a particular issue, it can’t matter that much. It would be a better use of time to not worry about it at all by increasing our fortitude toward trivialities. One heuristic we might use to recognize fixations is this: Before you offer a criticism in your next code review, ask yourself, “would I still be as concerned about this if the entire program fit on a half-page?” If the answer is “no”, then use the energy you would expend on criticism to figure out how to make the program shorter.
2. Requirements
From the customer’s and user’s perspective, correct behavior within the defined runtime and space constraints is the single most important goal of any software team. It is perhapstheultimate measure of success. This might seem so obvious as to not need mentioning, but I intentionally included requirements above fixations to “reset” our focus on what matters. Rigid adherence to “best practices” and other fixations easily distract us from the requirements by substituting an easier problem, bike-shedding and irrelevant puzzle-solving, for a harder one: meeting the requirements.
3. Models
The satisfaction of requirements may be the sole test of success, but it’s not easy to get there. Teams write a lot of code in pursuit of this goal. If there’s one thing that programmers can agree on, it’s that more code generally requires more time – time for writing and time for maintaining. A better model of the problem domain can greatly reduce the amount of code you write, and therefore the amount of time needed to implement and maintain a solution, but teams seldom give themselves the time needed to explore different models, perhaps because they’re busy with level 1 concerns. *****
Data structures and algorithms are the most concrete examples of good models, but there are other kinds of structural changes that make a tremendous difference in how your solution is expressed. Combinators, streams, knowledge bases, finite-state machines, virtual machines, etc. can be used to organize your solution along completely different axes from the first solution that pops into your head, and yield a big decrease in complexity.
4. Notations
A change in notation takes models a step further by integrating them deeply into the language itself. What would otherwise be implemented as a libarary in some language, subject to the model of control implicit in that language, is freed from those limitations of expressiveness. You can say exactly what you mean to say and no more. The most striking example of the power of notation that I’ve seen is in Alan Kay’s STEPS project, which used a pipeline of translators to produce a personal computing system with graphics and networking in nearly 20, 000 lines of code. Compare this to the several million lines of code used to implement Linux or Windows.
Language construction is treated as an esoteric skill best left to Answers or wizards, but I think this is a misconception largely due to a failure of computer science education. Though Aho and Ullman’s “dragon book” dedicates a few chapters to simply recognizing and parsing a token stream – topics which are interesting in and of themselves – tools exist that make these skills unnecessary to master if one simply wishes to experiment with new semantics . LISP in particular is a fantastic vehicle for doing exactly that.
It’s my view that language design needs to become a part of the “canon” of mainstream software engineering practice. Physicists and mathematicians routinely invent better notations to express bigger ideas, and do not complain about having to learn new ones. If we wish software engineering to be worthy of its name, we should not shrink from our responsibility to leverage every tool at our disposal, including the definition of more suitable notations.
5. Environments
Environments do with languages what languages do with models, by integrating the problem domain, and the language for expressing solutions in that problem domain, into the user’s environment. Smalltalk systems and Lisp machines are classic examples of this kind of integration, but systems such as Mathematica and MATLAB serve as excellent modern examples.
If you consider the set of all possible implementations of a particular software solution, it’s astounding how much time and energy is devoted to approximating onlyonepoint in that space. Even worse, there’s no way to tell if it’s any good because there’s nothing to compare it to. It would be better if we could make it cheap enough to explore that space more fully.
What we want is a vehicle for exploring that space – something like an IDE for your problem domain. It’s been joked that any problem in computing can be solved with another level of indirection, but I wonder if it would actually be better for software teams to build not the thing they want, but ratherthe thing that makes the thingthey want. Teams could have a “solution factory” in the time that it would take to build a single solution using current methods, and be able to experiment with a family of implementations rather than just one.
Conclusion
Looking back, we see that the hierarchy defined above moves from first-order, everyday programming concerns, to higher-order architectural concerns. Even more interesting, as we move up the hierarchy, aspects of the problem domain that were explicit at the lowest level become more and moreimplicitas we move up the hierarchy.
GIPHY App Key not set. Please check settings