Viewpoint Toward a Computer for Visual Thinkers (1988), Hacker News

When I started my PhD project in 1981, the IBM PC was brand new, and it would be years before the Mac and Microsoft Windows would appear. Nonetheless I was familiar with graphical user interfaces because of my internship at nearby Xerox PARC, the visionary research center that spawned, among other things, the laser printer and the bitmapped display. I was well-acquainted with Alan Kay’s vision of a dynabook (he envisioned the notebook computer in 1975), and understood the power of graphic user interfaces and graphic tools like paint programs to bring the power of computers to artists and visual thinkers.
At the time I was part of the digital typography program within the Stanford computer science department, which built computer programs for digital typeface designers. I was also enthralled by the Visual Thinking course at Stanford, which taught engineers in the product design program how to think visually.
It struck me as odd, and deeply wrong, that we were building tools for visual artists in a programming language that was utterly symbolic, and lacking in visual sophistication. I yearned for a programming language that had the same visual clarity that graphic user interfaces had.
So I set about wondering what a visual programming language might look like. If computers had been invented by artists and visually oriented people, instead of by mathematicians and engineers, how might they write programs? It seemed to me an important question, but one that hardly bothered most computer scientists. I read about a few attempts to build visual programming languages, and decided there was something fundamental I needed to understand.
My journey took me deep into the foundations of computer science, where I asked fundamental questions like “what is programming” and “what is a user interaction” – questions that often get passed over in computer science (any definition of “programmiing” that starts with “a sequence of symbols that…” is not deep enough to encompass visual programming languages). I never did build a visual programming language, but I did figure out a fundamental idea (things get interesting when the user and the computer share the same mental model of data), and built a rudimentary visual editor that demonstrated my ideas. I’m posting my dissertation and the accompanying video demo to re-open the conversation. What in my dissertation is interesting to you? What other work do you know of along these lines? Do you know of foundational research in interaction design and visual programming? And most importantly, what would be a good next step to push the work forward? (My research stalled for lack of a juicy application domain.) Pleaseemailme your thoughts. I’d love to hear them.