OutSystemsDev Zone

What Is Visual Programming?

Just what is visual programming? It’s interesting that not many leading programming sources have tried to answer that question. In fact, a colleague of mine complained that when she actually asked Google, “what is visual programming,” she didn’t get a straight answer. To me, that seemed odd, so here is my attempt to right that wrong. 

Ideas are visual. When things get complex, we reach for the whiteboard pen. Yet since the arrival of FORTRAN in 1954, we’ve sought ever more abstract ways of instructing computers using text.

While the mainstream of computer programming led us to today’s world of JavaScript, C# and Haskell, a parallel effort attempted to play to our visual strengths. Here we’ll explore the phenomenon known as visual programming, describe how it developed, ask why it hasn’t taken over the world, and look at the state of visual programming today.

The tl;dr Answer

So, what is visual programming?

Visual programming lets humans describe processes using illustration. Whereas a typical text-based programming language makes the programmer think like a computer, a visual programming language lets the programmer describe the process in terms that make sense to humans.

Just how big the gap is between visual programming and traditional programming depends on the visual programming tool. At one extreme, the tool shields the programmer almost entirely from the gap between human thinking and computers shuffling bits around memory.

Here’s an example. To create a to-do list in a visual programming tool, the programmer draws out the flow of the app. The resulting flow-chart describes screens, user interactions and what happens to the data at each stage. The tool then turns that into software.

As developers, we know that text-based programming languages focus entirely on the implementation: it’s all about the precise steps the computer must take in order to create the experience we want to give the user. Sure, higher-level languages and modern frameworks give us convenient shortcuts. But, the developer’s job is to translate human needs into processes that fit the computer’s limited abilities.

Other visual programming tools follow the same processes and paradigms as text-based programming. Imagine drawing a class and its relationship with the objects you instantiate, rather than typing it all out into a text editor.

All of this sounds great! But, you might ask, where are all the visual programmers? Why are we still writing code by hand?

To understand why, let’s start with the history of visual programming.

The Birth of the Flowchart

In the western world, the early 20th century was a time of Utopian efforts to drive humanity forward. As the subsequent wars would suggest, it’s fair to say that not everyone agreed on what form that progress should take.

There was a common theme, though. If only we could apply the scientific method to all our problems, we could better understand and solve them. That led to the Efficiency Movement, Huxley’s Brave New World and the Model T Ford. It also sparked interest in better documenting industrial processes. As a result, we got time and motion studies and, in 1921, the flow process chart.

What would become the modern flowchart gave us a visual language for describing processes. For millennia, humans had been using diagrams to communicate complex ideas. But, this formalized language spread quickly to become standardized in industry.

In 1949, pioneering computer scientists Herman Goldstine and John von Neumann became the first to apply flowcharts to describing computer programs, while working on the ENIAC computer. Such a visual language was necessary when programming was a matter of correctly setting thousands of ten-way ring-counter switches.

As computing developed over the coming years, computer scientists first used assembly languages. Then, they moved to higher-level languages, such as FORTRAN and COBOL, that set the template for the languages we use today. In some senses, each new language was just more of the same. The role of the programmer was to break human-scale problems into algorithms capable of running on computer hardware. And let’s not forget, this was a time when teletype was the cutting edge of human computer interaction.

Graphical Experiments

what is visual programming sketchpad

Ivan Sutherland using Sketchpad in 1962: Source: Sketchpad of Ivan Sutherland

As graphical computer interfaces became a reality–albeit rare–in the 1960s, computer scientists began to explore their possibilities.

The Sketchy 1960s

One experiment in particular was Sketchpad, the brainchild of Bert Sutherland’s brother, Ivan Sutherland. Known eventually as the progenitor of CAD, it set precedents that would crop up again and again in graphical user interfaces.

In particular, Sketchpad showed that it was possible for a computer to model objects that had, admittedly very simple, physical characteristics and were able to interact with each other. This led Ivan Sutherland to create what was arguably the world’s first visual programming tool as part of his PhD thesis.

Sutherland’s work was ahead of its time. Computers of the 1960s were largely used for batch processing, not the real-time interaction needed by graphical interfaces. In fact, it would be several years before Bell Labs’ Michael Noll first proposed modern bitmap-style displays that could efficiently handle the interfaces necessary for visual programming.

Smith Plays Pygmalion to His Own Galatea

In 1975, David Canfield Smith published Pygmalion. This visual programming language allowed the programmer to make “descriptions in terms of the concrete, which PYGMALION turn[ed] into the abstract.” His thesis was that traditional programming made it too hard to bridge the difference between a human’s mental model of a problem and the computer’s state.

what is visual programming Pygmalion

A HyperCard simulation of Pygmalion done on a Macintosh computer by Allen Cypher and David Canfield Smith. Source: Pygmalion: An Executable Electronic Blackboard

A programmer working with Pygmalion would instead be able to see the program’s state, rather than having to juggle with a mental representation of it. Canfield described it as a “computational extension of the brain’s short term memory” that provided an “alternative storage for mental images during thought.”

As such, a Pygmalion program would be a series of snapshots of the computer’s state, drawn using icons, at each stage of execution. The first snapshot (or frame, in Pygmalion’s terminology) would describe the opening state of the program. Subsequent frames would then describe how the programmer wanted that state to change.

While details would change, it’s easy to see Pygmalion echoed in later visual programming tools and, indeed, in modern low-code platforms. Pygmalion itself, though, was little more than a research project. Therefore, computers would remain primarily text-based beasts for at least another five years.

The PC Arrives

what is visual programming prographBy the 1980s, computing was mainstream. Desks increasingly had either computer terminals or personal computers. In the case of personal computers, graphical displays replaced the text of CP/M and MS DOS.

These two changes opened the way for visual programming to have its moment of glory. Except, as we know now, that moment never really happened.

Prograph came along in the mid-1980s for Apple’s Macintosh computers, following a discussion at Acadia University. The university noted that diagrams describing the flow of a computer program were clearer to read than the code itself. Much like Pygmalion before it, Prograph was an attempt to make it easier to visualize standard programming techniques. As such, it was an object oriented language, with objects appearing as clickable hexagons on a graph. Each hexagon was divided in two. Clicking on one side would provide access to the object’s methods. The other, to data. You drew methods as directed graphs, showing data flow through a series of actions that worked on that data.

Prograph, and the company behind it, lasted into the early 1990s but not beyond. Today, Prograph fans say the company’s decision to focus on Windows and not their loyal Mac fans caused its demise. This was a time when the PC’s dominance seemed certain, Yet, there was still much innovation and competition between home and business computer systems.

Outside the Mainstream

(Image source: The Personal Computer Museum, Canada’s video game museum)

The late 1980s and early 1990s were a time when x86-based PCs were dominant but not to the exclusion of all else. Today’s laptops, desktops and servers are pretty much all built around a compatible architecture. The operating system might vary between Windows, macOS or some flavor of Linux, but the machines underneath are all essentially the descendants of that original IBM PC. Back then, there was still competition for the underlying vision of the design of the personal computer.

Apple Macs, Commodore Amigas, Acorn Archimedes and Atari STs each had vastly different architectures and operating systems. And, although they found success in particular niches, they were all used as business computers. One important factor was that they each had a graphical operating system more sophisticated than contemporary versions of Windows.

The Amiga and Mac were particular hotbeds of visual programming experimentation, most famously with CanDo on the Amiga and HyperCard on the Mac. In the case of CanDo, and less so with HyperCard, these were true general purpose visual programming tools. They could create a range of software applications, from business tools to multimedia works.

In narrower niches were others. Commodore’s own AmigaVision, for example, is best thought of as a precursor to PowerPoint, Then there was the cross-platform Shoot-’Em-Up Construction Kit for games development.

Visual Programming Didn’t Scale Well

As computer hardware grew in power, software could solve ever more complex problems. That increased complexity led to something of a role reversal between visual programming and text-based programming. A simple employee management system running on a desktop computer and managing the details of a couple of hundred people is easy to represent visually. With three or four main classes and synchronous data operations, the entire system makes for a simple, easy to parse flow diagram.

The 1990s, though, saw companies network their PCs with Novell Netware and, later, directly with Windows. Network-aware applications and the growing expectations invested in software led to applications that became too complex to represent visually. With lines representing data flows, visual programming tools made the phrase “spaghetti code” rather more literal.

Perhaps, then, the visual programming paradigm was not suited to general problem-solving in a world of software that kept growing in complexity. However, where the task was of a known type, visual programming still had a role to play.

And so, in the 1990s, visual programming split into three branches, each of which tackled a specific problem domain: business applications, multimedia systems and games and educational development.

Games and Education

Non-PC computer systems each had their particular niche. Macs were in many US schools and were the go-to choice for designers. In the UK, the Archimedes was had a similar role in schools. Atari STs were in almost every recording studio, thanks to their built-in MIDI ports. And the Amiga’s advanced graphics made it ideally suited to television production.

Gaming the System

what is visual programming doomEach of these machines, though, owed at least some of their sales to computer gaming and, throughout the 1980s, visual programming tools for creating games flourished. Early systems focused on one type of game: Pinball Construction Set, Adventure Construction Set and Shoot-’Em-Up Construction Kit provided games enthusiasts with a visual method of creating their own games in a specific genre.

This was visual programming not of data flows but of sprites and collision detection. While most visual programming uses flowcharts to some extent, these games tools used the very substance of the games themselves as their primitives. “Add a spaceship sprite here. Make it loop around the screen like this. Have it fire this sprite downwards every two seconds. It’s game-over if the player’s sprite collides with the spaceship.”

The focus on one type of game made the tools easy to work with but also limited their potential. Publishing these games was rare; the role of these visual game programming systems was more as a fun distraction for hobbyists than a tool for creating commercial games.

Some games, such as Doom, even integrated a visual tool for extending the game. These level editors were an unwitting acknowledgement that games created with the visual programming tools of the day would always be limited by the imagination of the tool’s authors.

Hands-on Education

Some more general-purpose games creation tools—notably Klik and Play and GameMaker Studio—did come on the scene. But, beyond fun, their role was primarily educational. And that’s where visual programming has had one if its main successes.

Educational visual programming environments, such as MIT’s Scratch and Microsoft’s Kodu, have been taken up with vigor by national education systems.

Even Lego’s Mindstorms project offers a visual programming language, based on the general-purpose LabVIEW, for interacting with programmable Lego blocks.

Multimedia Visual Programming

For a brief period in the early to mid-1990s, the world entered the multimedia age. Just like other hype-based terminology, such as Web 2.0, multimedia could mean almost anything. Usually, it was some kind of software that featured full motion video. And, it seemed best suited to reference materials or adventure-style games.

Commodore with their CDTV, Phillips with the CD-I and, later, the 3DO company set out to create multimedia players that would sit in the living room. The goal was to be like any other piece of home entertainment equipment. In the PC world, multimedia software, such as Microsoft’s Encarta encyclopedia, was distributed on CD-ROM.

Each of these new formats led to a boom in content production. Much of this boom was more like the modern web than traditional software had been at that time. This was another great fit for visual programming. Content, rather than functionality, was the differentiator. And, so multimedia creators, rather than traditional programmers, were in charge.

Tools such as Authorware and Scala re-purposed flowchart-based visual programming techniques. Non-programmers were now able to create interactive tools, such as e-learning materials, that could incorporate text, video and audio.

what is visual programming authorware

Authorware in action. Image source: Softonic.

Diving into even more focused areas of interest, visual programming has had a long history in music. In the mid-1990s, Blue Ribbon SoundsWorks’ MIDI sequencing software, Bars and Pipes, used a unique visual way of representing and interacting with music tracks. More conventional sequencing tools, such as Steinberg’s Cubase, eventually won out. However, visual programming continues to play a strong role in synthesizer programming. Visual tools such as Reaktor enable people to create new sounds by threading together multiple types of sound processors.

Business Visual Programming

It all starts with databases. In modern software architectures, one or more databases sits somewhere beneath layers of middleware and application-level software. However, the distinction between database and application is not always so clear.

Before the world adopted SQL as the accepted standard for database query, each database system had its own method of querying data. SQL is different from most languages. It’s a declarative language. As a programmer, you tell SQL what data you want to find rather than telling it how to find that data. SQL competitors have most often been procedural languages. The developer tells the software what to do, just as with standard application programming.

Although there are procedural extensions to SQL—Oracle’s PL/SQL, for example—we tend to build modern software with C#, Java or some other multipurpose language. And, we leave the database management system to store and query the data.

what is visual programming dbase

The properties dialog (Inspector) for a menu item in dBase. (Image source: the dBase Knowledge Base.)

In the 1970s and 1980s, with limited computing resources, application software was often bundled together with the underlying database. C. Wayne Ratliff’s dBase used its own procedural database query language that grew to become a full application development language and platform. An entire industry of compatible database systems flourished, including names that would dominate custom business software such as FoxPro and Clipper.

dBase normalized the idea that database management systems were more than data stores. They could also be application development platforms. Lotus Notes, Microsoft Access and Apple FileMaker drove the combined database and application development platform into the GUI age. Although none of these was truly a visual programming tool, they laid the ground for one of the greatest misnomers in software history.

Visual, Sort of

Although Visual Basic predated Microsoft Access by about a year, it was the development of Access that led indirectly to Visual Basic. In the late 1980s, Microsoft set out to create their own relational database system. Named Project Omega, they eventually abandoned that plan in favor of what became Access. Yet, not before the development of a new macro language named Embedded Basic. Visual Basic was the combination of Omega’s Embedded Basic and another Microsoft project, a forms designer for Windows.

Now here’s the really interesting thing: Visual Basic is not a visual programming tool. None of Microsoft’s Visual languages are. Rather, they are rapid application development tools. Visual Basic made it easy to create user interfaces and attach code to actions in that interface. That laid the ground for Borland’s Delphi and the graphical IDEs that we have today. However, as the executable code was still written as text, it was not true visual programming.

Executable Flowcharts

That’s not to say that visual programming in the 1990s was confined to game creation kits, multimedia tools and databases. Rational Software had been building a non-GUI Ada IDE since the mid-1980s. In addition, they also turned their hand to defining software development process. Work on their Rational Unified Process and related efforts eventually led to the Unified Modelling Language.

UML had the potential to document every last part of a system, without ever writing a line of code. It looked like visual programming but without producing executable software.

UML provided a standardized and comprehensive language for describing object oriented systems. However, UML-fever struck some architects. Co-author of The Pragmatic Programmer, Andy Hunt, tells the story of a software project where an architect spent two years creating UML diagrams before even a line of code was written.

Just as agile was gaining momentum, UML seemed to enable all the worst aspects of the old ways of building software: too much planning and too little implementation. Executable UML was an attempt to add that missing piece—the executable software. Several implementations surfaced but without making too much of an impact on a world that was rapidly switching its focus to PHP, Ruby on Rails and other dynamic scripting languages.

One form of executable UML that did stick around, though, came out of Rational Software itself. Acquired by IBM, the erstwhile Rational Software developed Rational Rose, a suite of tools for creating software using UML and generating executable code in a target language, such as C++ or Java.

what is visual programming rational rose

Case view in Rational Rose. Image source: Assignment Help

Can’t Catch a Break

Visual programming enthusiasts will tell you that visual programming is far from dead. Ask them “what is visual programming?” and first, they’ll name an obscure domain-specific tool. Then, they’ll tell you that tool is proof that it’s alive and kicking. Meanwhile, you’ll be frantically using Google. As a result, you’ll learn not just about the tool they’ve mentioned, but also the highly specialized world in which it exists.

Without doubt, visual programming has its role, whether that’s programming synthesizers or giving UML enthusiasts a sense of accomplishment. For general purpose software, though, the world is just too complex to model purely visually. When your “code” looks like a CPU circuit diagram, it’s perhaps time to rethink the suitability of visual programming to the task.

Similarly, visual programming tools tend to be limited by the imagination of the creator in a way that doesn’t hamper general purpose textual programming languages.

And yet tools like Visual Basic, Delphi and their descendants have shown us that building software visually can be enormously efficient; it’s just that there’s a pragmatic trade-off where, sometimes, hand-written code is the right solution.

Superstars vs. Teams: Developers, They Are a Changin’

Those early days of programming were hard, that’s for sure. But one person could understand and be expert in everything that was necessary to create that software. If you’re old enough, think back to software titles of the 1980s. It was common for a single programmer to become a brand in their own right.

Sid Meier, Mitch Kapor and Jeff Minter gained some level of fame by creating well-known applications or games single handedly or, at most, with one other collaborator. Back then, software and hardware upgrade cycles took place over years. Today, we joke that there’s a new JavaScript library every day. Yet, there’s some truth in the idea that modern software development moves at a pace that many of us can’t keep up with.

Today, software is largely built by teams of specialists. Whereas early developers did everything themselves, a modern software development team might have one person whose only job is to look after the CI tool. Developers spend entire careers focused on one framework or platform. iOS developers are iOS developers, not mobile developers. Once or twice a decade, perhaps, a web developer might switch their preferred framework. Very few people are manually writing assembly language professionally.

It’s not just that the scope of software has changed. To some extent, developers themselves have changed, too. Software engineer is just another career, these days. In decades gone by it was a passion held by a few people who had the dedication to learn an entirely new system so they could write the Atari ST port, for example, of their successful Amiga game. But that’s understandable: computing is no longer niche.

Okay, So What Is Visual Programming Today?

So, the ideas of visual programming are in use today and the problems that visual programming wanted to solve haven’t gone away. In fact, they’re more relevant than ever.

We have a world where software development is made up of increasingly complex parts and where developers are ordinary people with extraordinary specializations. That complexity and specialization are badly suited to the pure visual programming of those early tools but it also makes it increasingly hard to build rounded software engineering teams.

Where pure visual programming environments have failed, there is a whole tranche of similar tools that take the best of visual programming and combine it with text-based coding. Whereas visual programming was “no-code,” these new tools are low-code.

Tools such as OutSystems let developers create software visually by drawing interaction flows, UIs and the relationships between objects, but supplementing it with hand-written code where that’s the better thing to do.

This pragmatic mix of visual programming and text-based programming is well suited to the needs of modern software development. Low-code tools reduce the complexity of software development and return us to a world where a single developer can create rich and complex systems, without having to learn all the underlying technologies.

Delivering on the Promise

Visual programming held so much promise. But real-world problems demand greater flexibility than visual programming could offer. Low-code takes that promise and applies it to reduce the complexity we find in modern software development. So, ask not “what is visual programming?” Instead, ask “what is low-code?” Or, view this two-minute video to see how low-code works. You can also schedule an online demo or even try OutSystems (it’s free).

 

About the author

Matthew Revell

When he's not pondering the answers to low-code questions and writing for OutSystems, Matthew Revell heads up Hoopy, the developer relations consultancy he founded. He also runs DevRelCon, the developer relations conference. the DevRel blog and DevRel Jobs.

Comments

Thanks for this article Matthew, Visual programming has been an unfulfilled prophecy for years. As so many other areas, like virtual reality, artificial intelligence, or speech recognition, when the hype was high, the underlying technology wasn’t there yet.

Leave Your Comment