In-Between Hardware and Software

Today in Tedium: In 2006, the software of the Nintendo Wii lagged a bit behind its hardware, causing issues for its initial release. How did Nintendo handle it? Well, it included a firmware disc with some early units, which installed the necessary software for the console to function. This disc, now recognized as one of the rarest pieces of Nintendo ephemera, wasn’t the first instance of installable firmware for a game console, but its existence, long-rumored and subsequently confirmed, highlighted the impact firmware had on our outlook towards computing devices. Firmware effectively became a game changer for video game consoles and computing in general. Suddenly, the device we purchased was not a permanent fixture. If it was problematic or incomplete, the manufacturer had the opportunity to rectify it post-purchase. Today’s Tedium delves into how firmware became the linchpin of computing. — Ernie @ Tedium

“Firmware does a lot of the work that computers had to do before. But firmware doesn’t cost like computers cost. And firmware is tiny.”

— A Burroughs ad, from 1969, which is one of the first newspaper ads explicitly referencing firmware. The fact that Burroughs, a significant company during this era, promoted firmware to the public speaks to its swift rise in computing, considering the term had only entered common usage two years prior.

Ascher Opler firmware

This article is the origin of all firmware. (Ted Nelson’s Junk Mail Cartons/Internet Archive)

Firmware’s potential pinpoint source didn’t see its eventual dominance over the computer

If we consider computing as a relationship between hardware and software, firmware neatly fits in the middle. It acts as an intermediary, bridging the gap between the information being processed and the managing electronics. It serves as a mediator—the marshmallow that lies between the chocolate and graham cracker.

And it existed long before the day when a few early Wii customers had to install a disc into their machine to begin using it. We have Ascher Opler to thank for introducing the world to this significant, albeit minuscule, concept. Opler, a computer scientist who collaborated with various luminaries of the era, detailed the concept in a January 1967 article for Datamation, in a piece titled “Fourth Generation Software.”

It was part of a series outlining the “fourth generation” of computing, as defined by the magazine—and it turned out to be the most crucial part, as it created an influential and well-known term. By accomplishing this, it can be said that they succeeded in their goal.

The moment Opler, an executive at Computer Usage Education, introduced the term seems like he possessed an awareness that the rest of us did not:

In fourth-generation computers, many microprograms will be available from the manufacturer. Software and user specialists will also prepare and use their own. This should throw the whole field wide open.

To better understand the nature of microprogramming a no-order-set/no-data-structure computer, I believe it worthwhile to introduce a new word into our vocabulary: firmware. I use this term to designate microprograms resident in the computer’s control memory, which specializes the logical design for a special purpose, e.g., the emulation of another computer. I project a tremendous expansion of firmware—obviously at the expense of hardware, but also at the expense of software.

The article later discusses microprogramming and the installation of small programs that computers frequently require for low-level tasks, as well as the potential cost-reduction benefits of firmware. For the forthcoming generation of computing that Opler was writing about, he argued that microprograms were well-positioned to leverage slow-write/fast-read (SW/FR) memory, as the data would be permanently accessible, but not easily modifiable.

In fourth-generation equipment using SW/FR micromemory, microprograms can be prepared by the firmware specialist-manufacturer, programming company or the user-to carry out the specific interrupt and input/output control function specified by the user. This alone will go far to simplify control programs.

Further simplification can be obtained by making the data structure and order set work for, not against, the implementors of control programs. The basic implementation involves techniques of queue management, control block handling, table reference, internal sorting, pointer handling, etc.

Since microprogramming permits extensive data structuring for control program implementors, it will permit the addition of instructions to enqueue, dequeue control blocks build search and sort tables of specified structure, etc. These new commands should prove a boon to expediting the running of supervisory programs.

In some ways, what Opler describes is an operating system, and in some ways, many firmware implementations can be seen as miniature or even full-fledged operating systems, managing specific needs that would otherwise fall on the back of individual software programs. (It’s worth noting in this context that many early graphical operating systems, such as MacOS, were implemented in ROM, effectively making them firmware.)

Opler’s insights on computing sadly outlived him—he died of a heart attack just two years after writing what turned out to be a strongly predictive article. Between the time he wrote it and his 1969 passing, he had taken a role at IBM’s research lab, working closely with one of the company’s most prominent luminaries of the era, Arthur Anderson.

Burroughs firmware

Imagine being this excited about firmware.

But his concept lived on. The Burroughs ad I mentioned above directly referenced the Fourth Generation concept that Opler had framed his discussion of firmware around. And soon enough, it would be more than just a selling point by a mainframe seller—it would be in every device under the sun, big and small.

What changed? In simple terms, the technology caught up with Opler’s vision expeditiously.

EEPROM

An example of non-volatile memory that can be electronically erased, but retains its data between restarts. When EEPROM was initially developed in the 1970s, it was typical to make physical modifications to programmable ROM chips to upgrade firmware, such as EPROM, which could be altered by exposing the chip’s window to ultraviolet light. EEPROM, essentially a less advanced form of flash memory, allowed firmware updates through software programs, without needing to open the machine. (NVRAM or PRAM are contemporary versions of EEPROM.)

IBM PC 5150 motherboard

An IBM PC 5150 motherboard. Can you name all the brand logos on this board?

The evolution of the BIOS into an indispensable element of computing

The most well-known term for firmware, and one that is still occasionally used today, is the Basic Input/Output System, also known as BIOS. It remains an excellent descriptor for our understanding of the fundamental functions of computers to this day, reflecting the system’s capabilities and its compatibility with software.

While it is often associated with the IBM PC, it actually predates it and was utilized by the CP/M operating system, one of the earliest PC-specific operating systems around. An industry legend, Gary Kildall of Digital Research, is credited with coining the term and conceptualizing it as a form of firmware that allowed software to have general compatibility with different devices—as long as they had the right firmware.

(By the way, Kildall’s life story deserves a mention here—he arguably is the man that should have been Bill Gates, but that he wasn’t was one of computing’s great failures.)

The IBM PC adopted the BIOS concept into its personal computers, which the machine’s technical reference guide explained as follows:

The ROM resident Basic I/O System (BIOS) provides the device level control of the major I/O devices in the System Unit. The BIOS routines allow the assembly language programmer to perform block (diskette and cassette) or character (video, communications, keyboard and printer) level I/O operations without any concern for device address and operating characteristics. Additionally, system services such as time of day and memory size determination are provided. The goal is to provide an operational interface to the system and relieve the programmer from concern over hardware device characteristics.

That final line is perhaps the most important, and it underlines the essential nature of firmware to this day. If you’re building software for a computer, a video game, even an Android-based smart toaster, for some reason, you need to have some sort of baseline for what to expect, one that doesn’t require you to know what every machine on the planet looks like. When there’s no guarantee that every machine will be exactly the same, you need a layer like the BIOS to help determine the rules at play for each component—rules implemented by the operating system and its component drivers.

On the IBM PC, the BIOS was initially intended to be proprietary, unlike most of its components. Nevertheless, companies like Phoenix Technologies, American Megatrends, and Award Software developed the firmware for the clone machines. Compaq was one of the first to circumvent IBM, convincing Microsoft to provide copies of MS-DOS and its BASIC program, and then constructing a BIOS through a complex series of approximations, which was only kind of like the first season of Halt and Catch Fire.

A 386 clone, booting with the help of an American Megatrends BIOS.

“We gave the black box inputs and saw what the results were,” said Gary Stimac, one of Compaq’s first employees, in 1984 comments to PC Magazine. “Typically, Microsoft gave us the specs, and we tested it—we didn’t look at the code.”

By the time Compaq was done, Stimac and similar figures knew more about the IBM PC than perhaps even IBM did.

Even though we don’t technically use BIOS anymore, transitioning to the standardized Unified Extensible Firmware Interface (UEFI) starting in the late 2000s, the original intention remains intact. While UEFI is more advanced, the purpose is similar: It creates a way for our computer’s innards to communicate with the software we use. BIOS was pretty long in the tooth by that point, as noted by Maximum PC in 2006:

As antiquated as it is essential, BIOS architecture is long overdue for a fundamental overhaul. We’ve gone through five major generations of Microsoft operating systems (DOS, Windows, Windows 3.1, Windows 98, Windows XP) and we’re poised to move to a sixth—Windows Vista—but BIOS architecture hasn’t changed much at all. It still relies on the 16-bit interface of x86-style CPUs. And it has very limited memory space—just 128 kilobytes—in which to execute the option ROM firmware that’s stored on expansion cards; this severely limits the number of cards that a system can host.

BIOS enhancements have been few and far between, with each improvement being laid atop the antiquated foundation of the original. The technology moved away from unalterable ROM (read-only memory) chips to a storage medium that allows for changeable settings and that can be flashed with entirely new versions. Because the underlying technology hasn’t changed, tweaking your BIOS remains fraught with danger for users who aren’t educated and careful. Updating your BIOS incorrectly—or just encountering a power outage during the flash process—still holds the potential to kill your motherboard.

(And as we highlighted just recently, yes, Intel was involved in herding the cats that gave us UEFI.)

BIOS has nonetheless stuck around as shorthand, the way many people refer to solid-state drives as hard drives, despite a new technology effectively replacing it completely. It’s that small little piece of computing, a liaison between your hardware and software, that makes the whole thing work. Without it, we’d be pushing a boulder up a mountain with every single app.

Rockbox made it possible to run Doom on an iPod. Because of course it did.

Five notable examples of firmware you might know about

  1. Open Firmware. This open standard, developed by Sun Microsystems and standardized by the Institute of Electrical and Electronics Engineers (IEEE), found use in many non-PC computing platforms. Non-Intel machines, particularly Macs and Sun machines, used Open Firmware for years—and I gotta say, I still have memories of trying to remember all those weird commands.
  2. Rockbox. This firmware modification, available on many portable media players developed throughout the 2000s, expanded what devices like iPods could do—for example, adding support for additional audio formats.
  3. OpenWrt. This well-known open-source firmware, widely used in wireless devices, became popular after Linksys accidentally sold a popular router with Linux-based firmware—something we wrote about in a well-read 2021 piece.
  4. Smart TV apps. If you’ve ever used a Roku, you’ve used firmware. It’s perhaps the most visible example of a modern embedded system, and it also shows how firmware often straddles the line between operating system and embedded tool.
  5. Android custom ROMs. If you’ve ever wanted to modify your operating system, it is certainly more than possible with Android if your device is unlocked. Its installation process is similar to more traditional firmware, but when you’re done, you get a brand-new OS interface.

DFU

Device firmware upgrade, the name of the process that USB devices, particularly iPhones, use for getting their firmware upgraded. Interestingly, Apple has implemented this approach to its recent Apple Silicon-based PCs, requiring end users to use a tool called Apple Configurator to install firmware when an error emerges. In a way, it highlights how Apple thinks about firmware on modern computers—as something that should work like it does on a smartphone.

These days, firmware is everywhere, gradually shaping the experiences we have with the devices we build. Perhaps the people making it aren’t burning the midnight oil, trying to recreate it in their garage, in an effort to work around IBM.

The cloak and dagger stuff is no longer necessary, and not quite this perfectly lit.

But I do think our relationship with firmware has definitely changed in the nearly 60 years since Ascher Opler dropped the mic and ended up revealing one of the most important computing concepts before departing way too soon, staying just long enough to share his best idea with us.

(Ascher Opler’s legacy sparks a great question for me: If you invented an important concept, but would not live to see it take over the world, what big idea would you share?)

In recent years, I’ve been keeping an eye on efforts to make firmware more user-modifiable and upgradable, such as Libreboot, which relies on the coreboot firmware platform to replace one of the few areas of computing that has proven surprisingly impervious to user freedom—the boot process. I like the way Libreboot talks about all this on its site:

Libreboot is a community-oriented project, with a focus on helping users escape proprietary boot firmware; we ourselves want to live in a world where all software is free, and so, Libreboot is an effort to help get closer to that world. Unlike the big vendors, we don’t try to stifle you in any way, nor do we see you as a security threat; we regard the ability to use, study, modify and redistribute software freely to be a human right that everyone must have. Extended to computers, these are products that you purchased, and so you should have the freedom to change them in any way you like. When you see Intel talk about their Boot Guard (which prevents Coreboot by only letting firmware signed by them be executed) or other vendors imposing similar restrictions, and you hear them talk about “security”, they are only talking about their security, not yours. In the Libreboot project, it is reversed; we see Intel Boot Guard and similar such technologies as an attack on your freedom over your own property (your computer), and so, we make it our mission to help you wrest back such control.

The thing is, if a computer manufacturer decides they don’t want to give us certain features in their firmware, they just won’t, limiting the experience for the end user, and offering a mechanism for control that limits trust. It makes it so that you’re having to work around their decisions, rather than the other way around.

(It also, arguably, makes the computing experience a bit more big-brotherish, as anyone who has ever been locked out of a work laptop after getting laid off might tell you.)

And this firmware sticks around even if every other part of the experience is otherwise opened up. You can use a completely open stack on your machine—FOSS operating systems, FOSS applications, the whole

Leave a Reply

Your email address will not be published. Required fields are marked *