Quoting Illuminata Perspectives

There is an accusation making the rounds that VMware ESX Server has an issue with the GPL because of the way the product blends Linux with proprietary (closed source) components. After looking into this for a couple of days, I don’t buy it. That said, this argument playing out in various blogs brings up some points that are worth discussing.

First off, some background. Mike McCana, writing on his VentureCake blog, first brought the issue to wide attention. It was subsequently reported in The Register (an article for which I was interviewed) and various blogs. The basic issue is as follows. As most folks involved with servers know by now, VMware ESX Server is a server virtualization product that allows multiple “guest” operating systems to co-exist on a single physical server independently of each other. ESX provides what is known as “native” virtualization—that is, the VMware software sits directly on the physical hardware; it isn’t “hosted” like an application in the manner of, for example, Microsoft’s Virtual Server. (My recent The Server Virtualization Bazaar, Circa 2007 provides a great deal of background about various virtualization technologies.) We usually use the term “hypervisor” to refer colloquially to this layer of software that lets virtual machines be created on top of it.

This is somewhat of an oversimplification, however. ESX Server, similarly to the Open Source Xen Project, actually has two major pieces. One is a virtual machine manager (VMM)—the layer that actually controls the virtual machines. In VMware’s case, it’s called the “VMkernel” and is proprietary code. The other is a service console that lets the user control and monitor the functions of the VMM. This “Console OS” is based on the Linux 2.4 kernel; it’s Open Source under the terms of the GPL.

The VMkernel and the Console OS are two separate pieces of code. VMware’s Zachary Amsden describes how they work (in a comment to the VentureCake blog post):

First, the vmkernel is not a Linux kernel module. The vmkernel is a completely isolated and separate piece of software which is loaded by a module called vmnix. The vmkernel has no knowledge or understanding of Linux data structures or symbols, and as a necessary result, does not depend on the Linux kernel for any services whatsoever.

Second, the vmkernel does not run inside or as part of the Linux kernel. It simply takes over control of the CPU and switches into a completely alien operating mode - one where Linux itself no longer exists. The former kernel used to boot the systems is still alive, but to switch back to it is a complex and involved process, similar to the well-defined copyright boundary of switching between two user processes, which are completely separate programs, running in their own address spaces. The vmkernel and the console OS Linux kernel are two completely separate entities, and the process of going from one to another is even a stronger separation than that given to user processes - more like rebooting the processor and re-creating the entire world.

At issue here is that Linux (vmnix is part of the Linux-based Console OS) is involved with bootstrapping the VMkernel. Does this constitute a form of linkage that would make the entire resulting “work” (i.e. the whole of ESX Server) subject to the GPL—thereby requiring that the source code be made public?

It seems unlikely. Linux is routinely used to load “binary blobs” such as closed source device drivers without suddenly making them subject to the terms of the GPL. Perhaps more directly on point, the bootstrap loaders used to bring up Linux (such as GRUB or LILO) can be freely used with Windows and other proprietary operating systems. Although some Linux kernel developers, such as Alan Cox, are on record as questioning the legality of loadable kernel modules with proprietary licenses, it’s a minority view and widespread common practice supports their use. I’m not qualified to give a legal opinion, but there would seem to be far more examples that support VMware’s use of GPL code than the opposite.

Furthermore, VMware’s use of Linux is hardly a secret that’s just been uncovered—as the usual corporation-hating online mob crying “scandal!” seems to be implying. The architecture of ESX Server has been known for years. Books have been written about it. Therefore, it’s hard for me to believe that any potential issue here hasn’t been thoroughly researched and discussed by people familiar with both the technical and the legal details. Certainly there have been cases where GPL violations buried deep in some network device’s firmware have only come to light belatedly. But that’s not the case here. VMware has never exactly trumpeted the fact that it leverages Linux. But it’s well documented. That something like this was sitting in plain sight for years and that even competitors (such as the various Xen proponents) never raised a public peep certainly suggests that there’s not much to raise a peep about.

But what, hypothetically speaking, if there were an issue here? Or, at least, many people became convinced that—buried in the complications of how software links to each other across license boundaries—there at least might be something of substance. Well, that would be an annoyance for VMware. It would be a very bad thing for Open Source—and the GPL most of all.

It would be an annoyance but hardly a fiasco for VMware. They would have the option of redoing the Console OS with a different foundation—say, BSD which has a more permissive license. There’s nothing special about the old version of Linux used for the Console OS today. Such a change would be somewhat disruptive but wouldn’t be the end of the world. VMware would also have the option to Open Source the VMkernel. This isn’t as radical as it sounds. Many partners already have access to the source code through VMware’s Community Source program. Furthermore, while the ESX Server code may once have truly been the company’s “crown jewels,” the value provided by VMware has increasingly moved to its Virtual Infrastructure products that sit on top of ESX Server and provide services such as workload balancing and management. Thus, even if there were legitimate issues here, VMware has various options to mitigate them.

The impact to Open Source in general and the GPL in particular would be far more profound. Critics have been knocking the GPL for years for its “viral” nature—a knock that’s based on language in the license that requires modified versions of GPL-ed code to be licensed under the GPL as well. One implication of this language is that if one makes changes to a program in the form of additional source code files and links them all together, the entire program is under the GPL—not just the original files. However, less tightly-coupled forms of using GPL code in conjunction with proprietary code have generally been considered legit. For example, just about no one disputes that it’s OK to run a proprietary application on top of Linux; using OS services and interfaces doesn’t make a program a derived work just because the application requires Linux to work.

Between these two extremes lies some ambiguity that revolves around the exact form that the linking takes place. What’s permissible has historically been very much tied to Unix-oriented concepts such as static vs. dynamic linking that don’t necessarily carry over cleanly to other contexts. (Some of these aspects of the GPL are discussed in detail here by attorney Lawrence Rosen.) When the linking takes a form that doesn’t have a direct analog in Unix—as is the case when an application implements some unique plug-in architecture—what’s allowed by the GPL and what isn’t becomes a bit blurry. (The Trac software bug tracking system switched from a GPL to a BSD licence for this reason.)

This ambiguity makes it at least understandable that GPL issues are being raised in borderline cases—especially given that there is precious little in the way of case law to help finely parse what sorts of linking are OK and which aren’t. However, if this sort of thing were to become commonplace, it would cast a pall over using GPL software in concert with any proprietary code. Open Source fans have been dismissing the “viral” tag for years—but if just about any form of connection between two programs can be viewed as leading to a derivative work, then viral seems a pretty good description of things. Easier it would be to just avoid Open Source entirely, or at least to limit choice to a license like BSD that allows allows mixing of proprietary and Open Source code with essentially no restrictions.

Read the original, here.