Groklaw Takes A Closer Look at the ABI Files, by Frank Sorenson et al

By Pamela Jones
Groklaw

January 30 2004

This article is the result of a group research project, compiled and primarily written by Frank Sorenson with Pamela Jones. The special footnoted article explaining some of the terms for nonprogrammers was written by Nick Richards. The research group was primarily composed of Frank Sorenson, Dr. Stupid, Harlan Wilkerson, Rand McNatt, Roland Buresund, and Pamela Jones, all of whom contributed both research and writing, with input from some Linux kernel contributors. Everyone worked on editing, including an invited group of Groklaw regulars. However, Frank carried the load more than anyone else, so his name is on the finished article.

We are now publishing the article and welcome Groklaw readers' further contributions, corrections, improvements, and comments. This is an ongoing project. This article is the first in a series where we'll discuss the System V UNIX ABI, or Application Binary Interface. We approached the research as, What if Linus Torvalds had not already claimed paternity [ http://www.groklaw.net/article.php?story=20031222174158852 ] of most of the header files? Then what?

The article will first explain what the ABI is and what it does, then discuss whether the code was released under the GPL and if so whether management at SCO knew and approved, and finally show how the Linux files that pertain to this do not appear to be infringing files that SCO can claim.

For those who are not programmers, such as myself, there is a footnoted section to explain in greater detail and in plain English what ABIs and APIs and shared libraries are and how they work. If you read it first, it will clarify the terminology for you and you will be able to follow the thread in the article more easily. At least, it helped me tremendously.

I think you will see from this article alone that if SCO is planning to sue anyone over the ABI files, unless there are facts we haven't unearthed, they seem to be leaning on a rickety bamboo reed.

**************************************************************

GROKLAW TAKES A CLOSER LOOK AT THE ABI FILES
~by Frank Sorenson et al

Back in January 2003, word leaked out [ http://www.linuxbusinessweek.com/story/34695.htm ] that SCO was planning to charge Linux users for "System V Intellectual Property" in Linux. SCO created a new business division called SCOSource [ http://www.sco.com/scosource/ ] to come up with new ways to make money from this "intellectual property". The original SCOSource Presentation [ http://www.caldera.com/scosource/SCOsource_Presentation.pdf ] (PowerPoint version [ http://www.caldera.com/scosource/SCOsource_Presentation.ppt ]) talks of licensing SCO's shared UNIX Libraries from OpenServer and UnixWare for use in Linux.

A Little Background: What Are ABI Files? [1]

As background information, shared libraries are files containing information to be loaded when an application is run. They usually implement common routines, and their inclusion simplifies programming, reduces file size, and standardizes interfaces. A simple example of this would be the "copy file" and "move file" commands: both commands check file permissions and manipulate the file system. When applications have a great deal of functionality in common, this functionality is often placed into shared libraries.

Shared libraries are architecture, operating-system, and version specific. Executables for different systems follow various standard formats, such as a.out, ELF, and COFF. To load an application, the system must do several things: the system interprets the format of the executable (or binary), loads any shared libraries referenced, and begins executing the code found in the binary.

If the executable is in a different format from those the system supports, or if the library files are for the wrong architecture or operating system version, the binary normally will not run. In 1991, Intel announced [ http://groups.google.com/groups?selm=20016%40scorn.sco.COM ] the availability of the iBCS-2 (Intel Binary Compatibility Specification), a specification designed by Intel, SCO, and Unix System Laboratories (USL) to enable binary compatibility and migration capabilities between systems. SCO's Tim Ruckle said this to encourage support:

'The goal of 'shrink wrap' software for the open systems marketplace will not be possible without the commitment of the entire industry to achieve full binary compatibility. This requires more than just the ability for binaries to run on a given kernel--it must address system installation scripts, I/O control instructions to devices, and the availability of standard system commands.

"For the benefit of the entire user base, as well as the industry as a whole, SCO encourages all UNIX System vendors for Intel processors to join SCO, USL, Intel, ISC and OSF in supporting the iBCS-2 standard for x86 applications.

"Tim Ruckle"

Say that you are building yourself a house. In order to make this construction project something that you can reasonably do within your lifetime, you choose to use pre-made construction materials, such as windows, doors, lumber, bricks, and plumbing supplies that you can get at the local Home Depot. By doing this, you don't need to cut down your own trees, fire bricks, or make glass. Someone else has gone to the trouble of building these components, and you can just build on what they've done.

This can save a lot of time and energy when building your house, but it does require that you conform to certain standards, such as 16-inch centers on studs, 3-foot-wide doors, specific diameters on the plumbing and electrical pipes, and vanities 18- or 24-inches deep. If you don't conform to these building standards, ductwork will not fit between the floor joists, and your pre-built and pre-hung doors will have gaps around them, may stick, or will require you to shave a couple of inches off them. If you don't follow the standards, either something isn't going to work, or you'll have to build it from scratch.

By conforming to these standards, you can just buy a window and install it. You can find 2-inch drain pipes that fit the 2-inch pipe coming out of the drain in your sink. You don't have to custom build a door to fit whatever space you left. You can just choose where to put doors, windows, sinks, and other things in your house, purchase the components, and build it.

It's somewhat comparable in programming. Someone has built some useful functions already, but to use them, you need to know and use the interface provided so as to put these building blocks into the right places. The components of the programs all behave the same, because they're the same library functions.

How ABIs Are Used

SCO UnixWare or OpenServer executables have a different binary format from Linux, and reference different versions of different shared library files, and therefore will not normally run under Linux. However, with iBCS-2 [ http://www.dee.isep.ipp.pt/docs/kernel-ibcs-2.2.14/ ] it is possible to execute a UnixWare or OpenServer program on a Linux machine. First the Linux machine must understand how to interpret the foreign executable file. This functionality of executing foreign binaries in the Linux 2.4 kernel is provided by an addon called "linux-abi [ http://linux-abi.sourceforge.net/ ]." On that linked page, you will find a list of SCO software that can be run with linux-abi, as well as this information:

"The Linux abi is a patch to the linux kernel that allows a linux system to run foreign binaries. This was developed and written by Christoph Hellwig and Joerg Ahrens as a follow on to the iBCS/iBCS2 project written for the older 2.2.x kernel by Mike Jagdis."
More on Christoph Hellwig later, but most of you remember his name from earlier [ http://www.groklaw.net/article.php?story=20031130210334574#c28934 ] Groklaw articles as a Caldera employee who contributed code to the Linux kernel.

When a SCO binary requires a given shared library, the correct version of this library must be present in the appropriate location on the system. Copying SCO's shared libraries from a UnixWare or OpenServer system to the Linux system makes it possible to run a SCO Unix application. One example of this (from the SCOSource Presentation) is a Red Hat Documentation page [ http://www.europe.redhat.com/documentation/mini-HOWTO/WordPerfect-5.php3 ] where the author explains how to run WordPerfect 6.0 for SCO under Linux. The instructions include this caution:

"SCO Shared Libraries

"DO NOT VIOLATE SCO's COPYRIGHTS! You should get a copy of SCO's shared libraries and install them in /shlib."

SCO's Shifting Claims on ABIs

When SCO's allegations against Linux first surfaced, there was concern over the ABI code, but Blake Stowell reassured everyone in a MozillaQuest interview [ http://www.mozillaquest.com/Linux03/ScoSource-01_Story01.html ] that the linux-abi add-on was "clean", although the libraries themselves were SCO's IP:

Blake Stowell: No, none of the code in the Linux ABI modules contains SCO IP. This code is under the GPL and it re-implements publicly documented interfaces. We do not have an issue with the Linux ABI modules. The IP that we are licensing is all in the shared libraries - these libraries are needed by many OpenServer applications *in addition* to the Linux ABI.

In SCO's amended complaint [ http://sco.tuxrocks.com/Docs/IBM/Doc-25.pdf ] (SCO's version [ http://www.sco.com/ibmlawsuit/amendedcomplaintjune16.html ]), they brought up the linux-abi add-on and their libraries again:

47. Linux offers a "SCO emulation module," originally called "iBCS" and now known as "linux-abi" which enables applications which were originally developed to run on SCO OpenServer to be run on Linux. ... SCO does not give permission for copying of the Shared Libraries for use outside OpenServer without payment of separate licensing fees.

A significant expansion of SCO's claims occurred in a December 2003 letter to Unix Licensees [ http://www.groklaw.net/article.php?story=20031222161942627 ]. The SCO Group claimed ownership of not just the libraries, but over the ABI code itself:

"Certain copyrighted application binary interfaces ('ABI Code') have been copied verbatim from the UNIX System V code base and contributed to Linux for distribution under the General Public License ('GPL') without proper authorization and without copyright attribution. While some application programming interfaces ('API Code') have been made available over the years through POSIX and other open standards, the UNIX System V ABI Code has only been made available under copyright restriction."

History of the Publicly Documented Standards

Stowell was correct when he said that the Linux ABI modules implement publicly documented interfaces. In September 1993, over 75 companies, including Novell and SCO, agreed to adopt [ http://www.byte.com/art/9401/sec4/art9.htm ] a single set of 1170 API calls. This standard was known as Spec 1170, and is now called the Single UNIX Specification [ http://www.unix-systems.org/version3/ ], or SuS for short. In January of 1994, that Byte article linked to described the purpose of the standard adopted by SCO and the 75 other companies:

"Novell has transferred the Unix trademark to the international X/Open standards organization. The transfer, which was announced last October, when combined with other standards efforts, may yet result in multiple implementations of Unix that conform to a single specification. . . .

"The idea behind the common API is to let developers write to a single set of memory, file- system, and other kernel-level calls so that they need to do only a source-level recompilation to support another Unix platform. With multiple compatible implementations of Unix, vendors will then compete on the basis of price, quality, service, and reliability, or as X/Open's president and CEO Geoff Morris said, 'a single specification, a single brand, and as much innovation as the industry can deliver.' . . .

"Novell will not give up its right to license Unix System V source code to other vendors, but once the test suites are available, Unix vendors will no longer be required to use Unix code developed at USL/Novell."

The System V ABI information has been known for decades, and the specifications are even now available from SCO's web site [ http://www.caldera.com/developers/gabi/ ]. The specifications are spelled out in several parts, a generic part (gABI) and a processor specific part (psABI), which together form the specification:

"The System V ABI is composed of two basic parts: A generic part of the specification describes those parts of the interface that remain constant across all hardware implementations of System V, and a processor-specific part of the specification describes the parts of the specification that are specific to a particular processor architecture. Together, the generic ABI (or gABI) and the processor specific supplement (or psABI) provide a complete interface specification for compiled application programs on systems that share a common hardware architecture.. . .

"What is presented here is two chapters of the System V gABI: Chapter 4, Object Files and Chapter 5, Program Loading and Dynamic Linking. These two System V gABI chapters form the definition of the ELF (Executable and Linking Format) file format. Some complete, but older System V gABIs as well as some psABIs are available on the developer specifications page. Besides on System V implementations, ELF is very commonly used as the object file, shared library, and execution file format. Separating these two chapters of the System V gABI also makes it easier for these implementations to reference an ELF specification.

"The contents of these chapters is being actively maintained (in part) through an industry committee chaired by Intel. This group's efforts are greatly reduced from when they first were extending ELF to support 64-bit architectures, but there is still some active development."

Not all of the information contained within the ABI even belongs to SCO, as they well know and acknowledge in both the gABI [ http://www.caldera.com/developers/devspecs/gabi41.pdf ] and the Intel386™ Architecture Processor Supplement [ http://www.caldera.com/developers/devspecs/abi386-4.pdf ]:

"We acknowledge the contributions of the 88OPEN Consortium Ltd., portions of whose System V ABI Implementation Guide for the M88000 Processor and the System V ABI M88000 Processor Networking Supplement have been incorporated in this section of the ABI with permission."

The entire IntelŪ ItaniumŪ Processor-specific Application Binary Interface (ABI) [ http://www.intel.com/design/itanium/downloads/245370.htm ] carries an 2001 Intel copyright, calls the System V ABI the "prime reference document," and says that the ABI was developed as the result of "consensus among operating system vendors...[including] Intel, Sun Microsystems, SCO, IBM, SGI, Cygnus Solutions, VA Linux Systems, HP, and Compaq."

SCO's web site touts the benefits of ABI Conformance [ http://ou800doc.caldera.com/DIFFS/_ABI_Conformance.html ], and encourages system and application developers to "seriously consider the benefits of building [their] products in conformance with the binary compatibility specification".

In the SCOSource Presentation [ http://www.caldera.com/scosource/SCOsource_Presentation.pdf ], The SCO Group admitted that "Because ibcs2 [the Intel Binary Compatibility Standard] is an open specification, the Linux community was able to freely copy this and rename it Linux ABI or Linux Application Binary Interface." These interfaces are also documented in a number of other places, including IEEE Std 1003.1 [ http://www.opengroup.org/onlinepubs/009696799/toc.htm ].

Did SCO Release the ABI Code Under the GPL?

The evidence indicates they not only did, they still are.

Let's follow the evidence trail. By SCO's own admission, the Linux ABI modules do not infringe on SCO IP, since they implement public standards. So how could any code implementing those standards infringe? You can read them but not use them? Moreover, though they claim they never made their own "ABI code" available outside of OpenServer, does the evidence support that claim?

In SCO's December 2003 letter, they named a number of Linux files, including errno.h, ctype.h, and signal.h, that they said they believe contain their copyrighted source code implementing the ABI. In the SCOSource FAQ [ http://www.caldera.com/scosource/linuxlicensefaq.html ], SCO claims "SCO has obviously never placed a notice indicating that our UNIX source code can be distributed under the terms of the GPL license." Is this a true statement? Has SCO ever released these particular files (or the information contained therein) under the GPL, or have they just been used without their knowledge and permission?

These Linux files can be found inside libc packages from the various versions of OpenLinux that SCO/Caldera produced, and many of the packages may still be found on their FTP site to this day. Here are just a few examples:

There can be no question that the LGPL [ http://www.gnu.org/copyleft/lesser.html ] files were present on the OpenLinux CDs that Caldera was producing and distributing. The SCO Group would have us believe that they were not aware of what they were distributing, but were only passing along packages created by someone else. Did Caldera know that the errno.h and other files existed inside those packages on their CDs and were they put there intentionally by Caldera? SCO says obviously not, but the evidence speaks loudly otherwise. At this point, we encounter Christoph Hellwig once again.

As we have already seen, according to the Linux A.B.I. homepage [ http://linux-abi.sourceforge.net/ ], Christoph Hellwig, working for Caldera, developed and wrote the linux-abi patches to allow the Linux kernel to run UnixWare and other foreign binaries. The entries in the ChangeLog [ http://www.kernel.org/pub/linux/kernel/people/hch/linux-abi/v2.4/ChangeLog ] show Christoph Hellwig as the principal maintainer of the patches, and the patch releases were announced by him as well (for example, 2.4.14 kernel [ http://lwn.net/2001/1122/a/linux-abi.php3 ] and 2.4.15 kernel [ http://lwn.net/2001/1129/a/linux-abi.php3 ]). Here are two one brief snips from the changelog to show you what we mean:

"2002-01-03 Christoph Hellwig (hch@caldera.de)
"linux-abi-2.4.17.0

"2001-11-24 Christoph Hellwig (hch@caldera.de)
linux-abi-2.4.15.0

This work by Caldera provided the linux-abi package add-ons for Linux, enabling programs compiled for UnixWare (and other systems) to run on a Linux system. However, as we have explained earlier, having the linux-abi package alone is not enough to run a UnixWare program. The program would need to make use of libraries compatible with the right SCO system.

To run binaries from older versions of SCO's operating systems, it is possible to use shared libraries from the Coherent Operating System [ http://www.uni-giessen.de/faq/archiv/coherent-faq.general/msg00000.html ]. Created by Mark Williams Company, in Northbrook, Illinois, Coherent was a multi-tasking, multi-user operating system similar to System V Unix, but it did not require either an AT&T or SCO license. Coherent had implemented their Unix so closely to SCO UNIX (in fact, more closely than Linux has ever been) that they could run SCO binaries directly, as they explain in the introduction to the Coherent FAQ:

"Coherent is a multi-tasking, multi-user operating system for IBM PC's and compatibles. Versions will run on most systems with a 286, 386, or 486 CPU. Coherent appears to be very similar to Version 7 Unix, or for those of you that have not had the pleasure, close to System 5 Unix. Coherent includes uucp (Coherent to Coherent Copy) :) and many more functions you would expect with Unix. Coherent is small and simple, easy to maintain, and does not require extensive CPU resources to operate well. It has serial networking, V4.2 has X Window support. It does not require an AT&T license for Unix. It is as advertised, a quick, elegant Unix-like multi-tasking, multi-user Operating System, with good documentation. The newest version is very much like SCO Unix, and version 4.2 will actually run out of the box applications."

Coherent's system and libraries were implemented by following the Intel386 Architecture BCS Supplement, and were therefore Intel BCS-compliant. AT&T examined Coherent's code [ http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&threadm=3A0A168D.F5C2053D%40bell-labs.com ] and found that while it had been written by someone with in-depth knowledge of Unix, nothing could be found that was copied or wasn't reproducible by using the manual. Coherent and other non-UNIX operating systems have used these System V ABI's for years with AT&T's knowledge. Neither AT&T, Novell, nor SCO ever had any problems with this, but then why would they, since iBCS was the defacto open standard?

If you wanted the official UnixWare shared libraries, one way to get them was to purchase UnixWare and to copy the libraries onto the Linux machine. However in the SCOSource Presentation, SCO claims that the SCO Unix shared libraries have never been licensed outside of the SCO products. The SCO Group's amended complaint [ http://sco.tuxrocks.com/Docs/IBM/Doc-25.pdf ] also asserts that SCO did not give permission for copying of the shared libraries for use outside of SCO's products, and the SCOSource Q&A [ http://www.caldera.com/scosource/Final_SCOsource_QandA.html ] claimed "the UNIX shared libraries, owned by SCO, are not Linux products. They are not open source software and they are not covered by the GPL."

While this may be true for other UNIX shared libraries, SCO did in fact provide some Unix shared libraries, as products for Linux, under the GPL. The evidence shows that SCO packaged and provided a set of Unix shared libraries, with a System V ABI, as a GPLed product, to Linux users.

The library set released implemented the basic functions needed for simple (non-graphical) programs. Rather than require customers who only wanted to run these programs to purchase an expensive UnixWare license, SCO released an add-on with these basic libraries. This add-on was released by Caldera under the GPL and was packaged by then-Caldera employee Tim Riker [ http://rikers.org/ ]. Want some proof? The source code (including the Unix shared libraries) is still available from SCO. Here are just a few examples:

How do we know that these packages were, in fact, created and distributed by SCO/Caldera and that they were not packaged by someone else and just passed along unwittingly by SCO/Caldera? The process of verifying the file signatures will prove that the packages were actually provided by someone at SCO/Caldera with the authority to sign, something only a trusted representative would have authority to do.

By checking the file signatures, we can see that these packages have been signed by SCO/Caldera's Key. The keys can be found here [ ftp://ftp.sco.com/pub/pgp-keys/PGP-KEY-integration ] and here [ ftp://ftp.sco.com//pub/pgp-keys/PGP-KEY-security ]. To verify the rpms, download and import the keys, then download the packages and check the package signature like this: 

$ rpm --import PGP-KEY-*`
$ rpm -qp --checksig iBCS-*`

The license for the package can be verified as GPL with this: 

$ rpm -qip iBCS-*

While these GPL'd source files contain 'lite' versions of SCO's shared libraries, which are significant in their own right, a closer look reveals the presence of source code files of great importance. Among the files contained in these packages are: 

ctype.h
ctype.c
ipc.h
errno.h
signal.h
stat.h

In other words, SCO's GPL'd packages contain many of the very same files that constitute the ABI code SCO recently identified [ http://www.groklaw.net/article.php?story=20031222161942627 ] as allegedly infringing. A close look at these files shows that none contain any copyright notice whatsoever and that they implement the majority of the ABI code in dispute. The errno.h file, for example, lists all of the error codes from System V, with the same numbering that System V uses. The same is true for the signal.h and the ioctl numbering.

Unpacking the source code RPM file reveals some 'patch files' - modifications made to the basic source by the packager. This shows that Caldera actively tested and tweaked the source code, even if some of it was gathered from other sources. Moreover, at least one of these patch files was specific to Caldera's own version of Linux.

This indicates that representatives of SCO/Caldera were the ones who collected, developed, tested, and maintained the source code to implement the ABI functionality, then placed the GPL notice on it, packaged it, and provided it to the entire Linux community. Given that SCO themselves did so much of the work to provide this functionality, and placed the GPL notice on it themselves, judging from these files, it seems hard to avoid concluding that SCO explicitly released the System V ABI under the GPL and that they did so knowingly and because they wanted to. Distributing code under the GPL under those circumstances is like losing your virginity. Even if you regret it later, there is no turning back to the way things were and no way to undo the consequences. SCO's claims are a little like falsely claiming rape after willingly consenting to sexual relations. Worse, from their standpoint. The evidence of the GPL distribution is there for the world to see, so it's not a He Said, She Said situation where you might have at least a 50% chance of being believed.

Moreover, as this ABI code was released under the GPL with no copyright management information, it is difficult to understand how SCO could now view it as infringing their copyrights. SCO worked side-by-side with IBM, Linux distributors, and others in defining and publishing the specifications and actively developed and distributed the material that they now wish to claim is infringing.

Did Management Know?

While we have shown that at least some employees at SCO were actively developing and releasing GPL code and libraries to support the linux-abi project, the question could be asked whether Hellwig or Riker were working on their own, without authorization. Let's see.

Not only was SCO involved with the linux-abi work, but SCO employees were involved in additional low-level coding activities. Both (old) SCO and Caldera were also heavily involved with the Linux Standards Base (LSB) [ http://www.linuxbase.org/ ], which had - and still has - this as their mission statement:

"To develop and promote a set of standards that will increase compatibility among Linux distributions and enable software applications to run on any compliant system. In addition, the LSB will help coordinate efforts to recruit software vendors to port and write products for Linux."

The LSB was formed in 1998, with Caldera supporting it [ http://old.lwn.net/1998/0528/a/lsb.html ] from day one. Notice the purpose and the top signatories to the proposal for formation of the LSB:

"The Linux(R) Operating System's rapid adoption by millions of computer users everywhere is direct recognition of the quality of the software and success of the freely distributable software development model. In order to ensure that large software application programs, from binary-only tools sold by the largest software companies, to freely distributable desktop environments built cooperatively over the net, run smoothly on as many Linux-based computers as possible - the Linux Standard Base (LSB) Project is an attempt to define the common core of components that can be expected to be found in any "Linux" system. . . .

"Participation in the base standard will assure the distributions of compatibility with each other for the set of applications that depend only on the files and libraries in the reference platform. As time passes, the standard will expand to include most of the files and libraries upon which a commercial application might depend.

"The Linux Standard Base System will be 100% compliant with the Open Source Definition. This assures all distributions that they can derive from it without concern over licensing problems for themselves or their users. Development will be carried out in the public, with anonymous access to the CVS archive and the developer mailing lists. The core group will be a mix of high-quality developers from the Linux community and the staff of commercial distributions, with an organization similar to the tremendously successful Linux kernel development team. Attention will be paid to standards such as POSIX and the FHS (the successor to the Linux Filesystem Standard). However, the project goes far beyond the utility of these standards, because rather than produce only paper documents, it will provide a complete implementation of the standard, ready to be integrated into Linux distributions or used as a reference platform for application developers. . . .

"The Linux Standard Base System will implement some of the goals of the 86open project, which proposes to establish an interoperability standard for all Unix-like operating systems.

"We, the undersigned, endorse this proposal, and ask that other distributions and ISVs also join us to help further define this proposal and then to help implement it:
Linus Torvalds, Creator of Linux
Jon A. Hall, Executive Director, Linux International
Bruce Perens, Director Linux International, proposed Project Leader
Ransom H. Love, Director Linux International, General Manager, OpenLinux Division, Caldera, Inc. . . ." [emphasis added]

In May 2000, both (old) SCO and Caldera became charter members [ http://members.freestandards.org/news.php?id=12 ] of the Free Standards Group [ http://www.freestandards.org/ ] to "accelerate the use and acceptance of open source technologies through the application, development and promotion of standards for open source development." Obviously, this was a company decision, not something a rogue employee arranged.

Both (old) SCO and Caldera participated in LSB Conference Calls [ http://www.linuxbase.org/talks/mindex.shtml ], including at least one [ http://www.linuxbase.org/talks/20000412.html ] in which the header files in question were discussed, and Caldera presented [ http://www.linuxbase.org/talks/scoforum082001/img0.htm ] about the LSB at SCOForum 2001. The participation was constant, beginning with the very first conference [ http://www.linuxbase.org/talks/19990609.html ] in 1999, which Caldera's Ralf Flaxa attended, to one on July 19, 2000 [ http://www.linuxbase.org/talks/20000719.html ] with SCO's Dave Prosser participating, and included this April 23, 2002 conference [ http://www.linuxbase.org/talks/20020424.html ], where both Flaxa and SCO's Doug Beattie were participants, to the LSB Workgroup meeting on June 26, 2002 [ http://www.linuxbase.org/talks/wg20020626.html ], where Flaxa is listed as the LSB Implementation Lead in charge of the LSB Sample Implementation project. On January 17, 2001 [ http://www.linuxbase.org/talks/20010117.html ], Beattie and Prosser and Caldera's John Terpstra participated in the LSB telephone conference and on the agenda that day, you find these tasks:

Dave Prosser, a senior engineer for (old) SCO, then Caldera, worked for AT&T back when Unix was younger. When the C programming language was being standardized, Dave Prosser served as chief editor [ http://groups.google.com/groups?selm=1993Aug11.133337.19199%40allegra.att.com&oe=UTF-8&output=gplain ] for the ANSI C standard. He stayed on as Unix was passed through Unix System Laboratories and Novell.

Since the gABI was a changing standard, Prosser maintained it and kept it up to date. Updates (such as this one [ http://sources.redhat.com/ml/binutils/2002-06/msg00123.html ] where Linux gets e_machine numbers for the PDP-10 and PDP-11) were sent to Prosser at registry@sco.com (later, registry@caldera.com). Prosser coordinated the approvals for the release of the updated gABI specifications at sco.com [ http://www.sco.com/developers/gabi/ ].

Not only did Dave Prosser contribute ABI information to the LSB, but he also contributed other "valuable" SCO IP on the LSB email lists. Here [ http://lists.debian.org/lsb-test/2001/lsb-test-200105/msg00008.html ], he is seen sharing UnixWare source code as an example of how tmpfile.c was programmed. As you see, he shared source code without mention of confidentiality or other restriction on use or adoption.

Doug Beattie, another senior SCO engineer, was involved [ http://members.freestandards.org/about/people.php#beattie ] with the Free Standards Group beginning in the fall of 2000, and was actively working with the LSB to disseminate information and keep everyone updated. Beattie indicated that he was cooperating to make sure that the LSB and ABI could work together, as can be seen in this post [ http://freestandards.org/pipermail/lsb-wg/2001-February/000275.html ] to the LSB working group, where he is listed as having the title Linux Test Architect for SCO:

"As before, these are just starting points and there is much work to be done in various areas to make these workable for the LSB."

In August 2000, Flaxa presented [ http://mail.freestandards.org/pipermail/lsb-wg/2000-August/000142.html ] Caldera's Linux Technology Preview to the LSB, and it subsequently became the LSB sample implementation. Here's the revealing exchange of email between Flaxa and IBM's George Kraft, just after the Caldera acquisition of UnixWare from SCO, regarding the LSB sample implementation. A little piece of history. First Flaxa [ http://mail.freestandards.org/pipermail/lsb-wg/2000-August/000143.html ] lets it be known that it's about ready, it's under the GPL, and it includes the 2.4 Linux kernel, and most LSB ABIs, with the rest to be added. He wonders if he should release now or wait:

"It has been extremely busy times for me the last weeks, and with the Linux Technology Preview product (announced last week) and us aquiring big parts of SCO (announcement went out today) you may now understand why.

"The Linux Technology Preview (ISO images are available for free download from our FTP site) contains all the stuff, that we have been working on, so it's a much more up-to-date snapshot of our stuff than I had at the LSB meeting. It contains the beta of glibc-2.2, the current linux-2.4.0-test kernel, ncurses-5.0 and so on and all packages fixed to compile and work with that. It also contains the FHS patches and RPM macros to support the new locations of /usr/share/man/LOCALE and so on.

"It's someting like a preview of the next generation development platform. (we stricly tell everybody not to use this in production use of course!)

"But before you misunderstand it - no this is not the sample implementation nor the stuff that I wanted to deliver to the LSB. This is a snapshot of the core of the next Caldera products, so much more than would be in the sample implementation and packaged just like a distritbution, [sic] with a graphical installer and all the bells and whistles we will not have in a LSB sample implementation.

"And some LSB ABI that are speced out (like the sysvinit proposal from Ted) are not yet in that snapshot because we ran out of time.

"So bottom-line: It contains all we have, which is many things the LSB requires, but it doesn't contain other things yet that the LSB requires and it contains way to much for a sample implementation.

"So basically I have now (still) the things to do, that I wanted to do right after the last LSB meeting, which are: "I can now reassign big parts of my engineering team back to LSB, at least the 4 people that had been working on it before. This can happen starting tomorrow.

"I want to make everything available to SourceForge now. The question is how. I don't want that people misinterpret what I currently have to be the sample implementation or say "this is all Caldera proprietary stuff" and ignore it. It's all free and GPL, but that doesn't matter, because it's not LSB sample implementation, it's a preview of a Caldera product and might lead people in the wrong direction (or argueing).

"The problem is, the Linux Technology preview is what we have, now we have to recompile it with LSB patches being turned on and enabled and then strip it down to only what is in the LSB spec. This will not be done in a day, but I think it can be done in 1 week (some things will break) and after another week of fixing it might be quite usable again.

"I am asking for your advise how to best proceed from here. Shall I make available what I have as soon as possible? How can I make clear what the LSB sample implemetation is and that the Linux Technology Preview is something that will be mutating to that over the next few days and weeks. . . ."
Kraft answers [ http://mail.freestandards.org/pipermail/lsb-wg/2000-August/000143.html ]:
"Let's run with it. Can you get the LSB sample implementation web page to point to ftp://ftp.caldera.com/pub/LTP/ or copy the images to the LSB's download area on SourceForge?

"Also, can you writeup a brief overview for the web site of what is interesting to the LSB (basically the fullfilment of the LSB requirements): "[BTW ftp://ftp.caldera.com/pub/LTP/ says glibc-2.2 and http://www.calderasystems.com/preview/ says glibc 2.1.91]

"and what is not necessarily within the scope fo [sic] the LSB: "Is there enough source and tools within the ISO images to reproduce itself? Can you writeup how someone else can take what you have and rebuild it to duplicate your results.

"Can you enter the sample implementation todos in the task manager? https://sourceforge.net/pm/?group_id=1107

Flaxa then replies [ http://mail.freestandards.org/pipermail/lsb-wg/2000-August/000144.html ] that he can and will do the tasks requested. The LTP is still available on Caldera's FPT site here [ ftp://ftp.caldera.com/pub/LTP/SRPMS ].

Caldera must have been proud of their LSB work, because in March 2001, Caldera jumped the gun by announcing [ http://lists.debian.org/lsb-discuss/2001/lsb-discuss-200103/msg00041.html ] Project 42, their beta version of OpenLinux, as having an LSB 1.0 implementation, even before the standard was finalized and a certification program in place. The next day, an updated press release [ http://ir.sco.com/ReleaseDetail.cfm?ReleaseID=39972 ] replaced the first, and (appropriately) made no mention of LSB compliance.

It would have been difficult to achieve point-by-point compliance with the LSB while completely missing the concept that the LSB implemented both the general and platform-unique ABI specifications. In order to achieve compliance, someone would have been knee-deep in the code and couldn't have ignored the ABI specification staring them in the face. If there was anything protectable about the code, that was the time to say so.

Both (old) SCO and Caldera made many public statements about their support for Linux and the LSB. David McCrabb, who was president of (old) SCO's server software division, then president and chief operating officer of Caldera, gave a presentation [ http://www.hkcs.org.hk/linux_presentation_240501.ppt ] before the Hong Kong Computer Society on the "Future of Linux," and listed five critical success factors in the adoption of Linux, including the LSB. While he fell short of giving his full support for the Linux development process and the GPL, it cannot be argued that he didn't know about the LSB or what it was.

Since the fundamentals of the Linux ABI have not changed between then and now, if Linux is now infringing on SCO's rights, it was back then as well. Yet Doug Michels, (old) SCO CEO, who felt that Linux could provide free research and development [ http://www.computerworld.com/news/1999/story/0,11280,35431,00.html ], and who had been trying [ http://www.vnunet.com/Analysis/1108193 ] to put a Linux personality on UnixWare, essentially going in the other direction from ABIs, so as to run Linux programs in UnixWare, presented [ http://www.uniforum.org/web/pubs/uninews/970411/thenews4.html ] a "Lifetime Achievement Award" to Linus Torvalds at UniForum '97. Here's an account of what he said:

"Linus received the UniForum "Lifetime Achievement Award" presented by Doug Michels, co-founder of SCO. Doug said that many people expected him to turn down the opportunity to present the award, or to say little when he did. He said that these people had him all wrong, that this was a special opportunity. Doug said he has been amazed at the success that Linux has had. Linux has rekindled the spirit and energy that first got him excited about Unix 20 years ago. The work of Linus and the greater Linux community has brought back a source base for real experimentation and innovation, something the computer industry badly needs again. Doug is proud of the selfless and true commitment that Linus and the Linux community have towards improvement of computing. Linus has brought the fun back in an otherwise static industry. He said that he thinks Linux is just what the world needs now."

Ransom Love, Manager of the OpenLinux Division at Caldera, then CEO of Caldera Systems made public statements [ http://freshmeat.net/articles/view/110/ ] in support of the LSB, and when Caldera purchased SCO's divisions, he became CEO of the new company.

Here's a fascinating segment from an article [ http://www.practical-tech.com/infrastructure/i08042000.htm ] on Love and his GPL plans for Unix:

"According to Ransom Love, Caldera Systems' CEO and soon to be CEO of the combined SCO divisions and Caldera Systems, 'Caldera has a proven track record of releasing the most important stuff to the open community. We haven't decided on which license to use yet. For standards, GPL makes a lot of sense and every product we'll ship with source code.' Some code, however, can't be open sourced because other companies own it.

"Caldera sources indicate the code that can be open sourced may be released under several different open-source licenses, including BSD, to the open-source developer community. Love promises more information at Forum2000, SCO's renamed annual partner/reseller get-together on Aug. 20 to 23 at the University of California, Santa Cruz. There he will present the broad outlines of Caldera's open-source plans.

"While some Linux purists would argue that there's no need to borrow from Unix. Others point out that such UnixWare features as ACL-based security, scalable SMP, logical volume management for storage, high-availability clustering and the VERITAS File System (VxFS) for fast file system recovery would be welcome additions to Linux.

"Of course, nothing happens in Linux without its founder's, Linus Torvalds, blessings. Fortunately for Caldera's plans, while he's waiting for the details, Torvalds generally approves of the idea of bringing over the best of Unix to Linux. Contacted by e-mail, Torvalds comments, 'I think the most likely scenario would be complementing each other especially in "non-core" technologies. It can actually go both ways, at least if the eventual license ends up being GPL -- UnixWare may end up picking up drivers, etc., from Linux (and stuff that was formerly an add-on, like the lxrun Linux binary emulation stuff).'

"While he thinks that, 'It's fairly unlikely that Linux/UnixWare would ever meld, but the same sort of thing that has been happening with other Unix components (like IBM's journaling file system getting ported over to Linux, etc.) is probably more likely to happen with UnixWare components.'"

So, IBM's porting of JFS was known back then, without a word of complaint from Caldera's Love, who was, according to this article, trying to do the same kind of thing with Unix code, the only blockage being that some of the code didn't belong to Caldera.

Love made numerous statements (such as here [ http://lwn.net/2001/0517/a/caldera-love-gpl.php3 ] and here [ http://www.interex.org/hpworldnews/hpw010/02nt.html ] ) regarding the merging of Unix with Linux, whenever possible, which was his stated goal:

"In conjunction with the announcement of the transaction, Love said, 'This acquisition is an industry-changing event that puts Caldera front and center as the answer to the enterprise question. Caldera will further broaden and validate both the Linux and UNIX industries and communities by providing open access to its unified Linux and UNIX technologies and by offering support, training, and professional services to customers worldwide. Caldera is fully committed to supporting and servicing the SCO OpenServer and UnixWare communities.'"

In January 2002, Caldera/SCO released [ http://www.groklaw.net/article.php?story=121 ] older versions of the Unix source code under a BSD-style license [ http://www.groklaw.net/article.php?story=246 ]. Some of these versions, such as 32V, contain the ABI files in question, and in fact, have no copyright notices in the files. Even if the Linux versions of these files had come from these older versions of Unix, which Linus says they didn't, no 'copyright management' information would have been removed in the copying.

This isn't even a case where the old management was aware and participating but the new management (McBride) didn't know what was happening. Darl McBride knew about the LSB work, and publicly touted SCO's involvement. When OpenLinux 3.1.1 achieved LSB Certification in September 2002, he said [ http://ir.sco.com/ReleaseDetail.cfm?ReleaseID=91330 ]:

"This is an important milestone for SCO. SCO is very dedicated to the development and promotion of standards. We see standards adherence as central to the growth and progression of the Linux industry, and are committed to again being LSB certified when we release SCO Linux, powered by UnitedLinux, this fall."

We have seen that both SCO and Caldera were involved in writing the LSB standard, and they made contributions of gABI information. Executives and programmers were actively involved. Does that not indicate that they approved of and intended to facilitate the use of the ABI files in the specification? The Free Standards Group specifications are licensed [ http://members.freestandards.org/policy/document-license-policy.txt ] under the GNU Free Documentation License version 1.1 [ http://www.gnu.org/licenses/old-licenses/old-licenses.html#FDL ], and require that copyrights be assigned to the Free Standards Group before being included in the specifications.

Why does the LSB matter? Because the LSB specifications [ http://www.linuxbase.org/spec/book/LSB-generic/LSB-generic/libc-ddefs.html ] contain the errno.h and signal.h files. How, then, can The SCO Group claim them now as their proprietary property and threaten to sue end users for using them? Caldera, when they were working with the LSB, helped to define what the mapping of error numbers to names was to be Linux. So an analogy for that would be someone sending in an article submission to the local newspaper and then suing the newspaper for copyright infringement when the article was published. Another way to look at it would be if a company suggested everyone use a certain symbol any time referring to a concept. Every other company agrees to follow the suggestion, and for many years everyone uses the symbol when discussing that particular concept. Then the original suggester claims that now that everyone is using the symbol, everyone needs to pay them because they never gave anyone the right to freely use the symbol. A separate question is what you can and can't copyright [ http://www.groklaw.net/article.php?story=20031228163637431 ], but we'll leave that for a later article.

If someone tells you that SCO didn't participate or was unaware that the System V ABI was being made available to Linux, or that Linux programmers "stole" it without authorization, what do you think? And if SCO goes forward, as they say they will, and sues an end user for "copyright infringement" of "their" ABI files, will they prevail? If you were on the jury, after reviewing this history, how would you vote?

To Be Continued

[Footnote 1] APIs and ABIs -- What are they and what does it all mean?

~By Nick Richards

API stands for Application Programming Interface and ABI stands for Application Binary Interface. Let's break this down.

What is an interface?

An interface is just a means or mechanism that allows two parties to connect or communicate with each other. When you flip on a light switch, you are using that particular interface (the light switch) to connect with the wiring of your home. The switch allows an abstraction of what is actually going on in your walls.

Another example is the way you communicate with a computer. How do you connect to a computer? You give it a command. How do you accomplish that? You have to use the mouse to click on a program, and then use the keyboard to type some parameters. Your desktop setup, the way the screen is painted, the way the mouse interacts with icons, the way the keyboard allows input, all of that is an interface, commonly called a graphical user interface, or GUI, since there are icons and the like. If you are using the Linux command line, your user interface leaves out the graphics part and the interface is primarily the keyboard alone.

So an interface allows one party or entity to connect or communicate with another party or entity.

The same thing happens behind the scenes with your computer. Programs have to connect and communicate with each other, and the way they do that is by means of interfaces.

Shared Libraries

When a programmer finds him or herself constantly having to reinvent the wheel by writing the same code over and over, it gets annoying. Not only is this time-consuming, but it can increase the number of times you make an error. So what programmers came up with long ago is the idea of putting commonly called functions or procedures into shared libraries. A library file contains source code just like a regular program, but this source code is generalized for use by multiple programs. For example, if you create an absurdly simple function that returns the sum of two integers, you can put that Add function in a shared library and reuse it across multiple programs or applications. All you have to do is tell the compiler to link that library into the project. That allows you to then just write Add (3, 4) in your code instead of also defining just what it is you mean by "Add." That definition has already been laid out in that shared library.

That's programming: when you tell the compiler what to do by writing near-English commands, you are creating a program. What the compiler does is take that human-readable programming code and turn it into computer-readable programming code. This is commonly called creating a binary file. Computers actually work with binary code, so programs have to be translated into that format before the computer can actually know what to do with your commands. Compilers are the translators of the computer world. More than that, they are the gatherers and translators, because at compile time they look through your code and find all the references to outside libraries and then go find those libraries. It doesn't matter if your instructions are found inside your main program or inside one of the libraries you referenced in your program. Wherever its location, that "Add" function will be found by the compiler and then translated into binary for the computer.

API vs ABI

We can define the difference between an API and an ABI like this:

An Application Programming Interface (API) is something that communicates with the programmer as code is written. It contains the near-English code that a human can read. A typical API will list all the functions the programmer can use, along with all the types of data the functions can be used to manipulate.

It is an interface, so it connects the program with information that program needs to compile properly. This connection occurs at compile time.

An Application Binary Interface (ABI), in contrast, is something that speaks to the computer. It contains binary information that a computer can read that talks about how the binary file is structured, how the program should be linked and loaded, and so on. It is also an interface, so it connects the program on that computer system with information that program needs to actually run properly. This connection occurs at run time.

It is possible for two computer systems to have the same APIs but have the associated ABIs be different. Compilers are programs that are specific to operating systems and computer chip sets. You can have the same source code on two different machines, but if you compile the source code on one machine with compiler A and on the other machine with compiler B, the ABIs could be different.

Header Files in Shared Libraries

So with all that in mind, what do we know about files claimed by SCO such as errno.h? That file is a header file, which is a type of library file spoken of earlier. Header files contain commonly defined variables that get used in many programs. Again, there's no point in reinventing the wheel every time you want to use one of these common variables, so you tell the compiler to gather in that header file instead at compile time.

Header Files Define Interfaces

The errno.h header file defines an interface, a way for the programmer to communicate with the end user by means of error messages. It is not, however, an implementation. Since this header file will be used by multiple programs, it is generalized, and that's why it's an interface but not an implementation. The light switch is how you turn on the light, the interface. When you actually flip the switch, you implement the interface. You have gone from the general to the specific and the light goes on. So too with errno.h. It is generalized in that it contains the error messages and the accompanying error abbreviations, but not the actual error numbers. Each system can define their own error numbers (1, 2, 17, 238, whatever) to those error abbreviations (E2BIG, EACCES, whatever). When you give it the actual error numbers, you are creating a specific implementation of that error interface.

Standard Interfaces

Everyone agrees to use these particular error abbreviations and messages. It's a standard. All you have to do in your particular implementation is to assign numbers to each abbreviation. When your function returns an error number, the system can look up the number, find the associated error abbreviation, and print out the corresponding error message.

Keep in mind that the abbreviations, such as E2BIG, are defined as integer (int) types. That means the computer sees E2BIG and thinks of it as a number, not a string of letters. Which number? The one you assign to it. Let's say you define E2BIG as being equivalent to the integer 23. Now when you have a function return the error E2BIG, the computer at compile time sees that as returning 23. That's what it compiles into. Then at run time, when that error occurs, a 23 gets returned from the function, the 23 gets looked up in the errno.h and the appropriate error message is displayed.

Had you defined E2BIG as being 218 instead, that would be fine, the same process would occur. The error abbreviations and messages are standards; the numbers assigned to the abbreviations are not and are part of the actual implementation.

The source code that makes up errno.h and the error numbers you assign as part of your implementation of errno.h in your particular program are part of the API. It's part of that near-English listing of words and numbers that humans can read. Once it gets complied to binary code, it forms part of the ABI that tells the computer how to actually run the program on your system.

11:53 PM EST

Copyright 2004