NetBSD Binary Emulation

Binary Emulation


Binary Emulation

What is Binary Emulation? Why does it exist?

A large amount of Unix software is distributed in source-code format. This means the authors actually provide the program code, and the installation process uses a compiler to generate an executable to run on the local machine. Carefully written source-code and installation utilities can allow the same program to be built on dozens of different operating systems.

Commercial software vendors are not inclined to distribute source code since it may contain trade secrets. Commercial vendors normally deliver the executable programs which can be run directly. They perform the compilation stage in-house, and delivery binary files from which their secrets are less easily discernible.

The result of this is that the vendor must make a choice to expend man-power for each different operating system they support, normally maintaining a system to do testing with, and at least one person to do compilation and testing.

This ties together the Operating System and the set of applications a consumer may wish to run. One may choose not to run a particular application because it is not available on their Operating System of choice, or one may be forced to run an Operating System one would rather not, due to the availability of some critical application.

Binary Emulation eliminates this forced linkage.

What other binary compatibility is there?

There is another compatibility feature worth being aware of. NetBSD uses the same binary object file format across NetBSD machines with the same CPU. If you build binaries on your NetBSD sun3, you can run them on your NetBSD amiga, NetBSD mac68k, NetBSD mvme68k, NetBSD next68k... however, you couldn't run them on Intel processor based machines.

How does it work?

Unix and Unix-like systems consist of two primary parts, the Kernel, and everything else. The kernel is the program which controls devices, security, and the programs which wish to use the machine's resources. Typically, the kernel provides these services to other programs through kernel system calls. An example would be a program requesting to OPEN a file, the program calls the kernel OPEN function with a set of parameters indicating what it wishes to do, and the kernel allows or denies the request, and replies with the information the program requires to continue.

Every Unix and Unix-like system supplies a very similar set of these system calls. They all have an OPEN for example.

From system to system, the primary differences in syscalls will be in the format of parameters passed to these calls. The names of the calls may also differ from system to system. If a NetBSD system wishes to run a Linux executable, each time the program performs a system call, the kernel performs a mapping function to the corresponding NetBSD system call, and re-orders/re-formats the parameters as required.

Another important issue is the format of the executable files. About every second operating system uses a different file format in which it saves its binaries, using different headers, magic cookies, hunks, whatever. The ones NetBSD supports natively are ELF and a.out. NetBSD's emulation knows how to handle the executable format for the emulated system.

Across different systems, syscalls are accessed in different ways, and parameters are passed differently. As an example, under AmigaOS, this is done with an index into a table containing pointers to functions pointed to by some address register and the data in some data register. Other operating systems use other methods, traps, tables, registers, stacks, etc. and this is handled by the binary emulation.

The final significant requirement is that the CPU the executable was compiled for must match the system it will run on. Besides system calls, executables consist of raw CPU instructions. Therefore, SCO Unix binaries (Intel 386 series CPU) will run on NetBSD i386, but can not be run on NetBSD Amiga (Motorola 680x0 CPU). Performing that function would require a much more complicated translation system, and would have a significant impact on application performance.

How well does it perform?

Since the only additional overhead is the mapping from emulated system calls to native NetBSD system calls, and the reformatting of any parameters, if needed, the performance is really, really good. A rough estimate would be at most a 1-2% performance impact. This varies depending on which system calls a program uses. Most mappings take <1% of the time the actual syscall takes to run.

Any other considerations?

In addition to the CPU being of the same type, and the mapping of system calls, there is one other requirement. Many Unix systems support shared libraries. This means that a compiled program does not come with all of its functions compiled in, but it requires an external set of libraries which must match the ones the program was compiled to use (not including minor modifications). If you wish to run a program under binary emulation, you can check whether it was statically or dynamically linked, by using the 'file' command...

% file qwsv
qwsv: BSD/OS i386 compact demand paged executable
% file arp
arp: NetBSD/i386 demand paged dynamically linked executable

The presence of 'dynamically linked' indicates exactly that, its absence indicates static linking. Shared object libraries for most freely available Unix systems are available from the NetBSD pkgsrc, under the /compat directory. Note that these shared library sets are _not required_ if you are only going to run statically linked binaries.

For commercial systems, you may need to supply your own set of libraries. See man -k compat for a list, and man compat_os (where os is the target OS) for some installation instructions:

% man -k compat
compat_freebsd(8) - setup procedure for running FreeBSD binaries
compat_linux(8) - setup procedure for running Linux binaries
compat_sunos(8) - setup procedure for m68k and sparc architectures
compat_ultrix(8) - setup procedure for Ultrix compatibility on mips and vax

Which OSs can I emulate on my machine?

NetBSD supports emulation of several systems. Generally speaking, you can run binaries from other unix operating systems which run on the same hardware as your NetBSD system.

aarch64
  • NetBSD 32bit
alpha
  • Linux (Alpha)
amd64
  • Linux 64bit
  • Linux 32bit
  • NetBSD 32bit
amiga
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
atari
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
hp300
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
i386
  • FreeBSD 32bit
  • Linux 32bit
mac68k
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
mvme68k
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
news68k
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
next68k
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
sgimips
  • Linux (mips)
  • Ultrix (mips)
sparc
  • SunOS (sparc)
sparc64
  • 32-bit NetBSD/sparc (both ELF and a.out)
  • SunOS (sparc)
sun3
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)
vax
  • Ultrix
x68k
  • Amiga Unix (SVR4)
  • Linux (m68k)
  • SunOS (68k)

Back to NetBSD Documentation