The Anykernel and Rump Kernels
A driver abstracts an underlying entity. For example, a TCP/IP driver abstracts the details required to perform networking, the Fast File System (FFS) driver abstracts how the file system blocks are laid out on the storage medium, and a PCI network card driver abstracts how to access device registers to send and receive packets. The kernel architecture controls how kernel drivers run with respect to other system components. Some examples of kernel architectures are the monolithic kernel, microkernel and exokernel. In contrast to the above architectures, NetBSD is an anykernel. This means that some kernel drivers can be run according to any kernel architecture instead of being limited to a single one.
When a driver is not run as part of the monolithic kernel, e.g. when it is run as a microkernel style server, the driver is hosted in a rump kernel. A rump kernel is an ultralightweight virtualized kernel running on top of high-level hypercall interface. Instead of a low-level hypercall API typically seen with operating systems with operations such as "modify page table", the rump kernel hypercall API provides high-level operations such as "run this code in a thread".
Currently, three implementations of the rump kernel hypercall interface exist.
- The POSIX implementation is included in the NetBSD tree and makes rump kernels to run as userspace processes on most operating systems such as NetBSD, Linux and Solaris.
- The Xen implementation allows running rump kernels directly as Xen DomU's without an intermediate operating system.
- The Linux kernel hypervisor allows rump kernels to run inside the Linux kernel.
Rump kernels are radically different from OS virtualization technologies such as KVM, containers and usermode operating systems. A rump kernel does not support hosting application processes because a rump kernel is aimed at virtualizing kernel drivers and application virtualization would be pure overhead. Instead, existing entities such as processes from a hosting OS are used as clients for the rump kernel ("application" in the figure).
As a result of the above design choices, rump kernels are extremely lightweight. The bootstrap time for rump kernels on POSIX hosts is measured in milliseconds and memory footprint in 100kB's. This means that a rump kernel can be bootstrapped for example as part of a command line tool for virtually no cost or user impact. Rump kernels also mandate very little from the hypercall implementation meaning that rump kernels, and by extension NetBSD kernel drivers, can be hosted in virtually any environment.
Use cases for rump kernels include:
- Code reuse: kernel drivers can be reused without having to run a whole OS. For example, a full-featured TCP/IP stack (IPv6, IPSec, etc.) can be included in an embedded appliance without having to write the stack from scratch or waste resources on running an entire OS.
- Kernel driver virtualization: every rump kernel has its own state. Furthermore, the functionality offered by multiple rump kernels running on the same host does not need to be equal. For example, multiple different networking stacks optimized for different purposes are possible.
- Security: when hosted on a POSIX system, a rump kernel runs in its own instance of a userspace process. For example, it is widely published that file system drivers are vulnerable to untrusted file system images. Unlike on other general purpose operating systems, on NetBSD it is possible to mount untrusted file systems, such as those on a USB stick, in an isolated server with the kernel file system driver. This isolates attacks and prevents kernel compromises while not requiring to maintain separate userspace implementations of the file system drivers or use other resource-intensive approaches such as virtual machines.
- Easy prototyping and development: kernel code can be developed as a normal userspace application. Once development is finished, the code can simply be complied into the kernel. This is a much more convenient and straightforward approach to kernel development than the use of virtual machines.
- Safe testing: kernel code can be tested in userspace on any host without risk of the test host being affected. Again, virtual machines are not required.
Further Reading (top)
The following is the definitive guide to the anykernel and rump kernels and supercedes all earlier publications and terminology on the subject.
These links are interesting for people who want to use rump kernels in addition to reading about them.
- Scripts for building rump kernels for POSIX systems
- Rump kernel hypercall implementation for Xen; rump kernels as Xen DomU's
- fs-utils: File system image access utilities
- Fast userspace packet processing: TCP/IP stack for use with DPDK or netmap
- Running rump kernels and applications on Xen without a full OS
- PCI device driver support in rump kernels on Xen
- Experiment with a rump kernel hypervisor for the Linux kernel (allows rump kernels to run in the Linux kernel)
- Kernel Servers using Rump
- Tutorial On Rump Kernel Servers and Clients
- Revolutionizing Kernel Development: Testing With Rump
- "The Anykernel and Rump Kernels" gives a general overview and demonstrates rump kernels on Windows and in Firefox. The video, slides and an interview are available. Presented at FOSDEM 2013 (Operating Systems track).
- "Rump Device Drivers: Shine On You Kernel Diamond" describes device driver and USB. The paper and video presentation are available. Presented at AsiaBSDCon 2010.
- "Fs-utils: File Systems Access Tools for Userland" describes fs-utils, an mtools-like utility kit which uses rump kernel file systems as a backend. The paper is available. Presented at EuroBSDCon 2009.
- "Rump File Systems: Kernel Code Reborn" describes kernel file system code and its uses in userspace. The paper and slides are available. Presented at the 2009 USENIX Annual Technical Conference.
- "Kernel Development in Userspace - The Rump Approach" describes doing kernel development with rump kernels. The paper and slides are available. Presented at BSDCan 2009.
- "Environmental Independence: BSD Kernel TCP/IP in Userspace" describes networking in rump kernels. The paper and video presentation are available. Presented at AsiaBSDCon 2009.
The manpages provide the usual type of information. Start from rump.3 and follow the cross-references in "SEE ALSO".
Any topic related to rump kernels can be discussed on the rumpkernel-users mailing list. Alternatively, you can use a NetBSD mailing which is related to a specific subtopic.
The IRC channel for rump kernels is #rumpkernel on irc.freenode.net.
The anykernel and rump kernels were first introduced as a prototype in NetBSD 5.0. A stable version with numerous new features and improvements was shipped with NetBSD 6.0.
Source Code (top)
All of the source code is available from the NetBSD source tree and can be obtained with the usual methods.