PiP - Process-in-Process
pip Overview of Process-in-Process (PiP)

Overview

PiP is a user-level library which allows a process to create sub-processes into the same virtual address space where the parent process runs. The parent process and sub-processes share the same address space, however, each process has its own variables. So, each process runs independently from the other process. If some or all processes agreed, then data own by a process can be accessed by the other processes.

Those processes share the same address space, just like pthreads, and each process has its own variables like a process. The parent process is called PiP process and sub-processes are called PiP task since it has the best of the both worlds of processes and pthreads.

PiP root can spawn one or more number of PiP tasks. The PiP root and PiP tasks shared the same address space. The executable of the PiP task must be compiled (with the "-fpie" compile option) and linked (with the "-pie" linker option) as PIE (Position Independent Executable).

When a PiP root or PiP task wants to be accessed the its own data by the other(s), firstly a memory region where the data to be accessed are located must be exported. Then the exported memory region is imported so that the exported and imported data can be accessed. The PiP library supports the functions to export and import the memory region to be accessible.

Execution mode

There are several PiP implementation modes which can be selected at the runtime. These implementations can be categorized into two according to the behavior of PiP tasks,

In the pthread mode, although each PiP task has its own variables unlike thread, PiP task behaves more like Pthread, having no PID, having the same file descriptor space, having the same signal delivery semantics as Pthread does, and so on.

In the process mode, PiP task behaves more like a process, having a PID, having an independent file descriptor space, having the same signal delivery semantics as Linux process does, and so on.

When the PIP_MODE environment variable set to "thread" then the PiP library runs based on the pthread mode, and it is set to "process" then it runs with the process mode. There are also two implementations in the process mode; "process:preload" and "process:pipclone" The former one must be with the LD_PRELOAD environment variable setting so that the clone() system call wrapper can work with. The latter one can only be specified with the PIP-patched glibc library (see below: GLIBC issues).

There several function provided by the PiP library to absorb the difference due to the execution mode

Limitation

PiP allows PiP root and PiP tasks to share the data, so the function pointer can be passed to the others. However, jumping into the code owned by the other may not work properly for some reasons.

Compile and Link User programs

The PiP root ust be linked with the PiP library and libpthread. The programs able to run as a PiP task must be compiled with the "-fpie" compile option and the "-pie -rdynamic" linker options.

GLIBC issues

The PiP library is implemented at the user-level, i.e. no need of kernel patches nor kernel modules. Due to the novel usage of combining dlmopn() GLIBC function and clone() systemcall, there are some issues found in the GLIBC. To avoid this issues, PiP users may have the patched GLIBC provided by the PiP development team.

GDB issue

Currently gdb debugger only works with the PiP root. PiP-aware GDB will be provided soon.

Author

Atsushi Hori (RIKEN, Japan) ahori@riken.jp

 All Files Functions