The NetBSD Foundation Press Release: NetBSD and the Google "Summer of Code" 2006 Summary
September 27, 2006 This report is about the results of NetBSD's participation in Google's "Summer of Code" 2006. After NetBSD was already part of the first GSoC last year, we had another chance to introduce a number of students to our operating system and get them paid by Google to work on projects defined by the NetBSD developers.
The NetBSD Foundation's Communication executive committee with the lead of Jan Schaumann and Hubert Feyrer took steering the event again, including organizing descriptions of possible projects, spawning discussions on the NetBSD mailing lists, reviewing of more than one hundred project proposals, assigning of mentoring NetBSD developers, setting up communication and development infrastructure on SourceForge as well as acting as a point of contact and communication for Google.
The projects and the students working on them were mentored by various NetBSD developers -- see below for for a list of projects that we were able to start, and the results stemming from the students' and mentors' efforts:
Summary and goal: The lack of journaling in the Berkeley Fast File System (FFS) and the result of lengthy file system checks if the system crashes is one of the less desirable features in NetBSD. As a result, high hopes were set into this project to add journaling to FFS to allow fast system startup in case of crashes. The project proposal was to do journaling on a file operations level, and replay the various operations in case of a crash.
Results: Let's tell the bad news first - this project didn't finish. After an initial design phase with some discussion about what the pros and cons of the various kinds of journaling are at about midterm, the student got the midterm payment, and further emails were not replied to. The project did not result in any code at all. Despite efforts of the mentoring NetBSD developers, only a rough outline of the intended work was proposed on the NetBSD mailing lists, with no further response or commits to the SourceForge CVS repository.
The lesson to learn from this project from the NetBSD side is to offer as much of a detailed design plan as possible, to make it easier for students to decide if they can implement a given proposal, and to be less generous about offering midterm payment if we're not really confident a project is enough on-track to reach the final(!) goal.
Summary and goal: NetBSD already runs on a number of 32- and 64bit machines, but on the MIPS platform it only supports 32bit machines so far. The goal of this project was to change this, and work on toolchain, cache, virtual memory and related userland changes that are necessary to get NetBSD working on 64bit MIPS machines.
Results: Unfortunately this is the second (of two) project that failed this summer -- the student had to go to hospital in Beijing during the first half of the project, and the code written by the student at the start of the project never got committed to the SourceForge CVS repository.
Six weeks after the end of the Summer of Code, the student was released from hospital and told us that he intends to work on the project. Given that the primary mentor moved on to other projects, a shift in the mentoring NetBSD developer will happen, and we are looking forward for progress.
Summary and goal: The goal of this project was to also add more hardware support for NetBSD, in this case the target was the G5 PowerPC CPU and peripheral hardware as found in Apple iMac G5, PowerMac G5 and Xserve machines. The original scope of the project as proposed by the student and agreed on by the NetBSD mentors was changed when, right at the start of the project, someone posted that they had basically done this project out of the sights of the open source community and was ready to contribute it. As a result, the project's goals were adjusted to still offer a win for NetBSD and meeting the student's goals for learning.
Results: After the initial change in direction, the project worked together with the party that did the initial port, and work was done do move from serial console to an OpenFirmware based console, debugging and optimisation in the cache and memory mapping code as well as in the serial driver to integrate it with NetBSD's machine-independent code framework. Yevgeny is currently in the process of getting a NetBSD developer account (AKA "a commit bit") to integrate his Summer-of-Code projects -- both from this year and from last year, to add HFS+ support and be able to boot from a Mac partition.
Summary and goal: In a multiuser/multitasking environment, many processes can issue read/write to a single disk at once, reaching a point where the underlying hardware cannot service requests fast enough and resulting in congestion. Effects of such congestion are unequal sharing of disk bandwidth between processes as well as unpredictable read and write delays. The goal of this project was to introduce an interface to the NetBSD virtual-memory system (UVM) that detects congestion and prevents it, as well as benchmarking the system and providing documentation.
Results: There are several possible approaches to the problem of detecting and preventing congestion, and the approach of observing a process' write behaviour and acting upon it was implemented within NetBSD's UVM system, and first benchmarks showed that that penalty of multiple writers was reduced from 42% to 15%. For the immediate future, the project will keep a status of "work in progress" to get better understanding of the dynamics of the page daemon and UVM, and to serve as framework for further research on how to detect and prevent congestion.
Developing student: Rui Paulo
Mentoring NetBSD developer: Kentaro A. Kurahone
Mentoring NetBSD developer: Allen Briggs
Mentoring NetBSD developer: Matt Thomas
Summary and goal: Explicit Congestion Notification as defined in RFC 3168 is one of several TCP congestion control algorithm to make delay-sensitive low-traffic applications perform smoother. It works by having routers indicating possible upcoming congestion situations, which allows to avoid dropped packets by having the sender adjusting the transmission window size.
Results: A working end-host ECN implementation was delivered by the student, along with test results that show the improved timing behaviour. The work was integrated into NetBSD already and will be part of NetBSD 5.0.
Developing student: Degroote Arnaud
Mentoring NetBSD developer: Sam Leffler
Mentoring NetBSD developer: Christos Zoulas
Mentoring NetBSD developer: Thor Lancelot Simon
Summary and goal: NetBSD currently has two IPsec implementations, ipsec(4) that came with KAME's IPv6 implementation but that lacks support for crypto hardware, and fast_ipsec(4) which supports crypto hardware but lacks support for IPv6. The goal of this project was to add IPv6 support to fast_ipsec(4).
Results: Both tunnel and transport mode for authentication (AH) and encrypted security payload (ESP) over IPv6 were added, compressed transport (IPcomp) was fixed to work with both IPv4 and IPv6, and TCP MD5 checksums work. We plan to integrate the work into NetBSD as soon as possible.
Developing student: Joerg Sonnenberger
Mentoring NetBSD developer: Dieter Baron
Mentoring NetBSD developer: Johnny C. Lam
Mentoring NetBSD developer: Alistair Crooks
Mentoring NetBSD developer: Thomas Klausner
Summary and goal: The core tools of pkgsrc, the NetBSD Packages Collection, have gone through a number of updates and maintenance cycles with added features on many sides, which led to the current state of the tools. The goal of this project is to re-collect all the requirements and perform a complete redesign and rewrite plus documentation.
Results: The student working on this project had previous experience from working on pkgsrc for NetBSD and DragonflyBSD, and it seems that the routine maintenance work he's conducting on these platforms turned out as a challenge for this project, keeping all the balls in the air and still proceeding. The results include a design plan with a new scheme for wildcard handling, partial code to add and create binary packages, but also a number of unresolved areas in the area of package registration that need further work. Given Joerg's involvement in the pkgsrc project we are looking forward to see him persue his ideas further.
The overall impression from this project was, similar to that of the Journaling for FFS (jffs) project, that too much time was needed to clean up or specify the design, which left too little time for implementation, testing and writing documentation.
Summary and goal: The mbuf functions and macros provide an easy and consistent way to handle a networking stack's memory management needs. A drawback of the current API is that it's easy to write code with incorrect assumptions, which will work in most but not all cases, which is leading to situations that are very hard to debug. Goal of the project was to improve the existing mbuf API, but in a way that was explicitly not adding new user-visible features, but to improve the current implementation and make the network stack more reliable.
Results: The mtod() macro was replaced by new mbuf accessor routines that encourage correct handing of noncontiguous mbuf chains and read-only mbuf storage and code correctness. Handling of read-only storage was implemented by using the "const" qualifier, using the compiler-provided checking. A start was made to convert existing code to the new API and constifying it in the process. As a result, the input path of IPv4 and IPv6 protocols is now safe with respect to read-only mbuf storage, another side-effect of this work could be zero-copy I/O in the NetBSD/Xen backend driver. Legacy protocols such as XNS, ISO or Appletalk were left as future work. These could also benefit from the documentation written during the project and published on the project's homepage, currently hosted on SourceForge. The project is expected to be soon integrated to the NetBSD mainline.
In summary, the results of this year's Google "Summer of Code" were a success for the NetBSD project as well as for the participating students. Even with two failed projects, the work achieved during this summer will have a noticable impact on the NetBSD project, its code structure, networking and platform support, even if they are not directly user visible in all cases.
Things we have learned as mentoring project is that we have to invest more work into details of project proposals, outlining design choices and offering decisions that would possibly delay projects if left open. Benefits for our (prospective) students and developers will be clearer guidance and more time for actual implementation, testing, debugging and last but not least documentation.
Another experience was that having a direct, interactive, and informal form of contact with the student -- and using it to just say, "how's it going?" if a mentor hadn't heard anything from a student for a couple of days -- made a big difference. It gives the student a chance to ask questions, explain the current issues, and for the mentor to work through some of the issues with the student in a fairly interactive fashion. Some of our mentors and student used AOL IM (via Adium and GAIM), which was sufficient.
Finally, we would like to thank Google for offering us this great opportunity and support, and Google's Leslie Hawthorn, Chris DiBona and Greg Stein as well as their whole support team working behind the scenes, for their organization and encouragement, all of the NetBSD developers who volunteered to act as mentors during the summer, and also the students who invested a lot of sweat blood and tears into their projects, and NetBSD. Thank you!
The NetBSD Foundation
Back to the NetBSD Foundation Inc. page