NetBSD in 2003 - A Report - Annual NetBSD Status Report - On February 7th, The NetBSD Foundation had its annual meeting, and here's a report on what happened in the past, what's in for the future, where we stand WRT the 2.0 release and many other things of general interest. This report gives a general overview on what happened in NetBSD in the past year, and then goes into details from each of the groups inside NetBSD to detail status of what was, is and will be. * General overview of happenings in the past NetBSD year First thing to note was the tenth anniversary year of the NetBSD Project. Over the past ten years, hundreds of people have come to develop a world-class freely redistributable operating system, NetBSD. Other groups have done the same, but NetBSD as a project was one of the first to try, and has done a good job. NetBSD started as a group of people who wanted to do something useful, and while most of the growth of NetBSD is spurred by similar-minded volunteers, NetBSD hasn't been limited to hobbyists. Companies have used NetBSD to build products, numerous engineers are employed building things with and based on NetBSD, and the NetBSD project helped save people a lot of time and effort, by both producing a good operating system, but also through tools like pkgsrc which help system administrators do their jobs more easily. Before going into detail on the various subgroups, let's mention a few achievements of the NetBSD Foundation from the past year. First is that we now have an active Board of Directors, elected by members of the Foundation, i.e. developers of the NetBSD project. Along with that, policies have been put to papers to create a better decision making process in the Foundation. There's also a new Core team in place, to guide the technical direction of the project even better than before. Another important thing to run the project is that the NetBSD Project has applied for US Internal Revenue Code 501(c)(3) tax-exempt non-profit status, and as of January 22nd 2004, the NetBSD Foundation is a 501(c)(3) tax-exempt organization, which will ease donations to the Foundation. Besides project financials the Foundation has also worked successfully towards registering the "NetBSD" trademark, and a similar undergoing for registering the "pkgsrc" trademark is in process to get appropriate recognition for the work done on the pkgsrc framework and it's now more than 4,400 packages. Great progress was made in the technical development arena, in particular the scheduler activations-based threads, the sysctl interface, better toolchain (compiler) support, Java, ports to new platforms, and a lot of work in pkgsrc, like pkgviews and buildlink. In the future, NetBSD will continue its tradition of technical excellence by working toward the upcoming release, NetBSD 2.0, complete with all the features that users are expecting, with multiprocessor support on common hardware and high-quality threading being the two most important. Second, it is an important point of the NetBSD project to increase the visibility of the operating system, of pkgsrc, and of the NetBSD Foundation. Everyone is welcome to publish high-quality information like articles, papers and other documentation, and participate in community events such as conferences, that help people learn about NetBSD. Getting people to know about NetBSD will help them build better products and do their jobs better and more efficiently. And it will get the NetBSD project more users, and more developers, which in turn helps NetBSD to grow and improve. Finally, problems that might harm NetBSD in the long term have to be addressed. Management of the NetBSD Foundation, continuing the high standard of technical leadership guiding NetBSD and pkgsrc development are key issues. Licensing is also important so vendors wishing to distribute NetBSD can comply more easily. The following sections will go into details for all the groups within the NetBSD project and their reporting of status. * The Board of The NetBSD Foundation Chris Demetriou reported about activities in NetBSD's project management, the NetBSD Board of Directors. The NetBSD Board of Directors gets its members elected for a period of two years. Two of the now-former board members then had their term finish at the date of the meeting, and two new members filled their position. Former board members that led the NetBSD project through the past were Scott Reynolds and Alistair Crooks, and with the new board members Tracy Di Marco White and Lex Wennmacher, the new board consists of (alphabetical order): - Chris G. Demetriou - Luke Mewburn, Vice President and Secretary - Lex Wennmacher, Treasurer - Tracy Di Marco White - Christos Zoulas, President and Assistant Treasurer In order to get better organization and better decision-making capabilities, Executive Committees (ECs) and Project Management Committees (PMCs) were formed. * Membership Executive Committee Lex Wennmacher gave the report on the project's "human resources" department. Notable things here includes a standardized membership application procedure, including a step where NetBSD developers are formally asked for comments to be considered before making a membership offer. New developers are now required to have PGP keys which are signed by at least one current NetBSD developer on a public key server. In the past year, 20 new membership applications were processed (and accepted). A list of "active members" is established for voting and elections, and accounts of inactive developers are currently being closed. Things currently being worked on are a new membership agreement form as well as documentation on PGP key management for developers. * Finance Executive Committee The Finance Committee maintains the financial records of the foundation, pays bills, accepts and documents donations, and files financial forms. The report was given by Christos Zoulas. The most important milestone, conversion of the Foundation to a 501(c)(3) publicly funded, non-profit organization was already mentioned above. In addition, the "NetBSD" trademark application has been published for opposition, and will soon be completed: http://tarr.uspto.gov/servlet/tarr?regser=serial&entry=78-025507 The financial status of the project is better than previous years thanks to many generous donations. In addition since the last annual meeting the number of donors increased significantly, while the amount of individual donation amounts dropped. This is a desirable change because it asserts that the NetBSD project can maintain a certain level of donations without depending on large individual contributions. Expenses last year (2003) were: - Corporate fees - Taxes - CVS server purchase - RealWeasel purchase - 501(c)(3) application form Plans for this year (2004) include purchase of: - four 1U servers - SSL certificates Everyone is welcome to keep donations coming. Donations are accepted via check or paypal. See http://www.NetBSD.org/donations/ for details. * Servers and Service Administration Tracy Di Marco White works in the team of administrators who "establish guidance over and are responsible for the operation of the services and servers used to develop and distribute the products of the Foundation." Other projects' security problems caused a reevaluation of the architecture of the NetBSD project's services. Among other things, SSH protocol 1 access as well as password based access are now disabled on all machines, and shell access to the CVS server is restricted. Software like cvs, ssh and others was upgraded numerous times to keep them up-to-date and avoid security problems. The machines of the release engineering group were moved from New York City to the machine rack kindly provided by the Internet Systems Consortium (ISC) which gives them a lot more bandwidth. A new and faster CVS server was bought, configured and installed at ISC too, and the Mail server was moved from Redback Networks to the ISC as well. There's also progress towards a standardized machine configuration based on the latest stable release of our fine operating system. Call duties for admins is now handled by a rotation scheme, with one admin being "on call" for a week. Plans for this year include a new console server, web, mail and admin server, moving to a ticket tracking system and getting more people involved. * NetBSD Communication and Publicity Committee Luke Mewburn talked about the Communications EC. Activities in the past year included updates on the web site about donations and the NetBSD Foundation, as well as setup of regional mailing lists, see the regional-* lists at http://www.NetBSD.org/gallery/groups.html#regionallists. A logo design contest was announced with more than 100 submissions three weeks before the competition closes - outcome of this will be made public when the competition is closed and a new logo has been chosen. * The WWW Group Jan Schaumann talked about the "WWW" group, which maintains the NetBSD project's web-presence, but also acts as the first point of contact for many users by performing what might qualify as ``customer-service'' or ``tech-support'' by answering every incoming mail to www@NetBSD.org quickly, politely and of course accurately. Many of the goals established in last year's meeting were achieved, among them are better internal documentation, updating a wide variety of documentation (e.g. building), and moving towards a better documentation framework based on XML. Ownership of the NetBSD Guide was taken and TNF licensed as well as getting it up to date - several chapters have undergone major work and corrections, several chapters have been written from scratch and been added to the NetBSD Guide. Initiative and leading this project was taken by Jason R. Fink. For the coming year, the plan is to provide more accurate information for mirror maintainers (esp. for anoncvs), simplifying maintenance of ports-pages for port-master, XMLizing all documents, bringing translations of the NetBSD Guide up to date as well as improving communication with other teams. Other issues that need addressing are the Projects Server (http://projects.NetBSD.org), fine-tuning CVS access, improving the mailing list archives, making public keys for developers (PGP) and machines and mirrors (SSH) available online, working more tightly with developers and other teams for press releases and news announcements as well as general advocacy by making printed documentation available for trade shows, e.g. via some web shop. * Committee for Technical Development Alistair Crooks gave this introduction to how technical development is managed. The committee for technical development is the board committee which oversees the technical development of the project. There are four Project Management Committees, or PMCs, which tech-exec oversees. These are (in no special order except importance) - pkgsrc - security-officer - release engineering - core ** Report of the pkgsrc Project Management Committee The pkgsrc PMC is the Project Management Committee which oversees the development of pkgsrc, NetBSD's third-party packages system. In the last year, pkgsrc gained 919 new packages, and many packages were updated numerous times. Over the last year, we have seen two branches in pkgsrc. A branch is created differently to that of the other modules in the CVS repository, since pkgsrc's needs are slightly different to the others, so the pkgsrc team just freezes the trunk, fix things, and then branch when the time is right. Branch names are netbsd-1-6-1 for the 1.6.1 release as well as netbsd-2003Q4 branch, which will be included in 1.6.2. Much work has taken place on porting pkgsrc to other platforms. The current list of platforms supported (to a greater or lesser degree) AIX, BSD/OS, Darwin, FreeBSD, IRIX, Linux, NetBSD, OpenBSD, Solaris. There are also patches in gnats for the Hurd and HP/UX. Preliminary patches may or may not be available for Digital Unix. Plans for the future include the aim to branch pkgsrc every 3 months. The reasons for this are ease of branch maintenance, branches which are up to date, and binary package management. Support will be given for trunk and last branch of pkgsrc. Procedure for the branches will be a two-week (maximum) freeze period for pkgsrc, after that time it's branched and "shipped". Plans for the next year are: - branch pkgsrc-2004Q1 in early March 2004, which will include some buildlink3, libtool 1.6, kde 3.2, gnome 2.4 etc. as well as self-hosting pkgsrc (i.e. integration of bootstrap-pkgsrc). - branching pkgsrc-2004Q2 in early June 2004, with complete buildlink3 - branching pkgsrc-2004Q3 in early September 2004 to include pkgviews - branching pkgsrc-2004Q4 in early December 2004 Work in progress include buildlink3 by Johnny Lam, including a developer's guide published to tech-pkg@ with a more up-to-date version available in pkgsrc/mk/buildlink3/BUILDLINK3_DG. Pkgviews is still an experimental addition to pkgsrc and is not yet supported. There is a user's guide published to tech-pkg@, and a more up-to-date version is available at pkgsrc/mk/buildlink3/PKGVIEWS_UG. Bootstrap is a project to merge bootstrap-pkgsrc into pkgsrc itself so that the pkgsrc tree is in some sense "self-hosting", meaning that users on all platforms will only need to download a single tarball to get up and running. This is an active project being developed by a small working group of developers who have just finished the design phase. The implementation is slated for completion by the 2004Q1 branch-point. There are several other items worth mentioning. The "pkgsrc" trademark was already mentioned above. Regular bulk build runs through all pkgs in pkgsrc are done, which take 5-6 days for a full build on fast i386 machines right now. The builds are done on the latest stable OS release as well as on NetBSD-current to identify and eliminate problems early. pkgsrc-wip was invoked by Thomas Klausner as a SourceForge project where people (usually those without NetBSD developer accounts) can commit packages easily, for review and later moving into pkgsrc. See http://pkgsrc-wip.sourceforge.net/ for more information. ** Security-Officer Team David Maxwell spoke for the NetBSD Security-Officer Team. First David gave an overview of their work areas, which are both proactive and reactive. Proactive measures include development of tools for handling reactive part of the job, encouraging good security practices by developers, auditing sources for additional instances of reported problems as well as encouraging availability of good security tools in NetBSD. List of things the security team reacts to is quite long and includes handling mail to the security-officer mail address, with a target response time of less than 24 hours, investigation of vulnerability reports coordinated with 3rd parties like CERT, FreeBSD security-officers and individual security researchers. Problems in NetBSD are coordinated with experts on the subject, who do analysis and resolve flaws. Other teams to coordinate with are admins to keep the NetBSD project machines' security up to date as well as the release engineering crew for pulling possible security changes into release branches. Ongoing research and discussions are followed, and queries to the tech-security mailing list are responded to. Problems addressed are documented publicly by publishing security advisories. Goals from last year that we've met include binary patches for advisories, re-organizing the Security Advisory publishing process, getting PGP keys for all developers, keeping security-related webpages up to date, improve mail response time and advisory publishing time and improving internal project tracking and communications. Goals for this year include more binary patches for a larger number of Security Advisories, add tools to track responsiveness of the security officers and for tracking ongoing issues, publish host and PGP keys, sign releases, issuing quick "security notes" like 'NetBSD is not affected' on issues which won't result in a Security Advisory (SA), working toward secure infrastructure for real-time communication between developers as well as recruiting more volunteers to help handle non-confidential issues, and contribute time to handle less critical flaws which are public and not yet SA'd and to followup with CERT and provide NetBSD references for older CERT issues. One of the big problems this year was that, in many cases, the security team received late (< 24h) or no notification before information about the security issue was made public. Possible areas where technical development for security is possible: - Suggest and perform security sweeps for your pet issues. - Review PRs in security category and send status updates - Generate example systrace policies for system daemons. - Write rc tweaks to run more daemons unpriv'd/chroot/etc. - Stomp out more suid programs. - Write release signature/verification tools. - Find a nice solution to the "local mail must work" problem without having sendmail or postfix listening outside. - Bring your PGP fingerprint on a piece of paper to NetBSD social events so people can sign it later! ** Release Engineering Erik Berls gave the report for the team handling releases of NetBSD. At the time of the presentation, the team was busy with preparing the NetBSD 1.6.2 release, with much of the work done by James Chacon. Major achievements last year: - NetBSD 1.6.1 released - The netbsd-1-5 branch is still in maintenance mode. - The team is in the final stages of kicking 1.6.2 out the door - the releng team took on maintaining, doing pullups for the pkgsrc branches - Improved autobuild - Releng machines were moved to a better location (in conjunction with admins.) Plans for this year: - Kick 2.0 out the door. - Kick 1.6.3 out the door. - Refine policy. - Improve autobuild some more. - Add more autobuild workhorses (in conjunction with admins.) Things we planned to do, but were not able to complete are adding additional req queues for 2.0 and pkgsrc, and the autobuild mechanism to (cross)compile NetBSD for all platforms on a daily base could be tuned more. ** NetBSD OS 'Core' development team The Core team traditionally managed all technical development of NetBSD. With the new structure of the NetBSD Foundation in place, overseeing pkgsrc, release engineering and security issues is now handled to separate groups, and "Core" is now back to managing technical development of the NetBSD operating system itself. Luke Mewburn gave the report of the Core group. Core is now a project management committee (PMC) under technical-exec, for the maintenance of the core "open-source" operating system software products. Some technical highlights: - Build.sh overhaul - Cross-buildable X11 (integrated into build.sh) - Toolchain update; gcc 3.3.2, gdb 5.3 - Dynamic sysctl - Thread improvements - Many programs updated - Storage: smbfs, FFSv2, vinum - Improved buffer cache memory allocation - Non executable mappings on many platforms - many others Things that need to be addressed for the NetBSD 2.0 release include: - GNATS audit with the goal of having no open PRs in the high or medium states going into a release cycle - gcc 3.3.2 for sh3, sh5, vax - gdb 5.3 for hppa, ns32k, sh5 - sigtramp: Gdb goes not handle the new signal trampoline so w cannot debug through signal handlers - Longjmp uses deprecated __sigreturn14. Programs that use longjmp will not work unless COMPAT_16 is defined. To fix: hppa, m68k, mips, sh5, sparc, sparc64, vax. - siginfo: Not supported yet on hppa, mips, ns32k, sh3. - and whatever releng needs * Closing This report gives an overview of past, present and future of the NetBSD Project, the NetBSD operating system, pkgsrc and the NetBSD Foundation both in general and from the perspective of each group, to give users and people interested in the NetBSD project insight into the project. Please join our mailing lists for participating in ongoing discussion, and see our web site for more information about the NetBSD project, http://www.NetBSD.org. - Hubert Feyrer Communications Executive Committee The NetBSD Foundation