If you would like to share your NetBSD experience please contact us.
- NetBSD and my life
- NetBSD/alpha on DEC 3000/300 (Tobias Ernst)
- NetBSD/sparc on an IPX (McMahill)
- Canadian Special Olympics 2000 Winter Games
- NetBSD/alpha scratch space server and portability (Seebs)
- NetBSD/alpha file and other servers (Isildur)
- Research and products at Effnet (Anders Magnusson)
- NetBSD at ASIM/LIP6 (Manuel Bouyer)
- Running a dedicated NetBSD Game Server (Half-Life/Counter Strike and Quake3)
- NeverWinter Nights Server on NetBSD under Linux emulation
- NetBSD and my life... (Gary Rolland)
NetBSD and my life (top)
This is a testimonial by Stephen Borrill on how NetBSD helped him in his professional life and on the products that he created based on it.
Back in the pre-1.1 days (1993?), I was doing a PhD and I was introduced to Unix for the first time (using Masscomp machines - anyone remember those?). It blew away Windows (or DOS as it was mainly then) and every other OS I'd used. I had an Acorn RiscPC and I wanted to run some form of Unix-alike on it. RiscBSD (which later became NetBSD/arm32) was very early on and not part of the NetBSD tree, but was the only option (interestingly RISCiX was a 4.3BSD OS written by Acorn for their older machines, but it wouldn't run on the RiscPC). Armed with about 40 floppies, I trudged elsewhere in the uni and downloaded it (a slow job!). Got it installed and I never looked back.
After the PhD, I decided that academia wasn't for me and so got a job at Acorn in tech support. The company then merged with Apple to become Acorn and Apple's sole provider to education in the UK. A new client came out which booted using bootp and NFS with no local storage. I took it and got it to boot from NetBSD. The company ran with this and we launched in January 1997 (using 1.2G IIRC). We sold these systems to schools all over the UK (so Apple UK were effectively selling BSD boxes into schools years before Mac OS X). In 1999, Apple bought out the whole company and anything non-Apple got dumped. I took all the IPR and set up my own company to continue development. We rebranded it and moved to Intel hardware. NetBSD's single source tree and cross-platform ability meant this was very easy). We expanded the services it provided and to this day continue to sell Internet/security servers to schools throughout the UK (In fact, I'm currently typing this on such a machine out on site while their Windows server rebuilds after a crash). See http://www.netmanager.info/.
We are also heavily into Citrix and thin-client computing. We sell a product based on NetBSD which converts pretty much any old PC into a centrally-managed thin-client. This is installed in the hundreds at some schools. We are also due to launch our own range of thin-client hardware at the end of the month which is, once again, NetBSD based (this time running 3.0_BETA and waiting for a full release!). http://www.thinit.info/.
I like to think that in our small way, we are introducing people to NetBSD (even if they don't see much of it). Many tens of thousands of pupils and teachers throughout the UK use NetBSD every day because of us.
NetBSD/alpha on DEC 3000/300 (Tobias Ernst) (top)
We are running NetBSD/alpha on a DEC 3000/300 used as a Webserver for the homepages of about 300 students and as CVS server for several open source projects. It runs rock solid and performs exceptionally well even on this elderly hardware. Support from the NetBSD team in the rare case of any problem is fast and very helpful.
[Tobias Ernst, Physics CIP Pool, University of Stuttgart, Germany]
NetBSD/sparc on an IPX (McMahill) (top)
"Well, my IPX has been happily running 1.4.1 since it came out. It runs a low traffic web server, a samba server, plus I sit at it for 8-15 hours a day with 25 windows open, play CD's with xmcd, run a scsi scanner...
"So, I guess that's a glowing recommendation for NetBSD/sparc"
Canadian Special Olympics 2000 Winter Games (top)
The Canadian Special Olympics 2000 Winter Games used several NetBSD servers to provide connectivity, file storage, and backup. At the main office, a NetBSD server provided shared and secure Internet dial service for up to 20 machines, as well as file storage and automated off-site backups from June 1999. As the Games approached, several other NetBSD servers were set up on lan's at the various event venues to provide Internet service for event officials, the media, and the athletes. These servers allow officials to post results to the web site immediately after the results are finalized, media to communicate stories and results to their offices, and athletes to send email back home and check out the results.
The CSO 2000 Winter Games ran from January 25th to January 29th, 2000 in Ottawa, Ontario.
NetBSD/alpha scratch space server and portability (Seebs) (top)
"I use NetBSD/alpha with a PCI Ultra-DMA controller, and a 100Mbps network card, as the scratch space server for my personal network. I also use it to test so-called `portable' programs against a system where `long' isn't exactly 32-bits. The system is fast and stable, and it builds from the same exact source tree (shared over NFS) as my i386 systems."
NetBSD/alpha file and other servers (Isildur) (top)
"I use NetBSD/alpha both at home and at work. I have two file servers (on alphastation 200's) running NetBSD for over a year and a half now, and one system for software development and testing, as well as occasional number/huge data file crunching. I just upgraded to 1.4.1 after a very long time with 1.3, and I've had nothing but smooth uninterrupted operation with both! never a crash, never an instability, it has been great. I also run it on a DEC3000/600 and a DEC3000/300XL at home for software development, and have nothing but good things to say about it. its stable and fast!
"The file servers serve a mix of win9 and NT clients using
www/apache, with pretty heavy
use. I've never had any downtime on the machines (the netbsd
boxen, that is ;) aside from power failures."
Research and products at Effnet (Anders Magnusson) (top)
Effnet uses NetBSD as its main research platform and also bases many of its products on NetBSD. Effnet is a router and firewall manufacturer with most of its development in Luleå in Sweden, with sales offices in Stockholm, Sweden and Boston, MA. Effnet have designed the world's fastest firewall.
Effnet's primary reasons for selecting NetBSD were:
- Well-functioning IP stack.
- Full source availability.
- Confidence in developer knowledge.
- Available for more platforms than x86 (such as Strongarm).
- Ability to keep certain locally developed source private. (No GPL).
NetBSD at ASIM/LIP6 (Manuel Bouyer) (top)
Manuel Bouyer, who works at the Architecture des Systèmes intégrés et Micro électronique of LIP6, explains how NetBSD is used in a mixed network:
NetBSD performs quite well in mixed environments. The network here is built around NetBSD and a few Solaris 7 servers for windows (95, 98, NT4 and 2000), Linux, SunOS4, Solaris 7 and a few Mac OS 9 clients. The local network is built around cisco 295x and 35xx (including a 3508G 8 gigabit ports), with 4 vlans. Some servers are using IEEE 802.1q tags to be connected to multiples vlans from one physical interface.
The main fileserver is an Alpha DS20 running NetBSD 1.6_STABLE serving NFS (v2 and v3) and smb requests through a gigabit ethernet. It has 30 SCSI disks spread over 6 scsi busses and can handle more than 1000 NFS operations per seconds, serving about 150 clients. During our tests we could have 3 or 4 clients doing NFS or smb access at almost full speed over a 100Mbs interface, without noticeable slowdown for the others clients.
All Unix stations and shell servers are part of a Nis domain. The master is a NetBSD/i386 system, with 2 slaves (a Solaris 7 and a NetBSD/i386). All windows NT4 and 2000 machines are in a windows domain. The domain controller is a NetBSD/i386 server running the samba package, which also handles authentification for other samba servers. The profiles are stored on the DS20 fileserver.
For users using a windows desktop and to provide powerful machines to users we have a few shell servers: 6 dual-PIII/1ghz boxes running Linux, one running NetBSD/i386 (1.6_STABLE+unofficial SMP patches), a Sun E420 (quad-CPU) running Solaris 7, an alphastation 600 running NetBSD, and a Sun SS10 running SunOS4. For users using a Unix desktop, we provide access to a windows 2000 terminal server.
All printing goes through a centralized print server. Clients can print via cups, lpd, netbios/smb or netatalk. The print queues themselves are managed via the cups server.
We have 2 NetBSD/i386 boxes acting as router: one routing between internal networks (with a gigabit ethernet connected to the different VLANs via 802.1q), one acting as a filtering router (using IPF) beteen the internet, local networks and subnets dedicated to the public servers (mail/web/ftp and the ssh gateways), connected to 5 different networks. The "internal router" is also a NFS/smb/appletalk translator (for, among others, the public server which is running NFS only), the windows domain controller, NIS slave server, dhcp server and provides services to do unattended installs of Windows, Linux or Solaris workstations over the network. The filtering router isn't running any other services, for security reasons.
The filtering router has, in addition to some 10/100 ethernet interfaces, a dual-port gigabit interface. It is able to route a TCP flow (between 2 gigabit-connected NetBSD boxes) at 20700KB/s with ipfilter disabled, and 19750KB/s with ipfiler enabled and 660 rules loaded (31500KB/s between the 2 NetBSD boxes without router in the path). CPU is only used at 40% on the router, so the slowdown is most likely related to latency. Unfortunably my setup doesn't easily allow testing with several 100Mbs flows from different hosts though the gogabit router.
A NetBSD/i386 box is running public services: mail (sendmail as MTA, Procmail for delivery, which allows to run some spam filtering rules, and the procmail sanitizer which defangs attachements, to protect windows boxes from some viruses; NFS, pop3, imap for MUAs, including horde and squirrelmail web mail agents), web (apache as server, with php4 for mysql as backend storage for dynamic contents), anonymous cvs and ftp. It is connected to the filtering router via a gigabit ethernet.
To give ssh access to the local network from the internet we have 3 hightly-secure boxes. These are running NetBSD/sparc, with a highly customized setup: system disks are read-only (physically, via a switch connected to the write enable jumper), log files are append-only (via system flags and kernel security level), users logging in from network are in a restricted, chrooted environnement. Users can't execute anything else than system binaries by the use of the noexec mount flag on all partitions which are not on the read-only disk. All users commands are logged through accounting. In 6 years this setup has never been breaken, and unauthorized accesses (from stolen passwords) have been detected, with the affected accounts and source IP addresses. The downside is that all administration tasks have to be done from console, with the server in single-user mode.
Network monitoring is done from a dual-PIII box running NetBSD 1.6_STABLE + unofficial SMP patches. It monitors all the ethernet switches using mrtg and cricket. It catches ethernet address changes with the arpwatch tool. It also manages the serial consoles of all the non-i386 boxes around with the rtty tool (using PCI 8-ports serial adapters).
All UPSes in the machine room are managed by NetBSD boxes via the apcupsd software. UPS paramters are recorded using mtrg. This also allows to cleanly shut down the servers in case of power failure (which unfortunably happens quite often on this site), and bring services back up without manual intervention when power is back. As the UPSes provide temperature informations, a custom script can deliver alarms in case of overheat, and eventually shut down and power off the machine room if noone is there to take care of the problem. Backups are done by a NetBSD/i386 box (also acting as the master NIS server) driving a 15-slot DLT library, using the amanda package.
In this photo of the server room, you can see some of the servers described: on the first floor, on the left, we have the Alpha DS20, with 4 of its 6 external SCSI enclosures and the associated UPS. Next to it we have the alphastation 600 shell server, and the dual PIII/1Ghz monitoring the network and acting as a serial console concentrator, On the second floor we have a Sun E420 (shell server, and slave NIS server). In the rack (on the right of the photo), from bottom to top: 2 Linux shell servers, the Windows 2000 terminal server (all 3 are dual PIII/1Ghz), the NetBSD/i386 web/ftp/mail server, the NetBSD/i386 filtering router, and the NetBSD/i386 routing internal networks (and also NFS/smb/appletalk fileserver, print server, etc). Then we have the 3 NetBSD/sparc ssh servers (running with case open, because the scsi disks would overheat). Note the add-on switch to switch the system drives between read-only and read/write mode. On the top of the rack we have the cisco 3500 layer 2 switches, including a 3508G (8 gigabit ports).
Running a dedicated NetBSD Game Server (Half-Life/Counter Strike and Quake3) (top)
One of the things that I do for fun is to organise a small lan party every couple of months. About 15 or so of us get together in a hall somewhere, hook up and play games until the wee hours of the morning. In previous lan parties I had been serving the games from my Win NT machine but I seemed to always run into stability problems. Anyone who has played multi-player computer games will tell you there is nothing ruder than the game server collapsing in a heap of bits just as you are at a critical point in the game. Due to the stability problems I decided to investigate more reliable ways of serving the games at my lan parties.
Most popular games these days have, at least, a dedicated server that runs under Linux. This would address my issues with Win NT causing me grief but, I have never really run Linux before and I did not feel confident in setting up a Linux server to perform such an important task. Enter the NetBSD Linux emulation. I had already been using the Linux emulation to run Linux binaries on my NetBSD computers (most notably Netscape) so I was confident I could make this work. As it turned out, the process was very smooth. I simply installed the SUSE libraries using the NetBSD packages system (if you don't now what to do then just install the linux Netscape, that will install all the Linux libaries) and made sure that I had the Linux emulation support in the kernel. Once this was done I downloaded the Half-Life dedicated server for Linux and also the Quake3 dedicated server for Linux.
The hardware I used for the server was my old 200Mhz PPro machine, this is getting a bit long in the tooth but it was what I had to hand and had a 9Gig SCSI drive plus 96Meg of memory. The disk space was great but I think I could have done with more memory to cope with both servers running at the same time. I loaded a recent copy of NetBSD-current onto the machine. The machine is actually a dual processor beast but, at the time, I deemed running it multiprocessor was a little too risky. What I was looking for was stability more than anything, pushing the envelope too much was not something I wanted to contemplate - people were going to be counting on this machine so I went the safe path and ran the machine uniprocessor. Perhaps the next LAN will see this machine running in full multiprocessor glory. As I said above, I loaded up the Linux libraries using the packages system in readiness for loading the game servers.
For the Half-Life dedicated server it was simply a matter of extracting the tar file into an appropriate directory, I chose /emul/linux/usr/local/games/hlds, I followed the instructions in the Half-Life install document, substituting paths as appropriate. I wanted to be able to play Counter-Strike (a Half-Life modification) so I downloaded and installed that under the hlds directory too. I was very pleasantly surprised that when I ran up the dedicated server it just worked, this was great progress. I also installed the admin_mod for Counter-Strike, which is a modification that allows adminstrator control of the Counter-Strike game from, this also worked fine. As an extra to Counter-Strike I wanted to use the hlstats application. Hlstats is used to track the performance of a player in Counter-Strike (and other games), giving you detailed statistics of what a player did and accumulating a "score" for the player. To run hlstats I needed to install a php aware version of apache, perl and a mysql database, all of these were installed using the packages system.
In addition to Counter-Strike I wanted to host Quake3 games, since I already had the Linux emulation set up adding Quake3 was a matter of downloading it and running the install script, just adjusting the paths as required and following the rest of the install document. At our LAN parties we rarely play plain Quake3 deathmatch, normally we play a couple of mods (specifically excessive and instagib), to get all the files required for the mods I simply copied my entire quake3 directory from my Win NT directory over the top of my NetBSD quake3 directory, since the binary names are different there was no problems in doing this. Much to my surprise, not only did the Quake3 server run fine, I was able to run both the excessive and instagib modifications with no changes.
Of course, we are talking about computers here so there must have been some glitches, right? Well, yes, there were but not where I expected them. The thing that gave me the most grief was actually hlstats - or rather the mysql database that underlies hlstats. I had this problem that if I ran the hlstats data collector I could not access the stats from the web server and vice versa. In the end, this turned out to be a simple fix in MySQL. There is a per user connection limit in MySQL, by the looks of this it should have not come into effect (because it was set to 0) but for some reason it was. Anyway, setting max_user_connections to 100 in the my.cnf file (located in /var/mysql in the default pkg install) cleared this problem up. This was really the only glitch I had with the set up of the server.
The ultimate test for the server was at the LAN party itself, the machine was used as the DHCP server, Counter-Strike server and Quake 3 server for about a dozen people. Since our LAN parties are so small most people play the same game at the same time - we had everyone in the Counter-Strike server and about 8 or so in the Quake3 server - not simultaneously mind you but still, the server performed without a fault. We did notice some glitches when playing the Excessive quake3 mod and someone tried to join the Counter-Strike server but I put this down to the extremely high network load that the Excessive Q3 mod puts on the network. Also, the server seemed a little slow on changing maps in Counter-Strike to the extent that some clients timed out but that may be due to memory issues or processor speed more than anything else. All in all the machine performed like a champ - absolutely rock-solid game serving all through the party, one cannot ask for more than that, apart from the instances I have previously noted the servers ran without a glitch at all, there were no complaints about the serving of the games at all from the people at the LAN. If anyone is looking to host game servers on something more reliable than the Win32 platform, my recommendation is to use NetBSD - you will not be disappointed. As a dedicated game server, NetBSD simply rocks.
NeverWinter Nights Server on NetBSD under Linux emulation (top)
ktracing the linux executable indicated that it was trying to mkdir two directories but failing. In order to run the linux executable, I had to wrap it in the following:if [ ! -d currentgame.0 ] then mkdir currentgame.0 fi if [ ! -d temp.0 ] then mkdir temp.0 fi ./nwserver [various options]
It worked fine after that. It loads custom modules, it runs well, and best of all, there is none of the lag associated with running the server on the same machine you're playing on. My friends are extremely happy about this. :)
back to NWN on a fly NetBSD server!
NetBSD and my life... (Gary Rolland) (top)
Taken from an email to the netbsd-advocacy mailing list on 09/10/2005.
I have been using NetBSD for about two years on my laptop and never had any problems. I use this laptop in work and at home. I knew NetBSD was capable of much more, and I was hell bent on using it at work too!
The main reason for this email is to let you guys know what we mainly use NetBSD for - supporting over 4,800 heavy users (some remote) and counting. It's my story about what NetBSD has done for my life, and how it has actually improved my personal life. I hope you will enjoy it and that it'll offer further encouragement to continue the truly excellent work.
I'll start off by introducing myself. My name is Gary Rolland and I live in the United Kingdom. I work in a team consisting of three other network admins. We all work for a large UK based company. Our task is to keep the servers up and to do maintenance. We work on shifts and can be called out at any time, night or day. We are not responsible for the client machines within the buildings. Sadly, I am unable to disclose the actual company name. I received permission to give you information about what we use NetBSD for and that only (I begged infact). Our network is mission critical - down time costs the company mega money. I certainly would not want to foot the bill!
(Note: I have also been told not to disclose network infrastructure in great detail. If you have any comments/questions, please do ask. I'll try my best to answer them the best I can)
Currently, our network consists of 29 high end servers. They all run NetBSD 2.0.2 and deal with extremely high loads. The servers handle:
- MySQL databases - This makes up most of our traffic/resource usage.
- Apache - Internal and external
- Postfix - Interval and external email
- Samba - Allow the 4,800 users to connect to the NFS.
(Note: Our file servers are connected to NetBSD. So it's like=20 Users->NetBSD->File server. Our file servers run Linux. Am not allowed to touch them! However, the email and httpd data is stored directly on the NetBSD servers. Just for your information, our file servers go down more than our servers;-))
Data facts(avg. per day):
- NetBSD pushes over 870GB of data per day.
- NetBSD pushes about 1,200 emails per day (not always work related. I have seen joke emails a while back with 12MB attachments! (12MB x 4500 :()).
- NetBSD during peak hours, our httpd servers can deal with 35 requests per minute(internal and external). The website is wrote in pretty heavy PHP (and bad PHP, but that's not my call).
Originally our network servers used Windows, running on more older hardware than we have now. We have recently been through an upgrade.
I can describe my admining of Windows as a complete f*cking nightmare. I was constantly worrying over when they would fall over. Here is a small story:
I had been promising my 12 year old daughter and her friend that I would take them to Alton Towers. The night before we had planned to go, I raced into the server rooms and checked everything was running. I was happy things would be OK, because I was on call out the next day and I couldn't let my daughter down again(this has happened more than once). We was having a great time at Alton Towers and the worst thing happened. My work phone starts to ring. My daughters face just dropped, she knew exactly what was coming next.
Boss: 'Gary, servers have all dropped to their knees. I need you here now.' Me: 'Sure, I'll be there in a few hours' (in a 'am going to kill someone voice').
It really upset me to see my daughter let down again, including a friend. They had been talking about going all week and Microsoft Windows of all things messes everything up.
Here I am, speeding up the motorway. My daughter completely pissed off, and for due reasons to. This has happened many other times before. I was even getting abit ratty at home since I'd be thinking of anything which could go wrong. This also lead to more arguments with the wife (which leads to a decrease in other things... ;)
I knew something had to change.
After I had fixed the problem, I stormed into my bosses office and explained we need more admins or need to change our servers. Cut a long story short, he allowed me to trial test two of our servers on NetBSD. He didn't know anything about it - just that I promised him I could increase the stability by a huge margin without disrupting business.
That day I was in a mood to change things(pissed), and so I took two servers home. One was what we call a 'MySQL' server and one httpd server(the bosses go nuts if the httpd goes down).
I had no room to fail.
I was up till 4 AM in the morning installing and configuring things to our exact needs. NetBSD as always installed flawlessly and I installed most required software from pkgsrc.
I roled these machines into the network in the morning. I knew I had to wait now, until something crashed on Windows and to show my boss which machines still stood strong.
That day, a few MySQL machines just decided to go for a quick break, and rebooted themselves a few times(MySQL queries began to be far too slow). The boss was storming(due to the day before also). I showed him which machines did not fall down. He agreed within two hours for me to role out more NetBSD machines slowly.
I was happy.
At first it was hard. The other admins had no clue how to use NetBSD. However, with the time we spent fixing the Windows servers, we now had spare time. I began to show them how to configure things and they actually took a personal interest. They installed it at home and began to tinker with it, making sure they understood how to compile kernels etc etc. You know the stuff am talking about! I even printed the whole handbook out for us.
Our whole Network was now running NetBSD.
Things changed. I was not as busy as I always was. My relationships where getting better. I had more time with my daughter to watch her grow up. The rest of my team agreed - this had been the best move we had ever made. The boss also agreed. He was delighted with our improved stability. Infact, he was so happy that none of us are now on call out of a weekend. We're allowed to admin from home using ssh(we take turns per weekend).
My team and I are constantly working and learning. We're becoming more and more efficient with NetBSD.
NetBSD changed my teams life for the better.
Last weekend I took my family to Alton Towers to finish what we started. We had a fantastic time! During the way home, this time not speeding, I has thinking about the people behind it. Those who put in time, little or large to make the project what it is. I decided to tell you guys my story, to show your work is greatly appreciated!
Once more, thanks again for the all the work. Please keep it up. Please use this email in any way you wish. Post it on your personal website or whatever.