Testing
Software
Test Bed
Miscellaneous
Blogs
|
|
|
Sun, 10 Jan 2010
-
[20100110]
ITX-220 ZFS FreeNAS project
-
My existing home file server is now 10 years old. It has grown organically
through the years. The original configuration used noisy SCSI drives with DAT
tape backup. The household was happy when I retired my original SCSI drives --
they sounded like a jet airplane taking off. Today's configuration uses
ATA/100 drives with rsync'ed backups to an external USB drive.
The usage cases have changed. Originally the server was expected to provide
email services for the household. Later, I added support to netboot my vaxen.
Now, the household wants file sharing, timemachine backups, samba shares. I
was spending too much time trying to fix the old server. Time to start over
again. I decided that using one server for all services might not be a good
idea anymore. I am already using another box for the firewall -- time to
continue this trend. The server becomes a collection of servers that I can
individually change, instead of a one big monolithic server. I will start with
a new box for serving files and for backups -- a NAS device.
I reviewed some of the NAS consumer devices that are available today. I
decided to avoid them. My existing server lasted for so many years because I
was able to make changes to the hardware and software. I was not convinced
that a consumer device would be so flexible. So, I am going to make my own NAS
-- what do I want it do?
- What services would be required for the household?
-
- daapd server support
- timemachine backup server support
- samba/NFS/AFP/tftp server support
- backup files using ssh/rsync interface
- What OS features are required?
-
- software RAID support
- fast snapshots
- detect and recover from disk data corruptions
- Hardware requirements?
-
- SATA drives
- intel x86 CPU
- low power footprint
- just needs to be fast enough to serve files
- PCI slot
- use existing parts, if possible
I discovered the Asus ITX-220 motherboard while searching an online catalog. I
will purchase it with 2G of RAM. I can use an existing ATX case. The
motherboard only supports 2 SATA drives -- I already have a SATA PCI card with
two more SATA ports. I figure that 4 SATA drives should be enough for now -- 3
for the RAID and 1 for a rsync'ed backup.
ZFS would be a match for my requirements, but this will limit my OS choices.
This was a worry, until I heard about the FreeNAS project. Using FreeNAS with
ZFS is a perfect match. I should be able to boot the new box off an old USB
key.
At first, I considered my ITX-220 ZFS FreeNAS project to have a low chance of
success. I would divide the project into a number of milestones and
evaluate the project when one of the milestones failed:
- hardware install
-
I put the new motherboard, in the old case. I needed a new cable to allow my
new motherboard to work with the my older ATX power supply. It booted without
problems.
- software install
-
I put the lastest FreeNAS image on an old USB key. It took a couple of
attempts to get FreeNAS to boot of the USB key. During last attempt, I
copied the raw image to the beginning of the key and ignored the parition
table.
- ZFS
-
I put in two SATA drives. I have never used ZFS before. I read an online
example and configure the two drives into a big raidz pool. This step took
the longest amount of time. I tried using "compression=gzip" and the box
started to run too slow -- the "lzjb" compression scheme seems to work
better.
- backup
-
I copied files to the NAS. Just in time -- I saw this error on my existing
older server:
wd1a: error reading fsbn 236276797 of 236276768-236276799 (wd1 bn 236276860; cn 234401 tn 10 sn 22)wd1: (uncorrectable data error)
- servers
-
I used the FreeNAS GUI to enable AFP. I followed the steps on a blog to
setup time-machine backups. I setup daapd and samba shares. The tricky
part was testing that the new server works, as expected, with the existing
clients. The speed of the NAS appears to be fast enough for the existing
clients.
- increase size of raidz pool
-
Not supported by ZFS. In the future, I will make a backup and re-create
the raidz pool with three drives.
In summary, it all just works. In about 9+ hours, I had a working NAS.
References:
Wed, 24 Jun 2009
-
[20090624]
fuzzed ELF -- debugging the NetBSD kernel
-
I discovered that using mangle.c to corrupt an ELF executable
can crash the NetBSD 5.0 kernel.
I recompiled a
kernel with full debug and tried to debug the crashdump.
Gdb was unable to get a backtrace of the corrupted ELF
process -- I got a bad address error instead.
Next, I tried to debug the kernel using GDB over a serial port. Setting this
up look longer than I had originally planned:
- Read the HOWTO.
- Run pkg_add gdb to get gdb6 on my remote system.
- Review the Debugging Kernel Problems paper. I actually took this
tutorial.
- Step through the exec() code while having gdb lose its connection.
This bug appears to be already fixed in kern/subr_kobj.c. I think that
bug would have taken me a long time to solve.
Here is the script I used to fuzz the ELF executable:
#
# Create a small ELF executable as a base.
#
rm -f true.c a.out
printf "int main() { return 0; }\n" > true.c
cc -o a.out true.c
#
# Loop forever.
# Create a corrupted ELF exeutable.
# Run this executable.
#
typeset -i acount
let acount=0
while true
do
#
# setup working directory
#
let acount=acount+1
DIRNAME=f${acount}s"$(date +'%s')"
echo $DIRNAME
mkdir $DIRNAME
cd $DIRNAME
#
# setup this ELF image
#
cp -f ../a.out ./curr.img
mangle curr.img "$( wc -c < ./curr.img)"
md5 curr.img > curr.img.md5
sync ; sync ; sync
#
# Run corrupted executable in a sub-shell.
# Prevent the generation of core files.
# Limit run time, in case executable has endless loop.
#
(
ulimit -c 0
ulimit -t 1
exec ./curr.img
) > /dev/null 2> /dev/null < /dev/null
cd ..
#
# kernel did not crash, try again
#
rm -rf $DIRNAME
done
Sat, 17 Nov 2007
-
[20071117]
update -- free gear available in Ottawa
-
Still available:
- AlphaServer 2100
- SPARCstation IPC
A DEC hobbyist drove by and took a van full. That is about 90% of my gear. Excellent!
Mon, 12 Nov 2007
-
[20071112]
free gear available in Ottawa
-
I need to clean out my basement. Free to a good home.
Available for pick-up in Ottawa. Will consider shipping if you are
willing to pay packaging and shipping costs.
- Sun 3/50 and Sun 3/160
- MicroVAX-II in a BA123 enclosure
- DEC RL02 drives
- AlphaServer 2100
- DECstation 3100
- SPARCstation IPC
- VT100 Tempest
There will be more gear available. I will update this webpage as
I sort out the gear -- http://www.employees.org/~kirk/page0290.html
Sun, 10 Jun 2007
-
[20070610]
DEC pdp8/a roadtrip
-
Went on a road trip to view a pdp8/a. This one was in a heavy desk.
The 8 was in great shape -- it looked brand new. I would assume that
one would have used a VT52 as a console.
I wrote a new tcl/expect script to help boot the
pdp11 .
Mon, 21 May 2007
-
[20070521]
Using crunchgen with OpenBSD Firewalls
-
After BSDCan, I try to think of some userland paper that I could present at the
next conference. I was wondering if crunchgen would be too boring. I used to
build my OpenBSD firewalls as tiny systems and I was considering revisiting
this technique. I would figure out the minimum set of executables and files
required, use the picobsd utilities, and use the openbsd installation media
tools to create a boot image. Does a minimal system offer any extra
security?
- I would use CDROM boot images.
- The firewall should not require a hard drive.
- The filesystem would be read-only.
- A shell will just appear on the serial console -- no login required.
- I want to introduce new configuration files without rebooting.
Why OpenBSD? I reported a sysctl() bug on Sunday and hours later the fixed
is committed. Since then, I have been purchasing OpenBSD t-shirts and CDROMs to help
support this project.
References:
Sun, 20 May 2007
-
[20070520]
BSDCan 2007
-
I went to BSDCan this weekend. My favorite presentations were about
ZFS
-- a file system on the move, and
Poisonous People
-- a lively
discussion about open source projects. I plan to attend next year.
-
[20070520]
TU56 DECtape Drive
-
I made a video of rk restoring his TU56 DECtape drive.
Mon, 09 Oct 2006
-
[20061009]
Automated installation of 4.3BSD Reno on simh VAX-11/780 emulator
-
Now that the simh emulator supports the VAX-11/780, I wrote some scripts to
automate the installation of 4.3BSD.
References:
Mon, 18 Sep 2006
-
[20060918]
DEC pdp8/e moving day
-
We rented a truck and moved the pdp8/e to its new home. The H960 style
cabinets were too tall to wheel under the truck's door, so we had carry the
cabinets, into the truck, on their side. Because of the 8's location, in the
house, we had to remove all the equipment from the cabinet. This fellow had
a wonderful collection of 8s, spare parts, paper tapes and documentation.
I got a VT100 terminal with the Tempest option -- the terminal and
keyboard were heavy. And a PDP11 programming card -- I want to
finish with my 4.3BSD SIMH project so I can start a PDP11 project.
Thu, 07 Sep 2006
-
[20060907]
DEC pdp8/e roadtrip
-
Went on a road trip to view a pdp8/e. This one was in an H960 style
cabinet. We should be moving this 8 to its new home next week.
Because of the 8's location, we might have to remove all the equipment,
from the cabinet, before moving it to the truck.
Mon, 04 Sep 2006
-
[20060904]
4.3BSD Reno on simh VAX-11/780 emulator
-
Now that the simh emulator supports the VAX-11/780, I decided to attempt to
install and run 4.3BSD Reno.
I have written some expect scripts to automate the installation -- will create
an archive and a webpage when I get the time.
References:
|
|
|
