FreeBSD PXEBoot Guide: Jumpstart installation

In this section covers how to set up a jumpstart installation. By the end of this section, you should be able to set up your server such that (almost) all you need to do to install FreeBSD is to connect your new machine to your network.

The jumpstart installation will load a memory disk as root device and fetch the release using ftp. We could also do this providing both root device and release files with nfs, if you want to to this, skip to the next section.

1. Building pxeboot
2. Building a custom kernel
3. The tftp root directory
4. The memory disk root device
5. Automating the installation
6. Speeding up booting
7. Rebooting

Building pxeboot

Build the pxeboot loader:

# cd /usr/src/sys/boot
# make -DLOADER_TFTP_SUPPORT=YES

and install it into the tftp directory under the path specified in dhcpd.conf:

# cp i386/pxeldr/pxeboot /var/tftp/boot/
# cp i386/boot0/boot0 /var/tftp/boot/
# cp i386/boot2/boot1 /var/tftp/boot/
# cp i386/boot2/boot2 /var/tftp/boot/
# cp i386/mbr/mbr /var/tftp/boot/

[ top ]
Building a custom kernel

First we create a new kernel configuration file, JUMPSTART:

# cd /usr/src/sys/i386/conf
# cp GENERIC JUMPSTART

Edit the configuration file, make sure to include the following lines:

# Filesystems
options MD_ROOT # MD is a potential root device
options PSEUDOFS # Pseudo-filesystem framework

# Memory pseudo devices
device mem # Memory and kernel memory devices
device md # Memory "disks" needed for root file system

NOTE: The default method for mounting a root device is using NFS. The current release 5.3-RELEASE and 6.0-CURRENT (2004-11-25) will override any attempts to use a memory file disk if NFS is supported. It is therefore important that the following linies are NOT in the kernel configuration file:

options NFSCLIENT
options NFS_ROOT

It is not sufficient to remove NFS_ROOT only!

Now we're ready to build the kernel:

# cd /usr/src
# make -DLOADER_TFTP_SUPPORT=YES KERNCONF=JUMPSTART buildkernel

This builds the base system and a kernel with support for tftp. If you are compiling the kernel for a different version of FreeBSD than the server system, you need to "make buildworld" before building the kernel.

We are not overwriting our system kernel, instead we will install directly into the directory /var/tftp. We first need to install a few files by hand:

# mkdir -p /var/tftp/boot/defaults
# cp /usr/src/sys/boot/forth/loader.conf /var/tftp/boot/defaults/
# cp /usr/src/sys/boot/forth/loader.4th /var/tftp/boot/defaults/
# cp /usr/src/sys/boot/forth/support.4th /var/tftp/boot/defaults/
# cp /usr/src/sys/boot/i386/loader/loader.rc /var/tftp/boot/
# cp /usr/src/sys/i386/conf/GENERIC.hints /var/tftp/boot/device.hints

Then install the kernel

# make KERNCONF=JUMPSTART DESTDIR=/var/tftp installkernel

This will create the base system directory layout, but all kernel files are installed into /var/tftp/boot.

[ top ]
The tftp root directory

Edit /var/tftp/boot/loader.rc to contain only the following lines:

include /boot/loader.4th
start

and edit /var/tftp/boot/loader.conf to the following content:

init_path="/stand/sysinstall"
rootfs_load="YES"
rootfs_name="/boot/jumpstart"
rootfs_type="mfs_root"
autoboot_delay=5

The init_path is convenient for debugging because it allows us to toggle wether to run sysinstall or init at boot, provided both are present. The default is something like /sbin/init:/stand/sysinstall, which will launch sysinstall if init is not found. If sysinstall fails for some reason, change above to /sbin/init.

We have specified that the loader should fetch a file for use as root device, this will be created in the next section. At this stage, the jumpstart client should complete stage 3 and load the kernel.

[ top ]
The memory disk root device

We first create a memory file system which is mounted as root device for the installation. The easy way to obtain such a memory disk is to grap it of the installation media, floppies or cd boot iso:

# mdconfig -a -t vnode -f boot.flp -u0
# mount /dev/md0 /mnt/flp
# cp /mnt/flp/boot/mfsroot.gz /var/tftp/boot/jumpstart.gz
# umount /mnt/flp
# mdconfig -d -u0

But, we want to create our own and customize it. It can be any size as long as the jumpstart client has enough RAM. We create a 16MB memory disk:

# dd if=/dev/zero of=jumpstart bs=1k count=16k
# mdconfig -a -t vnode -f jumpstart -u0
# disklabel -r -w md0 auto
# newfs /dev/md0c

Next mount the file system:

# mkdir -p /mnt/mfs
# mount /dev/md0 /mnt/mfs
# cd /mnt/mfs

We now have an empty memory disk which will be served the client. Now we need to populate the file system to start the system installation, first create the base directory layout:

# mkdir boot dev etc mnt stand
# ln -s stand bin
# ln -s stand sbin

In the boot directory we place the files for the boot manager to be installed:

# cp /var/tftp/boot/boot* boot/
# cp /var/tftp/boot/mbr boot/

Also we need a few files in the etc directory:

# cp /usr/src/etc/group etc/
# cp /usr/src/etc/services etc/
# cp /usr/src/etc/netconfig etc/
# cp /usr/src/etc/usbd.conf etc/

And add any other files you think might be usefull, it may be usefull to add all the files from the etc directory of a fresh install.

The memory file system distributed with the installation media is created using crunchgen(1) to build a compressed binary with all libraries and programs in one file. crunchgen needs a configuration file, to create a special Makefile. The configuration file used to create the installation floppies can be found in /usr/src/release/i386/boot_crunch.conf.

First create a directory to work in:

# mkdir /var/tftp/jumpstart
# cd /var/tftp/jumpstart
# cp /usr/src/release/i386/boot_crunch.conf .

Edit the configuration file to your taste, you can use this if you like. Now build the binary:

# crunchgen -m Makefile boot_crunch.conf
# make

This creates one binary, boot_crunch. To check that it works try running boot_crunch, will give you a list of the programs included, you can then test that each works by running boot_crunch , in particular make sure that the extra programs you included work. If generating the boot_crunch fails, you may consider to mount the root device using nfs, see Building the base system or take a look at Advanced topics on alternative methods to populate your root device. Copy this file into the memory disk:

# cp boot_crunch /mnt/mfs/stand
# cd /mnt/mfs/stand

We have only one binary containing all the programs. To make them available as usual we create hard links for each program like this:

# ln boot_crunch sysinstall

Creating links by hand is tiedous, so I created a perl script to do it. dhclient uses a shell script which must be installed separately:

# cp /usr/src/sbin/dhclient/dhclient-script.sh dhclient-script
# chmod +x dhclient-script

This memory file system is rather limited, for a more full equipped crunch file, checkout the 'rescue' distribution included with the source.

Unmount the file system and run fsck:

# cd /
# umount /mnt/mfs
# fsck -t ufs /dev/md0
# mdconfig -d -u 0

At this point we have a fully equipped memory file system that can be used for installation. Booting up your jumpstart client, you should get the standard interactive installation menu.

[ top ]
Automating the installation

sysinstall will first look for the file install.cfg which describes the automation of the installation proceedure and if this file does not exist or fails executing it will go into interactitive mode.

To automate the installation we create a install.cfg file. This file must be placed in the root of the memory file system we created or in stand. The syntax is described in sysinstall(8), however not all variables and commands are documented. To tweak it we need to take a look at the source.

One important thing is that it is read strictly top down, variables must be set before the function using them.

If anything fails in the script, sysinstall will abort the rest of the script and go interactive.

################################
# install.cfg for jumpstart of FreeBSD
#
# See sysinstall(8) for details about how to script the process
# This file has been edited from /usr/src/usr.sbin/sysinstall/install.cfg
# Turn on extra debugging.
debug=YES
nonInteractive=YES
noConfirm=YES
noWarn=NO
################################

We don't want any interactive questioning, the whole point is to power up the machine and drink coffee.

################################
# Disk partitioning
#
# WARNING: This will reformat the disk and delete all data!
disk=ad0
partition=all
bootManager=none
diskPartitionEditor
################################

This section selects a disk and creates the partitions. Multiple disks can be partitioned but you must end each disk defintion with diskPartitionEditor.

NOTE: The sample install.cfg provided with sysinstall source sets partition to exclusive, this will set your disk in "dangerously dedicated" mode and is not recommended unless you really need it.

################################
# Disk labeling
#
# All sizes are expressed in 512 byte blocks!
# / 512MB, swap 512MB, /usr 8192MB, /var 8192MB, /home remaining
ad0s1-1=ufs 1048576 /
ad0s1-2=swap 1048576 none
ad0s1-3=ufs 16777216 /usr
ad0s1-4=ufs 16777216 /var
ad0s1-5=ufs 1048576 /tmp
ad0s1-6=ufs 0 /home 1
diskLabelEditor
################################

This section slices up the chosen diskpartitions. The line labeling /home is special, this will use any remaining space, so it must be last.

################################
# Host specific configuration:
tryDHCP=YES
netDev=vr0
# The following optional if using dhcp to configure the network
hostname=jumpstart
#domainname=example.com
#defaultrouter=172.16.0.1
#ipaddr=172.16.1.1
#netmask=255.255.0.0
################################

Apparently sysinstall suffers from amnesia when it comes to the network configuration. Even though the network was configured on boot, sysinstall will need a network configuration as above unless you are installing from a disk.

This should be a general configuration file and all host configuration should be done with dhcp. For unknown reason, hostname is required even if passed with dhcp.

################################
# Select release and distribution to install
#
releaseName 5.3-RELEASE
# either use distSetCustom and choose the individual components or
# choose a collection. We use distSetKernDeveloper which installs base
# and kernel sources
distSetKernDeveloper
################################

This section defines the release and which collection of packages and sources to install. releaseName is by default determined by the release of the kernel and sysinstall will then look for a directory of that name. If you cvsup'ed your sources before building the custom kernel, releaseName will either be x.x-STABLE or x.x-CURRENT, but there are no such releases, so we need to specify it here. FreeBSD has some predefined distributions you can choose, but you can also customize defining the variable dists and then use distSetCustom.

################################
# Select installation method
#
# We want an FTP install, so we also need to specify the ftp server to
# fetch from
netDev=vr0
tryDHCP=YES
_ftpPath=ftp://ftp.example.com/pub/FreeBSD
mediaSetFTP
################################

Again (!?) we need to specify the interface we want to use and configure it using dhcp. For ftp installation we also need to specify a server to install from and a path. sysinstall will look for the directory in the ftp-path.

At this point, your old disk is still alive and happy, the next section will commit everything to the disk, and we have set noConfirm!

################################
#
# OK, everything is set. Do it!
installCommit
################################

You can customize further after install commit setting the variable package to any package you'd like installed and followed by the command packageAdd. At this point, when installation is finnished it will return to the well known sysinstall menu to allow you to add any further customization. To avoid this end the install.cfg with the command shutdown. It may be better though, if you compiled your memory disk with halt to terminate the installation with system halt -p.

If everything works well, simply power up your system, is should be busy installing installing and you busy drinking coffee :-)

[ top ]
Speeding up booting

Booting with PXE takes more time as all the files needs to be fetched over the network. The pxeboot program will first look for compressed files, then the non-compressed, the only file you cannot compres is pxeboot.

So, to speed up the transfer, you can gzip the kernel, any kernel modules and the memory file system. We may need to edit and customize the configuration files, so we let these be uncompressed.

[ top ]
Rebooting

Before you reboot, make sure that your new server won't boot again using pxeboot, hence repeating the process. My experience with the VIA board was that I had to disable LAN boot completely in the bios, not just have it as secondary boot device.