Booting Straigth from the Mirror

October 2009


This section describes how to boot a Linux distro straight from the mirror, using DHCP, PXE, Etherboot's gPXE and SysLinux. It is a precision of what is done at boot.kernel.org.

With the old way of booting via the network, both the configuration for PXELinux and the kernel and RAMdisk were fetched via TFTP. This had three disadvantages: fetching big files like kernels and RAMdisks via TFTP is slow and prone to failure, especially across many networks; TFTP is not server-side scriptable, so configuration had to be rather static; and the PXELinux configuration that came with many Linux distributions always carried the assumption that they were in the TFTP server's root, and they were the only distro there.

Now, there is a new way, which puts gPXE before PXELinux, thereby reducing TFTP traffic to a single, smallish file, and re-routes the rest of the traffic over HTTP. Here's how it works...

Figure 1.  Booting from the network the new way, via gPXE and then PXELinux

Flowchart describing booting a machine via the network the new way, via gPXE and then PXELinux

Procedure 70.  Setting up booting straight from the mirror via DHCP/PXE

  1. Setting up the TFTP server

    sudo mkdir -p /srv/tftp/tftpboot
    sudo apt-get install tftpd-hpa

    then edit /etc/default/tftpd-hpa:

    #Defaults for tftpd-hpa
    RUN_DAEMON="yes"
    OPTIONS="-vvv -l -s /srv/tftp"
    	

  2. Populating the TFTP server

    Fetch the latest gPXE release.

    wget http://kernel.org/pub/software/utils/boot/gpxe/gpxe-0.9.9.tar.gz
    tar zxvf gpxe-0.9.9.tar.gz
    cd gpxe-0.9.9/bin
    make bin/undionly.kpxe
    sudo cp bin/undionly.kpxe /srv/tftp/tftpboot/

    [Warning]Warning

    To be on the safe side, you want to compile on a 32-bit machine.

  3. Installing and populating the HTTP server

    (Basic setup of an HTTP server is left as an exercise to the reader.) Just set up a web server and make it so that http://my.server.org/boot/ points to /srv/www/boot.

    Then fetch the latest syslinux from kernel.org and compile it:

    wget http://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-3.83.tar.gz
    tar zxvf syslinux-3.83.tar.gz
    cd syslinux-3.83
    make
    sudo mkdir /srv/www/boot/syslinux/3.83/32-bits
    sudo cp  com32/menu/vesamenu.c32 com32/menu/menu.c32 com32/modules/cmd.c32 /srv/www/boot/syslinux/3.83/32-bits

  4. Configuring the DHCP server

    (Basic installation of a DHCP server is left as an exercise to the reader.) When the target machine boots via PXE, the DHCP server not only needs to send an IP number, but also information as to where to find the PXE bootstrap, and where to get its configuration information. So in you dhcpd.conf in the top level, you may need any of these:

    # Declare options needed for PXELinux
    option space pxelinux;
    option pxelinux.magic      code 208 = string;
    option pxelinux.configfile code 209 = text;
    option pxelinux.pathprefix code 210 = text;
    option pxelinux.reboottime code 211 = unsigned integer 32;
    
    #
    # gPXE-specific encapsulated options
    #
    option space gpxe;
    option gpxe-encap-opts code 175 = encapsulate gpxe;
    option gpxe.priority code 1 = signed integer 8;
    option gpxe.keep-san code 8 = unsigned integer 8;
    option gpxe.no-pxedhcp code 176 = unsigned integer 8;
    option gpxe.bus-id code 177 = string;
    option gpxe.bios-drive code 189 = unsigned integer 8;
    option gpxe.username code 190 = string;
    option gpxe.password code 191 = string;
    option gpxe.reverse-username code 192 = string;
    option gpxe.reverse-password code 193 = string;
    option gpxe.version code 235 = string;
    	

    To find out what exactly must be included, and what can be left out, study the docs on how to set up DHCP for PXELinux and for gPXE, the experiment. These settings work for me. Then to limit the experiment to a few hosts, set up a group in one of your subnets in the dhcpd.conf, like this:

    group boot-gpxe
    {
    if exists user-class and option user-class = "gPXE"
      { 1
        filename "http://www.mydomain.org/boot/boot.php"; 2
      }
    else
      { # This is the first step: the PXE bootROM loads the gPXE bootstrap
        next-server 192.168.0.5; 3
        filename "/tftpboot/undionly.kpxe"; 4
      }
    
    host example05  { hardware ethernet 01:23:45:67:89:AB ; fixed-address 192.168.0.14 ; }
    
      } # end group boot-gpxe
    	

    1

    This is for when gPXE is already loaded.

    2

    This is the configuration file for gPXE. We will point to a menu. Example contents follow later.

    3

    This is the address of your TFTP server. How to set one up is extensively documented on the Web, and I kind of assume you already know how to do that if you're fiddling with this.

    4

    This is the only file that is coming through TFTP. It can be taken from the gPXE source, which is available through their website.

  5. Configuring gPXE

    Once the target host has fetched the PXE bootstrap undionly.kpxe, it will load it into memory and execute it, and it in its trun will fetch the config file we specified, i.c. http://www.mydomain.org/boot/boot.php, which reads:

    #!gpxe
    imgfree
    chain http://my.domain.org/boot/pxelinux/3.83/32bit/vesamenu.c32 http://my.domain.org/boot/menu.php
    	

    In its turn, menu.php contains:

    menu background TuxOffZB.png
    prompt 0
    timeout 100
    allowoptions 0
    menu timeoutrow 29
    menu vshift 2
    menu rows 12
    menu color title  1;36;44   #ff8bc2ff #00000000 std
    menu color unsel  37;44     #ff1069c5 #00000000 std
    menu color sel    7;37;40   #ff000000 #ffff7518 all
    menu color hotkey 1;37;44   #ffffffff #00000000 std
    menu color hotsel 1;7;37;40 #ff000431 #ffff7518 all
    
    menu title LWP-devel Network Boot Menu
    
    label item3
    menu label ^3 UWP Menu
    kernel http://my.domain.org/boot/customermenu.php
    
    label item10
    menu label ^A Ubuntu_Jaunty_i386 as from CD
    kernel http://my.domain.org/boot/fromthemirror.php?arch=i386&distro=ubuntu&version=jaunty
    
    label item13
    menu label ^D Ubuntu_Karmic_amd64 as from CD
    kernel http://my.domain.org/boot/fromthemirror.php?arch=amd64&distro=ubuntu&version=karmic
    	

    while e.g. fromthemirror.php with parameters ?arch=amd64&distro=ubuntu&version=karmic renders:

    #!gpxe
    echo It is assumed that you have dhcp networking
    ifopen net0
    dhcp net0
    echo "set 209:string pxelinux.cfg/default\n"; 1
    
    
    set 210:string http://osmirror.rug.nl/ubuntu/dists/karmic/main/installer-amd64/current/images/netboot/ 2
    
    echo "Here we go"
    chain http://my.domain.org/boot/pxelinux.0.3.83 3
    echo "PxeKnife booting cancelled, using local disk instead.."
    	

    1

    This tells the PXELinux bootloader that we are going to chainload into where to find its configuration, relative to what we specify with option 210.

    2

    This is the prefix PXELinux will but in front of any file that doesn't start with a service specifier. It conveniently enables us to point to the root directory of the Ubuntu installer on any mirror, and the PXELinux config that comes with Ubuntu will work.

    3

    The pxelinux.0 that comes with Ubuntu Jaunty is fairly recent, but you need at least version 3.70 for the HTTP download to work (and even then it will only work when chainloading from gPXE). If the particular distro you are trying to boot is too old, you can just fetch pxelinux.0 from the SYSLinux website and put it on www.mydomain.org/boot or something. gPXE will happily work with that.

    [Warning]Warning

    Mind you, the above snippets are actually the outputs of *.php, as seen on the screen of your browser. I suppose I could rename it to another extension for clarity, but I assume that you are able to write PHP that outputs this...

[Note]Note

Yes, it is possible to boot a kernel and ramdisk from gPXE without chainloading PXELinux. And yes, then you can also make your PHP application determine what boot parameters a system needs. But that is not the point here. The point here is that many distros come with PXELinux config, and so it is very easy to offer the native installer of the distro without any additional configuration.