iPXE Booting Over the Network for Linux Diskless Desktops
BOOTING LINUX OVER THE NETWORK FOR DISKLESS DESKTOP DANCING
ow, that sounded dirty, didn't it?! Keeping my hands clean from manually managing hardware is what we are about in this blog post, so let's get down and dirty with iPXE booting over the network for diskless workstations. As most of you are likely tired of hearing, I really detest hardware overhead; those traditional "assets" are the drag and boon of my cloud-based, cloud-only approach. Jack can't be nimble and Jack can't be quick with a bunch of overhead hanging over him, tying him at the waist (or "waste", pun-intended); I am simply not going to be burdened with or by purchasing (mostly unnecessary) equipment that quickly deprecates, and having to secure, manage, update and replace them, incessantly.
I much prefer doing actual work for clients instead of wrangling with internal system maintenance, dealing with hardware and repetitively doing the same for the software needed for each instance. None of that is making me any money; instead, it's sifting it out of me one desktop at a time.
All that being said, and coming down off the old soapbox, I do of course have some physical equipment localized and vital to day-to-day business operations, performing test evaluations for clients, etc. Within my testing and analytics hackspace, my machines are purely purpose-built, in-house. For desktops only needing general applicational use, I've been employing iPXE to boot over the network for diskless workstations.
Don't Disk Me (Because I'm Diskless)
Ironically, I do admit to drooling over the latest cloud-enhancing chipsets with multiplying cores, slick and sleek motherboards, gpu cards that rival NSA computational power (just kidding) and often obsess over local drive storage speeds, specifications and swap space. Hardware stuff. It's fun. And, well, there's admittedly some pretty cool stuff happening across hardware these days in compute, networking and storage. But, it certainly didn't take long in dealing with my small network of machines before coming to what should have already been an obvious and foregone conclusion: if the primary applications needed for these desktops are basically web browsers and e-mail clients mixed in with some other tools inherent in Linux, if I didn't need to use any other proprietary or commercial applications (of course, and always, deploying Linux and open-source platforms) let alone having better things to do with my business day than manage machines? Then there was really no need for full blown desktop deployments. The functional application and use-case for these machines is ideal for diskless workstations.
Booting Over the Network
It isn't hard to imagine dealing with desktops over a large, ever-scaling network of clients, since we have all lived it, every day and for years, and some continue to do so. Nope, not me; given my reluctance for burdensome hardware to begin with? I don't even want to think about it... booting over the network is just the dance card I was looking for - nothing new here, but the environment that makes this accomplishable is the Preboot eXecution Environment (PXE). PXE is an environment for booting a computer using a network interface independently of any local data storage devices. In simplistic terms, when PXE is used a computer will look for a configuration to boot from a remote server rather than a local hard disk (a DHCP server points to a TFTP server and specified a pxelinux.0 file to load from where we can opt to install various distributions). You can do this with as few as 2 computers (1 server, 1 client) or with as many as you can get your hands on. The client PXE code is directly included within the NIC's own firmware and also as part of the UEFI firmware on UEFI hardware.
While PXE is a well-known and relative standard here, interesting variants have emerged with enhanced feature sets. As our working title indicates, I've been leveraging the open source derivative project iPXE and its highly expanded features, but we'll get to talking iPXE specifically in a few as it does build itself off standard PXE.
At the end of the day, I'm resulted with the ability to deploy workstations without need of local drives, an OS or IT oversight coupled with security and compliance advantages. Diskless Workstations are but one, albeit essential, piece to my consultancy, so how does that work and what are the benefits?
In my environment, the entire operating system is loaded via a compressed disk image over a Gigabit LAN, authentication is done via SSH, and home directories are mounted via NFS (although I do have one or more at a given time over iSCSI). All applications are run locally and are very fast – it is nearly unnoticeable that the machine is running entirely over the network. Behind the scenes there are two powerful servers in a hot-standby configuration that serve these diskless clients.
TCO + Headless IT Administration
¤ Cost of Ownership: The TCO of this solution is absurdly low. Generic cost advantages: cost of ssd/hdd, cost of sata, cost of ata, cost of operating system (if proprietary or under support licensing), all canceled out. Even within a relatively small hackspace network of 11 computers (1 server, 10 clients) we could potentially save (10 X (cost of hard drive)) + (10 X (cost of operating system)). Compare this to the cost of managing any other OS configuration and it just blows everything else away.
¤ Headless IT: Outside of my upfront + upgrade and ongoing maintenance costs for hard/software, far more advantageous with PXE-based booting is that I can have all of my configurations stored on one machine. This means I spend less time running around to resolve software and hardware issues. If you regularly install tens/hundreds/thousands of PC’s, you can start the installer on all those machines at once without needing to have individual boot/install media for each machine (heck, you can even use Linux PXE for starting Microsoft Windows network installers and tools).
If you've ever seen my analytics and testing lab and the machine builds done by my own hands, you know and realize I do love tinkering with hardware (the "cool stuff"). But doing so is a calculated measure towards highly purpose-built machines for very specific use-cases; hardware overheard only enters my fold to address an absolute and imperative necessity. Otherwise, this type of IT admin function would be better relegated to more of a weekend, garage-based hobby - in the throws of driving meaningful business operations managing hardware/software really only means I'd be losing money; the "cool factor" is a "cost factor". Even when necessitated towards securing physical hardware and software to exercise it, having this setup affords my consultancy a "headless IT" model and promotes tighter security around devices and data.
Benefits of Diskless Workstations:
- Secure: eliminates virus and malware threats
- Compliant: no sensitive information is ever stored on local disks
- Readily Accessible: all the great software needed to work and be entertained (mostly, Spotify!) is easily available
- Reduced Downtime: hardware failures are a non-issue
- Headless IT: support costs and IT support time is near zero
- TCO: purchase/raw hardware costs for diskless machines is minimal (additionally, see "Headless IT", above)
The negatives? Well, at times I've found ISO images slow to load over TFTP (an issue with TFTP, not my network or NIC cards/hardware, thank your very little). Other than some things to work through in the initial setup, this is really about the only negative I have, a minimal one at best. Enter (finally) iPXE, which allows for much faster booting / loading of ISO images; iPXE resolves this issue as it can work over HTTP / iSCSI and other protocols - even wireless! For some, the initial iPXE setup may be a little daunting, but the rewards have been many on my end.
It's worth mentioning here a highly useful PXE deployment utility: ERPXE. I've used this for network booting and to resolve a range of issues as one of my "stock & standard" server installs. ERPXE is a complete PXE solution featuring a broad range of recovery tools and various OS installations in one box, with a primary goal to make the PXE experience less "painful" for IT admins everywhere. ERPXE can do quite a lot, actually (all from its network location), more than I'm capable of adequately trying to deal with here in a post that is already long in the tooth... errr, mouth. I recommend taking the time to discover and apply ERPXE - quite handy and borrowing from CBS Golf commentary, "useful!" Version 2.0 of ERPXE will be released soon as a tar.gz file and as a Virtual Machine Template. The ERPXE project projects 2.0 will bring a big change to the way PXE is used!
You will first need a DHCP server with PXE capabilities or using Dnsmasq to add PXE options to your current router. You can use DD-WRT or a Linux/Windows DHCP Server, they all support PXE. If you want to use your Router DHCP and add PXE settings then you can ADD Dnsmasq to your network. You can also use Dnsmasq as your primary DHCP server - turning off the Router DHCP service. There is no difference in the PXE boot process between DHCPd and DNSMASQ. Both will allow you to boot using pxelinux.0/gpxe/ipxe.
ERPXE provides a handsome bevy of Plugins: Cloning and Deployment; Hardware Diagnostics; Operating System Installation; Linux Live; Recovery Tools; System Diagnostics; Windows PE. That's Awesome Sauce!
PXE booting with the mentioned advantages should certainly give you happy feet, dancing around the data center. It does me, but I've enhanced this model by employing the more rich open source project iPXE. I prefer iPXE because of its ability to perform tasks beyond the scope of a legacy PXE ROM: tasks such as booting via HTTP or iSCSI, controlling the boot process with a script, DNS, FCoE, creating dynamic menus, etc.
iPXE is developed by the persons who originally developed gPXE (which evolved from Etherboot), and is gPXE's official replacement. iPXE is the leading open source network boot firmware licensed under the GNU GPL (with some portions under GPL-compatible licences) and is included in products from several network card manufacturers and OEMs. It provides a full PXE implementation enhanced with additional features that allow booting from a web server via HTTP, from an iSCSI SAN, from a Fibre Channel SAN via FCoE, from an AoE SAN, from a wireless network, from a wide-area network, or from an Infiniband network. Much to like!
NOTE: iPXE supports the EFI and UEFI environments, as well as the standard PC BIOS but do your diligence here - although developing with priority, I'm not entirely confident UEFI support can be taken as stable.
Unlike a traditional PXE ROM, iPXE is able to boot over a wide area network such as the Internet and can be used on any platform that can boot an ISO image. This includes many cloud providers and physical hardware. Its approach is procedural with scripts, which gives you a benefit that you can react better to errors and make more runtime choices. You can use iPXE to replace the existing PXE ROM on your network card, or you can chainload into iPXE to obtain the features of iPXE without the hassle of reflashing.
iPXE - The Versatile Boot Loader
Dirty Dancing: The Requisite Stuff
Here's the basic requirements for iPXE Network Boots on Diskless Workstations:
Make sure you are running the latest pxelinux
Avoid issues by setting SELinux to Passive mode, and check your firewall settings configured properly to allow the required traffic and services communications
Configuring Linux Server to PXE boot / configure your Linux network to support PXE images (or validating it already does)
Hardware/client that supports PXE boot, with a minimum 2GB RAM
Hash-verified ISO of your chosen distro
A fast network; diskless booting is only useful if your network can transfer data quickly
While iPXE can be flashed to network cards – this seemed overkill for my environment and I prefer leaving the NIC ROM untouched, as provided by the OEM. In my case it was easier and preferred to ‘chainload’ iPXE from the normal PXE (Network Boot) process.
DO YOUR DUE DILIGENCE: INFO 2 GET YOU GOING
Do You Wanna Dance? Go Diskless!
Whether you use the extra features of iPXE or stick with standard PXE, you will need to build a PXE server to boot from image over the network. To get started building your PXE server, most Linux distros provide clear and concise directions (and don't forget ERPXE - easy!):
PXE MultiDistro Install (this is old, but provides a starting point for Ubuntu MultiDistro PXE boot)
A Few Vids For Your Reference:
iPXE Installation of Ubuntu Server
Boot Ubuntu (live) over iPXE and NFS
♠ If you found this information useful or of interest, you might also find worth exploring:
It's finally the Weekend, and in spite of still working that automatically puts me into a good mood. I could just go with The Ramones "Do You Wanna Dance?" here and leave it at that, but -
you knew it was coming... the obligatory "Dirty Dancing" film reference and insert...