No reboot patching comes to Linux 4.0

With Linux 4.0, you may never need to reboot your operating system again.

Using Linux means never having to reboot. -- SUSE One reason to love Linux on your servers or in your data-center is that you so seldom needed to reboot it. True, critical patches require a reboot, but you could go months without rebooting. Now, with the latest changes to the Linux kernel you may be able to go years between reboots.

This is actually a feature that was available in Linux in 2009 thanks to a program called Ksplice. This program compares the original and patched kernels and then uses a customized kernel module to patch the new code into the running kernel. Each Ksplice-enabled kernel comes with a special set of flags for each function that will be patched. The Ksplice process then watches for a moment when the code for the function being patched isn't in use, and ta-da, the patch is made and your server runs on.

Oracle acquired Ksplice in 2011, and kept it just for its own Oracle Linux, a Red Hat Enterprise Linux (RHEL) clone, and as a RHEL subscription service. That left all the other enterprise and server Linux back where they started.

Then KernelCare released a service that could provide bootless patches for most enterprise Linux distros. This program use proprietary software and is only available as a service with a monthly fee. That was a long way from satisfying many Linux system administrators.

So, Red Hat and SUSE both started working on their own purely open-source means of giving Linux the ability to keep running even while critical patches were being installed. Red Hat's program was named kpatch, while SUSE' is named kGraft.

The two companies took different approaches. Kpatch issues a stop_machine() command. After that it looks at the stack of existing processes using ftrace and, if the patch can be made safely, it redirects the running code to the patched functions and then removes the now outdated code.

Kgraft also uses ftrace, but it works on the thread level. When an old function is called it makes sure the thread reaches a point that it can switch to the new function.

While the end result is the same, the operating system keeps running while patches are made, there are significant differences in performance. Kpatch takes from one to forty milliseconds, while kGraft might take several minutes but there's never even a millisecond of down time.

At the Linux Plumbers Conference in October 2014, the two groups got together and started work on a way to patch Linux without rebooting that combines the best of both programs. Essentially, what they ended up doing was putting both kpatch and kGraft in the 4.0 Linux kernel.

Jiri Kosina, a SUSE software engineer and Linux kernel developer, explained, that live-patching in the Linux kernel will "provides a basic infrastructure for function "live patching" (i.e. code redirection), including API [application programming interface] for kernel modules containing the actual patches, and API/ABI [application binary interface] for userspace to be able to operate on the patches. This is "relatively simple and minimalistic, as it's making use of existing kernel infrastructure (namely ftrace) as much as possible. It's also self-contained, in a sense that it doesn't hook itself in any other kernel subsystem (it doesn't even touch any other code)."

The release candidate for Linux 4.0 is now out. Kosina stated that "It's now implemented for x86 only as a reference architecture, but support for powerpc, s390 and arm is already in the works." And, indeed, the source code for these architectures is already in the Live Patching Git code.

Simply having the code in there is just the start. Your Linux distribution will have to support it with patches that can make use of it. With both Red Hat and SUSE behind it, live patching will soon be the default in all serious business Linux distributions.