Introducing NUX, a kernel framework

History and motivation

Circa 2018, I decided that the Murgia Hack System needed a fresh start to support newer architectures.

MH's kernel is quite clean and simple, but suffers from an aging low level support. Incredibly, some of that i386 code can be traced back to my early experiments (in 1999!) and code that I wrote for my first SMP machine – a dual Pentium III bought in Akihabara in the early 2000s!

Unfortunately, emotional attachment to code doesn't create great engineering, and I had to start from scratch.

The driving principle behind this effort – that later became NUX – was to rationalise my kernel development.

At its core, a kernel is an executable, running in privileged mode. It's special because it handles exceptions, IRQs and syscalls, essentially events, so it can be seen as an event-based program. And it runs on multiple CPUs concurrently, we can even draw similarities with multi-threading.

The very annoying and often project specific part of a kernel is the bootstrap. A kernel usually starts in a mode that it's either very limited (think x86 legacy boot) or very different in terms of runtime (think EFI).

A kernel is thus required to set up its own data structures (and virtual memory), and then jump in it (through magic pieces of assembler called trampolines).

In a nutshell, NUX can be seen as an attempt to solve all the abovementioned problems that differentiate a kernel from a normal executable.

Solving the bootstrapping problem.

To solve the setup of the kernel executable data structures, NUX introduces APXH, an ELF bootloader. APXH – (upper case of αρχη), greek for beginning – is a portable bootloader whose goal is to load an ELF executable, create the page tables based on the ELF's Program Header, and jump to the entry point. It attempts to be the closest thing to an exec() you can possibly have at boot.

APXH also supports special program header entries, – such as Frame Buffer, 1:1 Physical Map, Boot Information page – that allows the kernel to immediately use system features discoverable at boot, further reducing low level initialisation.

APXH is extremely portable, and currently works on i386, AMD64 and RISCV64, and also supports booting from multiple environemnts, currently EFI, GRUB's multiboot and OpenSBI.

Creating an embedded executable: the need for a small libc.

In order to create an executable in C, you'll have to create against a C Runtime (crt) and a C Library.

This is why NUX introduces libec, an embedded quasi-standard libc.

libec is based on the NetBSD libc, guaranteeing extreme portability and simplicity. It is meant to be used as a small, embedded libc.

Every binary built by NUX – whether APXH, a NUX kernel, or the example kernel's userspace program – are all compiled against libec.

A kernel as a C executable.

As for any C-program, the kernel will have to define a main function, that is called after the C-runtime has initialised. The libec is complex enough to support constructors, so that you can, define initialisation functions that run before main.

A special function of NUX, that diverts from normal C-programs, is main_ap. This is a main funciton, that is called on secondary processors, that is other processors that are not the bootstrapping CPU.

Kernel entries as events.

As mentioned above, a kernel has to deal with requests from userspace and hardware events. In NUX, this is done by defining entry functions for these events.

The whole state of the running kernel can be defined by the actions of these entry functions.

A kernel entry has a uctxt passed as a parameters and returns a uctxt. uctxt is a User Context, the state of the userspace program. The kernel can modify the User Context passed as an argument and return the same one, or can return a completely new one.

The former is how system calls return a value, the latter is how you implement threads and process switches.

The NUX library interface

Finally, NUX provides three libraries:

1. libnux: a machine-independent library that provides the higher level funcitonalities you need to develop a fully functional OS kernel. The 'libnux' interface is here.

1. libhal: This is a machine-dependent layer. Exports a common interface to handle low level CPU functionalities. The HAL interface is here.

1. libplt: This is a machine-dependent layer. Exports a common interface to handle low level Platform functionalities, such as device discovery, interrupt controller configuration and timer handling. The Platform Driver interface is here.

The separation between hal and plt is possibly a unique choice of NUX, and allows, as many other design choices of NUX, for a gradual and quick porting to new architectures.

For example, when the AMD64 support was added, the ACPI platform library needed no changes, as the CPU mode was different but the platform was exactly the same.

Similarly, an upcoming support for ACPI support for Risc-V consists mostly on expanding the ACPI libplt to support Risc-V specific tables and the different interrupt controllers.

A useful tool for kernel prototyping.

NUX goal is to remove the burden of bootstrapping a kernel. And be portable.

The hope is that NUX will be useful to others the same way it has been useful to me: experimenting with kernel and OS architectures, while skipping the hard part of low level initialisation and handling.