I built the foundation of this operating system by following along with the Udemy course, "Developing a Multithreaded Kernel From Scratch", by Daniel McCarthy.
This kernel is built on a QEMU x86 System emulator.
You can find the emulated hardware specs here: i440fx PC.
sudo apt upgradesudo apt install nasmsudo apt install qemu-system-x86- Download binutils into
$HOME/src - Download gcc into
$HOME/src scripts/install_gcc_deps.shscripts/build_binutils.shscripts/build_gcc.sh
scripts/build.shmake run
The BIOS will load the boot sector into memory and execute it. Our FAT (File Allocation Table filesystem) boot sector is defined by boot.asm and gets written to the first sector in our disk image.
The bootloader enters protected mode and loads the kernel (100 sectors of the disk, 1 KB) into memory at 0x0100000 and then jumps to it.
Once the kernel is in memory, the first function to execute is
_start in kernel.asm. From here, I call `kernel_main``
which is in kernel.c.
The hardware emulated by QEMU provides two i8259 PIC devices. The following is a snippet of output from info qom-tree
of the QEMU monitor:
/device[4] (isa-i8259)
/elcr[0] (memory-region)
/pic[0] (memory-region)
/unnamed-gpio-in[0] (irq)
/unnamed-gpio-in[1] (irq)
/unnamed-gpio-in[2] (irq)
/unnamed-gpio-in[3] (irq)
/unnamed-gpio-in[4] (irq)
/unnamed-gpio-in[5] (irq)
/unnamed-gpio-in[6] (irq)
/unnamed-gpio-in[7] (irq)
/device[5] (isa-i8259)
/elcr[0] (memory-region)
/pic[0] (memory-region)
/unnamed-gpio-in[0] (irq)
/unnamed-gpio-in[1] (irq)
/unnamed-gpio-in[2] (irq)
/unnamed-gpio-in[3] (irq)
/unnamed-gpio-in[4] (irq)
/unnamed-gpio-in[5] (irq)
/unnamed-gpio-in[6] (irq)
/unnamed-gpio-in[7] (irq)
An interface to configure the interrupt descriptor table is provided by
idt.h. In kernel.c, I initialize the idt and load the idtr with idt_init().
Interrupt handlers can be dynamically registered with the idt_register_interrupt_handler function in idt.h.
The interrupt_handler function is responsible for calling the registered interrupt handler for the raised interrupt.
In kernel.asm, I've reinitialized the 8259 master PIC to map IRQs to IDT entries starting at 0x20. This is because the processor has already reserved earlier IDT entries for CPU exceptions. For example, now, if the 8259 has IRQ 0 raised, it will call the entry at index 0x20 in our IDT.
From kernel.c:
struct paging_desc *kernel_pages = 0;
kernel_pages = init_page_tables(PAGING_READ_WRITE | PAGING_PRESENT | PAGING_USER_SUPERVISOR);
paging_switch(kernel_pages);
enable_paging();
An interface to interact with the paging data structures and to enable paging is provided by paging.h. Currently, paging for kernel land is configured so that every linear (and virtual) address has a 1:1 relationship with the corresponding physical address. E.g. the linear address 0x20 maps to physical address 0x20. There's no concept of "high memory" for the kernel at the moment, and no split in the virtual address space between user land and kernel land.
In 32-bit x86 with 4-KByte pages, a two level paging scheme is used. Each linear address maps an index into the page directory, and index into the corresponding page table, and the byte offset within the page.
I'll talk about paging for user processes under "Processes and Tasks".
An interface to generate heaps, which manage a pool of memory (technically virtual address space), is defined in heap.h and the allocation algorithm is described in the heap README.
Since I want all kernel processes to use the same heap, kernel_heap.h provides the kernel code with a heap to use for allocating memory.
io.h provides an interface to interact with external hardware via the emulated system's I/O bus. This is crucial for interacting with external hardware like the hard disk drive.
disk.h provides structures to represent attached disks and to
read blocks from a given disk. The entry point to search for disks and generate the needed structures is disk_search_and_init().
However, dynamic disk mounting isn't available yet,
and therefore the code expects to have only one disk formatted for the FAT filesystem attached.
To make life easier, disk_stream.h allows us to read and write arbitrary sizes from disks by providing a disk stream interface. This is the foundation for filesystem drivers.
Much like how Linux provides a virtual file system independent of the underlying
filesystem implementation, file.h allows for filesystems to be dynamically inserted at runtime
(see fs_insert_filesystem). These dynamically inserted filesystems must conform to the
filesystem struct. This means they must provide several functions like fs_open, resolve, fs_read, etc.
The resolve function is especially important: when initializing disks, the disk code will attempt to resolve each
disk against a given filesystem. Thus, each filesystem must implement this resolve function so that the disk layer can pair
a filesystem implementation to a given disk.
The FAT filesystem is the only filesystem I've implemented so far. See fat16.h.
One important thing to remember with FAT16: Filename lengths have a strict limit: 8 characters for everything
before the ., and 3 characters for the extension. Forgetting this fact has caused me lots of pain numerous times.
The GDT is loaded into memory and initially configured with Code and Data segments in boot.asm. However, since the GDT must be modified again down the road, and interface for interacting with the GDT is provided in gdt.h. Much like the Linux Kernel, ConiferOS doesn't make explicit use of segmentation, and instead has kernel/user code and data segments share the entire address space.
A task is the CPU schedulable unit in ConiferOS. The task structure is defined in task.h. Each task has it's own set of page tables. All runnable tasks are stored together in a linked list structure. The task structure is also responsible for maintaing a task's hardware context; when we begin execution of a task, we load the values from the registers structure into the hardware registers before calling `iretd`` to drop into userland.
A task is owned by a process. For now, a process only owns a single task, but the intent was that a process can eventually own multiple process. The process structure is defined in process.h.
When a user loads an executable with process_load, a process image will be created from the specified executeable (filename).
Binary executables and ELF executables are supported. When the kernel initializes a process, it will allocate space for the executable and the stack,
and will map that memory into the page tables of the task that the process encapsulates. For example, see process_map_task_memory in process.c.
Eventually, I want to move away from the process abstraction. The Linux Kernel does not differentiate between processes and tasks, and I do not want to either.
The function process_load and task_exec is used to move the CPU into user mode.
To enter user mode, a process must be loaded with process_load. This will allocate memory for the executable and stack (see process_load_data).
process_load will also create and initialize a task structure. In task_init (task.h),
we set the hardware context as follows:
task->registers.ip = TASK_LOAD_VIRTUAL_ADDRESS;
if (process->format == ELF) {
struct elf32_ehdr *elf32_ehdr = elf_get_ehdr(process->elf_file);
task->registers.ip = elf32_ehdr->e_entry;
}
task->registers.ss = USER_DATA_SEGMENT;
task->registers.cs = USER_CODE_SEGMENT;
task->registers.esp = TASK_STACK_VIRT_ADDR;
Then, after process_load has initialized the task instance, it will modify the page tables of the task instance to
map these addresses to the memory that the process has allocated. See process_map_task_memory.
Once the task is ready to execute, task_exec will swap the task's page tables into the processor's current execution state
by modifying the CR3 register, and then will move the task's saved registers' values into the hardware registers, and then
will call iretd to drop into userland. This will set the code segment and data segment selector registers to map the user code and user data
segments. The values for these segments are defined in kernel.c (see /* Set up the GDT */).
When we switch to the task's page tables, before we call iretd, we can still
access the correct kernel code memory because in task_init, we do a one to one mapping of physical memory to virtual addresses.
However, at that point, we can only read the kernel memory, not write it.
The interrupt 0x80 is used to communicate with the kernel from userland. The userland program will put a command ID into the eax register, and push arguments onto the stack. Given the command ID, the kernel will know which operation (system call) to perform.
The ISR that handles 0x80 is isr80h_wrapper, which is defined in idt.asm. The wrapper saves the state of the user program's
general purpose registers, and then calls isr80h_handler. The definition for this function is in idt.h. This function finishes converting
the processor into kernel mode by loading the kernel page tables into memory, and then dispatches the appropriate system call, given the command id that
was passed to the eax register from the userland program.
An interface for kernel code to register systems calls, or kernel commands, is provided by isr80h_register_command, which is declared in idt.h. Right now, the only point at which I register system calls is isr80h_register_commands in isr80h.h.
Misc. kernel commands are stored in src/isr80h/misc.h, and IO related kernel commands are stored in src/isr80h/io.h.
Memory related system call are declared in src/isr80h/heap.h.
User programs are stored in the user_programs folder. The example programs here test the stdlib and ConiferOS system calls.
The stdlib folder contains our C standard library, stdlib.elf.
The stdlib contains useful functions for managing heap allocated memory, getting keyboard input, printing, etc.
It also contains an entry point to C user programs, start, which which defines the _start symbol and is responsible for calling the main function.
See user_programs/test_stdlib for a program which uses the stdlib.
User programs can launch other programs and pass them arguments by using the ConiferOS exec system calls. As an example, see the following code from user_programs/test_stdlib/test_stdlib.c:
char path_buf[1024];
strcpy(path_buf, "0:/echo.elf");
char *args[] = {
"first_argument",
NULL
};
coniferos_execve(path_buf, args, /*argc*/1);
I implemented the "echo" user space program to showcase command line arguments.
Each process structure has a keyboard_buffer, which is defined in process.h. An interface for interacting with a process's keyboard
buffer is defined in keyboard.h. The keyboard_push function will push a character to the keyboard buffer of the current process,
while keyboard_pop pops a character from the current task's process's keyboard buffer.
A userland program can invoke the system call SYSTEM_COMMAND_2_GET_KEY_PRESS via interrupt 0x80, to pop a character off the current process's keyboard buffer.
The handler for this system call is defined in isr80h/io.h.
Qemu emulates an Intel 8042 chip, which is the PS/2 Controller. For more info on this chip, see "'8042' PS/2 Controller" on OSDev. This is the chip that controls communication between the PS/2 Keyboard and the CPU.
If you open the qemu monitor and run info qtree, you'll notice that attached to the PCI bus is an ISA bridge
which supports legacy ISA devices by translating PCI I/O and PCI Memory space accesses into ISA I/O and ISA Memory accesses. Then, attached to the ISA bus,
is the i8042 device, which is our Intel 8042 chip.
dev: PIIX3, id ""
addr = 01.0
romfile = ""
romsize = 4294967295 (0xffffffff)
rombar = 1 (0x1)
multifunction = true
x-pcie-lnksta-dllla = true
x-pcie-extcap-init = true
failover_pair_id = ""
acpi-index = 0 (0x0)
class ISA bridge, addr 00:01.0, pci id 8086:7000 (sub 1af4:1100)
bus: isa.0
type ISA
dev: port92, id ""
gpio-out "a20" 1
dev: vmmouse, id ""
dev: vmport, id ""
x-read-set-eax = true
x-signal-unsupported-cmd = true
x-report-vmx-type = true
x-cmds-v2 = true
vmware-vmx-version = 6 (0x6)
vmware-vmx-type = 2 (0x2)
dev: i8042, id ""
gpio-out "a20" 1
extended-state = true
kbd-throttle = false
isa irqs 1,12
The PS/2 driver is in the works. IRQ 1 is typically raised by a keyboard device. We've mapped IRQ 1 to IDT entry 0x21.
As mentioned in "Processes and Tasks", ConiferOS contains an ELF loader which can be used to instantiate user processes from ELF executables. The loadable segments of the ELF file will be mapped into the address space of the user processes.
The ELF loader code can be found in src/loader/formats.
Currently, the ELF loader only supports executable files (e_type = ET_EXEC). Thus, the ELF loader
does not support relocatable files or dynamic shared libraries.
ELF Specification that was used as reference: https://refspecs.linuxfoundation.org/elf/elf.pdf
There are several things I want to experiment with:
- A networking stack. The emulator provides a network card to interact with.
- A dynamic device driver layer so that the kernel can support interacting with different hardware by loading in different drivers at runtime.
- Making the heap allocation algorithm more efficient. Right now, it's a fragmented mess.
- I'd like to implement an ext filesystem.
- The Kernel code disables interrupts because there aren't any safety mechanisms to support interleaved execution of functions. I'd like to implement some synchronization and concurrency primitives to support this.
- A PCI driver so I can dynamically find attached devices. Then I don't have to guess I/O port addresses.
mmapfunction to map files into memory.