Day 15 was about File System.

The book that I am following has come to an end with File System as the final chapter where it teaches to implement tar file as a filesystem.

tar is an archive format (like zip) that can contain multiple files. It contains file contents, filenames, creation dates, and other information necessary for a filesystem. Compared to common filesystem formats like FAT or ext2, tar has a much simpler data structure.

First a simple directory with two files is made as an example which is then archived/compressed into the tar format using the command line tar tool. Which, also is added to the buildscript before launching the qemu VM.

(cd disk && tar cf ../disk.tar --format=ustar *.txt)

# The following flag is also added to qemu.
-drive id=drive0,file=disk.tar,format=raw,if=none \

To read the filesystem, first, the data structures related to the tar filesystem is defined in kernel.h. This implementation has all files read from the disk at boot.

To write to the disk, the contents of the files variable is written back to the disk in the tar file format. The files variable contains the copy of the disk variable declared as a static variable (It is a temporary buffer and, since the stack has a limited size it is preferred to use static variable instead of local stack variable.) which contains the disk image.

File read/write system calls are then designed in the user.c and user.h files and then the system calls are implemented on the kernel files and then the system calls are implemented in the kernel. We also add the read and write commands to the shell.

But, we still have to address the user pointers.

In RISC-V, the behaviour of S-Mode (kernel) can be configured through sstatus CSR, including SUM (permit Supervisor User Memory access) bit. When SUM is not set, S-Mode programs (i.e. kernel) cannot access U-Mode (user) pages.

All we need to do is to set the SUM bit when entering user space and voila! Our OS with a filesystem implementation is now ready.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 14 was about Disk I/O.

A device driver for the virtio-blk, a virtual disk device was implemented.

Virtio is one of the APIs for device drivers to control devices. It is widely used in virtualization environments such as QEMU. The virtio devices have a structure called virtqueue and it is a queue shared between the driver and the device. It consists of Discriptor Table, Available Ring and Used Ring.

First we enable virtio drivers in our buildscript which is run.sh. We use the following flags for qemu:

    -drive id=drive0,file=lorem.txt,format=raw,if=none
    -device virtio-blk-device,drive=drive0,bus=virtio-mmio-bus.0

Here, lorem.txt is a file made for testing purposes beforehand.

Then, the definition of c structs and macros is done in kernel.h. To access the MMIO registers we add that functionality in kernel.c.

The create_process function is modified to map the virtio-blk MMIO region to the page table so the kernel can access the MMIO registers. MMIO region to the page table so the kernel can access the MMIO registers.

Then the Virtio Device Initialization and Virtqueue Initialization is done with the process described in the spec.

I/O requests to the disk is implemented by "adding processing requests to the virtqueue".

A request is sent in the following steps:

1. Construct a request in blk_req. Specify the sector number you want to access and the type of read/write.
2. Construct a descriptor chain pointing to each area of blk_req (see below).
3. Add the index of the head descriptor of the descriptor chain to the Available Ring.
4. Notify the device that there is a new pending request.
5. Wait until the device finishes processing (aka busy-waiting or polling).
6. Check the response from the device.

To test, we initialize the virtio device in the kernel main and try reading and writing to the disk with the implemented functions.

Device drivers are just "glue" between the OS and devices. Drivers don't control the hardware directly; drivers communicate with other software running on the device (e.g., firmware). Devices and their software, not the OS driver, will do the rest of the heavy lifting, like moving disk read/write heads.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 13 was about system call.

Similar to SBI Call implementation, the system calls are invoked in a similar way.

The syscall function sets the system call number in the a3 register and the system call arguments in the a0 to a2 registers, then executes the ecall instruction. The ecall instruction is a special instruction used to delegate processing to the kernel. When the ecall instruction is executed, an exception handler is called, and control is transferred to the kernel. The return value from the kernel is set in the a0 register.

The first systemcall implemented is putchar. It takes a character as the first argument and the second and subsequent unused arguments are set to 0.

Then the ecall instruction is handled in the kernel. The calling of ecall can by determined by checking the value of scause. Before calling the handle_syscall function we also add the size of ecall instruction (4) to the value of sepc.

This is because sepc points to the program counter that caused the exception, which points to the ecall instruction. If we don't change it, the kernel goes back to the same place, and the ecall instruction is executed repeatedly.

Then a system call handler is made and is called from trap handler. It receives a structure of registers at the time of exception that was saved in the trap handler. It determined the type of system call by checking the value of a3 register.

The system call was then tested with a main print function in shell.c.

An exit system call is also implemented where it takes the first argument as 3 which is defined as SYS_EXIT.

The system call changes the process state to PROC_EXITED, and calls yield to give up the CPU to other processes. The scheduler will only execute processes in PROC_RUNNABLE state, so it will never return to this process. However, PANIC macro is added to cause a panic in case it does return.

A bare-bones shell was then implemented to use the syscalls to perform actions from the user mode.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 12 was about user mode.

The application that we had made had to be run on the user mode. And since, the execution image is a raw binary, it needs to be prepared with a fixed binary.

#define USER_BASE 0x1000000

After defining the symbols to use in the embedded raw binary, the create process function is updated to start the application from the user_entry.

The create_process is modified to take the pointer to the execution image and image size as arguments. It copies the execution image page by page for the specified size and maps it to the process page table.

Finally, the create_process function is modified to create a user process.

After checking if the execution image is mapped as expected, to run applications we must transition the CPU to the user mode. Or, in RISC-V, the U-Mode.

This switch from S-Mode to U-mode is done with the sret instruction. It does two writes to CSRs while switching:

• PC is set for when transitioning in U-Mode in the sepc register where sret jumps to.
• Then, setting SPIE bit in the sstatus register enables hardware interrupts and the handler set in the stvec register will be called while entering the U-Mode.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 11 was about application.

First, a linker script named user.ln was made this time starting at address 1000000 so that the application doesn't overlap with the kernel space.

Then a simple userland library is created with minimal features just enough to indicate the existence of a userland application. A header file for the userland library is also made.

Then an application is made, very barebones, it is just an infinite loop for now since there is no way to print characters at the moment.

To build the application, we first compile it with cc to get an executable in a ELF format. The executable is then converted to raw binary format with the objcopy tool. Then the raw binary executable format is then again converted to a format that can be embedded in the C language.

Lastly, the shell.bin.o output is then passed to clang (kernel build section of the script) which gets embedded into the kernel image.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 10 was about testing and debugging paging and page table contents.

Running the buildscript should give us the exact same output of repeating letters similar to how it was before paging was implemented.

Then the page table contents were examined. To check about the registers, following command was run on the qemu console.

(qemu) info registers

Then the value of satp register is read and then interpreted by doing the following hexadecimal math:

hex((0x<val of satp> & 0x3fffff) * 4096)

Then the VPN[1] and VPN[0] physical addressed were also examined.

The consequences of forgetting to set the paging mode, specifying physical address instead of physical page number was also seen. qemu logs were also enabled.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 9 was about memory management and virtual addressing.

The structure of the virtual address is defined by the RISC-V paging mechanism called Sv32. It uses a two level page table where the 32 bit virtual address is divided into a first-level page table index, a second level and a page offset. The tables are named VPN[1] and VPN[0] respectively.

First the macros for the construction of the page table are defined. Then the function to map pages map_page is made which is utilized in process creation. For that we also add an element called page_table on the struct of process. And to make everything work we define the starting address of the kernel space __kernel__base in the linker script just after boot.

To utilize the above setup of page tables and switch them, we specify the first-level page table in satp (Supervisor Address Translation and Protection) register.

I still need to understand more about this. I am not satisfied with the explanation from the resource.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 8 was about Process. Though, modern operating systems use the concept of threads to provide execution context. We treat processes like individual threads being run for our implementation.

First we define a structure for PCB (Process Control Block). We first define PROC_UNUSED and 0 and PROC_RUNNABLE as 1. Then we create a structure for a unit process with the following slots:

• Process ID as an Integer.
• State (PROC_UNUSED or PROC_RUNNABLE) as an Integer as defined before.
• Stack Pointer as a vaddr_t (Virtual Address Type).
• Kernel Stack as a list of unsigned 8bit Integers.

The Kernel Stack is essential for saving the registers while context switching.

Then, we defined the switch_context function that does the context switching. It takes the previous and next stack pointer as arguments and then switches them during execution. It saves the callee-saved registers into the stack, switches the stack pointer then restores the callee-saved registers from the stack. The execution context is saved as a temporary local variable on the stack.

Then, we work on process creation. The process creation function create_process takes in the entry point of the process as a parameter and then returns the process struct.

The delay function is also created to act as a sleep style function which just does nop (Nothing) for 30000000 clock pulses.

Then a scheduler is made to make the context switching more autonomous. A scheduler is basically a Kernel program which decides the next process.

Then, in the exception handler, we make it so that each process has it's own independent kernel stack. While switching, the contents of sscratch are switched too to resume the execution of process from where it was interrupted as if nothing had happened.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 7 was about Memory Allocation. First the memory regions were defined in the linker script so that it can determine the position to avoid overlapping the memory to kernel's static data.

The size of the memory space was 64 * 1024 * 1024 bytes or 64MB and it is aligned to a 4KB boundary.

Then a function alloc_pages was implemented which allocated n pages of memory and returned it's starting address.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Day 6 was about Exceptions and handling those Exceptions in the kernel.

In RISC-V the CPU first checks medeleg register to determine which operation mode should handle the exception. In our case U-mode/S-mode is already handled by OpenSBI. Then, the CPU saves states into various CSRs. stvec register is set to pc then the exception is handled using the handler. Then sret is called to resume execution from the point where exception occurred.

The handle_trap function reads why the exception occurred and triggers the kernel panic. Which was implemented yesterday.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

Perfect topic for today because early morning I was shown why It was a stupid decision to do something, then at afternoon I got into a bike accident. Though it was minor with no injuries to both parties except some scratches on the bikes. It could have caused huge consequences. Then I did another mistake and then another.

PANIC!

It is day 5. Implementing panic was very easy, it is implemented as a macro because if we defined it as a function it would have printed the __LINE__ and __FILE__ where PANIC is defined and not where it is called. And to halt the kernel it uses a while true loop which goes on infinitely.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

To make a useful kernel we require it to perform tasks like setting a memory chunk to zero, or copying the memory from a source address to destination address. It may have to understand what a boolean is or what NULL is. So, for that we require the "C Standard Library" and hence we need to implement it (or the functions it contains as abstractions to commonly used functions).

For this, in day 4 I utilized clang's internal C library as well as resorted to writing the functions all by myself.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

As the days progress the journey is getting more fun and interesting. It was the part where actual text was going to be printed. It was a kernel hello-world.

Day 3 focused on printing characters on the console by talking to SBI finally implementing putchar and eventually printf.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

The actual development of OS started with making the build script run.sh in bash to setup and launch a qemu RISC-V virtual machine with OpenSBI as bios for QEMU.

Then a linker script kernel.ld was made and a basic kernel process kernel.c was written. Finally, the build script was updated to include compiler and execution commands and flags to properly boot the kernel.

Day 2 focused on writing a basic starting process of a kernel and running it in a virtual machine.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

I have set up the environment required to run RISC-V 64 toolchain using QEMU.

Installed a debian RISC-V 64 iso and set it up to my liking with autologin and proper terminal emulation and sudo. I then installed gcc, gdb and neovim for writing and compiling programs.

Details of what happened on Day 1 is tracked using git with codeberg as a remote.

I have decided to use Operating System in 1000 Lines as the learning material because it was featured as a beginner level learning material in github:riscv/learn.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.

I want to learn systems programming and I thought why not go with the open source one which is currency in development. So, I chose the RISC-V architecture. That way if I decide to keep pursuing it, I can grow along with the ISA slowly catching up to it.

I want to keep a streak by posting here every day about what I learn. Let's hope it goes well.

You might be wondering how I know that you're awesome!
Because I know you read this via RSS.