Writing A Linux Style Operating System From Scratch
Post Stastics
- This post has 2299 words.
- Estimated read time is 10.95 minute(s).
In Chapter 28, we cleaned up the user-program build system:
USER_PROGRAMS := demo counter
now automatically builds:
build/user/demo.elf build/user/counter.elf build/user/demo_elf_blob.o build/user/counter_elf_blob.o
But our user programs still duplicate small helper functions:
str_len() write_str() write_uint()
Both demo.c and counter.c contain nearly the same code.
This chapter adds the first tiny Toyix userland support library:
user/include/toyix.h user/lib/toyix.c
This is not a real libc yet, but it is the beginning of one.
After this chapter, user programs can simply call:
toyix_puts("hello");
toyix_write_uint(123);
toyix_putchar('\n');
The runtime behavior stays the same:
Program registry: registered 2 embedded program(s) Program test: starting background counter test counter: argc=3 counter: argv[0]=counter counter: argv[1]=alpha counter: argv[2]=beta counter: tick 1 counter: tick 2 counter: tick 3 Syscall: process counter pid=1 exited code 4 Program test: background counter cleanup sanity check passed
This is mostly a userland cleanup chapter.
1. What this chapter adds
Add:
user/
├── include/
│ └── toyix.h
└── lib/
└── toyix.c
Modify:
user/demo.c user/counter.c Makefile tests/smoke.sh
No kernel changes are required.
No syscall ABI changes are required.
2. Current problem
demo.c and counter.c both have local versions of:
static toyix_u32 str_len(const char *text); static void write_str(const char *text); static void write_uint(toyix_u32 value);
That was fine while we only had one or two programs.
But it will become annoying quickly.
We want user programs to look more like this:
#include "toyix.h"
int main(int argc, char **argv) {
toyix_puts("hello from userland");
return 0;
}
The syscall wrappers stay in:
user/include/toyix_syscall.h
Higher-level user helpers go in:
user/include/toyix.h user/lib/toyix.c
3. Add user/include/toyix.h
#ifndef TOYIX_USER_TOYIX_H #define TOYIX_USER_TOYIX_H #include "toyix_syscall.h" typedef unsigned int toyix_size_t; toyix_size_t toyix_strlen(const char *text); void toyix_putchar(char ch); void toyix_write_str(const char *text); void toyix_puts(const char *text); void toyix_write_uint(toyix_u32 value); void toyix_write_int(toyix_i32 value); int toyix_streq(const char *a, const char *b); #endif
Why not call these strlen, puts, and printf?
Because this is not a full libc yet.
For now, using a toyix_ prefix avoids pretending we have standard C library compatibility.
Later, we can decide whether to provide libc-like names.
4. Add user/lib/toyix.c
#include "toyix.h"
toyix_size_t toyix_strlen(const char *text) {
toyix_size_t len = 0;
if (text == 0) {
return 0;
}
while (text[len] != '\0') {
len++;
}
return len;
}
void toyix_putchar(char ch) {
toyix_write(FD_STDOUT, &ch, 1);
}
void toyix_write_str(const char *text) {
if (text == 0) {
return;
}
toyix_write(FD_STDOUT, text, (toyix_u32)toyix_strlen(text));
}
void toyix_puts(const char *text) {
toyix_write_str(text);
toyix_putchar('\n');
}
void toyix_write_uint(toyix_u32 value) {
char buffer[11];
toyix_u32 index = 0;
if (value == 0) {
toyix_putchar('0');
return;
}
while (value > 0 && index < sizeof(buffer)) {
buffer[index++] = (char)('0' + (value % 10u));
value /= 10u;
}
while (index > 0) {
toyix_putchar(buffer[--index]);
}
}
void toyix_write_int(toyix_i32 value) {
if (value < 0) {
toyix_putchar('-');
/*
* Avoid relying on undefined behavior for INT_MIN by converting
* through unsigned arithmetic.
*/
toyix_u32 magnitude = (toyix_u32)(-(value + 1)) + 1u;
toyix_write_uint(magnitude);
return;
}
toyix_write_uint((toyix_u32)value);
}
int toyix_streq(const char *a, const char *b) {
if (a == 0 || b == 0) {
return a == b;
}
while (*a != '\0' && *b != '\0') {
if (*a != *b) {
return 0;
}
a++;
b++;
}
return *a == *b;
}
This gives user programs a small common base.
5. Update user/demo.c
Replace demo.c with this smaller version:
#include "toyix.h"
int main(int argc, char **argv) {
char buffer[32];
toyix_write_str("argc=");
toyix_write_int(argc);
toyix_putchar('\n');
for (int i = 0; i < argc; ++i) {
toyix_write_str("argv[");
toyix_write_int(i);
toyix_write_str("]=");
toyix_puts(argv[i]);
}
toyix_write_str("user> ");
toyix_i32 got = toyix_read(FD_STDIN, buffer, sizeof(buffer));
if (got < 0) {
toyix_puts("read failed");
return 1;
}
toyix_write_str("echo: ");
if (got > 0) {
toyix_write(FD_STDOUT, buffer, (toyix_u32)got);
}
toyix_putchar('\n');
toyix_sleep(3);
return 9;
}
The behavior is unchanged.
The source is cleaner.
6. Update user/counter.c
Replace counter.c with this smaller version:
// user/counter.c
#include "toyix.h"
int main(int argc, char **argv) {
toyix_write_str("counter: argc=");
toyix_write_int(argc);
toyix_putchar('\n');
for (int i = 0; i < argc; ++i) {
toyix_write_str("counter: argv[");
toyix_write_int(i);
toyix_write_str("]=");
toyix_puts(argv[i]);
}
for (toyix_u32 i = 1; i <= 3; ++i) {
toyix_write_str("counter: tick ");
toyix_write_uint(i);
toyix_putchar('\n');
toyix_sleep(2);
}
return 4;
}
Again, behavior is unchanged.
But now counter.c is focused on the program’s logic.
7. Update Makefile user variables
Add a user library source list.
Near the user-program section:
USER_PROGRAMS := demo counter USER_ELFS := $(USER_PROGRAMS:%=build/user/%.elf) USER_BLOBS := $(USER_PROGRAMS:%=build/user/%_elf_blob.o) USER_LIB_SRCS := user/lib/toyix.c USER_LIB_OBJS := $(USER_LIB_SRCS:user/lib/%.c=build/user/lib/%.o)
This gives us:
USER_LIB_OBJS = build/user/lib/toyix.o
8. Add a build directory rule for build/user/lib
Add:
build/user/lib:
mkdir -p build/user/lib
Keep the existing:
build/user:
mkdir -p build/user
9. Add a pattern rule for user library objects
Add:
build/user/lib/%.o: user/lib/%.c user/include/toyix.h user/include/toyix_syscall.h | build/user/lib
$(CC) $(USER_CFLAGS) -c $< -o $@
This compiles:
user/lib/toyix.c
into:
build/user/lib/toyix.o
10. Update the user program object rule
The existing rule probably looks like this:
build/user/%.o: user/%.c user/include/toyix_syscall.h | build/user
$(CC) $(USER_CFLAGS) -c $< -o $@
Replace it with:
build/user/%.o: user/%.c user/include/toyix.h user/include/toyix_syscall.h | build/user
$(CC) $(USER_CFLAGS) -c $< -o $@
Now programs rebuild if the support library header changes.
11. Update the ELF link rule
The old rule links only:
crt0.o program.o
We now need:
crt0.o program.o user library objects
Replace:
build/user/%.elf: build/user/crt0.o build/user/%.o user/linker.ld | build/user
$(CC) $(USER_LDFLAGS) \
-Wl,-Map,build/user/$*.map \
build/user/crt0.o build/user/$*.o \
-o $@
with:
build/user/%.elf: build/user/crt0.o build/user/%.o $(USER_LIB_OBJS) user/linker.ld | build/user
$(CC) $(USER_LDFLAGS) \
-Wl,-Map,build/user/$*.map \
build/user/crt0.o build/user/$*.o $(USER_LIB_OBJS) \
-o $@
Now every user program links against the tiny support library.
12. Updated user build section
The user portion of the Makefile should now look roughly like this:
USER_PROGRAMS := demo counter
USER_ELFS := $(USER_PROGRAMS:%=build/user/%.elf)
USER_BLOBS := $(USER_PROGRAMS:%=build/user/%_elf_blob.o)
USER_LIB_SRCS := user/lib/toyix.c
USER_LIB_OBJS := $(USER_LIB_SRCS:user/lib/%.c=build/user/lib/%.o)
USER_CFLAGS := \
-std=gnu11 \
-ffreestanding \
-fno-builtin \
-fno-stack-protector \
-fno-pic \
-fno-pie \
-fno-asynchronous-unwind-tables \
-fno-unwind-tables \
-m32 \
-march=i686 \
-O2 \
-Wall \
-Wextra \
-Iuser/include
USER_LDFLAGS := \
-nostdlib \
-ffreestanding \
-m32 \
-Wl,-T,user/linker.ld \
-Wl,--build-id=none
OBJCOPY ?= i686-elf-objcopy
build/user:
mkdir -p build/user
build/user/lib:
mkdir -p build/user/lib
build/user/crt0.o: user/crt0.S | build/user
$(CC) $(USER_CFLAGS) -c $< -o $@
build/user/lib/%.o: user/lib/%.c user/include/toyix.h user/include/toyix_syscall.h | build/user/lib
$(CC) $(USER_CFLAGS) -c $< -o $@
build/user/%.o: user/%.c user/include/toyix.h user/include/toyix_syscall.h | build/user
$(CC) $(USER_CFLAGS) -c $< -o $@
build/user/%.elf: build/user/crt0.o build/user/%.o $(USER_LIB_OBJS) user/linker.ld | build/user
$(CC) $(USER_LDFLAGS) \
-Wl,-Map,build/user/$*.map \
build/user/crt0.o build/user/$*.o $(USER_LIB_OBJS) \
-o $@
build/user/%_elf_blob.o: build/user/%.elf | build/user
$(OBJCOPY) \
-I binary \
-O elf32-i386 \
-B i386 \
--rename-section .data=.rodata.user_$*,alloc,load,readonly,data,contents \
--redefine-sym _binary_build_user_$*_elf_start=user_$*_elf_start \
--redefine-sym _binary_build_user_$*_elf_end=user_$*_elf_end \
$< $@
user-programs: $(USER_LIB_OBJS) $(USER_ELFS)
user-blobs: $(USER_BLOBS)
list-user-programs:
@echo "$(USER_PROGRAMS)"
readelf-user: $(USER_ELFS)
@for elf in $(USER_ELFS); do \
echo "==== $$elf ===="; \
i686-elf-readelf -h $$elf | grep -E "Class:|Data:|Type:|Machine:|Entry point"; \
i686-elf-readelf -l $$elf | grep LOAD; \
done
13. Check for accidental libc calls
Because we now have a library source file, it is easy to accidentally introduce normal libc calls.
Avoid these in userland:
strlen memcpy printf puts exit malloc
Use Toyix helpers instead:
toyix_strlen toyix_write_str toyix_puts toyix_exit
A quick check:
i686-elf-nm -u build/user/demo.elf i686-elf-nm -u build/user/counter.elf
For this stage, you generally want no unresolved symbols except any toolchain-specific oddities you intentionally support.
Ideally, the output should be empty.
14. Inspect the user ELFs
Run:
make user-programs make readelf-user
You should still see:
ELF32 Intel 80386 EXEC Entry point 0x40100000
The segment sizes may grow slightly because both programs now link with toyix.o.
That is expected.
Your greps should not depend on exact filesz or memsz.
15. Runtime tests remain the same
The Chapter 27 and Chapter 28 runtime greps should still pass.
Keep:
grep -q "Program registry: registered 2 embedded program(s)" build/test.log
grep -q "demo - interactive stdin/stdout demo" build/test.log
grep -q "counter - background-safe counter demo" build/test.log
grep -q "Program test: starting background counter test" build/test.log
grep -q "Program: launching counter argc=3" build/test.log
grep -q "Process: created pid=1 name=counter" build/test.log
grep -q "counter: argc=" build/test.log
grep -q "counter: argv\\[0\\]=counter" build/test.log
grep -q "counter: argv\\[1\\]=alpha" build/test.log
grep -q "counter: argv\\[2\\]=beta" build/test.log
grep -q "counter: tick 1" build/test.log
grep -q "counter: tick 2" build/test.log
grep -q "counter: tick 3" build/test.log
grep -q "Syscall: process counter pid=1 exited code 4" build/test.log
grep -q "Process: destroyed pid=1 name=counter" build/test.log
grep -q "Program test: background counter cleanup sanity check passed" build/test.log
Add build-artifact checks:
@test -f build/user/lib/toyix.o
@test -f build/user/demo.elf
@test -f build/user/counter.elf
16. Update tests/smoke.sh
No structural change is needed.
#!/usr/bin/env bash set -euo pipefail make clean make test make test-exception make test-page-fault echo "All Chapter 29 checks passed."
17. Expected output
Runtime output should remain equivalent to Chapter 27 and Chapter 28:
Program registry: registered 2 embedded program(s) Embedded programs: demo - interactive stdin/stdout demo counter - background-safe counter demo Program test: starting background counter test Address space: created process page directory ELF32: loaded PT_LOAD vaddr=0x40100000 ... ELF32: entry=0x40100000 Process: initial stack argc=3 esp=0x6FFFF... Program: launching counter argc=3 Thread: created counter id=... Process: created pid=1 name=counter Program test: background pid=1 PID STATE EXIT NAME 1 running - counter counter: argc=3 counter: argv[0]=counter counter: argv[1]=alpha counter: argv[2]=beta counter: tick 1 counter: tick 2 counter: tick 3 Syscall: process counter pid=1 exited code 4 Threads: reaping zombie counter id=... PID STATE EXIT NAME 1 exited 4 counter Address space: destroyed process page directory, user pages=... tables=... Process: destroyed pid=1 name=counter Program test: background counter cleanup sanity check passed
The segment sizes may be larger now. That is fine.
18. Interactive check
After boot:
toyix> run counter one two
Expected:
Program: launching counter argc=3 Process: created pid=... counter: argc=3 counter: argv[0]=counter counter: argv[1]=one counter: argv[2]=two counter: tick 1 counter: tick 2 counter: tick 3 Syscall: process counter pid=... exited code 4 Process: destroyed pid=... name=counter run: counter exited code 4
Then:
toyix> run demo hello world
Expected:
argc=3 argv[0]=demo argv[1]=hello argv[2]=world user>
Type a line and press Enter.
19. Common failures
Failure: undefined reference to toyix_write_str
Check that the ELF link rule includes:
$(USER_LIB_OBJS)
The link command must include:
build/user/crt0.o build/user/$*.o $(USER_LIB_OBJS)
not only:
build/user/crt0.o build/user/$*.o
Failure: toyix.h not found
Check:
-Iuser/include
in USER_CFLAGS.
Also verify the include path in user programs:
#include "toyix.h"
Failure: library object directory missing
Make sure this rule exists:
build/user/lib:
mkdir -p build/user/lib
and the library object rule has:
| build/user/lib
as an order-only prerequisite.
Failure: user program accidentally links against host libc
Make sure USER_LDFLAGS includes:
-nostdlib
and user code does not call standard library functions.
Failure: segment layout changes and loader rejects ELF
The support library can change the section sizes. Usually this is fine.
But check:
make readelf-user
If a PT_LOAD segment has a non-page-aligned virtual address, our current ELF loader may reject it.
The current user/linker.ld should keep major sections page-aligned.
20. What this chapter achieved
Before this chapter:
demo.c and counter.c duplicated basic output helpers
After this chapter:
user/lib/toyix.c provides shared user helpers user/include/toyix.h declares user helpers demo.c and counter.c focus on program behavior
This is the first small step toward a Toyix userland C library.
21. Design limitations
This is still not libc.
Missing:
memcpy memset strcmp atoi printf malloc errno open/read/write/close wrappers with conventional names startup environment helpers file descriptor abstractions
But the structure is now in place.
Future user programs can share helper code instead of copying it.
Resources
- Chapter source: Toyix repository
- Chapter release: Chapter_29
Closure
Chapter 29 gives Toyix its first shared userland support library. The kernel behavior stays the same, but user programs now have a cleaner base to build on before the next runtime features arrive.
Happy Coding!