u/compgeek38400

A recent post got me thinking. Doing paging right requires a file system of some sort.

Do you think the paging file system belongs in user space, or kernel space. I am quick coming up on making such a decision myself

Continuing my series for those who are interested, if any. My source is here: https://github.com/tedavids/DragonOS

This was a long week, the first thing I learned put me back several days. Those of you following know this week I was finishing up my heap, allowing multi-page allocations.

This is what I learned:

1. Visual Studio Code when you 'Rename Symbol' it changes it EVERYWHERE in your project. This caused me to have to go back and redo code, and eventually just copy it back from Git, and start over.

2. I can't keep the whole project in my head anymore. I don't know if this is because of age, or because of the complexity of writing an OS. I hope it's the latter 😂

3. I'm going to have to take a couple of weeks to bring all my doco up to date.

I also have some questions:

Now that I have a working heap, I have lots of options on what to do next, these are some of the candidates:
a) multithreading -- I know this will entail at least creating atomic operations (atomic_t), sequencing stuff (mutex, semaphore), and retrofitting what I have to use them BEFORE I add this. What did I miss?

b) add a swap file system, I know this will involve writing a disk driver, should I do IDE/ATA or SCSI? If I did this, I'd be tempted to add both swap and a 'normal' file system.

c) something else?

I occasionally get page faults on startup. How would you debug this? I can't predict it, and never seem to get it when I am trying. How would you debug this? The only thing I can think of is to use line2addr, and hope

Thanks for reading, please chime in on what I should do next. I do know I'm going to spend this week updating doco. I may not change any code this week

In continuing letting those who care (if any) on what I'm learning making an OS, this is what I learned this week. My github link is here (https://github.com/tedavids/DragonOS).

This first thing I learned is the hardest part of writing a heap is deciding on the design. I read and reread the slab stuff found all over the net, and couldn't get my head around it. So I designed my own, when I look at this link (https://www.kernel.org/doc/gorman/html/understand/understand011.html) it looks similar to what I did but not quite. So some of it must have sunk in. This design is only appropriate for allocations under 1 page. I will do the page and above allocations this week. Probably using some sort of AVL binary tree structure, as I won't have to handle partial page allocations.

The second thing I learned is how much I've come to depend on malloc(), until you have a heap, you don't have malloc(). Free lists to the rescue.

Finally I learned how to make sub-bullets under bullet points in git. It makes the github readme easier for me to understand.

Thanks for reading.

I am working on physical memory management for my custom x86 32 bit kernel.

I have 5 functions: mapKernelPage, mapUserPage, unmapPage, setPageReadOnly, and setPageReadWrite.

Here is a code snippet from my kernel:

uint32_t physpage;
    uint32_t address = (uint32_t) mapKernelPage(0xC0158000, &physpage);


    printf("Successfully mapped page: 0x%Xl with physical address: 0x%Xl\n\r",
            address, physpage);


    printf("Testing Read only...");
    if (setPageReadOnly(address)) {
        printf("Success\n\r");
    } else {
        printf("Failed\n\r");
    }


    printf("Page 0x%Xl ", address);
    if (!unmapPage(address)) {
        printf("not unmapped\n\r");
    } else {
        printf("unmapped\n\r");
    }

I am testing via gdb. Before I do the map, my page is unavailable, afterwards it is. (I test my looking at memory), when I unmap the page it is again unavailable.

My issue is that when I set the page read only (it includes an invalidate page), I can still write to the memory.

Here is what I think is the pertinent info:

(gdb) info register cr0

cr0 0x80010011 [ PG WP ET PE ]

CR0 has the write protect bit set

Here is my page table entry:

(gdb) p/x pt[pte]

$7 = 0x1000001

Notice the read/write bit is zero

my page directory entry is:

(gdb) p/x page_directory[pde]

$13 = 0x114023

What am I missing?

TIA

From my reading the Page Table Entry sh ould override the Page Directory entry, what am I missing?

Thanks

I asked AI (I know, you all hate it but it has its place) how much faster a shift 12 bits (x >> 12) is than a divide by 0x1000. It came back 10 to 40 times. But also said modern compilers change a divide by 0x1000 to a shift.

I compiled and looked at the output, and darn if that isn't true.

So my question is, which do you do and why? To my mind either could be better depending on your audience. A divide is probably clearer to people that their first language isn't C (like me). But a shift would probably be clearer to someone that lives in C.

I look forward to both your flames and actual comments.

I've spent this week working on getting paging set up in my kernel. I have two bit arrays availMemMap, which tracks available physical memory, readwrtiteMap, which says what areas of memory are read/write (as opposed to read only, such as where the BIOS resides). I also now have the read/write bit in my page tables set appropriately. .text and .rodata are read only.

My main learnings are two things: a) when you try to write to a read only page, you don't get a GP fault, you get a page fault; and b) just because an area is 'reserved' in the multiboot info, does not necessarily mean it is read only, it might be, but not necessarily.

This coming week, I hope to get my page allocation/deallocation functions tested and working.

You can see my code (if you wish) here: https://github.com/tedavids/DragonOS

I do have a question. How to you allocate pages? Do you allocate kernel pages AFTER the kernel end, growing up. And user pages BEFORE the kernel, growing down? Or some other method? I'd be interested on hearing from those that have done this before.

Thanks for reading.

I have been working on a kernel for an OS, and after a month I have some lessons learned. Meaning what I would do differently if I was starting over from nothing. I'm posting this in the hope that it will hope someone else. I don't claim that this is the only, or even the best way. Just what I'd do if I was starting from scratch.

Firstly, I'd use OSDev.org as my guide. I'd join the forums and only after researching and failing for several days would I post a question. The regulars are really smart, and you don't want to wear out your welcome with questions you could have found in research.

Next I'd read the following sections: Introduction, Required knowledge, Beginner mistakes, Getting started, and how to ask questions. Especially Required knowledge, if you don't have it, get it before you start.

Next set up a Cross compiler (https://wiki.osdev.org/GCC_Cross-Compiler). Its really tempting to use the normal one, but DON'T

Once that is done do the bare bones tutorial. https://wiki.osdev.org/Bare_Bones Understand it then throw it away, and move on to the Meaty Skeleton Tutorial. https://wiki.osdev.org/Meaty_Skeleton

Once you understand it keep it for reference.

Next I'd move my kernel to the Higher Half. Tutorial here https://wiki.osdev.org/Multiboot_1_Higher_Half_x86_Bare_Bones

I'd start using what I learned in Meaty skeleton to start building my kernel. It is much easier to do it after you are in the Higher Half, than moving it. This is one place I started over.

Next I'd identity map (virtual address == real address) the lower meg of memory (0x0-0xFFFF). One thing I learned here is that the page directory table uses actual physical addresses, NOT virtual addresses.

In building the kernel this is the order I'd do things in after completing above:

1. Rewrite my snprintf routine to support hex, unsigned int, signed ints, boolean, hex, and string variables.

2. Make my GDT

3. Make my IDT, spend LOTS of time here, basically the GP (General Protection) fault, and Page Fault entries will become irreplaceable debugging tools. Really build them out. As part of the IDT also map the APIC ( https://wiki.osdev.org/APIC_Timer ), I kept getting random GP faults until I did this.

4. At this point I'd go back and move to Multiboot2, you will need a memory map for memory setting up useful paging and Multiboot2 can provide this to you. When you do this you will have to change QEMU from using "-kernel mykernel.abc" to "-cdrom mykernel.iso", be prepared to have this take some time. Once I had multiboot2, I'd extract the info to my own structure.

5. Next I'd make a keyboard handler https://wiki.osdev.org/PS/2_Keyboard#Commands This task isn't that hard but it is VERY putzy.

This brings us to some random thoughts.

a. when GRUB2 hands control over to your kernel, it places the multiboot info address in %ebx, save it before you use the register.

b. Mask the APIC so it sends less spurious interrupts here is the code:

# mask the APIC until we get it set up

xor %ax, %ax

mov	0xFF, %al

out	%al, $0x21		# Mask all master PIC IRQs

out %al, $0xA1		# Mask all slave PIC IRQs

c. when you compile remove the -O2 flag from your compiling, it optimizes out stuff as you try to debug. Get to know your debugger well before you start your kernel. I use gdb

d. Once you really are stuck (for several days), ask for help.

That's all for now. You can find my code here: https://github.com/tedavids/DragonOS

I hope this helps someone else.

PS my next steps are:

Set up my paging structures, I plan on using two bit arrays, one holding what memory is available for paging, and the other is it read only or read/write. Then start on functions to map/unmap pages. And then start working on a heap.

I'm putting this here in the hope that it helps someone else.

You can view the page descriptor table, and page entry as a 1024x1024 array. Where each element is a virtual page in memory. Array element [0,0] is virtual address 0, and element [1023,1023] is whatever that multiplies out as.

I teach my students to view array elements are like buckets. In these particular buckets are several things. First the 'present' bit, if this is zero, the buckets is empty. If it is not, the bucket also has a REAL address, which is the Real page the virtual address 'lives' at. Finally the buckets contains a bit that tells you if you can write to this area, or not.

I know for many, of you, this is obvious. But it took me several days on osdev.org using the paging pages and the bare bones paging page looking at code to come to this revelation.

I hope it helps someone