Little help with Executables!

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
nuno_silva_pt

Little help with Executables!

Post by nuno_silva_pt »

Hello everyone! After various months without activity, i have finally returned to this forum! (wooo! XD)
I've been learning C/C++ (now i do decent stuff), but i still have some trouble with Assembly.
Anyway, i finished coding a simple kernel using a tut on Boda Fine's (http://www.osdever.net/bkerndev/index.php?the_id=90), and i was wondering, after making the simple video output, setting up the GDT, IDT, ISR, IRQs, Keyboard input, etc, what should i do next till i can start having applications for the OS?
Thank you all for your help, and have a nice day! :D
Candamir

Re:Little help with Executables!

Post by Candamir »

I've also been through bkerndev, and if I remember well, the last chapter tells you what you should do next. I am now dealing with a memory manager, and afterwards, I will eventually set up file systems... Then you can add multitasking support and so on... Next tutorial to read (I did it that way) is 'Implementing Basic Paging', also on BonaFide, but make sure you understand everything that's mentioned there, if not, you're not going anywhere...
BTW, the code from that tutorial is no longer available (who knows why?), but we reconstructed it in a post a few days ago... It's called extern c functions or something like that...
earlz

Re:Little help with Executables!

Post by earlz »

maybe after memory management you could try doing dma and floppy disk IO
nuno_silva_pt

Re:Little help with Executables!

Post by nuno_silva_pt »

Thank you everyone! You've been quite helpfull!
Now i need to find some resources for implementing stuff like malloc , realloc and free, right? Or is it something different when you say i should implement a Memry Manager? :-\
Candamir

Re:Little help with Executables!

Post by Candamir »

Well, it will eventually come down to that, but first you should think about other things, like paging (the tutorial I mentioned). With paging, you can basically offer your applications (something you are interested in) protection from crashing each other; something that's almost a must in a multitasking system.

After implementing paging, you should write the physical memory manager, which allows you to allocate and free these memory pages. (Not sure if this is right, though).

The drawback in implementing the physical memory manager is that pages are always 4k big, and if you want to allocate heap for a struct that's 16 byte, it won't do. So you write a virtual memory manager, which allows you to allocate memory of variable size.

Note that all these functions are for the kernel, and when you call malloc() in a user program, the C library often does a system call (sbrk() or morecore()) and the kernel then gives the kernel more heap space. The virtual memory manager can also make that even if different pages in memory are separated from each other, they appear to the application relying on these pages as one continuos block.

Now, I'm not 100% sure on everything I wrote here, so I ask the ones who already have implemented their memory managers (I don't, I'm reading manuals now) to correct me where this shall be neccessary.

Candamir

BTW, I hope you can understand everything I said; I'm still learning English...
nuno_silva_pt

Re:Little help with Executables!

Post by nuno_silva_pt »

Hmm.. Then i must first implement paging, so the OS wont crash because of a app, and then i must implement a virtual memory manager... I did find that extern c declarations thread, and i should be able to use the source there! I'll look in the OSFAQ2 for info on virtual memory managers once i get paging done. Thanks for your help! ;D
earlz

Re:Little help with Executables!

Post by earlz »

reverse the order you have to have a memory manager before you can even think about paging
unless by paging you mean page allocation/deallacation in which case your using wrong terminology
Candamir

Re:Little help with Executables!

Post by Candamir »

Well, with implementing paging I meant to set up a page directory and page tables and so on and later on allocate these pages...
BTW, how can pages be allocated?
OZ

Re:Little help with Executables!

Post by OZ »

BTW, how can pages be allocated?
The tutorials that I've found and read on paging always just talk about a static address space. Perhaps I got the keyword wrong now, but I think he thought of the possibility to map/unmap pages while runtime in addresspace.
In those tutorials they always just start with mapping a certain amount of memory in and that's all.

Therefore one needs to know how much memory is in the machine and then provide a way to keep track of which page is free/used.
There are different approaches available like stack or bitmaps.
hope it helps
User avatar
Solar
Member
Member
Posts: 7615
Joined: Thu Nov 16, 2006 12:01 pm
Location: Germany
Contact:

Re:Little help with Executables!

Post by Solar »

Ahh, confusion... 8)

Have you visited Physical, Virtual, Paging, help? in the FAQ?

Note that malloc() / free() are part of the user-space library. They aren't connected to the memory manager in any way, other than that they require it to provide memory to the running process.
Every good solution is obvious once you've found it.
Candamir

Re:Little help with Executables!

Post by Candamir »

But doesn't malloc and free then use some system call if they've run out of memory? And then, a new page is allocated?

Another question, if I understood everything right, kalloc and kfree don't rely on paging?
OZ

Re:Little help with Executables!

Post by OZ »

Yeap, the userland malloc calls sbrk or something like that, to inform the kernel, that it needs to map more mem in for the process' heap.

Well if I don't got it wrong till now, I would say kalloc/kfree do rely on paging (aslong as there's paging support in the kernel).

You got to distinguish between basic physical memory mangement, the virtual memory management
(the paging part) and the higher memory allocation functionality for the kernelspace itself (kalloc/kfree).

As the for the phys mem part you mainly got functions to:
  • detect the amount of physical mem installed in the box
  • a mechanism to keep track of used/free pages (bitmaps/stacks)
  • a function to request/free (a) page/s
The virtual mem manager relies on the phys mem manager providing physical memory which it then uses in the different address spaces.
Here you will have to provide functionality for:
  • create page dirs / tables
  • un/map pages at certain virtual addresses
  • the rest I've forgotten ;D


and finally the higher memory management functions to allocate memory chunks that are smaller than the page size while avoiding memory loss for the kernel itself.
  • kalloc/kfree
hope it helps
User avatar
Solar
Member
Member
Posts: 7615
Joined: Thu Nov 16, 2006 12:01 pm
Location: Germany
Contact:

Re:Little help with Executables!

Post by Solar »

Candamir wrote: But doesn't malloc and free then use some system call if they've run out of memory? And then, a new page is allocated?
More memory is allocated. Whether that's a page, or whether the underlying OS supports paging at all, or if the OS is using a MMU (virtual address space) at all, doesn't really matter to malloc() as long as it has some way to demand and give back memory from / to the system.
Another question, if I understood everything right, kalloc and kfree don't rely on paging?
Completely up to you. ;-)
Every good solution is obvious once you've found it.
Candamir

Re:Little help with Executables!

Post by Candamir »

OZ wrote: As the for the phys mem part you mainly got functions to:
  • detect the amount of physical mem installed in the box
  • a mechanism to keep track of used/free pages (bitmaps/stacks)
  • a function to request/free (a) page/s
If I understood everything, I need the amount of physical memory installed in order to know if I can request more pages or not. But then, when there isn't any more space, must I write pages to harddisks or is that part of the vmm?
User avatar
Solar
Member
Member
Posts: 7615
Joined: Thu Nov 16, 2006 12:01 pm
Location: Germany
Contact:

Re:Little help with Executables!

Post by Solar »

OK, back to square one. Don't get hung up on names, especially not the name "virtual memory", because different people mean different things when they say that. ;)

Let's say you have a really, really basic OS. No MMU, which means no virtual-to-physical mapping, no pages.

Step 1 - Physical Memory

You determine how much RAM you have at which addresses. You assign areas of this memory to processes as they start up. You assign additional memory to running processes as requested by their [tt]malloc()[/tt] implementations. You collect memory from userspace [tt]free()[/tt] implementations and ending processes. This is your memory manager.

Step 2 - Virtual Addressing / Memory Protection

Now comes the MMU, which enables you to provide each running process with a virtual address space starting at 0 and reaching up to 2^32-1 (on a 32bit system). This is usually done page-wise, because mapping each virtual address to a physical address would require too much memory for mapping data. ;) You get the added headache of maintaining page tables and directories, but on the upside you don't have to fiddle with executable relocations, and you can protect processes from each other. You might want to call this your virtual memory manager, or you might reserve that for...

Step 3 - Virtual Memory

And because of some nice features provided by the MMU, you can map virtual memory pages to physical memory not residing in RAM but on hard drive. When such page is accessed, you stop execution, and swap in the requested page from hard drive (possibly swapping out another page in exchange).

Note that everything beyond step 1 is completely optional. It's a common illness that people tend to lump step 2 and 3 together, and aren't very precise in their usage of terms like "virtual memory" or "paging".
Every good solution is obvious once you've found it.
Post Reply