[Questions] Paging

[codemastersnake]

Hi all!

I have implemented some paging but I am confused...
I want to know:

* What is required to implement just malloc() and free();
* There are so many areas in memory which are reserved, So how to take care of them?
* How to create a new process and pass control to it, Like as kernel invoke shell and shell runs in less priviledged level.

There are many questions but right now these are enough....

P.S I have searched google and documents but they all give a very specific implementatio for their OS. I want to know a general algo.

Can you people tell the way everything about Memory should be built.

[AJ]

* What is required to implement just malloc() and free();

You need a heap manager. See the tutorial at http://www.jamesmolloy.co.uk .

* There are so many areas in memory which are reserved, So how to take care of them?

This is where your physical page frame allocator comes in. When you build the original page stack / bitmap, do not mark reserved areas as free.

* How to create a new process and pass control to it, Like as kernel invoke shell and shell runs in less priviledged level.

Again, JamesM's tutorials should help with this. The basic scheme is as follows:

1) Interrupt (timer) fires and runs your ISR, pushing ESP and SS (if in user privilege mode), EFLAGS, EIP and CS. You then push segment registers and general purpose registers.

2) You save the current ESP in a process control block structure (which you design).

3) You load CR3 and switch to the ESP of the incoming process (which is decided by you scheduler.

4) POP all of the general purpose registers and segment registers. An iret will pop CS, EIP, EFLAGS, SS and ESP. You are now running your new task.

It's not quite as simple as this, and you will need to load a TSS to start with and there are a few other considerations, but that's the basic scheme. IIRC, there is also something on http://www.osdever.net which quides you through this.

Can you people tell the way everything about Memory should be built.

If you are using GRUB, use the memory map feature. If using your own boot loader, you will need to query the BIOS while still in real mode. If you are referring to virtual RAM, see the tutorial link provided above.

Cheers,
Adam

[codemastersnake]

I'll clear my doubts one by one:

* Implementing malloc() and free()

To implement these I just need a heap manager and not paging. Is this right?

What I have:
* I know the amount of physical memeory. I use grub for that.
* I know the start address of my kernel i.e. 0x100000
* I don't know the size of kernel. Can I calculate it at runtime? I mean using some lables or something.

Thanks for above links

[AJ]

You don't need paging to implement malloc and free, but if you don't and you would like separate process spaces later on, you will need to either paging or segmentation to deal with this.

All a heap manager does is take a few variables (heap start, end and maximum limit), and allocates numbers based on these, marking allocated blocks of numbers. It really doesn't matter what system lies underneath. This means that a decent heap manager is easily portable.

e.g. My OS has i586 and x86_64 flavours at the moment - the kernel heap manager is exactly the same on both, and would be exactly the same in a real mode OS (albeit with smaller values for heap_start and heap_max). The underlying architecture deals with paging via a page fault exception handler. Oh - and yes, you can determine the kernel size by using labels in you linker script or a structured format such as ELF.

In addition to my suggestion about the task switching tutorial on osdever.net, have a general browse around the memory management articles. They're what I used to get started.

Cheers,
Adam

[AJ]

Sorry to have a dig at you, but two points about your website.

1) I'm still randomly seeing ads which are NSFW. There should be a warning about that in your sig.
2) How can your site be 'the best place for newbies to start from' in OS dev, if you are not happy with the concepts of memory management, paging and multitasking? I see what you are trying to achieve, but perhaps you should read more from the Intel books adn get more in to your own OS before creating a new os dev site in any big way.

Cheers,
Adam

[codemastersnake]

Sorry to have a dig at you, but two points about your website.

1) I'm still randomly seeing ads which are NSFW. There should be a warning about that in your sig.
2) How can your site be 'the best place for newbies to start from' in OS dev, if you are not happy with the concepts of memory management, paging and multitasking? I see what you are trying to achieve, but perhaps you should read more from the Intel books adn get more in to your own OS before creating a new os dev site in any big way.

Cheers,
Adam

Yeah I know what are you saying. But buddy I am trying hard to achieve it. Moreover, with no offence, I have been into osdev from past 6 years. it's only that I have to make a fresh start now. That banner in my sig. does feels okward but it's was what my previous website was. I have every tute in my PC dumped. you name it, I hv' got it. seriously I do have.

As far as reading is concered. I have taken my lectures and now I am clearing them with the lecturers. I think that is fine isn't it good thing.

Now about my website's ads. A lots of people have been complaining me about the NSFWs. So now I am going to talk to adbrite now about the issue and will hopefully resolve it.

Let me come back to Memory Management:
* How should be a heap stored in the phisical memory so that It does not confilict with the kernel that is loaded into the memory.
* Do I need to take care of reserved memory areas while implememting heap.

[AJ]

OK - thank for looking in to the ads thing.

To come back on topic:

1) You should have some kind of physical memory allocator to keep track of allocated pages. This is often one of the following:
a) A bitmap, each bit represents a space of one page. If a bit is set, the page is assigned, otherwise the page is free. Takes up not much space but potentially slow to allocate a page.
b) A Stack. All free pages are pushed on to the stack (you need to write push/pop functions). To allocate a page, pop it off the top of the stack. Takes more space and longer to set up, but potentially much faster to allocate a page.

2) The GRUB / BIOS memory map marks all memory mapped areas as unavailable, so don't worry about that.

Using the above system, you build your page map / stack using the BIOS memory map (which you get from the BIOS or GRUB). Then, ensure that all space containig your kernel is allocated (not available in your stack / bitmap). Then, whenever you allocate a page of physical RAM, you know it is not already in use.

Cheers,
Adam

[codemastersnake]

Things that I have learnt so far are
* I don't need to use/implement paging for using just malloc() or free().
* I will however need to implement a heap to use dynamic memory allocation.

Is that right?

More over for now I just want to use malloc() and free() efficiently. I don't want to get involved in any other thing that does not affect dynamic memory allocation like paging. This is because I want to clear every everything before I step forward. If paging is not nessacary for malloc() and free() then I'll skip it.

The GRUB / BIOS memory map marks all memory mapped areas as unavailable, so don't worry about that.

I didn't get it? can you please explain it.

[AJ]

No - paging is not required for memory allocation. You are most welcome to make a system without paging, but may have problems in the future when you want to have separate task spaces.

My second point was that presumably to find free physical areas of RAM, you are using a memory map (provided by a BIOS interrupt, or passed to your kernel by GRUB). Presumably you use this memory map to decide which physical memory is free. In which case, you do not need to worry about memory mapped devices as they are not marked as free in the memory map.

Moreover, with no offence, I have been into osdev from past 6 years.

Then how have you possibly got to this point without knowing how to write basic memory management routines?

Cheers,
ADam

[codemastersnake]

I have been into os development into six years, I wanted to say that I launcehd my first kernel on 13th Feb' 2002 when I was in 11th grade. I have been into only 16bit OS. I used BIOS to program my OS. Didi wrote FAT FS and created a DOS like Kernel. But never launcehd that KErnel because of some reasons... Though never mind.....

Now I have stepped into 32bit OS programming. And This may sound foolish but I was never able to understand paging very well. I know the concepts but have problems in implementing them. As always said by gurues Lil' knowledge is a dangerous thing. This is what is happning with me!

By the way you'll be happy to know that I have disabled ads containing offencive materials. But it will take upto 4-5 working days to vanish completely....

Back to topic:
So now what I'll do is that
* I'll only implement heap and not paging. I'll use paging after my kernel is stable enough to handle paging.

* I'll calc. the starting point of y kernel(which I know from the linker script) and size of my kernel image in memory. Lets say it comes aournd 40KB.

* Then I'll point the starting address of the Heap to the address where kernel image is ending.

* And the ending point of the Heap would be the last address of the accessicible RAM (Let's say 4MB).

* Then let code use malloc() and free().

Is that good? Do correct me if I am wrong.