Memory Handling

[paul]

I'm working on a small OS mostly just to learn how some things work at the lowest levels of a computer, and need some tips on memory handling.

My small OS (currently just a kernel booted by GRUB & a file loaded as a module), doesn't work like most others in that applications (and drivers) aren't native code but are written in a custom byte code which is interpreted (plan is to eventually JIT it, but for now interpreting will do). This byte code gives no access to pointers, so I don't think I need to use paging or segmentation (correct me if I'm wrong) as there's no way for apps to access random memory locations anyway. So, when there's no need to worry about paging or segmentation, whats the best way to handle memory?

Below is my current memory code:

Code: Select all

pointer memBase = 0x000001;
ulong memAllocated = 0;

pointer memAlloc(ulong size)
{
    pointer mem =  memBase + memAllocated;
	memAllocated += size;
	return mem;
}

void memFree(pointer mem)
{

}

pointer memReAlloc(pointer mem, ulong size)
{
	memFree(mem);
	return memAlloc(size);
}

Very simple, no freeing yet, and it doesn't detect when its out of memory. I know I can handle this as a linked list that tracks allocated memory, but how do I know what parts of memory it is OK to use? The kernel has to be in there somewhere, how do I know where GRUB loaded it to avoid overwriting something vital? Also, there's video memory at 0xB8000, is this always a given size so I avoid using that memory? Are there other areas of memory like the video memory that I shouldn't use?

Sorry if I don't understand this at all and sound completely stupid, I just don't really understand how to handle the memory.

[iammisc]

I know that pointers can be dangerous, but to code all your apps and drivers without them would be hell. Like how would you access the video memory or the memory where your dma controller wrote stuff to.

Maybe i am not understanding your bytecode idea.

[Crazed123]

It sounds like a fairly standard virtual machine interpreting safe bytecodes. A special API is probably used to control access to hardware resources from within the byte-code, probably via a special instruction that "drops down" into the real operating system on the real hardware.

Since your user-level code cannot, by nature, violate safety restrictions placed on it, you can probably just use perfectly normal heap algorithms to manage your byte-code interpreter's heap.

Also, can I offer one little bit of advice? You're better off using (and extending, if needed) an existing byte-code VM rather than writing your own, unless yours will have an important design or feature the others don't. The JVM and .NET VMs and byte-codes have loads of languages compiling to them now, so using one of those would give your application developers access to a much wider array of applications than they'd get from compiling to a hand-made proprietary byte code.

Best of luck!

[Kevin McGuire]

You can look into the multi boot header for you're kernel, which will allow GRUB to hand you another info header which can contain a system memory map.

http://www.gnu.org/software/grub/manual ... iboot.html

Code: Select all

void mbinit(struct tmbinfo *ifo){
	x = (unsigned int)ifo->mmap_addr + ifo->mmap_length;
	for(	mm = (struct tmbmm*)ifo->mmap_addr; 
		(unsigned int)mm < x; 
		mm = (struct tmbmm*)((unsigned int)mm + mm->size + 4)){
		if(mm->base_low + mm->size_low != 0){
			console_write(
				ksprintf(0, "mm: %u %u %u \x10", 
					mm->base_low, mm->size_low, mm->type)
				);
		}

		if(mm->type == 0x1){
                            // free memory range
		}
	}
}

[salil_bhagurkar]

Okay, using this byte code means you do not use processor opcodes and theres an interpreter inside your os which works just like a processor. Here the advantage lies that there is no much of overhead in case of system calls but in case of other bytecode processing, it's not fast. My idea:

To add two numbers it's "add eax,ebx" and that goes directly to the processor whereas for bytecodes it includes an "if" or "switch case" statement to decode the bytecode.

So is this method of making user applications better or optimal or it's best to go with windows and linux have been doing...

[paul]

I know that pointers can be dangerous, but to code all your apps and drivers without them would be hell. Like how would you access the video memory or the memory where your dma controller wrote stuff to.

Sorry, I should have explained that there is a way for code to write to and read from memory locations, but only drivers will be allowed to do so.

You're better off using (and extending, if needed) an existing byte-code VM rather than writing your own, unless yours will have an important design or feature the others don't.

I did initially think of using mono for this, and just writing a small kernel that would just run mono and that would in turn run .net stuff, but I see 2 problems with it: firstly, using mono would likely force me to open source my code, which I'm not sure if I want to do yet. Secondly, the kernel beneath mono would have to implement quite a lot of things in order to allow mono run, whereas by writing my own interpreter/future JIT I don't need to implement much in the C kernel, for example file i/o and multitasking aren't needed, as these can be done in the higher level bytecode or in a completely different manner to usual (eg. multitasking doesn't need to use TSS which has never made much sense to me).

using one of those would give your application developers access to a much wider array of applications than they'd get from compiling to a hand-made proprietary byte code.

In order to run most applications I would have to have the class library/APIs exactly the same as the platform the app was made for. I don't plan to do this, as the whole point of me writing my little OS is to try things my way, rather than following the same methods used by others. So, using an existing VM would offer me access to the languages that target it, but not access to any existing apps.

You can look into the multi boot header for you're kernel, which will allow GRUB to hand you another info header which can contain a system memory map.

Thanks, I use the multiboot header to retreive the modules (bytecode) loaded for me by GRUB. I hadn't read that part of the multiboot specification so I didn't realise that the type property could tell me if the memory was free. Am I correct in saying that I can use any memory within a range whose type is 1? If so, that makes it much easier than I had expected.

Okay, using this byte code means you do not use processor opcodes and theres an interpreter inside your os which works just like a processor. Here the advantage lies that there is no much of overhead in case of system calls but in case of other bytecode processing, it's not fast.

Yes, currently the bytecode is interpreted and it does involve a lot of overhead to run the bytecode, this is why I plan to eventually use a JIT compiler which will create native code and speed things up a lot. For now, an interpreter doesn't have any huge impact on performance because until I get more working in the C kernel, all the bytecode can do is load strings onto a 'stack' and tell the system to print whatevers on the stack (makes for a nice 'Hello World!' app )

it's best to go with windows and linux have been doing...

I stress again, the whole reason I'm writing this os is to try doing things my own way, and not to follow the same route most others have took. I don't want to have an OS which is just a clone of something else.

[Crazed123]

Good luck on the operating system then. However, I advise you to at least design your byte code as something to which many different types of high(er)-level languages can compile. Don't make something (like Java, for example) that can only easily represent one programming style at the byte-code level.