atomic process creation

[distantvoices]

Hi, folks!

Whilst doing widget design and programming all the while, I've been thinking about a way to actually create a process without the need to say *fork()* and *execve()* to load a image.

I'd say a systemcall like create_process(file,argv) should be perfectly ok too:

Imagine, you drop this CREATE_PROCESS to the memory manager:

It tells the fs service that a process is being created after determining the next valid pid.

fsservice opens standard input and standard output for the process. then it opens the file (if exists, else return an error to mmservice). Then it *passes* the crucial FD which points to the file pointer of the image to be loaded to mmservice.

MMservice checks the header of the image whether it is ok to load this file.

MM service orders a page directory from the pager, and then: it maps logically: the area of the image as a memory area bound to the file represented by fd. heap and stack (sufficient stack pages are to be mapped in for the argv image) and transfers the argv from the calling process to the new one.

MMservice tells system daemon to create a task control block.

Then the process is executed the first time. No 0xdeadbeef, but the usual address as is to be expcted in eip, say 0x1000: pager asks mmservice, what this is about.

mmservice tells pager to map one page. Pager executes this request and returns control to mmservice.

mmservice tells fsservice to load data from the file y on behalf of trapped process x into its adress space - in the length of a page. (0x1000).

this is supposed each time a pagefault happens. the faulting address is cut to page boundary and submitted in form of an offset to the file service, which performs the loading.

What do you think about this?

is it utterly rubbish?

[Pype.Clicker]

what sounds weird to me is why does MM have to know how to load a program and which program format are "valid" and which are not...

[Legend]

However, fork is one of the things that requires you to implement copy-on-write to do it efficiently.

[distantvoices]

@pype: You are perfectly right. I have not thought about it further than: MMsrv is the first one which receives a command for process creation and why not have it check whether the file to load is correct. Maybe I should put this kind of stuff into the fs service - to say it more exactly - into a kind of program loader subsystem which is able to parse all the different formats of executeables.

Meanwhile it is just a kind of lazyness to have this stuff located in the mm service - which is after all the root of all evil in my own os *gg*.

@Legend: This kind of stuff I have not yet dealt with althou it is on my neverending list of to-do's

[Dreamsmith]

Meanwhile it is just a kind of lazyness to have this stuff located in the mm service - which is after all the root of all evil in my own os *gg*.

My guess is, much like Minix, your "memory manager" would be better termed "the Task Manager" (or "Process Manager" if you prefer). If so, it makes perfect sense that a create process function would be located in it.

What's never made any sense to me is, err, what on earth would you need a seperate "memory manager" for? At the kernel level, about the only job it has is to give pages to tasks. Allocating and freeing pages is a job so trivial that it's a couple of inline functions in a header file in my OS, and altering task structures should be done by the task manager. Handling paging is a little more complicated, but it's also a job so intimately tied in with tasks/processes that it makes sense it would be handled by the task manager, too.

So, other than that you call your task manager "mm service" for some reason, I see nothing odd at all about where you've located this particular system call. Rename it "the task service" and call it a day.

As for the implementatio of the create process call, the fact that it bounces into and out of other services as many times as it does seems overly complicated to me. In my own load process call, the only time it might call some other service is when it's calling stream->read, depending on the kind of IOStream it was given to load the process from (might be a file, but it might just be a fifo/pipe, or it might be a TCP stream for all loadProcess knows).

When you have to bounce through three different services half a dozen times to perform one simple task, you know you've taken the microkernel approach way past the point of usefulness. Sometimes it's a good idea to keep certain things together and in one place, before you reach the point of having to have the read block service call the read byte service 512 times per disk block. ;D

[Legend]

I agree with Dreamsmith ... if you have X (with X > 2 for example) ring transistions to do a fucking simple task (like allocating a block of memory), you have to be slow. This is what all the L4-fans for some reason "overlook" when they speak of user level memory management ... stupid n00bs ...

[distantvoices]

@Legend: Your statement is petulant and inappropriate. Who the hell dost thou think thou arest that thou darest to call others taking another design approach than you names? Grow up, learn some manners, then come back. Period.

@Dreamsmith: well put. But I see, you didn't get it completely: There is a split between virtual memory management (address spaces of processes and the other implications) and physical memory management: mapping in and out of pages: thats merely two functions thou I put a bit of book keeping around it to keep track which page belongs to whom. Of course, the pager HAS to ask the virtual memory management if a faulting address is insida a valid range, if a pagefault comes along. Thats the way things work if one does demand paging.

Thanks for Feedback. I'll have to rethink the whole thing.

[Legend]

Apart from being a bit drunk, however, there are enough of those guys out there how take a look at an approach without understanding it's implications ...

[distantvoices]

*paf* and he woke up and saw there's light on earth...

I at least *know* there are bad designs in my work, that bad thought things have found their way into it. just look at my crappy malloc library and you know what I mean - dreamsmith 's nudged me to the right way - with *arguments* instead of insults and tantrums.

*me goes and does some kernel code cleaning*

Drunken Boy ... *tz* Shame on you.

[Solar]

@ BI:

Most of the times I found it very much enlightening to question the POSIX API in the first place. Unless, of course, you want to have a POSIX compliant system.

You implicitly say that your system has fork() and execve(). I've seen it before, just assuming they "have to be there" because "everybody says so". Those "everybodys" are most likely Linus and Tanenbaum, both of which are deeply rooted in the Unix world.

What's the reason for those functions? Why do you assume they're there? Why do you look for "another" function, instead of having a tabula rasa and ask yourself, "now how do I handle process generation"?

fork() et al. are only one example. I strongly believe the best designs come from people who, at least for the early stage, forget what they know or think to know. Then, when you've drafted a design, apply your experience to it and see if it holds together on its own.

Your mileage might vary, of course.

[distantvoices]

@Solar:
When I've started the whole OS dev thing, I told myself: walk in small steps. You don't know enough to be the tough OS dever. Try out the known. Then test it. Taste it. Keep it or change it. These days I am at the "change it" stage. Complete modules are ripped out and smashed in order to get something better. Getting rid of some olden idiosyncrasies is one part of this process. Getting things done is another one.

First, for me, it is a matter of learning to walk: Fork() and Execve() are two examples. I know what they do and it's been a thrilling experience for me to design/implement these two calls. I canna run ere I 've learned to walk, know ya?

Second, it is a matter of evolution: see, we try the probed, get positive feedback and become daring afteron. we want to explore the unknown land. For me it would mean to add some functionality like atomic process creation to the system. The means of doing it are there already. I just need to mix them together to a smooth thing.

Third, it is a matter of experience: working with fork() and execve() all the time just to *load* an executable - it is a bit tiresome. Why not providing one call which deals with this stuff - and permits demand loading? Would be nice to study whether it is feasible or not? What if two or three programs at "once" are started? would it crap out? would it deal with one after the other as long as there is need for the processes to fetch data from disk without one getting into the others way? See, I just want to *know*. Find a way to ease things.

ah and: why didn't I just make Tabula Rasa and asked "now how do I handle process creation"? Because this stuff I've already done. Fork() and execve() are *interfaces* to the process manager. the whole bunch of things done behind - it *is* a choice of protocol and that's design - I've done it on a lot of paper, with lots of drawing and thinking: How can I make a process?

Why I do assume fork() and execve() are there? They are well known interfaces - at least for me, the average Linux user/developer. Tons of information you find about what these two are doing - so they ve been a starting point for me.

to keep it short: yes, my mileage is different. But I read and understand your arguments and I accept them.

[Schol-R-LEA]

I have a similar view to BI's, but have taken a different approach to it: I am intentionally going through the OS development process at least twice, probably three times, one of which will be a relatively conventional, POSIX type OS - precisely to see how they do it, and understand why, and then decide if I find it a suitable way to design or not.

Of course, this is all moot so far, as my own development work hase been pretty much stuck for over a year - I have a lot of problems with motivation and determination, and spend far too much time second-guessing myself, but those are problems in my life in general.

[Dreamsmith]

@Dreamsmith: well put. But I see, you didn't get it completely: There is a split between virtual memory management (address spaces of processes and the other implications) and physical memory management: mapping in and out of pages: thats merely two functions thou I put a bit of book keeping around it to keep track which page belongs to whom. Of course, the pager HAS to ask the virtual memory management if a faulting address is insida a valid range, if a pagefault comes along. Thats the way things work if one does demand paging.

Sorry, what didn't I get? I said physical memory management is far too trivial a task to warrant a service of its own, and virtual memory management is too intimately tied in with task management to be sensibly split off from it into a seperate service. Of course, the pager HAS to know about what is or isn't inside a valid range, but being that it IS a part of the virtual memory management subsystem, I'm not sure it makes sense to say it has to ask anyone anything to know this -- it ought to just know it. If it doesn't, then as I said, the whole microkernel thing is being taken too far. You've either split things up too much, or put it in the wrong place, if it needs to ask someone else something this basic to its operation.

[Dreamsmith]

I have to agree with Solar here. First, determine what you need to do, and what makes the best sense as a way to do it, without regards to known APIs.

Later, if you really want to implement the POSIX API for compatibility reasons, do it as a library. Unless your idea of what your kernel should do was so parsimonious that you wouldn't even consider a POSIX API in any case, it shouldn't be a problem to implement it as an add-on library.

OTOH, if the main goal of your OS is to be "Unix" (as was the case for Linus Torvalds), maybe POSIX shouldn't be an afterthought...

[distantvoices]

@dreamsmith: sorry, I didn't use a proper wording, didn't want to upset you: I've got a split in my kernel between memory management (well - process adress space management seems to be a better word) and the handling of pages. That is what I have intended to say but didn't do so.

Your argument is valid. It would have been way easier for me to include the complete process management inside kernel space to avoid this asking about: is this page in a valid range? well - it is this influence from minix. Often I've thought about putting the whole thing into kernel land, but have refrained from it. I've thought of a service doing the management grunt work and a little working daemon inside kernel land which carries out the needed tasks and slips in and out of address spaces - so to say: to keep as little as possible in kernel space and move most management tasks into service processes. *shrugs* my daily programming work with it is filled with questions about "why not putting the complete thing into kernel space?"

By the way, how are you doing it?

regarding solars point: he is right: but still, ere one learns to run, one learns to walk. I prefer learning the tools first ere I go creative.