OSDev.org

I have been considering how to detect if a process (also driver in a microkernel) in some way fails or becomes unstable. My current thinking is this:

Each executable comes packaged with a 'contract' which specifies what it is supposed to do in simple, low level terms. For GUI programs this could be along the lines of "must be responsive to user input" (no I'm not saying that a word processor must say that it allows the user to use word art or something!). A storage device driver must respond to read and write requests in a timely manner. In addition, the contract would state what the task expects from the system, e.g. a game could say it wants direct access to the frame buffer and to be run at high priority.

The kernel would then try to balance out its obligations to each process, and assist those processes which are failing. If a gui program were to have its message queue continue to fill up, for example, the kernel could allocate it a greater timeslice or priority. On the other hand, if the reason for its message queue filling up is that the process doesn't read from it (rather than receiving too many requests), it could consider that process to have a problem.

Any process which needs help fulfilling its obligations could then have that information written to its contract, so the kernel would know for future reference.

For security reasons, the user would need to be informed of the contract when they install the program and when it changes (to stop malicious code nicely asking to be allowed direct access to io ports). No resources would be granted without a user-verified contract for that particular process.

Does anyone know if a similar mechanism been done before? I guess so, but it would be interesting to see how it was implemented.

Regards,
John.

jnc100 wrote:Any process which needs help fulfilling its obligations could then have that information written to its contract, so the kernel would know for future reference.

...

For security reasons, the user would need to be informed of the contract when they install the program and when it changes (to stop malicious code nicely asking to be allowed direct access to io ports). No resources would be granted without a user-verified contract for that particular process.

Does anyone know if a similar mechanism been done before? I guess so, but it would be interesting to see how it was implemented.

That sounds like another capability based system. They work, but the user must first fix up the contracts for it to work - which is the most problematic flaw in the concept.

How would capabilities keep a process within its contract in ways other than stopping overuse of resources?

Crazed123 wrote:How would capabilities keep a process within its contract in ways other than stopping overuse of resources?

Capabilities tell the OS what a process can do and what it can not - so if a capability is defined for doing X and it tries to do X without that capability, the action will be prohibited. Limits prevent overuse (abuse) of resources, capabilties are as fine-grained as you define them.

Right. That just covers naming and limiting the use of resources.

These contracts sound like they also specify what a process *must* do, what it *has an obligation* to do. For example, a contract can state that a shell process must respond to user input in a timely manner.

How can capabilities compel a process to use the resources it legitimately has in a manner specified by the contract?

Crazed123 wrote:These contracts sound like they also specify what a process *must* do, what it *has an obligation* to do. For example, a contract can state that a shell process must respond to user input in a timely manner.

Yes, that was my intention. To put it another way, the contract should be a two way agreement between the kernel and each process, that states:

- what the kernel should do for the process
- what the process should do
- what the process should not do

Note that what it should do is not necessarily the same as the opposite of what it shouldn't do - it is a requirement of functionality rather than just what it is allowed to do.

In most desktop operating systems, for example, the user is normally quite aware of when an application is not doing what they expect, and the user can then take appropriate action. Unfortunately, the user is normally only limited to killing the application. I hope to have the kernel be able to assist the process in fulfilling its obligations, by assigning more resources etc. This is something that the average user is not necessarily able to do. In addition, it has benefits in unattended systems like file servers. If the server application were to get stuck in a loop or whatever, it may take a few days for the users to feed back to the network administrator that something is going wrong and needs fixing, causing unnecessary delays in working. I have often been contacted about fixing some of the various networks I maintain, where all that needs doing is restarting samba.

I understand that some applications already contain built in fault tolerance, I was just hoping to make it more generic and involve the kernel more in the solution. As the contracts exist for this purpose, then it makes sense to extend them to include information similar to capabilities as well.

Regards,
John.

Crazed123 wrote:Right. That just covers naming and limiting the use of resources.

These contracts sound like they also specify what a process *must* do, what it *has an obligation* to do. For example, a contract can state that a shell process must respond to user input in a timely manner.

How can capabilities compel a process to use the resources it legitimately has in a manner specified by the contract?

The same way Electronic Arts paid their programmers to write Battlefield 2 not knowing one of the head programmers was a little ticked off just before it was uploaded to be published, and he embedded a trojan into it.

IE: Capabilities can't do that? Great, so some other mechanism is needed to implement these contracts. Hmm...

jnc100 wrote:

Crazed123 wrote:These contracts sound like they also specify what a process *must* do, what it *has an obligation* to do. For example, a contract can state that a shell process must respond to user input in a timely manner.

Yes, that was my intention. To put it another way, the contract should be a two way agreement between the kernel and each process, that states:

- what the kernel should do for the process
- what the process should do
- what the process should not do

Ok, that's a bit different. Why would you make a requirement on an application though? Moreover, how would you check that it did that without implementing the exact same behaviour to check it with? Could you catch "cheap fakes"? Would this not be a substitute for testing - if you satisfy the contract, release? Last but not least, who would be your target audience? Users will close & ignore applications that don't do what they should.

How does a Genetic Algorithm system test for fitness?

How does a Genetic Algorithm system test for fitness?

Do you mean:

1. How do you test a genetic algorithm system for fitness?
2. How do you test system fitness with a genetic algorithm?
3. How do I create a open ended time sink hidden inside another topic?

And, we have this button called "new topic". How am I supposed to find these cool new topics if they are all embedded inside another topic that is slightly different OR at least have no natural connection to the original question beyond a assumption which could be correct by expanding the question to include more information about the ..new.. topic or ..not..?

And.. if you do not like what I say then too bad because I am going to continue saying what I say until I see some reason to not say it.

I meant #1. Given several different genetic algorithms, how do I test them for fitness with respect to a problem whose solution is their purpose?

And the topics are quite related. Genetic algorithm systems somehow test algorithms' abilities to perform within certain requirements while solving a certain problem. Couldn't similar mechanisms implement the contracts we were talking about by checking running programs for fulfillment of their contracts?

Yes, there could easily be a horrifically stupid assumption hidden in that question. But it's worth checking.

Candy wrote:Why would you make a requirement on an application though? ... who would be your target audience? Users will close & ignore applications that don't do what they should.

Yes, I concede that they would. The contract is a way of improving the performance of an application before the user feels the need to close it. In addition, the user is less aware of the running of several system services. For example, if your word processor refuses to open a document, it is not necessarily the word processor that is at fault: it may be the file system or storage device. The contract would become invalid for one of these systems, and the kernel would take steps to remedy it (including restarting the service).

Candy wrote:Moreover, how would you check that it did that without
implementing the exact same behaviour to check it with?

The requirements are planned to be a very low-level arrangement that the kernel could monitor, or could monitor with the help of other services. In many cases, user interface responsiveness can be gauged by the activity in the applications message queue. Other sorts of functionality can be reported by other services. The tcp/ip service can report back if it is not getting correct data from the network card, for example. Obviously this would involve the response of a third service interoperating with one or the other of these to determine which is actually at fault.

Candy wrote:Could you catch "cheap fakes"?

Not necessarily. With any system you have to rely on the user being responsible enough to install trusted software. Besides, I'm not in the business of telling anyone what they can or cannot install on their own computer. If they wish to put something on which will fool the kernel into thinking its working then I will not stop them. Please elaborate if I have misinterpreted this qeustion.

Candy wrote:Would this not be a substitute for testing - if you satisfy the contract, release?

In a similar way to "if you satisfy MS logo requirements, then release"? Again, the onus is on the application developer, user and the community at large who will provide positive or negative reviews.

The contracts are designed to help the application/driver/service do what the user wants, but I'm taking up the points on capabilities mentioned here to and including them because it just kind of seems to make sense to keep this sort of information together. I stress that contracts are not a substitute for well written code.



Having just read the wikipedia article on genetic algorithms, I see you need some sort of fitness function, which I hope I have somehow described above (although my ideas are by no means finished). The method of 'natural selection' they describe seems a bit silly for this purpose though. I will not open up 20 slightly different versions of an application and see which works better. To continue the biological example, I propose a system more like 'selective breeding' where you have some knowledge of what effect a given alteration will have.

For example, if you're a short person and you want taller children you can be pretty sure its more likely if you marry a tall person rather than a short one. In the os example, I can be pretty confident that increasing a unresponsive program's timeslice will be more likely to make it more responsive than less responsive. If I need to make such a change to an unresponsive program, and it works, then I will remember that I have made that change and store that fact for the next time I run the program.

Regards,
John.