Page 2 of 2

Re: How do you use exceptions interrupts in your OS?

Posted: Wed Dec 10, 2014 6:31 am
by mallard
Brendan wrote: It requires kernel support to notify the process' standard library that the event occurred in the first place; regardless of how this notification is achieved.
As I've said repeatedly, C does not require any such notification! You can have a fully compliant C library with no support for signals except for those generated by "raise()" in the same process.
Brendan wrote: Everything has its own standards. C's "abstract machine" is just one. POSIX is another. Java's virtual machine is another. Most software does comply with the corresponding standards for the environment they're designed for. There's no reason to suspect that if those standards said "you're expected to implement an effective means of minimising data loss that may be caused by power failure, hardware faults or software crashes" that the majority of programmers designing software for that environment wouldn't do so.
If your "requirements" are too onerous, nobody will develop software for your environment. I prefer to make my OS as developer-friendly as possible. If you don't, that's your choice, don't act like it's the only possible option.
Brendan wrote:
mallard wrote: A safety net with "gaping holes" is still better than no net at all.
Wrong. A misleading/false sense of safety is worse than nothing.
I disagree. Please stop referring to people who hold different opinions to you as "wrong"/"stupid"/etc. It's very immature.
Brendan wrote: The key difference is that I won't be expecting code that's already crashed to handle crash recovery - the debugger is a completely separate/isolated process that has not crashed.
I don't "expect" any program to handle crash recovery. I just provide the option for them to do so if they so wish.
Brendan wrote: My OS's messaging? That's just signals. D-Bus? Yeah, that's just signals too. TCP/IP? That must be signals too! Obviously, all forms of communication (even spoken English) is 100% identical to Unix signals.
That is why "signals" was in quotes. "Signals" are a type of "messaging" that can exist in an OS. An implementation of POSIX signals can be built on another type of IPC and it's entirely possible to have a single messaging system that underpins all the kinds of IPC that you've mentioned. If you've implemented a way for another application to be notified when a program crashes, then you've implemented all the same mechanisms that are needed to support POSIX-like signals. I have no idea where you got the idea that "all forms of communication (even spoken English) is 100% identical to Unix signals" because I certainly did not say anything of the sort.
Brendan wrote: Users have no rights (until/unless the owner of the software grants them rights, including the right to use the software in the first place).
I disagree. The user owns the hardware (or has permission to use it from the actual owner). Any software that they run is there to serve them, not the other way around. The only entity with any right to restrict what a user does is another user, not an application vendor. While I may not be as fanatical about "software freedom" as the FSF, I certainly believe that I have the right to do what I want with hardware that I own, which includes debugging applications without needed special permission from their vendors.
Brendan wrote: The only thing required for "general purpose OS" is that the user is able to install software that wasn't a built-in part of the OS. Nobody is stupid enough to assume that an interpreter/JIT is a requirement.
The ability to run arbitrary applications is absolutely a requirement of a "general purpose" OS. An interpreter or JIT is a type of application. If it's not possible to run that entire class of application (with decent performance) on your OS, then it isn't "general purpose".

Re: How do you use exceptions interrupts in your OS?

Posted: Thu Dec 11, 2014 1:46 am
by Brendan
Hi,
mallard wrote:
Brendan wrote:The ISO C standard requires SIGABRT, SIGFPE, SIGILL, SIGINT, SIGSEGV, and SIGTERM to be defined; and (regardless of whether the C standard explicitly states it or not - I honestly don't know) programmers expect them to work (rather than merely being defined).
Brendan wrote: It requires kernel support to notify the process' standard library that the event occurred in the first place; regardless of how this notification is achieved.
As I've said repeatedly, C does not require any such notification! You can have a fully compliant C library with no support for signals except for those generated by "raise()" in the same process.
C doesn't strictly require them because C is designed to run on computers where there is no MMU or FPU, where it's technically impossible for a kernel to support them. On 80x86 (and other CPUs which have MMUs, etc); only a deluded fool would think it's sane to define them but provide no way of supporting them.
mallard wrote:
Brendan wrote:Everything has its own standards. C's "abstract machine" is just one. POSIX is another. Java's virtual machine is another. Most software does comply with the corresponding standards for the environment they're designed for. There's no reason to suspect that if those standards said "you're expected to implement an effective means of minimising data loss that may be caused by power failure, hardware faults or software crashes" that the majority of programmers designing software for that environment wouldn't do so.
If your "requirements" are too onerous, nobody will develop software for your environment. I prefer to make my OS as developer-friendly as possible. If you don't, that's your choice, don't act like it's the only possible option.
For competent developers it's standard practice, not onerous. Incompetent developers can learn to do it things right or find another OS that welcomes incompetence.
mallard wrote:
Brendan wrote:
mallard wrote: A safety net with "gaping holes" is still better than no net at all.
Wrong. A misleading/false sense of safety is worse than nothing.
I disagree. Please stop referring to people who hold different opinions to you as "wrong"/"stupid"/etc. It's very immature.
I provided a logical reason (if people make the mistake of thinking data loss can't happen when it can just because they've got a worthless signal handler that doesn't work for various cases including power failures, then that's worse than people knowing data loss can happen and providing adequate data loss prevention techniques). You've disagreed (repeatedly) without providing any reason other than "because I said so". If you don't want me to call your opinions wrong and stupid, either stop posting wrong/stupid opinions or at least try to give a reason why you think your opinion isn't wrong/stupid.
mallard wrote:
Brendan wrote:The key difference is that I won't be expecting code that's already crashed to handle crash recovery - the debugger is a completely separate/isolated process that has not crashed.
I don't "expect" any program to handle crash recovery. I just provide the option for them to do so if they so wish.
Translation: If a program wants to use signals; your kernel is happy to expect code that's crashed to handle its own crash recovery.
mallard wrote:
Brendan wrote:My OS's messaging? That's just signals. D-Bus? Yeah, that's just signals too. TCP/IP? That must be signals too! Obviously, all forms of communication (even spoken English) is 100% identical to Unix signals.
That is why "signals" was in quotes. "Signals" are a type of "messaging" that can exist in an OS. An implementation of POSIX signals can be built on another type of IPC and it's entirely possible to have a single messaging system that underpins all the kinds of IPC that you've mentioned. If you've implemented a way for another application to be notified when a program crashes, then you've implemented all the same mechanisms that are needed to support POSIX-like signals. I have no idea where you got the idea that "all forms of communication (even spoken English) is 100% identical to Unix signals" because I certainly did not say anything of the sort.
That's why "signals" was in quotes; and also why your previous comment was irrelevant nonsense. I gave clear reasons why Unix signals weren't ideal (e.g. the difficultly of correctly handling them when they can interrupt normal execution at any time) that were specifically directed at Unix signals, and you're trying to imply that something different (which never interrupts normal execution and doesn't have the problem) is the same.
mallard wrote:
Brendan wrote:Users have no rights (until/unless the owner of the software grants them rights, including the right to use the software in the first place).
I disagree. The user owns the hardware (or has permission to use it from the actual owner). Any software that they run is there to serve them, not the other way around. The only entity with any right to restrict what a user does is another user, not an application vendor. While I may not be as fanatical about "software freedom" as the FSF, I certainly believe that I have the right to do what I want with hardware that I own, which includes debugging applications without needed special permission from their vendors.
Users *do* have the right not to agree to the copyright owner's conditions and refuse to use the software.

For example, if I write a piece of software that manages the user's passwords and I disable debugging for that executable; then as a user you have to right to refuse to use my password manager and find (or write) an alternative password manager that doesn't disable debugging (and allows other processes, including malicious processes being run by other users, to use the debugging interface to access all of the passwords directly from the password manager's virtual address space).
mallard wrote:
Brendan wrote:The only thing required for "general purpose OS" is that the user is able to install software that wasn't a built-in part of the OS. Nobody is stupid enough to assume that an interpreter/JIT is a requirement.
The ability to run arbitrary applications is absolutely a requirement of a "general purpose" OS. An interpreter or JIT is a type of application. If it's not possible to run that entire class of application (with decent performance) on your OS, then it isn't "general purpose".
I honestly don't know if you're just pretending to believe this so you can argue, or if you really are a moron.


Cheers,

Brendan

Re: How do you use exceptions interrupts in your OS?

Posted: Thu Dec 11, 2014 3:41 am
by mallard
Brendan wrote:only a deluded fool would think it's sane to define them but provide no way of supporting them.
So, once again, you refer to people far more qualified and experience than yourself (in this case the writers of the C standard) in disparaging terms. Your arrogance really does know no bounds.
Brendan wrote: For competent developers it's standard practice, not onerous. Incompetent developers can learn to do it things right or find another OS that welcomes incompetence.
Name one commonly used desktop application that guarantees user data in the case of crash/power failure. Or are you just referring to the vast majority of experienced and qualified software developers and "incompetent"? I which case, see above.
Brendan wrote:I provided a logical reason (if people make the mistake of thinking data loss can't happen when it can just because they've got a worthless signal handler that doesn't work for various cases including power failures, then that's worse than people knowing data loss can happen and providing adequate data loss prevention techniques).
You provided your opinion. While your opinion is logically consistent, it is still an opinion, not fact. There is no strong evidence that the presence of extra safeguards causes developers to disregard good programming techniques. The presence of safety features in modern motorcars has not been shown to encourage dangerous driving, introduction of workplace safety equipment has not increased the number of accidents. Your hypothesis that extra safeguards in an OS might encourage poor programming has no compelling real-world support.
Brendan wrote:Translation: If a program wants to use signals; your kernel is happy to expect code that's crashed to handle its own crash recovery.
The kernel is not sentient, it cannot "expect" anything. All it does it send a message to an application saying "one of your threads crashed", if that application has asked to be sent that message. The provides the opportunity for it to attempt "recovery" or to save data before quitting, but there is no obligation for it to do so. It's perfectly fine for the application just to call "exit()" upon receipt of the message and attempt no recovery at all. An application with a plug-in architecture might use the feature simply to "blacklist" plugins that cause crashes, rather than attempt to "recover". There is no "expectation" of any particular behaviour.
Brendan wrote:That's why "signals" was in quotes; and also why your previous comment was irrelevant nonsense. I gave clear reasons why Unix signals weren't ideal (e.g. the difficultly of correctly handling them when they can interrupt normal execution at any time) that were specifically directed at Unix signals, and you're trying to imply that something different (which never interrupts normal execution and doesn't have the problem) is the same.
I said, several posts ago "I'm no fan how how POSIX does it". I agree that POSIX signals "aren't ideal". All I've ever advocated is providing POSIX-like behaviour (via another IPC system and some standard library code) to applications that need the compatibility. I pointed out that your messaging system is effectively the same thing, just more limited. You misinterpreted me (wilfully, I suspect).
Brendan wrote: Users *do* have the right not to agree to the copyright owner's conditions and refuse to use the software.
That's one attitude. Not one I share, but the entire discussion about "user rights" is opinion, not fact. I believe that a user has the right to do what they want with software that they've legally obtained. If you hold a different opinion, that's fine.
Brendan wrote: For example, if I write a piece of software that manages the user's passwords and I disable debugging for that executable; then as a user you have to right to refuse to use my password manager and find (or write) an alternative password manager that doesn't disable debugging (and allows other processes, including malicious processes being run by other users, to use the debugging interface to access all of the passwords directly from the password manager's virtual address spaaace).
If your security model allows "processes being run by other users" to debug applications without the user having a special access right, then your security model is severely deficient. If the user running the debugger has been given the "debug other users' applications" right, then they have been trusted by the owner of the hardware with that right and other users should be aware that their data may not be safe from that user. It's not the job of the OS to define relationships between humans.

To prevent malicious use of "debugging" rights, a system of mandatory user prompts is one possible mitigation.
Brendan wrote:I honestly don't know if you're just pretending to believe this so you can argue, or if you really are a moron.
So you seriously believe that an OS that can't run, for instance, a modern web browser (with a JavaScript interpreter) can be considered "general purpose"? Because I certainly don't.

Re: How do you use exceptions interrupts in your OS?

Posted: Thu Dec 11, 2014 8:54 am
by AndrewAPrice
mallard wrote:There is no strong evidence that the presence of extra safeguards causes developers to disregard good programming techniques. The presence of safety features in modern motorcars has not been shown to encourage dangerous driving, introduction of workplace safety equipment has not increased the number of accidents. Your hypothesis that extra safeguards in an OS might encourage poor programming has no compelling real-world support.
If we use the driving example, the more 'comfortable' a driver is, the more dangerous they are. When lanes are wider, people are more comfortable so they drive faster. When a traffic light turns green, they're comfortable that they have right away and they just go. When you place a driver in setting that's 'uncomfortable' (such as a shared zone full of trees, benches, pedestrians) they're forced to drive slow and cautiously, look around, make eye contact and signal people, and overall it's safer and more pleasant for everyone.

I don't know how easy this analogy works for programming. Perhaps assembly forces you to drive slow and cautiously because you expect there to be dangers (trashed registers, dangling pointers, memory leaks), compared to something like Java which forces you to declare explicit types, what exceptions are thrown, encourages you to design/implement specific interfaces, and use automatic garbage collection.

Re: How do you use exceptions interrupts in your OS?

Posted: Thu Dec 11, 2014 10:33 am
by Brendan
Hi,
mallard wrote:
Brendan wrote:only a deluded fool would think it's sane to define them but provide no way of supporting them.
So, once again, you refer to people far more qualified and experience than yourself (in this case the writers of the C standard) in disparaging terms. Your arrogance really does know no bounds.
No, I never said the people who designed the C standard are deluded fools that think it's sane to define SIGFPE, SIGILL, and SIGSEGV but provide no way of supporting them. In fact I did the opposite - I assumed the people who designed the C standard expect SIGFPE, SIGILL, and SIGSEGV to be supported on hardware that's capable of supporting it properly.

I also didn't say the people who designed the C standard were fools for including signals - everything was very different 35+ years ago when C was designed (and hindsight is far easier than foresight).

I will say people still using this crap after 35+ years are... actually I don't know (crippled?). It's too hard to tell the difference between the innocent (who are only victims of history) and the guilty (who perpetuated the disease until it became dogma) and I suspect they may all be both.
mallard wrote:
Brendan wrote:For competent developers it's standard practice, not onerous. Incompetent developers can learn to do it things right or find another OS that welcomes incompetence.
Name one commonly used desktop application that guarantees user data in the case of crash/power failure. Or are you just referring to the vast majority of experienced and qualified software developers and "incompetent"? I which case, see above.
Every major database manager provides strict guarantees, a lot of single-player games have auto-saves (or saves when changing areas, whatever), Google's Chrome browser saves all "session info" and uses it to restore everything after a crash (but then, so does Mozilla's Firefox). All of these things are pro-active (e.g. saving information in case there's a problem) and not re-active (e.g. saving information after a power failure).

Of course this is just a very quick list from memory. I'm not going to waste my time creating a comprehensive list just because you're too lazy to do a web search yourself (in addition to being so incompetent that you're not doing things like this yourself).
mallard wrote:
Brendan wrote:I provided a logical reason (if people make the mistake of thinking data loss can't happen when it can just because they've got a worthless signal handler that doesn't work for various cases including power failures, then that's worse than people knowing data loss can happen and providing adequate data loss prevention techniques).
You provided your opinion. While your opinion is logically consistent, it is still an opinion, not fact. There is no strong evidence that the presence of extra safeguards causes developers to disregard good programming techniques. The presence of safety features in modern motorcars has not been shown to encourage dangerous driving, introduction of workplace safety equipment has not increased the number of accidents. Your hypothesis that extra safeguards in an OS might encourage poor programming has no compelling real-world support.
My hypothesis is that encouraging poor programming practices (including providing signal handlers) tends to encouraging poor programming practices. There is no strong evidence because no strong evidence is needed. Of course it's extremely easy to find many people having problems with writing correct signal handlers so there is circumstantial evidence that signal handlers reduce safety (just not strong evidence).
mallard wrote:
Brendan wrote:That's why "signals" was in quotes; and also why your previous comment was irrelevant nonsense. I gave clear reasons why Unix signals weren't ideal (e.g. the difficultly of correctly handling them when they can interrupt normal execution at any time) that were specifically directed at Unix signals, and you're trying to imply that something different (which never interrupts normal execution and doesn't have the problem) is the same.
I said, several posts ago "I'm no fan how how POSIX does it". I agree that POSIX signals "aren't ideal". All I've ever advocated is providing POSIX-like behaviour (via another IPC system and some standard library code) to applications that need the compatibility. I pointed out that your messaging system is effectively the same thing, just more limited. You misinterpreted me (wilfully, I suspect).
None of my kernels have ever sent anything to a process that crashed (I have no idea why you think it changed since I told you this last time). Unix signals are the opposite - SIGFPE, SIGILL, and SIGSEGV are only ever sent (directly or indirectly) to the process that crashed.

Please note that it's the behaviour of signals that I've objected to. It makes no difference if it's implemented in a fast/direct way (e.g. diddling with the exception handler's stack) or if it's implemented in an slow/messy/indirect way (e.g. "message causes a task switch to a special thread that calls "raise()", which causes a second task switch back to the crashed thread"). Regardless of how it's implemented; it's still excessively difficult to handle signals correctly, and it's still silly to allow a crashed program to try to recover.

The only sane reason to do it at all is POSIX compatibility (which isn't something I think is useful - in fact I'd even be tempted to say POSIX may have caused more damage to both OS research/development and language design than anything else that's ever existed, and virtually crippled progress in multiple areas for over 20 years; and may also be the single largest contributor to Microsoft's dominance in the laptop/desktop market over the same period of time).
mallard wrote:
Brendan wrote:Users *do* have the right not to agree to the copyright owner's conditions and refuse to use the software.
That's one attitude. Not one I share, but the entire discussion about "user rights" is opinion, not fact. I believe that a user has the right to do what they want with software that they've legally obtained. If you hold a different opinion, that's fine.
If you'd like to test this; purchase any software from (e.g.) Microsoft, Sony or Electronic Arts, and then write them a letter/email claiming that as a user who has legally obtained their software you have the right to do what you want with their software (including providing copies of their software to your friends and family).

A user's rights to software isn't opinion, it's established copyright law.
mallard wrote:
Brendan wrote:For example, if I write a piece of software that manages the user's passwords and I disable debugging for that executable; then as a user you have to right to refuse to use my password manager and find (or write) an alternative password manager that doesn't disable debugging (and allows other processes, including malicious processes being run by other users, to use the debugging interface to access all of the passwords directly from the password manager's virtual address spaaace).
If your security model allows "processes being run by other users" to debug applications without the user having a special access right, then your security model is severely deficient. If the user running the debugger has been given the "debug other users' applications" right, then they have been trusted by the owner of the hardware with that right and other users should be aware that their data may not be safe from that user. It's not the job of the OS to define relationships between humans.
The "executable allows/disallows debugging" flag would only be one of multiple security measures.
mallard wrote:
Brendan wrote:I honestly don't know if you're just pretending to believe this so you can argue, or if you really are a moron.
So you seriously believe that an OS that can't run, for instance, a modern web browser (with a JavaScript interpreter) can be considered "general purpose"? Because I certainly don't.
Yes. If the user can use third party software for any purpose that the OS wasn't specifically designed for, then the OS is general purpose. My OS isn't designed for a specific purpose (e.g. hyper-visor, file server, etc) and will allow the end user to install (and create) third party software, therefore it is a general purpose OS. Whether or not it allows interpreters or JIT or java-script is entirely irrelevant.

You're trying to mix up "general purpose" with "standards compliant". An OS can be general purpose without supporting any established standards whatsoever. You can't claim that Linux is not a general purpose OS because it doesn't support Microsoft's file system permissions; or that Windows is not a general purpose OS because it doesn't support systemd; or that my OS isn't general purpose if it doesn't support Java-script.

Also note that an an OS can be (e.g.) usable for the purpose of viewing online content without supporting HTTP, HTML, CSS, Java-script, etc. You'd only have to create/invent/support your own alternative standards to use instead. Of course this would be a major compatibility problem (e.g. you'd only be able to view online content that supports the new/alternative standards), but this doesn't change the fact that it would be usable to view online content that does support your new/alternative standards.


Cheers,

Brendan

Re: How do you use exceptions interrupts in your OS?

Posted: Thu Dec 11, 2014 11:10 am
by mallard
Brendan wrote: No, I never said the people who designed the C standard are deluded fools that think it's sane to define SIGFPE, SIGILL, and SIGSEGV but provide no way of supporting them. In fact I did the opposite - I assumed the people who designed the C standard expect SIGFPE, SIGILL, and SIGSEGV to be supported on hardware that's capable of supporting it properly.
The only reason those names (which are given no defined behaviour) exist in the C standard is because C was originally developed with/for UNIX. When the UNIX standard (POSIX) was separated from C, those names ended up in C, but their behaviour (e.g. SIGFPE is raised on divide by zero, etc.) was left undefined in C, only defined in POSIX. If the people who separated C from UNIX expected that all C implementations should support the POSIX-like behaviour where possible, they should have said so in the standard. They did not, so you cannot claim that they intended that.
Brendan wrote: Every major database manager provides strict guarantees, a lot of single-player games have auto-saves (or saves when changing areas, whatever), Google's Chrome browser saves all "session info" and uses it to restore everything after a crash (but then, so does Mozilla's Firefox). All of these things are pro-active (e.g. saving information in case there's a problem) and not re-active (e.g. saving information after a power failure).
DBMSs are not "commonly used desktop applications". Web browsers and games do not guarantee data is saved, they make an effort (much like the autosave in common office applications), but if they lose power, you'll likely lose somewhere between the last few minutes work and everything in that session. No games, web browsers or office applications that I know of use fully transactional data storage (like DBMSs do), so if power is lost during the save, you've lost everything.
Brendan wrote:My hypothesis is that encouraging poor programming practices (including providing signal handlers) tends to encouraging poor programming practices. There is no strong evidence because no strong evidence is needed. Of course it's extremely easy to find many people having problems with writing correct signal handlers so there is circumstantial evidence that signal handlers reduce safety (just not strong evidence).
Define "poor programming practices". I take it you never actually looked at those Google results, because none of the first page results are references to problems writing signal handlers.

However, as I've said several times, POSIX handlers aren't great. Writing signal-safe code is hard, which is why I prefer alternative IPC systems, with the ability to "emulate" POSIX signals for compatibility with existing code only.
Brendan wrote: None of my kernels have ever sent anything to a process that crashed (I have no idea why you think it changed since I told you this last time). Unix signals are the opposite - SIGFPE, SIGILL, and SIGSEGV are only ever sent (directly or indirectly) to the process that crashed.
I don't consider that a multi-threaded program that has encountered a fault on one thread is "a process that crashed". One thread has crashed, not the entire process. Other threads are highly likely to still be viable. Your hypothetical kernel still has the capability to send a process a message when a thread crashes, you've just limited which process to send it to.
Brendan wrote:If you'd like to test this; purchase any software from (e.g.) Microsoft, Sony or Electronic Arts, and then write them a letter/email claiming that as a user who has legally obtained their software you have the right to do what you want with their software (including providing copies of their software to your friends and family).
What's illegal distribution got to do with my right to do what I want with software I've legally obtained? You haven't mentioned a "do not copy" bit or anything similar which could limit software distribution. It's not the OSs job to enforce copyright law (which changes between jurisdictions and over time).
Brendan wrote: A user's rights to software isn't opinion, it's established copyright law.
Where does copyright law say that I'm not allowed to attach a debugger to a program? In fact, "fair use" laws often specifically allow debugging/disassembly/reverse engineering for certain purposes.
Brendan wrote: The "executable allows/disallows debugging" flag would only be one of multiple security measures.
Good. It's an extremely weak mechanism on its own anyway. At the very least you'd need signed executables just to prevent somebody simply flipping the bit themselves. Then you'd have to have a method to prevent a user from re-signing it with their own signature, or prevent programs with different signatures reading each others files. All of which becomes irrelevant if somebody decides to run your OS in a VM/emulator with debugging support...
Brendan wrote: Yes. If the user can use third party software for any purpose that the OS wasn't specifically designed for, then the OS is general purpose.
But your design of an OS that has no floating-point exception support would make it impossible to implement a performant JavaScript interpreter. Thus, your "general purpose" OS isn't suitable for a common purpose; browsing the web. Maybe that was acceptable in an OS written 25 years ago, but certainly not now.

Re: How do you use exceptions interrupts in your OS?

Posted: Fri Dec 12, 2014 1:55 am
by Brendan
Hi,
mallard wrote:
Brendan wrote:No, I never said the people who designed the C standard are deluded fools that think it's sane to define SIGFPE, SIGILL, and SIGSEGV but provide no way of supporting them. In fact I did the opposite - I assumed the people who designed the C standard expect SIGFPE, SIGILL, and SIGSEGV to be supported on hardware that's capable of supporting it properly.
The only reason those names (which are given no defined behaviour) exist in the C standard is because C was originally developed with/for UNIX. When the UNIX standard (POSIX) was separated from C, those names ended up in C, but their behaviour (e.g. SIGFPE is raised on divide by zero, etc.) was left undefined in C, only defined in POSIX. If the people who separated C from UNIX expected that all C implementations should support the POSIX-like behaviour where possible, they should have said so in the standard. They did not, so you cannot claim that they intended that.
Awesome; I'll use a periodic timer and SIGSEGV to inform the process that 1 second has passed. If this causes compatibility problems for existing software, then obviously existing software is wrong because the meaning of SIGSEGV isn't defined by the C standard.
mallard wrote:
Brendan wrote:Every major database manager provides strict guarantees, a lot of single-player games have auto-saves (or saves when changing areas, whatever), Google's Chrome browser saves all "session info" and uses it to restore everything after a crash (but then, so does Mozilla's Firefox). All of these things are pro-active (e.g. saving information in case there's a problem) and not re-active (e.g. saving information after a power failure).
DBMSs are not "commonly used desktop applications". Web browsers and games do not guarantee data is saved, they make an effort (much like the autosave in common office applications), but if they lose power, you'll likely lose somewhere between the last few minutes work and everything in that session. No games, web browsers or office applications that I know of use fully transactional data storage (like DBMSs do), so if power is lost during the save, you've lost everything.
The more important the data is the more important avoiding data loss is. For some things (e.g. calculator app) nothing is needed; for other things (games, common office applications) losing a few minutes of work is good enough; and for other things (e.g. DMBSs being used for accountancy) strict guarantees aren't required.

The important thing to understand is that (for minimising data loss) signals aren't useful for any case.
mallard wrote:
Brendan wrote:My hypothesis is that encouraging poor programming practices (including providing signal handlers) tends to encouraging poor programming practices. There is no strong evidence because no strong evidence is needed. Of course it's extremely easy to find many people having problems with writing correct signal handlers so there is circumstantial evidence that signal handlers reduce safety (just not strong evidence).
Define "poor programming practices". I take it you never actually looked at those Google results, because none of the first page results are references to problems writing signal handlers.
I'm not sure which results you saw (Google has a nasty habit of giving different people different results based on their previous search history). The first page of results I see is almost all bug reports involving signals. The only one that isn't is a link to
a document by IBM explaining how to avoid signal handler bugs (and proposing "a model for dealing with reentrant functions at the compiler level" to help reduce the problem).
mallard wrote:However, as I've said several times, POSIX handlers aren't great. Writing signal-safe code is hard, which is why I prefer alternative IPC systems, with the ability to "emulate" POSIX signals for compatibility with existing code only.
So mostly the only thing you disagree with is whether or not it's worthwhile providing POSIX compatibility.
mallard wrote:
Brendan wrote: None of my kernels have ever sent anything to a process that crashed (I have no idea why you think it changed since I told you this last time). Unix signals are the opposite - SIGFPE, SIGILL, and SIGSEGV are only ever sent (directly or indirectly) to the process that crashed.
I don't consider that a multi-threaded program that has encountered a fault on one thread is "a process that crashed". One thread has crashed, not the entire process. Other threads are highly likely to still be viable. Your hypothetical kernel still has the capability to send a process a message when a thread crashes, you've just limited which process to send it to.
Let's test it by "accidentally" unmapping the wrong address (and freeing pages the executable is executing). This should trigger SIGSEGV and we can see how reliable the (now unmapped/freed) signal handler works. Alternatively, let's trash the heap with something like "memset(malloc(123), 0xCC, INT_MAX);" and see how well signal handler can allocate memory. Here's another idea - have some data protected by a mutex, and try to acquire the mutex twice (causing a deadlock), then (when the user tries to kill the process) see if the signal handler can acquire the same mutex (so it can safely access the data and write it to disk).

How about we have just have a loop that selects a random pointer and writes to it. It'd be interesting to see how often the signal handler crashes, how often the signal handler "works" (writes previously corrupted data to disk), and how often the signal handler actually works.

If something is wrong, a process may or may not crash (and I'm sure we've all seen dodgy C code that uses a dangling pointer or something that does not crash). If something is so wrong that the process actually does crash, then the risk involved in allowing it to attempt recovery is unacceptable.
mallard wrote:
Brendan wrote:If you'd like to test this; purchase any software from (e.g.) Microsoft, Sony or Electronic Arts, and then write them a letter/email claiming that as a user who has legally obtained their software you have the right to do what you want with their software (including providing copies of their software to your friends and family).
What's illegal distribution got to do with my right to do what I want with software I've legally obtained?
Your right to do what you want with software that you've legally obtained (including giving copies to friends/family and anything else), mostly doesn't exist. It's because you do not have the rights to do what you want that it's illegal to distribute it.
mallard wrote:You haven't mentioned a "do not copy" bit or anything similar which could limit software distribution. It's not the OSs job to enforce copyright law (which changes between jurisdictions and over time).
You're right; it's not necessarily the OS's job to enforce copyright law. However, if you make it hard for software developers to protect their investment, they simply won't bother spending their time and money designing, implementing and then supporting applications for the OS; and you end up with little more than crappy applications (either low quality stuff slapped together by volunteers, or ported open source stuff that makes the OS look bad because it doesn't take advantage of the OS's unique features).

For an example, consider SteamOS. I can guarantee that it will contain some form of DRM (because if it doesn't no decent games will support it, and if no decent games support it then no end users will want it).
mallard wrote:
Brendan wrote:A user's rights to software isn't opinion, it's established copyright law.
Where does copyright law say that I'm not allowed to attach a debugger to a program? In fact, "fair use" laws often specifically allow debugging/disassembly/reverse engineering for certain purposes.
Copyright law doesn't say that you're not allowed to attach a debugger to a program. Copyright law says you have no rights at all (excluding a few exceptions - educational purposes, etc). It's the copyright owner (and not copyright law) who grants you whichever rights they feel like granting, and it's the copyright owner (and not copyright law) who can say "I only grant you the right to use this software if you agree not to debug it".
mallard wrote:
Brendan wrote:Yes. If the user can use third party software for any purpose that the OS wasn't specifically designed for, then the OS is general purpose.
But your design of an OS that has no floating-point exception support would make it impossible to implement a performant JavaScript interpreter. Thus, your "general purpose" OS isn't suitable for a common purpose; browsing the web. Maybe that was acceptable in an OS written 25 years ago, but certainly not now.
"Browsing the web (with HTTP, HTML, CSS, Java-script, etc)" is just one of infinitely many possible ways of using a computer for the purpose of viewing online/remote content. Gopher is another one of those infinitely many ways. Java-script is NOT required for the purpose of viewing online/remote content.

Also note that it saddens me that people's minds are so crippled by the status quo that they can't even comprehend that alternative ways of achieving the same purpose are possible.


Cheers,

Brendan

Re: How do you use exceptions interrupts in your OS?

Posted: Fri Dec 12, 2014 1:57 am
by Combuster
mallard wrote:
Brendan wrote: The "executable allows/disallows debugging" flag would only be one of multiple security measures.
Good. It's an extremely weak mechanism on its own anyway. At the very least you'd need signed executables just to prevent somebody simply flipping the bit themselves. Then you'd have to have a method to prevent a user from re-signing it with their own signature, or prevent programs with different signatures reading each others files. All of which becomes irrelevant if somebody decides to run your OS in a VM/emulator with debugging support...
I don't intend to support anything that only serves DRM purposes in my own OS - in fact applications (by virtue of their developers) are to be considered evil by default and any auditing should be straightforward and easy. Uninstalling means uninstalling, no garbage left. All mutable and immutable resources will be on public display for the user, and anything the app wants to do outside it's extremely limited sandbox requires permission from the user in any of it's forms - and every such permission may be denied (and will be by default).

If the user can run both the application and the debugger, it can also attach that debugger by granting it that permission. Very few users will have a debugger installed though.

Re: How do you use exceptions interrupts in your OS?

Posted: Fri Dec 12, 2014 4:46 am
by mallard
Brendan wrote:Awesome; I'll use a periodic timer and SIGSEGV to inform the process that 1 second has passed. If this causes compatibility problems for existing software, then obviously existing software is wrong because the meaning of SIGSEGV isn't defined by the C standard.
Yes, strictly "standard C" software shouldn't have any adverse reaction in that case. However, most software assumes at least some POSIX-like behaviour and since your behaviour isn't POSIX complaint, that software would have difficulties. If you don't want to have any POSIX support, that's just fine.
Brendan wrote: The more important the data is the more important avoiding data loss is. For some things (e.g. calculator app) nothing is needed; for other things (games, common office applications) losing a few minutes of work is good enough; and for other things (e.g. DMBSs being used for accountancy) strict guarantees aren't required.
I agree completely.
Brendan wrote: The important thing to understand is that (for minimising data loss) signals aren't useful for any case.
Even if you don't use it to help prevent data loss, it has possible beneficial usecases (e.g. my blacklisting plugins scenario). I see no harm in supporting it. Of course my documentation won't promote it as a data-loss prevention strategy, but I won't "ban" that either.
Brendan wrote: I'm not sure which results you saw (Google has a nasty habit of giving different people different results based on their previous search history). The first page of results I see is almost all bug reports involving signals. The only one that isn't is a link to
a document by IBM explaining how to avoid signal handler bugs (and proposing "a model for dealing with reentrant functions at the compiler level" to help reduce the problem).
Most of the results I saw were problem/bug reports containing the line "signal handler triggered" and a stack trace. i.e. they were reports of other problems, with debugging information produced by a properly functioning signal handler (another possible usecase beyond data protection).
Brendan wrote:So mostly the only thing you disagree with is whether or not it's worthwhile providing POSIX compatibility.
Basically, yes.
Brendan wrote: Let's test it by "accidentally" unmapping the wrong address (and freeing pages the executable is executing). This should trigger SIGSEGV and we can see how reliable the (now unmapped/freed) signal handler works. Alternatively, let's trash the heap with something like "memset(malloc(123), 0xCC, INT_MAX);" and see how well signal handler can allocate memory. Here's another idea - have some data protected by a mutex, and try to acquire the mutex twice (causing a deadlock), then (when the user tries to kill the process) see if the signal handler can acquire the same mutex (so it can safely access the data and write it to disk).

How about we have just have a loop that selects a random pointer and writes to it. It'd be interesting to see how often the signal handler crashes, how often the signal handler "works" (writes previously corrupted data to disk), and how often the signal handler actually works.

If something is wrong, a process may or may not crash (and I'm sure we've all seen dodgy C code that uses a dangling pointer or something that does not crash). If something is so wrong that the process actually does crash, then the risk involved in allowing it to attempt recovery is unacceptable.
Worst-case, the program hangs in the signal handler and has to be forcefully killed, no different to if it hung for some other reason. If I were attempting data protection in a signal handler, I certainly wouldn't write the data to the same file, which would eliminate the possibility of overwriting good data with bad. (Microsoft Office auto-recovery works like this, including attempting to save data on crash.)
Brendan wrote: Your right to do what you want with software that you've legally obtained (including giving copies to friends/family and anything else), mostly doesn't exist. It's because you do not have the rights to do what you want that it's illegal to distribute it.
While it may just be semantics, I don't consider distributing copies to be "doing what I want with the software". It's something that happens externally to the software (unless it has a "copy me" feature).
Brendan wrote:You're right; it's not necessarily the OS's job to enforce copyright law. However, if you make it hard for software developers to protect their investment, they simply won't bother spending their time and money designing, implementing and then supporting applications for the OS; and you end up with little more than crappy applications (either low quality stuff slapped together by volunteers, or ported open source stuff that makes the OS look bad because it doesn't take advantage of the OS's unique features).

For an example, consider SteamOS. I can guarantee that it will contain some form of DRM (because if it doesn't no decent games will support it, and if no decent games support it then no end users will want it).
Honestly, I don't care about commercial development or DRM in my OS, it's a hobby, not a business plan. I won't write any, nor will I make any special effort to make it possible in my API design, but my kernel module API is completely open (and unlike Linux, has no licence restrictions), so if a commercial entity wants to add DRM to my OS, they're free to do so.

Note that SteamOS is a Linux distribution and there are existing implementations on Linux, which is an Open-Source OS.
Brendan wrote: Copyright law doesn't say that you're not allowed to attach a debugger to a program. Copyright law says you have no rights at all (excluding a few exceptions - educational purposes, etc). It's the copyright owner (and not copyright law) who grants you whichever rights they feel like granting, and it's the copyright owner (and not copyright law) who can say "I only grant you the right to use this software if you agree not to debug it".
Copyright law is way more complicated than that. Depending on the jurisdiction, there are certain rights you are assumed to have unless agreed otherwise (e.g. the ability to install/run software that you've legally obtained) and even some rights that you still have even if you do "agree" otherwise (e.g. the right to make backup copies, the right to reverse engineer software for interoperability purposes).

A copyright owner who says "I only grant you the right to use this software if you agree not to debug it", may or may not have the legal right to prevent you from debugging, depending on the purpose of the debugging and the legal jurisdiction you do it in.

See documents such as the EU Software Directive:

"In the absence of specific contractual provisions, the acts referred to [...] shall not require authorisation by the rightholder where they are necessary for the use of the computer program by the lawful acquirer in accordance with its intended purpose, including for error correction."

"The person having a right to use a copy of a computer program shall be entitled, without the authorisation of the rightholder, to observe, study or test the functioning of the program in order to determine the ideas and principles which underlie any element of the program if he does so while performing any of the acts of loading, displaying, running, transmitting or storing the program which he is entitled to do."

"The authorisation of the rightholder shall not be required where reproduction of the code and translation of its form [...] are indispensable to obtain the information necessary to achieve the interoperability of an independently created computer program with other programs, provided that the following [...]"
Brendan wrote:"Browsing the web (with HTTP, HTML, CSS, Java-script, etc)" is just one of infinitely many possible ways of using a computer for the purpose of viewing online/remote content. Gopher is another one of those infinitely many ways. Java-script is NOT required for the purpose of viewing online/remote content.
If you think that an OS without the capability to run a modern web browser will be successful and that Gopher is a valid alternative in 2014, sure, go ahead. I somehow doubt that history will vindicate that position.

Re: How do you use exceptions interrupts in your OS?

Posted: Fri Dec 12, 2014 6:50 am
by Brendan
Hi,
mallard wrote:
Brendan wrote:Let's test it by "accidentally" unmapping the wrong address (and freeing pages the executable is executing). This should trigger SIGSEGV and we can see how reliable the (now unmapped/freed) signal handler works. Alternatively, let's trash the heap with something like "memset(malloc(123), 0xCC, INT_MAX);" and see how well signal handler can allocate memory. Here's another idea - have some data protected by a mutex, and try to acquire the mutex twice (causing a deadlock), then (when the user tries to kill the process) see if the signal handler can acquire the same mutex (so it can safely access the data and write it to disk).

How about we have just have a loop that selects a random pointer and writes to it. It'd be interesting to see how often the signal handler crashes, how often the signal handler "works" (writes previously corrupted data to disk), and how often the signal handler actually works.

If something is wrong, a process may or may not crash (and I'm sure we've all seen dodgy C code that uses a dangling pointer or something that does not crash). If something is so wrong that the process actually does crash, then the risk involved in allowing it to attempt recovery is unacceptable.
Worst-case, the program hangs in the signal handler and has to be forcefully killed, no different to if it hung for some other reason. If I were attempting data protection in a signal handler, I certainly wouldn't write the data to the same file, which would eliminate the possibility of overwriting good data with bad. (Microsoft Office auto-recovery works like this, including attempting to save data on crash.)
Eww. That's almost as funny as the "do { try { something(); failed = false; } catch() { failed = true;} } while(failed);" pattern.
mallard wrote:Copyright law is way more complicated than that.
Yes; but that's not the point. The point is that my "software developer gets to choose if debugging is allowed", and Steam's DRM, and copyright law in general; all exists for the same reason - to ensure that developers/authors have an incentive to create/provide the products that users want. By weakening the incentive to create/provide the products that users want; you reduce the number of products users can choose from.
mallard wrote:
Brendan wrote:"Browsing the web (with HTTP, HTML, CSS, Java-script, etc)" is just one of infinitely many possible ways of using a computer for the purpose of viewing online/remote content. Gopher is another one of those infinitely many ways. Java-script is NOT required for the purpose of viewing online/remote content.
If you think that an OS without the capability to run a modern web browser will be successful and that Gopher is a valid alternative in 2014, sure, go ahead. I somehow doubt that history will vindicate that position.
My hope is that by creating a "legacy free" OS (with no sacrifices for compatibility and none of the burden/limitations compatibility causes) I'll be free to create something truly impressive; just like Unix developers did in the late 1960s, just like Apple did in the 1980s, and just like Microsoft did in the late 1990s. Of course I do know this is unlikely, but failing to try won't improve the already low probability.


Cheers,

Brendan

Re: How do you use exceptions interrupts in your OS?

Posted: Sun Dec 14, 2014 5:29 pm
by eryjus
Well, a couple of days has passed since my thread was highjacked for an inane (unless someone would kindly explain how it is relevant to my question) debate between 2 members. The original point of my post was to hopefully get some creative ideas on what I could do in my interrupt handlers. Unfortunately, I think the debate kept others from responding. Perhaps the problem is that my question was unclear. In either case, let me try again:

I am trying to get some ideas on how creative you can get in interrupt handlers, and more to the point how creative have you been able to get in the interrupt handlers in your OS. Can you please respond with some of the more creative things you do in your interrupt handlers to help spark my own creative thinking?

Re: How do you use exceptions interrupts in your OS?

Posted: Sun Dec 14, 2014 6:23 pm
by Brendan
Hi,
eryjus wrote:I am trying to get some ideas on how creative you can get in interrupt handlers, and more to the point how creative have you been able to get in the interrupt handlers in your OS. Can you please respond with some of the more creative things you do in your interrupt handlers to help spark my own creative thinking?
Are you after the list of common/normal things exception handlers do (that was already posted); or do you want whacky/insane things (e.g. like implementing Tetris in the invalid opcode handler)?

For the latter; I've used a small IDT (with only 14 entries) where if other interrupts occur the general protection fault handler determines which interrupt it was and passes control to the real interrupt handler. Of course this was in boot code (a "BIOS wrapper" where the general protection fault handler was in protected mode and switched to real mode before passing control to the BIOS interrupt handler), and is definitely not something I'd consider using for a kernel.

I also mess with the invalid opcode handler during boot; replacing it with an interrupt handler that returns to the address that happened to be in EDI at the time. I do this for CPU feature detection on old CPUs (e.g. set EDI to point to the address of the "feature didn't exist" code, then try the instruction and find out if the CPU supports it or not) - sort of like a crude "try/catch". This is also something I wouldn't consider using for a kernel (or for CPUs that support the CPUID instruction).

For my kernels; I tend to do tricks with GS and FS, where the kernel just assumes that GS and FS are never changed by CPL=3 code (and never saves/loads the segment registers when switching between CPL=3 and CPL=0, or when doing task switching), and if it was changed (and the kernel's assumption was wrong) a general protection fault or page fault occurs when the kernel tries to use it, and GS/FS is corrected in the exception handler. This is mostly just an optimisation (to avoid the overhead of segment register loads when I know it should always be unnecessary).

I can't think of anything else I've done with exception handlers that might qualify as "creative". Typically the exceptions are triggered by specific causes and need to be handled in a specific way that's suitable for those causes; so there's isn't that much scope for creativity.


Cheers,

Brendan