OSDev.org

Why can't we solve the protection in the software layer, instead of the hardware?

For example:
Protection rings. When you compile a user space application that needs to run in less priviliged mode, why can't the compiler complain when you're using priviliged code? And thats all there is.

Data execution prevention: Again, can be solved in software level. When you try to execute an address that sits in a R/W section, the compiler will complain.

The only real problem occures when some malwared user is trying to inject something. Then, hardware protection is good to go. I still think there can be a software solution for that, but maybe its a massive overhead.

Sourcer wrote:Data execution prevention: Again, can be solved in software level. When you try to execute an address that sits in a R/W section, the compiler will complain.

And if a rogue user compiles a program with a compiler that doesn't include such protection, or even hand crafts the assembler instructions... ?

Hi,

Sourcer wrote:Why can't we solve the protection in the software layer, instead of the hardware?

We've tried. The problem is that it's slow and expensive (because hardware can do a lot of the relevant checks very efficiently in parallel while software can't) and it's impossible to make it 100% effective (e.g. can't defend against things like compiler/virtual machine bugs and hardware flaws).


Cheers,

Brendan

I actually believe that managed code will be the future someday not too far from now, at least on general-purpose machines. You can already spot a couple of relevant trends:

• We are starting to see formally verified compilers emerge, which reduce the likelihood of vulnerabilities ("reduce" rather than "eliminate" because there can obviously still be errors in the specification---but you can't ever be certain of anything, software or hardware, you can only have different degrees of certainty).
• An increasing number of programs are written in managed languages, such as Java or C#. Furthermore, we can also see a shift towards a more functional programming style in both new and existing languages.

Protection is a matter of trust.
Does the hardware trusts itself ( Ram errors , for example) ?
Can you trust an unknown assembly binary blob ? ( Do you use flash, or Skype ?)

You can design a easily isolated binary blob, C# can do that with app domains and assemblies). But it has a cost : you have to make a virtual machine ( virtual hardware) which has its own protections .

Now, without #GPF, I'm pretty sure OSes would be a little less stable.

First off, it isn't required that you use the hardware protections; many systems don't, and quite a bit of research was done in the 1970s about eliminating hardware support for several common safety features.

However, these features became ubiquitous, and remain so, for a reason. A lot of these things are much more efficiently implemented in hardware, as Brendan states, while others would eliminate the use of many common software techniques used to improve efficiency in use today.

More importantly, as Boris said, is the matter of trust, and specifically, the scope of trust. You implicitly trust your hardware already, otherwise you wouldn't be using it; using a software implementation of something that can (often more effectively) be done in hardware adds to the network of trust, and each added level increases the complexity of the network by a factor equal to the number of interactions between the trusted elements.

Furthermore, it would require you to absolutely limit the ways that programs can be written to those which are themselves trusted - everything would have to be 'managed code', or at least generated by a tool that is itself 'trusted code'. That rules out, among other things, general use of assembly language or any tools that operate directly on the executable (regardless of whether it is native or for a pseudo-machine),

In the end, if you choose not to use a protection mechanism that the hardware already provides, you are removing a level of protection, even if you have a software implementation of that protection mechanism, and you are increasing the performance burden of the system. Even for a 'managed' system (and I have explained before - and more than once - why I consider that term to be meaningless marketing-speak), it makes sense for the system itself to use those mechanisms, abstracting them away but still applying them for the underlying super-OS.

We need hardware and software protection for normal users who don't quite know what is going on in a computer system or in their usual OSes.

It would also be nice to be able to completely disable all hardware and memory protection and run the whole system exclusively in Ring 0 such that we could manipulate processes and access hardware without worrying that things will crash randomly. It would be ideal for system developers that envision to constantly implement new OS architectures or that need the highest efficiency because they don't fear that software would destroy their data because they really know what they/we are doing and recognize the extreme value of a system fully run exclusively in Ring 0 and that it can be enabled or disabled.

There's even a name for that type of kernel in the OSDev Wiki.

It's called Megalithic Kernel, and it should be a mode to run the OS for Windows, Linux or any other OS that we should be able to enable or disable at will (enable for high/low level development, and disable for regular user usage):

http://wiki.osdev.org/Megalithic_Kernel

Hi,

~ wrote:It would also be nice to be able to completely disable all hardware and memory protection and run the whole system exclusively in Ring 0 such that we could manipulate processes and access hardware without worrying that things will crash randomly.

Even if you can guarantee that all code is 100% correct and 100% safe (not just for normal use, but under "deliberate/malicious attack" conditions), and can also guarantee all hardware is 100% reliable and 100% safe (e.g. "root-kit proof"); you'd still want to use the hardware's checks for various reasons. Most OSs use page faults to support things like swap space, memory mapped files, copy on write, etc. A lot of OSs use the invalid opcode exception for emulating features/instructions that the CPU doesn't actually support. Some OSs use the general protection fault for various tricks (e.g. "virtual8086 monitor", IO port permission schemes, "lazy GS load", etc).

If you disable/remove all the checks; then all of the other things (where the checks are also used to implement various features and/or improve performance) end up having to be done in pure software too (and not just security). Just implementing one tiny and trivial piece of it (e.g. having an "accessed" bit for each page so you're able to figure out "least recently used") would halve performance all by itself.


Cheers,

Brendan

At least mere hardware protection based on blocking access to hardware in levels (including memory) is different from hardware faults in response to things like pages marked as absent or hardware that somehow is signalling that we still need to wait to command, send or receive data, or initialize it fully or in stages.

So we could grant unrestricted access in Ring 0 while keeping fault capabilities running in Ring 0 for being able to manage paging and proper hardware manipulation but without blocking neither hardware or memory access in such a way that absolutely any piece of code running in that system could do absolutely anything possibly imaginable it wanted as if it was in Real Mode.

Hi,

~ wrote:So we could grant unrestricted access in Ring 0 while keeping fault capabilities running in Ring 0 for being able to manage paging and proper hardware manipulation but without blocking neither hardware or memory access in such a way that absolutely any piece of code running in that system could do absolutely anything possibly imaginable it wanted as if it was in Real Mode.

Yes, you can do protection much slower in software; even though the hardware supports it, and even though you're using the hardware's support for multiple other things anyway.

However; if the majority of users are complaining that your software too fast I'd still just take the easy way (e.g. insert dummy "while()" loops to slow things down) before I wasted my time trying to use "inadequate protection in bloatware" to fix the "software doesn't suck badly enough" problem.


Cheers,

Brendan

~ wrote:We need hardware and software protection for normal users who don't quite know what is going on in a computer system or in their usual OSes.

It would also be nice to be able to completely disable all hardware and memory protection and run the whole system exclusively in Ring 0 such that we could manipulate processes and access hardware without worrying that things will crash randomly. It would be ideal for system developers...

If an application clobbers your display driver and the user sees a bunch of garbage instead of a UI, that counts as a random crash. If an application masks interrupts and halts, or masks interrupts and goes into an infinite loop, that counts as a random crash. Neither of those can happen with proper protection in place, both happened fairly regularly on old systems that didn't implement protection, or ran on hardware where they couldn't implement protection. Furthermore, even worse than a crash is if an application clobbers your disk driver and the system appears tp continue running normally, except that as a result of the driver clobbering, all file writes end up silently corrupting filesystem data structures on disk, with the result that the system doesn't crash immediately, but a great number of files end up lost, and the system crashes the next time you try to boot it. With proper protection in place, the OS will kill the application when it tries to clobber the disk driver, so the application will randomly crash, but the integrity of the system will be maintained.

And don't give us this BS about it being ideal for system developers. The first step to becoming a good developer is to understand that *all* developers are fallible, and *this means you*.

~ wrote:There's even a name for that type of kernel in the OSDev Wiki.

It's called Megalithic Kernel, and it should be a mode to run the OS for Windows, Linux or any other OS that we should be able to enable or disable at will (enable for high/low level development, and disable for regular user usage):

http://wiki.osdev.org/Megalithic_Kernel

There is a reason why Paleolithic, oh, sorry, Megalithic kernels have been abandoned for all but the simplest embedded projects: they scale horribly, even (or especially, given real-time constraints when servicing multiple functions) for embedded systems. They are a relic of the past, and have no future to speak of, cannot have any future to speak of, because a system that lacks multitasking will never make adequate use of the CPU and other resources.

I keep telling you that this is not a match for what you seem to be looking for, and that an Exokernel would be closer to what you seem to want, but you seem to be fixated on MS-DOS as some kind of ideal.

Let me be as blunt as possible: FOR A CURRENT-GENERATION OS, RUNNING IN REAL MODE IS A FAILURE, NOT A FEATURE. It does not simplify the system, it makes it needlessly complicated because you spend your time getting around its limitations, usually by leaving real mode at the first opportunity. I remember just what a pain in the butt working in MS-DOS was, and trust me, while few people went out of their way to switch to Win95 immediately, almost no one regretted that they switched eventually (or more typically, never used MS-DOS in the first place, because in 1995-97 more computers pre-loaded with Windows 95 were sold to first-time owners than were sold for all computer types to all user types prior to 1995).

In any case, this is going to be irrelevant soon, as the next generation of Intel x86-64 processors is reportedly going to eliminate real mode and 32-bit protected mode entirely, and boot exclusively in long mode (with p-mode emulation similar to the old V86 mode). Some of the low-power embedded models (e.g., the Quark CPUs used in the Edison SBC) already have dropped real mode, and with the growing support for UEFI, it is likely that x86 will be 64-bit only by 2018 - assuming that the world hasn't finally come to its senses by then and dropped x86 as the junk it is, though I doubt that will happen even though everyone, including Intel, has been looking to do just that for nearly twenty years.

Schol-R-LEA wrote:In any case, this is going to be irrelevant soon, as the next generation of Intel x86-64 processors is reportedly going to eliminate real mode and 32-bit protected mode entirely, and boot exclusively in long mode (with p-mode emulation similar to the old V86 mode). Some of the low-power embedded models (e.g., the Quark CPUs used in the Edison SBC) already have dropped real mode, and with the growing support for UEFI, it is likely that x86 will be 64-bit only by 2018 - assuming that the world hasn't finally come to its senses by then and dropped x86 as the junk it is, though I doubt that will happen even though everyone, including Intel, has been looking to do just that for nearly twenty years.

Wait, is there any source for this? I can only find info about Linux distros dropping 32-bit builds.

Not questioning it though, the only reason why real mode was still around is to allow booting from BIOS, if you're using UEFI it's completely useless as it can do the job for you. Seeing as it results in a lot of clutter in the circuit (on an architecture where they already struggle out of sheer complexity) it'd be probably only a matter of time for it to be dropped.

In any case, at this point Linux is more capable already than Windows itself to run down to 16-bit Windows applications.

If you use for example Ubuntu x64 with Wine, you can perfectly run Win16 programs and screen savers perfectly.

It's something to thank because there are always engineering programs for Win16 that no longer run in x64 Windows, specially for people who are engineers but not programmers.

I'm sure that if 16 and 32-bit modes disappear from the x86 CPUs, a new processor architecture from Intel would appear which would use Intel-formatted instructions.

It could be called i64 to make completely clear that it would only run in 64-bit mode, such as the i386 was the first to give access to full 32-bit mode.

First off, it isn't required that you use the hardware protections; many systems don't

On the one hand, some because they're advanced research projects into the use of software protections and virtualization.

On the other, some are just old as balls. Of course, yet again, I'm saying this because of my screwing around in the guts of my old macs of late. I love the hell out of my classic macs, but goddamn is it funny to see the internals of an operating system that was designed before people ever really thought realistically about the possibility of malicious code on a personal machine.

"Oh, we'll just toss all of the applications into the same flat memory space and just tell them they all have to use IP-relative addressing and promise to behave. It'll be fine."

And, lo, for a time it was.