OSDev.org

cmdrcoriander wrote:

SoulofDeity wrote: Then I got to thinking, what's the point in using a virtual machine if the application is still platform dependent? It makes no sense. You're just sacrificing speed. This applies to Java as well. So, what I want to do is create a virtual machine/platform abstraction layer. It'll have a single interface for devices and absolutely will not allow peeking at hardware or executing/linking against programs and libraries not built for it. By doing this, all of the software written for it will be 100% platform independent. Porting the VM/PAL to a different platform would mean that anything ever written for it is guaranteed to work on that platform; thus the software never becomes obsolete. The only factor that matters is performance.

Uhhh... how is this not Java? Or at least conceptually really, really similar?

soulofdeity wrote:It'll have a single interface for devices and absolutely will not allow peeking at hardware or executing/linking against programs and libraries not built for it.

Java lets you do this. When someone builds a library trying to take advantage of this (eg. exposing an OpenGL interface via DLL), they are instantly locked to a specific hardware and platform. When the libraries become popular, you then have a whole bunch of hardware/platform dependant software. For 100% portability, hardware acceleration should only be performed by the VM via HLE and applications should be entirely ignorant of these things.

Rusky wrote:As soon as you start versioning the VM/PAL you're back where you were with some people stuck on the equivalent of GL 1.1.

soulofdeity wrote:It's a very ambitious project though. It requires defining new standards for pretty much everything. And they're not something that can just be changed on a whim; once they're written, they're set in stone (otherwise they'd become the cause of what they're seeking to fix).

There will never be a change in design. Only bug fixes and HLE hooks for the VM to speed things up.

That's not a solution. Even if you design a flawless system (you won't), the requirements people place on their systems will change out from under you someday and you'll still have to adjust. The only way to solve that problem is by accepting that things will change occasionally and designing the system to keep the impact of those changes to a minimum.

Rusky wrote:That's not a solution. Even if you design a flawless system (you won't), the requirements people place on their systems will change out from under you someday and you'll still have to adjust. The only way to solve that problem is by accepting that things will change occasionally and designing the system to keep the impact of those changes to a minimum.

I don't think I explained it well enough, so I'll take another stab at it. When I say, "VM" I mean literally, a Virtual Machine or console. The machine will have simulated virtual hardware. The drivers and libraries for this VM will be implemented in software for the VM itself. The libraries may change over time, but the (virtual) hardware will not. (Physical) hardware acceleration would work via HLE hooks. If the libraries update, the user just has to download the latest version of the library. But regardless, anything written form the VM will work on any version of the VM for any platform without question.

SoulofDeity wrote:

cmdrcoriander wrote: Uhhh... how is this not Java? Or at least conceptually really, really similar?

soulofdeity wrote:It'll have a single interface for devices and absolutely will not allow peeking at hardware or executing/linking against programs and libraries not built for it.

Java lets you do this. When someone builds a library trying to take advantage of this (eg. exposing an OpenGL interface via DLL), they are instantly locked to a specific hardware and platform. When the libraries become popular, you then have a whole bunch of hardware/platform dependant software. For 100% portability, hardware acceleration should only be performed by the VM via HLE and applications should be entirely ignorant of these things.

Okay, I see. I didn't realize it was possible to do that in Java (I'm not a Java programmer okay ). But isn't this basically what Java is *trying* to do? It seems to me like you're starting from the same base idea as a JVM, and your only difference is that they compromised a little, and let you use platform-specific features if you really, really want to.

That's no different. Imagine if someone had executed on this idea back in the DOS days- would their virtual hardware interface still be sufficient today? If so, would it have worked well at all back then?

cmdrcoriander wrote:Okay, I see. I didn't realize it was possible to do that in Java (I'm not a Java programmer okay ). But isn't this basically what Java is *trying* to do? It seems to me like you're starting from the same base idea as a JVM, and your only difference is that they compromised a little, and let you use platform-specific features if you really, really want to.

That's the gist of it. I basically want to provide a compromise that works the opposite way. VM software cannot ask for resources outside of the VM, but resources that are outside of the VM can poke into it with HLE hooks. The VM software has no way whatsoever of knowing about these hooks. They are totally ignorant of anything outside the VM.

Rusky wrote:That's no different. Imagine if someone had executed on this idea back in the DOS days- would their virtual hardware interface still be sufficient today? If so, would it have worked well at all back then?

It is very different. And yes, it would still be sufficient. As I said before, the hardware doesn't change. Think of it as how an emulator works. When it begins executing a rom, the first thing it does is search through the executable code and apply patches to hook OS calls. The code in the rom itself doesn't know this is happening. All it knows is that 'osLoadProgram' loads a program. It just executes faster or slower for some magical unknown reason.

So you want us to write software for what will become the equivalent of an SNES and never go beyond that? Sure, all the old stuff will continue to work as the VM gets ported to new systems, but it won't be able to take advantage of the new systems either.

cmdrcoriander wrote:

SoulofDeity wrote:

cmdrcoriander wrote: Uhhh... how is this not Java? Or at least conceptually really, really similar?

soulofdeity wrote:It'll have a single interface for devices and absolutely will not allow peeking at hardware or executing/linking against programs and libraries not built for it.

Java lets you do this. When someone builds a library trying to take advantage of this (eg. exposing an OpenGL interface via DLL), they are instantly locked to a specific hardware and platform. When the libraries become popular, you then have a whole bunch of hardware/platform dependant software. For 100% portability, hardware acceleration should only be performed by the VM via HLE and applications should be entirely ignorant of these things.

Okay, I see. I didn't realize it was possible to do that in Java (I'm not a Java programmer okay ). But isn't this basically what Java is *trying* to do? It seems to me like you're starting from the same base idea as a JVM, and your only difference is that they compromised a little, and let you use platform-specific features if you really, really want to.

Writing native (non-Java) code and executing it with Java is called the Java Native Interface (JNI).
Is this what your trying to accomplish, OP?

Rusky wrote:So you want us to write software for what will become the equivalent of an SNES and never go beyond that? Sure, all the old stuff will continue to work as the VM gets ported to new systems, but it won't be able to take advantage of the new systems either.

No. Alright, look at it like this. The (virtual) hardware give it's (virtual) drivers access to a (virtual) framebuffer. The (virtual) driver implements a software 3D renderer. Other libraries use this driver. The VM sees this driver and patches it with HLE hooks that redirect to (physical) driver functions when the code is recompiled. The VM software only knows and cares about it's virtual hardware and drivers. These work 100% standalone of the physical hardware or platform because it's all software implemented. Because it uses virtual hardware, people can write their own drivers. However, they won't run as fast unless they are recognized and patched by the VM.

I guess it might be of interest to people who don't care about the performance implications of virtualizing the hardware, but I'm not sure that's a very large target audience. It just sounds like an unholy mess to me with all sort of potential for failure and security issues. A complicated solution in search of a simple question.

iansjack wrote:I guess it might be of interest to people who don't care about the performance implications of virtualizing the hardware, but I'm not sure that's a very large target audience. It just sounds like an unholy mess to me with all sort of potential for failure and security issues. A complicated solution in search of a simple question.

When the drivers are hooked, there are no performance implications (aside from the time it takes to perform JIT compiling). Also, how can there be security issues when the software has no way, shape, or form of communicating directly with the hardware or operating system? That would be like saying that playing a gameboy game on my pc through an emulator is somehow more insecure than running the game natively.

When the drivers are hooked, there are no performance implications (aside from the time it takes to perform JIT compiling).

If you have discovered a way to virtualize, for example, a graphics card without any loss of performance then I congratulate you.

Also, how can there be security issues when the software has no way, shape, or form of communicating directly with the hardware or operating system?

Any time you allow external drivers to hook into the OS you are exposing it to security risks. It is naive to suppose that people won't be able to take advantage of these hooks.

Anyway, I wish you the best of luck and look forward to seeing your code in action.

iansjack wrote:

When the drivers are hooked, there are no performance implications (aside from the time it takes to perform JIT compiling).

If you have discovered a way to virtualize, for example, a graphics card without any loss of performance then I congratulate you.

Accepted.

iansjack wrote:

Also, how can there be security issues when the software has no way, shape, or form of communicating directly with the hardware or operating system?

Any time you allow external drivers to hook into the OS you are exposing it to security risks. It is naive to suppose that people won't be able to take advantage of these hooks.

What I'm trying to explain is that the drivers for the VM don't have anything to do with the OS. The VM is a thick iron wall between the systems applications and user applications. The user applications implement their own sandboxed drivers for the virtual hardware which are guaranteed to work, and the VM decides whether to execute these via LLE (slow), or use HLE hooks that translate those functions into functions for the physical drivers. At the very worst scenario, the software will run slow; at the very best scenario, all of the virtual driver functions translate directly to physical driver functions and the VM software is just as fast as .NET or Java. In either case, the software is 100% guaranteed to work regardless of the hardware or platform. This is something that can't be said for the other two because they allow native code execution.

The point is, the software must be written to some particular interface (which you intend to set in stone), but that interface cannot provide optimal performance for all hardware configurations and designs. This was hinted at by iansjack's comment here:

iansjack wrote:If you have discovered a way to virtualize, for example, a graphics card without any loss of performance then I congratulate you.

Try coming up with a specific design for one of these virtual drivers and we'll see how it fares.