strange bytecode idea...

[earlz]

Well, I designed this EPBP bytecode thing a while back... EPBP has a no-access code segment(not even readable) and jumping to just any address is not allowed

Code: Select all

mov r0,0x10043
jmp r0

is not allowed...

This gives the bytecode a very interesting property. It can be compiled into native code and the bytecode application would not know the difference.

Well, I thought about how there are some expensive things in an OS. The big one is context switching... another one is system calls(ring3->ring0->ring3)

Well, with this bytecode I could compile it so that paging protects memory accesses, it uses only special registers(so on context switch its ok if some registers get trashed), and a big thing: this could run from ring0 securely. (assuming no bugs in the compiler for access checking and such)

Lets ignore the IO bit and just say I have a magic in and out instruction in the bytecode for x86(even though I intend it to be portable)

The boot kernel would translate a bytecode kernel into assembly. The assembly turned into machine code and then executed. When the bytecode initiates the timer, there is a special interrupt vector table that is loaded and the bytecode kernel is interrupted much like an x86. The bytecode kernel then loads its first user program, so it will create a new virtual ring and translate-assemble the program and execute it. Now, the bytecode kernel has control over the user address space and IO access protection and all that. When the bytecode kernel does a context switch though, the way my registers work is it would take something like

Code: Select all

mov ebx,<requested register pointer> (registers are actually just a special pointer to regular memory)

Rather than a massive pop/push block.

Also, I intend for(if the bytecode OS has enough support/protection/emulation) any virtual ring 0 OS to run in a virtual ring 1(virtualization made very easy and nearly effortless)



Is there anyway this could give comparable speeds to a native OS? Is this a viable thing to attempt? And what immediate flaws do you see?

(btw, yes I know translate-assembling is very slow to do, and could take seconds)

[AndrewAPrice]

Mapping virtual registers to be pointers doesn't seem the fastest way after you translate it to run natively.

[earlz]

Mapping virtual registers to be pointers doesn't seem the fastest way after you translate it to run natively.

Yea, I thought about that when I started typing this... but I could make it so you must use like a synch instruction to synch registers with memory which would easily enable virtual register mapping onto hardware registers..

[JohnnyTheDon]

Well, I thought about how there are some expensive things in an OS. The big one is context switching... another one is system calls(ring3->ring0->ring3)

Well, with this bytecode I could compile it so that paging protects memory accesses, it uses only special registers(so on context switch its ok if some registers get trashed), and a big thing: this could run from ring0 securely. (assuming no bugs in the compiler for access checking and such)

So essentialy you're suggesting Software Isolated Processes, right? If so, yes, you can get speeds as fast as native OSes if your intermediate bytecode is good and your native assembler is good. The one thing you have to worry about is when the hardware has a hiccup. See the discussion on this thread.

As for the registers, I wouldn't have registers in bytecode. I'd kepp local variables as local variables so the assembler on the system can allocate registers and stack it how it wants. That allows your code to run on RISC, CISC, load-store, etc. without worrying about wasting registers.

(btw, yes I know translate-assembling is very slow to do, and could take seconds)

Yeah, but you could cache stuff that was previously assembled so it doesn't have to be assembled again, as long as nothing changed.

[earlz]

As for the registers, I wouldn't have registers in bytecode. I'd kepp local variables as local variables so the assembler on the system can allocate registers and stack it how it wants. That allows your code to run on RISC, CISC, load-store, etc. without worrying about wasting registers.

How would local variables be expressed in a bytecode though? and the register convention was really only meant as a space-saver so that memory addresses were not required every instruction. Oh, and no! I do not want a stack machine.. those are much harder to program, even if they are easy to optimize/emulate

[JohnnyTheDon]

You could just have numbers for them. Aka 0x0, 0x1, 0x2, etc. and then allocate them to the stack or registers when assembled.

[Troy Martin]

I had an idea for a 16-bit machine with no registers, all "control points" (the important control registers for calling functions and such) are somewhere in the last 4KB of memory, 0xF000.

Basically the machine code is built as a set of NASM macros that expand to DBs and DWs. NASM is the assembler for the machine even though it's nowhere near x86

[Colonel Kernel]

So essentialy you're suggesting Software Isolated Processes, right? If so, yes, you can get speeds as fast as native OSes if your intermediate bytecode is good and your native assembler is good. The one thing you have to worry about is when the hardware has a hiccup. See the discussion on this thread.

You also might need to worry about the overhead of extra run-time checks (array bounds, dereference, type tests, etc.)

Just want to mention that preemptively so that it doesn't start another religious war.

[earlz]

So essentialy you're suggesting Software Isolated Processes, right? If so, yes, you can get speeds as fast as native OSes if your intermediate bytecode is good and your native assembler is good. The one thing you have to worry about is when the hardware has a hiccup. See the discussion on this thread.

You also might need to worry about the overhead of extra run-time checks (array bounds, dereference, type tests, etc.)

Just want to mention that preemptively so that it doesn't start another religious war.

type-tests? I don't think that applies to my bytecode... it isn't a horrid high level bytecode like java and such. Also, it isn't "managed." (it uses good ol' malloc and free type allocation)

Array bounds? My bytecode is very similar to a real CPU arch. There is no built-in array type stuff.

Paging can be used to remap and check that addresses are within bounds and valid.

[JohnnyTheDon]

So essentialy you're suggesting Software Isolated Processes, right? If so, yes, you can get speeds as fast as native OSes if your intermediate bytecode is good and your native assembler is good. The one thing you have to worry about is when the hardware has a hiccup. See the discussion on this thread.

You also might need to worry about the overhead of extra run-time checks (array bounds, dereference, type tests, etc.)

Just want to mention that preemptively so that it doesn't start another religious war.

A very good idea. The only reason I didn't mention it because I took a look at earlz's project before and was aware it wouldn't need runtime checks for that type of stuff.

[Colonel Kernel]

It doesn't sound like software isolation then, since paging would still be used for memory protection. I saw "run safely in ring 0" in the original post and assumed that meant shared address space, but I guess it just means ensuring that the bytecode can never be translated into privileged instructions...?

[earlz]

ok, I'll look at that... and yes, the only protection is ensuring that the bytecode is not translated into priviledged instructions and also making sure it can't do any x86 code insertion

[gzaloprgm]

The Go! OS a while ago explored "code sanitizing" techniques.by tracing x86 code execution and rejecting loading a component if it used one of the privileged instructions. Since components were very small in size this didn't incur any major slowdown. Everything ran in ring0.

Hmm, but that has a big bug that can be exploited!

a) If the tracer only reads instructions from .text section, a shellcode may be in .data or .rdata.
b) If the tracer simulates every instruction, it may get hang or may skip some insecure code that can be activated later, once the process knows it has been loaded in memory and is really executting.
c) If the tracer doesn't implement some functions, its easy to exploit.

I think the solution to b is to split EACH conditional jump in two different threads, what do you think?

Cheers,
Gonzalo

[Combuster]

In any case that system's still leaky. I could overflow an array and write straight into kernel memory. You could lock everything down to nonconstant jumps but that'd break class support as normally implemented.

[gzaloprgm]

In any case that system's still leaky. I could overflow an array and write straight into kernel memory. You could lock everything down to nonconstant jumps but that'd break class support as normally implemented.

Yes, you are right. But the idea is still good for harvard architecture based microprocessors (For example, Pic, AVR MCUS, etc). There you have strictly separated code and data sections and you can't "execute data" to exploit things. The problem then would be predicting the execution and checking safety the fastest way possible. Unsafe instructions such as STI, CLI, HLT, etc may be detected, but pre detecting writes into kernel memory is almost impossible unless the code is 100% static and does not depend in ANY input from the outside.

Cheers,
Gonzalo