Real Mode (was: Some Questions about continuing with OS...)

[~]

If we feel better in Real Mode or if we feel constantly going back using, finding and learning from resources from the 90's and from the ISA/PS/2 PC's, then it's because that's our current level of knowledge and expertise, and we would do best to just agree with ourselves that fact, and accelerate our learning to other levels by learning the basics from the environment that actually holds all of the basics, which happens to be Real Mode/DOS/DPMI/Win9x/loadlin, whatever looks slightly like software fore bare-metal standard hardware/software/firmware, including usual and useful file formats of all kinds.

If you feel better in real mode, you haven't done very much in protected mode. The initial setup is a bit more work than real mode, but the improved flexibility is worth it.

To make a lot in Protected Mode and higher, I would need to make an OS, and I'd still need to stay capable to run Real Mode code natively (for that I need to learn everything about it and enjoy understanding the nice demos there, whereas I have never seen native 64-bit demos or a popular unprotected 64-bit OS, so why not start in 16-bit mode and build from there to make things more stable?). Other than that, there are HTML5 applications to do, native programs, and a little bit of instrumentation applications.

At the very least I would need to do something like reimplementing DPMI to natively compile and run DPMI programs.

The farther I go from Real Mode, the less freely-available or easily-understood material there is.

I almost pay no attention to the fact that I run under 32-bit Real Mode or under DOS because all the code I write is thought to be platform-independent.

I have rewritten my code so many times and I have realized that there's so much work for a single person that the only thing that could be acceptable to write as non-portable code is the base skeleton of an application, which is designed to actually provide an entry point for a specific OS/hardware, so it doesn't matter.

But the rest of the code doesn't have to ever worry about under which sort of environment it's running in, no matter if it's just written in assembly without a linker.

Add to it the fact that from Real Mode we can start DPMI/32-bit applications or native 32/64-bit ones that stay in that mode while it's running, and we can enjoy from ease, platform-independent convenience, and an environment that, unlike end-user OSes, doesn't restrict hardware access (aren't we in OS Development anyway? Why not take the capabilities of all modes to the same level of importance and with the same modern applications, from 8 to 64 bits and higher?).

The question is: Will there really be a way to intensively hack the CPU so that we can use EIP and ESP/EBP in pure Real Mode so we can use huge 16-bit programs without worrying about segment registers, leaving them loaded with 0?

Had you read the wiki page on unreal mode, you would already know that the answer to this question is yes. However, it has some severe limitations that make it useless for anything serious. It would be less work to just switch to protected mode and be done with it.

Could we maybe use 32-bit prefixes like...?:

I'm honestly not sure if that would work. That's pretty obviously going beyond what the CPU was ever intended to support, so you'll probably stumble across some unexpected behavior.

Protected mode without protection

Switching to protected mode without actually enabling any protection features is trivial. You should try it, it's actually less work than setting up unreal mode!

Will there be a possible way to enable 16/32/64-bit code/data/stack/instruction pointer from Real Mode, like Short Mode?

No.

This is something that will have to be proven under heavy hacking. At least a 16/32-bit code/data package in UnReal Mode seems possible. This is how Unreal Mode, Mode X graphics modes, going from 16 to 64-bit mode and similar things were discovered and documented anyway: Just trying the limits.

[Brendan]

Hi,

If you feel better in real mode, you haven't done very much in protected mode. The initial setup is a bit more work than real mode, but the improved flexibility is worth it.

To make a lot in Protected Mode and higher, I would need to make an OS, ...

You'd be an operating system developer that actually develops an operating system (on a forum dedicated to people that make operating systems)?

... and I'd still need to stay capable to run Real Mode code natively (for that I need to learn everything about it and enjoy understanding the nice demos there, whereas I have never seen native 64-bit demos or a popular unprotected 64-bit OS, so why not start in 16-bit mode and build from there to make things more stable?). Other than that, there are HTML5 applications to do, native programs, and a little bit of instrumentation applications.

Why? Real mode is deprecated puss for people with mental disabilities. The reason you've never seen native 64-bit demos is that people that don't have mental disabilities are more interested in doing things that are actually useful (and not interested in things like "how to get command line length in DOS" demos).

At the very least I would need to do something like reimplementing DPMI to natively compile and run DPMI programs.

Why? Nobody cares about ancient DPMI crud anymore, not even the people that invented it (Microsoft).

The farther I go from Real Mode, the less freely-available or easily-understood material there is.

The further you go from worthless puke that nobody cares about, the more useful your material will be.

I almost pay no attention to the fact that I run under 32-bit Real Mode or under DOS because all the code I write is thought to be platform-independent.

No, you're obsessed with retarded nonsense (like "how to get command line string length in DOS") that is tied directly to DOS and completely and utterly worthless for everyone because it is tied directly to DOS.

I have rewritten my code so many times and I have realized that there's so much work for a single person that the only thing that could be acceptable to write as non-portable code is the base skeleton of an application, which is designed to actually provide an entry point for a specific OS/hardware, so it doesn't matter.

But the rest of the code doesn't have to ever worry about under which sort of environment it's running in, no matter if it's just written in assembly without a linker.

Application development is for application developers and not for OS developers (and is therefore not something that really belongs on an OS development forum).

Add to it the fact that from Real Mode we can start DPMI/32-bit applications or native 32/64-bit ones that stay in that mode while it's running, and we can enjoy from ease, platform-independent convenience, and an environment that, unlike end-user OSes, doesn't restrict hardware access (aren't we in OS Development anyway?

You're standing in a room full of around 200 operating system developers, where every one of those ~200 people have unrestricted hardware access without DPMI (in their boot code, their kernel, etc); and you're trying to tell those ~200 people that they need DPMI for unrestricted hardware access?

Why not take the capabilities of all modes to the same level of importance and with the same modern applications, from 8 to 64 bits and higher?).

Because "<= 16-bit" imposes limitations, and modern applications have grown beyond those limitations. Essentially, modern applications and "<= 16-bit" are mutually exclusive, and can't co-exist.

Will there be a possible way to enable 16/32/64-bit code/data/stack/instruction pointer from Real Mode, like Short Mode?

No.

This is something that will have to be proven under heavy hacking. At least a 16/32-bit code/data package in UnReal Mode seems possible. This is how Unreal Mode, Mode X graphics modes, going from 16 to 64-bit mode and similar things were discovered and documented anyway: Just trying the limits.

For "anything 64-bit" the CPU decodes instructions differently and must be in "64-bit mode" so that it does decode instructions differently. If you're in real mode or unreal mode you can be in "64-bit mode"; so using anything 64-bit (64-bit addressing, half the CPU's general purpose registers, etc) is impossible.

For 32-bit EIP; you can probably load a 32-bit value like 0x12345678 into EIP using size overrides, but the CPU will only fetch instructions using IP and would fetch instructions from "cs:0x5678" so there's no point.

For using 32-bit stack; there's only 2 cases:

• You care about existing interrupt handlers designed for real mode (e.g. BIOS IRQ handlers), and therefore it can't work because the existing interrupt handlers won't work for 32-bit stacks (e.g. won't use the size overrides that would be necessary)
• You don't care about existing interrupt handlers designed for real mode; and (if "32-bit stack" can work at all) you'd gain nothing beneficial from it that you wouldn't have got from just using protected mode in the first place (and would only gain things that aren't beneficial, like complexity/hassle and a much higher risk of compatibility problems with things like firmware's SMM code).

Essentially, it either won't work or it'd be a pointless waste of time if it did work, so there's no (rational or irrational) reason to care whether it works or not.


Cheers,

Brendan

[Schol-R-LEA]

It's not impossible. Think about porting an application that has been properly encapsulated and defined, which is intended to write on the screen, or functions to perform arithmetic operations for a calculator.

Have you looked into the language Forth at all? Because this sort of fine-grained encapsulation of small packets of executable code is basically how a Forth interpreter works - depending on how the interpreter 'threads' the code, it can basically just turn the Forth words into branch or call instructions, and most forms of the language include an assembler as well.

If you think about how to compile x86 assembly to other native executable code, you will find that it's basically the same as compiling C, and we aren't using anything exotic, just clean x86 assembly well-defined to be cross-platform.

This makes no sense. Are you talking about an emulator running on some different CPU (or even the same CPU, as is the case with Bochs and DOSBox - DOSBox doesn't actually run x86 directly in hardware, it simulated a real mode CPU - or do you actually mean translating the x86 assembly language into an unrelated assembly language or machine code? By that I mean, map something like this in x86 assembly

Code: Select all

add rax, [rbp + 4]

where you are getting an argument off of the stack, into something like this in MIPS

Code: Select all

add $t0, $t0, $a0

where the size of register file, way that arguments are passed, the way the stack is used, the way subroutine calls are made, etc. are radically different? It could be done, for those parts of the code that don't reference hardware-specific features or external peripherals, but why?

If for example we are masters in using SSE and we want to use it in other architecture, it would be very useful if we already wrote an x86 application.

SSE only exists in x86, and only x86 processors within the past fifteen years at that. Most other processors have nothing equivalent to it, at least not as hardware.

I almost get the impression here that you think the CPU executes the assembly code itself. What do you think the assembler is actually doing?

If it wasn't possible to compile one assembler into a different object code, then it wouldn't be possible to use PCI, accelerated 3D graphics cards, IEEE-754-compliant FPUs or USB in architectures other than x86 PCs either.

That's entirely a non-sequitur. The bus isn't a processor; it's a means for transmitting signals from one device to another, and while PCI and it's successors do generally have some sort of DSP for managing the traffic, AFAIK, their internal software is not exposed to the CPU - they can be given certain commands, but they can't really be fully reprogrammed the way that, say, the APIC or the GPUs can. Similarly, the USB controllers, SATA controllers, etc. only receive operational commands from the CPU, not actual opcodes.

The GPUs have an instruction set that is entirely unlike that of the x86, and for the most part, the CPU never works on that code, either. It has more opportunity to do so indirectly, by way of the low-level representation the shading language (though in most cases that's actually emulated or re-translated into the actual GPU opcodes rather than run directly, I think), but when it does, it isn't sending x86 opcodes to the GPU, it is sending a block of raw binary with the GPU's shading language code. There is no overlap in them at all, either at the source code level (assembly language) or the machine code level (the actual pattern of bytes that represents the operations).

The FPU on a modern x86 family system is actually incorporated into the CPU (since the 80806DX), though it, too, has a separate instruction set; the fact that most modern x86 assembler can assemble the FPU instructions (or the similar SSE instructions, which are also a separate instruction set embedded in the x86 set) and generate this code as part of assembling an x86 program is a feature of the assembler, a convenience rather than an actual equivalence in the instruction sets.

About DOS, I know that many companies have tried to simply erase it due to the fact that everything that is written for it is so tiny that it's very easy to disassemble and port to other architectures and OSes, specially drivers.

What? This makes little sense, and sounds almost conspiratorial in mindset on your part. The reason that most companies today don't support DOS is because plain DOS has a hard limit or 640K plus the High Memory Area's 48K - it simply isn't capable of handling enough memory to support most modern peripherals, especially graphics cards. While a DOS-like monotasking OS could be written that does use protected mode (several do exist), they would not actually be MS-DOS or even FreeDOS - that's why FreeDOS-32 is a separate project from FreeDOS. It is also why 'DOS extenders' such as DPMI and Windows prior to Windows 95 were basically separate operating systems that took over from DOS once started.

As for 64-bit systems, once they are in long mode they are no longer able to run 16-bit real mode - not even as a virtual machine - except through simulation, which puts the DOBox programs on the same level as pseudocode interpreters such as Java's JVM or .Net CLI. Yes, if the code only uses the 156-bit anmd 8-bit registers, the assembly instructions - and in most, but not all, cases the opcodes - are the same, but the processor simply does not and cannot run them the same way it does in real mode.