running 32-bit code in LM64

[Octocontrabass]

I didn't see one to get AHSHIFT.

There isn't one, but you can calculate AHSHIFT from AHINCR since it's always a power of 2 anyway.

But is that the totally wrong thing for DPMI to be used for? It doesn't give you values for the current (RM16) situation, it only ever gives values for a future shift to PM16?

It doesn't give you anything in real mode. The DPMI INT 0x31 functions can only be called in protected mode.

If there is prior art (ie DPMI) to explain the *current* situation, I'm happy to reproduce that (small) functionality in PDOS-16-whatever.

Not as far as I know. (But there's nothing stopping you from providing your own "DPMI without PM" on INT 0x31 in real mode...)

[kerravon]

I didn't see one to get AHSHIFT.

There isn't one, but you can calculate AHSHIFT from AHINCR since it's always a power of 2 anyway.

Is there a neat way of doing that, or do I need to shift one bit at a time?

But is that the totally wrong thing for DPMI to be used for? It doesn't give you values for the current (RM16) situation, it only ever gives values for a future shift to PM16?

It doesn't give you anything in real mode. The DPMI INT 0x31 functions can only be called in protected mode.

Well that's exactly what I need then. So if the carry is set, either DPMI doesn't exist, or it exists, but I'm in RM, so - stay with the hardcoded defaults suitable for an 8086.

If there is prior art (ie DPMI) to explain the *current* situation, I'm happy to reproduce that (small) functionality in PDOS-16-whatever.

Not as far as I know. (But there's nothing stopping you from providing your own "DPMI without PM" on INT 0x31 in real mode...)

Exactly what I need again then - I'll provide my own suitable for the Turbo 186, and all "conforming apps" will get the proper values.

And while I'm at it - what does DPMI PM16 give you anyway? INT 21H works exactly as before, but you need to manipulate the selectors in accordance to the correct AHINCR?

And I would still need NE executables - it's just that instead of using the OS/2 or Windows DLL to obtain the Huge Shift value, they would instead be making INT 31H calls to get those values?

Which my code would have been doing anyway.

So - this is basically standard MSDOS (INT 21H) code, but relinked as an NE executable.

And I just need to take some care in places, like presumably not manipulating B8000H directly but stick to INT 21H. Which is something my apps are already doing.

That sounds pretty good to me.

[nullplan]

Is there a neat way of doing that, or do I need to shift one bit at a time?

On x86, there is the bsf instruction. More generally, the operation you seek is called "ctz" (count trailing zeroes), or, more rarely, "ffs" (find first set). The latter is actually the name of an XSI function that does that.

Generally, there are a couple of strategies to do it:

• Use a CPU instruction that does that, or does the opposite (count leading zeroes).
• Use the DeBruin method.
• Use a population count method on "(x-1)&~x".

Yeah, this rabbit hole goes deep.

[rdos]

DPMI was not designed to be implemented in RM. It's an interface to manage selectors in protected mode that has a bit of sharing functionality. Also, there is no "segment shift" value in PM. Each selector is allocated individually with a base & size, so segment shifts have no relevance to PM. Some poor 16-bit C compilers implement "huge segments" by stacking several selectors, but I wouldn't use that. If you need that kind of large segments, use PM32 instead.

[kerravon]

DPMI was not designed to be implemented in RM. It's an interface to manage selectors in protected mode that has a bit of sharing functionality. Also, there is no "segment shift" value in PM. Each selector is allocated individually with a base & size, so segment shifts have no relevance to PM. Some poor 16-bit C compilers implement "huge segments" by stacking several selectors, but I wouldn't use that. If you need that kind of large segments, use PM32 instead.

PM32 won't work on an 80286. PM16 will. I exactly want my 8086 applications to shine on an 80286 by suddenly getting access to 16 MiB of memory instead of being limited to 640k/1 MiB.

I believe some military has a slogan "be the best you can be". That's what I want for my 16-bit apps.

I'm organizing an OS to make those apps shine.

I'm probably going to end up doing it in reverse though. Allow OS/2 1.x apps to run on the 8086. But the Family API already allows that. So it's more that I want to move the Family API into the 8086 OS instead of polluting the executable.

And then I still need a somewhat clone of OS/2 1.x.

[rdos]

Sure, you would want to use all the 16MB of memory in an 286, but that requires using PM16. PM16 is a segmented environment with 8k global selectors (GDT) and 8k local selectors (LDT). You cannot access the 16MB as an uniform memory area in a 286, since segments can not be larger than 64k. That means you need to use multiple selectors that map your in-memory objects. Intel defined the segmented model so different memory objects would be mapped to different selectors. An address in PM16 consists of a selector and a 16-bit offset, not a 16MB offset from zero. In fact, and address in RM is the same: It consists of a segment and a 16-bit offset. While you can use pointer manipulations (shift segment register 4 bits and add offset), it never was meant to be used that way, and this is invalid in PM16.

DPMI was intended to manipulate PM local selectors (LDT) by defining base, size and access-rights. Kernel selectors (GDT) typically cannot be accessed or created with DPMI.

The typical 16-bit C compiler for PM will handle selector creation in the runtime library (using DPMI if available), or by the loader mapping sections to different selectors at load time. Depending on memory model, pointers will either be near (relative to DS or CS) or far.

[Octocontrabass]

And while I'm at it - what does DPMI PM16 give you anyway?

It gives you access to more than 640kiB of memory in a way that works under multitasking OSes. Same as DPMI in 32-bit mode, except it works on a 286.

And I would still need NE executables - it's just that instead of using the OS/2 or Windows DLL to obtain the Huge Shift value, they would instead be making INT 31H calls to get those values?

Programs that use DPMI are ordinary MZ or COM executables, not NE. If they need extra relocations to run in protected mode, they'll handle that part by themselves. They'll also load their own DPMI host (or prompt the user to load a DPMI host) if one isn't available.

And I just need to take some care in places, like presumably not manipulating B8000H directly but stick to INT 21H.

DPMI allows directly accessing 0xB8000 with the help of INT 0x31 AX=0x0002.

On x86, there is the bsf instruction.

That works, but it requires a 386 at minimum.

[kerravon]

Programs that use DPMI are ordinary MZ or COM executables, not NE. If they need extra relocations to run in protected mode, they'll handle that part by themselves.

Is there support from linkers to not split an object file over a 64k boundary? Because I couldn't see a way around that, which is why I was looking at NE in the first place.

And the linker would need to have support for the AHINCR to be set by the MZ itself after it is loaded.

So this would be extended MZ.

Any reason to not just make this NE?

The way HX does Win32 under MSDOS seems correct to me. Transparent to the user. I've requested Japeth to do the same for OS/2 1.x which he also supports.

[kerravon]

Is there support from linkers to not split an object file over a 64k boundary? Because I couldn't see a way around that, which is why I was looking at NE in the first place.

Actually variable length selectors should get around that problem. But I don't want to do that. Hence NE with linker support.

[Octocontrabass]

Is there support from linkers to not split an object file over a 64k boundary? Because I couldn't see a way around that, which is why I was looking at NE in the first place.

You can do whatever you want inside your MZ/COM binary, including packaging an entire NE loader and NE binary if that makes things easier for you. As far as I know, DPMI clients have to do something like that anyway.

Any reason to not just make this NE?

There's no point in copying a MS-DOS API if your programs aren't going to work in MS-DOS.

[kerravon]

Any reason to not just make this NE?

There's no point in copying a MS-DOS API if your programs aren't going to work in MS-DOS.

Ok - I wasn't expecting that!

PDOS was designed to be a 32-bit version of MSDOS. You can't run any of those 32-bit executables directly on an 8086 at all.

It was always unclear to me what "32-bit MSDOS" actually was.

But regardless of the answer to that, I produced an 8086 version regardless, more as a demo than anything else.

And that "demo" demonstrated that the OS itself (without trickery) occupied a large amount of the 640k that it was impractical to run other apps like SubC.

I need something like 2 MiB of memory.

And I can get that on an 80286.

So my "demo" can actually be "more than a demo" if I port the 32-bit version - incompatible with anything - although it does now run Win32 software - to the 80286. So a 16-bit version with room to actually do things.

And in recent days I have spent a lot of effort working on 16-bit OS/2.

And although PDOS-generic has its own API, I am expecting to be able to run limited OS/2 programs. Same limitation as with Win32 - mainly programs that use msvcrt.dll - and only a subset of that.

There doesn't appear to be an OS/2 1.x equivalent of msvcrt.dll - and indeed, even msvcrt.dll is undocumented and meant to be for internal use.

So there's no barrier to simply creating my own CRT as a DLL. And then it would be those programs that I run. (or else proper pdos-generic programs).

And without the C library, I will have more space in the lower 640k too.

So I may end up being able to even support the 8086 after all - enough to at least run SubC.

Either way - my OS/2 1.x or pdos-generic apps only execute 8086 instructions.

So they will nominally work on an 8086.

Neither the PDOS-generic nor the OS/2 1.x API (ie a DLL to replace msvcrt.dll) will be the MSDOS API anyway.

I could support INT 21H as well though. I do that on PDOS/386 already (a 32-bit version).

That way MSDOS assembler code would still be able to work. And no reason why I can't support MZ executables either - even though they (mine, anyway) only run on the 8086, not the 80286. Although even then - if the memory model is such that the MZ executable neither crosses a 64k boundary or uses huge pointers that require fixups - I can run them on PDOS/286 too.

So yeah - I don't see a reason to attempt to kludge MZ as the 80286 dos extenders do. I'll just support a clean subset of MZ plus a subset of OS/2 1.x I think. Plus pdos-generic.

Actually, the INT 21H calls may be difficult to implement. That's why I am nominally abandoning PDOS/386 and switching to PDOS-generic for the long term.

So MZ and INT 21H become irrelevant and it becomes OS/2 1.x and PDOS-generic as NE - even on the 8086. My OS/2 1.x executables are currently only dependent on a small number of functions in doscalls.dll.

[kerravon]

Actually, some more thoughts on that philosophical question you raised.

Ultimately I know that MSDOS was able to be run on low memory systems. MSDOS 1.0 anyway. And it's MSDOS 2.0 that has the API that I code to. But I assume MSDOS 2.0 and apps can run on low-memory systems too. ie much less than 640k.

Now I have no expectation (or desire) for PDOS/86 to run on extremely constrained systems. I want to cede that market to whoever is willing to write in assembler. ie Tim Paterson.

However, I want to "stay in touch" with those small systems.

So I have written Pos* wrappers on each of the INT 21H functions that I care about, so that if you do have "plenty" of memory, then your 16-bit C programs will work too. It will presumably seem inefficient to an assembler programmer to be making all these function calls (which push and pop registers they don't even change also), but - it works - even if it is slower and more memory-hungry. And so long as you are "rich" enough to afford an 80286 or 80386, it doesn't really matter that you're not very competitive on an 8086.

And if performance/memory on an 8086 is important, then by sticking with the MSDOS API (or wrappers over it), you know that your application can be hand-crafted in assembler if required. ie if that is cheaper than just asking your customers to at least buy an 80286.

So I'm sort of "starting big" and then going down.

Note that this is more thoughts/conjecture/musings than a definite opinion.

The main reason I started with MSDOS was because I already had it, it was simple/understandable, and the executables were so small that I thought it wouldn't take too much effort to reproduce something similar (but 32-bit was the goal). That was back in 1994. I'm not claiming I knew what I was doing back then. I'm not particularly claiming anything even in 2023.

[Octocontrabass]

PDOS was designed to be a 32-bit version of MSDOS. You can't run any of those 32-bit executables directly on an 8086 at all.

There are plenty of MS-DOS programs that require at least a 386 for 32-bit code. That's a valid target if you're aiming for binary compatibility.

If you're actually aiming for source compatibility instead of binary compatibility, then it doesn't matter which binary format you choose for the PDOS binary as long as you can still compile the same source code (except parts written in assembly) into a MS-DOS binary.

Either way your programs would work in MS-DOS, so it makes sense to copy the MS-DOS API.

[kerravon]

PDOS was designed to be a 32-bit version of MSDOS. You can't run any of those 32-bit executables directly on an 8086 at all.

There are plenty of MS-DOS programs that require at least a 386 for 32-bit code. That's a valid target if you're aiming for binary compatibility.

If you're actually aiming for source compatibility instead of binary compatibility, then it doesn't matter which binary format you choose for the PDOS binary as long as you can still compile the same source code (except parts written in assembly) into a MS-DOS binary.

Either way your programs would work in MS-DOS, so it makes sense to copy the MS-DOS API.

Ok, that's a neat way of putting it - thanks for that.

So yeah - I would start by saying "MSDOS-inspired" - I don't want to commit to being even source code compatible.

And then I want to be C-centric - so I want nice C functions to call instead of being required to do an INT 21H. In actual fact, the Pos* functions in PDOS was OS/2-inspired in that way. I liked the Dos* functions in OS/2, but I wanted the focus on MSDOS, and also I didn't like the ULONG etc - I wanted standard C types. So in that way it was Posix-inspired I guess.

And only after that is agreed on do I then consider assembler and the 8086 vs 80386 and low memory and existing executable formats or whatever.

Basically I don't want existing (or historical) practice to hold back "doing things right" (and what is "right" I am happy to debate).

In fact, PDOS-generic is what I eventually came up with as "right", but then I suddenly noticed I could twist it for expedience to support win64 executables that were dependent on msvcrt.dll. And today I am hoping that I can support win32 executables dependent on msvcrt.dll the same way under OS/2 2.0+.

But without the twisting, it would have just been PDOS-generic.

But PDOS-generic is still source compatible with the original MSDOS-inspired API.

Another thing is "division of labor/responsibility" or something like that. If the OS itself is deficient, I expect it to be cleaned up. If Microsoft have abandoned MSDOS - so be it - time to update a rival, like Freedos. If the Freedos people aren't interested either - so be it - time to write PDOS/86. In this specific case - ironically - it sounds like MSDOS actually had already been updated to support NE executables (with 4.0), but the changes were lost for managerial reasons. But someone at some point thought it was technically appropriate to support NE for 8086 MSDOS. They may well have been right, and I'm here to support them in that case. Or compete with them - whatever. I don't have a fixed goal.

And the support doesn't necessarily have to be in the (existing) OS itself. HX has a 16-bit version as well as a 32-bit version, and I'm not aware of a technical barrier to creating an hxldr16 that works the same way as hxldr32 so that it's equally as convenient. That gives OS/2 1.x rather than MSDOS, but if you stick to Family API functions (which I do in PDPCLIB), then that was something intended to run under MSDOS anyway. I just think that the "Family API" code should be moved into MSDOS (or equivalent) itself, rather than being in every executable. Again - either MSDOS itself, a clone, or an add-on equivalent to hxldr32. I raised two issues with HX that were preventing this from being done for what (I believe) I want.

And I don't necessarily want to be bound by Family API either. It's more Family API-inspired.

I don't want to gratuitously break compatibility, so if there's a way of coexisting, that would be cool. Which reminds me - I made a lot of effort to get my OS/2 programs (pdpclib) only dependent on doscalls.dll. Doing raw mode took a lot of effort (I didn't want thunks to 16-bit) although in hindsight it is simple enough (lots of things are). But I think I exceeded the capabilities of existing Family API (the functions exist, but may not fully work), so I may rework that to use different functions for the 16-bit version to support the existing Family API, or I may provide my own Family API (I assume there is no reason for the Family API to be set in stone for eternity), or I may abandon the Family API completely and put that same functionality directly into the OS. But then I would end up with OS/2 1.x rather than MSDOS. But that's not necessarily a bad thing. And that's all hidden in a C library anyway, so it's source-compatible with everything. It's only when I need to do a mkdir() or whatever that I need to decide which way to jump. Currently I have my own PosMakeDir() and haven't committed to jump anywhere. MSDOS is defined in terms of INT 21H so there is no pre-existing jump anyway. Sort of. I may adopt the Microsoft C extensions in dos.h and/or io.h. I've never used them before because they are beyond standard C90. If there was some formal MSDOS standard for C, then maybe I would be willing to use that. Or maybe I would/have created it. I'm not particularly expecting (or concerned) that no-one follows my "MSDOS standard" from the "non-POSIX committee". But I sort of expect such a thing to exist in principle. Or should have existed in the 1980s. Being late to the party doesn't bother me either. Lack of MSDOS users asking for a committee/consortium doesn't bother me either.

[kerravon]

In fact, PDOS-generic is what I eventually came up with as "right", but then I suddenly noticed I could twist it for expedience to support win64 executables that were dependent on msvcrt.dll. And today I am hoping that I can support win32 executables dependent on msvcrt.dll the same way under OS/2 2.0+.

This worked btw - the result can be found by searching for win32os2 at http://pdos.org