Separate Stack Segment in Protected mode?

[rdos]

How interesting that I found this thread just now. I've revived my old 80386DX just last week and wanted to experiment with segments in 32-bit extended DOS applications.

I did find the options for Watcom C/C++ to create a segmented executable, but as far as I can see the compiler still generates flat code. Maybe I need to use multiple OBJ-modules and/or exceed a certain minimum size of my data and code segments, I don't know at this moment.
I've also did some preliminary assembly tests but couldn't get anything to work. For example a simple "Hello World" program defined with full segment definitions and no DOS groups *STILL* generates a flat model with SS=DS=ES. The reality is that the link between generated assembly code -> object code -> executable binary is not clear enough at this moment.

32-bit DOS extender applications are flat. OpenWatcom does support non-flat 32-bit code, but there is no executable format that is non-flat except for the 32-bit RDOS kernel device driver format and some OS/2 executables or drivers.

[16bitPM]

32-bit DOS extender applications are flat. OpenWatcom does support non-flat 32-bit code, but there is no executable format that is non-flat except for the 32-bit RDOS kernel device driver format and some OS/2 executables or drivers.

Actually, the documentation of Causeway states that multiple segments are supported. But I couldn't get that to work so far.
I guess that answers a couple of my questions. It's confusing though: it's possible to select an extended DOS application with the "small" memory model, as opposed to "flat" in Watcom AppExpert.

[16bitPM]

But err... at least as far as assembly is concerned, I guess it wouldn't be too hard to modify the loader of, say, PMODE/W and set up a non-flat system, maybe extend it to set up multiple segments as defined in the LE/LX header.

EDIT: So I've done some reading in "Linkers & Loaders" by John. R. Levine, and did some testing armed with wdump, tdump and uhex.

My conclusion is that the final part of your statement should be: "there is no known linker that emits multi-segment output", given that more than one DOS exe-format actually support 32-bit segments. At least LE/LX ([1][2]) and PE ([3]). Outside the MS-World, ELF also has sections and segments and in fact, any readelf -a on any executable will tell you they really are there, abundantly.

On top of that, my suspicion is confirmed that the problem is also the loader (read under "Predefined Sections" and "Executable code section, .text", ref. [3]).
Or more simply put: linkers and/or loaders do the fixup in software instead of postponing it until execution when the x86 segment hardware could take over.

However, I'm sure that both of these obstacles can be dealt with.

We only need a linker that does NOT fuse segments but emit segment definitions for LE/LX or PE, and have a new loader that does NOT fuse segments but creates new selectors as needed.

Returning to the topic starter: usually, at least some segments ARE defined in the EXE format, even in a flat memory model (for the purpose of setting page attributes). This seems specifically true for stack segments (albeit optional for ELF).
If your execution model supports multiple segments, then it should be possible to create a separate stack segment (well, that rules out long mode of course).

References:

• • [1] LE File Format
  • [2] LE file format, also includes segment (object) flags
  • [3] PE File Format

[linguofreak]

Is GCC API does not allow SS to be on a different segment? I think System V API does allow that.

In general, C compilers will not support separate DS and SS, because of the & (address of) operator, which breaks if SS is not a subset of DS, and is more complicated to implement if they do not have the same base. Compilers whose primary audience was DOS/Windows/OS2 developers on 16 and 32 bit x86 often supported separate DS and SS via non-standard extensions, but compilers from the Unix world and those that aren't primarily meant to compile for pre-64-bit x86 generally do not support separate DS and SS (because they generally do not support x86 segmentation at all). About the only non-flatness you will ever see to the address space in the Unix world is that if the hardware supports separate page tables for code and data (which x86 *doesn't*, it supports separate code and data spaces via segmentation, but not via paging) Unix systems will often take advantage of that, and likewise if the hardware supports separate user and kernel page tables.

A potential source of confusion is that on Unix, executable file formats will define different "segments", but this is *not* to be interpreted in terms of x86 segmentation or a non-flat address space, it's just a way of describing the layout of the single flat address space.

[rdos]

I think LE format does support multiple segments in 32-bit mode, but PE and ELF are pure flat formats.

As for changing 32-bit DOS-extenders to support segmentation, I think that is a non-trivial issue.

[16bitPM]

I think LE format does support multiple segments in 32-bit mode, but PE and ELF are pure flat formats.

As for changing 32-bit DOS-extenders to support segmentation, I think that is a non-trivial issue.

Not true, as I explained earlier. PE and ELF support sections, which may or may not be fused together in one big segment. But that's the decision of the loader.
It's also possible that the compiler or assembler output one segment, but that's usually not the case because the sections are still useful to determine page-level protection attributes (for example NX for data/stack pages).

[rdos]

I think LE format does support multiple segments in 32-bit mode, but PE and ELF are pure flat formats.

As for changing 32-bit DOS-extenders to support segmentation, I think that is a non-trivial issue.

Not true, as I explained earlier. PE and ELF support sections, which may or may not be fused together in one big segment. But that's the decision of the loader.
It's also possible that the compiler or assembler output one segment, but that's usually not the case because the sections are still useful to determine page-level protection attributes (for example NX for data/stack pages).

Sure, it's possible to relocate sections in PE or ELF to be zero-based & segment oriented, but you also need far calls and far data references and there is no structure in PE or ELF to tell the loader that the segment part should be replaced with a selector allocated to a specific section. Also, if you use a standard compiler, the compiler will not be able to generate far calls or far data references, but instead will output a relocatable 32-bit offset.

When I do segmented device-drivers in RDOS, I only have two fixed selectors for each driver (one for code and one for data), and so the driver itself uses the compact segmented memory model. The driver does not assume that SS == DS, and it supports 48-bit pointers and far calls. In the OpenWatcom environment, I have fields to assign the code & data selector to a fixed value, and this is also output to the header of the driver file. When a driver is initialized, the code & data selectors will be initialized with the limits for the code & data segments and then the entry point will be called for initialization. The linker will take care of fixing up any references in the code to the code and data selector, so I can use "seg data" and "seg code". It also works to use C code in the driver since I build a libc that works with this memory model.

[TheHound]

I've looked into this sort of topic myself several months back. OpenWatcomC, TASM, and NASM can all produce 32-bit OMF object files. I thought the best approach was to make my own linker that produces my own executable format suited for segmentation from 32-bit OMF files. I read the specification found on openwatcom's site, it has a couple of mistakes, however, I couldn't figure out what to do exactly with segment groups and segment classes, which are left-overs from the 16-bit OMF. I tried to look at openwatcom's output files because it does generate groups and classes, but I can't exactly remember what I concluded. Eventually, I decided to just use them however I wanted, ditching groups entirely or using only specific classes such as 'rodata', 'data', text, etc.. Anyway, I made several hacky and initial versions of this linker, some that don't support groups, others that do, and my own executable format, but none of these were proper, and then I got busy and didn't touch on them again.
My findings were that the specification itself was not that good. Each assembler produces different object code for fixing up the addresses even though the assembly instructions are the same. And the specification was filled with stuff specific to the Microsoft C compiler such as the generated debugging information. Other tools such as TASM and NASM produced their own debugging information which is not in the specification. NASM can produce borland's (TASM) debugging information. I think the specification for it can be found here.

[16bitPM]

Sure, it's possible to relocate sections in PE or ELF to be zero-based & segment oriented, but you also need far calls and far data references and there is no structure in PE or ELF to tell the loader that the segment part should be replaced with a selector allocated to a specific section.

Maybe I misunderstood something then?

If you read here under "Intel 386 Processors":

IMAGE_REL_I386_SEG12 : Not supported.

This site tells me that this relocation field is a "Direct 16-bit reference to the segment-selector bits of a 32-bit virtual address".

[nexos]

@rdos, can OpenWatcom output LE or LX executables? That would very helpful for me if it could!

[rdos]

@rdos, can OpenWatcom output LE or LX executables? That would very helpful for me if it could!

I think it can output both.

[16bitPM]

Your examples are wrong.

Sorry, slip of the keyboard.

[16bitPM]

For those interested (I mean, if you're a fan of segmentation and need rethorical ammo to convince your peers): I found this paper : Checking Array Bound Violation Using Segmentation Hardware

[iansjack]

Not of relevance if you are using 64-bit processors in long mode.

[rdos]

Perhaps, but also a good argument for why you don't want to use long mode.