expand-down segments question

[mikeleany]

In the Intel's IA-32 manual (volume 3), it says that the base address for a segment descriptor "defines the location of byte 0 of the segment"(ch3 pg11). It doesn't give any exception for expand-down segments. That seems to imply that the base address for ALL 32-bit expand-down segments would HAVE TO be 0, otherwise the logical address 0xffffffff couldn't be translated to a 32-bit linear address. I wondered if maybe it wrapped around, but that doesn't seem to make sense either, because that could cause other problems. Anyone understand this? If so, could you explain?

[Anton]

Read(carefuly) about segment limit. ... Well it states there, that if you have a expand-down segment, then valid adress go from limit to 0xFFFF, or 0xFFFFFFFF depending on the bound flag.
Read for your self:

For expand-down
segments, the segment limit has the reverse function; the offset can range from
the segment limit to FFFFFFFFH or FFFFH, depending on the setting of the B
flag. Offsets less than the segment limit generate general-protection exceptions.
Decreasing the value in the segment limit field for an expand-down
segment allocates new memory at the bottom of the segment's address space,
rather than at the top. Intel Architecture stacks always grow downwards,
making this mechanism is convenient for expandable stacks.

Anton.

[mikeleany]

Okay, that answered something I already knew. My question was about the base address, not the limit. So let me explain the whole thing out.
Lets assume that we have an expand-down data segment with the following values:

Base Address: 0x1 (or any other non-zero value you want)
Limit: 0xfffff
Granularity Flag: set (meaning we multiply Limit by 4KB)
Bound Flag: set (meaning 32-bit segment)
(all the settings for an expand-down data segment)

This means that the valid logical addresses range from 0xfffff000 to 0xffffffff. And the logical address is an offset from linear address 0x1 (because the base address "defines the location of byte 0 of the segment within the 4-GByte linear address space"). Therefore (adding the logical address, or offset, to the base address) this segment ranges from linear address 0xfffff001 to 0x100000000, which is NOT a valid 32-bit linear address (because it's 33 bits). So this segment and all other 32-bit expand-down data segments with a non-zero base address appear to be invalid (unless of course the linear address wrapped back around to zero, but I would think that would create other problems). So if anyone can explain to me why I'm wrong (which I hope I am), I would greatly appreciate it.

P.S. As you should clearly see now, the limit has absolutely nothing to do with the question (and I wouldn't think it would have anything to do with the answer).

[tom1000000]

Hi,

I think you have got yourself confused.

Say for example we have a standard 32 bit data segment with a base of 80000000h, and limit of 0FFFFFFFFh. [DS:0] points to virtual address 080000000h.

What happens if you try and access [DS:080000000h]? As 080000000h + 080000000h is greater than virtual address 0FFFFFFFFh, you are going to get an exception.

Your question is basically the same as above. If you add an offset to the base address that exceeds the limit, you will get an exception.

Tom

[mikeleany]

So the processor will accept a descriptor which maps to invalid addresses, as long as you don't use those addresses? So if you have an expand-down segment with a non-zero base address, you just can't access the high addresses? So in that case you'd just have to be careful to watch out for the upper AND lower bounds? I guess that makes sense. Thanks.

[tom1000000]

You are getting the offset into a segment confused with virtual addresses.

The base and limit values in the GDT apply to the VIRTUAL ADDRESSES. The virtual address is (base + offset). So yes the offset can be too large for a specific descriptor, but thats obvious because you can use a 32 bit pointer in a 64K segment.

A descriptor NEVER maps to invalid VIRTUAL ADDRESSES.

[mikeleany]

Well, I'm pretty sure I know what you mean in this message, and I'm pretty sure it's the same as what I meant in my last message. Although, it sounds like you're a bit confused, because you and I both gave examples of segments which are mapped to invalid linear (or virtual) addresses. Any linear address above 0xffffffff is an invalid address (because it doens't fit in 32 bits). So if a segment limit allows offset large enough that base_address + offset > 0xffffffff, than the segment maps to invalid logical addresses. Based on what you said in the other message though, the descriptor for such a segment is valid, but the segmented addresses that map to invalid linear addresses are not valid. (However, I searched the IA manual for an official statement of what happens in that situation and found nothing. So I guess the behavior in that case is simply undefined. Unless there's an official statement on the subject, what happens on one processor may not be what happens on another.

[Bird]

Hi xSadar,
> So if a segment limit allows offset large enough that base_address + offset > > 0xffffffff, than the segment maps to invalid logical addresses.
It wraps around to base_address + offset - 0xffffffff.
Only the OFFSET is checked (against LIMIT, and the D bit).

[mikeleany]

Great, so now I have two different people telling me two different things which contradicts each other. tom1000000 says that if a segment allows an offset such that base_address + offset > 0xffffffff, and you use that address, than "you are going to get an exception." Bird says that "It wraps around" and "only the offset is checked." Unless you can prove to me that this is ALWAYS the case, than for all I know, you could both be wrong, or you could both be right, depending on the processor model.
The IA manual does say that the offset is checked, but, as far as I've seen, it says nothing about if base_address + offset is checked for overflow. So for all I know, it could be checked in some processors and not in others. It would be wrong to assume that it isn't checked because it doesn't say it's checked, and it would also be wrong to assume that it is checked, unless it says so. And it's also wrong to assume that because one processor does one thing that they all do. So does anybody know where there's and official statement about whether base_address + offset is checked for overflow?
As a side note, I would hope that Intel would be smart enough to not have it wrap around again, because that would make it so that really stupid stuff may have to be done to ensure compatibility, just like what happened with the A20 line.

[Bird]

I admit the IA-32 manuals miss a lot of things - they assume you already knew them. On a few occasions they make big mistakes explaining things.

Whenever found something vague in book III (which I often did), I suggest you refer to book II.

3.2 Instruction Set Reference - MOV - Protected Mode Exceptions
(It lists all the possible exceptions while executing MOV.)

[Anton]

I think you should go back to school and learn reading: the pointers are interpreted(by the CPU) as 32-bit numbers, so adding 0x1 + 0xffffffff whill mean
0x1 - 1, which is a valid adress. By setting the limit to 0xfffff yes you  will have the offset valid from 0xfffff000 to 0xffffffff, which means
from -1 to -0xfff, so the size of the stack will be 0xfff.

Anton.

[mikeleany]

Yes, addition wraps around. And I mentioned the possibility that segments wrapped around clear back in my first post. (Maybe you should go back to school and learn reading.) However, the wrapped-around result is not the TRUE result of adding two unsigned values. The TRUE result of adding the unsigned values 0x1 + 0xffffffff is 0x100000000, which is NOT a valid 32-bit address. Since addresses ARE unsigned values, a well-designed CPU would recognize that there was an overflow and generate and exception (in this case), because (1) it doesn't make any sense to have a segment that goes from 0xffff0001 to 0xffffffff then loops back around to 0x00000000, (2) doing so makes it a lot easier to create hard to find bugs, and (3) allowing that could cause compatibility problems just as it did with the 20-bit addresses looping back around. And there are probably other reasons as well, those are just the ones I could think of off the top of my head.

So tell me, what does any of this have to do with my reading ability? Hmm?

[mikeleany]

Well, I would hope that it listed all exceptions and how they're generated. As well, I would hope that it would list all ways of generating each exception in chapter 5 of volume 3. Unfortunately neither one says that they list ALL, but I guess it's probably what they meant. Anyway, it's probably enough to convince me not to ever use expand-down segments because of their apparantly psychotic design. Thanks for your help.

[Anton]

OK. Who told you that the pointers are unsigned. Asm is not a type language, which means you can do what ever you want with data, which means you can interpret any data as you like. BTW there is no special instruction to add signed integers(just in case you did't know). So the CPU adds the integers, and keeps only the 32bit result. Conserning 1) tell me, how will the following freqently used code be coded.

mov [ebp-1],5;
???
hint:
mov [ebp+0xff...ff],5;
And i can a shure you, that the code above works, and it does not generate any errors.
the cpu does not care if an overflow will hapen with the pointers(since there is not such data type(Remember asm is not Java)). AU is the one which adds the numbers, and it does't care what will be the use of these numbers, so when it adds 0x1 + 0xffffffff it gets a 0x0.
3)Yes, ofcorse there will be compatibility problems. ASM is not ment to be a portable language, nore is the byte-code generated. But if you programed on C, you would have those problems.

Anton

[mikeleany]

You seemed to have missed the point that we're talking about segments here, not pointers in general (although I don't know how you missed that). I stated that it should generate and exception "(in this case)". Also, my reasons had to do with segments, not pointers in general. In pointers in general, you are dealing with both addition and subtraction, however, the segment's base address and offset are not subtracted; they're added. I referred to pointers as being unsigned simply because that is the most logical way to interpret them, simply because address 0xffffffff is higher in memory than 0x0, not lower. Your correct that the AU doesn't distinguish between unsigned and signed addition, but that doesn't mean that the it doesn't provide means of detecting overflow in either case, and I'm saying that the CPU should use those means in this case (creating a linear address from a logical address). And YES, assembly is a portable language. I can port code from an 8088 to a Pentium 4 without any problems, because that's the way Intel wants it. However, to make that possible, Intel and other companies had to do all kinds of weird things that make assembly much more complicated today, because they didn't plan for the future. I would have hoped that Intel would learn their lesson the first time. And if you've taken any kind of design class (hardware or software) you should have learned that you should plan for future changes. One of the most obvious future changes that a mainstream CPU is going to have is a switch to a larger address size.