With or Without segments

[Pype.Clicker]

The segmentation unit of the IA-32 is almost always under-used in operating systems ... This is a pity while it could make YOUR OS more reliable that main-stream OSes

1. Using a separate segment for your stack can prevent stack overflows (even malicious ones). A common trick for stack overflow is to leave a non-allocated page at the top of your stack but

• this means that a char[4096] could go over that protection page and still result in a stack overflow
• this is not appliable to kernel stacks (and i suggest your kernel stack don't use on-demand memory allocation, but rather real allocation) because the page fault exception could not be handled if the stack is invalid (an option would be to have a task gate for paging exception, but you probably don't want it for performances purpose).

On the other side, using a expand-down segment that starts at the base address of the data segment and which limits matches the lowest allocated address for the stack offers you a full-proof protection against stack overflows (though it leaves you unshieded agaist stack underflows, but those are really less frequent and a blank page will usually stop them )

[Pype.Clicker]

2. Another possible use of segmentation is to strictly separate code and data ... or more precisely executable and writable bytes.
As a system designer, you should be aware that most Unix security problems comes from so-called buffers overflow : an unsafe function

Code: Select all

void fct() {
    char buffer[256];
    ...
    strcpy(buffer, <something that could possibly be larger than 256 bytes>)
}

could see the strcpy function go beyond the space allocated to buffer and starts erasing the stack content. As this stack contains (among others) the return address where we must branch to when the function is over, it is possible (using some reverse-engineering on the target program) to send the execution to bytes that were passed from an untrusted sources (a message coming from Internet, a user input, a file, whatever).

Unix & clones protects the text of the program to be overwritten, but they don't prevent the stack to be executed...
This can be done very easily by restricting the size of the user code segment to the size of the .text section at load time...
Note that it is absolutely not necessarily that code and data share the same addresses (unless you want to read the code instead of executing it), so you can safely locate code segment anywhere in memory, and using some quick remapping techniques (moving pages entries and segments bases accordingly) and even make it grow to add plugins, etc.

because you have no bits that is both writable and executable, the risk of buffer-overflow, worms, etc. is suppressed... Your OS is secure and only the kernel bugs can cause ennemy code to be executed.

[Tim]

I maintain that if you write decent code to begin with (strncpy instead of strcpy in the above example), this problem won't occur.

Separate code and data can work, since you should never need to run data, and you can get around accessing code as data. Various compilers (VC++ and gcc at least) like putting read-only data in the code section; this can be fixed, though.

The stack and data segments need to be the same for current C/C++ compilers to work without major modification. That is, you couldn't stop people writing beyond the end of the stack (since it would flow seamlessly into the data) but you could stop people running code from the stack.

I think that both the IA-64 and X86-64 architectures add a much-needed 'No Execute' bit to page protections.

[pskyboy]

One disadvantge of your approach is that you can't have your code dynamically changeable which is one of teh advantages of the von neuman architecture. The fact that code and data are stored together so you can have code change as it executes.

Another point to consider is the x86-64 platfrom doesn't support segmentation in long mode (64-bit mode) and im guessing that IA-64 doesn't either but i may be wrong. My point here is its probably not a good idea to use segmentation if you want to port your application to new processors or even other existing processors as only intel based chips support segmentation (i belive).

Peter

[DynatOS]

Segments are not going to protect you from faulty kernel logic.

[Pype.Clicker]

One disadvantge of your approach is that you can't have your code dynamically changeable which is one of teh advantages of the von neuman architecture. The fact that code and data are stored together so you can have code change as it executes.

i actually can't see the benefit of having modifiable code, except in some very hackish ASM programs ... but i can see many security problems of having the opportunity to execute things that were not in the program code (most often without the program's agreement, and in order to take over the machine's control)

Another point to consider is the x86-64 platfrom doesn't support segmentation in long mode (64-bit mode) and im guessing that IA-64 doesn't either but i may be wrong. My point here is its probably not a good idea to use segmentation if you want to port your application to new processors or even other existing processors as only intel based chips support segmentation (i belive).

you're right with the point that x86-64 won't have the segmentation unit once in long mode ... however, i never mentionned about changing the *application*: everything i proposed is to generate segments that will enforce some protections for the application, but which are set up and under the sole control of the kernel.
this is very different from having a separate segment per buffer (which is under the control of the app and which obviously won't be porttable)

on another architecture, the kernel would enforce the same rules, but using different techniques ...
Peter