MUlti-Segment Mode [GDT]

[origin of]

Hi, reading the intel manual 5, chapter 3.2.3, i found the Multi-Segment Mode....how can i set the GDT, for th MultiSegment Mode ??

[JAAman]

you dont have to 'set' the GDT for it, it is just a way of using the GDT -- and you can use it in any way you want,

however, most people dont use it (instead, they use the 'flat' mode discribed in volume 3a, section 3.2.1) because it can quickly become very complicated, most C compilers wont work well with it (assuming the OS doesnt use it, and will assume that DS.base=SS.base), and because making extensive use of segmentation will make it very difficult to use LMode (that is 64bit mode, called ia32e mode in the intel manuals) since segmentation is essentially dissabled in 64bit mode


i dont personally have experience using complicated segmentation, so i cannot say for sure the best way to do it, but basically, you allocate each segment only the memory that will be needed for it, so that writes to one segment, cannot be read by annother (though they can overlap, giving segments access to each others space -- where it really gets confusing for some people)

there is little practical use for this anymore, as paging provides simpler inter-process protection, and newer CPUs support the XD/EP/NX bit

if you do want to use multi-segment mode, mabey someone else has more experience with it and can provide more details

[bontanu]

there is little practical use for this anymore, as paging provides simpler inter-process protection, and newer CPUs support the XD/EP/NX bit

Paging is an old way of doing things in the mainframes of the 50s.
It is not only old but also bad design.

I does not help with interprocess comunication since a common segment woul haelp asmuch or better that a simgle common page and IPC is anyway done in many other ways than a shared memory area.

PAGING was so widely used because:
- compilers did not support segmentation (not the one in 32 bits)
- it helps the memory allocator by avoiding memory moves
- is alows to use page faults for virtual memory and THIS was the BIG issue back them when PCs had limited memory ...
today it is kind of useless with >1G of RAM
-it was well known and loved by the mainframe programmers that implemented the WinNT like OSes

IN FACT the Segementation was an advancement in computer technology and it alowed for unpreakable protection but it was rarely used and because of this dropped by capitalistic rules on the new design of 64bits long mode...

This way the human race "evolves"...

Anyway indeed one should be aware that using segmentation is not possible in long mode and this is a huge problem if you base your design on the 32bit segmmentation.

Otherwise it is quite simple to setup:
-just make a segment for code and a segment for data for each process you create
-setup the cs to code and ds,es,fs,gs,ss to data segment ... and start the process

Potential problems:
---------------------------
-be aware of the 8192 llimitation in the GDT this mean that you can only have approx 4000 or so of process running ... do not ask Intel for more
-be aware that if you enlarge the memory of the process you will probably have to memcpy it to another location if there is no free space after it.
-you need a continouse memory place for each process and you can have granularity problems. As a consequence you might have to move things in memory (physically).
-no compiler will suport this (almost)

Advantages:
-----------------------
-no malware code can beak away no matter what it does if you design it right --> the perfect security

-speed: changing a few selectors when a task or IRQ switch occurs is much faster than changing a whole set of pagetables


As you can see it will be a little more complicated and sometimes slower than paging when moving memory arround.

But has perfect security and greater task switch speed and overall speed if you avoid moving memory...

[matthias]

Interessting.. Interesting, wouldn't this be perfect for dedicated machines like servers or something (with pre-set memory limits for each process? Or is this bullshit? also I'd imagine that swapping is not quite easy to realize without having paging?). Only thing is, it is hard to realize Especially, the following bit

-no compiler will suport this Very Happy (almost)

I'm very interested in making such an OS, but do you happen to know which compiler supports this, e.g gcc ??

be aware of the 8192 llimitation in the GDT this mean that you can only have approx 4000 or so of process running ... do not ask Intel for more

And what about LDT?

[JAAman]

wouldn't this be perfect for dedicated machines like servers or something (with pre-set memory limits for each process?

exactly -- however, in a general use OS, it is too inflexable (in a dedicated machine, such as embeded platforms or gaming consoles, where the developer has absolute control over the system, this would be a much better solution)

I'd imagine that swapping is not quite easy to realize without having paging?

swapping is achieved for segmentation quite easily, however it is more complicated, and requires much larger portions to be swapped out more frequently

I'm very interested in making such an OS, but do you happen to know which compiler supports this, e.g gcc ??

GCC is the one that definately wont support it (i believe the intel compiler will (free only for linux), the MS compiler might -- have to do more checking, and watcom definately will -- this is often considered the primary reason to use it)

however, segmentation itself can be used with any compiler, as long as the following rule is always true -- DS.base = SS.base (some may also require ES.base and/or CS.base to be equal as well...)

And what about LDT?

the LDT cannot store processes, and it is local to each process (each process should have its own LDT), however, this limit is artificial, and easily extended indefinately (even OSs that use segmentation, do not use the GDT in this way) by editing the GDT at runtime -- the GDT will only contain a few entries, the GDT is edited on each task-switch, to add the appropriate entries (which are stored in the process table anyway)

[rexlunae]

Interessting.. Interesting, wouldn't this be perfect for dedicated machines like servers or something (with pre-set memory limits for each process? Or is this bullshit?

I would say the later. There is some overhead to paging, but in most cases it is worth it for the benefits, such as easier memory allocation, more granular swapping, and stronger protection mechanisms.

also I'd imagine that swapping is not quite easy to realize without having paging?). Only thing is, it is hard to realize Especially, the following bit

Without paging, you would probably need to swap out an entire task at a time instead of individual pages. This would still be feasible, but it would certainly be less useful.

I'm very interested in making such an OS, but do you happen to know which compiler supports this, e.g gcc ??

gcc doesn't. You would probably want one of the DOS compilers.

And what about LDT?

Each LDT can also contain 8192 segment descriptors.

[JAAman]

Each LDT can also contain 8192 segment descriptors.

but the LDT cannot contain processes -- all 8192 segment descriptors must be used for the same process (well... technically they dont have to be, but basically your still left tracking outside the CPU structures, which makes it essentially the same as altering the GDT -- which is easier)

[rexlunae]

Each LDT can also contain 8192 segment descriptors.

but the LDT cannot contain processes -- all 8192 segment descriptors must be used for the same process (well... technically they dont have to be, but basically your still left tracking outside the CPU structures, which makes it essentially the same as altering the GDT -- which is easier)

Well, these stuctures are largely whatever you make of them. Certainly, the intention of the LDTs is to allow each process to have its own descriptors, but if the goal is to exceed the 8192 descriptor limit of the GDT, it could be shared between processes. On the other hand, if you have 4000 processes at the same time, you are probably already running more than you should be at a time.

[matthias]

Each LDT can also contain 8192 segment descriptors.

but the LDT cannot contain processes -- all 8192 segment descriptors must be used for the same process (well... technically they dont have to be, but basically your still left tracking outside the CPU structures, which makes it essentially the same as altering the GDT -- which is easier)

Well, these stuctures are largely whatever you make of them. Certainly, the intention of the LDTs is to allow each process to have its own descriptors, but if the goal is to exceed the 8192 descriptor limit of the GDT, it could be shared between processes. On the other hand, if you have 4000 processes at the same time, you are probably already running more than you should be at a time.

hmmm, true, GDT limit of 4000 processes is enough for most applications of segmentation. Imagine you have:

512MB * 1024 = 524288 KB of RAM

524288 / 4000 = 131 KB of RAM for each process

And as we all know these days this isn't enough at all, but hey who needs 4000 processes at a time anyway?! So let's say I build a dedicated server OS with things like: http, ftp and email

64 MB -> OS and IO buffers
64 MB -> ftp server
128 MB -> mail server
256 MB -> webserver

So only (4 * 2) + 1 (null desc) descriptors needed in GDT (in the best situation), nice idea because I don't like paging at all, especially invlpg. The only thing you need is a function like malloc() to allocate memory. Also you're not wasting memory space since you don't use pages (ie if you need to allacote only 4 bytes and your allocator doesn't use the rest of the page that has been allocated.) Well actually you do of course but not with the cost of updating page tables (invlpg again ) This has great advantages. But is it worth the effort? I mean, will it be hard to program?

[bontanu]

BTW...

Do not forget that there is NO segmentation in upcoming 64bits architecture

[matthias]

BTW...

Do not forget that there is NO segmentation in upcoming 64bits architecture

I still aim at 32-bits.

btw, does anyone have good experience with open-watcom and OS-development??

[JAAman]

The only thing you need is a function like malloc() to allocate memory

except malloc is part of the application -- you need something to give the memory to malloc (this requires a syscall -- and you definately do not want to make a syscall every time you need to allocate memory), and then there is deallocation -- this cannot be handled properly with a simple segmentation adjustment

64 MB -> OS and IO buffers
64 MB -> ftp server
128 MB -> mail server
256 MB -> webserver

So only (4 * 2) + 1 (null desc) descriptors needed in GDT (in the best situation),

except, there are many more processes than that -- i currently have 24 processes running right now -- and that doesnt include all the system processes (that number is from windows task manager -- which includes some, but not all, OS processes)

nice idea because I don't like paging at all, especially invlpg.

not sure why -- and it is quite easy to do without invlpg completely (in fact, it didnt even exist until the P5) -- it is only needed for invalidating global pages (which are not invalidated on context change) -- though it can improve performance to use it elsewhere as well

Also you're not wasting memory space since you don't use pages (ie if you need to allacote only 4 bytes and your allocator doesn't use the rest of the page that has been allocated.

but it does -- malloc only requests full pages from the OS (some OSs will only allocate 4KB, or 4MB, or some other larger number, at a time) -- then malloc will reuse the page for several calls

Well actually you do of course but not with the cost of updating page tables (invlpg again Twisted Evil )

not true -- invlpg usually takes zero time -- the only thing it does, is invalidate any TLBs which currently contain that page (if the page isnt in the TLBs, then it will do absolutely nothing... and if it is in the TLBs, it will not take any extra time over what it would have taken if the entry had not been in the TLB) -- it doesnt load the TLB with any new values, it will only reload the page when the page is accessed -- just as if it hadnt been in the TLB at all

and you do have to invalidate if you change the segment register (by reloading the segment -- just as time consuming, and must always be done -- even if you do not access the changed area -- which paging wouldnt require any overhead at all)

This has great advantages. But is it worth the effort? I mean, will it be hard to program?

not as many as you assume -- you seam to missunderstand many things about paging
and yes, it will be difficult to program, (and even more difficult to program properly)