Some basic questions from the beginner

[Dario]

Hi!

I've been reading this forum for some time and found many interesting things. I'm one of those guys who wants to understand computing in it's lowest levels, so I plan to develop a small operating system with some basic file system and usual drivers. I'm not in the rush...so it should keep me writing it next 2-3 years.

For the start...I have couple of questions concerning the CPU registers which I can't find the solution to...
So, every process is segmented in memory (.code, .data, .stack and .heap)...question is, do the segment registers like CS, DS, SS...select these segments in memory or did I confused it with something else...What is the relation between these segments and the segments registers?

Another question is concerning the I/O sys. How does one device know to which memory location it should right some data, and from which memory location it should take instructions. Does it use some kind of standard, or does the BIOS make all the necessary mapping?

Is this simplified "drawing" correct?
If it is....how does the device know when it has received data or instruction?
My guess is using interrupts....

Code: Select all

                           SYS.MEM                                       DEVICE
                                                 READ DATA             Dev.mem. buffer
                 0x00A0  |__________|---------01011------------------>|__________|
                 0x00A1  |__________|<--------10110-------------------|__________|
                 0x00A2  |__________|           WRITE DATA
       CPU       0x00A3  |__________| 
                 0x00A4  |__________|
                 0x00A5  |__________|---------0001------------------->|__________|
                 0x00A6  |__________|           INSTRUCTIONS            DEVICE REGISTERS

Thank you for your time and sorry for my bad English.

[bewing]

There are too few words in english to describe the concept of "segments". So there are many completely unrelated things that end up being called "segments", for the lack of other names to call them. It is unfortunate that there are segments of programs, as you say, and there are also segment registers. They have nothing to do with each other. Segment registers are used for hardware memory mapping. Program segments are used by the compiler and loader software programs to divide applications into loadable chunks in memory.

Devices use interrupts to communicate "events" to the CPU. The CPU uses the IO Port Bus to communicate to devices. The CPU can send "instructions" and receive data over the IO Port Bus with devices. The CPU can also send and receive data (sometimes) through system memory with devices. The IO Port Bus is completely separate from memory.

A few devices have traditional IO port numbers. Most devices these days are PCI and get their IO port numbers assigned by the BIOS. (I was not able to really understand your drawing. )

[Dario]

There are too few words in english to describe the concept of "segments". So there are many completely unrelated things that end up being called "segments", for the lack of other names to call them. It is unfortunate that there are segments of programs, as you say, and there are also segment registers. They have nothing to do with each other. Segment registers are used for hardware memory mapping. Program segments are used by the compiler and loader software programs to divide applications into loadable chunks in memory.

Ok...so...are these segment registers used for task switching in HW?
I thought that they are being used for segments protection during the execution of a program...apparently I was wrong.
What are they used for?

* For these programs segments I assume that they are described in executable headers like ELF...?

Devices use interrupts to communicate "events" to the CPU. The CPU uses the IO Port Bus to communicate to devices. The CPU can send "instructions" and receive data over the IO Port Bus with devices. The CPU can also send and receive data (sometimes) through system memory with devices. The IO Port Bus is completely separate from memory.

That explains a lot....thanks.
So CPU is directly connected to these devices via system bus which has I/O lines, address lines, control lines, data lines...and every time device needs CPU's attention it sends an interrupt after which it can send data?
How are these I/O lines addressed?

A few devices have traditional IO port numbers. Most devices these days are PCI and get their IO port numbers assigned by the BIOS. (I was not able to really understand your drawing. )

The drawings shows CPU on the left side, sys.memory in the middle and some device on the right with it's memory buffer and registers...so devices writes and reads data from some memory locations and receives instructions from another mem. location/s.


Sorry if I ask too much

[bewing]

Ok...so...are these segment registers used for task switching in HW?
I thought that they are being used for segments protection during the execution of a program...apparently I was wrong.
What are they used for?

Unfortunately, that is a very complicated subject. They can be used for all of those things, or none of them. However, the segment registers do not really work anymore in 64 bit ("Long") Mode. So, most coders do not use them in 32 bit mode ("PMode" or Protected Mode) either -- so that the 64 bit and 32 bit versions work the same. You need to use segment registers in 16 bit (Real) mode, to get the system to boot. Once the OS is booted, you might only need to make sure that the segment registers always contain valid numbers. Other than that, most coders find them to be quite useless.

For these programs segments I assume that they are described in executable headers like ELF...?

That is exactly right.

So CPU is directly connected to these devices via system bus which has I/O lines, address lines, control lines, data lines...and every time device needs CPU's attention it sends an interrupt after which it can send data?

Yes.

How are these I/O lines addressed?

The IO Port Bus can only be addressed by the IN and OUT (or INS and OUTS) assembly language instructions in the CPU. You can create assembly language routines to do this for your OS, or you can use something called "inline assembler" inside some other programming language than assembly.
The IO Port Bus is an 8 bit bus -- it sends a byte at a time over the bus, and it is very slow. In almost all cases, you will access it to send or receive only one byte.

In all of the C code examples that you will find, you will see the IN and OUT functions accessed through C functions (or inline macros) that are named inb() and outb().

Make sure that you go and read the FAQ on our wiki, soon. http://wiki.osdev.org/Category:FAQ
So far, your questions are good ones.
The rest of our wiki contains a large amount of information. In the end, you will need to learn most of it to make an OS.
It is good that you are not in a hurry, because creating an OS takes many years -- no matter how fast you try to write it.

[Dario]

Thanks a lot bewing!

I got some more questions...concerning the FS and disk driver.
To create a disk driver I need controllers instructions...some kind of manual...and implement it using assembly, create calls to interface
some HL programming language like C?
What is FS anyway?
To my knowledge, I see it as a table which maps physical blocks to some logical structure....

[bewing]

Most people start with floppy disk drivers. They are standardized and there is a lot of info on the wiki about them. However, floppy drives are extremely unreliable and your driver needs to be able to do everything 3 times, if the first two times fail.

The most basic type of hard disk is called ATA or PATA. It is also standardized, and the basic operation mode (called PIO mode), is well documented on the wiki (ATA_PIO_Mode) and at T13.org (there are links from the wiki). Look for the ATA6 specifications. The wiki article has a complete driver in assembly, and explicit instructions for a driver in C.
If you try to create a driver for anything more complicated than PATA, it will be much more difficult.

A filesystem has two pieces. One of them is a map from physical blocks to a logical structure. The other half describes the "directories" in the filesystem, in detail. All current hard disks also contain an MBR with a Partition Table, which divides the physical blocks into smaller, "virtual" disk drives. It is usually easiest to choose an FS that is compatible with your development computer, so that you can copy files back and forth, automatically. (You will have to do that many many many times.) There are two main underlying designs for an FS. There is the FAT type, and the inode type. You will need to understand both very well. Each has advantages. Most people choose either FAT12 (DOS -- because it is very simple), FAT32 (Windows -- because it is so common), or Ext2 (Linux -- because most programmers develop on Linux, and Ext2 is better than FAT32). However, some of us are designing a custom FS, of our own.

[Dario]

Thanks bewing

[Love4Boobies]

The IO Port Bus can only be addressed by the IN and OUT (or INS and OUTS) assembly language instructions in the CPU.

Ah, but there's also memory-mapped I/O. Take for instance VGA. This makes it a lot faster (it is asynchronous) but also takes up memory.

All current hard disks also contain an MBR with a Partition Table, which divides the physical blocks into smaller, "virtual" disk drives.

There's also the notion of EBR (for Extended Boot Record; or EPBR for Extended Partition Boot Record), where extended partitions are defined. A MBR can only contain up to 4 partition but there can be numerous EBRs on a disk (they each have a link to the next one, forming a chain). The number of extended partitions is only limited by the amount of disk space available.

[Dario]

Hi,

some more questions:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------
1. In Linux, this is a description of a brk() system call:

Code: Select all

DESCRIPTION
       brk()  sets the end of the data segment to the value specified by end_data_segment, when that
       value is reasonable, the system does have enough memory and the process does not  exceed  its
       max data size (see setrlimit(2)).

It's confusing....I though that .data segment is fixed size?
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
2. Process layout in Linux viewed from pseudo FS.

Code: Select all

08048000-0804c000 r-xp 00000000 08:06 1553524    /bin/cat
0804c000-0804d000 rw-p 00003000 08:06 1553524    /bin/cat
0804d000-0806e000 rw-p 0804d000 00:00 0          [heap]
b7d04000-b7e2c000 r--p 00000000 08:06 524614     /usr/lib/locale/locale-archive
b7e2c000-b7e2d000 rw-p b7e2c000 00:00 0
b7e2d000-b7f54000 r-xp 00000000 08:06 2976106    /lib/tls/i686/cmov/libc-2.3.6.so
b7f54000-b7f59000 r--p 00127000 08:06 2976106    /lib/tls/i686/cmov/libc-2.3.6.so
b7f59000-b7f5b000 rw-p 0012c000 08:06 2976106    /lib/tls/i686/cmov/libc-2.3.6.so
b7f5b000-b7f5e000 rw-p b7f5b000 00:00 0
b7f6b000-b7f6d000 rw-p b7f6b000 00:00 0
b7f6d000-b7f82000 r-xp 00000000 08:06 2962615    /lib/ld-2.3.6.so
b7f82000-b7f84000 rw-p 00014000 08:06 2962615    /lib/ld-2.3.6.so
bf8b0000-bf8c5000 rw-p bffeb000 00:00 0          [stack]
ffffe000-fffff000 r-xp 00000000 00:00 0          [vdso]

First, why is the address space so huge...or does it really matter in the flat memory model?
Can you please say which segment is .code, .data....I know that first is .code and second should be .data but it has writable permission, then comes heap and then huge unallocated memory space and than....data again from a different object?
Also what is that segment, vdso, before the stack?

Thank you for your time.

[Dario]

Your google skills are really amazing, but it doesn't answer my questions.

[Solar]

The link provided goes to great length describing exactly what that [vdso] entry is about.

[Dario]

Please note, that my question was not only [vdso] related.

[Dario]

Data segment is not fixed in linux, obviously, brk/sbrk set process memory allocation for example for setting up a user-space heap in the process image. Afaik libc uses it in its malloc() allocator to dynamically provide more memory to the process but without growing the bookkeeping too much if it doesn't need it. But i'm not an expert in linux.

Then what's the point of .data segment if it can vary in its size?
Why would system alter it's size, if that portion of the memory is defined by the complier?

Also the questions which obviously you forgot about:
1. First, why is the address space so huge...or does it really matter in the flat memory model?
- I guess here the answer is to give enough unallocated space for stack and heap to grow. But the system maps only allocated memory pages to physical memory.

2. Can you please say which segment is .code, .data....I know that first is .code and second should be .data but it has writable permission, then comes heap and then huge unallocated memory space and than....data again from a different object?
-I also have some perception about it...every object has it's .data, .code and .heap I guess.

[Dario]

I realise, that my googling skills do not impress you, but asking google "what is sbrk" yields exactly the result you've been asking for for a few days here: what is brk/sbrk.

My job here is done, I'm not holding your hand anymore: http://www.google.com, "How to ask questions the smart way" - type it in search bar and press Enter for enlightenment.

My friend....for every question been asked, you can freely point to google, internet is large enough to provide that for you. Then this forum doesn't make any sense.
In fact, I find pointing people to google very rude...especially since my question wasn't very noobish.

Thank you for your precious time.

[Solar]

Deleted.

OK, after a few breaths I decided I'd re-phrase this one.

No, your question was not exactly noob-ish. However, it is rather impolite, when given a link that is very explicitly answering one out of two of your questions, to claim that it is "not answering your question". It also reinforces the impression of your first few posts that you don't like to read / research much (like, the Intel manuals, for example, which do a great job explaining segments and are required reading for any kind of OS work).

Combined, it gives the impression of someone who'd like to get his OS spoon-fed to him, something we encounter quite often here, and have grown rather impatient to.

Searching the fine web (STFW) is an excellent resource. This forum is - or at least, was - intended to answer those questions that are very specific, not answered by the documentation available one Google (or, back then, Yahoo, or even Altavista) away. Attacking the forum as being "not helpful" is another such thing that won't win you many friends here. Indeed I know of no better place to ask the real questions about writing your own OS, x86 or otherwise.

Please take a step back, reconsider your approach, and consider reading at least volume 1 of the Intel manuals before proceeding with your OS plans.