OSDev.org

Hello,

I am following Brokenthorn's series on OS dev. Things are going slowly but steadily (I've found a pretty convenient set-up and workflow now), and I'm able to load 32-bit code to 0x100000. This however, is still assembled code. When I try to follow the tutorial, which transitions from assembly to C++, it all becomes a big blur. In the series, the PE-executable format is used. This seems a bit of an overkill for me. Since I've also heard that using GCC to make a cross-compiler has advantages, I'm seriously considering to install cygwin and go with GCC.
On the other hand, I don't have ANY experience with GCC whatsoever (and very little with linux or bash scripts and the like). My experience with visual studio is limited to compiling normal user applications in various languages (I never bothered with the huge pile of options and settings), so I figure I'm gonna have a hard time getting a compiler to compile to the format that I want anyway.

Any tips or hints about what I should pick? I'm currently reading the OSDev pages about cross-compilers and GCC and I'm not entirely sure what to do, but maybe things will get clearer when I install cygwin and try to become more familiar with linux-like environments. Any directions on good reads are also more than welcome!

What are you asking? Which development environment to use?

Cygwin is straight forward, but you need half a day to set up the cross compilers. All tutorials you ever need can be found on the wiki.

However, you have Visual Studio (by far the nicest C++ IDE, but you can also use the VS IDE with Cygwin) and that has everything you need, but not many people here use the PE executable format, so if you find a good tutorial that can help you out it will do.

I'm currently trying to write an article about these issues, it's not finished yet, but you can already find some information about how to make C program that will work with several compilers, and how to interface safely between C and assembly, or between two C program compiled on different configurations.

viewtopic.php?f=8&t=27147 this thread is about that actually.

Agner frog articles are also well made about this topic.

But you can perfectly use visual studio to build an OS, i know a bit about PE and ELF format.

Typically the base assembly code will be assembled as flat binary object and loaded at a fixed location, then you need to be able to load exe file from it to be able to interface with C program compiled into a specific executable format. Or you can maybe link C program as flat binary and load them at specific location if you don't want to program a PE or ELF loader in assembler. But you'll still have to make a PE or ELF loader if you want to be able to use exe files produced by visual or gcc easily.

you can use mingw in visual studio as well, but i think under windows, both cygwin and mingw will produce PE format executable anyway, and they will potentially require mingw dll at runtime to function. So i'm not sure it's a very good solution.

Devcpp also use gcc under windows, it can worth a try if you want to use gcc compatible building environment, or you can define custom build rule that use mingw in visual. But it will still produce COFF object and/or PE executable.

cygwin and mingw are more made to be able to port GNU application to windows rather than the opposite. So you can use gcc in visual with mingw, but the intended use is more to compile GNU/POSIX code for windows, rather than to build GNU/POSIX compliant exe under windows.



And yes you'll most likely need to become familiar with visual studio compiling/linking options if you use the visual studio C compiler .

Yeah sorry, I tend to make really vague posts when I'm confused Basically I'm a little lost in choosing a compiler, and indeed, I want to know what's the best development environment.
Thanks for your reaction! I did a bit of searching, and combining visual studio and cygwin doesn't seam ideal (or perhaps you meant using VS as an editor and cygwin to compile the lot? That would probably work).


Phew, that sucks. I was under the impression that GCC can compile to a flat assembly format... Well, thank you too, I'm curious about your tutorial (edit: wait, you meant the one on the wiki? Found it )!

So, as far I understand, I basically got the choice of compiling in VS and get a PE-executable, or compile in GCC and get an ELF-executable?

I certainly do not recommend reading the article in question until it has been polished. Just a friendly warning.

yes it's not finished yet, it can just give some idea of the issues at hand, and reading the thread i posted will also give more clue as what is incorrect in it

the final exe format doesn't depend directly on the compiler, i think ld can produce flat binary file, but binary file can't include reference to any external symbol. So you can't use symbols defined in another module from a file linked as binary, unless you know the location of the symbol at runtime.

Not sure if you can build GNU/POSIX executable under windows, like producing elf file that will be compatible with a 'true POSIX' compliant environment, but cygwin and mingw are not made for that, they are made to make windows compatible exe from GNU/POSIX programs.

You can use gcc/ld to produce COFF/PE files, both under linux and windows, but it will be harder to produce elf file with visual studio or under windows in general. But maybe there are mingw or cygwin version of gcc that can be used to produce elf file, but it's not the objective normally.

The other choice is to make your own format and convert PE or ELF file to an internal format, which is what i do, but otherwise if you develop under windows, you'll most likely have to deal with COFF/PE format.

Oh, I see! Isn't that kind of thing (the thing which loads an executable file and it's dependancies) called a loader? I wanted to get some experience with linkers and loaders, but never really got to them. Too bad, because I'm confused now Ah well, I might learn a thing or two now

One more thing: in Brokenthorn's series, there are a lot of header files like _null.h, stdarg.h, stdint.h, ... in the include directory (in the VC++ project). What is this exactly? My first guess it was a minimal replacement of the C runtime library, but the code isn't even a library: it are just definitions and macro's...

Sorry, I'm kind of a newbie I guess
I try to find out as much as I can without asking other people, but I'm kind of stuck here...

yes it's what is called a loader, it will parse the exe format and resolve any symbols they import, and will recognize the symbol they export if other programs are linked with it. And they can also deal with relocation if the program is not loaded at the address the linker expects. With paged memory, each exe will have his own address space and the linker expect that the loader will setup this address space before to load exe, if you don't want to use the specific virtual memory layout, you need to use relocation.

Both relocation and import/export are entry in a table, stored in the exe format, that say at which position in the section a symbol reference will be located in the executable file, and the loader need to set the address of the symbol to either the location where the shared library linked with is loaded, or either with the absolute or relative address of the symbol if relocation are used because the exe is not loaded in the memory layout the linker expect.

they are probably replacement of the C Library programmed with macros If you would use the standard visual studio C Library, it would need to distribute visual studio runtime with the file being compiled, or linking it statically, so they probably use macros to have 'inlined' code that doesn't need to be resolved at runtime and doesn't depend on visual runtime dlls (like msvcXX.dll) .

The same behavior could be achieved by declaring the functions as 'static __inline' in the headers.

And yess, it works! Not like a charm: when I set Visual Studio to optimize speed it breaks some things (no idea why...), but at least I can compile my kernel now. It was easier than expected, just adding a couple of jumps in my stage 2 bootloader where enough.
I used brokenthorn's tutorial, it's all explained pretty well.

Thanks for your book suggestion btw, dozniak

Hello,

We actually recommend using a make file build system with well defined requirements for the build process in order to decouple development for a specific tool chain. Although the series uses the IDE, it only does so for simplicity. In the long run, a make file build system is the way to go.

It is of course possible to use an alternative compiler or linker. The only requirement is the use of a compatible object file format; for example, you can use CL (the compiler with MSVC) with LD using make file scripts. I am unaware if you can configure the IDE to use LD instead of LINK; although you can configure it to run separate programs I don't know if you could customize this aspect of the link process through the IDE.

In short, you don't have to use PE with Visual Studio. There are ways around that by changing the build system.

GCC is typically more recommended though depending on your design goals.

(1) MSVC is closed source, this could produce problems in the future and make self hosting impossible;
(2) MSVC does not support the latest C standard. The latest supported is C89; few later C extensions supported.

Building a cross compiler would also resolve possible hidden dependencies between the generated code and host operating system that would cause problems. These problems though could be avoided by careful programming.

We do encourage supporting GCC though in order to resolve (1) and (2) above.

If you don't want to use PE, you will need a different linker. LD is great; although it does support generating flat binaries, I don't know if that will work well with CL. It does work well with GCC though of course; but I really recommend against using a flat binary image. That is, I highly recommend using a standard executable format or a custom one.

If the end goal is self hosting, you will either need to write a compiler or supporting an open source one like TinyCC or GCC that is compatible with your build system.

Yeah, exactly what neon said.

You may not have that much experience with GCC, but it's totally worth it to put in the effort learning it. You can teach GCC and binutils what your OS is and tailor them exactly to your need if you are willing to modify its source code (I'd gladly help with that) and be in absolute control of your compilation environment. This is really great if you are thinking about going self-hosting. The reason we recommend using a cross-compiler is because it understands that it's not targeting the current system, but rather that something special is going on. If you use MSVC or a gcc that doesn't target your OS (or i586-elf or something generic), then you need all sorts of tricks to fool the compiler into compiling your OS and programs for your OS. This is naturally quite fragile and complicates things considerably - in contrast, things work reliably in a simple-ish manner when using a compiler that understands what's going on.

Note that unless you make some licensing deal with Microsoft, Visual Studio is a dead end if you ever consider going self-hosting (you may not want that today, but suddenly it seems possible and you want it).