OSDev.org

alexfru: i was able to achieve the things i needed from your compiler, i did the emulation with inline assembly with fake 64 bit number simulation. overally, the compiler is very good for unreal mode dev development purposes.

i most probably will not do any bootable stuff on x86 any more, but i have suggestions to make the software better for future users.

i list these from most important to less important features.

1. minimum optimisations must be introduced.

The generated code is extremely slow on SOME in-order cpus (like on the intel atom n4xx, at least 20x slower). but its suprisingly not SO slow on other in-order cpus, like vortex86. On superscalar cpu-s, the code is still much slower than it should be when compared to gcc (like 5x-10x). This is maybe not a focus in this project, but some minimal binary code optimisations must be done, at least related to the function calls, the for cycles, array addressing. I dont know the structural work of the compiler, but these simply optimisations must be possible to some extent (for example, storing the address computations if its possible for the next operation if its possible, if the variable is alreday copyed to a register then dont do a full resolve code on it again, maybe some very minimal cycle unrolling at least for register pre-computation, etc). This is required for the compiler to make general worldwide interest. (very important)

2. 64 bit long long must be implemented
64 bit signed and unsigned numbers must be implemented, it canot be avoided. i guess the problem with this is you use fix 32 bit wide data passing to function calls and return values. returning a 64 bit value is easy, you probably alreday just copy it to EAX (so the other part of the number can go for example to a different register). adding 64 bit to function parameters is a bit more tricky, but still must be done, since software (escpecially OS-es) alreday using long long if its necessary. having 64 bit numbers is a must have. (very important)

3. better usage of cpu with functions
For example, a function bswap32 and bswap64 should be implemented on language level. so where the cpu is capable of executing a bswap, then that can be very efficient compared to calling a function and doing shitloads of bitshifts. (important)

4. inline
Function inlining must be implemented. Its not complex to implement it, you can just basically copy the contects of the function to the place of the call, and set the variables as it should. (moderately important)

5. inline assembly errors
When its a typo on assembly, a bug writes out random source code line number, when an error is in the inline assembly code. Iit not writes out the place of the actual error. (not important)

Hi,

For example, a function bswap32 and bswap64 should be implemented on language level. so where the cpu is capable of executing a bswap, then that can be very efficient compared to calling a function and doing shitloads of bitshifts. (important)

The ideal solution would be to implement bswap32 and bswap64 as single-instruction assembly procedures and somehow have them get inlined in the caller code.

Function inlining must be implemented. Its not complex to implement it, you can just basically copy the contects of the function to the place of the call, and set the variables as it should. (moderately important)

If the compiler can "see" the whole code (either as source, AST or some kind of IR) at once, then it should be easy. If it uses machine-code object files, the compiler would also need to deal with the complication of at least determining and removing "ret" instructions (and possibly replacing them with jumps when they are in the middle of the functions).


Regards,
glauxosdever

Geri wrote: alexfru: i was able to achieve the things i needed from your compiler, i did the emulation with inline assembly with fake 64 bit number simulation. overally, the compiler is very good for unreal mode dev development purposes.

I'm glad you were able to get most things done and working with the compiler with very little help needed.

Geri wrote: i most probably will not do any bootable stuff on x86 any more, but i have suggestions to make the software better for future users.

i list these from most important to less important features.

1. minimum optimisations must be introduced.

The generated code is extremely slow on SOME in-order cpus (like on the intel atom n4xx, at least 20x slower). but its suprisingly not SO slow on other in-order cpus, like vortex86. On superscalar cpu-s, the code is still much slower than it should be when compared to gcc (like 5x-10x). This is maybe not a focus in this project, but some minimal binary code optimisations must be done, at least related to the function calls, the for cycles, array addressing. I dont know the structural work of the compiler, but these simply optimisations must be possible to some extent (for example, storing the address computations if its possible for the next operation if its possible, if the variable is alreday copyed to a register then dont do a full resolve code on it again, maybe some very minimal cycle unrolling at least for register pre-computation, etc). This is required for the compiler to make general worldwide interest. (very important)

The compiler is very simple and compact. It only keeps in memory the encountered declarations and the current expression (and a little bit of other accounting stuff like global/static identifiers, switch tables that are in use at the moment, etc). With this little info you can't make any serious optimizations. You can't, for example, keep an array index in a register because you don't know if there are pointers to the index (and therefore possible aliases of the index) and so you have to assume the worst and keep reloading the index from memory on every iteration.

As for importance, it's questionable. You want performance because you're emulating and your subleq-based code itself is quite slow. You normally should use the right tool for the task. Setting aside the questions of practicality of subleq, a non-optimizing compiler is hardly the right choice for computationally intensive tasks such as emulation. But for a boot loader which runs once and doesn't need to process mega- or gigabytes of data over and over again, it's OK. If you decide to get more performance, you should pay the cost of porting the code to protected mode, compiling it with gcc or clang, and either implementing protected mode device drivers or a virtual 8086 task to keep using the BIOS. That would be the right way/tool. Right now you're saving on protected mode and drivers development at the expense of performance. You can do it the other way around.

Geri wrote: 2. 64 bit long long must be implemented
64 bit signed and unsigned numbers must be implemented, it canot be avoided. i guess the problem with this is you use fix 32 bit wide data passing to function calls and return values. returning a 64 bit value is easy, you probably alreday just copy it to EAX (so the other part of the number can go for example to a different register). adding 64 bit to function parameters is a bit more tricky, but still must be done, since software (escpecially OS-es) alreday using long long if its necessary. having 64 bit numbers is a must have. (very important)

Again, the compiler effectively has just one type under the hood, a machine word. Introducing a type of the double size requires additional type checks and type conversions from and to this new type, code generator support for pairs of registers, a second type of the switch statement and a number of smaller changes (e.g. parsing and storing longer values).

Geri wrote: 3. better usage of cpu with functions
For example, a function bswap32 and bswap64 should be implemented on language level. so where the cpu is capable of executing a bswap, then that can be very efficient compared to calling a function and doing shitloads of bitshifts. (important)

This is probably a request to the C language standardization committee.

Geri wrote: 4. inline
Function inlining must be implemented. Its not complex to implement it, you can just basically copy the contects of the function to the place of the call, and set the variables as it should. (moderately important)

Except, like I said earlier, a whole function never exists in the compiler's memory. This makes it problematic to even figure out how much stack space to allocate at the beginning of the function because this information is only available when all of the function has been seen. Right now this is solved by emitting a string like "sub sp, <10 spaces>" at the beginning of the function, taking a note of its position in the output file and then at the end of the function going back and overwriting those spaces with the computed value.

Geri wrote: 5. inline assembly errors
When its a typo on assembly, a bug writes out random source code line number, when an error is in the inline assembly code. Iit not writes out the place of the actual error. (not important)

You can compile C code to assembly code (use the -c -S switch pair) and look at the latter. You've probably figured it out already. Again, it's better to have some inline assembly support than none and simply passing assembly text to the assembler is the cheapest way of doing it, which, of course, has its shortcomings. Just like you're saving on protected mode and drivers, I'm saving on some things in the compiler.

Smaller C runs in 96KB of RAM on a PIC32 MIPS microcontroller. [Belard's TinyCC won't fit, PCC won't fit, LCC won't fit. Hendrix'/Cain's/etc Small C fits, but is too limited.] Adding various improvements and optimizations will either make that impossible or require splitting the compiler into multiple stages, which means a substantial redesign/rewrite. I don't get paid to work on this compiler full time and so I do what I can and when I can. And a large part of that "I can" consists of what "I want". So, your "must" and "required" are viewed through the prism of these natural limitations and my personal choice. If there's a match between what you want and what I want and if I can do it, I may indeed do it. If there isn't, sorry. You can ask for something but there's no guarantee it will be given.

i understand your standpoint, i also refused to implement a lot of feature in my c compiler - for example, structs, array value passing to functions, typedefs, function pointers, so i basically left out every unnecessary features (it works very different ways than other c compilers, as subleq required special approach to have everything going relatively fast on a plaform without any kind of registers/stack/segments). and yeah, on the bootable emulator, i also decided to use unreal mode, as you mentioned, to avoid dealing with complex drivers and break the time down to fragments. so keeping the things as simple as possible is very important, becouse our time is limited.