Making compilation faster&

piranha · Post by **piranha** » Sat Feb 09, 2008 7:48 pm

I had an idea.
If you have a lot of code files, then why not start each 'gcc' command as a new process (by appending & to the end of the command)?
This may only work on a fast computer.
Problems:

1) The code files all need to be there when linking
Solution: When all commands are started, right before linking add a 'sleep 0.5' command which pauses for 1/2 second while the commands finish.

For me:
All my code files compiled in 4 seconds when doing one-at-a-time processing.
When each is a new process: 1.6 seconds. Thats a factor of 2.5! Much faster.

-JL

Alboin · Post by **Alboin** » Sat Feb 09, 2008 8:25 pm

Brynet-Inc · Post by **Brynet-Inc** » Sat Feb 09, 2008 9:00 pm

I'm officially stunned.. as is Alboin.

Are you serious?

babernat · Post by **babernat** » Sat Feb 09, 2008 11:47 pm

Wow...depending on the language this may or may not be possible. The most I've seen is a compiler starting up multiple threads for use only in the lexical and syntax analysis portion. The slowest part of compilation generally revolves around file i/o.

piranha · Post by **piranha** » Sun Feb 10, 2008 12:51 am

I'm officially stunned.. as is Alboin.

Are you serious? Rolling Eyes

What do you mean?

It works on my computer, and I like to not waste time.

-JL

ucosty · Post by **ucosty** » Sun Feb 10, 2008 1:02 am

you could use a makefile with the -j option.

http://unixhelp.ed.ac.uk/CGI/man-cgi?make

-j [jobs], --jobs[=jobs]
Specifies the number of jobs (commands) to run simultaneously. If
there is more than one -j option, the last one is effective. If
the -j option is given without an argument, make will not limit
the number of jobs that can run simultaneously.

piranha · Post by **piranha** » Sun Feb 10, 2008 1:28 am

Well, then can you control what links when?

...And I had some fun with the make file for 5 min.

-JL

ucosty · Post by **ucosty** » Sun Feb 10, 2008 1:58 am

The dependencies system should take care of that.

piranha · Post by **piranha** » Sun Feb 10, 2008 2:08 am

whatever, I was bored.

elfenix · Post by **elfenix** » Sun Feb 10, 2008 10:22 am

Errr.....

Wondering why you aren't using make files... Or Visual Studio Projects.... Or Sconscripts....

See - 'make -j', "Enable Parallel Builds", 'scons -j'

Edit: guess you were.... go back to your original Makefile and use,

make -j 4 (target) - if you have dual core or on single core

make -j 8 (target) - if you have more...

Further edit: not my day, I see this was already mentioned...

JamesM · Post by **JamesM** » Mon Feb 11, 2008 6:14 am

Then depending on how hard core (tm) you are, you can use distcc and ccache to distribute builds across multiple machines (we dispatch 12 files at a time

) and cache the object files.