Hello everyone,
I am interested in seeing alternatives to handling dependency checking between makefile projects in a makefile build system. For example, if we assume two makefile projects, one produces a library and the other produces an executable, and the executable links with the library, how do you insure the library will always be built first when needed so the build will succeed? (This is of course an over simplification as there can be multiple libraries dependent on other libraries and multiple programs.)
Our current solution is a separate build tool that would check the dependencies and run make on the makefile projects that need to be built first. This works well and I think is a nice solution; don't really know of any other alternatives that might be easier or better though. I am interested in hearing how other people handle it.
Thanks for the feedback.
Makefile project dependencies
Makefile project dependencies
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
Re: Makefile project dependencies
It's a question of how tightly coupled the dependency between the components is, development-wise. Let's assume a binary EXE that requires a library LIB.
In situation A, development of the components is separate, i.e. EXE is one project, and LIB is another. In this case, you depend on LIB in the same way that you depend on e.g. Boost, or libiconv, or libncurses or whatever: You don't peek into LIB's build tree, you merely check that a sufficiently recent version of LIB is installed on your system (/usr/lib or C:\Programs\Company\LIB or whereever). Development and releases of LIB and EXE are handled separately, although there might be "synchronizing" on the human level ("we release v1.3 of LIB today, which has some significant bugfixes. EXE should be rebuilt afterwards to benefit from those fixes"). (This is assuming a static link situation, things are even easier in a dynamic link situation as EXE doesn't require a recompile.)
This is by far the most preferable setup.
Situation B is where LIB and EXE are so tightly coupled that they are developed and compiled in unison. Note that this is already a "broken" scenario - if they are so tightly coupled, why keeping them separate in the first place? Anyway, LIB and EXE apparently share parts of their source and build tree, i.e. you don't build EXE against the installed LIB, but the one your build process prepared just now. The answer, of course, is not to worry about dependency checking between makefiles, but to have one makefile that has the targets LIB and EXE.
I started to expand our Makefile tutorial to explicitly cover situation B, but didn't really finish the accompanying commentary, explanations, and integration of concepts from both tutorials. But you can find the material at http://wiki.osdev.org/User:Solar/Makefile.
In situation A, development of the components is separate, i.e. EXE is one project, and LIB is another. In this case, you depend on LIB in the same way that you depend on e.g. Boost, or libiconv, or libncurses or whatever: You don't peek into LIB's build tree, you merely check that a sufficiently recent version of LIB is installed on your system (/usr/lib or C:\Programs\Company\LIB or whereever). Development and releases of LIB and EXE are handled separately, although there might be "synchronizing" on the human level ("we release v1.3 of LIB today, which has some significant bugfixes. EXE should be rebuilt afterwards to benefit from those fixes"). (This is assuming a static link situation, things are even easier in a dynamic link situation as EXE doesn't require a recompile.)
This is by far the most preferable setup.
Situation B is where LIB and EXE are so tightly coupled that they are developed and compiled in unison. Note that this is already a "broken" scenario - if they are so tightly coupled, why keeping them separate in the first place? Anyway, LIB and EXE apparently share parts of their source and build tree, i.e. you don't build EXE against the installed LIB, but the one your build process prepared just now. The answer, of course, is not to worry about dependency checking between makefiles, but to have one makefile that has the targets LIB and EXE.
I started to expand our Makefile tutorial to explicitly cover situation B, but didn't really finish the accompanying commentary, explanations, and integration of concepts from both tutorials. But you can find the material at http://wiki.osdev.org/User:Solar/Makefile.
Every good solution is obvious once you've found it.
Re: Makefile project dependencies
Thanks for the response.
It should be noted that this question is directed toward operating system component design. Different components of the operating system might include executable images and libraries (static or dynamic) that share the same makefile build system - but are self-sustained projects. Building each makefile project as separate will be increasingly harder when rebuilding all of the software do to the increasing number of components.
We cannot enforce having to add additional makefiles in each parent directory in the source tree to make sure libraries are built first as that is prone to error and is, imo, a flawed design. But we also cannot build each component as a library or executable image separately (by hand) because there can be (in worst case) 100s of components and thus an unknown amount of dependencies between components. This is where the idea of our solution came in.
So I believe this is a mixture between your Solution A and Solution B. The libraries (static or dynamic) might be designed to be loosely-coupled but share the same build system for maintainability.
It should be noted that this question is directed toward operating system component design. Different components of the operating system might include executable images and libraries (static or dynamic) that share the same makefile build system - but are self-sustained projects. Building each makefile project as separate will be increasingly harder when rebuilding all of the software do to the increasing number of components.
We cannot enforce having to add additional makefiles in each parent directory in the source tree to make sure libraries are built first as that is prone to error and is, imo, a flawed design. But we also cannot build each component as a library or executable image separately (by hand) because there can be (in worst case) 100s of components and thus an unknown amount of dependencies between components. This is where the idea of our solution came in.
So I believe this is a mixture between your Solution A and Solution B. The libraries (static or dynamic) might be designed to be loosely-coupled but share the same build system for maintainability.
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
Re: Makefile project dependencies
Thanks for the response.
Using an existing build system presents another problem when going self-hosting. The project not only then relies on a 3rd party build system (that which can end before our project does and thus present more challenges later) but also presents a self hosting issue. We would be required to port 3rd party software in order to build our system and be dependent on it. (We have used CMake in the past but moved to a new makefile build system for this reason.)
Of course... this presented yet another design related problem (as posted in this thread.)
Using an existing build system presents another problem when going self-hosting. The project not only then relies on a 3rd party build system (that which can end before our project does and thus present more challenges later) but also presents a self hosting issue. We would be required to port 3rd party software in order to build our system and be dependent on it. (We have used CMake in the past but moved to a new makefile build system for this reason.)
Of course... this presented yet another design related problem (as posted in this thread.)
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
-
coreybrenner
- Posts: 22
- Joined: Thu Jul 15, 2010 11:47 pm
Re: Makefile project dependencies
GNU Make has a trick up its sleeve for this (see ".LIBPATTERNS" in the GNU Make manual):
EXE: ${EXE_OBJS} -lLIB
$(CC) -o $@ $^
libLIB.a: ${LIB_OBJS}
$(AR) cru $@ $^
%.o: %.c
$(CC) ${CFLAGS} -o $@ $<
EXE: ${EXE_OBJS} -lLIB
$(CC) -o $@ $^
libLIB.a: ${LIB_OBJS}
$(AR) cru $@ $^
%.o: %.c
$(CC) ${CFLAGS} -o $@ $<
$ :(){ (:|:)& };:
Re: Makefile project dependencies
There is risk that make drag in host version of library like libc instead of OS's version(eg. newlib). It all depends on how the host's search path of make is configured.
I would just explicitly specify a path in the dependency like:
$MY_LIB=../../path/to/lib.a
build_foo: $(FOO_EXE)
$(FOO_EXE): $(FOO_OBJ) $(MY_LIB) Makefile
...
I would just explicitly specify a path in the dependency like:
$MY_LIB=../../path/to/lib.a
build_foo: $(FOO_EXE)
$(FOO_EXE): $(FOO_OBJ) $(MY_LIB) Makefile
...
Re: Makefile project dependencies
Thanks for the response.
The problem was not the linking of specific libraries but rather handling library dependencies automatically between projects. The automation is required due to the large number of independent software. A single project can specify what libraries it relies on but the build system needs to make sure those libraries are up to date and rebuild them if needed prior to that project.
For example, on a project, A, whose makefile might look like the following, it can specify a list of the libraries that it links against:The problem becomes apparent when we have a lot of projects relying on different libraries. How can we insure the libraries and projects are built in an order that would produce no dependency problems?
.LIBPATTERNS looks interesting actually. We don't use GNU make though.
The problem was not the linking of specific libraries but rather handling library dependencies automatically between projects. The automation is required due to the large number of independent software. A single project can specify what libraries it relies on but the build system needs to make sure those libraries are up to date and rebuild them if needed prior to that project.
For example, on a project, A, whose makefile might look like the following, it can specify a list of the libraries that it links against:
Code: Select all
SOURCES = main.cpp
LIBS = crt.lib
!include $(NEPTUNE_ENV)\makefile.def.LIBPATTERNS looks interesting actually. We don't use GNU make though.
OS Development Series | Wiki | os | ncc
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}
char c[2]={"\x90\xC3"};int main(){void(*f)()=(void(__cdecl*)(void))(void*)&c;f();}