OSDev.org

Currently, I have the entirety of my OS project, complete with necessary pieces for bootup (mostly drivers - it's a microkernel) and (eventually) the base system, all in separate directories within one common Git repository. I like the BSD model of developing the entire system together so things are consistent, which is why I initially wanted to keep things together like this.

However, I'm worried that if other people eventually join the project, having all the pieces of the OS in the same repository tree will become unwieldy; even though some pieces are very interdependent, other pieces are not, and I can't completely predict the dependency of each piece on the others yet. I don't want to have the revision history of the toolchain mixed up with the userland mixed up with the kernel and drivers. It also might be helpful to be able to download and compile each piece separately, even if I don't think the source code size will balloon much.

But conversely, I definitely want some pieces to be developed together, and having a common revision history would be helpful in those cases for debugging. For example, (as a future user) if a core format or library had to change *right now*, I could make sure the entire system was up to date by getting the latest system source tarball and updating everything at once.

Is it possible to properly manage such a setup using the features of Git? I could see having a branch for each sub-project, but still share a common master source tree - bugs could be tracked through the entire revision history tree, and everything would remain together for the end user. If this isn't fully possible or has some unforseen side effects, what would be a better organization of my project? If there is a better organization, how could I rearrange things within the repository without losing my revision history? (Even if this is not the case, it may be helpful for others to know for future reference.)

I don't use GIT, but at least in subversion you can check out subdirectories of the repository, and you can query for logs for subsets of the repository in a similar manner.

Some guru or the manpages might be able to give the corresponding GIT commands as they will most likely exist there as well.

From a 'make' perspective, if something depends on something else, it should reside in the same repository (so that 'make' can check time stamps and build the dependency graph).

Alternative, start worrying about package management including dependency handling now.

Solar wrote:From a 'make' perspective, if something depends on something else, it should reside in the same repository (so that 'make' can check time stamps and build the dependency graph).

I guess it could be a big advantage to have all dependencies in the same tree for bootstrapping purposes. If I wrote some scripts, I could make it work quite well: you would type "make bootstrap" in the root directory, it would build the toolchain and libraries for the foreign ABI (i.e. build a cross-compiler), double-recompile the toolchain for consistency, bootstrap the kernel, drivers, and userland with native tools, and build a native toolchain, all without user intervention or patching to make sure different versions play nicely.

I think the benefits of keeping things together outweigh the disadvantages, at least so far. I think it actually would work fine to have each program developed in a separate branch too - Git branches (because of their transparency in respect to the tree) should work in a way more amicable to this kind of parallel development than other revision control systems, which seem to already be capable of this kind of thing. I just need to set it up initially, and keep things organized from a branch standpoint, not necessarily a directory standpoint.

And as for the rearranging of sub-projects - after looking around for a while, I think the only way to do it is to create separate branches for each project, then remove the other projects' directories from each branch. I don't know how to separate the branches into separate repositories though... that may be tedious or impossible. I guess it's good that I started in a reasonable pattern.

Git has subprojects that you could look into. IIRC the master can track specific revisions, or tags in the sub repos.

As for N branches in one repo, that's not really what git branches are for. They're more for developing features on top of existing code, repo == full branch, 'git branch' == feature branch. At least, that appears to be the underlying intention in them.

Separating out into separate repos can be done, I don't have the commands on hand, but IIRC you need the SHA1's of every commit for the files under each subdirectory, then you can, I think, git cherry-pick them, one by one, while removing other files you changed in that commit. All in all, I recall it's rather tedious.

linuxfood wrote:At least, that appears to be the underlying intention in them.

Master is also a branch - logically, it should be possible to treat other branches as masters for each subproject. I doubt anyone will mind if I "misuse" the branch concept.

linuxfood wrote:Separating out into separate repos can be done, I don't have the commands on hand, but IIRC you need the SHA1's of every commit for the files under each subdirectory, then you can, I think, git cherry-pick them, one by one, while removing other files you changed in that commit. All in all, I recall it's rather tedious.

It would probably be less tedious to clone the entire repository a few times and specialize each one to have only one subproject, then make sure not to mix them back together. Maybe someone should ask the git maintainers for a branch -> repo conversion feature.

This post might have some information
http://stackoverflow.com/questions/3594 ... repository