concurent build fails in VM guest env but succeeds in host

[ggodw000]

I am using combination of cmake and make utilities to compile large number of code comparable to the size of linux kernel. However it is becoming regular occurrence, when concurrent build is specified i.e. make -j`nproc` it belches out with lot of errors messages if I do in VM guest environment (in this case linux libvirtd/kvm) but same build completes OK in same system that is hosting the VM guest environment but on host O/S. I am not very knowledgeable about compiler architecture specially concurrent builds only have very basic understanding compiler back and front end. Based on the what I am observing any expert share their opinin on this?

[alexfru]

If you don't build concurrently, do you still run into the same problem?

[ggodw000]

If you don't build concurrently, do you still run into the same problem?

that is very iffy and yes and no, meaning it is not consistent.
It is actually llvm toolset from llvm-project. I did notice for non-concurrent build (single thread), llvm project build OK. But there is variant of llvm-project which is part of ROCm AMD which, in single-thread in guest env, also failed and error was same.

I can not sure for certainty on my observation above, as once build working, I sort of moved on.
However guest environment build is bothering enough and definitely worth knowing.

Here is last few lines of build log, notice it says c++ internal compiler error:

[ 50%] Building CXX object tools/llvm-pdbutil/CMakeFiles/llvm-pdbutil.dir/TypeReferenceTracker.cpp.o
[ 50%] Building CXX object tools/llvm-pdbutil/CMakeFiles/llvm-pdbutil.dir/YAMLOutputStyle.cpp.o
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
lib/Target/AMDGPU/AsmParser/CMakeFiles/LLVMAMDGPUAsmParser.dir/build.make:62: recipe for target 'lib/Target/AMDGPU/AsmParser/CMakeFiles/LLVMAMDGPUAsmParser.dir/AMDGPUAsmParser.cpp.o' failed
make[2]: *** [lib/Target/AMDGPU/AsmParser/CMakeFiles/LLVMAMDGPUAsmParser.dir/AMDGPUAsmParser.cpp.o] Error 4
CMakeFiles/Makefile2 recipe for target 'lib/Target/AMDGPU/AsmParser/CMakeFiles/LLVMAMDGPUAsmParser.dir/all' failed
make[1]: *** [lib/Target/AMDGPU/AsmParser/CMakeFiles/LLVMAMDGPUAsmParser.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 50%] Linking CXX static library ../../libLLVMSymbolize.a
[ 50%] Built target llvm-dwarfdump
[ 50%] Built target LLVMSymbolize
[ 50%] Linking CXX executable ../../bin/llvm-pdbutil
[ 50%] Built target llvm-pdbutil
[ 50%] Linking CXX static library ../libLLVMAnalysis.a
[ 50%] Built target LLVMAnalysis
[ 50%] Linking CXX executable ../../bin/obj2yaml
[ 50%] Built target obj2yaml
[ 50%] Linking CXX executable ../../bin/llvm-readobj
[ 50%] Built target llvm-readobj
Makefile:151: recipe for target 'all' failed
make: *** [all] Error 2

[Octocontrabass]

c++: internal compiler error: Killed (program cc1plus)

Something is killing the compiler. Is it running out of memory?

[ggodw000]

not sure, may be that is possibility. the compile was no at least 46GB memory server on host. There were 8 same VM meaning at most 8GB mem allocated to each host.
Does compiling takes that much memory?

[nullplan]

not sure, may be that is possibility. the compile was no at least 46GB memory server on host. There were 8 same VM meaning at most 8GB mem allocated to each host.
Does compiling takes that much memory?

Depends on language and features used. And on the source files, of course. Regrettably, C++ does have the features to make compiling arbitrarily difficult. Templates and constexpr are the two big ones driving memory cost.

[alexfru]

not sure, may be that is possibility. the compile was no at least 46GB memory server on host. There were 8 same VM meaning at most 8GB mem allocated to each host.
Does compiling takes that much memory?

I've once tried building TensorFlow on an ARM board with 3GB of RAM. I had to configure a swap file of like 8 or 16GB to build the monster. And even then I ran out of memory and built the last chunk and linked everything non-concurrently.
So, yeah, some things may be pretty demanding.

[ggodw000]

i am going to try increase the VM memory and see if it works then.

[ggodw000]

it worked wonderfully. increased guest vm memory to 32-64G, and cpu-s to 32 core.