boost installation stalls when using parallel building

[fleason@…]

after doing: sudo port -d upgrade boost build stops here:

darwin.compile.c++ bin.v2/libs/math/build/darwin-4.2.1/debug/assoc_laguerre.o

    "/usr/bin/g++-4.2"  -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -I"bin.v2/libs/math/build/darwin-4.2.1/debug/../src/tr1" -I"." -I"libs/math/src/tr1" -c -o "bin.v2/libs/math/build/darwin-4.2.1/debug/assoc_laguerre.o" "libs/math/build/../src/tr1/assoc_laguerre.cpp"

libs/math/build/../src/tr1/assoc_laguerre.cpp:6:21: warning: bin.v2/libs/math/build/darwin-4.2.1/debug/../src/tr1/pch.hpp.gch: not used because `__OPTIMIZE__' not defined

No error code. Just stops. related processes:

    0  1952  1910   0   0:00.90 ttys000    0:01.77 /usr/bin/tclsh /opt/local/bin/port -d upgrade boost
    0  1972  1952   0   0:00.00 ttys000    0:00.00 sh -c  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_devel_boost/work/boost_1_41_0" && bjam -j5 -d2 --layout=tagged --debug-configuration --ignore-site-config --user-config=user-config.jam --without-python --without-mpi -sBZIP2_INCLUDE=/opt/local/include -sBZIP2_LIBPATH=/opt/local/lib -sEXPAT_INCLUDE=/opt/local/include -sEXPAT_LIBPATH=/opt/local/lib -sZLIB_INCLUDE=/opt/local/include -sZLIB_LIBPATH=/opt/local/lib threading=single,multi variant=debug,release 
    0  1973  1972   0  10:11.50 ttys000   20:06.50 bjam -j5 -d2 --layout=tagged --debug-configuration --ignore-site-config --user-config=user-config.jam --without-python --without-mpi -sBZIP2_INCLUDE=/opt/local/include -sBZIP2_LIBPATH=/opt/local/lib -sEXPAT_INCLUDE=/opt/local/include -sEXPAT_LIBPATH=/opt/local/lib -sZLIB_INCLUDE=/opt/local/include -sZLIB_LIBPATH=/opt/local/lib threading=single,multi variant=debug,release
    0  4202  1973   0   0:00.00 ttys000    0:00.00 /usr/bin/g++-4.2 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -no-cpp-precomp -gdwarf-2 -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -Ibin.v2/libs/math/build/darwin-4.2.1/debug/link-static/../src/tr1 -I. -Ilibs/math/src/tr1 -c -o bin.v2/libs/math/build/darwin-4.2.1/debug/link-static/assoc_laguerre.o libs/math/build/../src/tr1/assoc_laguerre.cpp
    0  4203  4202   0   0:00.00 ttys000    0:00.00 /usr/bin/i686-apple-darwin10-g++-4.2.1 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -no-cpp-precomp -gdwarf-2 -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -Ibin.v2/libs/math/build/darwin-4.2.1/debug/link-static/../src/tr1 -I. -Ilibs/math/src/tr1 -c libs/math/build/../src/tr1/assoc_laguerre.cpp -m64 -o bin.v2/libs/math/build/darwin-4.2.1/debug/link-static/assoc_laguerre.o
    0  4204  4203   0   8:46.09 ttys000   19:49.26 /usr/libexec/gcc/i686-apple-darwin10/4.2.1/cc1plus -quiet -Ibin.v2/libs/math/build/darwin-4.2.1/debug/link-static/../src/tr1 -I. -Ilibs/math/src/tr1 -imultilib x86_64 -D__DYNAMIC__ -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED libs/math/build/../src/tr1/assoc_laguerre.cpp -fPIC -feliminate-unused-debug-symbols -quiet -dumpbase assoc_laguerre.cpp -mmacosx-version-min=10.6 -m64 -mtune=core2 -auxbase-strip bin.v2/libs/math/build/darwin-4.2.1/debug/link-static/assoc_laguerre.o -g -gdwarf-2 -O0 -Wall -Winvalid-pch -ftemplate-depth-128 -fno-inline -D__private_extern__=extern -o /var/tmp//cccm11Bp.s
    0  4216  1973   0   0:00.00 ttys000    0:00.00 /usr/bin/g++-4.2 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -c -o bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/assoc_laguerre.o libs/math/build/../src/tr1/assoc_laguerre.cpp
    0  4217  4216   0   0:00.00 ttys000    0:00.00 /usr/bin/i686-apple-darwin10-g++-4.2.1 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -c libs/math/build/../src/tr1/assoc_laguerre.cpp -m64 -o bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/assoc_laguerre.o
    0  4218  4217   0   8:49.93 ttys000   19:48.88 /usr/libexec/gcc/i686-apple-darwin10/4.2.1/cc1plus -quiet -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -imultilib x86_64 -D__DYNAMIC__ -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 libs/math/build/../src/tr1/assoc_laguerre.cpp -fPIC -feliminate-unused-debug-symbols -quiet -dumpbase assoc_laguerre.cpp -mmacosx-version-min=10.6 -m64 -mtune=core2 -auxbase-strip bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/assoc_laguerre.o -g -gdwarf-2 -O0 -Wall -Winvalid-pch -ftemplate-depth-128 -fno-inline -fPIC -D__private_extern__=extern -o /var/tmp//ccZbqTWU.s
    0  4221  1973   0   0:00.00 ttys000    0:00.00 /usr/bin/g++-4.2 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -c -o bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/assoc_legendre.o libs/math/build/../src/tr1/assoc_legendre.cpp
    0  4222  4221   0   0:00.00 ttys000    0:00.00 /usr/bin/i686-apple-darwin10-g++-4.2.1 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -c libs/math/build/../src/tr1/assoc_legendre.cpp -m64 -o bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/assoc_legendre.o
    0  4223  4222   0   8:38.24 ttys000   19:48.77 /usr/libexec/gcc/i686-apple-darwin10/4.2.1/cc1plus -quiet -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -imultilib x86_64 -D__DYNAMIC__ -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 libs/math/build/../src/tr1/assoc_legendre.cpp -fPIC -feliminate-unused-debug-symbols -quiet -dumpbase assoc_legendre.cpp -mmacosx-version-min=10.6 -m64 -mtune=core2 -auxbase-strip bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/assoc_legendre.o -g -gdwarf-2 -O0 -Wall -Winvalid-pch -ftemplate-depth-128 -fno-inline -fPIC -D__private_extern__=extern -o /var/tmp//cck9Polr.s
    0  4224  1973   0   0:00.00 ttys000    0:00.00 /usr/bin/g++-4.2 -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -DNDEBUG -Ibin.v2/libs/math/build/darwin-4.2.1/release/../src/tr1 -I. -Ilibs/math/src/tr1 -c -o bin.v2/libs/math/build/darwin-4.2.1/release/assoc_laguerre.o libs/math/build/../src/tr1/assoc_laguerre.cpp
    0  4225  4224   0   0:00.00 ttys000    0:00.00 /usr/bin/i686-apple-darwin10-g++-4.2.1 -ftemplate-depth-128 -O3 -finline-functions -Wno-inline -Wall -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -DNDEBUG -Ibin.v2/libs/math/build/darwin-4.2.1/release/../src/tr1 -I. -Ilibs/math/src/tr1 -c libs/math/build/../src/tr1/assoc_laguerre.cpp -m64 -o bin.v2/libs/math/build/darwin-4.2.1/release/assoc_laguerre.o
    0  4226  4225   0   8:44.51 ttys000   19:48.68 /usr/libexec/gcc/i686-apple-darwin10/4.2.1/cc1plus -quiet -Ibin.v2/libs/math/build/darwin-4.2.1/release/../src/tr1 -I. -Ilibs/math/src/tr1 -imultilib x86_64 -D__DYNAMIC__ -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -DNDEBUG libs/math/build/../src/tr1/assoc_laguerre.cpp -fPIC -quiet -dumpbase assoc_laguerre.cpp -mmacosx-version-min=10.6 -m64 -mtune=core2 -auxbase-strip bin.v2/libs/math/build/darwin-4.2.1/release/assoc_laguerre.o -gdwarf-2 -O3 -Wno-inline -Wall -Winvalid-pch -ftemplate-depth-128 -finline-functions -fPIC -D__private_extern__=extern -o /var/tmp//ccuvpJTU.s
    0  4229  1973   0   0:00.00 ttys000    0:00.00 /usr/bin/g++-4.2 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -c -o bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/beta.o libs/math/build/../src/tr1/beta.cpp
    0  4230  4229   0   0:00.00 ttys000    0:00.00 /usr/bin/i686-apple-darwin10-g++-4.2.1 -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -c libs/math/build/../src/tr1/beta.cpp -m64 -o bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/beta.o
    0  4231  4230   0   8:47.86 ttys000   19:48.16 /usr/libexec/gcc/i686-apple-darwin10/4.2.1/cc1plus -quiet -Ibin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/../src/tr1 -I. -Ilibs/math/src/tr1 -imultilib x86_64 -D__DYNAMIC__ -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 libs/math/build/../src/tr1/beta.cpp -fPIC -feliminate-unused-debug-symbols -quiet -dumpbase beta.cpp -mmacosx-version-min=10.6 -m64 -mtune=core2 -auxbase-strip bin.v2/libs/math/build/darwin-4.2.1/debug/threading-multi/beta.o -g -gdwarf-2 -O0 -Wall -Winvalid-pch -ftemplate-depth-128 -fno-inline -fPIC -D__private_extern__=extern -o /var/tmp//cc23bQtj.s

environment: Mac OS X 10.6.2; 2.8 GHz Intel Core i7, 4GB 1067 MHz DDR3

The following installed ports are outdated:
boost                          1.41.0_1 < 1.41.0_2

[steve@…]

STDOUT from update script

[steve@…]

83% CPU on 8 cores!

[steve@…]

Build of 'boost' stalled for 4.5 hours with massive CPU load

This was also on a new i7 iMac running 10.6.2 (is this an i7 issue?).

My weekly 'port-selfupdate+upgrade' script ran as root from launchd at 03:00 this morning.

It uses 'port -v -p -u upgrade outdated' to update the ports.

Found it stalled at 07:30 this morning.

There were 8x 'g++' builds (one for each core?) under the 'bjam' process, each running 'cc1plus', each of which was using 83% of its own CPU!

No further output since 03:05 (according to the log time stamp, and assuming that there wasn't much unbuffered output).

Took samples of the 'bjam', 'g++' and 'cc1plus' processes.

Found this ticket with the same problem.

Killed build with 'sudo killall cc1plus' to get things back to normal.

Summary of stdout from the script (nothing bar the download in stderr):

STDOUT from /opt/local/bin/port-selfupdate+upgrade
Wed Feb  3 03:00:00 GMT 2010
--->  Updating the ports tree
--->  Updating MacPorts base sources using rsync
MacPorts base version 1.8.2 installed,
MacPorts base version 1.8.2 downloaded.
--->  MacPorts base is already the latest version

The ports tree has been updated. To upgrade your installed ports, you should run
  port upgrade outdated
Outdated ports:
apache2                        @2.2.14         www/apache2
bash-completion                @1.1            sysutils/bash-completion
boost                          @1.41.0         devel/boost
cfitsio                        @3.240          science/cfitsio
gmp                            @4.3.2          devel/gmp
graphviz                       @2.26.3         graphics/graphviz
libiconv                       @1.13.1         textproc/libiconv
nmap                           @5.21           net/nmap
wireshark                      @1.2.6          net/wireshark

Upgrading outdated ports...

--->  Computing dependencies for libiconv.
--->  Deactivating libiconv @1.13_0
--->  Computing dependencies for libiconv.
--->  Activating libiconv @1.13.1_0
--->  Computing dependencies for apache2.
--->  Deactivating apache2 @2.2.14_0+darwin+preforkmpm
--->  Computing dependencies for apache2.
--->  Activating apache2 @2.2.14_1+darwin+preforkmpm
--->  Computing dependencies for bash-completion.
--->  Deactivating bash-completion @1.1_3
--->  Computing dependencies for bash-completion.
--->  Activating bash-completion @1.1_4
--->  Computing dependencies for boost.
…

darwin.compile.c++ bin.v2/libs/math/build/darwin-4.2.1/debug/assoc_laguerre.o

    "/usr/bin/g++-4.2"  -ftemplate-depth-128 -O0 -fno-inline -Wall -g -dynamic -no-cpp-precomp -gdwarf-2 -fPIC -Winvalid-pch -DBOOST_ALL_NO_LIB=1 -DBOOST_BUILD_PCH_ENABLED -DBOOST_MATH_TR1_DYN_LINK=1 -I"bin.v2/libs/math/build/darwin-4.2.1/debug/../src/tr1" -I"." -I"libs/math/src/tr1" -c -o "bin.v2/libs/math/build/darwin-4.2.1/debug/assoc_laguerre.o" "libs/math/build/../src/tr1/assoc_laguerre.cpp"

libs/math/build/../src/tr1/assoc_laguerre.cpp:6:21: warning: bin.v2/libs/math/build/darwin-4.2.1/debug/../src/tr1/pch.hpp.gch: not used because `__OPTIMIZE__' not defined

Complete output from 'boost' build is in the attached file, together with a screenshot of Activity Monitor, and samples of the processes (in case that's relevant or helpful).

Hope this helps! Steve

[steve@…]

Cc Me!

[raphael@…]

I had the same problem. After that a sudo port clean boost && sudo port upgrade boost build.jobs=1 worked fine.

So, use_parallel_build no should be added to the boost portfile.

[steve@…]

Thanks, Raphael - much appreciated!

09:03 $ sudo port clean boost && sudo time port -v -u upgrade boost build.jobs=1
...
--->  Deactivating boost @1.41.0_1
--->  Computing dependencies for boost.
--->  Installing boost @1.41.0_2
--->  Activating boost @1.41.0_2
--->  Cleaning boost
--->  Removing build directory for boost
--->  Uninstalling boost @1.41.0_1
--->  Uninstall is removing boost from the port registry.
     3157.46 real      2515.38 user       371.64 sys

Successfully installed 53 minutes later. So your patch would work.

Best wishes, Steve

[jmr@…]

Please remember to cc the maintainer.

[nox@…]

r63389

[fleason@…]

Applied changes to boost Portfile as per r63389. boost installation still stalls. There were multiple processes left open. port upgrade boost build.jobs=1 seems to work fine.

environment: Mac OS X 10.6.2; 2.8 GHz Intel Core i7, 4GB 1067 MHz DDR3

[yves@…]

Cc Me!

[yves@…]

Replying to fleason@…:

Applied changes to boost Portfile as per r63389. boost installation still stalls. There were multiple processes left open. port upgrade boost build.jobs=1 seems to work fine.

environment: Mac OS X 10.6.2; 2.8 GHz Intel Core i7, 4GB 1067 MHz DDR3

Does not work for me, hangs on forever (10.6.2, 8 cores)

[yves@…]

Gandalf:~ yves$ ps acx | grep \+\+
73815 s000  S      0:00.00 g++-4.2
73818 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
73820 s000  S      0:00.00 g++-4.2
73821 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
73823 s000  S      0:00.00 g++-4.2
73824 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
73832 s000  S      0:00.00 g++-4.2
73835 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75711 s000  S      0:00.00 g++-4.2
75712 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75717 s000  S      0:00.00 g++-4.2
75719 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75721 s000  S      0:00.00 g++-4.2
75722 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75724 s000  S      0:00.00 g++-4.2
75725 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75732 s000  S      0:00.00 g++-4.2
75733 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75740 s000  S      0:00.00 g++-4.2
75741 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75747 s000  S      0:00.01 g++-4.2
75748 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75753 s000  S      0:00.00 g++-4.2
75754 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75845 s000  S      0:00.00 g++-4.2
75846 s000  S      0:00.00 g++-4.2
75847 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75848 s000  S      0:00.00 g++-4.2
75850 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75852 s000  S      0:00.00 g++-4.2
75853 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75854 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75857 s000  S      0:00.00 g++-4.2
75858 s000  S      0:00.00 g++-4.2
75859 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75861 s000  S      0:00.00 g++-4.2
75862 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75864 s000  S      0:00.00 g++-4.2
75865 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75866 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1
75886 s000  S      0:00.00 g++-4.2
75887 s000  S      0:00.00 i686-apple-darwin10-g++-4.2.1

[raphael@…]

The problem is that use_parallel_build no obviously does not set ${build.jobs} to 1 and the pre-build phase of the portfile still calls bjam with -j number of cores. So, the pre-build phase should be removed from the port. A patch is attached.

Nevertheless, a

sudo port clean boost
sudo port upgrade boost build.jobs=1

should still work.

[nox@…]

r63530