atlas @3.9.37 build failure - hangs with xztfc using 100% CPU

[alan@…]

I'm running XCode 4.0.2 on OSX 10.6.7. I'm trying to install gimp-app, which depends on atlas somewhere in its deps tree. During the build, it runs the command "xztfc" which just hangs forever.

Here is what it's doing:

$ ps aux|grep xzt |grep -v grep
root     44048 100.0  0.3  2463808  28564 s000  R+    4:24PM   1:07.08 /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/tune/blas/gemm/xztfc
root     44047   0.0  0.0  2435548    740 s000  S+    4:24PM   0:00.00 /bin/sh /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/bin/ATLrun.sh /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/tune/blas/gemm xztfc

here is the build log:

$ sudo port -v install atlas
--->  Computing dependencies for atlas.
--->  Building atlas
/usr/bin/make -f Make.top build
cd bin/ ; /usr/bin/make xatlas_build
make[2]: `xatlas_build' is up to date.
cd bin/ ; ./xatlas_build -1 0 -a 1 -l 1
make[2]: *** No rule to make target `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/..//CONFIG/ARCHS/UNKNOWNx8664SSE3.tgz', needed by `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/ARCHS/UNKNOWNx8664SSE3'.  Stop.
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/src/auxil ; make -s ATL_buildinfo.o
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/src/auxil ; make lib
make[3]: Nothing to be done for `lib'.
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/src/testing ; make lib
make -j 8 dlib.grd
make[4]: `dlib.grd' is up to date.
make -j 8 slib.grd
make[4]: `slib.grd' is up to date.
make -j 8 zlib.grd
make[4]: `zlib.grd' is up to date.
make -j 8 clib.grd
make[4]: `clib.grd' is up to date.
make INSTALL_LOG/L1CacheSize
make[3]: `INSTALL_LOG/L1CacheSize' is up to date.
make INSTALL_LOG/sMULADD pre=s
make[3]: `INSTALL_LOG/sMULADD' is up to date.
make INSTALL_LOG/dMULADD pre=d
make[3]: `INSTALL_LOG/dMULADD' is up to date.
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/tune/sysinfo ; make RunLamch
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/bin/ATLrun.sh /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/tune/sysinfo xemit_lamch /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/include
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/tune/blas/gemm ; make dinstall
make[3]: `dinstall' is up to date.
make[2]: `INSTALL_LOG/atlas_dtrsmXover.h' is up to date.
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/interfaces/blas/C/src ; make dlib
make -j 8 dlvl3.grd
make[4]: `dlvl3.grd' is up to date.
make -j 8 dlvl2.grd
make[4]: `dlvl2.grd' is up to date.
make -j 8 dlvl1.grd
make[4]: `dlvl1.grd' is up to date.
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/tune/blas/gemm ; make sinstall
make[3]: `sinstall' is up to date.
make[2]: `INSTALL_LOG/atlas_strsmXover.h' is up to date.
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/interfaces/blas/C/src ; make slib
make -j 8 slvl3.grd
make[4]: `slvl3.grd' is up to date.
make -j 8 slvl2.grd
make[4]: `slvl2.grd' is up to date.
make -j 8 slvl1.grd
make[4]: `slvl1.grd' is up to date.
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build/tune/blas/gemm ; make zinstall
make[3]: `zinstall' is up to date.

[compconsultant@…]

Same issue here. On new MBP 2011 i7.

[jmroot (Joshua Root)]

Please remember to cc the maintainer.

[alan@…]

Replying to compconsultant@…:

Same issue here. On new MBP 2011 i7.

I have the same configuration. MacBookPro8,2 Core i7, 8 GB.

[Veence (Vincent)]

That's no wonder. The 3.9.37 has no support for the newest Sandy bridge processors. That support comes in 3.9.38, which is buggy.

I'm working on the 3.9.40 and should have something usable tonight or tomorrow morning. If you are in a hurry, and don't need to build a universal x86-64/i386 build, but only a 64-bit Intel version, I can give you the temporary Portfile and the patches that you can use in a private repository.

3.9.40 is also buggy, but can more or less be fixed. The problem is that the maintainer/developer of Atlas does not seem to have access to a Mac, so new versions are never tested on OS X.

[compconsultant@…]

I should add this.... Not sure what atlas even is, but, it was required for py27-game I believe. Anyway, I let it go this time, for a few HOURS, and, it DID eventually install. The, the python application that used the py27-game port worked fine.

So, Atlas DOES install, it just takes FOREVER. So, the OP should retry and just let it sit. Mine did the same thing, showed 100% cpu, fan reved up, etc. HOWEVR, I started to notice the program using 100% would change every 5-20 minutes, so, just let it go over night.

There are a few other apps out there with the same symptoms, that run super slow on the new I7. Another example is VirtualBox, similar behavior. In their case, rebooting into 32 bit mode solves the issue. Not sure if the install would be any faster here.

[Veence (Vincent)]

ATLAS compiles pretty fast in 64-bit mode. It takes something like 15 min on my 3-year old MacBook Core2Duo. But, you're right, if you build the 32-bit version (in your case, that would mean universal), it can be very slow. That's because, in 32-bit mode, Atlas explores a lot of different parameters for each routine it compiles (eg. use of X87 vs SSE/SSE2/SSE3, spacing of data to optimize cache fetch/prefetch, and so on). Unfortunately, since about every routine is already written in assembler, there is little gain to be expected from one processor to the other. That explains why it takes, in 32-bit mode, about the same time with a basic Core processor than with a new Sandy bridge CPU.

[alan@…]

atlas main.log file

[alan@…]

I am now getting a different build error on atlas. I have attached the main.log file to this ticket:

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/main.log

Here are the last few lines that "sudo port -v install atlas" give me at the command line:

   BEGIN STAGE 2-2-4: L3BLAS TUNE at 13:52
make -f Makefile INSTALL_LOG/atlas_strsmXover.h pre=s 2>&1 | ./xatlas_tee INSTALL_LOG/sL3TUNE.LOG
make -f Makefile scblaslib 2>&1 | ./xatlas_tee INSTALL_LOG/sL3TUNE.LOG
   DONE  STAGE 2-2-4 at 13:52


   BEGIN STAGE 2-2-5: GEMV TUNE at 13:52
      gemvN : main kernel 900002:smvn_sse.c written by R. Clint Whaley
            mu=16, nu=4, using 36864 Cache Elements
              Performance = 7255.90 (32.48 of copy matmul, 329.81 of clock)
      gemvT : main kernel 900002:smvt_sse.c written by R. Clint Whaley
            mu=16, nu=4, using 49152 Cache Elements
              Performance = 7399.40 (33.12 of copy matmul, 336.34 of clock)
   DONE  STAGE 2-2-5 at 13:52


   BEGIN STAGE 2-2-6: GER TUNE at 13:52
      ger : main kernel 900002:sr2_sse.c written by R. Clint Whaley
            mu=16, nu=1, using 53248 Cache Elements
              Performance = 9333.41 (41.78 of copy matmul, 424.25 of clock)
   DONE  STAGE 2-2-6 at 13:52
DONE  STAGE 2-0-0 at 13:52


BEGIN STAGE 2-3-0: TUNING PREC='z' (precision 3 of 4) at 13:52


   BEGIN STAGE 2-3-1: BUILDING BLOCK MATMUL TUNE at 13:52
      The best matmul kernel was CASES/ATL_dmm6x1x72_sse2.c, NB=60, written by R. Clint Whaley
      Performance: 11168.32MFLOPS (507.65 percent of of detected clock rate)
        (Gen case got 5714.25MFLOPS)
      NCgemmNN : muladd=0, lat=1, pf=1, nb=40, mu=4, nu=2 ku=40,
                 ForceFetch=0, ifetch=6 nfetch=1
                 Performance = 5808.21 (52.01 of copy matmul, 264.01 of clock)
      NCgemmNT : muladd=0, lat=1, pf=1, nb=40, mu=4, nu=2 ku=40,
                 ForceFetch=0, ifetch=6 nfetch=1
                 Performance = 5600.93 (50.15 of copy matmul, 254.59 of clock)
      NCgemmTN : muladd=0, lat=1, pf=1, nb=40, mu=4, nu=2 ku=4,
                 ForceFetch=0, ifetch=6 nfetch=1
                 Performance = 5679.71 (50.86 of copy matmul, 258.17 of clock)
      NCgemmTT : muladd=0, lat=1, pf=1, nb=40, mu=4, nu=2 ku=40,
                 ForceFetch=0, ifetch=6 nfetch=1
                 Performance = 5677.05 (50.83 of copy matmul, 258.05 of clock)
make -f Makefile MMinstall pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/sL3TUNE.LOG
   DONE  STAGE 2-3-1 at 13:52


   BEGIN STAGE 2-3-2: CacheEdge DETECTION at 13:52
   DONE  STAGE 2-3-2 at 13:52


   BEGIN STAGE 2-3-3: SMALL/LARGE CROSSOVER at 13:52


   STAGE 2-3-3: COPY/NO-COPY CROSSOVER DETECTION
make -f Makefile INSTALL_LOG/zXover.h pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMCROSSOVER.LOG
      done.
   DONE  STAGE 2-3-3 at 13:52


   BEGIN STAGE 2-3-4: L3BLAS TUNE at 13:52
make -f Makefile Il3lib pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zL3TUNE.LOG
make -f Makefile zcblaslib 2>&1 | ./xatlas_tee INSTALL_LOG/zL3TUNE.LOG
   DONE  STAGE 2-3-4 at 13:52


   BEGIN STAGE 2-3-5: GEMV TUNE at 13:52
make -f Makefile INSTALL_LOG/zMVNK.sum pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMVNTUNE.LOG
make[1]: *** [build] Error 255
make: *** [build] Error 2
shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/work/atlas-3.9.37/build" && /usr/bin/make build " returned error 2
Error: Target org.macports.build returned: shell command failed (see log for details)
Warning: the following items did not execute (for atlas): org.macports.activate org.macports.build org.macports.destroot org.macports.install
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#pro

[Veence (Vincent)]

Please, for the sake of clarity, leave this bug closed and file a new one with your problem. Otherwise it becomes impossible to track anything. Thanks a lot.