Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#30280 closed defect (fixed)

Atlas @3.9.37 +GCC44 : STAGE 2-3-2: CacheEdge DETECTION FAILED

Reported by: pierre@… Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version: 2.0.99
Keywords: lion Cc: jameskyle@…, faisal.moledina@…, marc.schlaich@…, wojtekwn@…, rabbielvis@…, randompeach@…, ecbrown (Eric Brown), talezshin@…, jacobu@…, st.hennig@…, rex4539 (Dimitris Apostolou), ith140@…, MasakiOita, stromnov (Andrew Stromnov), cod3monk (Julian), axeljaeger@…, rsachdev@…, dmirkitanov@…, captainproton1971 (Captain Proton), nicholas.north@…, m.haller@…, kwasir@…, cdeil (Christoph Deil), mb86@…, melanochaitus@…, michelle.lynn.gill@…, gjznituv@…, sperka@…, patrice.kouame@…, sander@…, danmichaelo+macports@…, muchatel@…, fyu@…, jeff.wilson@…, geoffgrimwood, maurice@…, steve@…, lawrence.ong@…, jowens@…, yamada.manabu.1207@…, tzonghao@…, adolfo.benedetti@…, bonoba@…, a.y.harano@…, smcavoy@…, christoph.jacob@…, tonior@…, fyu+macports@…, thomas@…, axel.kirch@…, fleason (Fred Leason), derekathomas@…, lukas.reichlin@…
Port: atlas

Description

Fresh SVN install, Xcode 4.2 on Mac OS 10.7 Lion on MPB 13" Mid 2010. Atlas does not build (see log attached).

   DONE  STAGE 2-3-1 at 17:39


   BEGIN STAGE 2-3-2: CacheEdge DETECTION at 17:39
make -f Makefile INSTALL_LOG/atlas_zdNKB.h pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMCACHEEDGE.LOG
make[1]: *** [build] Error 255
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_math_atlas/atlas/work/atlas-3.9.37/build'
make: *** [build] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_math_atlas/atlas/work/atlas-3.9.37/build'
shell command " cd "/opt/local/var/macports/build/_opt_mports_trunk_dports_math_atlas/atlas/work/atlas-3.9.37/build" && /usr/bin/make -w build " returned error 2
Error: Target org.macports.build returned: shell command failed (see log for details)
DEBUG: Backtrace: shell command failed (see log for details)
    while executing
"command_exec build"
    (procedure "portbuild::build_main" line 8)
    invoked from within
"$procedure $targetname"
Warning: the following items did not execute (for atlas): org.macports.activate org.macports.build org.macports.destroot org.macports.install
Log for atlas is at: /opt/local/var/macports/logs/_opt_mports_trunk_dports_math_atlas/atlas/main.log
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#project.tickets>

Attachments (14)

main.log (2.3 MB) - added by pierre@… 10 years ago.
log
zMMCACHEEDGE.LOG (25.4 KB) - added by faisal.moledina@… 10 years ago.
The relevant zMMCACHEEDGE.LOG
atlas_10.7_gcc45.tgz (6.5 KB) - added by Veence (Vincent) 10 years ago.
Portfile and patch files for 3.9.46/3.3.1
py27-numpy+scipy-tests.txt (27.5 KB) - added by faisal.moledina@… 10 years ago.
numpy.test() and scipy.test() for py27 ports + vince' atlas fix
py26-numpy-test-lion.txt (17.9 KB) - added by michelle.lynn.gill@… 10 years ago.
py26-scipy-test-lion.txt.txt (6.9 KB) - added by michelle.lynn.gill@… 10 years ago.
py26-scipy-test-lion-mp1.9.2.txt (7.9 KB) - added by michelle.lynn.gill@… 10 years ago.
py26-scipy-test-snowleo-mp1.9.2 (7.6 KB) - added by michelle.lynn.gill@… 10 years ago.
main.2.log (70.2 KB) - added by wojtekwn@… 10 years ago.
log from iMac with Core i7
main_adolfo.benedetti_31.07.11_gcc45_lion.log (5.5 MB) - added by adolfo.benedetti@… 10 years ago.
main.3.log (23.3 KB) - added by christoph.jacob@… 10 years ago.
main.log for atlas-3.9.46 +gcc46 on Core i7
sysctl.out (1.5 KB) - added by christoph.jacob@… 10 years ago.
Output of sysctl -a | grep hw on my 2011 MBP
Portfile (11.7 KB) - added by Veence (Vincent) 10 years ago.
Patched portfile, should work with newest Sandy bridge processors and gcc45
atlas_3_9_46.tgz (7.0 KB) - added by Veence (Vincent) 10 years ago.
Updated atlas subdirectory containing Portfile and patches. Should work on any machine with gcc44 or 45.

Change History (163)

Changed 10 years ago by pierre@…

Attachment: main.log added

log

Changed 10 years ago by faisal.moledina@…

Attachment: zMMCACHEEDGE.LOG added

The relevant zMMCACHEEDGE.LOG

comment:1 Changed 10 years ago by faisal.moledina@…

In /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/bin/INSTALL_LOG. The file zMMCACHEEDGE.LOG ends with the following error.

/usr/bin/ranlib: file: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/lib/libatlas.a(ATL_dtrmv.o) has no symbols
/usr/bin/ranlib: file: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/lib/libatlas.a(ATL_strmv.o) has no symbols
ar: fatal error in /usr/bin/ranlib
make[6]: *** [dlib.grd] Error 1
make[6]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/src/blas/gemm'
make[5]: *** [dlib] Error 2
make[5]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/src/blas/gemm'
make[4]: *** [dmmlib] Error 2
make[4]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/tune/blas/gemm'
make[3]: *** [res/atlas_zdNKB.h] Error 2
make[3]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/tune/blas/gemm'
make[2]: *** [/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/tune/blas/gemm/res/atlas_zdNKB.h] Error 2
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_atlas/atlas/work/atlas-3.9.37/build/bin'

comment:2 Changed 10 years ago by faisal.moledina@…

Cc: faisal.moledina@… added

Cc Me!

comment:3 Changed 10 years ago by marc.schlaich@…

Cc: marc.schlaich@… added

Cc Me!

comment:4 Changed 10 years ago by carbncl@…

Cc: carbncl@… added

Cc Me!

comment:5 Changed 10 years ago by markus.doits@…

Cc: markus.doits@… added

Cc Me!

comment:6 Changed 10 years ago by wojtekwn@…

Cc: wojtekwn@… added

Cc Me!

comment:7 Changed 10 years ago by moninec@…

Cc: moninec@… added

Cc Me!

comment:8 Changed 10 years ago by rabbielvis@…

Cc: rabbielvis@… added

Cc Me!

comment:9 Changed 10 years ago by randompeach@…

Cc: randompeach@… added

Cc Me!

comment:10 Changed 10 years ago by randompeach@…

Cc: randompeach@… removed

Cc Me!

comment:11 Changed 10 years ago by randompeach@…

Cc: randompeach@… added

Cc Me!

comment:12 Changed 10 years ago by ryandesign (Ryan Schmidt)

Cc: eric.c.brown@… shengcer@… talezshin@… added

Has duplicate #30296.

comment:13 Changed 10 years ago by jacobu@…

Cc: jacobu@… added

Cc Me!

comment:14 Changed 10 years ago by st.hennig@…

Cc: st.hennig@… added

Cc Me!

comment:15 Changed 10 years ago by rex4539 (Dimitris Apostolou)

Cc: rex_4539@… added

Cc Me!

comment:16 Changed 10 years ago by ith140@…

Cc: ith140@… added

Cc Me!

comment:17 Changed 10 years ago by MasakiOita

Cc: epimetheus314@… added

Cc Me!

comment:18 Changed 10 years ago by stromnov (Andrew Stromnov)

Cc: stromnov@… added

Cc Me!

comment:19 Changed 10 years ago by cod3monk (Julian)

Cc: codemonk@… added

Cc Me!

comment:20 Changed 10 years ago by axeljaeger@…

CC Me!

comment:21 Changed 10 years ago by axeljaeger@…

Cc: axeljaeger@… added

Cc Me!

comment:22 Changed 10 years ago by rsachdev@…

Cc: rsachdev@… added

Cc Me!

comment:23 Changed 10 years ago by dmirkitanov@…

Cc Me!

comment:24 Changed 10 years ago by dmirkitanov@…

Cc: dmirkitanov@… added

Cc Me!

comment:25 Changed 10 years ago by shengcer@…

Cc: shengcer@… removed

Cc Me!

comment:26 Changed 10 years ago by captainproton1971 (Captain Proton)

Cc: captainproton1971@… added

Cc Me!

comment:27 Changed 10 years ago by nicholas.north@…

Cc: nicholas.north@… added

Cc Me!

comment:28 Changed 10 years ago by m.haller@…

Cc: m.haller@… added

Cc Me!

comment:29 Changed 10 years ago by kwasir@…

Cc: kwasir@… added

Cc Me!

comment:30 Changed 10 years ago by cdeil (Christoph Deil)

Cc: Deil.Christoph@… added

Cc Me!

comment:31 Changed 10 years ago by mb86@…

Cc: mb86@… added

Cc Me!

comment:32 Changed 10 years ago by melanochaitus@…

Cc: melanochaitus@… added

Cc Me!

comment:33 Changed 10 years ago by michelle.lynn.gill@…

Cc: michelle.lynn.gill@… added

Cc Me!

comment:34 Changed 10 years ago by raimue (Rainer Müller)

Keywords: lion added

comment:35 Changed 10 years ago by gjznituv@…

Cc: gjznituv@… added

Cc Me!

comment:36 Changed 10 years ago by sperka@…

Cc: sperka@… added

Cc Me!

comment:37 Changed 10 years ago by patrice.kouame@…

Cc: patrice.kouame@… added

Cc Me!

comment:38 Changed 10 years ago by patrice.kouame@…

I have some relevant attachments from unsuccessful build/install @

#30335

comment:39 Changed 10 years ago by Veence (Vincent)

I am working on the newest version of Atlas (3.9.46) with lapack 3.3.1 and a mix from clang/gfortran. I'll keep you posted.

comment:40 Changed 10 years ago by rui.pereira@…

Cc: rui.pereira@… added

Cc Me!

comment:41 in reply to:  39 Changed 10 years ago by patrice.kouame@…

Replying to vince@…:

I am working on the newest version of Atlas (3.9.46) with lapack 3.3.1 and a mix from clang/gfortran. I'll keep you posted.

FYI... I just tried building (standalone from website) 3.9.45 on my system with Xcode 4.2 (latest beta4) installed and it also fails...Same CacheEdge Detection issue.

my system gcc is currently:

gcc -v
Using built-in specs.
Target: i686-apple-darwin11
Configured with: /private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/src/configure --disable-checking
 --enable-werror --prefix=/Developer/usr/llvm-gcc-4.2 --mandir=/share/man --enable-languages=c,objc,c++,obj-c++ 
--program-prefix=llvm- --program-transform-name=/^[cg][^.-]*$/s/$/-4.2/ 
--with-slibdir=/usr/lib --build=i686-apple-darwin11 --enable-llvm=/private/var/tmp/llvmgcc42/llvmgcc42-2335.15~108/dst-llvmCore/Developer/usr/local 
--program-prefix=i686-apple-darwin11- --host=x86_64-apple-darwin11 
--target=i686-apple-darwin11 --with-gxx-include-dir=/usr/include/c++/4.2.1
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)

NOTE:

I configure with --nof77

I have zipped logs still available if it helps...

comment:42 Changed 10 years ago by sander@…

Cc: sander@… added

Cc Me!

comment:43 Changed 10 years ago by danmichaelo+macports@…

Cc: danmichaelo+macports@… added

Cc Me!

comment:44 Changed 10 years ago by muchatel@…

Cc: muchatel@… added

Cc Me!

comment:45 Changed 10 years ago by fyu@…

Cc: fyu@… added

Cc Me!

comment:46 Changed 10 years ago by jeff.wilson@…

Cc: jeff.wilson@… added

Cc Me!

comment:47 Changed 10 years ago by geoffgrimwood

Cc: geoff@… added

Cc Me!

comment:48 Changed 10 years ago by gaochangjian@…

Cc: gaochangjian@… added

Cc Me!

comment:49 Changed 10 years ago by Veence (Vincent)

I am making significant progress, but since compiling Atlas on my old MacBook (late 2008) takes more than 5 hours, it is painful. I have no cache edge detection problems. The biggest hurdle I've encountered so far is a bug in ar on Lion: "ar r" crashes while calling libtool. I've written a little sh script to use libtool directly instead, as a workaround, but I expect the final archive to be unusable (libtool does not replace .o in the archive like ar r does. It just simply adds them warning of a duplicate; maybe I should call ranlib just after). I have filed a radar about this.

There are some other issues like some non-void functions returning no value that causes error with clang, assembler instructions lacking size specification, etc. I'm going to forwards them upstream to Clint.

comment:50 Changed 10 years ago by maurice@…

Cc Me!

comment:51 Changed 10 years ago by maurice@…

Cc: maurice@… added

Cc Me!

comment:52 Changed 10 years ago by steve@…

Cc: steve@… added

Cc Me!

comment:53 Changed 10 years ago by lawrence.ong@…

Cc: lawrence.ong@… added

Cc Me!

comment:54 Changed 10 years ago by hans.vonkaraman@…

Cc Me!

comment:55 Changed 10 years ago by adolfo.benedetti@…

Cc Me!

comment:56 Changed 10 years ago by Veence (Vincent)

Ok. The situation is thus: Atlas *compiles* all right. But, on 10.7, ar is buggy, so it does not link (fatal error in ar). I have written a shell script that simulates 'ar r' by calling libtool directly, but the shell is yet too primitive and the dylib generation at the end fails because of duplicate symbols. I am going to try to modify the script so that it emulates perfectly 'ar r' (replacing already stored .o instead of storing everything blindly). I'll keep you posted.

comment:57 Changed 10 years ago by Veence (Vincent)

Good. I have a 10.7 clean build with the latest version 3.9.46/lapack 3.3.1 and gcc45. I'll post the updated Portfile and patch files later today.

comment:58 Changed 10 years ago by jowens@…

Cc: jowens@… added

Cc Me!

Changed 10 years ago by Veence (Vincent)

Attachment: atlas_10.7_gcc45.tgz added

Portfile and patch files for 3.9.46/3.3.1

comment:59 Changed 10 years ago by Veence (Vincent)

So, I append a .tgz file containing the Portfile and all the patch files. Since this is highly experimental, it is given as is (not in the form of a diff), without any guarantee, and I strongly advise you to unpack it in a private section of your ports tree (see Macports manual to figure out how to setup one). If you feel adventurous, you can extract it directly in the atlas repertory of the main tree: You can always revert to previous state by wiping out everything and syncing again.

It should work on all MacOS versions with gcc45 in 64-bit mode. All other combinations are not yet tested. I've an old MacMini CoreDuo/10.6 machine on which I should test building it tonight, and a G5 server too.

Support for clang is almost here too, but since clang is stricter than gcc, there are some extra patch files to be generated. Besides, building with clang takes hours, whereas building with gcc is a matter of minutes. I don't know why.

Good luck, and please tell me if you succeed or not.

comment:60 Changed 10 years ago by Veence (Vincent)

Some additional elements I forgot:

  1. When I say "wiping out everything", I meant in the atlas directory, of course;
  2. It is normal, on Lion, to get ar warnings about not found object files. This is part of the workaround.

comment:61 in reply to:  59 Changed 10 years ago by carbncl@…

Replying to vince@…:

Good luck, and please tell me if you succeed or not.

Strangely I did not need atlas anymore, it might have been removed of dependencies of <no idea> package. Anyway, just tried to build it with your attached files, works perfectly, good job! :)

comment:62 Changed 10 years ago by Veence (Vincent)

Thanks! ;) I am currently building it the CoreDuo/10.6 macmini and the G5/10.5 server. We'll see in a few hours if it also succeed on these configurations.

comment:63 in reply to:  62 Changed 10 years ago by carbncl@…

Replying to vince@…:

Thanks! ;) I am currently building it the CoreDuo/10.6 macmini and the G5/10.5 server. We'll see in a few hours if it also succeed on these configurations.

You're welcome :) FYI, I built it on a Core 2 duo MBP / 10.7.

comment:64 Changed 10 years ago by Veence (Vincent)

Great. That's actually almost the same configuration as mine (MB Core 2 duo late 2008). I wonder what will come out the newer machines (various Core i3/5/7). This version of Atlas should be able to use the latest AVX instructions set, too.

comment:65 Changed 10 years ago by Veence (Vincent)

The same ar bug appears to be present on the 32-bit SL build. I'll to modify the port file accordingly.

comment:66 Changed 10 years ago by faisal.moledina@…

@vince: builds perfectly for me. Same configuration, MB C2D late 2008. On my way to building NumPy now. Thanks.

comment:67 Changed 10 years ago by Veence (Vincent)

@faisal: cool. Glad it works. gcc45 seems not to be usable on G4/G5: internal compiler error on some file. I'll have to disable the option for those processors.

comment:68 Changed 10 years ago by jowens@…

@vince: Also works fine for me on a newer MBP (Core i7). Woo! Thanks Vince, I know that was a lot of work.

Took under an hour to compile, for the record.

comment:69 Changed 10 years ago by Veence (Vincent)

Nice! Since you have a powerful machine, maybe I'll ask your help to build the future clang/dragonegg option, if you don't mind. Meanwhile, enjoy, and thanks again for your message.

comment:70 in reply to:  64 Changed 10 years ago by michelle.lynn.gill@…

Replying to vince@…:

Great. That's actually almost the same configuration as mine (MB Core 2 duo late 2008). I wonder what will come out the newer machines (various Core i3/5/7). This version of Atlas should be able to use the latest AVX instructions set, too.

@vince: I just built your version of atlas using gcc44 in Lion on my MBP mid2010, which has an i5 processor.

Atlas built in around 10 minutes for me. I assume this was so fast because the tests atlas performs during normal installation have been disabled?

Also, I've been able to build py26-numpy, py26-scipy, and py26-cvxopt against your version of atlas. Fantastic! Thanks!

comment:71 Changed 10 years ago by Veence (Vincent)

Great to know it works with gcc44 too on a "modern" processor.

I have no real answer to why it compiles so quickly. I will ask upstream. Maybe some configurations are pre-recorded somewhere, recognized, and all the lengthy test phase is skipped? Right now, I have no better hypothesis.

Please run the scipy/numpy tests, tell me if it works properly.

Changed 10 years ago by faisal.moledina@…

Attachment: py27-numpy+scipy-tests.txt added

numpy.test() and scipy.test() for py27 ports + vince' atlas fix

comment:72 Changed 10 years ago by faisal.moledina@…

Just attached the output of numpy.test() and scipy.test() after building py27-numpy and py27-scipy agains the fixed atlas port. Some failures; not sure if they're atlas related or if it's just a combination of new NumPy version + 10.7 issues.

comment:73 Changed 10 years ago by Veence (Vincent)

Hmmm… The scipy errors look suspicious. I'll try myself tomorrow. Thanks for your time.

Changed 10 years ago by michelle.lynn.gill@…

Attachment: py26-numpy-test-lion.txt added

Changed 10 years ago by michelle.lynn.gill@…

comment:74 in reply to:  73 Changed 10 years ago by michelle.lynn.gill@…

Replying to vince@…:

Hmmm… The scipy errors look suspicious. I'll try myself tomorrow. Thanks for your time.

Just attached the results of my tests.

I had some failures as well using py26-numpy and py26-scipy compiled on Lion against the experimental atlas. I have my old MacPorts library (compiled on Snow Leopard) at home and I can repeat the tests the same hardware using the old MacPorts install with Lion and then with the old MacPorts install and a clone of my old (Snow Leopard) system.

comment:75 Changed 10 years ago by Veence (Vincent)

I don't think the failures you have are really caused by atlas, so I am not too much worried. Numpy failures seem to be related to borderline computations involving infinite and NaN, these are not caused by atlas and anyhow not worrisome. The scipy failure does not appear to be caused by atlas either, but, what is a bit strange, is that you do not get the same errors as faisal… I'll try on my own, see how it turns out, and keep you posted. V.

comment:76 Changed 10 years ago by talezshin@…

Mac Mini 2010 (not server) which has almost identical configuration with MBP 2008 late confirmed successful on atlas. Now working on octave!

comment:77 Changed 10 years ago by brandon.zacharie@…

Cc: brandon.zacharie@… added

Cc Me!

comment:78 Changed 10 years ago by yamada.manabu.1207@…

Cc: yamada.manabu.1207@… added

Cc Me!

comment:79 Changed 10 years ago by moninec@…

Cc: moninec@… removed

Cc Me!

comment:80 Changed 10 years ago by moninec@…

Cc: moninec@… added

Cc Me!

comment:81 Changed 10 years ago by moninec@…

Cc: moninec@… removed

Cc Me!

Changed 10 years ago by michelle.lynn.gill@…

Changed 10 years ago by michelle.lynn.gill@…

comment:82 Changed 10 years ago by michelle.lynn.gill@…

Above I have attached two more results from scipy.test() on my MBP X,2. The first was performed using Mac OS X 10.7 with scipy/atlas from the most recent version of my MacPorts library that was compiled under Mac OS X 10.6.8. (This version of the library utilized MacPorts 1.9.2 since I waited to upgrade to MacPorts 2.0 until I installed Lion.) The second version utilizes the old version of the library under Snow Leopard (Mac OS X 10.6.8).

Based on all three scipy tests (the third being the MacPorts library I compiled under Lion using Vince's version of atlas), the same test fails every time--"test_morestats.TestAnderson". I believe this suggests that, at least in my case, any issues encountered are not related to either Lion or Vince's atlas compilation.

Identical hardware was used for the tests and for the compilation of both versions of the MacPorts library.

If you'd like me to perform any other tests on this system, let me know.

comment:83 Changed 10 years ago by michelle.lynn.gill@…

Oops, that should say "MBP 6,2" not "MBP X,2"

comment:84 in reply to:  59 Changed 10 years ago by patrice.kouame@…

Replying to vince@…:

So, I append a .tgz file containing the Portfile and all the patch files. Since this is highly experimental, it is given as is (not in the form of a diff), without any guarantee, and I strongly advise you to unpack it in a private section of your ports tree (see Macports manual to figure out how to setup one). If you feel adventurous, you can extract it directly in the atlas repertory of the main tree: You can always revert to previous state by wiping out everything and syncing again.

It should work on all MacOS versions with gcc45 in 64-bit mode. All other combinations are not yet tested. I've an old MacMini CoreDuo/10.6 machine on which I should test building it tonight, and a G5 server too.

Support for clang is almost here too, but since clang is stricter than gcc, there are some extra patch files to be generated. Besides, building with clang takes hours, whereas building with gcc is a matter of minutes. I don't know why.

Good luck, and please tell me if you succeed or not.

V! Thank you so much for this prompt fix!

I built against gcc44 and gcc45 on Lion with the latest Xcode 4.2 (from the bleeding edge iOS 5 beta 4) with great success. All in a local ports repository as you suggested. I tested meld with all the python dependencies and it all works fine.

By the way the builds on my box take about 15 minutes. I have a Mac Pro dual quad-core Xeon. I know I'm lucky. If you need help with your long compiles/build just ask...you have my email...

Also decided no to push my luck with gcc46 and clang or llvm 3.0. Will wait for your official distros.

I think your ar workaround may give me clues to other stuff that currently breaks on Lion. You filed a radar bug, may I have the number?

Thanks again, Patrice

comment:85 Changed 10 years ago by Veence (Vincent)

You're welcome. Thanks for testing with gcc44 and 45, and for the proposal. I will probably add a dragonegg port soon, so that we can enjoy LLVM superior performance both for C and for Fortran. I'll tell you as soon as it is ready, so maybe you can do some performance comparisons. For the 'ar r' bug, that is also present on SL (but has obviously gone unnoticed), the radar # is 9830754.

Cheers, have fun! Vincent

comment:86 Changed 10 years ago by tzonghao@…

Cc: tzonghao@… added

Cc Me!

comment:87 Changed 10 years ago by spam-macports.com@…

Cc: spam-macports.com@… added

Cc Me!

comment:88 Changed 10 years ago by adolfo.benedetti@…

Cc: adolfo.benedetti@… added

Cc Me!

comment:89 in reply to:  85 Changed 10 years ago by michelle.lynn.gill@…

Replying to vince@…:

For the 'ar r' bug, that is also present on SL (but has obviously gone unnoticed), the radar # is 9830754.

@Vince

If the 'ar r' bug existed under Snow Leopard, do you know why it (apparently) didn't cause an issue with atlas builds on that system? I'm asking this out of nothing more than curiosity, so no worries if you're busy with other things.

Thanks.

comment:90 in reply to:  85 Changed 10 years ago by patrice.kouame@…

Replying to vince@…:

You're welcome. Thanks for testing with gcc44 and 45, and for the proposal. I will probably add a dragonegg port soon, so that we can enjoy LLVM superior performance both for C and for Fortran. I'll tell you as soon as it is ready, so maybe you can do some performance comparisons. For the 'ar r' bug, that is also present on SL (but has obviously gone unnoticed), the radar # is 9830754.

Cheers, have fun! Vincent

Yes to Dragonegg. I personally am impressed with all the LLVM goodness wrapped in the latest Xcode builds (3.0 is impressive). Dragonegg seems pretty solid for C. The authors claim there are a few quirks with C++ and boost which worry me though (and I use boost a lot). Arguably a good direction for you since Fortran support is critical. Pretty bleeding edge, but then you know what you're doing (smile)...Good luck.

Any interest in my build logs? The patch blew them away but I can re-generate.

Patrice

comment:91 in reply to:  64 Changed 10 years ago by wojtekwn@…

Replying to vince@…:

Great. That's actually almost the same configuration as mine (MB Core 2 duo late 2008). I wonder what will come out the newer machines (various Core i3/5/7). This version of Atlas should be able to use the latest AVX instructions set, too.

Works great on my 2009 MB Pro (Core 2 Duo), but not on my 2011 iMac (Core i7).

Changed 10 years ago by wojtekwn@…

Attachment: main.2.log added

log from iMac with Core i7

comment:92 Changed 10 years ago by Veence (Vincent)

The log suggests that gcc45 might be outdated for the Sandy bridge line of processors: the gas45 assembler seems not to be aware of the brand new AVX instructions atlas already uses. Try to build gcc46 and use the +gcc46 option instead.

comment:93 Changed 10 years ago by adolfo.benedetti@…

I just tried with gcc45 on Lion;

$ sudo port -v install atlas +gcc45

and after 4 hours of build process, I got;

make -f Makefile INSTALL_LOG/zMMRES.sum pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG
      The best matmul kernel was SSEGENOUTDIR/zgenmm_sse.c, NB=60, written by Zalkin & Whaley
      Performance: 8979.05MFLOPS (374.13 percent of of detected clock rate)
        (Gen case got 4546.46MFLOPS)
make -f Makefile INSTALL_LOG/zNCNB pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOGmake -f Makefile INSTALL_LOG/zbestNN_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG      NCgemmNN : muladd=1, lat=2, pf=1, nb=40, mu=4, nu=2 ku=40,
                 ForceFetch=1, ifetch=4 nfetch=2
                 Performance = 4273.02 (47.59 of copy matmul, 178.04 of clock)
make -f Makefile INSTALL_LOG/zbestNT_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG      NCgemmNT : muladd=1, lat=4, pf=1, nb=40, mu=4, nu=2 ku=40,
                 ForceFetch=1, ifetch=4 nfetch=2
                 Performance = 3959.74 (44.10 of copy matmul, 164.99 of clock)
make -f Makefile INSTALL_LOG/zbestTN_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG      NCgemmTN : muladd=1, lat=2, pf=1, nb=40, mu=4, nu=2 ku=40,
                 ForceFetch=1, ifetch=4 nfetch=2
                 Performance = 4321.79 (48.13 of copy matmul, 180.07 of clock)
make -f Makefile INSTALL_LOG/zbestTT_40x40x40 pre=z nb=40 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG      NCgemmTT : muladd=1, lat=5, pf=1, nb=40, mu=4, nu=2 ku=40,
                 ForceFetch=1, ifetch=4 nfetch=2
                 Performance = 4064.43 (45.27 of copy matmul, 169.35 of clock)
make -f Makefile MMinstall pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMSEARCH.LOG
   DONE  STAGE 2-3-1 at 13:13


   BEGIN STAGE 2-3-2: CacheEdge DETECTION at 13:13
make -f Makefile INSTALL_LOG/atlas_zdNKB.h pre=z 2>&1 | ./xatlas_tee INSTALL_LOG/zMMCACHEEDGE.LOG
make[1]: *** [build] Error 255
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build'
make: *** [build] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build'
shell command " cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.9.37/build" && /usr/bin/make -w build " returned error 2
Error: Target org.macports.build returned: shell command failed (see log for details)
Warning: the following items did not execute (for atlas): org.macports.activate org.macports.build org.macports.destroot org.macports.install
Log for atlas is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/main.log
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#project.tickets>

Changed 10 years ago by adolfo.benedetti@…

comment:94 Changed 10 years ago by bonoba@…

Cc: bonoba@… added

Cc Me!

comment:95 in reply to:  93 Changed 10 years ago by Veence (Vincent)

Ciao Adolfo,

Replying to adolfo.benedetti@…:

I just tried with gcc45 on Lion;

$ sudo port -v install atlas +gcc45

Sure, you tried with the old version (.37). Pick up the new one that is append in this page as atlas_10.7_gcc45.tgz and try again. Buona fortuna, mi dica se tutto è andato bene.

comment:96 in reply to:  59 Changed 10 years ago by ke.wu@…

Replying to vince@…: That works for me, MBP 2009 13', Lion and Xcode 4.1.

So, I append a .tgz file containing the Portfile and all the patch files. Since this is highly experimental, it is given as is (not in the form of a diff), without any guarantee, and I strongly advise you to unpack it in a private section of your ports tree (see Macports manual to figure out how to setup one). If you feel adventurous, you can extract it directly in the atlas repertory of the main tree: You can always revert to previous state by wiping out everything and syncing again.

It should work on all MacOS versions with gcc45 in 64-bit mode. All other combinations are not yet tested. I've an old MacMini CoreDuo/10.6 machine on which I should test building it tonight, and a G5 server too.

Support for clang is almost here too, but since clang is stricter than gcc, there are some extra patch files to be generated. Besides, building with clang takes hours, whereas building with gcc is a matter of minutes. I don't know why.

Good luck, and please tell me if you succeed or not.

comment:97 Changed 10 years ago by Veence (Vincent)

Thanks.

I upload a new version of the Portfile and patch files. It should now build on x86_64, i386 and ppc alike. I apologize because at the same time I altered the Portfile to clean up spaces and indentation: The original was a mess. I know I should have done that separately, but the changes are so extensive anyhow that I took on me to do both at the same time, so that we have now a firm ground on which to start afresh.

comment:98 Changed 10 years ago by a.y.harano@…

Cc: a.y.harano@… added

Cc Me!

comment:99 Changed 10 years ago by smcavoy@…

Cc: smcavoy@… added

Cc Me!

comment:100 Changed 10 years ago by christoph.jacob@…

Cc: christoph.jacob@… added

Cc Me!

comment:101 in reply to:  92 Changed 10 years ago by christoph.jacob@…

Replying to vince@…:

The log suggests that gcc45 might be outdated for the Sandy bridge line of processors: the gas45 assembler seems not to be aware of the brand new AVX instructions atlas already uses. Try to build gcc46 and use the +gcc46 option instead.

I got the same problem as wojtekwn on my 2011 MBP (Core i7).

I also tried +gcc46, which fails with a different problem. The log is attached. If there is anything I can test, please let me know.

Changed 10 years ago by christoph.jacob@…

Attachment: main.3.log added

main.log for atlas-3.9.46 +gcc46 on Core i7

comment:102 Changed 10 years ago by Veence (Vincent)

Sorry, there was a typo in the Portfile I gave. Try this corrected one and tell me.

comment:103 Changed 10 years ago by tonior@…

Cc: tonior@… added

Cc Me!

comment:104 in reply to:  102 Changed 10 years ago by christoph.jacob@…

Replying to vince@…:

Sorry, there was a typo in the Portfile I gave. Try this corrected one and tell me.

I tried, but I still get the same error in the log file.

comment:105 Changed 10 years ago by Veence (Vincent)

Did you upgrade to macports 2.0 ?

comment:106 in reply to:  105 Changed 10 years ago by christoph.jacob@…

Replying to vince@…:

Did you upgrade to macports 2.0 ?

Yes, I run MacPorts 2.0.99 from SVN.

The configure problem also only shows up with gcc46, with gcc45 the configure step finishes without problems.

comment:107 Changed 10 years ago by Veence (Vincent)

Hmmm… I have not tested that myself because Macports gcc46 is outdated (current version is 4.6.1).

Try, at line 103 to delete "${os.major} < 11
" and tell me.

comment:108 Changed 10 years ago by christoph.jacob@…

No, still the same error.

comment:109 Changed 10 years ago by gaochangjian@…

Cc: gaochangjian@… removed

Cc Me!

comment:110 Changed 10 years ago by Veence (Vincent)

Okay: break! :) I am going to install gcc 4.6.1 from a private modified version of the gcc46 Portfile and try by myself. As soon as I've figured out what's wrong, I'll post a fix.

comment:111 Changed 10 years ago by rui.pereira@…

Cc: rui.pereira@… removed

Cc Me!

comment:112 Changed 10 years ago by a.y.harano@…

I use Xcode 4.1 build 4B110; MacPorts 4.0.1 and when I try to build atlas @3.9.37+gcc45-universal, it fails, even after port selfupdate.

comment:113 Changed 10 years ago by a.y.harano@…

Sorry, I meant MacPorts 2.0.1.

comment:114 Changed 10 years ago by Veence (Vincent)

3.9.37 is dead: Forget it. Use the atlas_3_9_46.tgz package I have uploaded instead.

comment:115 Changed 10 years ago by fyu+macports@…

Cc: fyu+macports@… added

Cc Me!

comment:116 Changed 10 years ago by carbncl@…

Cc: carbncl@… removed

Cc Me!

comment:117 in reply to:  110 Changed 10 years ago by christoph.jacob@…

Ok, I found the problem, and sorry, it was my mistake: gcc46 does not build gfortran by default. So, of course atlas didnt find the fortran compiler. Building it with +gfortran doesnt work for me, but that is another problem.

comment:118 Changed 10 years ago by a.y.harano@…

Thanks, it worked really well. Just for curiosity, is there any estimated date to replace the official atlas's Portfile with yours, since .37 is dead?

comment:119 Changed 10 years ago by Veence (Vincent)

@a.y.harano: I let the official maintainer decide. Well, if nothing is done by the end of the week, I will commit it anyway.

@christoph: bad news :( Neither gcc 4.6.0 nor 4.6.1 will compile because of a ld (1) bug I filed a radar about. Gcc and g++ do compile, but gfortran does not. Worse, gfortran is not enabled nor tested on the beta version currently available in Macports.

To wrap up: you’re stuck.

At this point, the only workaround I see is to somehow downgrade the processor to make use of SSE instructions only. It’ll half the performance, but, at least, it should work…

comment:120 in reply to:  119 Changed 10 years ago by christoph.jacob@…

At this point, the only workaround I see is to somehow downgrade the processor to make use of SSE instructions only. It’ll half the performance, but, at least, it should work…

Thanks a lot anyway. What I did get to work is the "old" 3.9.37 atlas by just including your ar2 workaround into the old Portfile. So I have at least something ...

comment:121 Changed 10 years ago by Veence (Vincent)

@christoph: Could you please tell me what is the output of

sysctl -a | grep hw

Thanks!

comment:122 Changed 10 years ago by spam-macports.com@…

Cc: spam-macports.com@… removed

Cc Me!

comment:123 Changed 10 years ago by brandon.zacharie@…

Cc: brandon.zacharie@… removed

Cc Me!

Changed 10 years ago by christoph.jacob@…

Attachment: sysctl.out added

Output of sysctl -a | grep hw on my 2011 MBP

comment:124 Changed 10 years ago by thomas@…

Cc: thomas@… added

Cc Me!

comment:125 Changed 10 years ago by axel.kirch@…

Cc: axel.kirch@… added

Cc Me!

comment:126 Changed 10 years ago by fleason (Fred Leason)

Cc: fleason@… added

Cc Me!

comment:127 Changed 10 years ago by michelbuck@…

Hi Vince,

I was just referred to this page due to duplication. Can I ask how to install the tgz attached to this page? I'm pretty new to all of this. Thanks,

m

comment:128 Changed 10 years ago by Veence (Vincent)

@michelbuck: Get the tgz file, unpack it in a local directory of your choice, then follow the instruction of section 4.6 of the manual (http://guide.macports.org/#development.local-repositories) Vincent

comment:129 Changed 10 years ago by Veence (Vincent)

I append a new version of the Portfile that should force atlas to downgrade corei2 CPU to corei1, thereby enabling gcc45 compilation. Please test and report any error.

Changed 10 years ago by Veence (Vincent)

Attachment: Portfile added

Patched portfile, should work with newest Sandy bridge processors and gcc45

comment:130 in reply to:  128 Changed 10 years ago by michelbuck@…

Replying to vince@…:

@michelbuck: Get the tgz file, unpack it in a local directory of your choice, then follow the instruction of section 4.6 of the manual (http://guide.macports.org/#development.local-repositories) Vincent

Sorry to bother again, I followed the instructions - I've decompressed the tar file to ~/ports/atlas but when I do cd ~/ports portindex

I get

Total number of ports parsed: 0 Ports successfully parsed: 0 Ports failed: 0 Up-to-date ports skipped: 0

Am I doing something wrong? If there's a beginner's forum I should rather post these questions to please let me know. m

comment:131 Changed 10 years ago by markus.doits@…

Cc: markus.doits@… removed

Cc Me!

comment:132 Changed 10 years ago by fleason (Fred Leason)

@michelbuck

Here is a simpler list of instructions than what I saw on section 4.6. of the manual.

$ locate atlas/files
/opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files
/opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-ATL_AVgcc-fix.diff
/opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-SpewMakeInc.c.diff
/opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-archinfo_freebsd.c.diff
/opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-archinfo_freebsd_c.diff
/opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-build-Make.top.diff
/opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/patch-emit_mm_c.diff

Will show you the directory where the files need to go. I have a plain vanilla implementation, so in my case /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files/
After you open and expand the tgz you will have a directory called files. It will have the ar2 program and a bunch of patch*.diff files. In your case I'm guessing

$ cd ~/ports/files
$ sudo cp * /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/files

This will copy the work-around into your normal MacPorts workspace.

Then cd up a level and copy the portfile.

$ cd ..
$ cp Portfile /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas/

Then you can go ahead and try an upgrade

$ sudo port upgrade atlas

If you make a mistake, you can always start from scratch as follows:

$ sudo port clean --all atlas
$ sudo port install atlas

Let it blow up and then repeat the patching process above.

comment:133 Changed 10 years ago by christoph.jacob@…

Hi Vince, if I change hw.optional_avx1_0 to hw.optional.avx1_0, then it works.

comment:134 Changed 10 years ago by Veence (Vincent)

Darn! I know I had to do a typo! :) Thanks so much for testing. I’ll correct the typo and repost an updated (& hopefully final) tar file.

Changed 10 years ago by Veence (Vincent)

Attachment: atlas_3_9_46.tgz added

Updated atlas subdirectory containing Portfile and patches. Should work on any machine with gcc44 or 45.

comment:135 in reply to:  132 Changed 10 years ago by michelbuck@…

@fleason

Hi, Thanks for the help. So I installed atlas, getting he standard error message

Error: Target org.macports.build returned: shell command failed (see log for details) Log for atlas is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/main.log Error: Status 1 encountered during processing. To report a bug, see <http://guide.macports.org/#project.tickets>

and I then copied the files into the relevant folders. However, upon updating I get

0 Sat Aug 06 20:03:29 michelbuck ~/atlas/atlas$ sudo port upgrade atlas Error: atlas is not installed To report a bug, see <http://guide.macports.org/#project.tickets>

Is there something obvious going wrong? m

comment:136 Changed 10 years ago by fleason (Fred Leason)

@michaelbuck

Try

$ sudo port install atlas

comment:137 Changed 10 years ago by matthew.g.russell@…

I have the same problem.

I followed the instructions from michelbuck using the files from atlas_3_9_46.tgz, and I never get it into a state where I can upgrade it.

So, if I clean, install, replace the Portfile and files/*, then try to upgrade ==> I can't. So I run the install .. the install will see that the Portfile has been replaced and completely re-run and bail a couple hours later with the same error.

If I instead only replace the files in files/* at that point, the install will jump to the error right away.

Just in case, I ran a selfupdate and selfupgrade today.. Didn't change anything.

Am I doing something wrong?

comment:138 Changed 10 years ago by fleason (Fred Leason)

I wish I saved the steps that worked for me. I cannot remember the sequence of clean, clean --all, install, upgrade that made it work for me. I don't understand the MacPorts environment to help. But it did work.
The only other thing I did different is I used the patched Portfile that is downloaded separate from the tgz. And I went ahead and did the gcc46 install with gfortran. Although my script indicated that it used gcc44. I will attach my log.

comment:139 in reply to:  138 Changed 10 years ago by fleason (Fred Leason)

Replying to fleason@…:

I will attach my log.

Oops. Log gets deleted if it installs correctly.

comment:140 Changed 10 years ago by Veence (Vincent)

I am going to commit my version of atlas. It should avoid all this hocus-pocus.

comment:141 Changed 10 years ago by Veence (Vincent)

Committed in r82161. Please update and try again.

comment:142 Changed 10 years ago by cdeil (Christoph Deil)

When I update the atlas directory using svn I get the following error:

svn: In directory '/opt/local/var/macports/sources/svn.macports.org/trunk/dports/math/atlas/files'
svn: Can't open file '/opt/local/var/macports/sources/svn.macports.org/trunk/dports/math/atlas/files/.svn/tmp/text-base/patch-Make_ttune.diff.svn-base': No such file or directory
shell command "/opt/local/bin/svn update --non-interactive /opt/local/var/macports/sources/svn.macports.org/trunk/dports" returned error 1

I also tried removing the atlas directory and checking out a fresh version. Same error.

Any advice?

comment:143 Changed 10 years ago by derekathomas@…

Cc: derekathomas@… added

Cc Me!

comment:144 in reply to:  141 ; Changed 10 years ago by geoffgrimwood

Replying to vince@…:

Committed in r82161. Please update and try again.

Successfully installed in less than 10 minutes on 2011 MBP running Lion and XCode 4.1 (latest i7 processor).

Great!

Thank you very much Vince for your efforts over the last couple of weeks.

Fingers now crossed that py27-matplotlib will build and install.

comment:145 Changed 10 years ago by lukas.reichlin@…

Cc: lukas.reichlin@… added

Cc Me!

comment:146 in reply to:  144 ; Changed 10 years ago by Veence (Vincent)

Resolution: fixed
Status: newclosed

Replying to geoff@…:

Replying to vince@…:

Committed in r82161. Please update and try again.

Successfully installed in less than 10 minutes on 2011 MBP running Lion and XCode 4.1 (latest i7 processor).

Great!

Thank you very much Vince for your efforts over the last couple of weeks.

My pleasure. Glad to know it works on the latest Core i7 with gcc45. But please note that due to the inability to compile gcc46, you get only half the performance you could draw out of your brand new chip.

I'll now report all the patches upstream, so that we may have a cleaner future version.

Cheers.

PS: I close the ticket, since it seems to work for everyone.

comment:147 in reply to:  144 Changed 10 years ago by michelle.lynn.gill@…

Replying to geoff@…:

Thank you very much Vince for your efforts over the last couple of weeks.

Yes, thank you, Vincent.

Fingers now crossed that py27-matplotlib will build and install.

Worked fine for me on Lion.

comment:148 in reply to:  146 Changed 10 years ago by fleason (Fred Leason)

Replying to vince@…:

Replying to geoff@…:

My pleasure. Glad to know it works on the latest Core i7 with gcc45. But please note that due to the inability to compile gcc46, you get only half the performance you could draw out of your brand new chip.

I got gcc46 to build on a first generation i7. Is it only an issue with the second generation?

comment:149 Changed 10 years ago by Veence (Vincent)

It is an issue with the ld (1) utility (the linker) on 10.7, it does not depend on the CPU. GCC45 does not support the additional AVX instruction set introduced with Sandy bridge, therefore Atlas assembler routines that use it fails. As a workaround, I hardwired the detection routine to consider any CoreiX processor to be first generation. That way, Atlas makes use of instructions known to gcc45. But it is a waste of power. If you have succeeded in building gcc46, you can easily patch the Portfile to make use of it.

Note: See TracTickets for help on using tickets.