Opened 14 years ago

Closed 12 years ago

Last modified 11 years ago

#25662 closed defect (fixed)

qrupdate: +atlas variant fails to build: undefined symbols

Reported by: moe.nicholas@… Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version: 1.9.1
Keywords: haspatch Cc: penyuan@…, stefan@…, michaelld (Michael Dickens), mfuhrer@…, AlonzoQuixote@…, jsnodgra@…, ejtttje@…, Veence (Vincent), ryandesign (Ryan Carsten Schmidt), nabble@…, mauricio_ville@…, MacPorts@…, m1cha3ld0m1n0@…, enderw88@…, sewebster@…, jerome.lelong@…, ben.pts@…, mojca (Mojca Miklavec), jovo@…, lawrence.ong@…, anddam (Andrea D'Amore), sam.petulla@…
Port: qrupdate

Description (last modified by Veence (Vincent))

Hello,

I am trying to install octave on my PowerBook G4. I have run

$ sudo port selfupdate

and

$ sudo port upgrade outdated

The last dependency to install, qrupdate, fails to build and gives the following output:

$ sudo port install octave
--->  Computing dependencies for octave
--->  Dependencies to be installed: qrupdate
--->  Fetching qrupdate
--->  Attempting to fetch qrupdate-1.1.1.tar.gz from http://superb-east.dl.sourceforge.net/qrupdate
--->  Attempting to fetch qrupdate-1.1.1.tar.gz from http://downloads.sourceforge.net/qrupdate
--->  Verifying checksum(s) for qrupdate
--->  Extracting qrupdate
--->  Applying patches to qrupdate
--->  Configuring qrupdate
--->  Building qrupdate
Error: Target org.macports.build returned: shell command failed
Log for qrupdate is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_qrupdate/main.log
Error: The following dependencies failed to build: qrupdate
Error: Status 1 encountered during processing.
To report a bug, see <http://guide.macports.org/#project.tickets>
$

I am attaching the log file cited above.

Attachments (8)

main.log (42.0 KB) - added by moe.nicholas@… 14 years ago.
qrupdate_Portfile.diff (868 bytes) - added by michaelld (Michael Dickens) 13 years ago.
"svn diff" of 'qrupdate' Portfile, to add in missing atlas symbols
main.2.log (29.7 KB) - added by stefan@… 13 years ago.
main.log file of my failed qrupdate build. iBook G4 OS X 10.5.8
qrupdate_Portfile_2.diff (862 bytes) - added by mfuhrer@… 13 years ago.
Modified patch with -lf77blas
qrupdate_Portfile3.diff (937 bytes) - added by nabble@… 12 years ago.
qrupdate_Portfile_4.diff (1.0 KB) - added by jerome.lelong@… 12 years ago.
Add blas dependency for +atlas variant
patch-port_qrupdate.diff (1.1 KB) - added by anddam (Andrea D'Amore) 12 years ago.
qrupdate_Portfile_5.diff (316 bytes) - added by jerome.lelong@… 12 years ago.
Fix conflict between gcc45 and gcc46

Download all attachments as: .zip

Change History (70)

Changed 14 years ago by moe.nicholas@…

Attachment: main.log added

comment:1 Changed 14 years ago by jmroot (Joshua Root)

Keywords: qrupdate octave removed

The symbols that are undefined here are present in /opt/local/lib/liblapack.dylib on my system (10.6/x86_64).

comment:2 Changed 14 years ago by penyuan@…

The same thing happens on my MacBook Pro running Mac OS X 10.6.4 when trying to install Octave. Is there a workaround?

comment:3 Changed 14 years ago by penyuan@…

Cc: penyuan@… added

Cc Me!

comment:4 Changed 13 years ago by stefan@…

Same on my iBook G4 with 10.5.8. liblapack.dylib is present but not recognized.

comment:5 Changed 13 years ago by stefan@…

Cc: stefan@… added

Cc Me!

comment:6 Changed 13 years ago by michaelld (Michael Dickens)

On my 10.6.4 x86_64 box, the undefined symbols are actually in -lptf77blas (either .a or .dylib) -- which is interesting because that library isn't directly referenced during the link stage (nor is the linking told to ignore undefined symbols), but the linking still works. I'm attached a patch that might work; please try it out.

comment:7 Changed 13 years ago by michaelld (Michael Dickens)

The Octave issue is on another ticket already. For -devel, that's #26640. IDK which one it is for octave alone, but I'm sure it's there.

comment:8 Changed 13 years ago by michaelld (Michael Dickens)

@stefan: Can you attach a debug-log of your install, to see if it's the same issue as for this ticket?

Changed 13 years ago by michaelld (Michael Dickens)

Attachment: qrupdate_Portfile.diff added

"svn diff" of 'qrupdate' Portfile, to add in missing atlas symbols

comment:9 Changed 13 years ago by michaelld (Michael Dickens)

Cc: michaelld@… added

Cc Me!

Changed 13 years ago by stefan@…

Attachment: main.2.log added

main.log file of my failed qrupdate build. iBook G4 OS X 10.5.8

comment:10 in reply to:  description Changed 13 years ago by mfuhrer@…

Ran into this link problem this evening on my Powerbook G4 with Mac OS X 10.5.8. Tried Michael's patch with no success, as I don't have -lptf77blas in /opt/local/lib. Then I found -lf77blas which does define the missing symbols. Modified Michael's patch and was able to link and install qrupdate successfully.

Changed 13 years ago by mfuhrer@…

Attachment: qrupdate_Portfile_2.diff added

Modified patch with -lf77blas

comment:11 Changed 13 years ago by mfuhrer@…

Cc: mfuhrer@… added

Cc Me!

comment:12 Changed 12 years ago by AlonzoQuixote@…

Cc: AlonzoQuixote@… added

Cc Me!

comment:13 Changed 12 years ago by jsnodgra@…

Cc: jsnodgra@… added

Cc Me!

comment:14 Changed 12 years ago by ejtttje@…

I'm seeing this now following an update to atlas (#34295) which builds the blas libraries in question. I also had to add -latlas.

The -ptf77blas and -ltatlas give multi-threaded usage, whereas -lf77blas and -latlas give single-threaded libraries. I'm not sure which is more appropriate here.

I still wound up running into issues with __gfortran_transfer_character_write and __gfortran_transfer_integer_write being undefined, which is supposedly fixed by setting the linker/loader to use gfortran, but I couldn't figure that out. I wound up adding -undefined dynamic_lookup to get around this, which probably isn't the right solution.

comment:15 Changed 12 years ago by ejtttje@…

Cc: ejtttje@… added

Cc Me!

comment:16 Changed 12 years ago by Veence (Vincent)

The dylibs automatically built by atlas 3.9.75 differ from those that were (manually) built by 3.9.47. They are named libtatlas.dylib and libsatlas.dylib. You must look after the undefined symbols using nm then maybe patch qrupdate accordingly.

Changed 12 years ago by nabble@…

Attachment: qrupdate_Portfile3.diff added

comment:17 Changed 12 years ago by nabble@…

I had a similar problem trying to install octave, link failing on qrupdate, but I had to tweak my Portfile differently to get it to build - attached as qrupdate_Portfile3.diff

comment:18 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: vince@… ryandesign@… added

Vincent, is the patch attached by makuch.org correct—add "-latlas -lsatlas -ltatlas -lf77blas -lptf77blas"?

I've rearranged things a bit in my update to 1.1.2 so you may want to keep that in mind when preparing a fix for this issue as well.

comment:19 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: nabble@… added

comment:20 Changed 12 years ago by nabble@…

I seriously doubt it's correct, but it worked for me :-) so I just thought I'd share it with you guys knowing you would fix it correctly.

comment:21 Changed 12 years ago by Veence (Vincent)

-latlas -lf77blas -lptf77blas is redundant with -libsatlas and -libtatlas. The first three are static archives, and the last two dynamic libraries. Then you can choose between satlas or tatlas whether you need threaded Atlas or not. So, to wrap up, only one of libsatlas or libtatlas is needed.

comment:22 Changed 12 years ago by mauricio_ville@…

Cc: mauricio_ville@… added

Cc Me!

comment:23 Changed 12 years ago by MacPorts@…

Cc: MacPorts@… added

Cc Me!

comment:24 Changed 12 years ago by m1cha3ld0m1n0@…

Cc: m1cha3ld0m1n0@… added

Cc Me!

comment:25 Changed 12 years ago by enderw88@…

I was able to get qrupdate to build and install by

  1. cd'ing into the working directory /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/math/qrupdate/work/qrupdate-1.1.1
  2. executing sudo make solib.
  3. executing sudo port install qrupdate.

comment:26 Changed 12 years ago by m1cha3ld0m1n0@…

Using enderw88's steps, was able to build on 10.5.8. Thanks!

comment:27 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: enderw88@… added

enderw88, that's not a fix. When you do that, you're bypassing the PREFIX, BLAS and LAPACK arguments MacPorts would have set for you, so at that point you're not actually using atlas, plus the library will get the wrong install_name. So you may as well install qrupdate without atlas support at that point.

sudo port selfupdate
sudo port clean qrupdate
sudo port install qrupdate -atlas

comment:28 in reply to:  27 Changed 12 years ago by enderw88@…

Replying to ryandesign@…:

enderw88, that's not a fix. When you do that, you're bypassing the PREFIX, BLAS and LAPACK arguments MacPorts would have set for you, so at that point you're not actually using atlas, plus the library will get the wrong install_name. So you may as well install qrupdate without atlas support at that point.

sudo port selfupdate
sudo port clean qrupdate
sudo port install qrupdate -atlas

Thanks, at this point I just really want to get Octave built. But I can see why this isn't a real solution.

comment:29 Changed 12 years ago by sewebster@…

Cc: sewebster@… added

Cc Me!

Changed 12 years ago by jerome.lelong@…

Attachment: qrupdate_Portfile_4.diff added

Add blas dependency for +atlas variant

comment:30 Changed 12 years ago by jerome.lelong@…

qrupdate does not build with variant +atlas because of missing symbols at link actually provided by Blas. It seems that the patch qrupdate_Portfile_4.diff solves the issue at least if atlas was installed with gcc45. If atlas was installed with variant +gcc46, then you should consider adding the corresponding variant to qrupdate to achieve the build.

comment:31 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: jerome.lelong@… added
Keywords: haspatch added
Summary: qrupdate fails to buildqrupdate: +atlas variant fails to build: undefined symbols

comment:32 Changed 12 years ago by ben.pts@…

Cc: ben.pts@… added

Cc Me!

comment:33 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

Has duplicate #34654.

comment:34 Changed 12 years ago by weasel@…

It appears a diff four fixes the problem I noted in #34654 with qrupdate, but forces atlas +gcc46 be installed, which oddly enough completes without gcc46 being installed. There is still no /opt/local/lib/libsatlas.dylib, maybe that's intentional. I haven't yet tested if octave ultimately works correctly.

comment:35 Changed 12 years ago by mojca (Mojca Miklavec)

Cc: mojca.miklavec.lists@… added

Cc Me!

comment:36 Changed 12 years ago by jovo@…

Cc: jovo@… added

Cc Me!

comment:37 Changed 12 years ago by beunjb@…

Octave works with the diff patch

cd $(port dir qrupdate)
sudo patch -p0 < ~/Downloads/qrupdate_Portfile_4.diff

Thanks for the good work.

comment:38 in reply to:  37 Changed 12 years ago by rjfergus@…

Replying to beunjb@…:

Octave works with the diff patch

cd $(port dir qrupdate)
sudo patch -p0 < ~/Downloads/qrupdate_Portfile_4.diff

Thanks for the good work.

Hi,

I have the patch file in my $(port dir qrupdate) directory but when I run the patch command above, I get "can't find file to patch at input line 3" and "Perhaps you used the wrong -p or --strip option?". I notice in the diff file that there are some strange tabs but I don't know the syntax ...

comment:39 Changed 12 years ago by lawrence.ong@…

Cc: lawrence.ong@… added

Cc Me!

comment:40 Changed 12 years ago by lawrence.ong@…

Using the forth patch, qrupdate and octave both install without error. But when launching octave, I get the following error message:

dyld: Library not loaded: /opt/local/lib/libptcblas.dylib
  Referenced from: /opt/local/lib/libarpack.2.dylib
  Reason: image not found
Trace/BPT trap: 5

comment:41 Changed 12 years ago by Veence (Vincent)

The patch should read: BLAS="-L${prefix}/lib -ltatlas" and nothing else behind.

Arpack needs rebuilding too. The Atlas dynamic libraries have changed into libsatlas (single threaded) and libtalas (threaded).

comment:42 Changed 12 years ago by Veence (Vincent)

I have committed r93765 that should fix this bug. GCC46 is now default, like atlas, to avoid messing up gfortran libraries. Please test and report.

comment:43 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

We want all ports having gcc variants to default to the same version. gcc45 is the version we have chosen at this time. If you want to change that default for all ports, you should start a discussion about it on the mailing list.

comment:44 Changed 12 years ago by Veence (Vincent)

I wasn’t aware of this. Then you can make the default gcc depend on whether the +atlas variant is set or not. gcc46 in the former case, gcc45 in the latter.

comment:45 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

Why shouldn't atlas default to gcc45 as well?

comment:46 Changed 12 years ago by Veence (Vincent)

Because atlas is all about performance and a gcc46 compiled atlas performs 10 to 15 % better than a gcc45 one. On the other hand, gcc47 improvements over gcc46 (at this time) are moot.

comment:47 in reply to:  46 ; Changed 12 years ago by anddam (Andrea D'Amore)

Replying to vince@…:

gcc46 compiled atlas performs 10 to 15 % better than a gcc45 one.

That's not a really big enhancement, I vote for keeping the default at gcc45 like other fortran-needing ports.

If an user really needs the speed bonus he/she can enable +gcc46, speaking in a broad sense I'd keep dependencies more streamlined.

comment:48 Changed 12 years ago by anddam (Andrea D'Amore)

Cc: and.damore@… added

Cc Me!

comment:49 Changed 12 years ago by ryandesign (Ryan Carsten Schmidt)

The reason we default everything to the same gcc version is that subtle hard-to-track-down problems can occur if programs linked with different versions of the C library (i.e. from different versions of the gcc ports) are linked together. I don't know if that affects atlas or not.

comment:50 in reply to:  49 Changed 12 years ago by jerome.lelong@…

If atlas was compiled with gcc46, then qrupdate must alo be compiled with gcc46 otherwise linking fails with missing Fortran symbols. Hence, a gcc46 variant should be added to qrupdate Portfile even if gcc45 remains the default.

comment:51 Changed 12 years ago by Veence (Vincent)

The update I have committed have removed obsolete gcc43/44 variants in favor of gcc46/47.

comment:52 in reply to:  47 ; Changed 12 years ago by Veence (Vincent)

Description: modified (diff)

Replying to and.damore@…:

Replying to vince@…:

gcc46 compiled atlas performs 10 to 15 % better than a gcc45 one.

That's not a really big enhancement

That’s your opinion. In the field of HPC, 10% is a lot, especially when all you have to do is change compiler. It’s about half an hour win every four hours. If a user is not concerned with Atlas, he can just opt out by selecting the no_atlas variants and get standard Accelerate framework performance. If he cares about Atlas, then it’s probably to squeeze the most out of his hardware.

comment:53 Changed 12 years ago by mf2k (Frank Schima)

@vince: Any user concerned about tweaking their machine's performance can - and should - be using variants.conf to set their gcc version to 4.6 or 4.7 anyway. The vast majority of Macports users don't know or care about this. Instead they will (rightly) complain that they have two versions of gcc installed. So gcc45 should be used as the default to keep in line with the other fortran ports.

comment:54 Changed 12 years ago by Veence (Vincent)

Well, I only half agree because the people who are likely to use Atlas, Octave, and so forth are probably only those in need of processing power, and not the typical Macports user, if that notion makes any sense. But okay, I gracefully abide by the majority’s opinion. I’ll update Atlas accordingly. Anyhow, there is always this major flaw in Atlas whereby AVX instructions cannot be used, and that alone halves the performance of some kernels…

comment:55 in reply to:  52 Changed 12 years ago by anddam (Andrea D'Amore)

Replying to vince@…:

That’s your opinion. In the field of HPC, 10% is a lot, especially when all you have to do is change compiler.

Yes, it is my opinion that a 0.1 factor is an order of magnitude smaller than the total time and it's not meaningful to common users. That we should focus on people doing intensive computing is yours.

I believe that anyone doing HPC has the responsibility to tune his/her system.

If a user is not concerned with Atlas, he can just opt out by selecting the no_atlas variants and get standard Accelerate framework performance. If he cares about Atlas, then it’s probably to squeeze the most out of his hardware.

He could be using atlas just for a dependency.

Back to this ticket

This ticket is messy, it's not clear if users after comment:12 are experiencing the same issue as reporter.

I stumbled into a similar one with different symbols' name after the name change in single- and multi-threaded atlas library.

I'm attaching a new patch that adds -latlas to BLAS and bumps revision to force rebuild.

Please anyone who had the issue confirm it's working so we can close this ticket.

Changed 12 years ago by anddam (Andrea D'Amore)

Attachment: patch-port_qrupdate.diff added

comment:56 in reply to:  41 Changed 12 years ago by lawrence.ong@…

Replying to vince@…:

The patch should read: BLAS="-L${prefix}/lib -ltatlas" and nothing else behind.

Arpack needs rebuilding too. The Atlas dynamic libraries have changed into libsatlas (single threaded) and libtalas (threaded).

Thanks. Rebuilding arpack solves the following problem:

dyld: Library not loaded: /opt/local/lib/libptcblas.dylib
  Referenced from: /opt/local/lib/libarpack.2.dylib
  Reason: image not found
Trace/BPT trap: 5

comment:57 Changed 12 years ago by anddam (Andrea D'Amore)

Resolution: fixed
Status: newclosed

Vince had already committed r93765 by the time I last replied, I was mistakenly using a previous version of Portfile.

I've switched default variant back to +gcc45 and bumped revision in r93783, r93785.

Closing the ticket as fixed.

comment:58 in reply to:  57 ; Changed 12 years ago by jerome.lelong@…

Resolution: fixed
Status: closedreopened

There is still a conflict in the Portfile between gcc45 and gcc46. The qrupdate_Portfile_5.diff patch solves the issue.

Changed 12 years ago by jerome.lelong@…

Attachment: qrupdate_Portfile_5.diff added

Fix conflict between gcc45 and gcc46

comment:59 in reply to:  58 Changed 12 years ago by anddam (Andrea D'Amore)

Resolution: fixed
Status: reopenedclosed

Replying to jerome.lelong@…:

The qrupdate_Portfile_5.diff patch solves the issue.

The patch was not made from current Portfile, anyway I committed the change in r93792.

Ticket closed.

comment:60 in reply to:  57 Changed 12 years ago by anddam (Andrea D'Amore)

Replying to and.damore@…:

I've switched default variant back to +gcc45 […] in r93783

That's actually r93782.

comment:61 Changed 11 years ago by sam.petulla@…

Hi

I'm having all kinds of problems installing qrupdate and cannot install the patch. I get the following error after running: sudo patch -p0 < ~/Downloads/qrupdate_Portfile_4.diff

patching file /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/math/qrupdate/Portfile
Hunk #1 FAILED at 69.
1 out of 1 hunk FAILED -- saving rejects to file /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/math/qrupdate/Portfile.rej
Last edited 11 years ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:62 Changed 11 years ago by ryandesign (Ryan Carsten Schmidt)

Cc: sam.petulla@… added

This ticket is closed and fixed and the patch was already committed 11 months ago; you do not need to apply it manually. Just sudo port selfupdate to receive the changes, as usual. If you are unable to install the port, file a new ticket.

Note: See TracTickets for help on using tickets.