openmpi-default not compiling using clang on Mac OSX 10.6.8

[andrewhw (Andrew Wright)]

When running "port upgrade outdated" on Mac OSX 10.6.8, openmpi-devel fails. As can be seen in the log, the test for clang seems to determine that the C compiler cannot produce an executable.

This is particularly odd, as this is a machine used regularly for C/C++ work, under both clang and gcc.

As an example: talos [ 80 ]% cd /tmp talos [ 81 ]% cat > hello.c #include <stdio.h>

int main() {

printf("Hello, world?\n"); return 0;

}

talos [ 82 ]% clang -o hello hello.c talos [ 83 ]% ./hello Hello, world? talos [ 84 ]%

uname -a reports: Darwin talos 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64

[andrewhw (Andrew Wright)]

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/main.log

[seanfarley (Sean Farley)]

Thanks for the log. I'm looking into it now.

[seanfarley (Sean Farley)]

I don't know if the description has a typo but this error is for openmpi-default (which makes sense because before my update of macports there was no openmpi-devel port). Could you attach config.log from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/opal/mca/event/libevent2021/libevent? It might be in a directory above that.

Also, the buildbot should have this port for binary download now, so maybe you could use that? (just rerun the upgrade command)

[andrewhw (Andrew Wright)]

Sorry for the delay -- I was out of town yesterday.

I have re-run update, and still see the problem. There was a typo in the ticket -- we are definitely talking about openmpi-devel. New config.log attached as requested...

[andrewhw (Andrew Wright)]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/opal/mca/event/libevent2021/libevent/config.log

[seanfarley (Sean Farley)]

Something is very wrong here. For one, the work directory is 'openmpi-default' which should be 'openmpi-devel-default'. Second, it's missing the hwloc dependency which should be installed automatically. Third, you said this comes from 'port upgrade outdated' which isn't possible since I added that in r116342. Are you using a local source which provides openmpi-devel?

[andrewhw (Andrew Wright)]

Not intentionally. The installation is not intentionally changed from the defaults in any way. I have been using ports for some years, but there are not that many ports installed on this machine. The top-level ports that have been installed are the following: gnuplot, gv, octave, xfig, xpdf, xv.

I am somewhat loathe to deinstall/reinstall everything as I am currently fighting a problem in octave due to the lack of a fortran compiler in clang on another machine (based on Mavericks).

Here is what I get when I attempt some queries regarding openmpi (extra newlines lines added to avoid reflow):

talos [ 25 ]% port rdependents openmpi

The following ports are dependent on openmpi:

arpack

octave

talos [ 26 ]% port rdependents openmpi-devel

Error: Registry error: openmpi-devel not registered as installed.

talos [ 27 ]% port rdependents openmpi-default

Error: Registry error: openmpi-default not registered as installed.

Currently there are three openmpi packages installed:

openmpi @1.7.2_0+gcc47

openmpi @1.7.2_1+gcc47

openmpi @1.7.3_0+gcc47 (active)

I am attaching the full output of "port installed" as a file.

[andrewhw (Andrew Wright)]

Output of "port installed"

[seanfarley (Sean Farley)]

Right, so as I thought you only have openmpi installed, not openmpi-devel. Where is openmpi-devel coming from? Either way, it should be configuring so maybe the old openmpi is conflicting with it? Could you try deactivating it first and see if configure goes through after that?

[seanfarley (Sean Farley)]

Replying to sean@…:

Right, so as I thought you only have openmpi installed, not openmpi-devel. Where is openmpi-devel coming from? Either way, it should be configuring so maybe the old openmpi is conflicting with it? Could you try deactivating it first and see if configure goes through after that?

Specifically, run these commands:

sudo port deactivate octave arpack openmpi
sudo port configure openmpi-default

and let me know how that goes.

[andrewhw (Andrew Wright)]

Hmm. Deactivate seemed to go well:

% sudo port deactivate octave arpack openmpi
--->  Deactivating octave @3.6.4_10+atlas+gcc47
--->  Cleaning octave
--->  Deactivating arpack @3.1.3_0+accelerate+openmpi
--->  Cleaning arpack
--->  Deactivating openmpi @1.7.3_0+gcc47
--->  Cleaning openmpi
%

Configure, not so well:

% sudo port configure openmpi-default
--->  Computing dependencies for openmpi-default
--->  Configuring openmpi-default
Error: Failed to configure openmpi-default, consult /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/config.log
Error: org.macports.configure for port openmpi-default returned: configure failure: command execution failed
Please see the log file for port openmpi-default for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/main.log
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port openmpi-default failed
%

Attaching the new log now...

(it looks much the same as the first one -- I am leaving "main.log" as the original and providing this one a new name)

[andrewhw (Andrew Wright)]

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/main.log (reupload)

[andrewhw (Andrew Wright)]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/config.log (reupload)

[andrewhw (Andrew Wright)]

... and looking above, my correction from "openmpi-devel -> openmpi-default" was itself a miscorrection. While I am sure I have seen the string "openmpi-devel" somewhere in poking through the logs, you are quite right in stating that we are only looking at "openmpi-default", and my earlier attempt at clarification simply re-affirmed the error through bad typing/proofreading.

[seanfarley (Sean Farley)]

No worries :-) Thanks for doing the re-configure. I forgot to mention to clean the build with sudo port clean openmpi-default. After cleaning and re-running configure, could you upload the log at: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/openmpi-1.7.3/opal/mca/event/libevent2021/libevent/?

[andrewhw (Andrew Wright)]

Still no success. Clean is good:

% sudo port clean openmpi-default
--->  Cleaning openmpi-default
%

but configure doesn't succeed

% sudo port configure openmpi-default
--->  Computing dependencies for openmpi-default
--->  Fetching distfiles for openmpi-default
--->  Verifying checksums for openmpi-default
--->  Extracting openmpi-default
--->  Configuring openmpi-default
Error: Failed to configure openmpi-default, consult /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/config.log
Error: org.macports.configure for port openmpi-default returned: configure failure: command execution failed
Please see the log file for port openmpi-default for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/main.log
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port openmpi-default failed
%

In addition, I'm not sure which log you mean, as I don't have any log file in that directory:

% ls /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/openmpi-1.7.3/opal/mca/event/libevent2021/libevent/*.log
ls: No match.
% ls /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/openmpi-1.7.3/opal/mca/event/libevent2021/libevent*/*.log
ls: No match.
% 

Perhaps you meant the config log again? I'll upload it and tag it "v3"...

[andrewhw (Andrew Wright)]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/config.log (upload version 3)

[andrewhw (Andrew Wright)]

I worked out that you must have meant the one in /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/opal/mca/event/libevent2021/libevent

Uploading now...

[andrewhw (Andrew Wright)]

/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi-default/work/build/opal/mca/event/libevent2021/libevent/config.log

[seanfarley (Sean Farley)]

Thanks again! The error still seems to be hwloc:

ld: library not found for -lhwloc

What does hwloc show for its libraries? Here is what I have:

$ port contents hwloc | grep /lib
  /opt/local/lib/libhwloc.5.dylib
  /opt/local/lib/libhwloc.dylib
  /opt/local/lib/libhwloc.la
  /opt/local/lib/pkgconfig/hwloc.pc

[seanfarley (Sean Farley)]

Ok, I was finally able to reproduce this on my machine. Hopefully, I will be able to figure out why -L/opt/local isn't being passed through.

[andrewhw (Andrew Wright)]

Strangely, this is the same:

% port contents hwloc | grep /lib
  /opt/local/lib/libhwloc.5.dylib
  /opt/local/lib/libhwloc.dylib
  /opt/local/lib/libhwloc.la
  /opt/local/lib/pkgconfig/hwloc.pc
% 

For filetimes I have this:

% ls -l `!!`
ls -l `port contents hwloc | grep /lib`
-rwxr-xr-x  1 root  admin  152072 Nov 29 19:41 /opt/local/lib/libhwloc.5.dylib
lrwxr-xr-x  1 root  admin      16 Nov 29 19:41 /opt/local/lib/libhwloc.dylib -> libhwloc.5.dylib
-rwxr-xr-x  1 root  admin     923 Nov 29 19:41 /opt/local/lib/libhwloc.la
-rw-r--r--  1 root  admin     285 Nov 29 19:41 /opt/local/lib/pkgconfig/hwloc.pc
% 

[andrewhw (Andrew Wright)]

Ah, great (I guess). Let me know if there is anything else you need from me ...

[seanfarley (Sean Farley)]

Ok, well, I figured out the problem:

​https://trac.macports.org/browser/trunk/dports/science/openmpi/Portfile?rev=116532#L110

The port deletes ${prefix}/lib to prevent linking to a previous version already installed. The strange part is why my version worked … Well, anyway, I will test to make sure version >=1.7 don't link against a previously installed version of openmpi and get back to you.

[seanfarley (Sean Farley)]

Hopefully, r116536 fixes this. Try to update, clean, and reconfigure now.

[andrewhw (Andrew Wright)]

Some improvement! I ran the following, and all was well:

% sudo port clean openmpi-default
% sudo port configure openmpi-default

After that I tried this:

% sudo port activate octave arpack openmpi

and was told that it wanted a specific version (for octave), so I tried this:

% sudo port activate "octave @3.6.4_10+atlas+gcc47"
--->  Activating octave @3.6.4_10+atlas+gcc47

which looked good, so I continued with

% sudo port activate arpack
--->  Computing dependencies for arpack
--->  Dependencies to be installed: openmpi
--->  Fetching distfiles for openmpi
--->  Verifying checksums for openmpi
--->  Extracting openmpi
--->  Configuring openmpi
Error: openmpi has been made obsolete by the port openmpi-default. Please install openmpi-default instead.
Error: org.macports.configure for port openmpi returned: obsolete port
Error: Failed to install openmpi
Please see the log file for port openmpi for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_science_openmpi/openmpi/main.log
Error: The following dependencies were not installed: openmpi
Warning: Failed to execute portfile from registry for arpack @3.1.3_0+accelerate+openmpi
--->  Activating arpack @3.1.3_0+accelerate+openmpi
% 

at this point, it seemed sensible to install "openmpi-default", so I did that:

% sudo port install "openmpi-default"
--->  Computing dependencies for openmpi-default
--->  Building openmpi-default
--->  Staging openmpi-default into destroot
--->  Installing openmpi-default @1.7.3_2+gcc48
--->  Activating openmpi-default @1.7.3_2+gcc48


+--- OpenMPI Usage note ----
| The mpi[cc|cxx|f77|f90] weappers are installed as
| mpi[cc|cxx|f77|f90]-mp. To make openmpi-default the default (i.e. the
| version you get when you run 'mpicc/mpicxx'), please run:
|
| sudo port select --set mpi openmpi-mp-fortran
+-------------------------


--->  Cleaning openmpi-default
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  Found 6 broken file(s), matching files to ports
--->  Found 1 broken port(s), determining rebuild order
--->  Rebuilding in order
     arpack @3.1.3 +accelerate+openmpi
--->  Computing dependencies for arpack
--->  Fetching archive for arpack
--->  Attempting to fetch arpack-3.1.4_1+accelerate+gfortran+openmpi.darwin_10.x86_64.tbz2 from http://lil.fr.packages.macports.org/arpack
--->  Attempting to fetch arpack-3.1.4_1+accelerate+gfortran+openmpi.darwin_10.x86_64.tbz2 from http://packages.macports.org/arpack
--->  Attempting to fetch arpack-3.1.4_1+accelerate+gfortran+openmpi.darwin_10.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/arpack
--->  Fetching distfiles for arpack
--->  Attempting to fetch arpack-ng_3.1.4.tar.gz from http://ykf.ca.distfiles.macports.org/MacPorts/mpdistfiles/arpack
--->  Verifying checksums for arpack
--->  Extracting arpack
--->  Applying patches to arpack
--->  Configuring arpack
--->  Building arpack
Error: org.macports.build for port arpack returned: command execution failed
Please see the log file for port arpack for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_arpack/arpack/main.log
Error: Unable to upgrade port: 1
Error rebuilding arpack
    while executing
"error "Error rebuilding $portname""
    (procedure "revupgrade_scanandrebuild" line 382)
    invoked from within
"revupgrade_scanandrebuild broken_port_counts $opts"
    (procedure "macports::revupgrade" line 5)
    invoked from within
"macports::revupgrade $opts"
    (procedure "action_revupgrade" line 2)
    invoked from within
"action_revupgrade $action $portlist $opts"
    (procedure "action_target" line 96)
    invoked from within
"$action_proc $action $portlist [array get global_options]"
    (procedure "process_cmd" line 93)
    invoked from within
"process_cmd $remaining_args"
    invoked from within
"if { [llength $remaining_args] > 0 } {

    # If there are remaining arguments, process those as a command
    set exit_status [process_cmd $remaining..."
    (file "/opt/local/bin/port" line 4857)
%

... so have I lost the path in there? Should I be running the "port select" line? Or is openmpi now no longer the problem and I have triggered something new in arpack?

I will note that I earlier received a warning about switching arpack from +accelerate to +atlas to avoid triggering a floating point bug on Apple, but have not yet done that as I feel that concurrent changes will only confuse thing further.

I am attaching the arpack log file, as well as the results of "port list inactive"...

[andrewhw (Andrew Wright)]

/opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_math_arpack/arpack/main.log

[andrewhw (Andrew Wright)]

output of "port list inactive"

[andrewhw (Andrew Wright)]

Sorry for the edit -- I had included "sudo port configure openmpi-default" in the trace twice, when the second one should have been the attempted activate of octave. I have edited and updated the above post so that it now reads correctly.

[seanfarley (Sean Farley)]

You will probably have to uninstall the dependents of openmpi since ports built against the old version won't work.

$ sudo port uninstall --follow-dependents openmpi and installed
$ sudo port uninstall --follow-dependents openmpi-default

Then reinstall openmpi-default,

$ sudo port install openmpi-default

if that works, continue with arpack and octave. I'm going to close the ticket because I'm pretty sure that will fix this error but if something else is still wrong, definitely report back.