py37-scipy: broken port fails rebuild

[mf2k (Frank Schima)]

I am not able to get a working py37-scipy. It keeps trying to rebuild and fails after 3 attempts. I'm not sure if this is related to #56954? Here are my relevant installed ports.

$ port installed OpenBLAS-devel ld64 cctools py37-numpy py37-scipy
The following ports are currently installed:
  cctools @921_1+llvm70 (active)
  ld64 @3_1+ld64_xcode (active)
  OpenBLAS-devel @20190210_0+gcc8+lapack (active)
  py37-numpy @1.16.1_0+gcc8+openblas (active)
  py37-scipy @1.2.1_0+gcc8+openblas (active)

I have tried various combinations with no success including using openblas vs. openblas-devel and installing py37-numpy without +openblas. Can anyone share a working combo?

Error is:

--->  Found 1 broken port, determining rebuild order
You can always run 'port rev-upgrade' again to fix errors.
The following ports will be rebuilt: py37-scipy @1.2.1+gcc8+openblas
Continue? [Y/n]: 
--->  Computing dependencies for py37-scipy
--->  Cleaning py37-scipy
--->  Scanning binaries for linking errors
--->  No broken files found.                             
--->  Found 1 broken port, determining rebuild order
--->  Rebuilding in order
     py37-scipy @1.2.1 +gcc8+openblas
--->  Computing dependencies for py37-scipy
--->  Fetching distfiles for py37-scipy
--->  Verifying checksums for py37-scipy
--->  Extracting py37-scipy
--->  Configuring py37-scipy
--->  Building py37-scipy
--->  Staging py37-scipy into destroot
--->  Unable to uninstall py37-scipy @1.2.1_0+gcc8+openblas, the following ports depend on it:
--->  	py37-pandas @0.24.1_0
--->  	py37-spyder @3.3.3_0
Warning: Uninstall forced.  Proceeding despite dependencies.
--->  Deactivating py37-scipy @1.2.1_0+gcc8+openblas
--->  Cleaning py37-scipy
--->  Uninstalling py37-scipy @1.2.1_0+gcc8+openblas
--->  Cleaning py37-scipy
--->  Computing dependencies for py37-scipy
--->  Installing py37-scipy @1.2.1_0+gcc8+openblas
--->  Activating py37-scipy @1.2.1_0+gcc8+openblas
--->  Cleaning py37-scipy
--->  Scanning binaries for linking errors
--->  No broken files found.                             
--->  Found 1 broken port, determining rebuild order
--->  Rebuilding in order
     py37-scipy @1.2.1 +gcc8+openblas
--->  Computing dependencies for py37-scipy
--->  Fetching distfiles for py37-scipy
--->  Verifying checksums for py37-scipy
--->  Extracting py37-scipy
--->  Configuring py37-scipy
--->  Building py37-scipy
...

[mf2k (Frank Schima)]

It looks like the problem is this:

py37-scipy is using libstdc++ (this installation is configured to use libc++)

[reneeotten (Renee Otten)]

hi Frank, I haven't seen this issue... I have the following (relevant) packages installed:

> port installed OpenBLAS ld64 cctools py37-numpy py37-scipy
The following ports are currently installed:
  cctools @921_1+xcode (active)
  ld64 @3_1+ld64_xcode (active)
  OpenBLAS @0.3.5_1+gcc8+lapack (active)
  py37-numpy @1.16.1_0+gcc8+openblas (active)
  py37-scipy @1.2.1_0+gcc8+openblas (active)

it appears the only difference is in cctools where I have the +xcode variant whereas you have +llvm70. There have been too many issues with py-numpy and py-scipy recently... unfortunately they all seem to be compiler related and I am not familiar with most of that to figure out the cause/solution, but it would be really nice if this could get fixed!

[mf2k (Frank Schima)]

@reneeotten: Thanks for responding. My understanding, on High Sierra at least, that we are supposed to use cctools +llvm70. There was a discussion about it on the mailing list and the default variant was changed to that. Anyway, Here is the combo that works for me.

$ port installed OpenBLAS ld64 cctools py37-numpy py37-scipy
The following ports are currently installed:
  cctools @921_1+xcode (active)
  ld64 @3_1+ld64_xcode (active)
  OpenBLAS @0.3.5_1+gcc8+lapack (active)
  py37-numpy @1.16.1_0+gcc8+openblas (active)
  py37-scipy @1.2.1_0+gfortran+openblas (active)

I don't like having to use different variants for scipy and numpy and wonder if that might cause problems?

[mf2k (Frank Schima)]

Oh hah. Let me rebuild my cctools and ld64.

[mf2k (Frank Schima)]

Well that did not help. So here is the only working configuration for me so far.

$ port installed OpenBLAS ld64 cctools py37-numpy py37-scipy
The following ports are currently installed:
  cctools @921_1+llvm70 (active)
  ld64 @3_1+ld64_xcode (active)
  OpenBLAS @0.3.5_1+gcc8+lapack (active)
  py37-numpy @1.16.1_0+gcc8+openblas (active)
  py37-scipy @1.2.1_0+gfortran+openblas (active)

[michaelld (Michael Dickens)]

Interesting: scipy imports correctly for both +gcc8+openblas and+gfortran+openblas; here are the test results for my setup (OSX 10.14 latest, MP latest, latest ports):

*** py27-scipy @1.2.1_0+gfortran+openblas

38 failed, 14028 passed, 1326 skipped, 1223 deselected, 77 xfailed, 7 xpassed, 39583 warnings

*** py27-scipy @1.2.1_0+gcc8+openblas

26 failed, 14040 passed, 1326 skipped, 1223 deselected, 77 xfailed, 7 xpassed, 39579 warnings

I'd say what you've found is a MP base issue.

[michaelld (Michael Dickens)]

Well ... maybe or not. If I do

otool -L $(port contents py27-scipy | grep "\.so" | grep -v plist | grep -v dSYM) | grep -v : | sort -u

Then for py27-scipy @1.2.1_0+gfortran+openblas I get back:

        /opt/local/lib/libgcc/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/opt/local/lib/libgcc/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	/opt/local/lib/libgcc/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/opt/local/lib/libopenblas-r1.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)

while for I get back:

	/opt/local/lib/libgcc/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
	/opt/local/lib/libgcc/libgfortran.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	/opt/local/lib/libgcc/libquadmath.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/opt/local/lib/libgcc/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.25.0)
	/opt/local/lib/libopenblas-r1.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.200.5)

.... so, +gcc8 does link against libstdc++ while +gfortran links against libc++ ... hmmmm ...

[michaelld (Michael Dickens)]

OK so the difference is that with +gfortran, the compiler used to link C++ object code is /usr/bin/clang++, while for +gcc8 it is /opt/local/bin/g++-mp-8 ... so ... we need to not use the whole compiler suite, but rather just the fortran compiler from the selected variant if the variant supports doing so (as all GNU ones would).

[mf2k (Frank Schima)]

This is working for me now.

[mf2k (Frank Schima)]

$ port installed OpenBLAS ld64 cctools py37-numpy py37-scipy
The following ports are currently installed:
  cctools @921_1+llvm70 (active)
  ld64 @3_1+ld64_xcode (active)
  OpenBLAS @0.3.5_1+gcc8+lapack (active)
  py37-numpy @1.16.2_1+gfortran+openblas (active)
  py37-scipy @1.2.1_0+gfortran+openblas (active)