openmpi-gcc12 @4.1.6_0+fortran: problem linking MPI-2 memory routines from c++

[floquet-cxx]

The application uses C++, C & F77, and has used MPI-1 routines (and openmpi) for many years.

When I started using some MPI-2 routines to do parallel I/O, some linkages failed, I believe associated with MPI_File_set_view (and perhaps with MPI_File_write_all, MPI_File_read_all - these 3 routines were the only new ones introduced). The routines (in fact the whole code) linked and worked fine with mpich (though I currently have a separate issue with mpich and cmake, for which I plan a separate ticket), so I think the problem relates to openmpi. The top-level source is C++.

Mac Studio, Sonoma/14.2.1 and Xcode15.2, but had the same problems earlier (with Ventura and Xcode14.3.1). I did a clean reinstall of macports and required ports for Sonoma. Also had same problem using gcc13 variant of openmpi.

Here is a cut-down summary of the linkage errors issued by cmake:

[50%] Linking CXX executable elliptic_mp
Undefined symbols for architecture arm64:
  "__ZN3MPI3Win4FreeEv", referenced from:
...
 "__ZN3MPI4CommC2Ev", referenced from:
...
"__ZN3MPI8Datatype4FreeEv", referenced from:
...
"_ompi_mpi_cxx_op_intercept", referenced from:

...
ld: symbol(s) not found for architecture arm64

Perhaps this is just associated with C++ mangling of the names, and a missing "extern" in a header file?

[floquet-cxx]

I went to the openmpi team with these concerns. After some exchanges, it seems to me that the openmpi preprocessing system is broken with openmpi 4.1.6 (and maybe some earlier versions - since openmpi used to work for me a while back, I think it was probably OK at one stage). When used with g++ (either Apple or gnu) as the C++ compiler (which is what cmake is handing me), a bunch of MPI routines get asked for once mpi.h is included. The ones reported for my errors are actually MPI routines with C++ bindings, even though I'm only using C bindings.

For example (were NO mpi routines are used):

semtex-xxt (xxt) >$ cat foo.cpp
#include <mpi.h>

int main() { return 0 ; }
semtex-xxt (xxt) >$ gcc -I /opt/local/include/openmpi-mp foo.cpp
-macosx_version_min has been renamed to -macos_version_min
Undefined symbols for architecture arm64:
  "_MPI_Abort", referenced from:
      __ZN3MPI4Comm5AbortEi in ccZsRNtd.o
  "_MPI_Accumulate", referenced from:
      __ZNK3MPI3Win10AccumulateEPKviRKNS_8DatatypeEiliS5_RKNS_2OpE in ccZsRNtd.o
  "_MPI_Allgather", referenced from:
      __ZNK3MPI4Comm9AllgatherEPKviRKNS_8DatatypeEPviS5_ in ccZsRNtd.o
  "_MPI_Allgatherv", referenced from:
      __ZNK3MPI4Comm10AllgathervEPKviRKNS_8DatatypeEPvPKiS8_S5_ in ccZsRNtd.o
  "_MPI_Allreduce", referenced from:
      __ZNK3MPI4Comm9AllreduceEPKvPviRKNS_8DatatypeERKNS_2OpE in ccZsRNtd.o
  "_MPI_Alltoall", referenced from:
      __ZNK3MPI4Comm8AlltoallEPKviRKNS_8DatatypeEPviS5_ in ccZsRNtd.o
  "_MPI_Alltoallv", referenced from:
      __ZNK3MPI4Comm9AlltoallvEPKvPKiS4_RKNS_8DatatypeEPvS4_S4_S7_ in ccZsRNtd.o
  "_MPI_Alltoallw", referenced from:
      __ZNK3MPI4Comm9AlltoallwEPKvPKiS4_PKNS_8DatatypeEPvS4_S4_S7_ in ccZsRNtd.o
...
... many more lines ...
...
"__Znwm", referenced from:
      __ZNK3MPI9Intracomm5CloneEv in ccZsRNtd.o
      __ZNK3MPI8Cartcomm5CloneEv in ccZsRNtd.o
      __ZNK3MPI9Graphcomm5CloneEv in ccZsRNtd.o
      __ZNK3MPI9Intercomm5CloneEv in ccZsRNtd.o
  "___cxa_pure_virtual", referenced from:
      __ZTVN3MPI4CommE in ccZsRNtd.o
  "___cxa_throw_bad_array_new_length", referenced from:
      __ZNK3MPI8Datatype12Get_contentsEiiiPiPlPS0_ in ccZsRNtd.o
      __ZNK3MPI4Comm9AlltoallwEPKvPKiS4_PKNS_8DatatypeEPvS4_S4_S7_ in ccZsRNtd.o
      __ZNK3MPI9Intracomm11Create_cartEiPKiPKbb in ccZsRNtd.o
      __ZN3MPI9Intracomm24convert_info_to_mpi_infoEiPKNS_4InfoE in ccZsRNtd.o
      __ZNK3MPI8Cartcomm8Get_topoEiPiPbS1_ in ccZsRNtd.o
      __ZNK3MPI8Cartcomm3SubEPKb in ccZsRNtd.o
      __ZNK3MPI8Cartcomm3MapEiPKiPKb in ccZsRNtd.o
      ...
  "___gxx_personality_v0", referenced from:
      Dwarf Exception Unwind Info (__eh_frame) in ccZsRNtd.o
  "_ompi_mpi_comm_null", referenced from:
      __ZN3MPI9IntracommC1EP19ompi_communicator_t in ccZsRNtd.o
      __ZN3MPI8CartcommC1ERKP19ompi_communicator_t in ccZsRNtd.o
      __ZN3MPI9GraphcommC1ERKP19ompi_communicator_t in ccZsRNtd.o
  "_ompi_mpi_cxx_op_intercept", referenced from:
      __ZN3MPI2Op4InitEPFvPKvPviRKNS_8DatatypeEEb in ccZsRNtd.o
  "_ompi_op_set_cxx_callback", referenced from:
      __ZN3MPI2Op4InitEPFvPKvPviRKNS_8DatatypeEEb in ccZsRNtd.o
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status
semtex-xxt (xxt) >$ 

Now with mpich:

semtex-xxt (xxt) >$ gcc -I /opt/local/include/mpich-gcc12 foo.cpp
-macosx_version_min has been renamed to -macos_version_min
semtex-xxt (xxt) >$ nm a.out | grep MPI
semtex-xxt (xxt) >$ 

Everything is as one would expect: no MPI routines requested, none pulled by the preprocessor.

The openmpi team suggested that these issues will be/are resolved in openmpi-5 (and that they are unlikely to be fixed in older/current versions), but gave some workarounds for 4.1.6. For example, to use mpicxx in place of g++ --- this works for foo.cpp; at least the "spurious" MPI routines are resolved. However, for compiling my larger code, that workaround causes still further errors which I'm so far unable to fix. I am a bit frustrated with trying to resolve this.

So, I will see is I can get mpich to work with cmake (you may see I presently have another issue with that not finding the MPI_C and MPI_CXX compilers, but I suspect that may be easier to fix). Maybe using openmpi-5 will work, too; I'm guessing you might be working to adopt that in macports.