mpich: several subports failing on certain OS versions

[jmroot (Joshua Root)]

Similar to #62878 for openmpi, a lot of the subports of mpich fail to build on some subset of OS versions. If they can be fixed immediately, great, please do so; if they can't be fixed or it will take some time, please set known_fail yes and remove if/when they are fixed.

[mascguy (Christopher Nielsen)]

As part of our simplification/refactoring effort (issue:62807), we'll use shared logic regarding known failures and such. But that's a few weeks out, so we should probably add some immediate fixes.

Ultimately I'd really like to share the same logic across not only openmpi/mpich, but also among our various portgroups: The latter is going to take plenty of thought, planning, and effort. (Not to mention a LOT of local testing!) But if we can eliminate logic duplication, it would make things a lot simpler.

[Christopher Nielsen <mascguy@…>]

In 1e95463a3fdae55dc339b60a2ff8e24d0cda2e7e/macports-ports (master):

mpich: disable gcc9+ subports for MacOS 10.7, as builds fail

See: #62887

[kencu (Ken)]

success on 10.7 (with AVX enabled, as well):

$ port -v installed openmpi-gcc10
The following ports are currently installed:
  openmpi-gcc10 @4.1.1_0+fortran (active) platform='darwin 11' archs='x86_64' date='2021-05-14T19:55:14-0700'

SO -- how....

all the gcc versions on MacPorts (and all the macports-clang-N compilers) are built to use cctools assembler ${prefix}/bin/as. For all older systems this is the cctools-supplied version (for newer systems it might be a symlink to the one provided by Xcode).

${prefix}/bin/as has been set up to use a newer clang as the assembler if a newer clang is available, and it looks for macports-clang-9.0 ==> macports-clang-5.0 and then clang on 10.7+.

clang on 10.7 and 10.8 is very old now, and it doesn't understand the newer assembly instructions. macports-clang-9.0 is pretty recent, and easy to build for all systems all the way back to 10.5 Leopard Intel. So it's a pretty good choice for an assembler right now.

It understands the failing instruction, and also understands the previously-failed AVX instructions.

SO you can re-enable openmip-gcc10 on 10.7, and the AVX instructions on 10.7 and 10.8, if you make sure that clang-9.0 is installed when openmpi-gcc10 (etc) is built.

Something like this:

    # For MacOS 10.8 and earlier, system-provided clang assembler doesn't support newer instructions
    if {${os.major} <= 12} {
        ui_debug "installed clang-9.0 to use as assembler"
        depends_build-append port:clang-9.0
    }

I'm not at this second 100% sure if it is also needed at runtime as an assembler. It might be, and for caution, I would suggest perhaps it just be a depends_lib-append so it's always available.

[mascguy (Christopher Nielsen)]

While this fix works for openmpi, I'm seeing other issues with mpich that aren't related to AVX instructions.

So we still have more unrelated work to do for latter, if we want to fix the 10.7 builds.

[kencu (Ken)]

can you give me a specific port to fix? There are so many variations.

Which one do you need fixed?

[mascguy (Christopher Nielsen)]

Yes, it's difficult to keep track of what issues are occurring where.

For the 10.7 gcc9 case, it looks like configure is complaining about a lack of a C99-compliant compiler:

checking for /opt/local/bin/gcc-mp-9 option to accept ISO C99... unsupported
configure: WARNING: Libfabric requires a C99-compliant compiler
configure: error: Cannot continue
configure: error: libfabric configure failed

​https://build.macports.org/builders/ports-10.7_x86_64-builder/builds/52814/steps/install-port/logs/stdio

[mascguy (Christopher Nielsen)]

The C99-compliant compiler issue is also occurring for 10.7/gcc10:

​https://build.macports.org/builders/ports-10.7_x86_64-builder/builds/52880

[kencu (Ken)]

ok, I'll look.

clearly gcc supports C99, so whatever it is, it's not that.

[mascguy (Christopher Nielsen)]

Exactly, which is why I simply punted on trying to diagnose the issue.

[Christopher Nielsen <mascguy@…>]

In 6ff08717228b7f28b09c463555a3004ac9416f6d/macports-ports (master):

pg mpi: disable gcc11 for mpich unconditionally

See: #62887

[mascguy (Christopher Nielsen)]

The list of failures has been reduced significantly, thanks to the completed pruning effort; see issue:62807.

There's still more work to be done, but getting there.

[Christopher Nielsen <mascguy@…>]

In 83662481aa05dd5ffa5dfbb9857b4d1d2eae0782/macports-ports (master):

openmpi/mpich: gcc builds: add clang 9 build dependency for macOS 10.8 and earlier

• clang 9 provides a modern assembler, allowing openmpi/mpich to build successfully

See: #62878
See: #62887

[Christopher Nielsen <mascguy@…>]

In 733aa5de852c084e87c7a01f595975efe6454bc7/macports-ports (master):

openmpi/mpich: disable Xcode clang builds for MacOS 10.6 and earlier

See: #62878
See: #62887

[Christopher Nielsen <mascguy@…>]

In 15d5dc2fe3fb22241ca35a1f93afcb25024b2795/macports-ports (master):

openmpi/mpich: disable default (gcc) build when necessary; cleanup

See: #62878
See: #62887

[Christopher Nielsen <mascguy@…>]

In 8ea6c676e0df04f9f90b2b9eb6190901e20a5e03/macports-ports (master):

openmpi/mpich: refactor note generation to pg mpiutil; cleanup

See: #62878
See: #62887

[mascguy (Christopher Nielsen)]

The vast majority of failures have either been fixed, or disabled. The few remaining are due to missing compiler dependencies - for example, clang 11 is not yet available for macOS 10.6 - but these fail quickly during the dependency install phase.

Marking as fixed.

[mascguy (Christopher Nielsen)]

Several of the disabled subports were still trying to build, due to unintended dependencies. (Both on the default subport, as well as legacy-support for earlier macOS releases.)

That's been fixed as of this writing, and that should further reduce any wasted buildbot time.

[mascguy (Christopher Nielsen)]

The aforementioned fixed are present in the following two commits:

​https://github.com/macports/macports-ports/commit/883e402ae5c48131366196332fc874ed0a26ecb2

​https://github.com/macports/macports-ports/commit/3c28142a5299c165c1a38a60a819bd82229d0c67