Multicore support for math/R

[webbp (Webb Phillips)]

Currently, math/R is built (by default) using Apple's clang. This causes both R itself and any packages installed by R to be single-core because Apple clang doesn't support OpenMP. This very much sucks and is ultimately Apple's fault. However, there is an easy solution, which is to build R with Macports clang. More specifically, when I install macports clang-10 llvm-10 and libomp, port select clang mp-clang-10 and llvm mp-llvm-10, and macports PATH is prefixed to my PATH, sudo port install R installs a multicore R capable of building multicore R packages. Anecdotally, this R on my 9-year-old MacBook Air is faster for many common operations than the default Macports R on my 2020 MacBook Air.

The vast majority of users will not know how to do this. Therefore, please update the default R build to build a multicore-capable R.

[mf2k (Frank Schima)]

Note that a "request" ticket type is only for requesting a new port.

In the future, please add the port maintainer(s) to Cc (port info --maintainers R), if any.

[i0ntempest]

How do I know if R on my system has multicore support? If it's that easy as you said, I might just blacklist apple clang.

[webbp (Webb Phillips)]

My description of the solution isn't quite right. The way I solved this was with:

~/.R/Makevars:
LLVM_LOC = /opt/local/libexec/llvm-10
CC=$(LLVM_LOC)/bin/clang -fopenmp
CXX=$(LLVM_LOC)/bin/clang++ -fopenmp
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L/opt/local/lib -L$(LLVM_LOC)/lib -Wl,-rpath,$(LLVM_LOC)/lib
CPPFLAGS=-I/opt/local/include -I$(LLVM_LOC)/include

That alone on my new machine breaks installing R packages, so there are some missing steps, and some of the other steps I took above may have been irrelevant.

[i0ntempest]

Are those just for compiling R packages? I'm not very experienced in manipulating complex builds (like R) yet, so can you try building R with multicore support on your new machine and tell me exactly what change did you make? Then I could try putting them into the portfile.

[chrstphrchvz (Christopher Chavez)]

compiler.openmp_version should be used to specify OpenMP requirement; an OpenMP-capable compiler will be chosen and libomp dependency added automatically. See CompilerSelection#GettingtheRightCompilerinthePortfile

[webbp (Webb Phillips)]

compiler.openmp_version 4.5 did not enable R to build packages with OpenMP, e.g., with compiler.openmp_version 4.5:

clang: error: unknown argument: '-pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64'
*** OpenMP not supported! data.table uses OpenMP to automatically
***   parallelize operations like sorting, grouping, file reading, etc.

I've investigated the problem a bit more. The issue mostly or entirely affects packages installed by R rather than the R binary itself, that is, there are many R packages which benefit greatly from being built with openmp, but the R binary may not. The above .R/Makevars cause R to build packages with OpenMP in Mac OS 10.11, whereas these below work for 10.15. (Those are the only two Mac OS versions I have readily available.)

CC=/opt/local/bin/clang -fopenmp
CXX=/opt/local/bin/clang++ -fopenmp
# -O3 should be faster than -O2 (default) level optimisation ..
CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CCFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
CXXFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L/opt/local/lib -Wl,-rpath,/opt/local/lib
CPPFLAGS=-I/opt/local/include

Source: ​https://github.com/Rdatatable/data.table/wiki/Installation

Also, I believe that R uses the flags with which it was built as the default flags for building packages, so specifying these flags the Portfile should solve the problem. However, the specific flags that work differ for different Mac OS versions, and they are also known to be incompatible with some R packages. So maybe the best thing to do here is just to add some lines to the Portfile long_description, e.g.: to install multicore packages in R, do sudo port install clang-10 pkgconfig && sudo port select clang mp-clang-10 and create ~/.R/Makevars as above.

[i0ntempest]

It's working for me:

* installing *source* package ‘data.table’ ...
** package ‘data.table’ successfully unpacked and MD5 sums checked
** using staged installation
zlib 1.2.11 is available ok
OpenMP supported
** libs
/opt/local/bin/clang-mp-10 -fopenmp -I"/opt/local/Library/Frameworks/R.framework/Resources/include" -DNDEBUG   -I/opt/local/include  -fopenmp -fPIC  -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe -c assign.c -o assign.o
In file included from assign.c:1:
In file included from ./data.table.h:13:
In file included from ./myomp.h:2:
/opt/local/include/libomp/omp.h:53:9: warning: ISO C restricts enumerator values to range of 'int' (2147483648 is too large) [-Wpedantic]
        omp_sched_monotonic = 0x80000000
        ^                     ~~~~~~~~~~
...

My Makevar file is same as the one you have except I explicitly specified /opt/local/bin/clang-mp-10 and /opt/local/bin/clang++-mp-10. I've added openmp in as a variant, do you wanna try my portfile?

[i0ntempest]

Also I just tried with the default macports R, it can build openmp packages despite itself is not built explicitly with openmp support (maybe it's automatically enabled). But it's still better to add a variant since it'll ensure one has the correct compiler and openmp library installed.

[i0ntempest]

PR is up ​https://github.com/macports/macports-ports/pull/8614

[i0ntempest]

In 71899c63ad7c5953b9313aeca35bd7aa44c019c9/macports-ports (master):

R: Add openmp variant & change openblas variant to lowercase (​https://github.com/macports/macports-ports/pull/8614)

This new variant makes sure the user has OpenMP library and a OpenMP supported compiler installed to build and use OpenMP enabled R packages. OpenMP support is now explicitly disabled before build if the variant is not selected to avoid undeclared dependency.
Closes: #61256