Opened 6 months ago

Last modified 4 weeks ago

#59022 assigned defect

py-numpy: Anything that imports numpy crashes python on sierra/10.12 in a vm/virtual machine

Reported by: ryandesign (Ryan Schmidt) Owned by: michaelld (Michael Dickens)
Priority: Normal Milestone:
Component: ports Version:
Keywords: sierra Cc: jmroot (Joshua Root), stromnov (Andrew Stromnov), reneeotten (Renee Otten), JackDunnNZ (Jack Dunn)
Port: py-numpy

Description

py36-matplotlib and py37-matplotlib do not build:

running build_ext
sh: line 1: 98187 Illegal instruction: 4  /opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 setup.py --no-user-cfg build
Command failed:  cd "/opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_python_py-matplotlib/py37-matplotlib/work/matplotlib-3.1.1" && /opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 setup.py --no-user-cfg build 
Exit code: 132

py35-healpy fails with a similar error

Change History (10)

comment:1 Changed 6 months ago by reneeotten (Renee Otten)

It seems this only happens for matplotlib version 3.1.1 and specific versions of macOS. I am currently traveling, but will take a closer look after next week.

comment:2 Changed 5 months ago by reneeotten (Renee Otten)

I am not really sure how to proceed... I can't really tell why it does not work on macOS 10.12 (on 10.11 and lower the failures seem to be caused by its dependencies).

The failure on macOS 10.12 happens after: running build_ext

On macOS 10.13 and 10.14, the build continues after that with the following:

building 'matplotlib.ft2font' extension
creating build/temp.macosx-10.13-x86_64-3.7
creating build/temp.macosx-10.13-x86_64-3.7/src
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -arch x86_64 -DFREETYPE_BUILD_TYPE=system -DPY_ARRAY_UNIQUE_SYMBOL=MPL_matplotlib_ft2font_ARRAY_API -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -D__STDC_FORMAT_MACROS=1 -Iextern/agg24-svn/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/numpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c src/checkdep_freetype2.c -o build/temp.macosx-10.13-x86_64-3.7/src/checkdep_freetype2.o -I/opt/local/include/freetype2 -I/opt/local/include/libpng16
src/checkdep_freetype2.c:7:9: warning: Compiling with FreeType version 2.10.1. [-W#pragma-messages]
#pragma message("Compiling with FreeType version " \
        ^
1 warning generated.

somehow it appears that this clang command does not work on macOS 10.12. I do not have access to a machine with that OS installed and don't really know how to continue with this - does anyone have a suggestion?

comment:3 Changed 5 months ago by reneeotten (Renee Otten)

also py35-svipc fails with a similar error and only on macOS 10.12.

To me this suggests it's something specific with macOS 10.12, and it started to happen more recently: at least py{36/37}-matplotlib used to build fine at the time of the last update (July 3) (and before the rev-bump due to qhull on August 26). From the log file it seems unlikely that the qhull update has anything to do with it, so is there anything else that has changed on the buildbot in between?

comment:4 Changed 4 months ago by ryandesign (Ryan Schmidt)

In #59383 we've found that it seems to be a NumPy problem.

comment:5 Changed 6 weeks ago by ryandesign (Ryan Schmidt)

Cc: jmroot stromnov reneeotten added
Keywords: sierra added
Owner: changed from reneeotten to michaelld
Port: py-numpy added; py-healpy py-matplotlib removed
Summary: py35-healpy, py36-matplotlib, py37-matplotlib: Illegal instruction: 4py-numpy: Anything that imports numpy crashes python on sierra/10.12 in a vm/virtual machine

Has duplicates #59383, #59979. Crash logs are attached to #59979.

The upstream bug report we have been presuming this relates to is https://github.com/numpy/numpy/issues/10330. However it is closed because it was thought not to be a numpy bug but rather a defect in the Xen hypervisor that that user was using. There is a sample program which was reporting that AVX was supported but trying to use AVX functions was causing a crash.

I have tried that test program on our Sierra and High Sierra buildbot workers. On both, it reports that AVX is not available. Also, the upstream bug report says the problem occurs when doing "simple array additions", but in our case, the problem occurs immediately after "import numpy". So perhaps our problem is not the same one.

Our buildbot workers are virtual machines (running under VMware ESXi on Xserves) so that part may still be relevant. If anybody experiences this problem on Sierra running directly on a Mac, not in a VM, let us know.

comment:6 Changed 6 weeks ago by ryandesign (Ryan Schmidt)

A more likely upstream bug report is https://github.com/numpy/numpy/issues/13059 which is still open and in which a user on Linux reports "import numpy" crashing in a vm with a backtrace that looks slightly different from ours but we're both crashing in npy_cpu_supports.

comment:7 Changed 6 weeks ago by ryandesign (Ryan Schmidt)

I wrote up my own upstream bug report: https://github.com/numpy/numpy/issues/15342; we'll see what they say.

comment:8 Changed 5 weeks ago by lpsinger (Leo Singer)

Cc: lpsinger removed

comment:9 Changed 5 weeks ago by JackDunnNZ (Jack Dunn)

Cc: JackDunnNZ added

comment:10 Changed 4 weeks ago by ryandesign (Ryan Schmidt)

They pointed me to a PR from April that solves the problem by rewriting their CPU detection code. But it conflicts with another PR they want so I'm not sure when it will be merged.

Note: See TracTickets for help on using tickets.