Opened 10 months ago
Closed 10 months ago
#67900 closed defect (fixed)
pyarrow Segmentation Fault on Import
| Reported by: | essandess (Steve Smith) | Owned by: | essandess (Steve Smith) |
|---|---|---|---|
| Priority: | Normal | Milestone: | |
| Component: | ports | Version: | |
| Keywords: | Cc: | mascguy (Christopher Nielsen) | |
| Port: | apache-arrow |
Description
The ports py*-pyarrow create a segfault on import. This happens on both arm64 and x86_64 boxes, installed from prebuilt or built from source, using either the current MacPorts version and the latest available version from https://github.com/macports/macports-ports/pull/19664.
$ port installed py310-pyarrow The following ports are currently installed: py310-pyarrow @12.0.0_3 (active) $ PYTHONFAULTHANDLER=1 python3.10 -c 'import pyarrow' Fatal Python error: Segmentation fault Current thread 0x00007ff84d123700 (most recent call first): File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 1176 in create_module File "<frozen importlib._bootstrap>", line 571 in module_from_spec File "<frozen importlib._bootstrap>", line 674 in _load_unlocked File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1027 in _find_and_load File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyarrow/__init__.py", line 65 in <module> File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed File "<frozen importlib._bootstrap_external>", line 883 in exec_module File "<frozen importlib._bootstrap>", line 688 in _load_unlocked File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1027 in _find_and_load File "<string>", line 1 in <module> Segmentation fault: 11
Change History (7)
comment:1 Changed 10 months ago by essandess (Steve Smith)
comment:2 Changed 10 months ago by mascguy (Christopher Nielsen)
| Cc: | mascguy added |
|---|
comment:3 Changed 10 months ago by jmroot (Joshua Root)
The information in the macOS crash report would probably be helpful.
comment:4 Changed 10 months ago by essandess (Steve Smith)
Same crash dump as in https://trac.macports.org/ticket/67896:
FWIW, I've tried setting a few more cmake flags to be more consistent with upstream's build script at https://github.com/apache/arrow/blob/main/ci/scripts/python_wheel_macos_build.sh, but keep hitting the same issue.
Process: Python [51251]
Path: /opt/local/Library/Frameworks/Python.framework/Versions/3.10/Resources/Python.app/Contents/MacOS/Python
Identifier: org.python.python
Version: 3.10.12 (3.10.12)
Code Type: X86-64 (Native)
Parent Process: bash [15799]
Responsible: Terminal [15797]
User ID: 502
Date/Time: 2023-08-03 16:08:12.4564 -0400
OS Version: macOS 13.5 (22G74)
Report Version: 12
Bridge OS Version: 7.6 (20P6072)
Time Awake Since Boot: 810000 seconds
Time Since Wake: 607027 seconds
System Integrity Protection: enabled
Crashed Thread: 0 Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000012340
Exception Codes: 0x0000000000000001, 0x0000000000012340
Termination Reason: Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process: exc handler [51251]
VM Region Info: 0x12340 is not in any region. Bytes before following region: 4352785600
REGION TYPE START - END [ VSIZE] PRT/MAX SHRMOD REGION DETAIL
UNUSED SPACE AT START
--->
__TEXT 103736000-10373a000 [ 16K] r-x/r-x SM=COW .../MacOS/Python
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libjemalloc.2.dylib 0x1059ec3df je_free_default + 193
1 libprotobuf.3.21.12.0.dylib 0x105688477 google::protobuf::internal::ArenaStringPtr::Destroy() + 39
2 libprotobuf.3.21.12.0.dylib 0x10571d5a5 google::protobuf::FileDescriptorProto::SharedDtor() + 149
3 libprotobuf.3.21.12.0.dylib 0x10571d6bd google::protobuf::FileDescriptorProto::~FileDescriptorProto() + 45
4 libprotobuf.3.21.12.0.dylib 0x105738657 google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) + 167
5 libprotobuf.3.21.12.0.dylib 0x1056e34f4 google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) + 36
6 libprotobuf.3.21.12.0.dylib 0x1057544d9 google::protobuf::(anonymous namespace)::AddDescriptors(google::protobuf::internal::DescriptorTable const*) + 105
7 dyld 0x7ff81b4f93fb invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 175
8 dyld 0x7ff81b537b7a invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 242
9 dyld 0x7ff81b52bf22 invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 577
10 dyld 0x7ff81b4dc0af dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 245
11 dyld 0x7ff81b52b0bf dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 175
12 dyld 0x7ff81b53773a dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 470
13 dyld 0x7ff81b4f666c dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 220
14 dyld 0x7ff81b4f685a dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 178
…
comment:5 Changed 10 months ago by essandess (Steve Smith)
Here's the lldb trace:
/opt/local/bin/lldb-mp-16 (lldb) command script import pyarrow PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump: 0. Program arguments: /opt/local/libexec/llvm-16/bin/lldb 1. HandleCommand(command = "command script import pyarrow") Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 libLLVM.dylib 0x0000000113a91c6c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56 1 libLLVM.dylib 0x0000000113a90c38 llvm::sys::RunSignalHandlers() + 112 2 libLLVM.dylib 0x0000000113a922f8 SignalHandler(int) + 344 3 libsystem_platform.dylib 0x000000018d386a24 _sigtramp + 56 4 libjemalloc.2.dylib 0x000000010f5940e0 je_free_default + 1104 5 libprotobuf.3.21.12.0.dylib 0x000000010f2471f0 google::protobuf::internal::ArenaStringPtr::Destroy() + 56 6 libprotobuf.3.21.12.0.dylib 0x000000010f2d0000 google::protobuf::FileDescriptorProto::SharedDtor() + 144 7 libprotobuf.3.21.12.0.dylib 0x000000010f2d00e4 google::protobuf::FileDescriptorProto::~FileDescriptorProto() + 48 8 libprotobuf.3.21.12.0.dylib 0x000000010f2ea634 google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) + 168 9 libprotobuf.3.21.12.0.dylib 0x000000010f29e878 google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) + 40 10 libprotobuf.3.21.12.0.dylib 0x000000010f3056a8 google::protobuf::(anonymous namespace)::AddDescriptors(google::protobuf::internal::DescriptorTable const*) + 136 11 libprotobuf.3.21.12.0.dylib 0x000000010f3056dc google::protobuf::internal::AddDescriptorsRunner::AddDescriptorsRunner(google::protobuf::internal::DescriptorTable const*) + 24 12 dyld 0x000000018d01c1d8 invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 168 13 dyld 0x000000018d05de94 invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 340 14 dyld 0x000000018d0511a4 invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 528 15 dyld 0x000000018cffc2d8 dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 296 16 dyld 0x000000018d0501cc dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 192 17 dyld 0x000000018d05d958 dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 516 18 dyld 0x000000018d01885c dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 448 19 dyld 0x000000018d018c10 dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 220 20 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 21 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 22 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 23 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 24 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184 25 dyld 0x000000018d01c264 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_1::operator()() const + 112 26 dyld 0x000000018d018d90 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 304 27 dyld 0x000000018d036d58 dyld4::APIs::dlopen_from(char const*, int, void*) + 1440 28 Python 0x000000010378ae64 _imp_create_dynamic + 616 29 Python 0x00000001036ce5d0 cfunction_vectorcall_FASTCALL + 80 30 Python 0x0000000103760d94 _PyEval_EvalFrameDefault + 50396 31 Python 0x0000000103763424 _PyEval_Vector + 116 32 Python 0x000000010368859c object_vacall + 224 33 Python 0x0000000103688458 PyObject_CallMethodObjArgs + 92 34 Python 0x00000001037873a8 PyImport_ImportModuleLevelObject + 1224 35 Python 0x000000010375b8f0 _PyEval_EvalFrameDefault + 28728 36 Python 0x0000000103753d74 PyEval_EvalCode + 168 37 Python 0x0000000103750174 builtin_exec + 332 38 Python 0x00000001036ce6cc cfunction_vectorcall_FASTCALL_KEYWORDS + 76 39 Python 0x0000000103760d94 _PyEval_EvalFrameDefault + 50396 40 Python 0x0000000103763424 _PyEval_Vector + 116 41 Python 0x000000010368859c object_vacall + 224 42 Python 0x0000000103688458 PyObject_CallMethodObjArgs + 92 43 Python 0x00000001037873a8 PyImport_ImportModuleLevelObject + 1224 44 Python 0x000000010375b8f0 _PyEval_EvalFrameDefault + 28728 45 Python 0x0000000103753d74 PyEval_EvalCode + 168 46 Python 0x00000001037a5d38 run_eval_code_obj + 84 47 Python 0x00000001037a5c9c run_mod + 112 48 Python 0x00000001037a8084 PyRun_StringFlags + 112 49 liblldb.16.0.6.dylib 0x000000010469effc lldb_private::python::runStringMultiLine(llvm::Twine const&, lldb_private::python::PythonDictionary const&, lldb_private::python::PythonDictionary const&) + 120 50 liblldb.16.0.6.dylib 0x00000001046a5838 lldb_private::ScriptInterpreterPythonImpl::ExecuteMultipleLines(char const*, lldb_private::ExecuteScriptOptions const&) + 960 51 liblldb.16.0.6.dylib 0x00000001046ab314 lldb_private::ScriptInterpreterPythonImpl::LoadScriptingModule(char const*, lldb_private::LoadScriptOptions const&, lldb_private::Status&, std::__1::shared_ptr<lldb_private::StructuredData::Object>*, lldb_private::FileSpec) + 2092 52 liblldb.16.0.6.dylib 0x0000000104777158 CommandObjectCommandsScriptImport::DoExecute(lldb_private::Args&, lldb_private::CommandReturnObject&) + 328 53 liblldb.16.0.6.dylib 0x00000001043469c0 lldb_private::CommandObjectParsed::Execute(char const*, lldb_private::CommandReturnObject&) + 484 54 liblldb.16.0.6.dylib 0x000000010433d2e8 lldb_private::CommandInterpreter::HandleCommand(char const*, lldb_private::LazyBool, lldb_private::CommandReturnObject&) + 2136 55 liblldb.16.0.6.dylib 0x00000001043409a0 lldb_private::CommandInterpreter::IOHandlerInputComplete(lldb_private::IOHandler&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) + 852 56 liblldb.16.0.6.dylib 0x00000001042731a0 lldb_private::IOHandlerEditline::Run() + 304 57 liblldb.16.0.6.dylib 0x0000000104256968 lldb_private::Debugger::RunIOHandlers() + 140 58 liblldb.16.0.6.dylib 0x0000000104341c14 lldb_private::CommandInterpreter::RunCommandInterpreter(lldb_private::CommandInterpreterRunOptions&) + 156 59 liblldb.16.0.6.dylib 0x00000001040a0614 lldb::SBDebugger::RunCommandInterpreter(bool, bool) + 124 60 lldb 0x0000000102ff48e0 Driver::MainLoop() + 2784 61 lldb 0x0000000102ff541c main + 2168 62 dyld 0x000000018cffff28 start + 2236 LLDB diagnostics will be written to /var/folders/2s/r9z6gsps7gn5wd89c0sqs8k80000gq/T/diagnostics-5f4976 Please include the directory content when filing a bug report Segmentation fault: 11
comment:6 Changed 10 months ago by essandess (Steve Smith)
This issue goes away if jemalloc isn’t used and is superseded by https://github.com/jemalloc/jemalloc/issues/2511.
comment:7 Changed 10 months ago by essandess (Steve Smith)
| Owner: | set to essandess |
|---|---|
| Resolution: | → fixed |
| Status: | new → closed |
Note: See
TracTickets for help on using
tickets.
Upstream issue: https://github.com/apache/arrow/issues/37010