neovim @0.6.0 +debug: segfaults during build process

[cooljeanius (Eric Gallager)]

Trying to upgrade from neovim 0.5.1 to 0.6.0 (with the +debug variant) results in the following build error for me:

make[2]: Entering directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build'
[100%] Generating syntax/vim/generated.vim
[100%] Generating doc/tags
[100%] Generating pack/dist/opt/matchit/doc/tags
[100%] Generating pack/dist/opt/vimball/doc/tags
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/pack/dist/opt/vimball && /opt/local/bin/cmake -E copy_directory /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/neovim-0.6.0/runtime/pack/dist/opt/vimball /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/pack/dist/opt/vimball
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/pack/dist/opt/matchit && /opt/local/bin/cmake -E copy_directory /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/neovim-0.6.0/runtime/pack/dist/opt/matchit /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/pack/dist/opt/matchit
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime && /opt/local/bin/cmake -E remove doc/*
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime && /opt/local/bin/luajit /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/neovim-0.6.0/scripts/genvimvim.lua /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/neovim-0.6.0/src/nvim /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/syntax/vim/generated.vim /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/funcs_data.mpack
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime && /opt/local/bin/cmake -E copy_directory /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/neovim-0.6.0/runtime/doc doc
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/pack/dist/opt/matchit && /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/pack/dist/opt/vimball && /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit
/bin/sh: line 1: 79771 Segmentation fault: 11  /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit
/bin/sh: line 1: 79770 Segmentation fault: 11  /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit
make[2]: *** [runtime/pack/dist/opt/vimball/doc/tags] Error 139
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [runtime/pack/dist/opt/matchit/doc/tags] Error 139
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime && /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ ++t\ doc -c quit
/bin/sh: line 1: 79773 Segmentation fault: 11  /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ ++t\ doc -c quit
make[2]: *** [runtime/doc/tags] Error 139
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build'
make[1]: *** [runtime/CMakeFiles/runtime.dir/all] Error 2
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build'
make: *** [all] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build'
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build" && /usr/bin/make -j16 -w all VERBOSE=ON 
Exit code: 2
Error: Failed to build neovim: command execution failed
DEBUG: Error code: CHILDSTATUS 77904 2

I'm on Big Sur (11.6.1) with Xcode 13.1.

[sainnhe (Sainnhe Park)]

Same, executing sudo port -vst install neovim will print detailed build information, here is the end of my full output:

cd /usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime/pack/dist/opt/vimball && /usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit cd /usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build/runtime && /usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ ++t\ doc -c quit /bin/sh: line 1: 82884 Segmentation fault: 11 /usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ ++t\ doc -c quit make[2]: * [runtime/doc/tags] Error 139 make[2]: * Waiting for unfinished jobs.... /bin/sh: line 1: 82882 Segmentation fault: 11 /usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit /bin/sh: line 1: 82881 Segmentation fault: 11 /usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit make[2]: * [runtime/pack/dist/opt/vimball/doc/tags] Error 139 make[2]: * [runtime/pack/dist/opt/matchit/doc/tags] Error 139 make[2]: Leaving directory `/usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build' make[1]: * [runtime/CMakeFiles/runtime.dir/all] Error 2 make[1]: Leaving directory `/usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build' make: * [all] Error 2 make: Leaving directory `/usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build' Command failed: cd "/usr/local/var/macports/build/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/work/build" && /usr/bin/make -j8 -w all VERBOSE=ON Exit code: 2 Error: Failed to build neovim: command execution failed Error: See /usr/local/var/macports/logs/_usr_local_var_macports_sources_mirrors.tuna.tsinghua.edu.cn_macports_release_tarballs_ports_editors_neovim/neovim/main.log for details. Error: Processing of port neovim failed

Any ideas?

[l2dy (Zero King)]

Could be related to our libtermkey port, which uses ncurses. ​https://github.com/neovim/neovim/issues/16190#issuecomment-955741551

libtermkey will use unibilium if it was installed. Also, libtermkey will use ncurses if unibilium is not installed and ncurses is installed. neovim recommends that you use unibilium to build libtermkey (The bundled libtermkey will be built using unibilium unless you change the cmake options). Perhaps your broken libtermkey is built using ncurses (This is indicated by openat(AT_FDCWD, "/lib/libncursesw.so.6", O_RDONLY|O_CLOEXEC) = 3 in strace).

libtermkey using ncurses has not been well tested by CI or other means, so it is not surprising that it has bugs.

[l2dy (Zero King)]

Those who could reliably reproduce the build failure, please try libtermkey from this PR ​https://github.com/macports/macports-ports/pull/13483.

[cooljeanius (Eric Gallager)]

Replying to l2dy:

Could be related to our libtermkey port, which uses ncurses. ​https://github.com/neovim/neovim/issues/16190#issuecomment-955741551

libtermkey will use unibilium if it was installed. Also, libtermkey will use ncurses if unibilium is not installed and ncurses is installed. neovim recommends that you use unibilium to build libtermkey (The bundled libtermkey will be built using unibilium unless you change the cmake options). Perhaps your broken libtermkey is built using ncurses (This is indicated by openat(AT_FDCWD, "/lib/libncursesw.so.6", O_RDONLY|O_CLOEXEC) = 3 in strace).

libtermkey using ncurses has not been well tested by CI or other means, so it is not surprising that it has bugs.

hm...

$ port installed libtermkey unibilium ncurses cmake
The following ports are currently installed:
  cmake @3.22.1_0+docs+python39+qt5 (active)
  libtermkey @0.22_0 (active)
  ncurses @6.3_0 (active)
  unibilium @2.0.0_1 (active)

[l2dy (Zero King)]

Our libtermkey port is patched to always use ncursesw, see ​https://github.com/macports/macports-ports/pull/13483.

[l2dy (Zero King)]

In 1f31fffe579875c6ed7ef6a2bcaf9b43fa3b0c19/macports-ports (master):

libtermkey: use unibilium

See: #64159

[l2dy (Zero King)]

PR merged. Please try the latest version of neovim and libtermkey.

[cooljeanius (Eric Gallager)]

I'm still getting this error with neovim @0.6.0_1 and libtermkey @0.22_1

[l2dy (Zero King)]

Unfortunately I can not reproduce this error, and our buildbot has successfully built binary packages for all macOS versions above 10.7.

I need more information to continue investigating, stack traces and otool -L <path/to/segfault/nvim> output could be helpful.

Everyone who can reproduce, please provide your macOS and Xcode versions. This could also be related to a specific Xcode release.

[cooljeanius (Eric Gallager)]

hm, I'm trying to get a stack trace, but it doesn't seem to be doing anything when run inside gdb:

$ /opt/local/libexec/gnubin/gdb /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim
GNU gdb (GDB) 11.1
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin20.6.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim...
(gdb) set args -u NONE -i NONE -e --headless -c helptags\ doc -c quit
(gdb) run
Starting program: /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim -u NONE -i NONE -e --headless -c helptags\ doc -c quit
[New Thread 0x2b03 of process 19069]

(I haven't learned how to transition my gdb commands to lldb commands yet)

otool -L says:

$ otool -L /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_editors_neovim/neovim/work/build/bin/nvim:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1775.118.101)
	/opt/local/lib/libintl.8.dylib (compatibility version 11.0.0, current version 11.0.0)
	/opt/local/lib/libiconv.2.dylib (compatibility version 9.0.0, current version 9.1.0)
	/opt/local/lib/libluv.1.dylib (compatibility version 1.0.0, current version 1.41.0)
	/opt/local/lib/libuv.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.100.5)
	/opt/local/lib/libmsgpackc.2.dylib (compatibility version 2.0.0, current version 2.0.0)
	/opt/local/lib/libvterm.0.dylib (compatibility version 1.0.0, current version 1.4.0)
	/opt/local/lib/libtermkey.1.dylib (compatibility version 16.0.0, current version 16.2.0)
	/opt/local/lib/libunibilium.4.dylib (compatibility version 5.0.0, current version 5.0.0)
	/opt/local/lib/libtree-sitter.0.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libutil.dylib (compatibility version 1.0.0, current version 1.0.0)
	/opt/local/lib/libluajit-5.1.2.dylib (compatibility version 2.1.0, current version 2.1.0)

macOS and Xcode version info:

macOS 11.6.2 20G314 x86_64
Xcode 13.2.1 13C100

[cooljeanius (Eric Gallager)]

Whatever, I'm just going to give up on the +debug variant; it installs perfectly fine without it...

[l2dy (Zero King)]

The debug variant is added by the cmake PortGroup. It's probably an unsupported configuration in the first place.

I have tested it on macOS 10.15 and 12 (x86_64) but could not reproduce the segfault.