wireshark @1.12.7: Build fails with lua 5.3

[mndavidoff (Monte Davidoff)]

Building wireshark @1.12.7 fails after the upgrade to lua @5.3.1.

$ sudo port -vs install wireshark
...
  CC       lua_bitop.lo
  CC       lrexlib.lo
  CC       lrexlib_glib.lo
  CC       lrexlib_glib_f.lo
  CC       wslua_tvb.lo
  CC       wslua_proto.lo
  CC       wslua_int64.lo
  CC       wslua_tree.lo
./lua_bitop.c:85:2: error: "Unknown number type, check LUA_NUMBER_* in luaconf.h"
#error "Unknown number type, check LUA_NUMBER_* in luaconf.h"
 ^
1 error generated.
make[4]: *** [lua_bitop.lo] Error 1
...
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_net_wireshark/wireshark/work/wireshark-1.12.7" && /usr/bin/make -j8 -w all 
Exit code: 2
Error: org.macports.build for port wireshark returned: command execution failed
Warning: targets not executed for wireshark: org.macports.activate org.macports.build org.macports.destroot org.macports.install
Please see the log file for port wireshark for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_net_wireshark/wireshark/main.log
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port wireshark failed

The build succeeds if the lua variant is disabled:

$ sudo port -vs install wireshark -lua

For a similar issue with gnuplot and lua, see #48826.

[mojca (Mojca Miklavec)]

Cc Me!

[mopihopi]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #48893.

[smith.kennedy+macports@…]

Cc Me!

[laughingtiger]

Cc Me!

[mp@…]

Cc Me! Same issue on Mavericks.

[p.ohanlon@…]

This issues seems to be down to the fact that Wireshark doesn't yet support lua-5.3 as may be seen from the Wireshark mail list: ​https://www.wireshark.org/lists/wireshark-dev/201501/msg00211.html

Maybe it is possible to modify the portfile and patch wireshark configure scripts so it depends upon port:lua52 - I tried a simple change (lua -> lua52) in the portfile but it didn't work. It needs more work on Wireshark configure script...

[mojca (Mojca Miklavec)]

Does it work if you temporary force-deactivate lua before installing wireshark built against lua52?

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #49006.

[jere@…]

Cc Me!

[aque (Allan Que)]

Cc Me!

I got further by doing the following:

• uninstalled lua and installed lua52
• edited the portfile to depend on lua52
• sudo ln -s /opt/local/include/lua-5.2 /opt/local/include/lua5.2 (the configure script seems to be looking for lua 5.2 there)

checking whether to use liblua for the Lua scripting plugin... yes
checking Lua version... Lua 5.2
checking lua.h usability... no
checking lua.h presence... no
checking for lua.h... no
checking lua5.2/lua.h usability... yes
checking lua5.2/lua.h presence... yes
checking for lua5.2/lua.h... yes
checking lua5.2/lualib.h usability... yes
checking lua5.2/lualib.h presence... yes
checking for lua5.2/lualib.h... yes
checking lua5.2/lauxlib.h usability... yes
checking lua5.2/lauxlib.h presence... yes
checking for lua5.2/lauxlib.h... yes
checking lualib.h usability... no
checking lualib.h presence... no
checking for lualib.h... no
checking for lua5.2/lua.h... (cached) yes
checking for lua5.2/lualib.h... (cached) yes
checking for lua5.2/lauxlib.h... (cached) yes
checking lauxlib.h usability... no
checking lauxlib.h presence... no
checking for lauxlib.h... no
checking for lua5.2/lua.h... (cached) yes
checking for lua5.2/lualib.h... (cached) yes
checking for lua5.2/lauxlib.h... (cached) yes
checking for luaL_openlibs in -llua... no
checking for luaL_openlibs in -llua5.2... no
configure: error: Linking with liblua failed.
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_net_wireshark/wireshark/work/wireshark-1.12.7" && ./configure --prefix=/opt/local --disable-usr-local --without-adns --with-c-ares=/opt/local --with-pcap=/opt/local --with-ssl=/opt/local --with-zlib=/opt/local --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --disable-warnings-as-errors --with-krb5=/opt/local --with-gnutls=yes --with-libgcrypt-prefix=/opt/local --enable-ipv6 --with-lua=/opt/local --with-libsmi=/opt/local --with-geoip=/opt/local --with-portaudio=/opt/local --with-qt=no --with-gtk2=no --with-gtk3=yes 
Exit code: 1

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #49054.

[breiter (Brian Reiter)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #49063.

[Schamschula (Marius Schamschula)]

Cc Me!

[fracai]

Cc Me!

[rlhamil]

Cc Me!

[christian@…]

Cc Me!

[ghosthound]

Building wireshark with '-lua' works around the problem. This has been added to the Portfile, will be added once svn is back up.

[Schamschula (Marius Schamschula)]

As lua52 can be installed alongside with the current lua, I patched configure and one source file to point to the correct headers and library.

Along the way wireshark just got upgraded to version 1.12.8. so I built against it.

[ghosthound]

Removed lua from the default variants in r141357.

Patches to build wireshark against lua52 welcome.

[Schamschula (Marius Schamschula)]

Replying to ricci@…:

Removed lua from the default variants in r141357.

Patches to build wireshark against lua52 welcome.

See attached patches.

[macports_org@…]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #49315.

[ryandesign (Ryan Carsten Schmidt)]

Replying to mschamschula@…:

See attached patches.

Thanks, but this line looks completely wrong:

        configure.env-append    "-I/${prefix}/include/lua-5.2 -L/${prefix}/lib/lua-5.2" 

First, quotation marks are not required, but more importantly, those flags are not environment variables. To have any effect, the -I flag would need to go in the CPPFLAGS environment variable (represented in MacPorts by the configure.cppflags variable) and the -L flag would need to go in the LDFLAGS variable (represented by the configure.ldflags variable). If the port built fine with this line the way it was written, then the line can be removed, since it can't possibly be doing anything in its current state.

[dershow]

Cc Me!

[Schamschula (Marius Schamschula)]

Replying to ryandesign@…:

Replying to mschamschula@…:

See attached patches.

Thanks, but this line looks completely wrong:

        configure.env-append    "-I/${prefix}/include/lua-5.2 -L/${prefix}/lib/lua-5.2" 

First, quotation marks are not required, but more importantly, those flags are not environment variables. To have any effect, the -I flag would need to go in the CPPFLAGS environment variable (represented in MacPorts by the configure.cppflags variable) and the -L flag would need to go in the LDFLAGS variable (represented by the configure.ldflags variable). If the port built fine with this line the way it was written, then the line can be removed, since it can't possibly be doing anything in its current state.

Ryan,

Point well taken. It must have been very early in the morning when I added that line. I turns out that removing

configure.args-append	--with-lua=${prefix}

actually did the trick. I had added this to deal with the configure script. However, the configure script is smart enough to look for lua5.2, if not forced to look in ${prefix}. The patch for epan/wslua/lua_bitop.c is what is needed to build.

[Schamschula (Marius Schamschula)]

Removed unneeded configure.env arguments

[raimue (Rainer Müller)]

Cc Me!

[ben@…]

Thank you for the previously attached patches to the Portfile. It built successfully for me, but then crashed on startup. I made an additional change to the Portfile to fix the linker path. It looks like this was attempted above using environment variables, but my reading of the Macports docs makes me think this is the more correct way to do it, with configure.ldflags-append. I now have a working Wireshark build. My fixes are re-attached here as a complete unified diff.

[ben@…]

I spoke incorrectly - I was accidentally compiling -lua instead of +lua. Even with my patch above, I'm still getting this when starting Wireshark:

 wireshark
dyld: lazy symbol binding failed: Symbol not found: _luaL_optint
  Referenced from: /opt/local/lib/libwireshark.5.dylib
  Expected in: flat namespace

dyld: Symbol not found: _luaL_optint
  Referenced from: /opt/local/lib/libwireshark.5.dylib
  Expected in: flat namespace

Trace/BPT trap: 5

[ben@…]

Ok, I figured out the problem: I had both Lua 5.3 and Lua 5.2 installed, and Wireshark was ended up being linked against both /opt/local/lib/liblua.dylib (the 5.3 version) and /opt/local/lib/lua-5.2/liblua-5.2.dylib. You can see this by running it with DYLD_PRINT_LIBRIARIES:

$ DYLD_PRINT_LIBRARIES=true  wireshark
dyld: loaded: /opt/local/bin/wireshark
dyld: loaded: /opt/local/lib/libfiletap.0.dylib
dyld: loaded: /opt/local/lib/libwiretap.4.dylib
dyld: loaded: /opt/local/lib/libwireshark.5.dylib
dyld: loaded: /opt/local/lib/liblua.dylib
dyld: loaded: /opt/local/lib/libGeoIP.1.dylib
dyld: loaded: /opt/local/lib/libwsutil.4.dylib
dyld: loaded: /opt/local/lib/lua-5.2/liblua-5.2.dylib
# ...snip a lot more

By uninstalling Lua 5.3 and Wireshark, the rebuilding/re-installing Wireshark, I have a working build now. Perhaps the solution is to mark Wireshark as conflicting Lua 5.3, or fix the build so that it only links against the 5.2 version?

[ben@…]

...and I ended up with yet another Wireshark build without Lua. (Sorry for the noise, I'm about done for tonight).

The build reports as being +lua but when you actually launch Wireshark, it reports without Lua. This is because the proposed patch removed the --with-lua flag that compels ./configure to include it. Putting that back in causes the configure step to fail because it can't find Lua.

Here's a relevant excerpt:

:info:configure checking whether to use liblua for the Lua scripting plugin... yes
:info:configure checking Lua version... Lua 5.2
:info:configure checking lua.h usability... yes
:info:configure checking lua.h presence... no
:info:configure configure: WARNING: lua.h: accepted by the compiler, rejected by the preprocessor!
:info:configure configure: WARNING: lua.h: proceeding with the compiler's result
:info:configure checking for lua.h... yes
:info:configure checking lualib.h usability... yes
:info:configure checking lualib.h presence... no
:info:configure configure: WARNING: lualib.h: accepted by the compiler, rejected by the preprocessor!
:info:configure configure: WARNING: lualib.h: proceeding with the compiler's result
:info:configure checking for lualib.h... yes
:info:configure checking lauxlib.h usability... yes
:info:configure checking lauxlib.h presence... no
:info:configure configure: WARNING: lauxlib.h: accepted by the compiler, rejected by the preprocessor!
:info:configure configure: WARNING: lauxlib.h: proceeding with the compiler's result
:info:configure checking for lauxlib.h... yes
:info:configure checking for luaL_openlibs in -llua... no
:info:configure checking for luaL_openlibs in -llua5.2... no
:info:configure configure: error: Linking with liblua failed.
:info:configure Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_net_wireshark/wireshark/work/wireshark-1.12.8" && ./configure --prefix=/opt/local --disable-osx-deploy-target --disable-usr-local --with-pcap=/opt/local --with-ssl=/opt/local --with-zlib=/opt/local --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --disable-warnings-as-errors --with-krb5=/opt/local --with-adns=/opt/local --without-c-ares --with-gnutls=yes --with-libgcrypt-prefix=/opt/local --enable-ipv6 --with-lua --with-libsmi=/opt/local --with-geoip=/opt/local --with-portaudio=/opt/local --with-qt=no --with-gtk2=yes --with-gtk3=no
:info:configure Exit code: 1

Note that Macports installs Lua 5.2 at liblua-5.2 instead of liblua5.2. Even if you pass the relevant CFLAGS and LDFLAGS, it doesn't fix it:

	configure.cflags-append		-I${prefix}/include/lua-5.2
	configure.ldflags-append	-L${prefix}/lib/lua-5.2 -llua-5.2  # does not work

I guess the next thing to try is to patch the build so that it looks for Lua in lua-5.2 instead of lua5.2. Any other ideas?

[Schamschula (Marius Schamschula)]

Ben,

Thanks for the feedback.

The dual link situation is a problem that I (inadvertently) circumvented by building in trace mode, i.e. sudo port -t install wireshark.

We do need a better solution.

[ryandesign (Ryan Carsten Schmidt)]

Has duplicate #49533.

[Schamschula (Marius Schamschula)]

I submitted a ticket @ wireshark.org ​https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=11660 and got a response that the lua detection algorithm had been updated for wireshark 2.0.0, which currently is in release candidate status (rc2).

I've attached a diff file to wireshark 2.0.0rc2.

[ben@…]

Thanks mschamschula. However, this still isn't working to detect lua-5.2 from Macports:

info:configure checking for the location of lua.h... not found
:info:configure configure: error: Lua support was requested, but is not available
:info:configure Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_net_wireshark/wireshark/work/wireshark-2.0.0rc2" && ./configure --prefix=/opt/local --disable-osx-deploy-target --disable-usr-local --disable-osx-deploy-target --with-pcap=/opt/local --with-ssl=/opt/local --with-zlib=/opt/local --mandir=\${prefix}/share/man --infodir=\${prefix}/share/info --disable-warnings-as-errors --with-krb5=/opt/local --with-adns=/opt/local --without-c-ares --with-gnutls=yes --with-libgcrypt-prefix=/opt/local --enable-ipv6 --with-lua=/opt/local --with-libsmi=/opt/local --with-geoip=/opt/local --with-portaudio=/opt/local --with-qt=no --with-gtk2=yes --with-gtk3=no
:info:configure Exit code: 1
:error:configure Failed to configure wireshark, consult /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_net_wireshark/wireshark/work/wireshark-2.0.0rc2/config.log

I'll see if I can hack together a fix to the configure script.

[raimue (Rainer Müller)]

Ports are supposed to track stable variants. Updating to a release candidate would only be appropriate for the wireshark-devel port, which seems to be in a strange limbo anyway (wireshark @1.12.8 > wireshark-devel @1.9.0).

[ben@…]

Ok, I got it working. It required specifying linker flags as well as hacking up the configure script, but I got it working:

$ tshark --version
TShark (Wireshark) 2.0.0rc2 (SVN Rev Unknown from unknown)

Copyright 1998-2015 Gerald Combs <gerald@wireshark.org> and contributors.
License GPLv2+: GNU GPL version 2 or later <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (64-bit) with libpcap, without POSIX capabilities, with libz 1.2.8,
with GLib 2.46.1, with SMI 0.5.0, without c-ares, with ADNS, with Lua 5.2, with
GnuTLS 3.3.18, with Gcrypt 1.6.4, with MIT Kerberos, with GeoIP.

Running on Mac OS X 10.11.1, build 15B42 (Darwin 15.0.0), with locale
en_US.UTF-8, with libpcap version 1.7.4, with libz 1.2.8, with GnuTLS 3.3.18,
with Gcrypt 1.6.4.
        Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz (with SSE4.2)

Built using clang 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.1.76).

I'll see about getting a patch contributed upstream as well.

[Schamschula (Marius Schamschula)]

Raim,

I'm aware that wireshark is not a -devel port. Unfortunately only version 2.0.0 has the fix for the MP lua issue. We have a choice between installing the current port -lua or building 2.0.0rc2 at this time.

Ben,

Strange. Under Yosemite, my patch worked with and w/o trace mode and with and w/o the patch on epan/wslua/lua_bitop.c:

otool -L /opt/local/bin/tshark
/opt/local/bin/tshark:
	/opt/local/lib/libwiretap.5.dylib (compatibility version 6.0.0, current version 6.0.0)
	/opt/local/lib/libwireshark.6.dylib (compatibility version 7.0.0, current version 7.0.0)
	/opt/local/lib/lua-5.2/liblua-5.2.dylib (compatibility version 5.2.0, current version 5.2.3)
	/opt/local/lib/libGeoIP.1.dylib (compatibility version 8.0.0, current version 8.6.0)
	/opt/local/lib/libwsutil.6.dylib (compatibility version 7.0.0, current version 7.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
	/opt/local/lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
	/opt/local/lib/libgthread-2.0.0.dylib (compatibility version 4601.0.0, current version 4601.1.0)
	/opt/local/lib/libgmodule-2.0.0.dylib (compatibility version 4601.0.0, current version 4601.1.0)
	/opt/local/lib/libglib-2.0.0.dylib (compatibility version 4601.0.0, current version 4601.1.0)
	/opt/local/lib/libintl.8.dylib (compatibility version 10.0.0, current version 10.4.0)
	/opt/local/lib/libpcap.A.dylib (compatibility version 1.0.0, current version 1.7.4)
	/opt/local/lib/libcares.2.dylib (compatibility version 4.0.0, current version 4.0.0)
	/opt/local/lib/libkrb5.3.3.dylib (compatibility version 3.0.0, current version 3.3.0)
	/opt/local/lib/libk5crypto.3.1.dylib (compatibility version 3.0.0, current version 3.1.0)
	/opt/local/lib/libcom_err.1.1.dylib (compatibility version 1.0.0, current version 1.1.0)
	/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration (compatibility version 1.0.0, current version 699.40.2)
	/opt/local/lib/libgcrypt.20.dylib (compatibility version 21.0.0, current version 21.4.0)
	/opt/local/lib/libgpg-error.0.dylib (compatibility version 17.0.0, current version 17.0.0)
	/opt/local/lib/libgnutls.28.dylib (compatibility version 70.0.0, current version 70.10.0)
	/opt/local/lib/libsmi.2.dylib (compatibility version 3.0.0, current version 3.27.0)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1153.18.0)

[raimue (Rainer Müller)]

Replying to mschamschula@…:

I'm aware that wireshark is not a -devel port. Unfortunately only version 2.0.0 has the fix for the MP lua issue. We have a choice between installing the current port -lua or building 2.0.0rc2 at this time.

As you say using 2.0.0rc2 is a valid workaround for those who want +lua, my intention was that wireshark-devel could actually provide @2.0.0rc2 at this point.

However, we can also just fix it for 1.12.x by patching the build system. Attached is a patch​ that finds Lua 5.2 with pkg-config and links against the lua52 port. Note this patch is specifically for MacPorts as it now hardcodes the Lua version.

[Schamschula (Marius Schamschula)]

wireshark 2.0.0 is out of release candidate status. I've updated the patch accordingly.

[ghosthound]

FYI- Planning on making the current wireshark port into 'wireshark1' and moving the 'wireshark' port to 2.0.0 so those that want access to latest wireshark 1.x can still get it.

[mdeaudelin (Mathieu Deaudelin-Lemay)]

There is an updated wireshark-devel portfile for 1.99.8 that has been waiting in #40782 for some time - it might be pertinent to consider it, or some of its changes, for 2.0.0. It uses the cmake build system and provides a Wireshark.app for the Applications folder.

[mdeaudelin (Mathieu Deaudelin-Lemay)]

Cc Me!

[posita (Matt Bogosian)]

Cc Me!

[mike-savory]

Cc Me!

[mike-savory]

I just upgraded the 1.99.8 port file in #40782 to 2.0.0 and it compiles and runs for me (10.11.1). The app in /Applications/MacPorts/Wireshark.app fails to run, but the command line version or wireshark launches it fine (I assume this may be a ElCap protection issue). Packet captures working as well.

[Schamschula (Marius Schamschula)]

Bumped patch file to wireshark 2.0.1.

[mike-savory]

Confirming wireshark 2.0.1 on qt5 builds fine for me. Are we ready to do something with this?

[mojca (Mojca Miklavec)]

mshamschula's patch is already a bit outdated by now, but applying it manually almost works (btw: port:lua52 doesn't help if lua is installed; and with +x11 I get quartz-based gtk3 instead).

However it would be probably better to do a major step and also switch to CMake.

See also comment:54:ticket:40782 with a patch as well as the port abandonment ticket #50243.

I'm ready to fork current port to wireshark1 (if there is demand for that), commit the update and retire wireshark-devel, but ricci wanted to do it, so I'm leaving it up to him.

None of that solves the problem with lua though, so these issues should better be discussed in another ticket.