Opened 4 years ago

Closed 3 years ago

#52414 closed defect (wontfix)

libarchive port fails problem with -pthread argument to clang

Reported by: JustinCB Owned by: tobypeterson
Priority: Normal Milestone:
Component: ports Version: 2.3.4
Keywords: Cc:
Port: libarchive

Description


Attachments (1)

libarchive_fail.log (213.8 KB) - added by JustinCB 4 years ago.
The main log of the libarchive port

Download all attachments as: .zip

Change History (23)

Changed 4 years ago by JustinCB

Attachment: libarchive_fail.log added

The main log of the libarchive port

comment:1 Changed 4 years ago by JustinCB

Cc: JustinCB@… removed

Cc Me!

comment:2 Changed 4 years ago by JustinCB

Cc: JustinCB@… added

Cc Me!

comment:3 Changed 4 years ago by JustinCB

The problem seems to be that the libarchive port needs libpthread, but it doesn't provide the right argument to clang to get the library.

comment:4 Changed 4 years ago by JustinCB

And that it thinks lzma is multithreaded, which prevents it from working.

comment:5 Changed 4 years ago by mf2k (Frank Schima)

Cc: JustinCB@… removed
Owner: changed from macports-tickets@… to toby@…

In the future, please Cc the port maintainers (port info --maintainers libarchive), if any. As reporter, you do not need to Cc yourself.

comment:6 Changed 4 years ago by tobypeterson

Doesn't look like this has anything to do with pthreads. That's just a compilation warning.

Looks more like something sketchy going on with linking.

comment:7 Changed 4 years ago by tobypeterson

CPATH='/Users/justin/include' LIBRARY_PATH='/Users/justin/lib'

That doesn't seem like a good idea. Try removing those and seeing if it works.

comment:8 Changed 4 years ago by JustinCB

Apparently macports assumes that it's libraries are in the same directory as the system libraries. I installed macports in my home directory and added that to my path. It keeps it from accidentally overwriting system files and me from getting used to typing sudo all the time.

comment:9 Changed 4 years ago by neverpanic (Clemens Lang)

Weird, the call of libtool seems to be correct:

./libtool  --tag=CC   --mode=link /usr/bin/clang ... -L/Users/justin/lib ... -L/Users/justin/lib ... -llzma ...

but the actual linker invocation generated by libtool isn't:

libtool: link: /usr/bin/clang ... -L/Users/justin/lib ... /usr/lib/liblzma.dylib ...

For some reason, libtool turns the valid spec into /usr/lib/liblzma.dylib, which doesn't match the headers and doesn't have multithread support, so linking fails. This might be a side effect of libtool filtering some uncommon arguments in an effort to try to be helpful ("surely you don't want to link against these files in your $HOME, let me helpfully remove those for you").

I assume you do have liblzma.dylib with those symbols in ~/lib?

comment:10 Changed 4 years ago by JustinCB

~/liblzma.dylib exists. It is the version installed by macports with xz.

comment:11 Changed 4 years ago by JustinCB

I compiled it without macports, and I moved *libiconv* in ~/lib/ to ~/lib/noexec and iconv.h in ~/include to ~/include/noinclude Is there a way to make macports used those that I compiled myself? To compile it, I also had to compile cmake.

Last edited 4 years ago by JustinCB (previous) (diff)

comment:12 in reply to:  11 Changed 4 years ago by larryv (Lawrence Velázquez)

Replying to JustinCB@…:

Is there a way to make macports used those that I compiled myself?

No.

To compile it, I also had to compile cmake.

Which part of this required CMake?

comment:13 Changed 4 years ago by JustinCB

Configuring libarchive requires CMake(If it's from source). Macports CMake somehow requires libarchive. Macports libarchive has a bug that I filed this ticket for. That's why I'm trying to compile it myself right now.

comment:14 in reply to:  13 ; Changed 4 years ago by larryv (Lawrence Velázquez)

Replying to JustinCB@…:

Configuring libarchive requires CMake(If it's from source). Macports CMake somehow requires libarchive.

It does not require CMake. Upstream provides an Autotools-based build system, which is what we use.

Macports libarchive has a bug that I filed this ticket for.

Debatable. I wouldn’t be surprised if your highly unorthodox configuration were causing the issue.

comment:15 in reply to:  14 ; Changed 4 years ago by JustinCB

Replying to larryv@…:

Replying to JustinCB@…:

Configuring libarchive requires CMake(If it's from source). Macports CMake somehow requires libarchive.

It does not require CMake. Upstream provides an Autotools-based build system, which is what we use.

Macports libarchive has a bug that I filed this ticket for.

Debatable. I wouldn’t be surprised if your highly unorthodox configuration were causing the issue.

It isn't highly unorthodox, only slightly unorthodox. It is a part of the official macports source, but not the default. You can't use libraries you compiled yourself in macports, but you can use tools that you either compiled yourself or are included with the system(like CMake), and I was trying to build something that required CMake. It can use the CMake I compiled. I don't think this bug is in the libarchive build system, it is a bug in libtool that causes it to automatically link against the system libraries, even if you told it explicitly that you want it to link to different libraries. It might be so that it will make binaries that work on any system. It needs to do that to make binaries that work on any system because Mach-O links to a specific path instead of a path set by the environment.

comment:16 Changed 4 years ago by JustinCB

I don't particularly like CMake, though. It's like a version of autotools that requires autotools to configure instead of generating a portable shell script.

comment:17 in reply to:  15 ; Changed 4 years ago by larryv (Lawrence Velázquez)

Replying to JustinCB@…:

It isn't highly unorthodox, only slightly unorthodox.

Installing MacPorts somewhere under $HOME is not uncommon. Using $HOME directly as the prefix and overriding CPATH and LIBRARY_PATH is.

You can't use libraries you compiled yourself in macports, but you can use tools that you either compiled yourself or are included with the system(like CMake), and I was trying to build something that required CMake. It can use the CMake I compiled.

You can technically override MacPorts’ toolchain, yes, but we do not support doing so and are not obligated to accommodate such configurations.

I don't think this bug is in the libarchive build system, it is a bug in libtool that causes it to automatically link against the system libraries, even if you told it explicitly that you want it to link to different libraries.

There very well might be a bug in libtool, but it’s not quite what you are describing. Anyone using MacPorts in the standard fashion gets a libarchive that is not linked to the system libraries.

% otool -L /opt/local/lib/libarchive.13.dylib
/opt/local/lib/libarchive.13.dylib:
	/opt/local/lib/libarchive.13.dylib (compatibility version 16.0.0, current version 16.1.0)
	/opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current version 8.1.0)
	/opt/local/lib/liblzo2.2.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.2.0)
	/opt/local/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/opt/local/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.6)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/opt/local/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

comment:18 Changed 4 years ago by tobypeterson

Yeah, something in libarchive's libtool must be doing something cute. The libtool invocation looks fine, it just somehow decides to use /usr/lib instead.

comment:19 in reply to:  17 ; Changed 4 years ago by JustinCB

Installing MacPorts somewhere under $HOME is not uncommon. Using $HOME directly as the prefix and overriding CPATH and LIBRARY_PATH is.

You mean installing the port commands under $HOME is not uncommon, but $HOME as the prefix is? CPATH and LIBRARY_PATH are set to (prefix)/include and (prefix)/lib respectively.

You can't use libraries you compiled yourself in macports, but you can use tools that you either compiled yourself or are included with the system(like CMake), and I was trying to build something that required CMake. It can use the CMake I compiled.

You can technically override MacPorts’ toolchain, yes, but we do not support doing so and are not obligated to accommodate such configurations.

Macports defaults to using toolchains that already exist, even if they aren't Macports toolchains.

I don't think this bug is in the libarchive build system, it is a bug in libtool that causes it to automatically link against the system libraries, even if you told it explicitly that you want it to link to different libraries.

There very well might be a bug in libtool, but it’s not quite what you are describing. Anyone using MacPorts in the standard fashion gets a libarchive that is not linked to the system libraries.

% otool -L /opt/local/lib/libarchive.13.dylib
/opt/local/lib/libarchive.13.dylib:
	/opt/local/lib/libarchive.13.dylib (compatibility version 16.0.0, current version 16.1.0)
	/opt/local/lib/libiconv.2.dylib (compatibility version 8.0.0, current version 8.1.0)
	/opt/local/lib/liblzo2.2.dylib (compatibility version 3.0.0, current version 3.0.0)
	/opt/local/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.2.0)
	/opt/local/lib/libcharset.1.dylib (compatibility version 2.0.0, current version 2.0.0)
	/opt/local/lib/libbz2.1.0.dylib (compatibility version 1.0.0, current version 1.0.6)
	/opt/local/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.8)
	/opt/local/lib/libxml2.2.dylib (compatibility version 12.0.0, current version 12.4.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

I think that the bug is that if it tries to link to a library that is not in a "standard" path, it will replace it with one that is in a "standard" path if one exists.

comment:20 in reply to:  19 Changed 4 years ago by larryv (Lawrence Velázquez)

Replying to JustinCB@…:

You mean installing the port commands under $HOME is not uncommon, but $HOME as the prefix is?

Yes.

Macports defaults to using toolchains that already exist, even if they aren't Macports toolchains.

No, it does not. It uses tools in one of the following directories, expecting them to be provided by MacPorts or Apple.

  • ${prefix}/bin
  • ${prefix}/sbin
  • /bin
  • /sbin
  • /usr/bin
  • /usr/sbin

Modifying this search behavior or installing your own tools to those locations is absolutely unsupported.

I think that the bug is that if it tries to link to a library that is not in a "standard" path, it will replace it with one that is in a "standard" path if one exists.

What is “standard”? Who says that's a bug? It may very well be intentional.

Last edited 4 years ago by larryv (Lawrence Velázquez) (previous) (diff)

comment:21 Changed 3 years ago by tobypeterson

wacky config, marking wontfix

comment:22 Changed 3 years ago by tobypeterson

Resolution: wontfix
Status: newclosed
Note: See TracTickets for help on using tickets.