Opened 7 months ago

Last modified 3 months ago

#60230 assigned defect

bsdtar --hfsCompression fails for non-root users

Reported by: AgilentGCMS Owned by: eborisch (Eric A. Borisch)
Priority: Normal Milestone:
Component: ports Version: 2.6.2
Keywords: Cc: eborisch (Eric A. Borisch), tobypeterson
Port: libarchive

Description

I recently ran 'port upgrade outdated', and after building some packages it failed with the error message

--->  Activating libarchive @3.4.2_0
Error: Failed to activate libarchive: command execution failed

after which I am unable to do much with port, because it seems that port wants to install libarchive to fix broken packages first, which fails. The 'main.log' for libarchive is attached.

Attachments (1)

main.log (10.3 KB) - added by AgilentGCMS 7 months ago.

Download all attachments as: .zip

Change History (33)

Changed 7 months ago by AgilentGCMS

Attachment: main.log added

comment:1 Changed 7 months ago by AgilentGCMS

I found a similar issue in #56563. Even though a comment there said that the underlying issue was fixed in libarchive 3.4.0, and my libarchive is 3.4.2

$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4

I can confirm that the problem reported by #56563 is the same one I'm having, and the fix (disabling hfscompression in macports.conf for non-root installs) is the same as well. So perhaps the libarchive bug is back.

Last edited 7 months ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:2 Changed 7 months ago by mf2k (Frank Schima)

In the future, please add the port maintainer(s) to Cc (port info --maintainers libarchive), if any.

comment:3 Changed 7 months ago by mf2k (Frank Schima)

Owner: set to tobypeterson
Status: newassigned

comment:4 Changed 7 months ago by eborisch (Eric A. Borisch)

You're operating with a custom (not /opt/local) MacPorts location, so it's always possible we've missed something in testing...

A few quick questions:

  • When you're in the state where activating fails, what's the output of which bsdtar and bsdtar --version? (Be sure to check after a sudo port deactivate libarchive, as this would happen before activating a new version.)
  • What type of filesystem is /Users/sbasu1/packages/macports on?

Thanks!

comment:5 Changed 7 months ago by eborisch (Eric A. Borisch)

Cc: eborisch added

comment:6 Changed 7 months ago by iplasma

This happens to me for cctools. Tried libarchive 3.4.0_1 and 3.4.2_0, both have same problem with activation. Setting "hfscompression no" in macports.conf fixes it.

Last edited 7 months ago by iplasma (previous) (diff)

comment:7 in reply to:  6 Changed 7 months ago by eborisch (Eric A. Borisch)

Replying to iplasma:

This happens to me for cctools. Tried libarchive 3.4.0_1 and 3.4.2_0, both have same problem with activation. Setting "hfscompression no" in macports.conf fixes it.

Can you answer the first of the bulleted questions I had in my comment above? Also what OS version are you on? Thanks!

Last edited 7 months ago by eborisch (Eric A. Borisch) (previous) (diff)

comment:8 Changed 7 months ago by AgilentGCMS

As I mentioned in my original ticket, before trying port upgrade outdated, this was the bsdtar version

$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4

Also,

$ which bsdtar
/Users/sbasu1/packages/macports/bin/bsdtar

As for filesystem type, diskutil list says APFS Volume Macintosh HD.

comment:9 in reply to:  8 Changed 7 months ago by eborisch (Eric A. Borisch)

Replying to AgilentGCMS:

As I mentioned in my original ticket, before trying port upgrade outdated, this was the bsdtar version

$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4

Also,

$ which bsdtar
/Users/sbasu1/packages/macports/bin/bsdtar

As for filesystem type, diskutil list says APFS Volume Macintosh HD.

During an upgrade, it first deactivates the (old version) port and then activates the new version. It is the result of which bsdtar during this deactivated state that I’m looking for. Sorry if my original question was not sufficiently clear.

So, sudo port deactivate bsdtar && which bsdtar would be the desired commands to run and report the result from. (It will not be the one from MacPorts as you have reported here unless something is wrong, as it shouldn’t be present at that point.)

comment:10 Changed 7 months ago by AgilentGCMS

OK, before I post the results, I need to clear up one thing. I *did* manage to port upgrade outdated 10 days ago by disabling hfscompression. If that makes my answer useless, then I apologize, but I don't know how to go back to the state before I did port upgrade outdated. Having said that, here's what happens currently.

$ port deactivate bsdtar
Error: port deactivate failed: Image error: port bsdtar is not active.
$ bsdtar --version
bsdtar 3.4.2 - libarchive 3.4.2 zlib/1.2.11 liblzma/5.2.4 bz2lib/1.0.8 liblz4/1.9.2 libzstd/1.4.4
$ which bsdtar
/Users/sbasu1/packages/macports/bin/bsdtar

comment:11 Changed 7 months ago by eborisch (Eric A. Borisch)

Is this location where it is trying to install into, as well, or do you have multiple macports installations?

If this is where your macports installs into, then your on-disk state and MP’s database aren’t lining up — MacPorts thinks libarchive is not active, yet its contents (including bsdtar) clearly are in place, at least partially. I’m not sure how it got into this state, but it will be difficult to determine what is causing the activate/compression issue until that is resolved.

If you have two macports installations, then all bets are off; this isn’t a supported configuration.

comment:12 Changed 7 months ago by AgilentGCMS

No, I only have one macports installation. The 'port' command resolves to the correct location

$ which port
/Users/sbasu1/packages/macports/bin/port

comment:13 Changed 6 months ago by tobypeterson

Rebuilt base to force usage of bsdtar; cannot reproduce this issue. I'm using the standard location.

comment:14 Changed 6 months ago by tobypeterson

Ok, I _can_ repro this manually by using /usr/bin/bsdtar, which is confusing.

/usr/bin/bsdtar -xvp --hfsCompression -f /opt/local/var/macports/software/libarchive/libarchive-3.4.2_0.darwin_19.x86_64.tbz2

For some reason, it doesn't fail (for me) during activate, even though it's using /usr/bin/bsdtar there as well.

Last edited 6 months ago by tobypeterson (previous) (diff)

comment:15 Changed 6 months ago by tobypeterson

Ah, got it! It's because I run as root. From the attached log: :debug:main Privilege de-escalation not attempted as not running as root.

comment:16 Changed 6 months ago by tobypeterson

This is really a base bug; /usr/bin/bsdtar (at least on macOS 10.15) does not appear to correctly support --hfsCompression when running as non-root.

comment:17 Changed 6 months ago by tobypeterson

Component: portsbase

comment:18 Changed 6 months ago by tobypeterson

Summary: Failed to activate libarchive/usr/bin/bsdtar --hfsCompression fails for non-root users (Failed to activate libarchive)

comment:19 Changed 6 months ago by tobypeterson

Based on #60176, macOS 10.14's /usr/bin/bsdtar is also unsuitable.

comment:20 Changed 6 months ago by tobypeterson

Owner: changed from tobypeterson to eborisch

comment:21 Changed 6 months ago by tobypeterson

Cc: tobypeterson added

comment:22 in reply to:  19 Changed 6 months ago by jmroot (Joshua Root)

Summary: /usr/bin/bsdtar --hfsCompression fails for non-root users (Failed to activate libarchive)bsdtar --hfsCompression fails for non-root users

Replying to tobypeterson:

Based on #60176, macOS 10.14's /usr/bin/bsdtar is also unsuitable.

We don't attempt to use it though. The 10.14 user's issue was with the libarchive port's bsdtar. Whereas on 10.15 /usr/bin/bsdtar claims to support hfscompression so we do use it.

All the information so far points to this being a new bug introduced in libarchive 3.4.2. It was confirmed that 3.4.0 worked fine with non-root installs and it seemed OK with 3.4.1 as well AFAIK.

comment:23 Changed 6 months ago by jmroot (Joshua Root)

Milestone: MacPorts 2.6.3

This is both a bug in a port, and an OS bug that base needs to work around. We need to:

  1. Fix the libarchive port and get the fix accepted upstream. If someone could bisect the commits since 3.4.1, that would be very helpful.
  2. Make base not use hfscompression if it would be using a broken version of /usr/bin/bsdtar and not running as root.
Last edited 6 months ago by jmroot (Joshua Root) (previous) (diff)

comment:24 Changed 6 months ago by tobypeterson

Sorry, I may be misunderstanding; what's the bug in the port?

/usr/bin/bsdtar --hfsCompression generates a ton of errors; /opt/local/bin/bsdtar (libarchive-3.4.2_0) works fine.

edit: I see, some suspicion that libarchive 3.4.2 may be broken on 10.14. I'll poke around in a VM to see if I can repro that.

Last edited 6 months ago by tobypeterson (previous) (diff)

comment:25 Changed 6 months ago by tobypeterson

Reopened #60176 to continue investigating that. Let's track the base bug here.

Last edited 6 months ago by tobypeterson (previous) (diff)

comment:26 Changed 6 months ago by eborisch (Eric A. Borisch)

#60176 and this are both likely due to issues with extended attributes that shouldn't be there (show up with '@' on ls -l listing) combined with an issue within libarchive if:

  • the activated file is set read-only
  • AND has extended attributes (not compression iteslf, as this is applied separately)
  • AND is not being extracted by root

See also my comment here on libarchive.

For #60176, this can be fixed by squashing stray resource forks (as the port already does for a number of files) in the manpages; I haven't looked further at this one yet.

comment:27 Changed 6 months ago by eborisch (Eric A. Borisch)

I don't have 10.15 running, but from here it appears it has libarchive 3.3.2, which has --hfsCompression, but does not have my fix for supporting non-root extraction in it (which went into 3.4.0).

We may just want to change our test to see if our libarchive is installed rather than if it claims to support --hfsCompression.

I still think supporting compression is a big win, especially for developers (header files are excellent compression fodder, and compilers have lots of them) or anyone on a laptop (fixed size storage; every byte counts.)

comment:28 in reply to:  27 Changed 6 months ago by jmroot (Joshua Root)

Replying to eborisch:

We may just want to change our test to see if our libarchive is installed rather than if it claims to support --hfsCompression.

That won't help, the reporter of #60176 was using our libarchive on 10.14.

comment:29 Changed 3 months ago by jmroot (Joshua Root)

Upon further investigation, the case in #60176 has been broken all along, it's not a regression since 3.4.0 as I previously thought.

comment:30 in reply to:  23 Changed 3 months ago by jmroot (Joshua Root)

Replying to jmroot:

  1. Fix the libarchive port and get the fix accepted upstream. If someone could bisect the commits since 3.4.1, that would be very helpful.

It doesn't look like anyone ever took this upstream, so: https://github.com/libarchive/libarchive/issues/1415

comment:31 Changed 3 months ago by jmroot (Joshua Root)

In 9d0ebd5b5b1a7fd0f3a05be85bf0e7267c63b70a/macports-base (master):

Disable hfscompression for non-root installs (again)

See: #60230

comment:32 Changed 3 months ago by jmroot (Joshua Root)

Component: baseports
Milestone: MacPorts 2.6.3

Workaround is back in base 2.6.3 but the bug in libarchive still needs to be addressed, so leaving this open for that.

Note: See TracTickets for help on using tickets.