port -vst install fails with "/usr/bin/tar: No such file or directory"

[janstary (Jan Starý)]

This is 2.4.99 on MacOS 13.10.2 Trying to install a port with sudo port -vst install repeatably fails. For example, sudo port -vst install pstree will fail, the log saying

:info:extract Executing:  cd "/opt/local/var/macports/build/_opt_mports_macports-ports_sysutils_pstree/pstree/work/pstree-2.39" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/pstree/pstree-2.39.tar.gz' | /usr/bin/tar -xf -
:debug:extract system:  cd "/opt/local/var/macports/build/_opt_mports_macports-ports_sysutils_pstree/pstree/work/pstree-2.39" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/pstree/pstree-2.39.tar.gz' | /usr/bin/tar -xf -
:info:extract sh: /usr/bin/tar: No such file or directory
:info:extract gzip: error writing to output: Broken pipe
:info:extract gzip: /opt/local/var/macports/distfiles/pstree/pstree-2.39.tar.gz: uncompress failed
:info:extract Command failed:  cd "/opt/local/var/macports/build/_opt_mports_macports-ports_sysutils_pstree/pstree/work/pstree-2.39" && /usr/bin/gzip -dc '/opt/local/var/macports/distfiles/pstree/pstree-2.39.tar.gz' | /usr/bin/tar -xf -
:info:extract Exit code: 127

The same happens with sudpo port -t intall, with any other port. On the other hand, sudo port -vs install pstree will work, so I suspect the -t to be the culprit.

[janstary (Jan Starý)]

log of the failed sudo port -vst install

[ryandesign (Ryan Carsten Schmidt)]

This appears to be a bug in macOS High Sierra introduced in 10.13.2. It was ​previously discussed on the mailing list.

[raimue (Rainer Müller)]

Filed as rdar://problem/36469208 with the Apple Bug Reporter.

[raimue (Rainer Müller)]

Reply from Apple on the bug report:

This issue behaves as intended based on the following:

This change was intentional as following symlinks allows you to clone directories (which is disallowed due to potential system slowdown in apfs). We will update the man page to reflect this.

This is not a bug in clonefileat(), this was an intentional change in copyfile itself.

[janstary (Jan Starý)]

Can someone please explain how following symlinks allows you to "clone directories"?

[janstary (Jan Starý)]

Without COPYFILE_CLONE, copyfile(3) seems to copy the destination file OK. ​https://lists.macports.org/pipermail/macports-dev/2017-December/037091.html

[neverpanic (Clemens Lang)]

Yeah, I believe the intent of the original change was to use COPYFILE_CLONE so that no additional storage would be required on APFS.

Since Apple now intentionally broke their own API, we only have the option of following the symlinks ourselves, or going back to the old model that requires double the storage.

[raimue (Rainer Müller)]

I have no idea what Apple meant in the answer with "clone directories". I got no answer on a rebuttal and the rdar was closed without further comment.

[raimue (Rainer Müller)]

In 995dde8476c48580db4f6eedfde09e90dc5e8c99/macports-base:

sip_copy_proc: Avoid COPYFILE_CLONE

Apple broke the copyfile(3) interface with macOS 10.13.3 High Sierra as
there is no way any more to make it follow symlinks when COPYFILE_CLONE
is used. According to rdar://problem/36469208 this change was
intentional.

Instead of adding more conditionals to check whether the file to clone
is actually a symlink, just do not attempt to clone it at all but make
a normal copy. This will usually only take a few megabytes of disk space
anyway.

Closes: #55575

[raimue (Rainer Müller)]

This is actually broken in the released version on macOS 10.13.3 High Sierra and needs to be backported to MacPorts 2.4.3.

[raimue (Rainer Müller)]

In b58d578a8e1aed0fa186ca059c337589a588e0af/macports-base:

sip_copy_proc: Avoid COPYFILE_CLONE

Apple broke the copyfile(3) interface with macOS 10.13.3 High Sierra as
there is no way any more to make it follow symlinks when COPYFILE_CLONE
is used. According to rdar://problem/36469208 this change was
intentional.

Instead of adding more conditionals to check whether the file to clone
is actually a symlink, just do not attempt to clone it at all but make
a normal copy. This will usually only take a few megabytes of disk space
anyway.

Closes: #55575
(cherry picked from commit 995dde8476c48580db4f6eedfde09e90dc5e8c99)