rsync: re-enable HFS compression

[ryandesign (Ryan Carsten Schmidt)]

The rsync port used to use an official patch to add support for HFS compression. As far as I know, "HFS compression" continues to be supported on APFS and is a valuable feature that we want to enable if we can.

This patch was removed when the port was updated to 3.1.3 with the remark:

hfs-compression.diff is marked by upstream as broken as of 3.1.3

Homebrew seems to have followed our lead in removing this patch, but nobody seems to know why we did this. The rsync mailing list has been ​twice ​asked and nobody has replied. John, could you clarify where upstream says this patch is broken, or explain in what way it is broken?

In addition, we have the problem that when HFS compression support was added to the rsync port, we added not only hfs-compression.diff from the official patches directory but also hfs-compression-options.diff. Where did that second patch come from? Jim, did you make it?

[ryandesign (Ryan Carsten Schmidt)]

John responded by email:

Hello, I don’t recall where I found where samba.org marked it as broken. My recollection is that it was in a readme file included with either rsync or rsync-patches. ​http://rsync.samba.org/ftp/rsync/rsync-patches-3.1.3.tar.gz is no longer available, and the new location for rsync-patches, GitHub, doesn’t contain any such info.

I can tell you that I submitted the PR because I have an old PowerPC mac and a) rsync from an intel mac running rsync 3.1.3 to PowerPC mac running 3.1.2 was crashing. I’m pretty sure I tried to get the port working with the hfs-compression patch but was not successful, found _something_ that said it was broken, so I removed it. See: ​https://github.com/macports/macports-ports/pull/1299

FWIW, the patch itself is labeled as UNTESTED, and doesn’t appear to have been updated in the past 9 years, since 3.1.0.dev. Mike Bombich’s original patch isn’t around any more, either. ​http://www.bombich.com/rsync.html - last snapshot on the wayback machine, ​http://web.archive.org/web/20120805080158/http://www.bombich.com/rsync.html.

[ryandesign (Ryan Carsten Schmidt)]

John sent another reply by email:

I’m not a member so my email bounced…I seem to have found where it was marked as broken:

​https://github.com/WayneD/rsync-patches/blame/master/hfs-compression.diff#L551

If you try applying the patch & compiling, this error will be kicked out:

xattrs.c:274:1: error: unknown type name '$$$ERROR$$$'
$$$ERROR$$$ the old patch needs reworking since rsync_xal_get() has totally changed!

This was first added for 3.1.3, still present in 3.2.2.

[ryandesign (Ryan Carsten Schmidt)]

Replying to jbenninghoff:

Hello, I don’t recall where I found where samba.org marked it as broken. My recollection is that it was in a readme file included with either rsync or rsync-patches. ​http://rsync.samba.org/ftp/rsync/rsync-patches-3.1.3.tar.gz is no longer available

It's available at ​https://download.samba.org/pub/rsync/src/rsync-patches-3.1.3.tar.gz. I had already looked in there and didn't see anything at the top of the patch that would indicate it didn't work. I looked now at rsync-3.1.3.tar.gz, and didn't find any mention of "hfs" in it.

I can tell you that I submitted the PR because I have an old PowerPC mac and a) rsync from an intel mac running rsync 3.1.3 to PowerPC mac running 3.1.2 was crashing.

Ok thanks, that's at least something I can try to reproduce the issue. Do you happen to remember what OS versions you were running?

I’m pretty sure I tried to get the port working with the hfs-compression patch but was not successful, found _something_ that said it was broken, so I removed it. See: ​https://github.com/macports/macports-ports/pull/1299

Yup, I had read that PR, but it didn't say why it was broken.

FWIW, the patch itself is labeled as UNTESTED, and doesn’t appear to have been updated in the past 9 years, since 3.1.0.dev.

The patch seems to at least get refreshed in each rsync-patches tar.gz. For example, the patchfile did change between 3.2.1 and 3.2.2. (Not its functionality, but the line numbers in the patch.)

Replying to jbenninghoff:

I seem to have found where it was marked as broken:

​https://github.com/WayneD/rsync-patches/blame/master/hfs-compression.diff#L551

If you try applying the patch & compiling, this error will be kicked out:

xattrs.c:274:1: error: unknown type name '$$$ERROR$$$'
$$$ERROR$$$ the old patch needs reworking since rsync_xal_get() has totally changed!

This was first added for 3.1.3, still present in 3.2.2.

Ahhhh that explains it and gives me somewhere to look. Thanks!

[ryandesign (Ryan Carsten Schmidt)]

John responded by email:

Yes, the patchfile was deliberately broken before the 3.1.3 release with the $$$ERROR$$$ message. It has been maintained since then, but only minimally (just line numbers as you say).

Seeing that error, I’m quite sure that’s why I took out the patch.

[ryandesign (Ryan Carsten Schmidt)]

I'm asking upstream about this patch here: ​https://github.com/WayneD/rsync-patches/issues/1

[mf2k (Frank Schima)]

:)

[mascguy (Christopher Nielsen)]

This is on my official to-do list. Ryan/anyone, please assign this to me.

[Haravikk]

Was any progress made on this? I always found --hfs-compression extremely useful for saving space when syncing into a disk image, so its loss for newer versions of rsync was a bit of a blow. While my eventual aim *should* be to use ZFS or such instead, that's easier said than done.

If a full fix requires a lot more work, would it be possible as an alternative/stop-gap to create an "rsync-3.1.2" port specifically for 3.1.2 to make this easier for people to install and retain in the same way that we have multiple ports for different versions of mariadb etc.?

[mascguy (Christopher Nielsen)]

I briefly looked at this several months ago, but had to defer it due to other commitments. However, it's still on my long-term To Do list.

If I can't make any progress over the next few weeks, I'll look at adding a 3.1.2-specifc port.

[Christopher Nielsen <mascguy@…>]

In 4f145ee6bd117ce0285d5743642115075daf5600/macports-ports (master):

rsync-hfscomp: new port for rsync 3.1.2, with HFS compression
Fixes: #60792

[mascguy (Christopher Nielsen)]

The new port rsync-hfscomp should be available within the next hour or so. To use, first resync your ports, and then do the following:

$ sudo port -f deactivate rsync
$ sudo port -N install rsync-hfscomp
$ sudo port -f uninstall rsync

Also, the various dependents, like git, have been updated to support either rsync-hfscomp or rsync. So going forward, you shouldn't have any issues.

Give it a try, and let us know if all is well.

[mascguy (Christopher Nielsen)]

There's just one caveat with new port rsync-hfscomp: Builds are presently failing on Big Sur, due to gettimeofday().

Tracked by issue:63669

[mascguy (Christopher Nielsen)]

While we now provide a separate port for rsync 3.1.2 with HFS compression (rsync-hfscomp), that's only a short-term fix. The ultimate goal is to rework those patches for the latest upstream release.

Reopening this, to cover work for the latter.