Opened 3 years ago

Closed 15 months ago

#56294 closed update (fixed)

boost: update to 1.70.0

Reported by: michaelld (Michael Dickens) Owned by: ryandesign (Ryan Schmidt)
Priority: Normal Milestone:
Component: ports Version:
Keywords: haspatch maintainer Cc: elfprince13 (Thomas Dickerson), ra1nb0w, stanimura, olupton (Olli Lupton), gallafent
Port: boost

Description (last modified by michaelld (Michael Dickens))

Boost 1.70.0 is out as of 20190412.

Prior Description, which is all likely still relevant: [[ Boost 1.69.0 is out as of 120181212.

Because the Portfile currently uses "--layout=tagged", the resulting libraries names change from the Boost 1.68.0 to include an abbreviated architecture (e.g., "x64" for Intel 64-bit; "p32" for PPC 32-bit); thus for example, the prior naming might be "libboost_system-mt.dylib", while the new naming is "libbboost_system-mt-x64.dylib". This slight change combined with a defect in how CMake works with Clang results in the FindBoost.cmake script failing to detect the new library names with significant coercion.

This issue is resolved by moving to "--layout=system", which reverts the library names to just the minimum for each component; for example here "libboost_system.dylib".

Most of the changes to this latest Boost itself seem to be compatible with the ports I've tried building with it, once the Portfile is fixed to properly detect the changed name. That said, the 10 or so ports I checked pale compared with the total number of ports that depend on Boost (some 250 in total).

Verifying that "enough" of these ports build and function properly "enough" is an arduous task. We in MacPorts have yet to determine a reasonable approach to updating Boost, so the version provided in MacPorts regularly lags the latest release. Please be patient. ]]

Attachments (2)

boost_1.67.0_0.diff (1.0 KB) - added by michaelld (Michael Dickens) 3 years ago.
boost_1.66.0_to_1.68.0.diff (1.9 KB) - added by michaelld (Michael Dickens) 2 years ago.

Download all attachments as: .zip

Change History (38)

Changed 3 years ago by michaelld (Michael Dickens)

Attachment: boost_1.67.0_0.diff added

comment:1 Changed 3 years ago by mf2k (Frank Schima)

Owner: set to ryandesign
Status: newassigned

comment:2 Changed 3 years ago by michaelld (Michael Dickens)

OK so there are some issues I'm coming upon with changes to boost::thread and boost::posix_time ... hopefully it's just slopping programming on the project's part. Or maybe the Boost folks fixed some type conversation issues. I'll work on fixing & verify back backward compatibility with (at least) boost 1.66.0.

comment:3 Changed 2 years ago by elfprince13 (Thomas Dickerson)

There are also some changes in the interaction between CMake and Boost::python, which may be non-backwards compatible.

comment:4 Changed 2 years ago by michaelld (Michael Dickens)

Description: modified (diff)
Summary: boost: update to 1.67.0boost: update to 1.68.0

comment:5 Changed 2 years ago by michaelld (Michael Dickens)

Given the ~4 months for projects to fix their issues with Boost 1.67.0, and possible more issues now with 1.68.0, let's test this release with ports that require Boost to make sure the vast majority work. I would like to get this updated Boost in place sooner rather than later, assuming compatibility is good.

comment:6 Changed 2 years ago by mf2k (Frank Schima)

Keywords: haspatch maintainer added

comment:7 Changed 2 years ago by michaelld (Michael Dickens)

Updated the patch to reset the revision to 0.

comment:8 Changed 2 years ago by ryandesign (Ryan Schmidt)

I'm not working on this; go ahead with what you think is best.

Changed 2 years ago by michaelld (Michael Dickens)

Attachment: boost_1.66.0_to_1.68.0.diff added

comment:9 Changed 2 years ago by michaelld (Michael Dickens)

Updated the patch to remove part of a patchfile incorporated into Boost master for +python3X only.

comment:10 Changed 2 years ago by michaelld (Michael Dickens)

One major ABI naming change: the Boost::Python library is renamed to include the major.minor Python version but with no "." ... so, "27" for Python 2.7 and "36" for Python 3.6 (for 2.,7 was "libboost_python-mt.dylib" is now "libboost_python27-mt.dylib"). Not sure if this is a change from 1.67.0 or 1.66.0 ... but it's important since any project using CMake to find Boost must now specify "python27" or "python36" as a component to look for, not just "python" or "python2" or "python3"; guessing other build systems will be impacted as well.

comment:11 Changed 2 years ago by michaelld (Michael Dickens)

The ABI name change was in Boost 1.67.0 ... so, projects might already be updating to check for it. Not sure where CMake is on this change.

comment:12 Changed 2 years ago by michaelld (Michael Dickens)

CMake is aware of this change: https://gitlab.kitware.com/cmake/cmake/blob/master/Modules/FindBoost.cmake#L43 . One just has to use the correct module name, which now depends on which version of Boost is installed ... so, some logic needs to be in place for projects looking for any Boost Python-based ABI (e.g., "python" or "numpy"). Fun fun!

comment:13 Changed 21 months ago by luzpaz

FYI 1.69.0 is latest stable

comment:14 Changed 21 months ago by michaelld (Michael Dickens)

Description: modified (diff)
Summary: boost: update to 1.68.0boost: update to 1.69.0

comment:15 Changed 20 months ago by elfprince13 (Thomas Dickerson)

Any update on the progress on this? I'm working on a project where my collaborators are on Linux with 1.67, and constantly having to git stash/git stash pop this patch in our CMake:

-find_package(Boost 1.65 REQUIRED COMPONENTS system context python27)
+find_package(Boost 1.65 REQUIRED COMPONENTS system context python)

is...annoying.

Last edited 20 months ago by elfprince13 (Thomas Dickerson) (previous) (diff)

comment:16 Changed 20 months ago by michaelld (Michael Dickens)

I haven't had or taken the time to work over the boost dependencies to make sure that 90%+ of them work. There are some 250 ports that depend on Boost, so doing this check isn't trivial. It's also a challenge to know which ports don't build with the current Boost and ignore them if they still don't build.

MacPorts needs a better way to do testing / verification of such ports that are depended on so heavily; doing it manually as we currently do means that someone has to find and take the time to do all of this checking -- and, then fixing and/or issuing tickets for those that aren't easily fixable. Maybe crowdsourcing this checking is the way to go?

If you need something right now, use the patch attached to this ticket to update to Boost 1.68.0. There is no patch for 1.69.0 because the Boost folks changed the default API naming scheme to include the "ARCH" type & that wreaks havoc on CMake-based builds on macOS / OSX (but, not in general). Again, I have't had or taken the time to figure out the change to revert the "ARCH" to get back to what MacPorts desires (and what works for CMake).

comment:17 Changed 19 months ago by elfprince13 (Thomas Dickerson)

Cc: elfprince13 added

comment:18 Changed 19 months ago by mohd-akram (Mohamed Akram)

comment:19 Changed 19 months ago by michaelld (Michael Dickens)

Description: modified (diff)
Summary: boost: update to 1.69.0boost: update to 1.70.0

comment:20 Changed 19 months ago by michaelld (Michael Dickens)

Thanks for the info on Boost 1.70.0 @mohd-akram . Any incompatibilities you know of already?

comment:21 Changed 19 months ago by michaelld (Michael Dickens)

OK so i've figured out how to revert library names back to before the added <ARCH> ... building using this patch does indeed result in the correct ABI naming as we currently have with 1.66.0 ... yay! One issue down. Here's the whole patch in case anyone wants to use it right now for testing (-p0)

--- boostcpp.jam.orig
+++ boostcpp.jam
@@ -163,7 +163,7 @@
             <base> <threading> <runtime> ;
     case 1.69 :
         .format-name-args =
-            <base> <threading> <runtime> <arch-and-model> ;
+            <base> <threading> <runtime> ;
     }
 }
 else if $(layout) = system

comment:22 Changed 19 months ago by michaelld (Michael Dickens)

1.70.0 provides a CMake scripts for Boost detection using modern targets(any of 1.67-1.69 might have also; I don't recall seeing these being installed by default, but who knows). There are multiple scripts per Boost component .. so a -ton- of these.

I'm having (an) issue/s getting them to work; not sure what's going on yet, hence the "(an)".

This detection is kinda critical for -lots- of ports, so it needs solid testing / fixing if necessary ... that or we just don't install the CMake scripts & use the (meh) one provided by CMake itself.

Once the CMake detection is worked out, my hope is to work on a testing / verification script that iterates over all of the ports that depend on Boost && tests each in turn all the way through "install" && tallies the successes and failures in some reasonable way.

comment:23 Changed 19 months ago by michaelld (Michael Dickens)

The Portfile section to get the CMake scripts installed where at least some ports do so (share/cmake), is:

pre-destroot {
    destroot.args {*}${build.args} --prefix=${destroot}${prefix} \
        --cmakedir=${destroot}${prefix}/share/cmake
    system "find ${worksrcpath} -type f -name '*.gch' -exec rm {} \\;"
}

note to --cmakedir addition. Without this tweak they are installed into lib/cmake.

Not sure which is the better location, but after a quick review of the ports I have that install into both of these, I'm actually inclines to leave the install in lib/cmake. Anyone have an opinion about this?

comment:24 Changed 18 months ago by ra1nb0w

Cc: ra1nb0w added

comment:25 Changed 18 months ago by cjones051073 (Chris Jones)

Could I suggest it would be very useful to do something similar to the on going openssl 1.1.x update here, so push your changes to a WIP github PR and also start to revbump dependents as required.

https://github.com/macports/macports-ports/pull/3822

I think that would make it much easier for others to review and test (I personally am not going to faff about downloading and installing patches by hand, but if its a branch in github thats a different story).

I would also suggest aiming for perfection here is counter productive. We should aim to fix things as far as is reasonable, but at some point just merge the changes, and deal with whatever that causes.

comment:26 Changed 18 months ago by michaelld (Michael Dickens)

Agreed, Chris. I'll do a WIP PR hopefully later today.

Updates from the past week, mostly discussing the new Boost 1.70.0 with others who have also been trying it out:

Disable the new CMake build scripts provided by Boost, as they are not yet ready for prime time; just use the one provided by CMake as it should work as currently expected by other ports. I'll post that portfile tweak once I've confirmed it, per ...

I'm updating Boost on my 10.14 build system today & will start testing dependent ports as the week goes on, hopefully creating something of an automated script to do so. Others can also test dependent ports. Once we hit a reasonable threshold (80%? 90%?) of them being functional, I'd be ready to commit & work on fixing up the remainder afterwards.

comment:27 Changed 18 months ago by cjones051073 (Chris Jones)

Cc: cjones051073 added

comment:28 Changed 18 months ago by michaelld (Michael Dickens)

Cc: cjones051073 removed

OK I have a WIP PR for testing < https://github.com/macports/macports-ports/pull/4243 >. In my testing thus far, it works very well overall, with the primary -port- issue being the MPI variant, which fails in destroot & I'm not sure how to fix it yet. Please test / verify & report back successes / failures!

comment:29 Changed 18 months ago by michaelld (Michael Dickens)

I know that the gnuradio ports currently fail because of an ASIO API change. I'm working on fixing them.

comment:30 Changed 17 months ago by ryandesign (Ryan Schmidt)

Cc: stanimura added

Has duplicate #58511.

comment:31 Changed 17 months ago by michaelld (Michael Dickens)

I've fixed gnuradio-next to work with this Boost, and have pushed the fix to an upstream PR. I need to start adding rev-bumps of compatible ports to the WIP PR listed above.

comment:32 Changed 17 months ago by olupton (Olli Lupton)

Cc: olupton added

comment:33 Changed 16 months ago by gallafent

Cc: gallafent added

comment:34 Changed 15 months ago by blair (Blair Zajac)

I started working on a PR for the boost package. Two of the patches needed work, you can find them in my local branch: https://github.com/blair/macports-ports/tree/boost-1.70.0 .

comment:35 Changed 15 months ago by michaelld (Michael Dickens)

@blair: Thanks for your work!

Did you try the +mpi option? NP if not; just wondering since I think it was broken when I tried it with Boost 1.70.0.

Also: Looks like you deleted one (patch-random-uniform_on_sphere.hpp.diff) & removed a chunk from another (patch-boost-python3.diff) ... but that's it, yes?

Last edited 15 months ago by michaelld (Michael Dickens) (previous) (diff)

comment:36 Changed 15 months ago by michaelld (Michael Dickens)

Resolution: fixed
Status: assignedclosed

In b09ef91d8de72290191eb87b0b33641b37e5a5ca/macports-ports (master):

boost: update to 1.70.0

1) tweak current patches to apply cleanly.

2) Add a patch to revert the ABI naming to what we have historically had in MacPorts: "libboost_COMPONENT-mt.dylib", where COMPONENT is the component name e.g., "system" or "thread" or the like.

Closes: #56294

Note: See TracTickets for help on using tickets.