subversion +no_bdb still requires Berkeley DB

[com-macosforge@…]

% port installed | grep db
  apr-util @1.3.4_1+no_bdb (active)
  db46 @4.6.21_5 (active)
  subversion @1.6.2_0+no_bdb+no_neon (active)
  subversion-perlbindings @1.6.2_0+no_bdb+no_neon (active)
% sudo port uninstall db46
--->  Unable to uninstall db46 4.6.21_5, the following ports depend on it:
--->  	apr-util
--->  	subversion
--->  	subversion-perlbindings
Error: port uninstall failed: Please uninstall the ports that depend on db46 first.

Note that no other variants of the ports are installed. So I try anyway:

% sudo port uninstall -f db46
--->  Unable to uninstall db46 4.6.21_5, the following ports depend on it:
--->  	apr-util
--->  	subversion
--->  	subversion-perlbindings
Warning: Uninstall forced.  Proceeding despite dependencies.
--->  Deactivating db46 @4.6.21_5
--->  Uninstalling db46 @4.6.21_5
% svn
dyld: Library not loaded: /opt/local/lib/db46/libdb-4.6.dylib
  Referenced from: /opt/local/bin/svn
  Reason: image not found
[1]    88696 trace trap  svn

While you're at it, the same dependency problem happens with neon, but at least if you force an uninstall of neon, svn doesn't encounter a link error.

[danielluke (Daniel J. Luke)]

I can replicate this by installing apr-util without +no_bdb first, installing serf (which ends up with an undeclared dependency on db46) uninstalling apr-util, installing apr-util with +no_bdb and then installing subversion with +no_bdb.

Is this what happened for you?

In this case, if I remove serf, install apr-util with +no_bdb, install serf, and then install subversion with +no_bdb, I don't end up with subversion linked to bdb.

[com-macosforge@…]

I just installed subversion, which gave me serf whether I wanted it or not (as well as neon). After that, I reinstalled with the no_bdb+no_neon variants; although I ended up reinstalling neon because I didn't realize serf can't do digest auth yet.

[danielluke (Daniel J. Luke)]

OK, so at least part of the problem is that if serf builds against an apr-util that is linked to bdb, it ends up linked to bdb but doesn't declare a dependency on bdb.

Blair - you have any other input on this? If serf is linked to bdb, subversion ends up linked to it even when built with --without-berkeley-db

[blair (Blair Zajac)]

A couple of thoughts.

1) serf should not list the dependencies of it's direct dependents.

That gets serf into having the same variants as apr-util, which I don't want.

2) MacPorts is not like RPM or other packaging systems that can

dynamically determine the .so's or .dylibs that the package requires, so that's why it falls down here. For example, rpmbuild would have seen that libserf-0.dylib depends upon libdb-4.6.dylib so it would not have let you uninstall db46.

3) If one recompiles a base library port and changes the variants that

it uses that involve linking other shared libraries, then all the dependent ports needs to be recompiled. This can be accomplished with the -R -f option to port I believe, which I haven't tried.

So in the end, this is a known problem and I think the ticket should be closed as won't fix since it's an issue with MacPorts, not this specific port. The reporter should just rebuild the entire stack of software that uses apr-util.

[danielluke (Daniel J. Luke)]

Replying to blair@…:

A couple of thoughts.

1) serf should not list the dependencies of it's direct dependents.

That gets serf into having the same variants as apr-util, which I don't want.

While that might be true, serf does have a library dependency on bdb when built against an apr-util that also has a library dependency on bdb (which is the default).

It should probably declare a dependency on bdb (and have a +no_bdb variant that just removes that dependency) so that MacPorts warns the end user that removing bdb will break serf.

2) MacPorts is not like RPM or other packaging systems that can

dynamically determine the .so's or .dylibs that the package requires, so that's why it falls down here. For example, rpmbuild would have seen that libserf-0.dylib depends upon libdb-4.6.dylib so it would not have let you uninstall db46.

true, I'm sure a patch to base would be welcome ;-)

[blair (Blair Zajac)]

I'm not going to do that to serf. That implies I need to implement all these variants that apr-util has:

$ port variants apr-util
apr-util has the variants:
        no_bdb: Build without BerkeleyDB support
        no_sqlite: Build without sqlite support
        openldap: Build with OpenLDAP support
        postgresql83: Build with postgres support (using postgresql v8.3)
        postgresql82: Build with postgres support (using postgresql v8.2)
        mysql5: Build with MySQL5 suppoprt

What happens when somebody upgrades from postgresql83 to 84 when that version comes out of beta? That now becomes my problem? No thank you.

The real solution is to have MacPorts scan the port's *.dylib and:

a) add those files to the files that the port provides b) add the otool -L output on those *dylib that are not provided by the

port to the port's requirements.

For now, people need to recompile all dependent ports when changing variants.

[danielluke (Daniel J. Luke)]

Replying to blair@…:

I'm not going to do that to serf. That implies I need to implement all these variants that apr-util has

only if the resulting serf links against those libraries

an alternative would be to fix serf to not link against libraries it doesn't need.

The real solution is to have MacPorts scan the port's *.dylib and:

a) add those files to the files that the port provides b) add the otool -L output on those *dylib that are not provided by the

port to the port's requirements.

A patch to base to do this would be nice, but I would guess that many people would say that even if port could auto-detect these kinds of dependencies, they should still be declared in the portfile.

[blair (Blair Zajac)]

Serf doesn't need to explicitly list those dependencies since it lists a dependency upon apr-util.

Serf links against all these libraries for me:

$ otool -L /opt/local/lib/libserf-0.dylib
/opt/local/lib/libserf-0.dylib:
        /Users/blair/my-macports/lib/libserf-0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
        /Users/blair/my-macports/lib/libaprutil-1.0.dylib (compatibility version 4.0.0, current version 4.4.0)
        /Users/blair/my-macports/lib/libldap-2.3.0.dylib (compatibility version 3.0.0, current version 3.23.0)
        /Users/blair/my-macports/lib/libsasl2.2.dylib (compatibility version 3.0.0, current version 3.22.0)
        /Users/blair/my-macports/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
        /Users/blair/my-macports/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
        /Users/blair/my-macports/lib/liblber-2.3.0.dylib (compatibility version 3.0.0, current version 3.23.0)
        /Users/blair/my-macports/lib/db46/libdb-4.6.dylib (compatibility version 0.0.0, current version 0.0.0)
        /Users/blair/my-macports/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0)
        /Users/blair/my-macports/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
        /Users/blair/my-macports/lib/libapr-1.0.dylib (compatibility version 4.0.0, current version 4.3.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.10)
        /Users/blair/my-macports/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
        /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

I'm not going to add explicit dependencies to them and then have to track each change in apr-util.

This should be taken on macports-dev instead.

[danielluke (Daniel J. Luke)]

Replying to blair@…:

Serf doesn't need to explicitly list those dependencies since it lists a dependency upon apr-util.

The MacPorts convention would be to include all libraries that a port depends directly on in the portfile even if one of its dependents also brings in that library.

I'm not going to add explicit dependencies to them and then have to track each change in apr-util.

The other option would be to fix the serf build process so it doesn't link in those libraries unless it needs symbols from them.

One small change to configure.in (line 174) would prevent SERF_LIBS from getting the extra libraries, but it will still link against them because we install the .la files for apr-util.

Perhaps that needs to change (I think other package mangers don't install the .la files along with the libraries)? I see a 2005 reference to ArchLinux switching over here ​http://www.archlinux.org/news/174/

This should be taken on macports-dev instead.

[blair (Blair Zajac)]

Replying to dluke@…:

Replying to blair@…:

Serf doesn't need to explicitly list those dependencies since it lists a dependency upon apr-util.

The MacPorts convention would be to include all libraries that a port depends directly on in the portfile even if one of its dependents also brings in that library.

That's a suggestion I'm not in support of as a convention for MacPorts.

I'm not going to add explicit dependencies to them and then have to track each change in apr-util.

The other option would be to fix the serf build process so it doesn't link in those libraries unless it needs symbols from them.

One small change to configure.in (line 174) would prevent SERF_LIBS from getting the extra libraries, but it will still link against them because we install the .la files for apr-util.

If you have a patch to reduce the number of libraries, I can apply it to serf.

Perhaps that needs to change (I think other package mangers don't install the .la files along with the libraries)? I see a 2005 reference to ArchLinux switching over here ​http://www.archlinux.org/news/174/

I have third party internal C++ binaries that use the Subversion API and need the .la files to properly link against them, so removing them is a not a good idea.

I should also note that Ubuntu includes all the .la files in the dev packages. So we can split the .la files into another package, say subversion-dev.

[blair (Blair Zajac)]

It looks like this issue got resolved in the latest apr-util 1.3.7 where it now links against BDB at runtime.

​http://www.apache.org/dist/apr/CHANGES-APR-UTIL-1.3

  *) Introduce DSO handling of the db, gdbm and ndbm drivers, so these are
     loaded as .so's on first demand, unless --disable-util-dso is configured.
     [William Rowe]

The differences between 1.3.4 and 1.3.7's otool -L output:

 /opt/local/lib/libaprutil-1.dylib:
-	/Users/blair/my-macports/lib/libaprutil-1.0.dylib (compatibility version 4.0.0, current version 4.4.0)
-	/Users/blair/my-macports/lib/db46/libdb-4.6.dylib (compatibility version 0.0.0, current version 0.0.0)
+	/Users/blair/my-macports/lib/libaprutil-1.0.dylib (compatibility version 4.0.0, current version 4.7.0)
 	/Users/blair/my-macports/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0)
 	/Users/blair/my-macports/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
-	/Users/blair/my-macports/lib/libapr-1.0.dylib (compatibility version 4.0.0, current version 4.3.0)
-	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.3)
+	/Users/blair/my-macports/lib/libapr-1.0.dylib (compatibility version 4.0.0, current version 4.5.0)
+	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
 	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

Note that libdb-4.6.dylib is missing in the new lines. I then recompiled serf and got this difference:

 /opt/local/lib/libserf-0.dylib:
 	/Users/blair/my-macports/lib/libserf-0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
-	/Users/blair/my-macports/lib/libaprutil-1.0.dylib (compatibility version 4.0.0, current version 4.4.0)
+	/Users/blair/my-macports/lib/libaprutil-1.0.dylib (compatibility version 4.0.0, current version 4.7.0)
 	/Users/blair/my-macports/lib/libldap-2.3.0.dylib (compatibility version 3.0.0, current version 3.23.0)
 	/Users/blair/my-macports/lib/libsasl2.2.dylib (compatibility version 3.0.0, current version 3.22.0)
 	/Users/blair/my-macports/lib/libssl.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
 	/Users/blair/my-macports/lib/libcrypto.0.9.8.dylib (compatibility version 0.9.8, current version 0.9.8)
 	/Users/blair/my-macports/lib/liblber-2.3.0.dylib (compatibility version 3.0.0, current version 3.23.0)
-	/Users/blair/my-macports/lib/db46/libdb-4.6.dylib (compatibility version 0.0.0, current version 0.0.0)
 	/Users/blair/my-macports/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 7.2.0)
 	/Users/blair/my-macports/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
-	/Users/blair/my-macports/lib/libapr-1.0.dylib (compatibility version 4.0.0, current version 4.3.0)
-	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.3)
+	/Users/blair/my-macports/lib/libapr-1.0.dylib (compatibility version 4.0.0, current version 4.5.0)
+	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.4)
 	/Users/blair/my-macports/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.3)
 	/usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)

Again libdb-4.6.dylib is missing. So this is good progress.

[danielluke (Daniel J. Luke)]

Excellent. The new apr-util .la file includes the -L to db46, but not the db46 lib in dependency_libs, so this should be resolved now. Please re-open this ticket if there are still problems (with a fresh build of apr-util and serf).