cctools 895 breaks port:audacity build

[RJVB (René Bertin)]

After upgrading from cctools 886_6 to 895_4 port:audacity builds fail with the error reported in ticket #53438 .

Audacity doesn't depend on cctools, not even indirectly, but the same error occurs on the 10.12 build bot. This suggests a bug in cctools itself.

[RJVB (René Bertin)]

This error when creating the liblv2.a static library (the one causing a link failure later on) may be related?

/opt/local/bin/ranlib: archive member: ../liblv2.a() size too large (archive member extends past the end of the file)
ar: internal ranlib command failed

[RJVB (René Bertin)]

OK, so the underlying error comes from the way Audacity builds liblv2.a:

function waf
{
   pkg=$1
   pushd >/dev/null ${pkg}
   shift
   /opt/local/bin/python2.7 waf --prefix="." --include="." $@ build || exit 1
   popd >/dev/null 

   . .buildvars

   if [ -e ${pkg}/build/*.a ]
   then
      mkdir -p obj
      pushd obj
      ar vx ../${pkg}/build/*.a
      ar vq ../liblv2.a *
      popd
      rm -rf obj
   fi
}

[ -e liblv2.a ] && exit 0

waf lv2    -vvv --no-plugins
waf serd   -vvv --static --no-shared --no-utils
waf sord   -vvv --static --no-shared --no-utils
waf sratom -vvv --static --no-shared
waf lilv   -vvv --static --no-shared --no-utils
waf suil   -vvv --static --no-shared --no-qt

IOW, different "modules" are built, and if that produces a static lib it is unpacked in a temp. directory, and the archive elements are added in cumulative fashion to liblv2.a . The first step succeeds, but subsequent steps fail in the ranlib stage (possibly because the temp. directory also contains a file with the symbol table?).

Curiously, this took me back in time to an similar issue I had with one of my own projects, and which I solved by using a different approach which also works here:

function waf
{
   pkg=$1
   pushd >/dev/null ${pkg}
   shift
   /opt/local/bin/python2.7 waf --prefix="." --include="." $@ build || exit 1
   popd >/dev/null 

   . .buildvars

   if [ -e ${pkg}/build/*.a ]
   then
      mkdir -p obj.${pkg}
      pushd obj.${pkg}
      ar vx ../${pkg}/build/*.a
      popd
   fi
}

[ -e liblv2.a ] && exit 0

waf lv2    -vvv --no-plugins
waf serd   -vvv --static --no-shared --no-utils
waf sord   -vvv --static --no-shared --no-utils
waf sratom -vvv --static --no-shared
waf lilv   -vvv --static --no-shared --no-utils
waf suil   -vvv --static --no-shared --no-qt

# now create liblv2.a in a single call
libtool -static -o liblv2.a obj.serd/*.o obj.sord/*.o obj.sratom/*.o obj.lilv/*.o obj.suil/*.o
ar -sv liblv2.a

IOW, unpack the intermediate archives into dedicated directories (they'll contain files with the same names) and then use libtool -static -o to create the definitive archive.

[jeremyhu (Jeremy Huddleston Sequoia)]

It sounds like this is an audacity bug and not a cctools bug then ... yes?

[RJVB (René Bertin)]

Replying to jeremyhu:

It sounds like this is an audacity bug and not a cctools bug then ... yes?

Honestly, I don't think so. If cctools' ar command doesn't support adding object files to existing static libraries that is not audacity's fault.

Idem if one cannot do

> mkdir combine
> cd combine
> ar x /lib/libfoo.a
> ar x /lib/libbar.a
> ar vq libfoobar.a *

To the best of my knowledge that is standard (if possibly somewhat forgotten) UNIX functionality. It's OK for cctools not to support it as long as there are alternative ways to achieve the same thing, but in that case it should raise an error, not create a corrupt archive that (much) later on gives a useless error.

When something works for a whole series of releases and then breaks this way it has a bug. Either a down-to-earth regression or a (long-standing) one in the documentation and input checking/error handling.

[jeremyhu (Jeremy Huddleston Sequoia)]

The issue isn't that it can't. The error indicates (at least as I read it) that the archive being constructed is too large and thus cannot be expressed by the spec. I saw this issue with WebKit a while ago. I suspect you're hitting the same filesize limit.

[RJVB (René Bertin)]

No, size is not an issue here. The working version is 1234544 bytes, the faulty version was the same or almost the same size.

libWebKit is a few orders of magnitude larger; in my builds it's 3Gb (and still works).The problem with that library is that when built with debug info it becomes too large for 32bit systems, at least that's what a comment in the Qt Portfile says.