Opened 2 years ago

Last modified 14 months ago

#61967 reopened defect

cctools @927.0.2: not fully compatible with llvm after llvm10: Unknown command line argument '-macho'.

Reported by: blair (Blair Zajac) Owned by: jeremyhu (Jeremy Huddleston Sequoia)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: michaelld (Michael Dickens), jungx098
Port: cctools

Description (last modified by kencu (Ken))

When cctools +llvm11 is installed than gobject-introspection fails to build:

CPPFLAGS="-I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk" CFLAGS="-pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum" LDFLAGS="-L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64" CC="/usr/bin/clang -arch x86_64" PKG_CONFIG="/opt/local/bin/pkg-config" GI_HOST_OS="" DLLTOOL="false"  env PATH=".libs:/opt/local/bin:/opt/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin" LPATH=.libs CC="/usr/bin/clang -arch x86_64" PYTHONPATH=".:.:" UNINSTALLED_INTROSPECTION_SRCDIR=. UNINSTALLED_INTROSPECTION_BUILDDIR=. ./g-ir-scanner  --verbose -I. --add-include-path=. --add-include-path=./gir --add-include-path=. --add-include-path=. --add-include-path=./gir --namespace=GLib --nsversion=2.0 --libtool="/bin/sh ./libtool" --pkg=glib-2.0     --library=glib-2.0 --library=gobject-2.0 --external-library --reparse-validate --identifier-prefix=G --symbol-prefix=g --symbol-prefix=glib --c-include="glib.h"  --cflags-begin -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -DGETTEXT_PACKAGE=Dummy -DGLIB_COMPILATION -D__G_I18N_LIB_H__ --cflags-end  /opt/local/lib/glib-2.0/include/glibconfig.h /opt/local/include/glib-2.0/glib/galloca.h /opt/local/include/glib-2.0/glib/garray.h /opt/local/include/glib-2.0/glib/gasyncqueue.h /opt/local/include/glib-2.0/glib/gatomic.h /opt/local/include/glib-2.0/glib/gbacktrace.h /opt/local/include/glib-2.0/glib/gbase64.h /opt/local/include/glib-2.0/glib/gbitlock.h /opt/local/include/glib-2.0/glib/gbookmarkfile.h /opt/local/include/glib-2.0/glib/gbytes.h /opt/local/include/glib-2.0/glib/gcharset.h /opt/local/include/glib-2.0/glib/gchecksum.h /opt/local/include/glib-2.0/glib/gconvert.h /opt/local/include/glib-2.0/glib/gdataset.h /opt/local/include/glib-2.0/glib/gdate.h /opt/local/include/glib-2.0/glib/gdatetime.h /opt/local/include/glib-2.0/glib/gdir.h /opt/local/include/glib-2.0/glib/genviron.h /opt/local/include/glib-2.0/glib/gerror.h /opt/local/include/glib-2.0/glib/gfileutils.h /opt/local/include/glib-2.0/glib/ggettext.h /opt/local/include/glib-2.0/glib/ghash.h /opt/local/include/glib-2.0/glib/ghmac.h /opt/local/include/glib-2.0/glib/ghook.h /opt/local/include/glib-2.0/glib/ghostutils.h /opt/local/include/glib-2.0/glib/gi18n-lib.h /opt/local/include/glib-2.0/glib/gi18n.h /opt/local/include/glib-2.0/glib/giochannel.h /opt/local/include/glib-2.0/glib/gkeyfile.h /opt/local/include/glib-2.0/glib/glist.h /opt/local/include/glib-2.0/glib/gmacros.h /opt/local/include/glib-2.0/glib/gmain.h /opt/local/include/glib-2.0/glib/gmappedfile.h /opt/local/include/glib-2.0/glib/gmarkup.h /opt/local/include/glib-2.0/glib/gmem.h /opt/local/include/glib-2.0/glib/gmessages.h /opt/local/include/glib-2.0/glib/gnode.h /opt/local/include/glib-2.0/glib/goption.h /opt/local/include/glib-2.0/glib/gpattern.h /opt/local/include/glib-2.0/glib/gpoll.h /opt/local/include/glib-2.0/glib/gprimes.h /opt/local/include/glib-2.0/glib/gprintf.h /opt/local/include/glib-2.0/glib/gqsort.h /opt/local/include/glib-2.0/glib/gquark.h /opt/local/include/glib-2.0/glib/gqueue.h /opt/local/include/glib-2.0/glib/grand.h /opt/local/include/glib-2.0/glib/grcbox.h /opt/local/include/glib-2.0/glib/grefcount.h /opt/local/include/glib-2.0/glib/grefstring.h /opt/local/include/glib-2.0/glib/gregex.h /opt/local/include/glib-2.0/glib/gscanner.h /opt/local/include/glib-2.0/glib/gsequence.h /opt/local/include/glib-2.0/glib/gshell.h /opt/local/include/glib-2.0/glib/gslice.h /opt/local/include/glib-2.0/glib/gslist.h /opt/local/include/glib-2.0/glib/gspawn.h /opt/local/include/glib-2.0/glib/gstdio.h /opt/local/include/glib-2.0/glib/gstrfuncs.h /opt/local/include/glib-2.0/glib/gstring.h /opt/local/include/glib-2.0/glib/gstringchunk.h /opt/local/include/glib-2.0/glib/gtestutils.h /opt/local/include/glib-2.0/glib/gthread.h /opt/local/include/glib-2.0/glib/gthreadpool.h /opt/local/include/glib-2.0/glib/gtimer.h /opt/local/include/glib-2.0/glib/gtimezone.h /opt/local/include/glib-2.0/glib/gtrashstack.h /opt/local/include/glib-2.0/glib/gtree.h /opt/local/include/glib-2.0/glib/gtypes.h /opt/local/include/glib-2.0/glib/gunicode.h /opt/local/include/glib-2.0/glib/gurifuncs.h /opt/local/include/glib-2.0/glib/gutils.h /opt/local/include/glib-2.0/glib/guuid.h /opt/local/include/glib-2.0/glib/gvariant.h /opt/local/include/glib-2.0/glib/gvarianttype.h /opt/local/include/glib-2.0/glib/gversion.h /opt/local/include/glib-2.0/glib/gversionmacros.h /opt/local/include/glib-2.0/glib/gwin32.h /opt/local/include/glib-2.0/gobject/glib-types.h gir/glib-2.0.c /opt/local/include/glib-2.0/glib-unix.h g-ir-scanner g-ir-compiler --output GLib-2.0.gir
/usr/bin/clang -arch x86_64 -E -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -DGETTEXT_PACKAGE=Dummy -DGLIB_COMPILATION -D__G_I18N_LIB_H__ -U__BLOCKS__ -I. -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2 -o g-ir-cpp-ygnxvj6t.i -C /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/g-ir-cpp-ygnxvj6t.c
/usr/bin/clang -arch x86_64 -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -D_REENTRANT -DGETTEXT_PACKAGE=Dummy -DGLIB_COMPILATION -D__G_I18N_LIB_H__ -I/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2 -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -I/opt/local/include/glib-2.0 -I/opt/local/lib/glib-2.0/include -c /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.c -o /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.o -Wno-deprecated-declarations
warning: unknown warning option '-Wno-duplicated-branches' [-Wunknown-warning-option]
1 warning generated.
g-ir-scanner: link: /bin/sh ./libtool --mode=link --tag=CC /usr/bin/clang -arch x86_64 -o /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0 -export-dynamic -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.o -lgio-2.0 -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -lintl -Wl,-framework -Wl,CoreFoundation -lglib-2.0 -lgobject-2.0 -L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64
libtool: link: /usr/bin/clang -arch x86_64 -o /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0 -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -Wno-duplicated-branches -Wno-switch-enum /opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0.o -Wl,-framework -Wl,CoreFoundation -Wl,-headerpad_max_install_names -Wl,-syslibroot -Wl,/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64  -lgio-2.0 -lgmodule-2.0 -lintl -lglib-2.0 -lgobject-2.0 -L/opt/local/lib
llvm-objdump: Unknown command line argument '-macho'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '-h'?
llvm-objdump: Unknown command line argument '-dylibs-used'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--dylibs-used'?
llvm-objdump: Unknown command line argument '-non-verbose'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--non-verbose'?
Traceback (most recent call last):
  File "./g-ir-scanner", line 99, in <module>
    sys.exit(scanner_main(sys.argv))
  File "./giscanner/scannermain.py", line 579, in scanner_main
    shlibs = create_binary(transformer, options, args)
  File "./giscanner/scannermain.py", line 419, in create_binary
    shlibs = resolve_shlibs(options, binary, options.libraries)
  File "./giscanner/shlibs.py", line 183, in resolve_shlibs
    _resolve_non_libtool(options, binary, non_libtool))
  File "./giscanner/shlibs.py", line 103, in _resolve_non_libtool
    output = subprocess.check_output(args)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/bin/sh', './libtool', '--mode=execute', 'otool', '-L', '/opt/local/var/macports/build/_Users_blairzajac_Code_MacPorts_macports-ports.git_gnome_gobject-introspection/gobject-introspection/work/gobject-introspection-1.60.2/tmp-introspectnchs_fhs/GLib-2.0']' returned non-zero exit status 1.
make[2]: *** [GLib-2.0.gir] Error 1

When cctools +llvm90 is installed than gobject-introspection builds fine.

Do all the calls to add_execute_list() in otool/main.c need to be updated to add another - to the argument?

Change History (27)

comment:1 Changed 2 years ago by kencu (Ken)

Hi Blair, this is a known issue, and is not a bug -- they did this on purpose. There are a number of other tickets against various software builds that ran into the same thing.

Newer versions of llvm come with a new version of llvm-objdump, and they decided upstream to change the arguments that llvm-objdump accepts. llvm-objdump will no longer accept -macho. You can use -m and you can use --macho now. All the old scripts that use -macho are broken, and will need to be gradually updated to no longer do that.

If you want to get on with your work, please do this:

sudo port -f uninstall cctools
sudo port -v install cctools

and then you should get a proper working version of cctools for your system, which is cctools +xcode.

If you want to help fix all the scripts that are broken, you can stay on cctools +llvm11 and start submitting PRs (upstream, preferably) to get them all updated.

Last edited 2 years ago by kencu (Ken) (previous) (diff)

comment:2 Changed 2 years ago by jmroot (Joshua Root)

Cc: kencu added
Owner: set to jeremyhu
Status: newassigned

comment:3 Changed 2 years ago by kencu (Ken)

Resolution: wontfix
Status: assignedclosed

comment:4 Changed 2 years ago by kencu (Ken)

Resolution: wontfix
Status: closedreopened

comment:5 Changed 2 years ago by kencu (Ken)

have to do a bit of sleuthing to sort out if it's gobject-introspection or python that needs to be updated here to the new call.

comment:6 Changed 2 years ago by michaelld (Michael Dickens)

Cc: michaelld added

comment:7 Changed 2 years ago by ryandesign (Ryan Schmidt)

Has duplicate #61901.

comment:8 Changed 23 months ago by jungx098

Cc: jungx098 added

comment:9 Changed 23 months ago by michaelld (Michael Dickens)

has duplicate #62095

comment:10 Changed 23 months ago by michaelld (Michael Dickens)

Copied from #62095 ... where we're telling poppler to save-temps and thus we can test this issue outside the MacPorts' environment.

Looking at the cctools code in cctools-927.0.2/otool/main.c, line 80 reads:

enum bool show_objdump_command = FALSE; /* print the objdump command */

change this to TRUE; build and install. then run the otool command again:

$ otool -L /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_graphics_poppler/poppler/work/build/glib/tmp-introspectdnhi97mq/Poppler-0.18
+ /opt/local/bin/llvm-objdump-mp-11 -macho -dylibs-used -non-verbose /opt/local/var/macports/build/_opt_sources_MacPorts_ports_github_macports_graphics_poppler/poppler/work/build/glib/tmp-introspectdnhi97mq/Poppler-0.18
llvm-objdump: Unknown command line argument '-macho'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '-h'?
llvm-objdump: Unknown command line argument '-dylibs-used'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--dylibs-used'?
llvm-objdump: Unknown command line argument '-non-verbose'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--non-verbose'?

... so ... this is a cctools issue, not GO-I or poppler.

comment:11 Changed 23 months ago by kencu (Ken)

Uhm -- this is expected behaviour from cctools/llvm-objdump (this command comes from the llvm tree), it does not accept "-macho" as an argument any more, it is not a bug, they did this on purpose, they won't be fixing it or reverting it.

The error is this:

/opt/local/bin/llvm-objdump-mp-11 -macho

as -macho is not an accepted flag any more.

Am I backwards on this somehow? I'm multitasking here...

Last edited 23 months ago by kencu (Ken) (previous) (diff)

comment:12 Changed 23 months ago by kencu (Ken)

Oh -- do you mean that somehow the cctools otools driver is calling llvm-objdump and that is what is adding the -macho flag?

If that is what is happening, then yes, 100%, we should patch that out of cctools (or update cctools). Sorry if I missed this first time round.

comment:13 Changed 23 months ago by michaelld (Michael Dickens)

Yes it's this latter: cctools otools driver is calling llvm-objdump and that is what is adding the -macho flag! It's easy to fix in the cctools code luckily!

comment:14 Changed 23 months ago by michaelld (Michael Dickens)

Given that the current cctools variants are just "llvm10, llvm11, llvmdev, [+]xcode", we can patch the cctools otool code to use -- instead of - for those arguments. I think that's the only change required.

comment:15 Changed 23 months ago by kencu (Ken)

those are the only llvm variants on 10.19+, anyway

on 10.4 and up, I need to see if the old llvms accept "--macho"; otherwise we can just use "-m" which is accepted still, for no good reason :>

comment:16 Changed 23 months ago by kencu (Ken)

looks like llvm-objdump-3.4 accepts "--macho" as well as "-m" and "-macho".

So I can patch it to "--macho".

comment:17 Changed 23 months ago by michaelld (Michael Dickens)

makes sense! make it so!

comment:18 Changed 23 months ago by kencu (Ken)

oh oh. Looks like there is a whole mess of single-hyphen arguments that are no longer accepted:

% otool -l /opt/local/bin/mtoc
llvm-objdump: Unknown command line argument '-private-headers'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--private-headers'?
llvm-objdump: Unknown command line argument '-non-verbose'.  Try: '/opt/local/libexec/llvm-11/bin/llvm-objdump --help'
llvm-objdump: Did you mean '--non-verbose'?
Last edited 23 months ago by kencu (Ken) (previous) (diff)

comment:19 Changed 23 months ago by kencu (Ken)

I smell my cctools update coming ...

comment:20 Changed 23 months ago by kencu (Ken)

Description: modified (diff)
Summary: cctools +llvm11 fails to build gobject-introspection: Unknown command line argument '-macho'.cctools @927.0.2: not fully compatible with llvm11 Unknown command line argument '-macho'.

comment:21 Changed 23 months ago by kencu (Ken)

Resolution: fixed
Status: reopenedclosed

In 3c027d9f80b35954bd37e5d715874527643e6bab/macports-ports (master):

cctools: remove llvm11 variant

there is no released version of cctools
that is compatible with the llvm11
calling conventions.

we'll remove the llvm11 variant until such
time as an appropriate version of cctools
becomes available.

users who have cctools with an llvm variant
of < 11 will see an upgrade.

users who have cctools with llvm11 will be
updated to the appropriate version for their
systems

closes: #61967

comment:22 Changed 23 months ago by kencu (Ken)

So -- why?

a lot of scripts, and people, use otool. otool is binary driver for llvm-objdump. otool has compiled into it the flags for llvm-objdump that map to the old otool flags. It translates the flags, then calls llvm-objdump.

but the flags that llvm-objdump accepts have changed, and otools now adds flags that llvm-objdump in llvm11 does not support, ergo the errors.

It's not just one -- there are a whole bunch of them.

And, perhaps more such issues, in other parts of cctools. This is the only one we've found so far. The newer version of cctools that is out there, 949, has the same issues, and it has not been fixed there yet either. No doubt some newer open source version of cctools will be coming out eventually that does support the newer llvm calling conventions.

Now sure, I could take a stab at patching otool to emit the new flags. One or two, easy -- but there are lots. And maybe more? And maybe in other tools?

There is no good reason to cause this grief upon our users. AFAIK, llvm-10 was OK, and that makes sense, timeline wise... cctools is about that vintage.

So we will have most users on newer systems using cctools +xcode, as they should be using. And we'll make available llvm10, as that was not visibly broken.

And for those few users out there who really want the newest, know what they are doing, and can hack in on it -- cctools +llvmdev is still available.

I'll re-enable cctools +llvm11 when it works properly.

Last edited 23 months ago by kencu (Ken) (previous) (diff)

comment:23 Changed 23 months ago by michaelld (Michael Dickens)

Thanks for your detailed thoughts here, @kencu . Makes sense all around IMHO! Glad to get this issue fixed!

comment:24 Changed 15 months ago by kencu (Ken)

Resolution: fixed
Status: closedreopened

This issue is still open, until I update cctools to some new version that has the new calling conventions in it. A few folks can hit it when playing with cctools +llvmdev (unless I decide to remove that option to stop that).

Last edited 15 months ago by kencu (Ken) (previous) (diff)

comment:25 Changed 15 months ago by kencu (Ken)

Summary: cctools @927.0.2: not fully compatible with llvm11 Unknown command line argument '-macho'.cctools @927.0.2: not fully compatible with llvm after llvm10: Unknown command line argument '-macho'.

comment:26 Changed 15 months ago by kencu (Ken)

has duplicate #63448

comment:27 Changed 14 months ago by kencu (Ken)

Cc: kencu removed
Note: See TracTickets for help on using tickets.