Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#59510 closed defect (fixed)

gdal @3.0.1_3: Build fails, argument list too long

Reported by: Dave-Allured (Dave Allured) Owned by: Veence (Vincent)
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: cooljeanius (Eric Gallager), AgilentGCMS
Port: gdal

Description

I am building gdal 3.0.1 from source using Macports. I am building into a custom user directory, and not using sudo. "Port install" proceeds to compile many gdal components without trouble. Then it fails with "Argument list too long" on a very long libtool command. See main.log which I am attaching.

Here is the point of failure from main.log. The offending command line is 252733 characters long. There are 1267 arguments to the libtool command, most of them containing long full paths (only 3 of those shown here). I added line breaks for clarity, and truncated. See main.log for the original, unmodified long command.

:info:build /Library/Developer/CommandLineTools/usr/bin/make libgdal.la
:info:build make[1]: Entering directory `/Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal'
:info:build /bin/sh /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/libtool
        --mode=link
        --silent
        /usr/bin/clang++
        -std=c++14
        -L/Users/dallured/port/262/lib
        -Wl,-headerpad_max_install_names
        -stdlib=libc++
        -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk
        -arch x86_64
        -lcrypto
        -lqhull
        -lqhull
        -L/Users/dallured/port/262/lib
        -lgeos_c
        -lwebp
<snip>
        /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/alg/contour.lo
        /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/alg/delaunay.lo
        /Volumes/User/dallured/port/262/var/macports/build/_Volumes_User_dallured_port_262_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.0.1/gdal/alg/gdal_crs.lo
<snip>
:info:build 	    -rpath /Users/dallured/port/262/lib \
:info:build 	    -no-undefined \
:info:build 	    -version-info 26:1:0
:info:build make[1]: /bin/sh: Argument list too long
:info:build make[1]: *** [libgdal.la] Error 1

I think this is a corner case, convergence of two factors:

  1. Major version update from gdal 2.x to 3.x. Version 2.x built fine for me recently.
  2. Custom install prefix which is longer than the usual /opt/local.

What is the best approach to fix this? Devise a Macports patch, or request a fix upstream? Is there a Macports approach to reduce duplication of the same 1267 very long file prefixes?

Attachments (1)

main.log (6.6 MB) - added by Dave-Allured (Dave Allured) 3 years ago.
Main.log from build failure, Argument list too long, October 28

Change History (14)

Changed 3 years ago by Dave-Allured (Dave Allured)

Attachment: main.log added

Main.log from build failure, Argument list too long, October 28

comment:1 in reply to:  description Changed 3 years ago by ryandesign (Ryan Schmidt)

Replying to Dave-Allured:

I think this is a corner case, convergence of two factors:

  1. Major version update from gdal 2.x to 3.x. Version 2.x built fine for me recently.
  2. Custom install prefix which is longer than the usual /opt/local.

You're probably right. Version 3 probably has more files than version 2.

What is the best approach to fix this?

Can you shorten your custom MacPorts prefix?

Devise a Macports patch, or request a fix upstream?

We are probably not doing anything special to build this, so if anyone can fix it it's upstream.

Is there a Macports approach to reduce duplication of the same 1267 very long file prefixes?

Nope. It's their build system causing the absolute paths to be used. You could ask them to use relative paths instead of absolute ones, but if they are just using autotools and libtool in the normal way then it may not be under their control.

comment:2 in reply to:  description Changed 3 years ago by ryandesign (Ryan Schmidt)

Replying to Dave-Allured:

  1. Custom install prefix which is longer than the usual /opt/local.

Not just the length of the prefix, but also the length of the path to the ports tree. You could sync with git, saving the clone in a shorter path like /opt/ports.

comment:3 Changed 3 years ago by Dave-Allured (Dave Allured)

Problem solved. There is now a makefile change which will link using much shorter relative paths. I will await the next GDAL release.

https://github.com/OSGeo/gdal/issues/2075#issuecomment-560321087

comment:4 Changed 3 years ago by cooljeanius (Eric Gallager)

Cc: cooljeanius added

comment:5 Changed 3 years ago by reneeotten (Renee Otten)

Cc: AgilentGCMS added

comment:6 Changed 3 years ago by AgilentGCMS

So what does this mean? The issue has been fixed in gdal, and we wait for that fix to make its way into the macports tree?

comment:7 Changed 3 years ago by mf2k (Frank Schima)

Yes. If you want to help move things along, submit a GitHub pull request with the fixed version.

comment:8 Changed 3 years ago by Dave-Allured (Dave Allured)

Has duplicate: #60011

Last edited 3 years ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:9 Changed 3 years ago by Dave-Allured (Dave Allured)

GDAL version 3.0.3 was released on 2020 January 8. This version should include the fix for "Argument list too long".

Update request was filed on January 24: #60009

Last edited 3 years ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:10 Changed 3 years ago by Veence (Vincent)

I have committed the new version. Please let me know if that new release fixes this issue

comment:11 Changed 3 years ago by AgilentGCMS

Indeed, macports now builds gdal without errors.

comment:12 Changed 3 years ago by Veence (Vincent)

Resolution: fixed
Status: assignedclosed

Excellent! Then I can close this ticket. Thanks for testing!

comment:13 Changed 3 years ago by Dave-Allured (Dave Allured)

The new version (gdal 3.0.3 and portfile) works for me too. Very good to have this problem cleaned up. Thanks, Veence and Even of OSGeo/GDAL.

Note: See TracTickets for help on using tickets.