Opened 4 weeks ago

Closed 13 days ago

#60625 closed defect (fixed)

Failed to build gdal (Catalina 10.15.5 Macports 2.6.2)

Reported by: avillasenorh (Antonio Villaseñor) Owned by: Veence (Vincent)
Priority: Normal Milestone:
Component: ports Version: 2.6.2
Keywords: Cc: dershow, 3add3287 (3add3287)
Port: gdal

Description (last modified by ryandesign (Ryan Schmidt))

I am installing gdal in a new MacBook Pro 16" with latest version of Catalina (10.15.5) and Macports (2.6.2)

I run:

sudo port install gdal +hdf5 +netcdf +openjpeg

And I get the following error:

.
.
.
--->  Building gdal
Error: Failed to build gdal: command execution failed
Error: See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/main.log for details.
Error: Follow https://guide.macports.org/#project.tickets to report a bug.
Error: Processing of port gdal failed

The last lines of the log file are:

:info:build make[1]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/frmts'
:info:build make: *** [frmts-target] Error 2
:info:build make: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal'
:info:build Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal" && /usr/bin/make -j16 -w
:info:build Exit code: 2
:error:build Failed to build gdal: command execution failed
:debug:build Error code: CHILDSTATUS 44466 2
:debug:build Backtrace: command execution failed
:debug:build     while executing
:debug:build "system {*}$notty {*}$nice $fullcmdstring"
:debug:build     invoked from within
:debug:build "command_exec build"
:debug:build     (procedure "portbuild::build_main" line 8)
:debug:build     invoked from within
:debug:build "$procedure $targetname"
:error:build See /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/main.log for details.

Attachments (2)

main.log (5.8 MB) - added by avillasenorh (Antonio Villaseñor) 3 weeks ago.
main.log file
main.2.log (5.8 MB) - added by avillasenorh (Antonio Villaseñor) 3 weeks ago.
main.log file without the +openjpeg variant

Change History (23)

comment:1 Changed 4 weeks ago by ryandesign (Ryan Schmidt)

Description: modified (diff)
Owner: set to Veence
Port: gdal added
Status: newassigned

Please attach the entire main.log. We cannot see what's wrong based just on the few lines you posted.

comment:2 Changed 3 weeks ago by ryandesign (Ryan Schmidt)

gdal @3.1.0 built fine on our Catalina buildbot worker so if you're seeing a build error you'll need to provide us with your main.log file so we can see what went wrong.

Changed 3 weeks ago by avillasenorh (Antonio Villaseñor)

Attachment: main.log added

main.log file

comment:3 Changed 3 weeks ago by kencu (Ken)

try building it without the +openjpeg variant.

Changed 3 weeks ago by avillasenorh (Antonio Villaseñor)

Attachment: main.2.log added

main.log file without the +openjpeg variant

comment:4 Changed 3 weeks ago by avillasenorh (Antonio Villaseñor)

I left out the +openjpeg variant and run:

sudo port install gdal +hdf5 +netcdf 

I still got an error. I attached the new main.log file

Last edited 13 days ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:5 Changed 3 weeks ago by kencu (Ken)

Ok. I had hoped that turning off the openjpeg variant would stop this, but it didn't

:info:build /bin/sh /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/libtool --mode=compile --silent --tag=CXX /usr/bin/clang++ -std=c++14 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gcore -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/alg -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr/ogrsf_frmts -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gnm -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/apps -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -pipe -DGDAL_COMPILATION -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64  -Wall -Wextra -Winit-self -Wunused-parameter -Wformat -Werror=format-security -Wno-format-nonliteral -Wshorten-64-to-32 -Wshadow -Werror=vla -Wdate-time -Wnull-dereference -Wextra-semi -Wcomma -Wfloat-conversion -Wdocumentation -Wno-documentation-deprecated-sync -Wunused-private-field -Wmissing-declarations -Wnon-virtual-dtor -Woverloaded-virtual -fno-operator-names -Wzero-as-null-pointer-constant -Wimplicit-fallthrough  -I..  -I/opt/local/include -I/opt/local/include -I/opt/local/lib/proj6/include  -DGNM_ENABLED -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port  -DGDAL_COMPILATION -I/opt/local/include -DHAVE_SPATIALITE -DSPATIALITE_412_OR_LATER -DHAVE_PCRE -DSQLITE_HAS_COLUMN_METADATA -c -o ../o/ogrsqliteutility.lo ogrsqliteutility.cpp
:info:build jpeglsdataset.cpp:126:35: error: no member named 'ImageTypeNotSupported' in 'charls::jpegls_errc'
:info:build         case CharlsApiResultType::ImageTypeNotSupported: return "ImageTypeNotSupported";
:info:build              ~~~~~~~~~~~~~~~~~~~~~^
:info:build jpeglsdataset.cpp:127:35: error: no member named 'UnsupportedBitDepthForTransform' in 'charls::jpegls_errc'; did you mean 'UnsupportedColorTransform'?
:info:build         case CharlsApiResultType::UnsupportedBitDepthForTransform: return "UnsupportedBitDepthForTransform";
:info:build              ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:info:build                                   UnsupportedColorTransform
:info:build /opt/local/include/CharLS/public_types.h:360:5: note: 'UnsupportedColorTransform' declared here
:info:build     UnsupportedColorTransform = color_transform_not_supported,
:info:build     ^
:info:build jpeglsdataset.cpp:125:14: error: duplicate case value: 'ParameterValueNotSupported' and 'InvalidCompressedData' both equal '5'
:info:build         case CharlsApiResultType::InvalidCompressedData: return "InvalidCompressedData";
:info:build              ^
:info:build jpeglsdataset.cpp:121:14: note: previous case defined here
:info:build         case CharlsApiResultType::ParameterValueNotSupported: return "ParameterValueNotSupported";
:info:build              ^
:info:build jpeglsdataset.cpp:129:14: error: duplicate case value 'UnsupportedColorTransform'
:info:build         case CharlsApiResultType::UnsupportedColorTransform: return "UnsupportedColorTransform";
:info:build              ^
:info:build jpeglsdataset.cpp:127:14: note: previous case defined here
:info:build         case CharlsApiResultType::UnsupportedBitDepthForTransform: return "UnsupportedBitDepthForTransform";
:info:build              ^
:info:build /bin/sh /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/libtool --mode=compile --silent --tag=CXX /usr/bin/clang++ -std=c++14 -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gcore -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/alg -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/ogr/ogrsf_frmts -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/gnm -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/apps -DHAVE_AVX_AT_COMPILE_TIME -DHAVE_SSSE3_AT_COMPILE_TIME -DHAVE_SSE_AT_COMPILE_TIME -pipe -DGDAL_COMPILATION -stdlib=libc++ -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64  -Wall -Wextra -Winit-self -Wunused-parameter -Wformat -Werror=format-security -Wno-format-nonliteral -Wshorten-64-to-32 -Wshadow -Werror=vla -Wdate-time -Wnull-dereference -Wextra-semi -Wcomma -Wfloat-conversion -Wdocumentation -Wno-documentation-deprecated-sync -Wunused-private-field -Wmissing-declarations -Wnon-virtual-dtor -Woverloaded-virtual -fno-operator-names -Wzero-as-null-pointer-constant -Wimplicit-fallthrough  -DGNM_ENABLED -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/port  -DGDAL_COMPILATION -I/opt/local/include -DHAVE_HDF5 -DNETCDF_HAS_NC4   -c -o ../o/netcdf_sentinel3_sral_mwr.lo netcdf_sentinel3_sral_mwr.cpp
:info:build 4 errors generated.
:info:build make[2]: *** [../o/jpeglsdataset.lo] Error 1
:info:build make[2]: Leaving directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_macports_release_tarballs_ports_gis_gdal/gdal/work/gdal-3.1.0/gdal/frmts/jpegls'
:info:build make[1]: *** [jpegls-install-obj] Error 2
:info:build make[1]: *** Waiting for unfinished jobs....

gdal has a cornucopia of variants, and a little quick factorial math will show you the number of potential combinations of them is astronomical. To be frank, the only build of any port that is ever really tested is the default build, and then usually only on the port maintainer's system. So lots of room for trouble.

Would you consider going with the default build options, which apparently does build according to the buildbot, and then once you are at least in business, look at these variants carefully and see if you really need one or the other of them.

If you do really need one or the other, and that variant won't build, then that is a better place for someone to start to help you.

On the broader front, other package managers on macOS have headed completely away from all variants for this reason. There is one, single, default build available, and no variants are allowed. Less flexible, but much less headache.

comment:6 Changed 3 weeks ago by kencu (Ken)

on another note -- looking at the log, it is surprisingly not very easy for me to confirm the actual variant selection you typed into the command line.

I'm sure it is in that 6MB file somewhere -- maybe -- but it's not obvious (like right at the top of the file, for example, where you would expect it to be) and searching for "variant" in the log shows dozens of hits for everything but what variants you typed into the command line.

comment:7 Changed 3 weeks ago by kencu (Ken)

I think I see what might have gone on.

charls was updated several weeks ago. gdal might need some tweaking to build against the new charls headers.

comment:8 Changed 3 weeks ago by avillasenorh (Antonio Villaseñor)

Many thanks for your help.

I appreciate very much your quick responses and dedication.

I use that combination of gdal because it is in the instructions of the software that I am really interested in (gmt6):

https://github.com/GenericMappingTools/gmt/blob/master/INSTALL.md#install-via-macports

I have installed recently a slightly older version in another Mac with Catalina without any problems. This is the version in the other Mac:

$ port list gdal
gdal                           @3.0.3          gis/gdal

The current version I think it is 3.1.0

Thanks again!!!

Last edited 13 days ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:9 in reply to:  7 ; Changed 3 weeks ago by reneeotten (Renee Otten)

Replying to kencu:

I think I see what might have gone on.

charls was updated several weeks ago. gdal might need some tweaking to build against the new charls headers.

perhaps the changes from this upstream PR are needed? Let's hope that the maintainer can take a look at this.

comment:10 Changed 3 weeks ago by dershow

Cc: dershow added

comment:11 in reply to:  9 Changed 2 weeks ago by Dave-Allured (Dave Allured)

Replying to reneeotten:

Replying to kencu:

I think I see what might have gone on.

charls was updated several weeks ago. gdal might need some tweaking to build against the new charls headers.

perhaps the changes from this upstream PR are needed? Let's hope that the maintainer can take a look at this.

That upstream PR predated the gdal 3.1.0 release by a few days, so it should have been included. The 3.1.0 release notes say something to this effect, "add support for CharLS 2.1".

comment:12 Changed 2 weeks ago by Dave-Allured (Dave Allured)

There may be insights in #60696. Quoting:

What appears to have happened is that charls changed the capitalization of its include directory and library between 2.0.0 and 2.1.0. In 2.0.0, it's CharLS; in 2.1.0, it's charls.

Last edited 13 days ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:13 Changed 2 weeks ago by reneeotten (Renee Otten)

well, how about this upstream PR then ;) Feel free to give that a try and if it works please submit a PR.

Again, I am not using this software - just attempted to point to a possible solution. But yes, on closer inspection you are right that the first PR I listed was indeed already part of the 3.1.0-release.

comment:14 in reply to:  13 Changed 2 weeks ago by Dave-Allured (Dave Allured)

Replying to reneeotten:

well, how about this upstream PR then ;)

I missed that, sorry. Good catch. Indeed, that will gum up the works on case insensitive file systems.

@avillasenorh, by any chance, is /opt/local in a case insensitive file system on your new MacBook Pro?

Feel free to give that a try and if it works please submit a PR.

Again, I am not using this software - just attempted to point to a possible solution. But yes, on closer inspection you are right that the first PR I listed was indeed already part of the 3.1.0-release.

GDAL 3.1.1 will include this new fix. When released, I will update on Macports, if someone else does not get to it first.

https://lists.osgeo.org/pipermail/gdal-dev/2020-June/052314.html

comment:15 Changed 2 weeks ago by 3add3287 (3add3287)

Cc: 3add3287 added

comment:16 Changed 13 days ago by avillasenorh (Antonio Villaseñor)

I use an APFS file system in my MacBook Pro. I assume that /opt/local is case insensitive, since that is the default installation, and I haven't done anything to change it to case sensitive.

Thanks again!

Antonio

comment:17 Changed 13 days ago by ryandesign (Ryan Schmidt)

Cc: bal-agates added
Summary: Failed to build gdal (Catalina 10.15.5 Macports 2.6.2)gdal @3.1.0 is not compatible with charls @2.1.0

Has duplicates #60693 and #60694.

Vince committed [bcc04bf03c5f6579a5a1a85ff795eb9785b2642c/macports-ports] to gdal. Did that fix it?

comment:18 in reply to:  8 Changed 13 days ago by ryandesign (Ryan Schmidt)

Replying to avillasenorh:

$ port list gdal
gdal                           @3.0.3          gis/gdal

"port list" doesn't tell you what you think it tells you.

comment:19 Changed 13 days ago by Veence (Vincent)

Cc: bal-agates removed
Summary: gdal @3.1.0 is not compatible with charls @2.1.0Failed to build gdal (Catalina 10.15.5 Macports 2.6.2)

I have committed a patch that should solve this issue. As a matter of fact, I just compiled/installed GDAL on my brand new MacBook Air with the default FS (Case insensitive), and it worked like a charm, so there’s no reason it should fail.

comment:20 Changed 13 days ago by avillasenorh (Antonio Villaseñor)

I run first: sudo port selfupdate sudo port upgrade outdated

And then the command that gave me problems: sudo port install gdal +hdf5 +netcdf +openjpeg

It now compiles without problems and installs gdal!!!

Thanks everybody! Antonio

comment:21 Changed 13 days ago by Veence (Vincent)

Resolution: fixed
Status: assignedclosed

Perfetto ! :) I close this issue, then.

Note: See TracTickets for help on using tickets.