hdf5, hdf5-18 header & library runtime error

[almage (Roy Miller)]

Hi, we are experiencing runtime errors with netcdf and nacre (ncl commands) that have to do with the hdf5 headers used for compilation, and the hdf5-18 library used during compilation of netcdf.

system on which this occurs: Mac OS X 10.10.5 Xcode 7.2.1

The runtime error can be generated by doing a simple "ncdump" on a netcdf4 format file, shown here below:

Tue Sep 20 10:45:33 beowulf test> ncdump tro3_Amon_reanalysis_ENS_198101-201012.3D.cg.ltm.nc 
Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.10.0, library is 1.8.16
	    SUMMARY OF THE HDF5 CONFIGURATION
	    =================================

General Information:
-------------------
		   HDF5 Version: 1.8.16
		  Configured on: Wed Aug  3 06:43:51 PDT 2016
		  Configured by: macports@tenten-slave.macports.org
		 Configure mode: production
		    Host system: i386-apple-darwin14.5.0
	      Uname information: Darwin tenten-slave.macports.org 14.5.0 Darwin Kernel Version 14.5.0: Mon Jan 11 18:48:35 PST 2016; root:xnu-2782.50.2~1/RELEASE_X86_64 x86_64
		       Byte sex: little-endian
		      Libraries: static, shared
	     Installation point: /opt/local/lib/hdf5-18

Compiling Options:
------------------
               Compilation Mode: production
                     C Compiler: /usr/bin/clang
                         CFLAGS: -pipe -Os 
                      H5_CFLAGS:  
                      AM_CFLAGS: 
                       CPPFLAGS: -I/opt/local/include
                    H5_CPPFLAGS:   -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS: 
               Shared C Library: yes
               Static C Library: yes
  Statically Linked Executables: no
                        LDFLAGS: -L/opt/local/lib -Wl,-headerpad_max_install_names 
                     H5_LDFLAGS: 
                     AM_LDFLAGS: 
 	 	Extra libraries: -lz -ldl -lm 
 		       Archiver: ar
 		 	 Ranlib: ranlib
 	      Debugged Packages: 
		    API Tracing: no

Languages:
----------
                        Fortran: no

                            C++: yes
                   C++ Compiler: /usr/bin/clang++
                      C++ Flags: -pipe -Os  -stdlib=libc++
                   H5 C++ Flags:  
                   AM C++ Flags: 
             Shared C++ Library: yes
             Static C++ Library: yes

Features:
---------
                  Parallel HDF5: no
             High Level library: yes
                   Threadsafety: no
            Default API Mapping: v18
 With Deprecated Public Symbols: yes
         I/O filters (external): deflate(zlib)
                            MPE: no
                     Direct VFD: no
                        dmalloc: no
Clear file buffers before write: yes
           Using memory checker: no
         Function Stack Tracing: no
      Strict File Format Checks: no
   Optimization Instrumentation: no
Bye...
Abort trap: 6

It appears that hdf5-18 is necessary for the netcdf and hdfeos5 ports, while hdf5 is necessary for a handful of others. These are the ports we are using that are dependent upon a version of hdf5:

Tue Sep 20 12:26:35 beowulf ~> port depend hdf5
cdo depends on hdf5
grads depends on hdf5
grib_api depends on hdf5
nco depends on hdf5
ncview depends on hdf5
wgrib2 depends on hdf5
Tue Sep 20 12:26:42 beowulf ~> port depend hdf5-18
hdfeos5 depends on hdf5-18
netcdf depends on hdf5-18

I've been unable to get all these dependent ports to install, using only one of the hdf5 ports, it appears I must use both for the compilations. Then we have the runtime headers/library problem.

ideas?

thanks, Roy

[dershow]

Cc Me!

[aaschwanden (Andy Aschwanden)]

That's been a constant source of frustration for me too.

As far as I can tell, there is not good reason to keep hdf5-18 around anymore, since the latest release of hdf5 is now compatible with netcdf. Can't we just get rid of the hdf5-18 port?

As a work around, I've replaced "hdf5-18" with "hdf5" in all port files that request hdf5-18 (netcdf, py-tables). Works for me, but it's not a nice solution.

[mf2k (Frank Schima)]

Here are all the ports that depend on hdf5-18. They must be updated to use hdf5 and tested first.

$ port echo depends:hdf5-18
h4h5tools                       
hdfeos5                         
netcdf                          
octave-devel                    
py27-tables                     
py34-tables                     
py35-tables                     
py36-tables                     
ncarg                

[mamoll (Mark Moll)]

hdf5 1.10 is still not supported in py-tables: ​https://github.com/PyTables/PyTables/issues/545

[mf2k (Frank Schima)]

Replying to aaschwanden:

As far as I can tell, there is not good reason to keep hdf5-18 around anymore, since the latest release of hdf5 is now compatible with netcdf. Can't we just get rid of the hdf5-18 port?

I have created ticket:54101 for this issue.

[Dave-Allured (Dave Allured)]

Replying to aaschwanden:

As far as I can tell, there is not good reason to keep hdf5-18 around anymore, since the latest release of hdf5 is now compatible with netcdf. Can't we just get rid of the hdf5-18 port?

(2017 May 10) Hdf5-18 solves two particular problems: creation of backward incompatible hdf5 data files, and broken network file access with hdf5 version 1.10.0. The first problem was solved only in netcdf-4.4.1, and not necessarily in any other hdf5-using packages. AFIK, the second problem is still extant for many if not all Macs using network file servers. Hdf5 version 1.10.1 provides an improved solution for the network file problem.

I urge caution. Please see ticket:54101 for more details.

[mf2k (Frank Schima)]

Now that no port depends on hdf5-18 anymore (#54101), can we close this ticket?

[Dave-Allured (Dave Allured)]

Replying to mf2k:

Now that no port depends on hdf5-18 anymore (#54101), can we close this ticket?

Yes, please close this ticket. Thanks.