hdf4: macOS arm64 version produces wrong results

[svniemeijer (Sander Niemeijer)]

When running the following program against any of the files from ​https://nsidc.org/data/NISE

#include <assert.h>
#include <stdio.h>
#include "hdf.h"
#include "mfhdf.h"

int main(int argc, char *argv[])
{
    int32 sd_id, sds_id, rank, dimsizes[H4_MAX_VAR_DIMS], data_type, num_attributes, i;
    char sds_name[H4_MAX_NC_NAME + 1];
    int result;

    sd_id = SDstart("NISE_SSMISF18_20210111.HDFEOS", DFACC_READ);
    sds_id = SDselect(sd_id, 0);
    result = SDgetinfo(sds_id, sds_name, &rank, dimsizes, &data_type, &num_attributes);
    assert(result == 0);
    for (i = 0; i < rank; i++)
    {
        printf("%d\n", dimsizes[i]);
    }
    return 0;
}

On x86_64 this gives 721 (twice). But on arm64 it gives -788398080 (twice).

The data is rank 2, 721 x 721. So the x86_64 output is correct.

It looks like the arm64 build of HDF4 is incorrectly treating the platform as big endian.

[mf2k (Frank Schima)]

In the future, please fill in the Port field and add the port maintainer(s) to Cc (port info --maintainers hdf4), if any.

[ryandesign (Ryan Carsten Schmidt)]

Replying to svniemeijer:

When running the following program against any of the files from ​https://nsidc.org/data/NISE

#include <assert.h>
#include <stdio.h>
#include "hdf.h"
#include "mfhdf.h"

int main(int argc, char *argv[])
{
    int32 sd_id, sds_id, rank, dimsizes[H4_MAX_VAR_DIMS], data_type, num_attributes, i;
    char sds_name[H4_MAX_NC_NAME + 1];
    int result;

    sd_id = SDstart("NISE_SSMISF18_20210111.HDFEOS", DFACC_READ);
    sds_id = SDselect(sd_id, 0);
    result = SDgetinfo(sds_id, sds_name, &rank, dimsizes, &data_type, &num_attributes);
    assert(result == 0);
    for (i = 0; i < rank; i++)
    {
        printf("%d\n", dimsizes[i]);
    }
    return 0;
}

To compile this program, I saved it as test.c and ran:

cc -o test test.c -I/opt/local/include -L/opt/local/lib -lmfhdf -ldf

On x86_64 this gives 721 (twice).

I agree, I got the same output on macOS 10.15.7 on x86_64.

But on arm64 it gives -788398080 (twice).

I agree, I got the same output on macOS 11.6.5 on arm64.

The data is rank 2, 721 x 721. So the x86_64 output is correct.

It looks like the arm64 build of HDF4 is incorrectly treating the platform as big endian.

On both x86_64 and arm64, configure output says:

checking whether byte ordering is bigendian... no

so that seems right, but it appears that this result isn't actually used everywhere (or perhaps anywhere). Instead, many places in the code decide what to do based on CPU-specific defines, and an arm64 define is not included, resulting in the wrong decision being made.

Debian and Fedora each have different patchsets for fixing this compatibility problem with arm64 and the other systems they support.

I built my own patch focused just on adding arm64 support for macOS and it worked for me: your test program now returns the right result on arm64 and the test suite also comes much closer to passing on arm64.

​https://github.com/macports/macports-ports/pull/14595

[ryandesign (Ryan Carsten Schmidt)]

In 96e97e5126e54fbddf51ca1049fe8d773f1f9f1a/macports-ports (master):

hdf4: Fix test and runtime failures on arm64

Closes: #62341