dav1d @0.8.2: build fails on leopard PPC (at least) with several errors

[kencu (Ken)]

/usr/include/stdlib.h:266:1: error: unknown type name 'u_int32_t'; did you mean 'uint32_t'?

leopard doesn't have this:

src/thread.h:150:5: warning: implicit declaration of function 'pthread_setname_np'; did you mean 'pthread_main_np'?

and then

:info:build  #error Missing aligned alloc implementation

[kencu (Ken)]

adding legacysupport didn't fix all this, although it will take care of the last one I believe.

[ballapete (Peter "Pete" Dyballa)]

This data type is defined here:

/usr/include/ppc/types.h:90:typedef	unsigned int		u_int32_t;
/usr/include/i386/types.h:90:typedef	unsigned int		u_int32_t;

[ballapete (Peter "Pete" Dyballa)]

In ffmpeg-devel's Portfile:

188     if {${os.major} > 9} {
189         # libsdl2 requires minimum Xcode 10.7 SDK to build successfully
190         # but builds on Snow Leopard
191         configure.args-replace --disable-sdl2 --enable-sdl2
192         depends_lib-append     port:libsdl2
193 
194         # Dav1d: 10.6+ No PowerPC or Apple Silicon (for now)
195         if {![regexp -all (arm64|ppc*) [get_canonical_archs]]} {
196             configure.args-append   --enable-libdav1d
197             depends_lib-append      port:dav1d
198         }
199     }

[kencu (Ken)]

It is -- too bad, in a way -- that we can't somehow just "ask" dav1d if it supports ppc, or a given system version, and if it says it does, then include it... but no, we just need to know that in all the portfiles that might include it.,

[kencu (Ken)]

I suppose dav1d should set it's supported archs to what it can support. at least that would give us the right error faster...

[ballapete (Peter "Pete" Dyballa)]

Could be the problem is much simple – stdlib.h on PPC Leopard has:

  253	#if !defined(_ANSI_SOURCE) && (!defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE))
  254	#include <machine/types.h>
  255	
  256	#ifndef _DEV_T
  257	typedef	__darwin_dev_t	dev_t;
  258	#define _DEV_T
  259	#endif
  260	
  261	#ifndef	_MODE_T
  262	typedef	__darwin_mode_t	mode_t;
  263	#define _MODE_T
  264	#endif
  265	
  266	u_int32_t
  267		 arc4random(void);

<machine/types.h> then includes the proper C header file, either /usr/include/ppc/types.h or /usr/include/i386/types.h. Could be the guard prevents <machine/types.h> from being included… I'll find that in a few hours because right now port is trying to upgrade some trivial ports like sqlite3 or python37…

I am trying to solve such problems with C header files by letting port try to build the port and fail. Then I have in *compilation* buffer the compiler invocation. By changing to the proper working directory I can try to establish the same environment settings and invoke the compiler with a small change. By adding -dD and -E to the C compiler switches I can see in the pre-processed output file why an inclusion happened or not. Whether the C header was included can be found by adding -H to the C compiler switches.

[ballapete (Peter "Pete" Dyballa)]

To me it looks as if just an implementation of some memalign function is missing:

   58	/*
   59	 * Allocate align-byte aligned memory. The return value can be released
   60	 * by calling the dav1d_free_aligned() function.
   61	 */
   62	static inline void *dav1d_alloc_aligned(size_t sz, size_t align) {
   63	    assert(!(align & (align - 1)));
   64	#ifdef HAVE_POSIX_MEMALIGN
   65	    void *ptr;
   66	    if (posix_memalign(&ptr, align, sz)) return NULL;
   67	    return ptr;
   68	#elif defined(HAVE_ALIGNED_MALLOC)
   69	    return _aligned_malloc(sz, align);
   70	#elif defined(HAVE_MEMALIGN)
   71	    return memalign(align, sz);
   72	#else
   73	#error Missing aligned alloc implementation
   74	#endif
   75	}

[ballapete (Peter "Pete" Dyballa)]

The trouble starts with private shadowed C header files:

. ./config.h
. /usr/include/string.h
.. /usr/include/_types.h
... /usr/include/sys/_types.h
.... /usr/include/sys/cdefs.h
.... /usr/include/machine/_types.h
..... /usr/include/ppc/_types.h
. ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/common/intops.h
.. /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stdint.h
... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include-fixed/stdint.h
.. ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/common/attributes.h
... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
. ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/cdef_apply.h
.. ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/common/bitdepth.h
.. ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/internal.h
... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stdatomic.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/data.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
.... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/common.h
..... /usr/include/errno.h
...... /usr/include/sys/errno.h
..... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/cdef.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/cdf.h
.... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/levels.h
..... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/headers.h
...... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
.... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/ref.h
..... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/dav1d.h
...... /usr/include/errno.h
...... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stdarg.h
...... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/picture.h
....... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
....... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/headers.h
...... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/data.h
...... include/dav1d/version.h

..... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/mem.h
...... /usr/include/stdlib.h
....... /usr/include/available.h
....... /usr/include/sys/wait.h
........ /usr/include/sys/signal.h
......... /usr/include/sys/appleapiopts.h
......... /usr/include/machine/signal.h
.......... /usr/include/ppc/signal.h
........... /usr/include/ppc/_structs.h
......... /usr/include/sys/_structs.h
.......... /usr/include/machine/_structs.h
........... /usr/include/ppc/_structs.h
............ /usr/include/mach/ppc/_structs.h
........ /usr/include/sys/resource.h
......... /usr/include/sys/_structs.h
........ /usr/include/machine/endian.h
......... /usr/include/ppc/endian.h
.......... /usr/include/sys/_endian.h
....... /usr/include/alloca.h
....... /usr/include/machine/types.h
........ ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/ppc/types.h     ————» /usr/include/ppc/types.h
......... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/altivec.h
...... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/thread.h          «———— line #39 of ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/mem.h
....... /usr/include/pthread.h
........ /usr/include/pthread_impl.h
........ /usr/include/sched.h
........ /usr/include/time.h
......... /usr/include/_structs.h
.......... /usr/include/sys/_structs.h
In file included from ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/ref.h:33:0,
                 from ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/cdf.h:34,
                 from ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/internal.h:44,
                 from ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/cdef_apply.h:33,
                 from ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/cdef_apply_tmpl.c:34:
../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/mem.h:73:2: error: #error Missing aligned alloc implementation
 #error Missing aligned alloc implementation
  ^~~~~
..... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
.... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/thread_data.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/data.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/env.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
.... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/refmvs.h
..... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/intra_edge.h
..... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/tables.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/film_grain.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/ipred.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/itx.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/lf_mask.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/loopfilter.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/looprestoration.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/mc.h
.... /opt/local/lib/gcc7/gcc/ppc-apple-darwin9/7.5.0/include/stddef.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/msac.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/picture.h
.... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/include/dav1d/picture.h
... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/recon.h
.... ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/internal.h
Multiple include guards may be useful for:
/usr/include/_structs.h
/usr/include/machine/_structs.h

Above is the list of C header files included when trying to compile .../dav1d/work/dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/cdef_apply_tmpl.c.

[ballapete (Peter "Pete" Dyballa)]

dav1d's private ppc/types.h just defines some useful data types and conversions for its own use:

#define u8x16 vector unsigned char
#define i8x16 vector signed char
#define b8x16 vector bool char
#define u16x8 vector unsigned short
#define i16x8 vector signed short
#define b16x8 vector bool short
#define u32x4 vector unsigned int
#define i32x4 vector signed int
#define b32x4 vector bool int
#define u64x2 vector unsigned long long
#define i64x2 vector signed long long
#define b64x2 vector bool long long

#define u8h_to_u16(v) ((u16x8) vec_mergeh((u8x16) v, vec_splat_u8(0)))
#define u8l_to_u16(v) ((u16x8) vec_mergel((u8x16) v, vec_splat_u8(0)))
#define u16h_to_i32(v) ((i32x4) vec_mergeh((u16x8) v, vec_splat_u16(0)))
#define i16h_to_i32(v) ((i32x4) vec_unpackh((i16x8)v))
#define u16l_to_i32(v) ((i32x4) vec_mergel((u16x8) v, vec_splat_u16(0)))
#define i16l_to_i32(v) ((i32x4) vec_unpackl((i16x8)v))

[ballapete (Peter "Pete" Dyballa)]

At least there is some hope:

# Roadmap

The plan is the following:

### Reached
1. Complete C implementation of the decoder,
2. Provide a usable API,
3. Port to most platforms,
4. Make it fast on desktop, by writing asm for AVX2 chips.
5. Make it fast on mobile, by writing asm for ARMv8 chips,
6. Make it fast on older desktop, by writing asm for SSSE3+ chips,
7. Make high bit-depth fast on mobile, by writing asm for ARMv8 chips.

### On-going
8. Make it fast on older mobile, by writing asm for ARMv7 chips,
9. Make high bit-depth fast on older mobile, by writing asm for ARMv7 chips,
10. Improve C code base with [various tweaks](https://code.videolan.org/videolan/dav1d/wikis/task-list),
11. Accelerate for less common architectures, like PPC, SSE2 or AVX-512.

### After
12. Make high bit-depth fast on desktop, by writing asm for AVX2 chips,
13. Make high bit-depth fast on older desktop, by writing asm for SSSE3+ chips,
14. Use more GPU, when possible.


## Will you care about <my_arch>? <my_os>?

- We do, but we don't have either the time or the knowledge. Therefore, patches and contributions welcome.

(from README.md)

[kencu (Ken)]

Replying to ballapete:

To me it looks as if just an implementation of some memalign function is missing:

   58	/*
   59	 * Allocate align-byte aligned memory. The return value can be released
   60	 * by calling the dav1d_free_aligned() function.
   61	 */
   62	static inline void *dav1d_alloc_aligned(size_t sz, size_t align) {
   63	    assert(!(align & (align - 1)));
   64	#ifdef HAVE_POSIX_MEMALIGN
   65	    void *ptr;
   66	    if (posix_memalign(&ptr, align, sz)) return NULL;
   67	    return ptr;
   68	#elif defined(HAVE_ALIGNED_MALLOC)
   69	    return _aligned_malloc(sz, align);
   70	#elif defined(HAVE_MEMALIGN)
   71	    return memalign(align, sz);
   72	#else
   73	#error Missing aligned alloc implementation
   74	#endif
   75	}

This I put in legacysupport.

[ballapete (Peter "Pete" Dyballa)]

Replying to kencu:

This I put in legacysupport.

This cannot be the solution, because config.h contains only:

#define ARCH_AARCH64 0

#define ARCH_ARM 0

#define ARCH_PPC64LE 0

#define ARCH_X86 0

#define ARCH_X86_32 0

#define ARCH_X86_64 0

#define CONFIG_16BPC 1

#define CONFIG_8BPC 1

#define CONFIG_LOG 1

#define ENDIANNESS_BIG 1

#define HAVE_ASM 0

#define HAVE_UNISTD_H 1

#define PREFIX 1

Some real function is needed instead of #error.

(HAVE_ASM is 0 although I have:

  nasm @2.15.05_0 (active)
  yasm @1.3.0_0 (active)

which indicates that ARCH_PPC64BE is not ready yet.)

[kencu (Ken)]

there are other problems, but the aligned memory problem should be fixed by adding legacysupport.

[kencu (Ken)]

In 731f7fca56b6aaa982a408db028ead3e4e927daa/macports-ports (master):

dav1d: use legacysupport

for posix_memalign, on < 10.6

see: #62618

[kencu (Ken)]

In c97f750d7a9fda9c0fbfad21cf4f8940f831c787/macports-ports (master):

dav1d: no pthread_setname on Leopard

see: #62618

[kencu (Ken)]

with those two minor changes, dav1d builds back to Tiger i386.

Now we can see about the PPC issue.

[kencu (Ken)]

well, success:

$ port -v installed dav1d
The following ports are currently installed:
  dav1d @0.8.2_0 (active) platform='darwin 9' archs='ppc' date='2021-04-03T18:08:16-0700'

but -- there is some kind of a problem with legacysupport, it appears. u_int32_t is supposed to be defined when stdlib.h has run through it's various headers, but it is not being defined, for reasons that I am as yet not clear on.

I added a manual definition in legacysupport's stdlib.h to make it work, but that is not supposed to be needed, so there is some kind of an issue. In:

/opt/local/include/LegacySupport/stdlib.h

I added this:

  #endif /*__ENABLE_MP_LEGACY_SUPPORT_REALPATH_WRAP__*/
+ typedef	unsigned int		u_int32_t;
  /* Include the primary system stdlib.h */
  #include_next <stdlib.h>

and the build succeeded.

[kencu (Ken)]

Oh, wait -- this is not a legacysupport issue, as it was happening before legacysupport was added. This is some other kind of issue involving stdlib.h.

Whew. At least that makes it less cumbersome to fix.

[ballapete (Peter "Pete" Dyballa)]

Replying to kencu:

Oh, wait -- this is not a legacysupport issue, as it was happening before legacysupport was added. This is some other kind of issue involving stdlib.h.

Ken, the problem is that dav1d uses its own private machine/«CPU arch»/types.h implementation so that Apple's definition of u_int32_t is not met. We should patch ../dav1d-0.8.2-f06148e7c755098666b9c0ed97a672a51785413a/src/ppc/types.h because *this* is the cause.

I'll try that now!

[kencu (Ken)]

Replying to ballapete:

the problem is that dav1d uses its own private machine/«CPU arch»/types.h implementation

Oh, didn't notice that in any of your previous comments. Should be trivial to patch then.

[ballapete (Peter "Pete" Dyballa)]

Patch to add definition of u_int32_t data type to dav1d's src/ppc/types.h

[ballapete (Peter "Pete" Dyballa)]

Ken, I do not understand how to apply the patch, because there is this hex number component in the path. I am also not allowed to submit this patch to the dav1d developers because I have no hardware devices for two factor authorisation – if you can handle this you have permission to contribute it to the code base.

[kencu (Ken)]

Yes, the problem seems to really be that they have chosen a particularly bad name for their internal header, ppc/types.h, which is overriding the system header of the same name..

At least I think that is what must be causing this, as including stdlib.h should not cause this error to occur, and adding that typedef in that location should not fix it.

[kencu (Ken)]

[eaa16ef5b64fdbded8e47d239b8dceb05f86ef97/macports-ports]

[ctreleaven (Craig Treleaven)]

Are you going to notify upstream?

[kencu (Ken)]

Avec plaisir, mon ami <​https://code.videolan.org/videolan/dav1d/-/issues/363>