Opened 3 weeks ago

Last modified 31 hours ago

#71162 assigned defect

xar @1.8.0.501_0: error: implicit declaration of function 'xar_prop_get_expect_notnull' is invalid in C99 [-Werror,-Wimplicit-function-declaration]

Reported by: ryandesign (Ryan Carsten Schmidt) Owned by: Schamschula (Marius Schamschula)
Priority: High Milestone:
Component: ports Version: 2.10.2
Keywords: bigsur, monterey Cc: lukaso (Lukas Oberhuber)
Port: xar

Description

On Monterey arm64 and x86_64, xar fails to build:

src/xar.c:678:11: error: implicit declaration of function 'xar_prop_get_expect_notnull' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        if( 0 != xar_prop_get_expect_notnull((xar_file_t)x, "checksum/offset" ,&value) ) {
                 ^

xar is a dependency of llvm ports so this will prevent the build of any port needing a newer clang.

Attachments (2)

xar.diff (4.0 KB) - added by ryandesign (Ryan Carsten Schmidt) 3 weeks ago.
work in progress
xar-main.log (79.0 KB) - added by lukaso (Lukas Oberhuber) 3 days ago.
main.log

Download all attachments as: .zip

Change History (20)

comment:1 Changed 3 weeks ago by Schamschula (Marius Schamschula)

I am well aware of this!

However, when I rolled back the failing builds to the previous version, the build bot insisted in attempting to build the incorrect version of xar.

I don't see the logic error in the Portfile.

I attempted to fix the very old macOS versions by removing Big Sur and Monterey from the if statements in [9df0a503e8490c0bb7eda2af37c6cfd9df0c9e89/macports-ports] but that changed nothing.

These two OS versions need their own patch file anyway, as there are different implicit function definitions that are missing. The problem is that I have no way of testing on anything older than macOS Sonoma.

Last edited 3 weeks ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:2 Changed 3 weeks ago by Schamschula (Marius Schamschula)

Also see: #71151

Last edited 3 weeks ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:3 in reply to:  1 Changed 3 weeks ago by ryandesign (Ryan Carsten Schmidt)

Replying to Schamschula:

However, when I rolled back the failing builds to the previous version, the build bot insisted in attempting to build the incorrect version of xar.

I don't see the logic error in the Portfile.

I'm not sure what you mean. The port versions seem to be what they should be based on the portfile code:

% for osmajor in $(seq 8 24); do printf %2d: $osmajor; port info --version xar os.major=$osmajor; done 
 8:version: 1.8.0.498
 9:version: 1.8.0.498
10:version: 1.8.0.498
11:version: 1.8.0.498
12:version: 1.8.0.498
13:version: 1.8.0.498
14:version: 1.8.0.501
15:version: 1.8.0.501
16:version: 1.8.0.501
17:version: 1.8.0.501
18:version: 1.8.0.501
19:version: 1.8.0.501
20:version: 1.8.0.501
21:version: 1.8.0.501
22:version: 1.8.0.501
23:version: 1.8.0.501
24:version: 1.8.0.501

Buildbot is building xar 501 on macOS 11 and 12 like you're telling it to. Buildbot is not special; the same thing that happens on Buildbot workers happens on user systems as well.

These two OS versions need their own patch file anyway, as there are different implicit function definitions that are missing.

I haven't yet dug into their code but do you know why this differs by OS version?

The problem is that I have no way of testing on anything older than macOS Sonoma.

It is possible to set up virtual machines running older macOS versions, though that of course takes time and disk space to set up and I wouldn't expect anyone to do that.

comment:4 in reply to:  description Changed 3 weeks ago by ryandesign (Ryan Carsten Schmidt)

Keywords: bigsur added

Replying to ryandesign:

src/xar.c:678:11: error: implicit declaration of function 'xar_prop_get_expect_notnull' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        if( 0 != xar_prop_get_expect_notnull((xar_file_t)x, "checksum/offset" ,&value) ) {
                 ^

xar_prop_get_expect_notnull is declared in xar.h and defined in xar.c. xar.c does #include "xar.h" thanks to patch-stdout.diff. Seems like that should be enough.

However, I see that /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/xar/xar.h also exists and it does not declare xar_prop_get_expect_notnull. It seems to be including this file instead of the one included with the xar source. I'm not sure why. The command looks right to me:

/usr/bin/clang -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -arch x86_64  -Iinclude -Iinclude -I/opt/local/include -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/opt/local/include/libxml2 -c src/xar.c -o src/xar.o

I thought -I paths would be looked in first, before -isysroot and -isystem paths. But the log shows it is accessing the file from the SDK:

src/xar.c:186:6: warning: 'xar_open' is deprecated: first deprecated in macOS 12.0 - xar is a deprecated file format and should not be used. [-Wdeprecated-declarations]
        x = xar_open(filename, WRITE);
            ^
/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk/usr/include/xar/xar.h:158:7: note: 'xar_open' has been explicitly marked deprecated here

Different OS versions including different versions of xar.h would certainly account for why this problem occurs on some OS versions and not others, but I don't think we want to use the OS version of xar.h on any OS version.

I see that xar.c does also #include "xar_internal.h", before including xar.h. xar_internal.h does this:

#ifdef XARSIG_BUILDING_WITH_XAR
#include "xar.h"
#else
#include <xar/xar.h>
#endif // XARSIG_BUILDING_WITH_XAR

So if XARSIG_BUILDING_WITH_XAR is defined then we wouldn't need to patch xar.c to include xar.h. And if XARSIG_BUILDING_WITH_XAR is not defined then the include of <xar/xar.h> will certainly find the system version, not the one in the xar source.

xar.xcodeproj/project.pbxproj defines XARSIG_BUILDING_WITH_XAR:

				OTHER_CFLAGS = "-DXARSIG_BUILDING_WITH_XAR";

Maybe all we need to do is define that in the portfile as well, since the portfile doesn't build using the Xcode project. I can try it in a moment.

comment:5 Changed 3 weeks ago by ryandesign (Ryan Carsten Schmidt)

Regardless whether I define XARSIG_BUILDING_WITH_XAR or reinstate the part of the patch of xar_internal.h that you removed in the 501 update, I get this error on macOS 12:

lib/archive.h:45:10: fatal error: 'CommonCrypto/CommonDigestSPI.h' file not found
#include <CommonCrypto/CommonDigestSPI.h>
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.

I don't have such a header in /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/CommonCrypto on macOS 12 but I don't see it on a macOS 15 machine either. I see a hint in webkit source code that this is an Apple internal header, so we can't use it.

Wherever this header is included, it is only when __APPLE__ is defined. There is a fallback to openssl for other systems. Maybe we need to take that fallback in MacPorts too.

Attaching my work in progress.

Changed 3 weeks ago by ryandesign (Ryan Carsten Schmidt)

Attachment: xar.diff added

work in progress

comment:6 Changed 3 weeks ago by Schamschula (Marius Schamschula)

I think I mistakenly uploaded a newer version of patch-src-xar_internal.h.old.diff. I've uploaded the correct version: [a3d54e33fd4716dcfb060c320750d64b1050f533/macports-ports]

Note: for version 501 xar_internal.h has a line

xar_t xar_fdopen_digest_verify(int fd, int32_t flags, void *expected_toc_digest, size_t expected_toc_digest_len) API_AVAILABLE(macos(15.0));

Does that not mean that this is only available with the macOS 15 API, i.e. Xcode 16.x? However, it compiled fine with Xcode 15.4, after I patched the file.

P.S. Yes, I'm aware that I can run older OS versions in virtual machines. I've done so in the past. It isn't a hardware issue (I have a collection of Macs going back to the original item) as much as a time issue: I'm teaching a double load this semester!

Last edited 3 weeks ago by ryandesign (Ryan Carsten Schmidt) (previous) (diff)

comment:7 in reply to:  6 Changed 3 weeks ago by ryandesign (Ryan Carsten Schmidt)

Replying to Schamschula:

Note: for version 501 xar_internal.h has a line

xar_t xar_fdopen_digest_verify(int fd, int32_t flags, void *expected_toc_digest, size_t expected_toc_digest_len) API_AVAILABLE(macos(15.0));

Does that not mean that this is only available with the macOS 15 API, i.e. Xcode 16.x?

I'm not really familiar with this. I have read Marking API Availability in Objective-C. I think it means that xar_fdopen_digest_verify can only be used when the deployment target is 15.0 or later. xar doesn't ever call xar_fdopen_digest_verify. It appears to be just a library function that will be in libxar for others to use if they want. And if they did want to use it, they would have to do so only when the deployment target is 15.0 or later. I'm not sure if one should be able to build this library when the deployment target is less than 15.0.

But that seems orthogonal to the point that the xar build is using the system xar.h header rather than the one that ships with xar because you removed the fix for #64842.

comment:8 Changed 3 weeks ago by Schamschula (Marius Schamschula)

The portion I removed from the patch file is included in the upstream version of src/xar_internal.h in version 501.

comment:9 Changed 3 weeks ago by Marius Schamschula <mschamschula@…>

In 458829893a027c296ee62b2c054d8a4fa641ba66/macports-ports (master):

xar: add missing funcxtion definition to patch-src-xar_internal.h.diff

See: #71162

comment:10 Changed 5 days ago by lukaso (Lukas Oberhuber)

Is there a working version of xar? My build is now broken and looking at the history I can't tell when it was broken so I don't know what to build an overlay of. Or could the brokenness be rolled back? Many thanks!

comment:11 Changed 5 days ago by Schamschula (Marius Schamschula)

Please be more specific: what OS version and variant.

comment:12 in reply to:  10 Changed 5 days ago by ryandesign (Ryan Carsten Schmidt)

Cc: lukaso added

Replying to Schamschula:

The portion I removed from the patch file is included in the upstream version of src/xar_internal.h in version 501.

I was referring to the fact that you removed this change from the patchfile. You reinstated it for 498 on older systems but it is still missing for 501 on current systems. Therefore 501 on current systems is using the system xar.h header instead of the one from the port, and this is at least part of the reason why 501 isn't building on older systems and you had to downgrade older systems to 498.

Replying to lukaso:

My build is now broken

In what way?

comment:13 Changed 5 days ago by lukaso (Lukas Oberhuber)

My build is now broken

In what way?

Simply that xar won't build (I have to build everything from source for the GIMP build because I target older SDKs).

comment:14 Changed 3 days ago by ryandesign (Ryan Carsten Schmidt)

Please attach the main.log file.

Changed 3 days ago by lukaso (Lukas Oberhuber)

Attachment: xar-main.log added

main.log

comment:15 in reply to:  13 Changed 37 hours ago by barracuda156

Replying to lukaso:

My build is now broken

In what way?

Simply that xar won't build (I have to build everything from source for the GIMP build because I target older SDKs).

On a side note, maybe you could take a look at this when you get a chance? #69525 The same problem was confirmed on 10.5.8 ppc.

comment:16 in reply to:  13 Changed 35 hours ago by ryandesign (Ryan Carsten Schmidt)

Replying to lukaso:

Simply that xar won't build (I have to build everything from source for the GIMP build because I target older SDKs).

Ok, that probably explains it. I see in your log that you are on macOS 15 but using the macOS 11 SDK and deployment target. As we know from this ticket, our xar build inadvertently uses the xar.h header from the SDK instead of the one from the new version of xar. This works ok with the current macOS SDK but not with older SDKs that have the older xar.h header. The port should be fixed so that it uses the xar.h header from the port, not the SDK, by reinstating the fix for #64842. However, per my work in progress, that hasn't been sufficient to allow building again; I have not investigated further.

The quickest fix for your situation is to use version 498 instead of 501.

comment:17 Changed 31 hours ago by lukaso (Lukas Oberhuber)

The quickest fix for your situation is to use version 498 instead of 501.

I did an overlay with a version of the xar port from this summer. That worked so I'm OK until this is fixed. Thanks.

comment:18 Changed 31 hours ago by lukaso (Lukas Oberhuber)

On a side note, maybe you could take a look at this when you get a chance? #69525 The same problem was confirmed on 10.5.8 ppc.

I would love to. However, I've had to stop supporting 2.10.38 (time constraints) to focus on 3.0, and have also dropped support for MacOS below 11. (All related to challenges of cross-compiling dependencies to older macs and to intel from arm and resources while being the only maintainer for GIMP on the Mac -- with a few people helping out.)

So we are counting on the kindness of others on this.

Note: See TracTickets for help on using tickets.