Opened 7 years ago

Closed 7 years ago

Last modified 7 years ago

#39605 closed defect (fixed)

Move ports: xnu-headers, libc-headers, CarbonHeaders, etc

Reported by: watsodw Owned by: mfeiri
Priority: Normal Milestone:
Component: ports Version: 2.1.3
Keywords: Cc: bernard.j.kelly@…, mymacports@…, jeremyhu (Jeremy Huddleston Sequoia), ryandesign (Ryan Schmidt), cooljeanius (Eric Gallager)
Port: xnu-headers libc-headers dyld-headers CarbonHeaders

Description

clang-3.3 fails to upgrade with error Availability.h not found.

Attachments (2)

main.log (2.8 MB) - added by watsodw 7 years ago.
main_MountainLion.log (4.5 MB) - added by bernard.j.kelly@… 7 years ago.
clang-3.3 install error on Mountain Lion machine

Change History (40)

Changed 7 years ago by watsodw

Attachment: main.log added

comment:1 Changed 7 years ago by cooljeanius (Eric Gallager)

The CarbonHeaders port contains an Availability.h header; try installing that.

comment:2 Changed 7 years ago by larryv (Lawrence Velázquez)

Cc: jeremyhu@… removed
Keywords: snowleopard added
Owner: changed from macports-tickets@… to jeremyhu@…
Summary: clang-3.3 fails to upgrade - Availability.h not foundclang-3.3: upgrade failure on 10.6 - Availability.h not found

comment:3 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Resolution: invalid
Status: newclosed

Availability.h exists in /usr/include on 10.5+

You're missing your dependencies.

comment:4 Changed 7 years ago by watsodw

Availability does exist in my /usr/include, but my setup kept clang from finding it. An install of CarbonHeaders allowed clang to install.

comment:5 Changed 7 years ago by bernard.j.kelly@…

Cc: bernard.j.kelly@… added

Cc Me!

comment:6 Changed 7 years ago by bernard.j.kelly@…

This just happened to me on my Mountain Lion (10.8.4) system.

I've just installed CarbonHeaders, and that appears to have fixed the issue, but I'm wondering why this got flagged as "invalid"? Did earlier clang versions not need this header? If an installed port breaks on "upgrade outdated" when nothing was (consciously) changed by the user, isn't that a real problem?

comment:7 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

See comment:3 for why this is invalid. This is a system header, and if you're missing it, you either deleted it or never installed it.

Last edited 7 years ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:8 in reply to:  7 Changed 7 years ago by bernard.j.kelly@…

Replying to jeremyhu@…:

See comment:3 for why this is invalid. This is a system header, and if you're missing it, you either deleted it or never installed it.

As for david.w.watson's machine above, Availability.h was always sitting in my /usr/include, but MacPorts (or the clang build process) couldn't find it. What environment variable(s) should it be using to look there? Is it possible that this particular release of clang is relying on an environment variable previous versions didn't look at?

Last edited 7 years ago by ryandesign (Ryan Schmidt) (previous) (diff)

comment:9 Changed 7 years ago by larryv (Lawrence Velázquez)

Keywords: snowleopard removed

comment:10 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Well I'm not hitting this on any of my machines. This is almost certainly a configuration error. If you don't think so, please provide a reduced case showing the bug.

comment:11 Changed 7 years ago by bernard.j.kelly@…

Hi Jeremy. I just did the following:

port uninstall CarbonHeaders
port uninstall clang-3.3
port selfupdate
port upgrade outdated
port install clang-3.3 +python27 +assertions

... and hit the same install error. I'll attach the error log, but to give some "reduced case" as you suggest sounds like I'd have to get a clean machine with no other ports installed, which I'm afraid I cannot do. Do you have any suggestion for what I could supply to show the current configuration?

Changed 7 years ago by bernard.j.kelly@…

Attachment: main_MountainLion.log added

clang-3.3 install error on Mountain Lion machine

comment:12 Changed 7 years ago by mymacports@…

Cc: mymacports@… added

Cc Me!

comment:13 Changed 7 years ago by mymacports@…

same problem here.

clang-3.3 fails to upgrade.

OS X 10.8.4

Availability.h is in /usr/include

comment:14 in reply to:  13 Changed 7 years ago by larryv (Lawrence Velázquez)

Resolution: invalid
Status: closedreopened

Replying to mymacports@…:

same problem here.

clang-3.3 fails to upgrade.

OS X 10.8.4

Availability.h is in /usr/include

Please attach a clean main.log.

comment:15 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

:info:build /opt/local/include/sys/stat.h:75:10: fatal error: 'Availability.h' file not found
:info:build #include <Availability.h>

Remove libc-headers, CarbonHeaders, and all those other ports that conflict with system headers and try again.

Also, please file tickets against whatever port depended on those ports, so they can fix the real issue rather than forcing users to break their system with broken ports that don't match system libraries.

I will not support these configurations.

comment:16 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Owner: changed from jeremyhu@… to mfeiri@…
Status: reopenednew

comment:17 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Cc: jeremyhu@… added

Cc Me!

comment:18 Changed 7 years ago by mymacports@…

  1. "... and all those other ports that conflict with system headers"
    There are no other problems except the one with clang-3.3
  1. I do not have libc-headers and/or CarbonHeaders installed.

port installed *headers*
The following ports are currently installed:
  cctools-headers @836_0
  cctools-headers @836_1
  cctools-headers @839_0 (active)
  dyld-headers @210.2.3_0 (active)
  libunwind-headers @35.1_0 (active)
  xnu-headers @2050.18.24_0 (active)


there are no ports in the system that depend on dyld-headers and/or libunwind-headers and/or xnu-headers.

any ideas?

comment:19 in reply to:  18 ; Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Replying to mymacports@…:

  1. "... and all those other ports that conflict with system headers"
    There are no other problems except the one with clang-3.3

As one of the engineers who maintains those projects for OS X, and a MacPorts maintainer who has seen enough tickets from their fallout, I assure you there are.

  1. I do not have libc-headers and/or CarbonHeaders installed.

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

comment:20 in reply to:  19 ; Changed 7 years ago by beany_kelly@…

Replying to jeremyhu@…:

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

Hi Jeremy. I just did the "clean" step before attempting to reinstall clang-3.3, but with the same result.

BUT ...

Your mention of dyld-headers reminded me of an issue I've had for months with MacPorts. Any new installation or upgrade requires that I execute

sudo port [install|upgrade|blah] ...

The first thing I see after this is the message:

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid

Could this be related? It's cropped up in OSX-related boards, and appears within some Macports ticket discussions, but doesn't seem to have its own dedicated ticket ...

comment:21 in reply to:  20 ; Changed 7 years ago by larryv (Lawrence Velázquez)

Replying to beany_kelly@…:

Replying to jeremyhu@…:

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

Hi Jeremy. I just did the "clean" step before attempting to reinstall clang-3.3, but with the same result.

That’s because you need to deactivate or uninstall those ports entirely.

BUT ...

Your mention of dyld-headers reminded me of an issue I've had for months with MacPorts. Any new installation or upgrade requires that I execute

sudo port [install|upgrade|blah] ...

The first thing I see after this is the message:

dyld: DYLD_ environment variables being ignored because main executable (/usr/bin/sudo) is setuid or setgid

Could this be related? It's cropped up in OSX-related boards, and appears within some Macports ticket discussions, but doesn't seem to have its own dedicated ticket ...

It doesn’t have a ticket because it’s not a problem. It’s a security measure hard-coded into OS X’s dynamic linker, and it has no effect on searching for headers. If you want the warning to go away, remove the DYLD_ variables from your environment; otherwise just deal with the message.

comment:22 in reply to:  21 ; Changed 7 years ago by beany_kelly@…

Replying to larryv@…:

It doesn’t have a ticket because it’s not a problem. It’s a security measure hard-coded into OS X’s dynamic linker, and it has no effect on searching for headers. If you want the warning to go away, remove the DYLD_ variables from your environment; otherwise just deal with the message.

In fact, I didn't *have* any DYLD_ environment variables set. I *did* have LD_LIBRARY_PATH set, and the warning went away when I unset that. So LD_LIBRARY_PATH was being interpreted as DYLD_LIBRARY_PATH (or DYLD_something_else) for some reason.

Anyway, I agree that it doesn't seem to be related to this ticket.

comment:23 in reply to:  22 Changed 7 years ago by larryv (Lawrence Velázquez)

Replying to beany_kelly@…:

In fact, I didn't *have* any DYLD_ environment variables set. I *did* have LD_LIBRARY_PATH set, and the warning went away when I unset that. So LD_LIBRARY_PATH was being interpreted as DYLD_LIBRARY_PATH (or DYLD_something_else) for some reason.

My fault, I always forget about that. The linker ignores LD_LIBRARY_PATH too; it just doesn’t bother to warn you about it.

Anyway, I agree that it doesn't seem to be related to this ticket.

Any results after deactivating the header ports?

comment:24 in reply to:  19 Changed 7 years ago by mymacports@…

I have uninstalled dyld-headers and xnu-headers.
After that clang-3.3 was compiled without errors.
Thanks!


Replying to jeremyhu@…:

Replying to mymacports@…:

  1. "... and all those other ports that conflict with system headers"
    There are no other problems except the one with clang-3.3

As one of the engineers who maintains those projects for OS X, and a MacPorts maintainer who has seen enough tickets from their fallout, I assure you there are.

  1. I do not have libc-headers and/or CarbonHeaders installed.

Ok, in your case, it's the dyld-headers and xnu-headers. Please nuke those.

comment:25 Changed 7 years ago by beany_kelly@…

Mine compiled fine also! Tried to post that a few hours ago, but the system wasn't responding.

Presumably those header ports *were* needed once upon a time, or they wouldn't have appeared in our installed ports list.

Perhaps running

sudo port uninstall leaves

would have avoided this? I'm running it now, but didn't think of checking for this functionality before I began composing this update.

comment:26 in reply to:  25 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Replying to beany_kelly@…:

Mine compiled fine also! Tried to post that a few hours ago, but the system wasn't responding.

Presumably those header ports *were* needed once upon a time, or they wouldn't have appeared in our installed ports list.

I think I recall seeing some port pulling them in when I did an upgrade months ago. When I noticed that, I fixed that port to not depend on them. It's likely that people's systems got polluted with those ports...

comment:27 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Port: xnu-headers libc-headers dyld-headers CarbonHeaders added
Summary: clang-3.3: upgrade failure on 10.6 - Availability.h not foundRemove ports: dyld-headers, xnu-headers, libc-headers, CarbonHeaders, etc

comment:28 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Port: clang-3.3 removed

comment:29 Changed 7 years ago by neverpanic (Clemens Lang)

$ port echo depends:dyld-headers or depends:xnu-headers or depends:libc-headers or depends:CarbonHeaders
ld64

comment:30 Changed 7 years ago by ryandesign (Ryan Schmidt)

Cc: ryandesign@… added

The newly-submitted lsyncd port (#42129) needs xnu-headers. Can we fix xnu-headers to install the correct headers for each OS X version?

comment:31 Changed 7 years ago by mfeiri

Status: newassigned

Ah, this is something I want to fix for quite a while. I'll update the ports next weekend. To avoid compatibility problems related to the *-headers ports, I plan to move these files from ${prefix}/include/ to ${prefix}/SDKs/Darwin${os.major}.sdk/. This way ports can opt-in to use these files through the -I or -isysroot flag.

One open question is to consider forwardporting old platform support (e.g. powerpc) to the current SDK. This would avoid the need to maintain versioned content in these ports. But it might require more effort to validate compatibility than continuing to maintain the versioned ports (I guess upstream dropped maintenance for a reason).

comment:32 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

The newly-submitted lsyncd port (#42129) needs xnu-headers.

*why*?

Can we fix xnu-headers to install the correct headers for each OS X version?

No, you should remove them. They don't belong in dports.

comment:33 Changed 7 years ago by jeremyhu (Jeremy Huddleston Sequoia)

Summary: Remove ports: dyld-headers, xnu-headers, libc-headers, CarbonHeaders, etcRemove ports: xnu-headers, libc-headers, CarbonHeaders, etc

comment:34 Changed 7 years ago by mfeiri

Summary: Remove ports: xnu-headers, libc-headers, CarbonHeaders, etcMove ports: xnu-headers, libc-headers, CarbonHeaders, etc

I’ve updated xnu-headers and libc-headers to the latest versions, but I haven’t yet finished the move from ${prefix} to ${prefix}/Developer/SDKs/Darwin${os.major}. I also want to do some more testing, e.g. to find out if the other *-headers ports can/need to be moved as well and if the isysroot flag works as expected. ETA is next weekend.

Moving these ports to an "SDKs" subdirectory will make them entirely opt-in. This way we will never see any involuntary interferences again. And using these headers through isysroot implies using the same precedence rules that some ports (such as clang apparently) seem to rely on for system headers. This would probably fix the problem described in this bugreport. Meanwhile the use of isystem in the upcoming macports 2.3 (#40656) might also solve the issue at a lower level. I plan to investigate that as well.

comment:35 Changed 7 years ago by mfeiri

Resolution: fixed
Status: assignedclosed

Resolved in r117043 for xnu-headers, libc-headers, and libm-headers. The other system header ports cctools-headers, dyld-headers, and libunwind-header should follow soon, but I've opened a seperate issue #42500 for that. The CarbonHeaders port will be merged into xnu-headers.

It turned out that the isystem flag wouldn't change anything for this particular issue. So following upstream to a "relocatable SDKs" model for opt-in usage seems to be the best solution.

comment:36 Changed 7 years ago by cooljeanius (Eric Gallager)

Cc: egall@… added

Cc Me!

comment:37 in reply to:  32 ; Changed 7 years ago by ryandesign (Ryan Schmidt)

Replying to jeremyhu@…:

The newly-submitted lsyncd port (#42129) needs xnu-headers.

*why*?

The developers of lsyncd have determined that the only interface capable of meeting their needs is /dev/fsevents but Apple considers it internal, so they need the internal fsevents.h header from xnu-headers. See https://github.com/axkibe/lsyncd/issues/257

comment:38 in reply to:  37 Changed 7 years ago by seanfarley (Sean Farley)

Replying to ryandesign@…:

Replying to jeremyhu@…:

The newly-submitted lsyncd port (#42129) needs xnu-headers.

*why*?

The developers of lsyncd have determined that the only interface capable of meeting their needs is /dev/fsevents but Apple considers it internal, so they need the internal fsevents.h header from xnu-headers. See https://github.com/axkibe/lsyncd/issues/257

Their response is really just a copt out. I posted a reply to their reasoning pointing out why.

Note: See TracTickets for help on using tickets.