defect: policykit 0.112_1 cannot be built on PPC Tiger and Leopard because it needs Clang 3.4 which cannot be built

[ballapete (Peter "Pete" Dyballa)]

See #44320. This must be a defect inside the Portfile, because

port info policykit

mentions

Build Dependencies:   pkgconfig, intltool, gtk-doc, autoconf, automake, libtool
Library Dependencies: glib2, gobject-introspection, mozjs17

and no Clang – although I have no idea what mozjs17 might me. Part of *some* Firefox?

I have installed on PPC Mac OS X 10.5.8, Leopard:

port installed | egrep 'pkgconfig|intltool|gtk-doc|autoconf|automake|libtool|glib2|gobject-introspection|mozjs17|gcc|clang|llvm'
  autoconf @2.69_2 (active)
  autoconf213 @2.13_1 (active)
  automake @1.14.1_2 (active)
  cctools @806_3+llvm33 (active)
  clang-3.3 @3.3_5+analyzer+python27 (active)
  clang_select @1.0_0 (active)
  gcc48 @4.8.3_0
  gcc48 @4.8.3_1 (active)
  gcc49 @4.9.1_0 (active)
  gcc_select @0.1_8 (active)
  gccmakedep @1.0.3_0 (active)
  glib2 @2.42.0_1 (active)
  gobject-introspection @1.42.0_0 (active)
  gtk-doc @1.21_1 (active)
  intltool @0.50.2_3+perl5_16 (active)
  ld64 @97.17_3+llvm31 (active)
  libgcc @4.9.1_0
  libgcc @4.9.1_1 (active)
  libtool @2.4.2_5 (active)
  llvm-3.1 @3.1_8 (active)
  llvm-3.3 @3.3_4 (active)
  llvm-3.4 @3.4.2_0 (active)
  llvm_select @1.0_0 (active)
  pkgconfig @0.28_0 (active)

[larryv (Lawrence Velázquez)]

What do you mean “it needs Clang”? What exactly happens when you try to install it?

[udbraumann]

PPC support of clang is less than weak. Either you can build it (e.g. versions 3.1 and 3.3) and then clang compiles more or less garbage, or you cannot build clang itself (like 3.4; I think the experimental 3.5 or 3.6 are not worthwhile to try), see a small portion of this sad story here: #43484 where two days ago the maintainer announced that the compiling issues I was reporting months ago concerning cctools are now considered as "wontfix". I actually have forgotten how clang versions 2.9, 3.0 and 3.3 were behaving, but I do not think these were much different from those mentioned above.

So for PPC and clang we are lost at present.

Practically spoken, any macports utility which should have a chance to be built on PPC cannot rely on clang. Instead, good old gcc-4.2 or macports-gcc-4.8 should be accepted (have not tried macports-gcc-4.9 so far).

For mozjs17 I have tried both of them but always get this "architecture was not detected as supported" error (also see my build log):

...
:info:build /Volumes/Data/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/mfbt/double-conversion/utils.h:71:2: error: #error Target architecture was not detected as supported by Double-Conversion.
...

Looking into utils.h one can see that the architecture check obviously has failed, i.e. __powerpc__ has not been defined. So my first guess is that this symbol needs to be defined before in order to let gcc-4.2 do the compilation of mozjs17. Hope this can be achieved via an official portfile modification, since I am sure it is not difficult to work around this issue.

[udbraumann]

sudo port install mozjs17 configure.compiler=gcc-4.2

[udbraumann]

Cc Me!

[ballapete (Peter "Pete" Dyballa)]

Replying to braumann@…:

For mozjs17 I have tried both of them but always get this "architecture was not detected as supported" error (also see my build log):

...
:info:build /Volumes/Data/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/mfbt/double-conversion/utils.h:71:2: error: #error Target architecture was not detected as supported by Double-Conversion.
...

Looking into utils.h one can see that the architecture check obviously has failed, i.e. __powerpc__ has not been defined. So my first guess is that this symbols needs to be defined before in order to let gcc-4.2 do the compilation of mozjs17. Hope this can be achieved via an official portfile modification, since I am sure it is not difficult to work around this issue.

Could be utils.h is a bit incorrect…

Apple's GCC 4.2 defines

#define __ppc__ 1
#define __POWERPC__ 1

The file could be patched. I did try that last night but change Portfile a bit:

 37 autoconf.cmd        autoconf213
 38 #compiler.blacklist  *gcc-*
 39 configure.args      --with-system-nspr \
 40                     --enable-system-ffi \
 41                     --disable-readline \
 42                     --enable-threadsafe \
 43                     --enable-macos-target=10.5 \
 44                     --with-arch=G4
 45 
 46 configure.ccache    no

but had no success. The configure option --with-arch=… leads to setting a compiler switch -march=… which GCC 4.2 does not accept. Changing configure to output -mcpu=… was successful. I finally left port out but used its environment to run this command from /opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src:

env PATH=/opt/local/bin:$PATH CC='/usr/bin/gcc-4.2' CC_PRINT_OPTIONS='YES' CC_PRINT_OPTIONS_FILE='/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_lang_mozjs17/mozjs17/work/.CC_PRINT_OPTIONS' CFLAGS='-pipe -Os -arch ppc' CPATH='/opt/local/include' CPPFLAGS='-I/opt/local/include -D__powerpc__' CXX='/usr/bin/g++-4.2' CXXFLAGS='-pipe -Os -arch ppc' F77FLAGS='-m32' F90FLAGS='-pipe -Os -m32' FCFLAGS='-pipe -Os -m32' FFLAGS='-pipe -Os' INSTALL='/usr/bin/install -c' LDFLAGS='-L/opt/local/lib -Wl,-headerpad_max_install_names -arch ppc' LIBRARY_PATH='/opt/local/lib' MACOSX_DEPLOYMENT_TARGET='10.5' OBJC='/usr/bin/gcc-4.2' OBJCFLAGS='-pipe -Os -arch ppc' OBJCXX='/usr/bin/g++-4.2' OBJCXXFLAGS='-pipe -Os -arch ppc' ./configure --prefix=/opt/local --with-system-nspr --enable-system-ffi --disable-readline --enable-threadsafe --enable-macos-target=10.5 --with-arch=G4

Notice the CPPFLAGS='-I/opt/local/include -D__powerpc__'! This lead a successful compilation of mozjs17, I also could invoke make check. I could not wait until this step finished because the bed was calling for me…

So I think that it is possible to compile mozjs17 for PPC Mac OS X 10.5.8/Leopard.

P.S.: With GCC it's possible to get the #defines with cpp -dD < /dev/null for example. This does not seem to work with Clang, it seems to need an empty C file, but then it emits:

#define __ppc__ 1
#define _ARCH_PPC 1
#define __powerpc__ 1
#define __POWERPC__ 1
#define __ppc__ 1

llvm-gcc gives

#define _ARCH_PPC 1
#define _ARCH_PPCGR 1
#define __ppc__ 1
#define __POWERPC__ 1

so it's as unusable as GCC. Tonight I'm going to test with port's Clang mp-clang-3.3.

[ballapete (Peter "Pete" Dyballa)]

Replying to Peter_Dyballa@…:

Tonight I'm going to test with port's Clang mp-clang-3.3.

The sources don't seem to made for Clang. Or Clang not ripe for these sources…

[udbraumann]

Replying to Peter_Dyballa@…:

The sources don't seem to made for Clang. Or Clang not ripe for these sources…

Try this one-liner

printf '#include<stdio.h> \n int main(void) {fprintf(stderr,"Hello, world\\n");}' | clang-mp-3.1 -arch ppc -xc -o out.exe - && ./out.exe

and vary your versions of clang. For me it worked for clang-mp-2.9 to clang-mp-3.1. Apparently the -arch ppc can be used even on 10.6.8 Intel Macs (have tried for x86_64). As far as I can remember clang-mp-3.2 to clang-mp-3.4 were unable to build even this very basic example. Moreover, clang-mp-3.4 cannot be built on PPC. And if you try clang-mp-3.5 on Intel Mac (with 10.6.8), you get the particular message ld: unknown/unsupported architecture name for: -arch powerpc, so even the formal support which was given so far has been dropped recently.

So how to proceed further for our search for a solution?

[ballapete (Peter "Pete" Dyballa)]

Replying to braumann@…:

So how to proceed further for our search for a solution?

I'd patch utils.h to contain __POWERPC__ instead of __powerpc__ and use Apple's GCC 4.2 for compilation.

[ballapete (Peter "Pete" Dyballa)]

Patch to fix ../../mfbt/double-conversion/utils.h

[ballapete (Peter "Pete" Dyballa)]

I made my first test on PPC Tiger, Mac OS X 10.4.11. It failed here:

/opt/local/bin/g++-apple-4.2 -o jsgc.o -c  -I./dist/system_wrappers_js -include ./config/gcc_hidden.h -DIMPL_MFBT -DEXPORT_JS_API -DNO_NSPR_10_SUPPORT -DUSE_ZLIB -I./../../mfbt/double-conversion -I. -I. -I./dist/include  -I/opt/local/include/nspr      -I. -I./assembler -I./yarr  -fPIC -I/opt/local/include -Wall -Wpointer-arith -Woverloaded-virtual -Werror=return-type -Wempty-body -Wno-ctor-dtor-privacy -Wno-overlength-strings -Wno-invalid-offsetof -Wno-variadic-macros -Wcast-align -pipe -Os -arch ppc -fno-common -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -pthread -pipe  -DNDEBUG -DTRIMMED -g -O3 -fno-stack-protector -fomit-frame-pointer -DUSE_SYSTEM_MALLOC=1 -DENABLE_ASSEMBLER=1  -I/opt/local/include -DMOZILLA_CLIENT -include ./js-confdefs.h -MD -MF .deps/jsgc.o.pp /opt/local/var/macports/build/_opt_mports_trunk_dports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src/jsgc.cpp
/opt/local/var/macports/build/_opt_mports_trunk_dports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src/jsgc.cpp: In function 'unsigned int js::GetCPUCount()':
/opt/local/var/macports/build/_opt_mports_trunk_dports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src/jsgc.cpp:2868: error: '_SC_NPROCESSORS_ONLN' was not declared in this scope
config/rules.mk:1017: recipe for target 'jsgc.o' failed
gmake[1]: *** [jsgc.o] Error 1
gmake[1]: Leaving directory '/opt/local/var/macports/build/_opt_mports_trunk_dports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src'
config/rules.mk:576: recipe for target 'all' failed
gmake: *** [all] Error 2
gmake: Leaving directory '/opt/local/var/macports/build/_opt_mports_trunk_dports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src'
Command failed:  cd "/opt/local/var/macports/build/_opt_mports_trunk_dports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src" && gmake -w all 
Exit code: 2
Error: org.macports.build for port mozjs17 returned: command execution failed

I made three changes to the Portfile:

1. added the patch file
2. commented the blackmail line
3. added build.cmd gmake because Apple's /usr/bin/make instantly quit telling that virtual space was exhausted.

One of the next few days, could be even tomorrow, I'll try the same on Leopard.

[ballapete (Peter "Pete" Dyballa)]

Failed build with patch applied

[udbraumann]

I did these experimental steps and had success building both mozjs17 @17.0.0.2 and policykit @0.104_1 on 10.5.8. PPC:

$ sudo port configure mozjs17 configure.compiler=gcc-4.2

$ sudo nedit /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/mfbt/double-conversion/utils.h

Added the pattern as variant in capitals inside utils.h:

defined(__powerpc__) || defined(__POWERPC__)

Continued building:

$ sudo port install mozjs17 configure.compiler=gcc-4.2

--->  Computing dependencies for mozjs17
--->  Building mozjs17
--->  Staging mozjs17 into destroot
--->  Installing mozjs17 @17.0.0_2
--->  Activating mozjs17 @17.0.0_2
--->  Cleaning mozjs17
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  No broken files found.

Then have built policykit:

$ sudo port upgrade policykit

--->  Computing dependencies for policykit
--->  Fetching archive for policykit
--->  Attempting to fetch policykit-0.112_1.darwin_9.ppc.tbz2 from http://nue.de.packages.macports.org/macports/package/policykit
--->  Attempting to fetch policykit-0.112_1.darwin_9.ppc.tbz2 from http://lil.fr.packages.macports.org/policykit
--->  Attempting to fetch policykit-0.112_1.darwin_9.ppc.tbz2 from http://mse.uk.packages.macports.org/site/packages.macports.org/policykit
--->  Fetching distfiles for policykit
--->  Attempting to fetch polkit-0.112.tar.gz from http://nue.de.distfiles.macports.org/macports/distfiles/policykit
--->  Verifying checksums for policykit
--->  Extracting policykit
--->  Applying patches to policykit
--->  Configuring policykit
--->  Building policykit
--->  Staging policykit into destroot
--->  Installing policykit @0.112_1
--->  Cleaning policykit
--->  Computing dependencies for policykit
--->  Deactivating policykit @0.104_1
--->  Cleaning policykit
--->  Activating policykit @0.112_1
--->  Cleaning policykit
--->  Updating database of binaries
--->  Scanning binaries for linking errors
--->  No broken files found.

[ballapete (Peter "Pete" Dyballa)]

Worked for me on Leopard as well. I had to manually edit utils.h because port had this splendid idea:

DEBUG: Assembled command: 'cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src" && /usr/bin/patch -p0'
DEBUG: Executing command line:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src" && /usr/bin/patch -p0 < '/opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/lang/mozjs17/files/fix-utils.h.diff'
/usr/bin/patch: **** rejecting file name with ".." component: ../../mfbt/double-conversion/utils.h
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_nue.de.rsync.macports.org_macports_release_tarballs_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/js/src" && /usr/bin/patch -p0 < '/opt/local/var/macports/sources/nue.de.rsync.macports.org/macports/release/tarballs/ports/lang/mozjs17/files/fix-utils.h.diff'
Exit code: 2
Error: org.macports.patch for port mozjs17 returned: command execution failed

I also reported the bugs to bugzilla.

[udbraumann]

Did you test the abovementioned experimental steps also on Tiger? I btw. have doubts that gmake is required instead of make.

I think, we now should ask the maintainer(s) to do the following:

1. Patch the header file utils.h under /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_mozjs17/mozjs17/work/mozjs17.0.0/mfbt/double-conversion according to attachment:fix-utils.h.diff​

2. Clear the blacklisting of *gcc-* in browser:trunk/dports/lang/mozjs17/Portfile?rev=120613​ and instead add some configure.compiler apple-gcc-4.2 or configure.compiler gcc-4.2, as tests (at least on PPC 10.5.8) have demonstrated that in fact gcc-4.2 is able to build mozjs17 @17.0.0_2. However, if there would be some substantial reason to use macports-clang-3.4 here for non-PPC systems, than please implement a switching mechanism for compiler section here.

Thanks!

[ballapete (Peter "Pete" Dyballa)]

Replying to braumann@…:

Did you test the abovementioned experimental steps also on Tiger?

No. On Tiger, where some GNU Emacs builds took many hours' time, I created the patch – and port had no problem to apply it with those .. in the path! But the usual /usr/bin/make (3.80) failed at start, upon invocation, with that virtual memory exhausted report. Look at this: ​http://stackoverflow.com/questions/20286653/make-error-virtual-memory-exhausted-stop! And ​http://stackoverflow.com/questions/10903367/virtual-memory-exhausted-error. Gmake (4.1) managed to start, and failed later. Anyway, I can retry this in a few days (right now my PowerBook G4 is running Leopard). And also can try your steps.

I think, we now should ask the maintainer(s) to do the following:

D'accord! Except – I think clang-3.4 did not build for me … In any case it does not build on Tiger. Apple's GCC 4.2 should be used as compiler. It is prepared to build any software On Tiger and Leopard.

[dgonyier (Dwaine Gonyier)]

Cc Me!

[ballapete (Peter "Pete" Dyballa)]

My rather private fix-jsgc.cpp.diff for PPC Tiger 10.4.11

[ballapete (Peter "Pete" Dyballa)]

With this patch file I could cure the missing definition of _SC_NPROCESSORS_ONLN. This symbol is not configured in my kernel, the command line utility getconf also fails.

[ballapete (Peter "Pete" Dyballa)]

After mozjs17 was successfully built. policykit failed here:

Making all in polkit
make[3]: Entering directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_security_policykit/policykit/work/polkit-0.112/src/polkit'
/opt/local/share/gobject-introspection-1.0/Makefile.introspection:152: *** Need to define Polkit_1_0_gir_LIBS or Polkit_1_0_gir_PROGRAM.  Stop.
make[3]: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_security_policykit/policykit/work/polkit-0.112/src/polkit'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_security_policykit/policykit/work/polkit-0.112/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_security_policykit/policykit/work/polkit-0.112'
make: *** [all] Error 2
make: Leaving directory `/opt/local/var/macports/build/_opt_mports_trunk_dports_security_policykit/policykit/work/polkit-0.112'
Command failed:  cd "/opt/local/var/macports/build/_opt_mports_trunk_dports_security_policykit/policykit/work/polkit-0.112" && /usr/bin/make -w all CC="/opt/local/bin/gcc-apple-4.2 -arch ppc" 
Exit code: 2

[ballapete (Peter "Pete" Dyballa)]

Real policykit 0.122 main.log

[ballapete (Peter "Pete" Dyballa)]

By using

     62                 --enable-introspection=no \

in Portfile policykit finally built on Tiger PPC Mac OS X 10.4.11.

[csanchezdll]

I am using 10.4.11 on a PPC (iBook G4) and --enable-instrospection=no does not seem to be required.

System make is version 3.80 and indeed fails with some weird virtual space exhausted error, so this has to use gmake (at least in darwin 8). Attached patch changes make just for Tiger.

BTW, I cleaned up mozjs17 changes discussed on this ticket and found some extra information on which compilers should be blacklisted. See #46567.

[csanchezdll]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

Fixed mozjs17 build on Leopard in r135937. Tiger fixes forthcoming.

There should no longer be any need to modify the policykit portfile, neither to disable introspection nor to manually force the use of gmake, because the gobject_introspection portgroup which the policykit port now uses already does that.

[ryandesign (Ryan Carsten Schmidt)]

Replying to Peter_Dyballa@…:

I also reported the bugs to bugzilla.

Could you give us the URLs of those reports, please? I could not, for example, find a bug report in ​Mozilla's Bugzilla about the use of _SC_NPROCESSORS_ONLN. A quick Google search did reveal ​this code snippet purporting to show a Mac-compatible equivalent.

[ballapete (Peter "Pete" Dyballa)]

Replying to ryandesign@…:

Replying to Peter_Dyballa@…:

I also reported the bugs to bugzilla.

Could you give us the URLs of those reports, please?

That's it: ​https://bugzilla.mozilla.org/show_bug.cgi?id=1106323

[ryandesign (Ryan Carsten Schmidt)]

They closed it as wontfix and referred you to a newer version of mozjs, in which I presume the issue still exists.

[ryandesign (Ryan Carsten Schmidt)]

Using newer gmake on Tiger as of r136465. That leaves the _SC_NPROCESSORS_ONLN problem.

[ballapete (Peter "Pete" Dyballa)]

Replying to ryandesign@…:

I could not, for example, find a bug report in ​Mozilla's Bugzilla about the use of _SC_NPROCESSORS_ONLN. A quick Google search did reveal ​this code snippet purporting to show a Mac-compatible equivalent.

unistd.h does not contain a definition of _SC_NPROCESSORS_ONLN. Another option to provide the value would be, taken from the command sysctl -b hw.availcpu | awk '{print $NF}' to "patch" jsgc.cpp. In Shell it would be something like that:

 FROM='sysconf(_SC_NPROCESSORS_ONLN)'
   TO=`sysctl -b hw.availcpu | awk '{print $NF}'`

sed -ie s/$FROM/${TO}L/ mozjs17.0.0/js/src/jsgc.cpp

Maybe hw.logicalcpu or hw.logicalcpu_max or simple hw.ncpu are better names to determine the number of (available) CPU cores.

Anyway, I think I'm going to try the code snippet. (Because I also found a cure for the not building policykit 0.113_1 – not yet reported.)

[ballapete (Peter "Pete" Dyballa)]

Replying to ryandesign@…:

A quick Google search did reveal ​this code snippet purporting to show a Mac-compatible equivalent.

I managed to patch jsgc.cpp – and it compiled!

I am attaching the patch files for Portfile and js/src/jsgc.cpp.

[ballapete (Peter "Pete" Dyballa)]

Patch for js/src/jsgc.cpp

[ballapete (Peter "Pete" Dyballa)]

Patch Portfile to make port patch js/src/jsgc.cpp as well

[dbevans (David B. Evans)]

Closing. Reporter states that the issue is now fixed (#49235).

[ballapete (Peter "Pete" Dyballa)]

On PPC Tiger, Mac OS X 10.4.11, I am missing for mozjs17 17.0.0_4 patch-js-src-jsgc.cpp.diff​ and Portfile.2.diff. So this port does not build … With the changes it builds.

[ballapete (Peter "Pete" Dyballa)]

Since policykit 0.113 is out and builds on PPC Tiger it can be closed, I think.

[dbevans (David B. Evans)]

Done.