poxml @4.11.2 (kde, kde4) build failure on OSX 10.9 on x86-64, also blocks kdesdk4 build

[FoolishEwe@…]

After upgrading to mavericks, I cannot upgrade/install kdesdk4, due in part to the dependency, poxml, which fails to build. After uninstalling and attempting a fresh install of poxml I see:

sudo port info poxml
poxml @4.11.2 (kde, kde4)
Variants:             debug, docs, universal

Description:          KDE developer utility
Homepage:             http://www.kde.org/

Extract Dependencies: xz
Build Dependencies:   cmake, pkgconfig, automoc
Library Dependencies: qt4-mac, phonon, kdelibs4, antlr
Platforms:            darwin
License:              GPL-2+
Maintainers:          nicos@macports.org, openmaintainer@macports.org

sudo port install poxml
--->  Computing dependencies for poxml
--->  Building poxml
Error: org.macports.build for port poxml returned: command execution failed
Please see the log file for port poxml for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/main.log
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port poxml failed

I had successfully installed this on my Macbook Pro before the upgrading the OS to Mavericks. Please find the indicated main.log attached.

[FoolishEwe@…]

Forgot to show that kdesdk4 won't build

 sudo port install kdesdk4
--->  Computing dependencies for kdesdk4
--->  Dependencies to be installed: poxml umbrello
--->  Building poxml
Error: org.macports.build for port poxml returned: command execution failed
Error: Failed to install poxml
Please see the log file for port poxml for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/main.log
Error: The following dependencies were not installed: poxml umbrello
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port kdesdk4 failed

[ryandesign (Ryan Carsten Schmidt)]

This was not a clean build. Please ​clean and try again.

[FoolishEwe@…]

cleaned and retried main.log

[FoolishEwe@…]

I just posted the cleaned and retried main.log from this run, the upload renamed it to main.2.log. Please find the sequence of commands I ran (selfupdate, outdated, upgrade outdated, uninstall poxml, clean poxml and install poxml) and their console output. I think I did this before filing the bug, but I repeated and recorded this time (I did upgrade recently before doing this run, hence no packages were rendered outdated).

$ sudo port selfupdate
--->  Updating MacPorts base sources using rsync
MacPorts base version 2.2.1 installed,
MacPorts base version 2.2.1 downloaded.
--->  Updating the ports tree
--->  MacPorts base is already the latest version

The ports tree has been updated. To upgrade your installed ports, you should run
  port upgrade outdated
$ sudo port outdated
No installed ports are outdated.
$ sudo port uninstall poxml
$ sudo port clean poxml
--->  Cleaning poxml
$ sudo port install poxml
--->  Computing dependencies for poxml
--->  Fetching archive for poxml
--->  Attempting to fetch poxml-4.11.2_0.darwin_13.x86_64.tbz2 from http://packages.macports.org/poxml
--->  Attempting to fetch poxml-4.11.2_0.darwin_13.x86_64.tbz2 from http://lil.fr.packages.macports.org/poxml
--->  Attempting to fetch poxml-4.11.2_0.darwin_13.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/poxml
--->  Fetching distfiles for poxml
--->  Verifying checksums for poxml
--->  Extracting poxml
--->  Applying patches to poxml
--->  Configuring poxml
--->  Building poxml
Error: org.macports.build for port poxml returned: command execution failed
Please see the log file for port poxml for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/main.log
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port poxml failed

[ryandesign (Ryan Carsten Schmidt)]

Ok, thanks; the log says:

:info:build [ 90%] Generating lauri.pot
:info:build cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/work/build/tests && checkXML /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/work/poxml-4.11.2/tests/lauri.xml
:info:build make[2]: *** [tests/lauri.pot] Error 139

Code 139 means a crash, I think. Is there a crash log in ~/Library/Logs/DiagnosticReports/ created around the time the build finished? Possibly for this "checkXML" program? If so please attach it.

[FoolishEwe@…]

Recent version the main.log

[FoolishEwe@…]

Sanitized Log, includes port -d install poxml

[FoolishEwe@…]

I repeated the attempt but did not notice a crash log, is there another place I should be looking? In any case I attached the main.log (named main.3.log) of this attempt and the output of my attempt to do this using the command:

date; sudo port clean poxml; date ; sudo port -d install poxml; date; ls -lart  ~/Library/Logs/DiagnosticReports/ ; date

The output was sanitized (i.e. I did a global search and replace on my true user name to FoolishEwe) however, the date stamps on the files observed in the DiagnosticReports directory do not appear to include a sufficiently recent crash log.

[ryandesign (Ryan Carsten Schmidt)]

What about in the system-wide log directory /Library/Logs/DiagnosticReports/?

[FoolishEwe@…]

I tried:

date; sudo port clean poxml; date ; sudo port -d install poxml; date; ls -lart  ~/Library/Logs/DiagnosticReports/ /Library/Logs/DiagnosticReports/ ; date

Again I sanitized it and will add after I post. Your intuition seems good, there was a file in /Library/Logs/DiagnosticReport/, I'll attach the corresponding file as well. I'm no expert but it looked to me like minproc4 may have crashed. I will add the crash file and what appears to be an associated plist as well.

[FoolishEwe@…]

minproc4 crash file

[FoolishEwe@…]

I had to tar the plist file becuase the leading . foiled the file picker tool

[ryandesign (Ryan Carsten Schmidt)]

meinproc4 crashes were supposed to have been resolved 2 weeks ago by making +docs no longer a default variant (see #30162) but your output does not show you using the +docs variant so perhaps there is another cause we need to investigate. Any ideas, Nicolas?

[FoolishEwe@…]

The script associated with the most recent crash, shows the unix commands run

[NicosPavlov]

I did not meddle up to now since I did not have anything useful to add, and I could not reproduce the issue. It should however not be directly linked to the issue of 2 weeks ago, because the default variant was only for kdelibs4, in order to have the basic files needed by other ports for the documentation. All the other ports were still up to the user.

Thanks to the analysis above, it became easier to understand the issue. As the poxml port is a utility to generate XML files, it uses checkXML to validate the generated XML files at the test phase, independently of the build of documentation, and checkXML seems to use meinproc4 (which I was not aware of).

An easy way would be to disable tests completely, but I will try to look if it is possible to keep some of them. I should be able to prepare the patches once I get access to my computer later in the day.

[NicosPavlov]

Here are the patches which should disable the use of checkXML during the test phase, while retaining the other tests, directly related to poxml. The patch to the Portfile has been made from version 4.11.4, which is the present one, and not 4.11.2, from which this ticket originated (but the issue is the same).

If these do not work, it would also be possible to just remove the tests from the build process, but this should be ok.

[NicosPavlov]

Patch to remove checkXML from tests

[NicosPavlov]

patch to Portfile to add new patch to code

[FoolishEwe@…]

Replying to nicos@…:

Here are the patches which should disable the use of checkXML during the test phase, while retaining the other tests, directly related to poxml. The patch to the Portfile has been made from version 4.11.4, which is the present one, and not 4.11.2, from which this ticket originated (but the issue is the same).

If these do not work, it would also be possible to just remove the tests from the build process, but this should be ok.

Thanks for looking into this.

Regarding these patches, how can we validate them? Should I be applying them on my own ports tree (if so, how, and if something goes wrong, how do I check point and roll back?), or is there another mechanism I should use (e.g. port selfupdate ...)?

[ryandesign (Ryan Carsten Schmidt)]

Why is meinproc4 crashing in the first place? Has an upstream issue been filed about that?

[NicosPavlov]

Replying to FoolishEwe@…:

Regarding these patches, how can we validate them? Should I be applying them on my own ports tree (if so, how, and if something goes wrong, how do I check point and roll back?), or is there another mechanism I should use (e.g. port selfupdate ...)?

A way or another, you will have to test them on your port tree, I think. You can either ​create a local port tree, but this seems overkill if you intend to use only once. Otherwise, you can replace the files in the port tree to test them. Anyway, running "selfupdate" will purge the directory and resync all files with the ones of the servers, and thus rolling back your changes if you need to.

[NicosPavlov]

Replying to ryandesign@…:

Why is meinproc4 crashing in the first place? Has an upstream issue been filed about that?

A bug has been reported for quite some time now (​https://bugs.kde.org/show_bug.cgi?id=261509), and I pinged the report with our last issues, but not much response has been provided up to now.

[mkae (Marko Käning)]

Cc Me!

[ryandesign (Ryan Carsten Schmidt)]

source for a small C program to print the current locale and country

[ryandesign (Ryan Carsten Schmidt)]

As best I can tell from the crash log, the crash is occurring when KDE is trying to determine the user's country. It is expecting OS X to return a two-character string like "US" for the United States, but instead it is receiving an invalid string and crashing when trying to convert it to another format.

I have written a small program to print the current OS X locale value and the value of the country code extracted from that. Could anyone experiencing this meinproc4 crash please download testlocale.c​, open Terminal, cd to the directory where you downloaded it, compile it, and run it, and post the output?

cd ~/Downloads/
cc testlocale.c -framework CoreFoundation -o testlocale
./testlocale

For example on my system I get:

<CFLocale 0x7feab1d20950 [0x7fff78e00eb0]>{type = user, identifier = 'en_US'}
US

And I have not experienced a meinproc4 crash.

[FoolishEwe@…]

running testlocale yields for me:

machineName:testdir FoolishEwe$ ./testlocale
<CFLocale 0x7fa043600680 [0x7fff77ed1eb0]>{type = user, identifier = 'en_US'}
US

I have not tried the patches yet.

Replying to ryandesign@…:

As best I can tell from the crash log, the crash is occurring when KDE is trying to determine the user's country. It is expecting OS X to return a two-character string like "US" for the United States, but instead it is receiving an invalid string and crashing when trying to convert it to another format.

I have written a small program to print the current OS X locale value and the value of the country code extracted from that. Could anyone experiencing this meinproc4 crash please download testlocale.c​, open Terminal, cd to the directory where you downloaded it, compile it, and run it, and post the output?

cd ~/Downloads/
cc testlocale.c -framework CoreFoundation -o testlocale
./testlocale

For example on my system I get:

<CFLocale 0x7feab1d20950 [0x7fff78e00eb0]>{type = user, identifier = 'en_US'}
US

And I have not experienced a meinproc4 crash.

[ryandesign (Ryan Carsten Schmidt)]

Replying to FoolishEwe@…:

running testlocale yields for me:

machineName:testdir FoolishEwe$ ./testlocale
<CFLocale 0x7fa043600680 [0x7fff77ed1eb0]>{type = user, identifier = 'en_US'}
US

Hmm. The same values as on my system. So we're back to square one on understanding why meinproc4 crashes on some people's systems and not others.

[FoolishEwe@…]

Thanks for all your help. Regarding the patches that were kindly provided, could I please get some clarification on how to apply the patches supplied?

In particular I'm not sure what to do with the patch-noCheckXML.diff file. I tried the to follow the approach described at ​http://guide.macports.org/chunked/development.patches.html#development.patches.applying, however I did not have success. Informally I tried the following sequence of commands

Script started on Mon Dec 23 16:56:38 2013
sh-3.2$ sudo port selfupdate
--->  Updating MacPorts base sources using rsync
MacPorts base version 2.2.1 installed,
MacPorts base version 2.2.1 downloaded.
--->  Updating the ports tree
--->  MacPorts base is already the latest version

The ports tree has been updated. To upgrade your installed ports, you should run
  port upgrade outdated
sh-3.2$ port outdated
No installed ports are outdated.
sh-3.2$ cd $(port dir poxml)
sh-3.2$ sudo patch -p0 < ~/testdir/patch-Portfile-poxml.diff 
patching file Portfile
sh-3.2$ sudo cp ~/testdir/patch-noCheckXML.diff files/
sh-3.2$ sudo port -d install poxml
[Output Ommitted for brevity]
--->  Applying patch-noCheckXML.diff
DEBUG: Environment: CPATH='/opt/local/include' CC_PRINT_OPTIONS_FILE='/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/work/.CC_PRINT_OPTIONS' LIBRARY_PATH='/opt/local/lib' CC_PRINT_OPTIONS='YES' MACOSX_DEPLOYMENT_TARGET='10.9'
DEBUG: Assembled command: 'cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/work/poxml-4.11.4" && /usr/bin/patch -p0'
DEBUG: Executing command line:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/work/poxml-4.11.4" && /usr/bin/patch -p0 < '/opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/kde/poxml/files/patch-noCheckXML.diff'
sh: /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/kde/poxml/files/patch-noCheckXML.diff: Permission denied
Command failed:  cd "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/work/poxml-4.11.4" && /usr/bin/patch -p0 < '/opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/kde/poxml/files/patch-noCheckXML.diff'
Exit code: 1
Error: org.macports.patch for port poxml returned: command execution failed
DEBUG: Error code: CHILDSTATUS 34866 1
DEBUG: Backtrace: command execution failed    while executing
"system $fullcmdstring"
    ("eval" body line 1)
    invoked from within
"eval system $notty $nice \$fullcmdstring"
    invoked from within
"command_exec patch "" "< '$patch'""
    (procedure "portpatch::patch_main" line 35)
    invoked from within
"$procedure $targetname"
Warning: targets not executed for poxml: org.macports.activate org.macports.patch org.macports.configure org.macports.build org.macports.destroot org.macports.install
Please see the log file for port poxml for details:
    /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_kde_poxml/poxml/main.log
To report a bug, follow the instructions in the guide:
    http://guide.macports.org/#project.tickets
Error: Processing of port poxml failed

This raises some questions.

Did I follow the correct procedure?

If not, could I please get the exact and correct steps to follow?

I plan to attach the script file output (sanitized and with control characters removed when possible for readability) as well as the main.log.

[FoolishEwe@…]

First failed attempt to apply the supplied diff files for the poxml build.

[FoolishEwe@…]

main.log from failed application of diff files.

[ryandesign (Ryan Carsten Schmidt)]

Replying to FoolishEwe@…:

sh: /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/kde/poxml/files/patch-noCheckXML.diff: Permission denied

Looks like the permissions of this file are wrong. Try:

sudo chmod 644 /opt/local/var/macports/sources/rsync.macports.org/release/tarballs/ports/kde/poxml/files/patch-noCheckXML.diff

[FoolishEwe@…]

Thanks, that appears to have worked, it looks like I was able to get poxml to install, i.e.

sh-3.2$ sudo port install poxml
--->  Computing dependencies for poxml
--->  Fetching archive for poxml
--->  Attempting to fetch poxml-4.11.4_0.darwin_13.x86_64.tbz2 from http://packages.macports.org/poxml
--->  Attempting to fetch poxml-4.11.4_0.darwin_13.x86_64.tbz2 from http://lil.fr.packages.macports.org/poxml
--->  Attempting to fetch poxml-4.11.4_0.darwin_13.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/poxml
--->  Fetching distfiles for poxml
--->  Verifying checksums for poxml
--->  Extracting poxml
--->  Applying patches to poxml
--->  Configuring poxml
--->  Building poxml
--->  Staging poxml into destroot
--->  Installing poxml @4.11.4_0
--->  Activating poxml @4.11.4_0

Don't forget that dbus needs to be started as the local user (not with sudo) before any
KDE programs will launch.
To start it run the following command:
 launchctl load -w /Library/LaunchAgents/org.freedesktop.dbus-session.plist

--->  Cleaning poxml
--->  Updating database of binaries: 100.0%
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.

Although the command completes successfully, it is odd that we had this error.

Regarding the problem, the symptom and the patch (fix/workaround), is this the right place to discuss next steps?

If so,

1) Do the maintainers need more information from me (since I don't think they can personally observe this on their configurations)? 2) Is this patch a fix suitable for general consumption, or is it a workaround for a small group (perhaps only me) of users? 3) Will this patch (or some related fix) be propagated into the mainline (i.e. can I safely do a port selfupdate and port upgrade outdated)?

[NicosPavlov]

Thanks for testing the patch, I committed it in r115311, implying that you can run selfupdate and still have it.

I don't think there will be a need for more information. I already had to ask you several things since I could not reproduce the issue and was not sure that I had totally fixed the issue. The crash of meinproc4 is irritating and quite more general than just this particular issue, but it has been around for quite some time (bug report ​https://bugs.kde.org/show_bug.cgi?id=261509) and nothing has been done upstream yet. In this context, suppressing some tests in poxml is a fix which is as satisfying as it can be.

[FoolishEwe@…]

Just to let you know that I commented on the kde thread and offered to help track this down. I ran port selfupdate and then port uninstall poxml and port install poxml which worked, so it looks like the fix has propagated to the release and is working for me. If the kde maintainers want help tracking this down, I may request help reverting this patch so that I can locally recreate the build error (i.e. the meinproc4 crash).

[ryandesign (Ryan Carsten Schmidt)]

Replying to FoolishEwe@…:

If the kde maintainers want help tracking this down, I may request help reverting this patch so that I can locally recreate the build error (i.e. the meinproc4 crash).

wiki:howto/InstallingOlderPort