poppler -- new version -- more variants

[jens.traube@…]

I submitted a new port "TeXworks" (Ticket #16055). This port needs port "poppler" with Qt4 bindings, but the current port of poppler has disabled by default the build of the Qt4 bindings. Therefore I added a variant qt4.

I studied the port "poppler" in greater detail and realized that much more dependencies to other ports exists, than in the portfile are registered. With the command line tool "otool" I checked the dependencies and added the ports to depends_lib.

The portfile of "poppler" is written in such a way, as to provide you with a maximal installation. But the port "TeXworks" does neither need port "XFree86" nor "GTK2". Therefore I added more variants: qt3Splash, qt4Splash, glibSplash, glibCairo, and abiword. Each of these variants provides you with a minimal installation. Of course, these variants can be combined, but not with variant "qt4". Variant "qt4" is for maximal installation of poppler and therefore contradicts to the other variations.

[jens.traube@…]

poppler -- version 0.8.4

new variants: qt4, qt3Splash, qt4Splash, glibSplash, glibCairo, abiword

[rhwood@…]

CCing those CCed in #16076

[rhwood@…]

This seems like an excessively complex solution to a simple issue that may not be an issue...

Some notes:

Abiword going forward is going to be requiring gtk and all that entails. I would simply remove the abiword variant and make libxml2 a dependency in all cases (libxml2 is small with only a couple of dependencies).

Since poppler is not (as far as I can tell) dependent on X11 unless gtk or qt are, I would simply remove the dependencies on XFree86 and xrender from the port.

I would make the variants all separate, so that instead of installing poppler +qt4Splash, I would install poppler +qt4 +Splash this will make the portfile maintainer's life much simpler.

[jens.traube@…]

Replying to rhwood@macports.org:

This seems like an excessively complex solution to a simple issue that may not be an issue... Some notes: Abiword going forward is going to be requiring gtk and all that entails. I would simply remove the abiword variant and make libxml2 a dependency in all cases (libxml2 is small with only a couple of dependencies). Since poppler is not (as far as I can tell) dependent on X11 unless gtk or qt are, I would simply remove the dependencies on XFree86 and xrender from the port. I would make the variants all separate, so that instead of installing poppler +qt4Splash, I would install poppler +qt4 +Splash this will make the portfile maintainer's life much simpler.

You are right about variant "abiword". Anybody, who have port "abiword" installed (from source), must have installed port "ImageMagick" and its dependencies (XFree86, freetype, fontconfig, libiconv, expat, zlib, libpng, jpeg, tiff, and libxml2).

But about the dependencies on XFree86 you are wrong. I installed all variants (except qt3Splash) to test them and to check the dynamic libraries. I used the command line tool "otool" with option "-L" to get a list of the needed libraries. The variants "glibSplash" and "glibCairo" depend on ports "XFree86" and "xrender". The ports "gtk2", "pango", and "cairo" are all dependent on XFree86, but these three ports can be build without x11, there is a variant "no_x11". I checked this and compiled these ports without x11 bindings, and then compiled poppler again with variant "quartz". Then library "libpoppler-glib.3.0.0.dylib" is not dependend on XFree86 and xrender.

These dependencies are indirect. For instance, in the source files of poppler you can not find any inclusion of a header file <expat.h> or <iconv.h>. I checked this: grep -lrE ".*<expat.h>.*" ./ Nevertheless, poppler depends on port "expat" and "libiconv" (any variant).

To your last argument: This was my first attempt to enhance the portfile. I had at first these "atomic" variants "qt3", "qt4", "glib", "splash", "cairo". But these variants cannot combined freely, there is no installation "poppler+splash+cairo". If you choose a frontend qt4 or glib, you must also choose a backend "splash" or "cairo", they are logically dependend. Therefore I revised the portfile and invented these "molecule" variants.

[jens.traube@…]

port dependencies of poppler

[rhwood@…]

Variants can explicitly conflict and require other variants. See  http://guide.macports.org/#reference.variants

[jens.traube@…]

I am sorry, but I do not want to start this task again. I spent a lot of time to think about this port. And the result is a working port, I tested that. You do not like my solution, that's a pity. Feel free to take it or ignore it.

[jens.traube@…]

poppler update -- version 0.8.5

[raimue@…]

This patch is so long it is very hard to review.

	# Poppler with variants +glibSplash +glibCairo +abiword (+qt3Splash) 
	# is the same as Poppler without any variant. 

What is the purpose of these variants? Why not use default_variants instead?

Do these variants build multiple different .dylibs or do all backends end up in one .dylib? Maybe the port could be splitted into several subports.

[rhwood@…]

This port remains excessively over-engineered to be able to (what seems to me) avoid using some already developed best practices for handling variants, writes new functions to avoid using established functions, which makes it difficult to read. The commenting is good, but its not enough to save this port submission. I recommend rejecting it unless its rewritten, but since the ticket submitter is not interested in rewriting this port submission to follow recommended practices, I don' think that's going to happen.

Poppler has a qt4 variant as of r38921 and is upgraded to version 0.8.5 as of r38910

[jens.traube@…]

otool output listings

[jens.traube@…]

configure script -- help text

[jens.traube@…]

poppler -- header file inclusion

[jens.traube@…]

I am new to MacPorts and still do not have the practice with what you call "developed best practices for handling variants". Also, I am about to revise my port enhancements, but do not have time for it on every day. At last, you can see, that Rainer Mueller asked me yesterday some questions and today I prepared my answers.

[jens.traube@…]

(1) What is the purpose of these variants?

I want to provide for a minimal installation of poppler. The port "TeXworks" only needs the Qt4 bindings of poppler. But the current portfile of "poppler" (version 0.8.3) provides for a maximal installation. It leaves it to the configuration script, which of the available modules gets compiled. Only three conditions are set: (A) "Don't compile poppler Qt4 wrapper," (B) "Install unsupported xpdf headers," and (C) "Build with zlib." Condition (A) bothers me. About (B) I can't tell you anything. Condition (C) means simply: "use port zlib." That's it. Hence I started the work on the portfile of poppler to resolve (A), but got more involved in this task, than I wanted.

I installed poppler, based on the current portfile, without variant "quartz". Apart from several command line tools, it consists of two dynamic libraries:

libpoppler-glib.3.0.0.dylib
libpoppler.3.0.0.dylib

These libraries are linked to a lot of other libraries, but the current portfile specifies only the dependencies to port "cairo" and "gtk2", which is questionable to me, because poppler clearly depends on ports "freetype" and "fontconfig" (see attached file "poppler-includes.txt"). The command line tool "otool" displays the following dependencies (I removed the info about the compatibility version):

$ otool -L /opt/local/lib/libpoppler-glib.3.0.0.dylib
/opt/local/lib/libpoppler-glib.3.0.0.dylib:
        /opt/local/lib/libpoppler-glib.3.dylib
        /opt/local/lib/libpoppler.3.dylib
        /opt/local/lib/libjpeg.62.dylib
        /opt/local/lib/libxml2.2.dylib
        /opt/local/lib/libz.1.dylib
        /opt/local/lib/libgdk-x11-2.0.0.dylib
        /opt/local/lib/libgdk_pixbuf-2.0.0.dylib
        /opt/local/lib/libtiff.3.dylib
        /opt/local/lib/libpangocairo-1.0.0.dylib
        /opt/X11R6/lib/libSM.6.dylib
        /opt/X11R6/lib/libICE.6.dylib
        /opt/X11R6/lib/libX11.6.dylib
        /opt/local/lib/libpangoft2-1.0.0.dylib
        /opt/local/lib/libpango-1.0.0.dylib
        /usr/lib/libSystem.B.dylib
        /opt/local/lib/libgobject-2.0.0.dylib
        /opt/local/lib/libgmodule-2.0.0.dylib
        /opt/local/lib/libglib-2.0.0.dylib
        /opt/local/lib/libintl.8.dylib
        /opt/local/lib/libiconv.2.dylib
        /opt/local/lib/libcairo.2.dylib
        /opt/local/lib/libfreetype.6.dylib
        /opt/local/lib/libfontconfig.1.dylib
        /opt/local/lib/libexpat.1.dylib
        /opt/local/lib/libpng12.0.dylib
        /opt/local/lib/libXrender.1.dylib
        /opt/local/lib/libpixman-1.0.dylib
        /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
        /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
        /usr/lib/libstdc++.6.dylib
        /usr/lib/libgcc_s.1.dylib
$ 
$ 
$ otool -L /opt/local/lib/libpoppler.3.0.0.dylib
/opt/local/lib/libpoppler.3.0.0.dylib:
        /opt/local/lib/libpoppler.3.dylib
        /opt/local/lib/libjpeg.62.dylib
        /opt/local/lib/libxml2.2.dylib
        /opt/local/lib/libz.1.dylib
        /usr/lib/libSystem.B.dylib
        /opt/local/lib/libiconv.2.dylib
        /opt/local/lib/libfreetype.6.dylib
        /opt/local/lib/libfontconfig.1.dylib
        /opt/local/lib/libexpat.1.dylib
        /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
        /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
        /usr/lib/libstdc++.6.dylib
        /usr/lib/libgcc_s.1.dylib
$ 

See attached file "poppler-dependencies.pdf" for a listing of the port dependencies. The first column of the table shows: port "poppler" is dependend on following ports -- freetype -- fontconfig -- gtk2 -- cairo -- pango -- glib2 -- gettext -- expat -- libiconv -- libpixman -- XFree86 -- xrender -- jpeg -- tiff -- libpng -- libxml2 -- zlib

Apart from the default installation of poppler (i.e. without any variant), I also installed the variants and combinations of them:

$ port installed poppler
The following ports are currently installed:
  poppler @0.8.4_0+abiword+darwin_8
  poppler @0.8.4_0+abiword+darwin_8+glibCairo+glibSplash
  poppler @0.8.4_0+darwin_8 (active)
  poppler @0.8.4_0+darwin_8+glibCairo
  poppler @0.8.4_0+darwin_8+glibCairo+quartz
  poppler @0.8.4_0+darwin_8+glibSplash
  poppler @0.8.4_0+darwin_8+glibSplash+quartz
  poppler @0.8.4_0+darwin_8+qt4Splash
  poppler @0.8.4_0+darwin_8+quartz
  poppler @0.8.5_0+darwin_8+qt4
$ 

The table in file "poppler-dependencies.pdf" shows that poppler @0.8.4_0+abiword+darwin_8+glibCairo+glibSplash has the same port dependencies as poppler @0.8.4_0+darwin_8 That is what I wanted to achieve. Each of the new variants, when used alone, should provide for a minimal installation of poppler, and when used in a certain combination, should provide for the same installation as the default (i.e. poppler without variants).

Variant "qt4Splash" is the only variant I want to have. It has no dependencies on ports "gtk2", "cairo", pango", "gettext2, "libpixman", "XFree86", "xrender". "tiff", and "libxml2". Only this variant is needed for port "TeXworks", if one wants to have a minimal installation. Variant "qt4" is the other choice, if you want to have a maximal installation of poppler. The variants "glibSplash", "glibCairo", and "abiword" are for completeness.

If used alone, the variants "glibSplash" and "glibCairo" do not have much less dependencies compared to the default installation of poppler, apart from port "libxml2" (see file "poppler-dependencies.pdf"). The same applies to installations "poppler+quartz" and "poppler+glibSplash+quartz". There is an advantage to use "poppler+glibCairo+quartz", because of the missing dependencies on port "gtk2" and "pango". And of course the installation of "poppler+abiword" provides you with the least installation requirements, which is interesting for those who installed port "abiword" with variant "use_binary".

If these advantages do not justify the variants "glibSplash", "glibCairo", and "abiword", then I shall remove them from the portfile, together with the unusual and lengthy tcl procedures I have written ("setConfigureOptionsForMinimalInstallationOfSelectedPopplerVariant", "setListOfPortDependenciesOfSelectedPopplerVariant").

Do you understand, how these procedures work?

Do you understand the meaning of the pre-fetch procedure?

(2) Do these variants build multiple different .dylibs or do all backends end up in one .dylib?

Common to all variants is the library:

libpoppler.3.0.0.dylib

Depending on the selected frontend (GLib and/or Qt4), the installation of poppler contains additionally one of, or both of the following libraries:

libpoppler-glib.3.0.0.dylib
libpoppler-qt4.3.0.0.dylib

The source code for the Splash output device is contained in the poppler distribution. Therefore symbols related to the Splash output device are defined in libpoppler.3.0.0.dylib:

$ otool -Rv /opt/local/lib/libpoppler.3.0.0.dylib
/opt/local/lib/libpoppler.3.0.0.dylib:
Reference table (6091 entries)
    module single module 6091 entries, at index 0
   (...)
   __ZN15SplashOutputDev10takeBitmapEv defined
   (...)

Symbols related to the Cairo output device are defined in libpoppler-glib.3.0.0.dylib:

$ otool -Rv /opt/local/lib/libpoppler-glib.3.0.0.dylib
/opt/local/lib/libpoppler-glib.3.0.0.dylib:
Reference table (1005 entries)
    module single module 1005 entries, at index 0
   (...)
   __ZN14CairoOutputDev8startDocEP4XRef defined
   (...)

[rhwood@…]

Replying to jens.traube@t-online.de:

I am new to MacPorts and still do not have the practice with what you call "developed best practices for handling variants". Also, I am about to revise my port enhancements, but do not have time for it on every day. At last, you can see, that Rainer Mueller asked me yesterday some questions and today I prepared my answers.

Best practices are documented in the guide, to which you referred to, and you replied "I am sorry, but I do not want to start this task again. I spent a lot of time to think about this port. And the result is a working port, I tested that. You do not like my solution, that's a pity. Feel free to take it or ignore it."

I apologize for having taken you at your word.

I do understand your functions, but I do not understand what wrapping the documented instructions for modifying dependency lists and configuration arguments in new functions buys you other than a difficult to maintain and difficult to read port.

[ryandesign@…]

On the one hand I would like to accept Jens's changes because he's clearly invested a lot of time into making this work. I also know the dependency on qt4-mac is extremely heavy, limiting the number of people with the patience and resources to test and improve this. We could commit the changes now, then make subsequent commits to improve it bit by bit to make it more "MacPorts-like", e.g. in the ways described below.

---

Like Randall, I have some reservations about Portfile.diff, which I'm reading today for the first time. For example, these lines should be removed:

#   Variant "qt4" completes the maximal installation of poppler.
#   This variant cannot be used together with any of the variants 
#   "qt3Splash", "qt4Splash", "glibSplash", "glibCairo", and "abiword".
#   These variants are each for a minimal installation of poppler.

    if { [variant_isset qt4] && ($A || $B || $C || $D || $E) } then {
        ui_error "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
        ui_error "variant \"qt4\" cannot be used together with any of the variants"
        ui_error "\"qt3Splash\", \"qt4Splash\", \"glibSplash\", \"glibCairo\", and \"abiword\""
        ui_error "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
        return -code 1
    }

The same effect can be achieved by changing this line:

variant qt4 {

To this:

variant qt4 conflicts qt3Splash qt4Splash glibSplash glibCairo abiword {

---

From this comment, I gather that using the "quartz" variant requires that you use the "glibCairo" variant?

#   variant "quartz" is only necessary if your active port "cairo"
#   has been build without "X Window" support (variant "no_x11")

If so, it should be marked as such:

variant quartz conflicts x11 requires glibCairo description { Support for the native Mac OS X interface } {

---

This part is not good:

# Poppler with variants +glibSplash +glibCairo +abiword (+qt3Splash)
# is the same as Poppler without any variant.

This would be a case when you should use the default_variants command. Make it so that if the user does not select any variants, the variants +glibSplash +glibCairo and +abiword get selected:

if {![variant_isset qt3Splash] && ![variant_isset qt4Splash] && ![variant_isset glibSplash] && ![variant_isset glibCairo] && ![variant_isset abiword] && ![variant_isset quartz]} {
    default_variants +glibSplash +glibCairo +abiword
}

---

Then you can get rid of the global pre-fetch block and the cairoWithX11 procedure, and replace the glibCairo variant placeholder with this:

variant glibCairo description {GLib bindings to poppler & Cairo output device} {
    pre-fetch {
        if {![variant_isset quartz] && ![file exists "${prefix}/include/cairo/cairo-xlib.h"]} {
            ui_error "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
            ui_error "The active port \"cairo\" has been built without X11 support."
            ui_error "To build Poppler, use the \"+quartz\" variant."
            ui_error "+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
            return -code error
        }
    }
}

All of the above begin to make the port more "normal", and I think it's this kind of thing Randall was talking about when he said the port felt over-engineered.

---

Of course, optionToAppend and optionToDelete are big sticking points too. Why is it necessary to use these instead of configure.args-append and configure.args-delete? Same goes for dependencyToAppend. Why not use depends_lib-append?

---

I would remove this bit because it is not necessary:

platform darwin 9 {
    configure.env-append MACOSX_DEPLOYMENT_TARGET=10.5
}

On Mac OS X 10.5, MACOSX_DEPLOYMENT_TARGET is already set to 10.5 by Apple. In contrast, on Mac OS X 10.4, 10.3 and 10.2, it is necessary to set MACOSX_DEPLOYMENT_TARGET because the default value there is 10.1.

---

This is a comparatively unimportant stylistic nit-pick, but I would use underscore_separated_words instead of camelCaseWords; I haven't seen camel case used elsewhere in MacPorts.

[ryandesign@…]

I forgot to mention that you should not declare a dependency on X11 by writing port:XFree86. You want either XFree86 or xorg or Apple X11 to be able to satisfy the X11 dependency, so you should write lib:libX11.6:XFree86 as other ports do.

[jens.traube@…]

update of patch -- based on Randell's recent update

[jens.traube@…]

Replying to ryandesign@macports.org:

Thank you for your interest and contribution. I just submitted an update of the portfile patch ("Portfile_v2.diff). It is based on Randell's recent update and includes your contributions.

I did not testet this portfile yet. It is just for an intermediate and quick feedback to your and Randell's contribution.

I want to reconsider the pre-configure procedure now.

[jens.traube@…]

update of patch -- third version

[jens.traube@…]

check of poppler installation

[jens.traube@…]

debug output of failed installation

[jens.traube@…]

Now I can present you a third version of the portfile (Portfile_v3.diff).

The "pre-configure" procedure is reduced to "ui_debug" commands, which informs about the configure arguments and the port dependencies.

I removed the lengthy tcl procedures ("setConfigureOptionsForMinimalInstallationOfSelectedPopplerVariant", "setListOfPortDependenciesOfSelectedPopplerVariant").

To check this revised Portfile completely, I had to install qt4-x11. After hours it got installed without any problems.

But I could not install the variant "poppler @0.8.5_0+darwin_8+qt4Splash+x11" due to a link error ( see attached file "poppler +qt4Splash +x11 (debug output).txt ). I leave this problem to others.

Following poppler variants of poppler are now installed on my computer:

$ port installed poppler
The following ports are currently installed:
  poppler @0.8.5_0+abiword+darwin_8+glibCairo+glibSplash+quartz
  poppler @0.8.5_0+abiword+darwin_8+glibCairo+glibSplash+x11
  poppler @0.8.5_0+abiword+darwin_8+x11
  poppler @0.8.5_0+darwin_8+glibCairo+quartz
  poppler @0.8.5_0+darwin_8+glibCairo+x11
  poppler @0.8.5_0+darwin_8+glibSplash+quartz
  poppler @0.8.5_0+darwin_8+glibSplash+x11
  poppler @0.8.5_0+darwin_8+qt4Splash+quartz (active)
$ 

The active variant is the variant needed for port "TeXworks", and it works.

The variant "poppler @0.8.5_0+darwin_8+glibSplash+x11" works also. I have testet it with port "epdfview".

I had no luck with variant "poppler @0.8.5_0+darwin_8+glibCairo+x11". To test it, I kept the build directory during installation of this variant.

The directory ${worksrcpath}/test contains five shell scripts: (1) gtk-cairo-test (2) gtk-splash-test (3) pdf_inspector (4) pdf-fullrewrite (5) perf-test

I launched "pdf_inspector" in a window of the X Window system ("XDarwin", port "XFree86"), it opens a dialog window. Then I tried to open a pdf file, but got a bus error. I also had no success with "gtk-cairo-test", it outputs "Error loading ..."

The test of variant "poppler @0.8.5_0+abiword+darwin_8+x11" can be done by someone else.

[rhwood@…]

This is much more readable. Thank you for the changes.

Abiword for Mac OS X is no longer in active development and the last "stable" version had some serious breakage. Unless you have successfully tested it, I would recommend removing the "abiword" variant.

I would split splash and cairo from the qt4 and glib variants:

variant cairo conflicts splash ... variant splash conflicts cairo ... variant qt4 requires splash ...

it looks like it is not possible to install poppler with qt4-x11 - the code to determine that is wrapped in a variant that requires quartz

also note that qt4-x11 depends on glib for its run loop, so poppler most likely needs to be configured with glib support as well in that case

[jens.traube@…]

update of patch -- fourth version

[jens.traube@…]

debug output of failed installation (based on fourth version of portfile)

[jens.traube@…]

Replying to rhwood@macports.org: Variant "abiword" is now removed, as you recommended.

Yesterday I tested the installation of poppler with qt4-x11, but without success. Therefore I added the phrase "requires quartz" to the variant "qt4Splash". Now I removed this phrase, so it is possible again to try an installation of poppler with qt4-x11. In addition, I provided for configure arguments and port dependencies. But this did not remove the link error, as you can see in the new debug output
(attached file "poppler +qt4Splash +x11 (debug output 2).txt").

Once again: I leave this problem to others.

I added "conflicts qt3Splash" to variant "qt4Splash", and the corresponding phrase to variant "qt3Splash".

I know, you do not like the "molecule" variants "qt4Splash", "glibCairo", ...

Hence, I have written down some thoughts concerning "atomic" variants:

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

frontends:

qt3, qt4, glib

---

backends:

splash, cairo,

and in a future release of poppler: "arthur"

---

gui:

x11, quartz

---

variant requirements:

variant glib requires (splash OR cairo)

variant qt3 requires (splash AND quartz)

variant qt4 requires (splash AND (quartz OR x11))

Future release of poppler:
variant qt4 requires ((splash OR arthur) AND (quartz OR x11))

variant cairo requires glib

variant splash requires (qt3 OR qt4 OR glib)

Future release of poppler:
variant arthur requires qt4

---

variant conflicts:

variant qt3 conflicts qt4
variant qt4 conflicts qt3

exclusive OR (XOR):
variant quartz conflicts x11
variant x11 conflicts quartz

At least one backend (cairo, splash) must be selected (inclusive OR)
You can select both of them.
For this reason, the following two lines are wrong:
variant cairo conflicts splash
variant splash conflicts cairo

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Tell me, how can I carry out this phrase: "variant splash requires (qt3 OR qt4 OR glib)"

The backend "splash" needs at least one frontend. It is possible to select qt4 and glib at the same time.

$ port installed poppler
The following ports are currently installed:

  (...)

  poppler @0.8.5_0+darwin_8+glibCairo+glibSplash+qt4Splash+x11 (active)

  (...)

$ 

[jens.traube@…]

update of patch -- fifth version

[jens.traube@…]

I discovered an error: In my last message I showed you a detail of the listing of installed poppler variants. The active variant has been built with port "qt4-mac" active. Therefore, it is not +x11, it should be +quartz.

$ port installed poppler
The following ports are currently installed:

  (...)

  poppler @0.8.5_0+darwin_8+glibCairo+glibSplash+qt4Splash+quartz (active)

  (...)

$ 

If variant "qt4Splash" is set, I test now the active Qt4 port: Which one of the ports "qt4-mac" and "qt4-x11" is active? I put this test in the pre-fetch procedure.

In addition, I moved the pre-fetch block in variant "glibCairo" to the pre-fetch procedure in the global namespace.

[anonymous]

Milestone Port Enhancements deleted

[blb@…]

Assigning to current poppler maintainer for input.