poppler conflicts with xpdf

[gale@…]

poppler 0.10.1 cannot be installed simultaneously with xpdf 3.02 because there are many files that both ports try to install.

poppler advertises itself as "based on the xpdf code base". If so, it should depend on xpdf instead of installing some of xpdf's files itself. Better yet, create an xpdf-tools port that contains the files common to both.

The list of conflicting files is:

/opt/local/bin/pdffonts
/opt/local/bin/pdfimages
/opt/local/bin/pdfinfo
/opt/local/bin/pdftoppm
/opt/local/bin/pdftops
/opt/local/bin/pdftotext
/opt/local/share/man/man1/pdffonts.1.gz
/opt/local/share/man/man1/pdfimages.1.gz
/opt/local/share/man/man1/pdfinfo.1.gz
/opt/local/share/man/man1/pdftoppm.1.gz
/opt/local/share/man/man1/pdftops.1.gz
/opt/local/share/man/man1/pdftotext.1.gz

[blb@…]

Cc reporter of dup #18267.

[jeremyhu@…]

I think the with_poppler variant should be default since not enabling it causes this conflict. I propose:

Index: Portfile
===================================================================
--- Portfile	(revision 46844)
+++ Portfile	(working copy)
@@ -45,7 +45,7 @@
 
 depends_lib	 port:xorg-libs lib:libfreetype.6:freetype \
 		     port:libpaper port:openmotif port:t1lib
-depends_run  path:share/ghostscript/fonts:ghostscript
+depends_run  path:share/ghostscript/fonts:ghostscript port:poppler
 
 patchfiles   xpdf-3.02-salvage-your-eye.patch \
              patch-xpdf-NameToUnicodeTable.h.diff \
@@ -89,16 +89,6 @@
     configure.args-append --enable-a4-paper
 }
 
-variant with_poppler description {Add Poppler support} {
-	depends_run-append port:poppler
-	post-destroot {
-		foreach i { pdffonts pdfimages pdfinfo pdftoppm pdftops pdftotext } {
-			delete ${destroot}${prefix}/bin/${i}
-			delete ${destroot}${prefix}/share/man/man1/${i}.1
-		}
-	}
-}
-
 platform puredarwin {
     depends_lib-delete port:openmotif
     depends_lib-append port:lesstif
@@ -115,6 +105,13 @@
     patchfiles-delete xpdf-3.02-salvage-your-eye.patch
 }
 
+post-destroot {
+	foreach i { pdffonts pdfimages pdfinfo pdftoppm pdftops pdftotext } {
+		delete ${destroot}${prefix}/bin/${i}
+		delete ${destroot}${prefix}/share/man/man1/${i}.1
+	}
+}
+
 post-install {
     ui_msg " \
         Xpdf language support packages are also available at \

[gale@…]

Replying to jeremyhu@…:

I think the with_poppler variant should be default since not enabling it causes this conflict. I propose...

+depends_run ... port:poppler

...

+post-destroot {
+ foreach i { pdffonts pdfimages pdfinfo pdftoppm pdftops pdftotext } {
+ delete ${destroot}${prefix}/bin/${i}
+ delete ${destroot}${prefix}/share/man/man1/${i}.1
+ }

I strongly object to this proposal.

1. This adds many more gratuitous dependencies to xpdf
2. The xpdf binaries are silently replaced by poppler binaries. That is a dangerous hack.

Personally, I only really need xpdf, and I require binary compatibility of the utilities. I don't want poppler; it's deep inside some dependency tree and will hopefully disappear from my system someday. If poppler silently replaces my xpdf utilities with the wrong version, I'm hosed. The resulting critical failures would be very difficult to diagnose by someone who doesn't realize that the xpdf package is using the proposed hack.

Note that Debian does something similar to what I suggested in my original bug post - xpdf-utils is a separate package, also provided by poppler-utils. They each provide the utilities and their man pages. You can choose either one, independently of whether xpdf and/or libpoppler-dev are installed.

For MacPorts, perhaps the right thing to do is to have xpdf-utils and poppler-utils conflict with each other, and xpdf and poppler with no further dependency/conflict relationships. Or perhaps there is some way to achieve the same thing using variants, I don't know how.

It would be nice if the *-utils packages could be built without all of the X dependencies. I have checked that this is possible with xpdf - use "./configure --without-x" I know nothing about poppler.

Thanks,
Yitz

[jeremyhu@…]

How is it "dangerous" , and how is it a "hack" ?

Right now, the default is broken. A variant is in place in xpdf to "make it work". The opposite way (deleting the apps from poppler) is pretty much the same. The utilities ARE binary compatible. As a solution to satisfy you, we could add a 'no_poppler' variant which would then result in the current default situation and the current +poppler would become the new default.

[gale@…]

Replying to jeremyhu@…:

How is it "dangerous",

Because it tricks people into running a different executable than they think they are running.

and how is it a "hack" ?

Wouldn't you agree that a package that silently deletes files belonging to a different package and clobbers them with its own is a hack? Especially if some of those files are executables?

A variant is in place in xpdf to "make it work".

Not for me, I also would need a variant in poppler to be able to use this hack.

The utilities ARE binary compatible.

Could be at this moment. But even if so, that could change without warning at any time.

As a solution to satisfy you, we could add a 'no_poppler' variant which would then result in the current default situation and the current +poppler would become the new default.

Yes, assuming that we also add a +with_xpdf variant to poppler, that would satisfy my personal needs currently. But it would still be a dangerous hack. Unless there is some way to specify a conflict that is conditional on variants across two different packages, the only way to do this is to separate out the utilities from the X app for xpdf, and from the library for poppler.

But really, why not do that, like the others are doing? It is easy to build the utilities separately (at least for xpdf, I assume also for poppler). And you gain the additional advantage of vastly reduced dependencies.

[jeremyhu@…]

Replying to gale@…:

Replying to jeremyhu@…:

How is it "dangerous",

Because it tricks people into running a different executable than they think they are running.

It doesn't "trick" anyone. It just makes MacPorts use poppler for the pdf utils and xpdf just installs xpdf. It runs the app people think they run. It's not like I replaced /bin/cp with /bin/mv or something...

and how is it a "hack" ?

Wouldn't you agree that a package that silently deletes files belonging to a different package and clobbers them with its own is a hack? Especially if some of those files are executables?

As mentioned, it's not "silently deleting" files. It's just making the xpdf package no longer provide the pdf utils. They're now provided by poppler instead (as was the case with the existing poppler variant).

A variant is in place in xpdf to "make it work".

Not for me, I also would need a variant in poppler to be able to use this hack.

Yeah, well you're the exception. That's who variants are for. The default variant set should "just work". It doesn't now. My solution fixes that. If you want to use the pdf utils provided by xpdf and not have poppler on your system, you can do that with a variant.

The utilities ARE binary compatible.

Could be at this moment. But even if so, that could change without warning at any time.

But for now they are, and if they ever aren't, we can deal with it at that point.

Yes, assuming that we also add a +with_xpdf variant to poppler, that would satisfy my personal needs currently. But it would still be a dangerous hack. Unless there is some way to specify a conflict that is conditional on variants across two different packages, the only way to do this is to separate out the utilities from the X app for xpdf, and from the library for poppler. But really, why not do that, like the others are doing? It is easy to build the utilities separately (at least for xpdf, I assume also for poppler). And you gain the additional advantage of vastly reduced dependencies.

Fine. I'll just leave it "broken" for ricci to fix. This isn't my mess.

[gale@…]

Replying to jeremyhu@…:

Fine. I'll just leave it "broken" for ricci to fix. This isn't my mess.

Well, I'm lobbying for a different fix because I think it is the right fix, but don't let me get in the way of progress. :)

Your fix - if you provide variants in both xpdf and poppler and set some consistent defaults - does solve the immediate problem of the conflict, as you point out. If you think it would take a long time to get (what I believe is) the proper fix, then perhaps you should apply your simple hack now. We could leave this bug open - or close this one and open a new clean bug with a clear summary of where we are and where we need to go, based on our discussion above.

otoh if ricci could do the proper fix now, that would make me the happiest.

[ricci@…]

Replying to gale@…:

Replying to jeremyhu@…:

Fine. I'll just leave it "broken" for ricci to fix. This isn't my mess.

Well, I'm lobbying for a different fix because I think it is the right fix, but don't let me get in the way of progress. :) Your fix - if you provide variants in both xpdf and poppler and set some consistent defaults - does solve the immediate problem of the conflict, as you point out. If you think it would take a long time to get (what I believe is) the proper fix, then perhaps you should apply your simple hack now. We could leave this bug open - or close this one and open a new clean bug with a clear summary of where we are and where we need to go, based on our discussion above. otoh if ricci could do the proper fix now, that would make me the happiest.

I'll have a look at the "mess" as soon as I can, might be a few more days before I can get to it.

[ricci@…]

Replying to jeremyhu@…:

How is it "dangerous" , and how is it a "hack" ? Right now, the default is broken. A variant is in place in xpdf to "make it work". The opposite way (deleting the apps from poppler) is pretty much the same. The utilities ARE binary compatible. As a solution to satisfy you, we could add a 'no_poppler' variant which would then result in the current default situation and the current +poppler would become the new default.

What is (was) broken about the default? That poppler tries to install some of the same files that xpdf installs? That seems like a problem with poppler to me - its based on xpdf, maybe it should be picking new names for files (within the MP world that is). So far as having MP install the poppler versions of the utilities when someone installs xpdf, I think that's a bad idea. I don't know what (if any) differences there are, I do know that if I say "I'm using xpdf" but I'm really using poppler, that will be a problem for upstream debugging and such (I asked for xpdf, I expect to get the xpdf versions of utilities unless I request otherwise). As is, xpdf is broken because of the dependency on poppler that was introduced in r47030. Unless there's a good reason to keep that dependency, I'm going to back it out and put it back into the variant.

[jeremyhu@…]

yes, the brokenness is that they can not coexist with the default variants and they're both listed as dependencies in other Ports, so that needs to be addressed.

[ricci@…]

Replying to jeremyhu@…:

yes, the brokenness is that they can not coexist with the default variants and they're both listed as dependencies in other Ports, so that needs to be addressed.

I'm not finding any Portfiles that have both "xpdf" and "poppler" in them other than xpdf, poppler, and kdegraphics3. In kdegraphics3, it appears that the references to xpdf and poppler ports are commented out, and kdegraphics3 uses an internal copy of xpdf. Do you have a list of ports that use both xpdf and poppler ports? Perhaps there were some previously that have been altered?

[gale@…]

Replying to ricci@…:

I'm not finding any Portfiles that have both "xpdf" and "poppler" in them other than xpdf, poppler, and kdegraphics3. In kdegraphics3, it appears that the references to xpdf and poppler ports are commented out, and kdegraphics3 uses an internal copy of xpdf. Do you have a list of ports that use both xpdf and poppler ports? Perhaps there were some previously that have been altered?

No, the problem is that there are many ports - including gimp and inkscape - that depend on poppler. These are library dependencies. There is no reason that they should conflict with xpdf or its command-line utilities.

[ricci@…]

Replying to gale@…:

Replying to ricci@…:

I'm not finding any Portfiles that have both "xpdf" and "poppler" in them other than xpdf, poppler, and kdegraphics3. In kdegraphics3, it appears that the references to xpdf and poppler ports are commented out, and kdegraphics3 uses an internal copy of xpdf. Do you have a list of ports that use both xpdf and poppler ports? Perhaps there were some previously that have been altered?

No, the problem is that there are many ports - including gimp and inkscape - that depend on poppler. These are library dependencies. There is no reason that they should conflict with xpdf or its command-line utilities.

Ok, got it. I'll have a (further) look at it when I can.

[anonymous]

Milestone Port Bugs deleted

[mcalhoun@…]

Cc Me!

[ryandesign@…]

Has duplicate #26852.

Marking these ports as conflicting in r72435 until you find a better solution.

[gale@…]

Replying to ryandesign@…:

Marking these ports as conflicting in r72435 until you find a better solution.

Please don't do that. xpdf utilities are critically important to me, as are a whole bunch of important ports that depend on poppler.

I'd rather deal with the minor inconvenience of xpdf and poppler temporarily stomping on each other's files until this is fixed, rather than the major project of moving from MacPorts to something else. And for such a silly reason, too - a false dependency.

This is not a problem in any other distribution I know of - everyone just separates out the utilities as a separate port for each of xpdf and poppler, and makes those conflict with each other. Then everything else can coexist happily. Is there a special reason why this is so much harder in MacPorts than everywhere else?

[ricci@…]

Replying to gale@…:

Replying to ryandesign@…:

Marking these ports as conflicting in r72435 until you find a better solution.

Please don't do that. xpdf utilities are critically important to me, as are a whole bunch of important ports that depend on poppler. I'd rather deal with the minor inconvenience of xpdf and poppler temporarily stomping on each other's files until this is fixed, rather than the major project of moving from MacPorts to something else. And for such a silly reason, too - a false dependency. This is not a problem in any other distribution I know of - everyone just separates out the utilities as a separate port for each of xpdf and poppler, and makes those conflict with each other. Then everything else can coexist happily. Is there a special reason why this is so much harder in MacPorts than everywhere else?

No special reason, it just needs to get done, patches are welcome. Both xpdf and poppler are marked w/ openmaintainer, so anyone can step up and help out with them.

[devans@…]

To avoid the conflict with poppler, make the functionality of the +with_poppler variant the default and remove the variant and the conflicts statement. See  discussion on macports-users.

Committed in r72519.

Statement conflicts xpdf removed from poppler in r72520.

[ricci@…]

Did anyone do any testing to verify that the poppler command line utilities perform in the same way the xpdf command line utilities do? If not, (and there are differences) then I agree that this is a "dangerous" sort of change - people who know what the xpdf package contains (not just the xpdf X11 pdf viewer) won't get what they are expecting. Making an xpdf-tools and poppler-tools set of ports (or similar) and making them not conflict seems like the best solution. I'd suggest that the poppler-tools alter the binary and man page names to reflect their origin (as they came later than the xpdf tools).

[ricci@…]

Replying to ricci@…:

Did anyone do any testing to verify that the poppler command line utilities perform in the same way the xpdf command line utilities do? If not, (and there are differences) then I agree that this is a "dangerous" sort of change - people who know what the xpdf package contains (not just the xpdf X11 pdf viewer) won't get what they are expecting.

I was able to run a test of 'pdftotext' on ~48k PDFs, here's the summary:

Command used is: pdftotext -layout -enc ASCII7 -nopgbrk INPUT.pdf OUTPUT.txt

Both xpdf's pdftotext and poppler's pdftotext fail on certain types of data, for example poppler's tends to fail on 'ff' characters (like in the word "offer"), possibly due to ligatures. However there were a few instances where poppler worked on an 'ff' combination and xpdf didn't (go figure). Overall it seemed like xpdf did a better here.

Ordering of text elements can be different, noticed in tabular data (I suspect that each cell of the tables is its own text element in the PDF). Where I could find the data, xpdf appeared to do a better job of ordering, though neither was perfect. While there were a fair number of instances where the ordering was different, I was only able to find a few files where I could be sure I'd identified the ordering in the original document.

Poppler did a better at spacing words in some of the data, having fewer instances of run-together words.

There's also a significant difference in the text that the two versions create when the PDF has a crop box - xpdf's pdftotext appears to pull text from inside the crop box only, poppler's will pull all of the text from the document. Which of these is "better" is probably subjective - to me the user is expecting to get the text they can see when they open the PDF, which would make the xpdf version "better".

Note that this is only one test on a single set of data, and with only one application. More testing would be a good thing.

Based on the above, I do think that overwriting the xpdf command line utilities with poppler is not the right answer, we need to give people a choice here. So far as a 'default' goes (presuming we need one), if we don't get more test data then I'd vote for the xpdf utilities as they provided the text that the user would see when opening the PDF.

[gale@…]

Due to this bug, the xpdf versions of the pdf tools have now disappeared from my system, and I can't get them back with the current port versions. This is a critical showstopper issue for me; please fix it as soon as possible. Thanks. In the meantime, I have reverted xpdf to r70558 and temporarily blocked it from further upgrade in my personal upgrade script.

Replying to ricci@…:

I'd suggest that the poppler-tools alter the binary and man page names to reflect their origin (as they came later than the xpdf tools).

As an expansion on this idea, let me suggest a simple way forward that I think would solve the problem for everyone:

1. Rename the binary and man page names to reflect their origin for *both* xpdf and poppler. This allows the original ports to build the tools as before, without conflict, and without the risk of causing someone data loss by unknowingly running the wrong binary.
2. Optionally, create two new nearly-trivial ports:
   1. xpdf-tools, depends on xpdf, conflicts with poppler-tools
   2. poppler-tools, depends on poppler, conflicts with xpdf-tools

Each of these *-tools ports merely creates symlinks to the traditional names.

This requires no significant change to the existing ports, so it's really easy to implement. It allows anyone to use whichever version of the tools they need, optionally even with the traditional names. And there is no risk of confusion or conflict.

[ricci@…]

Replying to gale@…:

Due to this bug, the xpdf versions of the pdf tools have now disappeared from my system, and I can't get them back with the current port versions. This is a critical showstopper issue for me; please fix it as soon as possible. Thanks. In the meantime, I have reverted xpdf to r70558 and temporarily blocked it from further upgrade in my personal upgrade script. Replying to ricci@…:

I'd suggest that the poppler-tools alter the binary and man page names to reflect their origin (as they came later than the xpdf tools).

As an expansion on this idea, let me suggest a simple way forward that I think would solve the problem for everyone: 1. Rename the binary and man page names to reflect their origin for *both* xpdf and poppler. This allows the original ports to build the tools as before, without conflict, and without the risk of causing someone data loss by unknowingly running the wrong binary. 1. Optionally, create two new nearly-trivial ports: a. xpdf-tools, depends on xpdf, conflicts with poppler-tools a. poppler-tools, depends on poppler, conflicts with xpdf-tools Each of these *-tools ports merely creates symlinks to the traditional names. This requires no significant change to the existing ports, so it's really easy to implement. It allows anyone to use whichever version of the tools they need, optionally even with the traditional names. And there is no risk of confusion or conflict.

This idea works for me. Do any of the dependents of xpdf/poppler use the CLI tools? If so we'll need to change those to depend on the binary rather than a port name, or patch them to use the poppler-* or xpdf-* version. I like the latter better, I think it maintains the principle of least surprise.

[ricci@…]

partial resolution committed in r73295:

xpdf installs its tools with mangled names (xpdf-*) xpdf-tools links xpdf-* to its non-mangled name poppler conflicts with xpdf-tools

This should allow those of us who want the xpdf version of the CLI tools to have them with minimal impact.

[gale@…]

Replying to ricci@…:

partial resolution committed in r73295: xpdf installs its tools with mangled names (xpdf-*) xpdf-tools links xpdf-* to its non-mangled name poppler conflicts with xpdf-tools This should allow those of us who want the xpdf version of the CLI tools to have them with minimal impact.

Thanks, this is a giant step forward for me. xpdf will now get the version I need built, albeit with the mangled names.

The xpdf-tools port is going to be nearly useless in the current state, though. So many important ports depend at least indirectly on poppler that almost no one will be able to install xpdf-tools. That's why I suggested that xpdf-tools conflict only with poppler-tools, which also will only create symlinks to its own set of mangled names.