gpgme: install headers in a private location to avoid conflict and/or header confusion

[RJVB (René Bertin)]

port:gpgme was recently updated to 1.7.0 and to build the C++ wrappers (but only when using libc++, for some reason).

Several issues result from this:

• port:kdepimlibs4 installs its own gpgme++ libraries and headers; unmodified those go into ${prefix}/include/gpgme++ and thus clash with the ones from port:gpgme

• KDE actually maintains its own C++ wrappers (fork or original I don't know). It seems that (very recent) KF5 code can co-exist with the headers from port:gpgme, but the KDE4 code that uses these C++ wrappers is now old and mostly unmaintained. It is unlikely that it will build and run against the 1.7.0 gpgme++ wrappers without significant modification of the build system and code (see #52342 and #52470). The 1.7.0 gpgme C libraries are fine, though.

The library binaries from port:kdepimlibs4 and port:gpgpme do not clash, so the main issue at hand is to avoid headerfile confusion where a gpgme++ header is included from port:gpgme when one from port:kdepimlibs4 is required, or vice versa.

It is trivial to reconfigure port:kdelibs4 such that headers from KDE4 ports are installed into, say, ${prefix}/include/KDE4. This has been proposed already as it is required for concurrent installation of KDE4 and KF5 ports.

It is relatively trivial to do something similar with port:gpgme, and the attached patch contains an implementation. I tested this approach and can confirm that

• The KDEPIM4 ports as well as port:kde4-runtime build and run provided the KDE4 headers are installed as described above
• kf5-gpgmepp and kf5-kwallet also build (with an upstream patch to support gpgme 1.7). I have not yet made ports for other KF5 software that uses gpgme.

As far as I know, installing the C++ headers is a new feature in port:gpgme 1.7 and as such I *presume* that only few ports depend on it as yet. Either way, the proposed change should be completely transparent and doesn't change anything link dependencies. No rev-bumping of dependents is required. However, it might be useful to add a variant to control building (or omission) of the C++ wrappers.

See also ​https://bugs.gnupg.org/gnupg/issue2733

[larryv (Lawrence Velázquez)]

Why should gpgme move anything? kdepimlibs4 is the one installing its own private stuff.

[larryv (Lawrence Velázquez)]

Please don’t set ticket priority; that is for committers to decide.

[RJVB (René Bertin)]

Replying to larryv@…:

Why should gpgme move anything? kdepimlibs4 is the one installing its own private stuff.

And why would kdepim4 have to be broken because another port suddenly starts installing files with the same name at the same location?

If anything, kdepimlibs4 was installing those headers long before port:gpgme did, and if you read correctly the fix involves BOTH ports moving them. That's a direct consequence of how compiler include paths work: it's simply not feasible to instruct the compiler to ignore include/gpgme++ when looking for <gpgme++/foo.h>.

We should think of this as 2 ports installing different implementations of a same idea which sadly can clash. Either both ports cannot be allowed to install the C++ wrappers, or both must make some changes - changes which in both cases are foreseen by the buildsystem.

EDit: you'd have a point if a significant number of high-value ports could now finally be installed (without depending on kdepimlibs4). It's impossible for me to know if there are any such ports (for that the C++ wrappers would have to be installed via a subport). But even in that case: well behaved dependents will use the metadata provided by port:gpgme to learn how to find the gpgme++ headerfiles. In other words, moving them to a dedicated location shouldn't be an issue at all, and doesn't even require a rebuild of the dependents.

[larryv (Lawrence Velázquez)]

Replying to rjvbertin@…:

And why would kdepim4 have to be broken because another port suddenly starts installing files with the same name at the same location?

It doesn’t have to be broken. It can be patched to hide its gpgme++ stuff.

If anything, kdepimlibs4 was installing those headers long before port:gpgme did

And? My concern is that future ports will all have to be told to look in a nonstandard location to use the public headers now provided by GPGME. (Granted, I’m not actually expecting many such ports.)

That's a direct consequence of how compiler include paths work: it's simply not feasible to instruct the compiler to ignore include/gpgme++ when looking for <gpgme++/foo.h>.

Setting -I/opt/local/include/KDE4 -I/opt/local/include wouldn’t cause that compilation to find /opt/local/include/KDE4/gpgme++/foo.h first?

If that’s false, then the rest of my argument is void (obviously).

We should think of this as 2 ports installing different implementations of a same idea which sadly can clash. Either both ports cannot be allowed to install the C++ wrappers, or both must make some changes - changes which in both cases are foreseen by the buildsystem.

I don’t understand. The few ports that already use private headers/libraries are expected to keep them out of sight to prevent issues with other builds. The ports that publicly provide the same headers/libraries are not expected to do the same.

you'd have a point if a significant number of high-value ports could now finally be installed (without depending on kdepimlibs4). It's impossible for me to know if there are any such ports (for that the C++ wrappers would have to be installed via a subport). But even in that case: well behaved dependents will use the metadata provided by port:gpgme to learn how to find the gpgme++ headerfiles.

The port provides no such metadata. Does GPGME provide a program or configuration file that dependents are expected to query for the header location? Otherwise ports that want to use gpgme’s headers will have to explicitly look elsewhere for the headers.

(Again, I admit that this is more philosophical than anything. I am not expecting a flood of ports that need GPGME’s headers.)

[larryv (Lawrence Velázquez)]

A tangent: If GPGME is only now providing public C++ bindings, then what is KDE installing? I don’t really understand what’s going on there.

[RJVB (René Bertin)]

Replying to larryv@…:

It doesn’t have to be broken. It can be patched to hide its gpgme++ stuff.

That is part of the solution, but even when that's done it still must not find the newer gpgme++ stuff.

And? My concern is that future ports will all have to be told to look in a nonstandard location to use the public headers now provided by GPGME.

The point is that this is not true for well-behaved ports. They will use the cmake file provided by gpgme, which will tell them exactly to look. Just like it does now, except that the location provided will be slightly different.

Setting -I/opt/local/include/KDE4 -I/opt/local/include wouldn’t cause that compilation to find /opt/local/include/KDE4/gpgme++/foo.h first?

You'd expect that because /opt/local/include isn't a default location like /usr/include or /usr/local/include, but that's not my experience. It's also my experience that KDE and in general any project of even moderate complexity that uses cmake will have lots of -I and -isystem arguments in its compiler invocations, which makes it almost impossible to hierarchise the search path.

This is easier with the linker search path. There you can precede a library spec with a -L option that gives its location. But headers are included in the source, *after* the full search path has been set. To make things a bit more complicated: you can easily have a KDE4 header that calls for a gpgme++ header, but somewhere else a non-KDE4 header is included that also calls for a gpgme++ header. How do you decide which to include?

Either way, I have grappled with this quite a bit back when I was setting up my KF5 ports so that they can co-install with the KDE4 equivalents. It's probably not impossible to do that without putting the KDE4 headers somewhere where they cannot be picked up by accident (the KF5 headers already are), but it's practically not feasible. With autoconf-based build systems it's still doable (usually) to patch the resulting Makefile(s) or make.conf, but with CMake that becomes a nightmare. There are lots of make files, most that contain the actual tidbits to patch nicely hidden ... and they can be regenerated at any moment during a build.

I don’t understand. The few ports that already use private headers/libraries are expected to keep them out of sight to prevent issues with other builds. The ports that publicly provide the same headers/libraries are not expected to do the same.

I don't know the exact story behind gpgme++, and why KDE have their own version/fork/branch. I think there's been a confusion here between private headers and installing public headers to what I called a private location, for lack of a better term. Until now, kdepimlibs4 provided the gpgme++ headers and libraries, period. This explains why certain ports depend on kdepimlibs4: for gpgme++ (in kde4-runtime it's the kwalletd which requires gpgme++). This is how my KDE4-based Linux box provides gpgme++, and even if you look at the latest Ubuntu you'll see that gpgme++ is provided through KDE (KF5) gpgmepp. The gpgme tarball is used only to install the C interface, not the C++ wrappers.

The port provides no such metadata. Does GPGME provide a program or configuration file that dependents are expected to query for the header location?

Yes: GpgmeppConfig.cmake

Otherwise ports that want to use gpgme’s headers will have to explicitly look elsewhere for the headers.

Ports that don't use cmake indeed will. But I wonder how many of those there are. Remember, Ubuntu only provides the KF5 gpgmepp variant, which puts its headers into /usr/include/KF5/gpgme++ . I think they would have reconsidered that if there were a lot of software that doesn't use cmake (or even requires the non-KDE gpgme++ variant).

But what's worse, allow a conflict situation to occur where you can either install a whole family of ports (KDE) or else a few other ports that don't work with KDE's gpgme++ ... or a situation where some of those few ports have to be told to look in a specific location for some of their headers?

(Again, I admit that this is more philosophical than anything. I am not expecting a flood of ports that need GPGME’s headers.)

I understand it's a matter of principal, and I also would have liked an easy way to use just port:gpgme instead of the C libraries from that port and then C++ wrappers (for those C libraries!) from either KDE or possibly gpgme. That's why I made time for looking into this as soon as I caught wind of the issue. But it really seems we have to be practical here, at least to implement a fast solution for the current deadlock. I've asked about kdepim4 on the kdepim ML and I have contacted one of the KF5 gpgmepp maintainers who's also involved with gpgme itself about their roadmap and intentions. And the gpgme bug report I filed explains why I install gpgme's headers to a dedicated location so there too feedback can come about other solutions.

If GPGME is only now providing public C++ bindings, then what is KDE installing? I don’t really understand what’s going on there.

KDE installs a slightly different variant of the same thing, with slightly different headers and using an additional library (for threaded operation). KDE code based on KDE's latest version can build using the headers from gpgme by including boost headers itself that were dropped from gpgme's C++ wrappers. This is at least the case for the KWallet framework. I don't know if this is indeed a general principle, but it might not be impossible to get kdepim4 to build with gpgme's headers too. I hope to have another look at that tomorrow. BTW: KF5 KWallet picks up the gpgme++ headers from their new location without any need for manual intervention.

[RJVB (René Bertin)]

it might not be impossible to get kdepim4 to build with gpgme's headers too. I hope to have another look at that tomorrow.

About that: there are 218 files in kdepim4 that include file(s) from a gpgme++ or qgpgme directory... a "bit" more than I hoped.

[jjstickel (Jonathan Stickel)]

Cc Me!

[NicosPavlov]

Same answer as in #52481 about the conflict between gpgme new headers and kdepimlibs4 internal ones, as the two tickets are related.

Just my two cents as the supposed maintainer of kdepimlibs4. This port has been installing its own version of the gpgme headers for its internal use. Its seems logical to me that we should move kdepimlibs4 files out of the way to leave the "general" ones in their standard place.

KDE provides its own FindGpgme.cmake file for finding its components. Hopefully I could tweak it so that the other KDE ports could find the internal KDE headers and libraries.

[RJVB (René Bertin)]

Replying to nicos@…:

Its seems logical to me that we should move kdepimlibs4 files out of the way to leave the "general" ones in their standard place.

Yes, that would be logical if it worked. It works for kdepimlibs4, but port:kdepim4 will fail to build because as I explained to Larry earlier, it finds the "general" ones first. At least it did for me, and I haven't found a way to prevent it from happening. I'll add another post to the other, more specific ticket (here: ​https://trac.macports.org/ticket/52481#comment:11).

[hjs@…]

Cc Me!

[NicosPavlov]

Kdepimlibs4 now install the headers in a private location (r153562) and dependents were changed to compile accordingly (r153563). There should be no need to perform any change on gpgme's side.