Add new dependency type "port"

[jberry@…]

Add a new style of dependency for Portfiles.

The port::portname dependency allows a simple dependency on another port, without having to resort to the lib:XXX:portname hack.

Patch attached.

[jberry@…]

patches to base/src/port1.0/portdepends.tcl and base/src/darwinports1.0/darwinports.tcl

[pguyot (Paul Guyot)]

I don't like such a patch much because it confirms a behavior that shouldn't happen in DP, namely dependency on ports only (instead of trying to see if the required software is present on the host system).

A key feature of DP is to be able to exist on various systems that already have various software installed and the way to do it with tests on libraries/binaries/paths avoid virtual packages, which is good.

We fail to do this in some very specific cases because the tests aren't complex enough yet. For example, a port might depend on a specific version of a software and to make sure that this version is there, we currently have dependencies on ports achieved with the lib:XXX: syntax.

However, this behavior should disappear in favor of a better syntax for dependencies (associated with a better dependency engine). See for example Landon & Kevin's proposal (​http://www.opendarwin.org/~landonf/papers/Dependencies_RFC.pdf).

Adding a port dependency will favor laziness among port maintainers and let them use port: syntax in cases where a regular lib:, bin: or path: syntax would do it.

[jberry@…]

Paul,

While your argument that port dependencies aren't necessary strikes a high moral tone, I believe that it nevertheless neglects some common cases where port dependencies are necessary and correct. Let me try to explain.

The argument that dependencies should be described only by bin:, lib:, or path: ignores the fact that ports often should be considered simply as providers of some functionality that it needed, and are best described in those terms, rather than by attempting to snoop for side effects of their installation though the use of path:, for instance.

A very good argument for this is the case of the port DarwinPortsStartup. This is a piece of functionality which is best described only in that way ('I rely on DarwinPortsStartup'). It breaks encapselation for another port to think that it should specify DarwinPortsStarup though a path dependency, for intance, into /System/Library/StartupItems...this assumes that the implementation of the functionality required won't change, which is a bad assumption, and make other ports far more aware than they should be of the internal implementation of DarwinPortsStartup. The dependent port simply wants to specify that it needs the functionality brough by DarwinPortsStartup; this is a perfect example of the need for a port: dependency.

The lib:XXX hack is also used heavilly in the Java ports. I've had a hand in propogating that hack through the java ports I'm submitted, a fact of which I'm not particularly proud, and which spurred me to contribute this (port dependency) patch.

In thinking about how to solve the java case, several options have occurred to me:

(1) Create a new jar: dependency which would specify a search for a jar file in the /share/java directory. (2) Use the path: dependency to look for jar files in the /share/java directory.

Either of these solutions would work, but I don't believe either is as good as using a port dependency. Let me explain why. First, let's get rid of the jar dependency by saying that it would basically be a more conveniant way of specifying the path dependency. It's convenient, and more specific, but it doesn't really imply anything different.

The problem with using the path: dependency (or a specialization like a jar: dependency) in a case like this is that it basically says: "I need a file named this at this location, and if it's not there then this port can provide it to you". This raises two interesting questions. (1) if there's a file needed at a particular location, and it's in ${prefix}, then there shouldn't be multiple ways for it to get there (multiple ports supplying the same behavior). So this is really just a complicated way of saying "I need what this port gives"--in other words, a port dependency). But (2) if the port does produce that file, and it's the only port that does produce that file, then it seems much clearer to say "I need this port". "Sniffing" for a given port based on a single file it produces is often a lie anyway, since what is often being requested is a set of functionality (or perhaps a whole set of files), a whole set of assumptions, about what a port produces, which is only hinted at by the fact that a single particular file exists in a particular location.

I argue that in almost all cases where the path dependency is needed, could be needed, or is used, a far better option would be a straightforward port dependency. If I need Paul's help on something, it's far more direct to say "I need Paul" rather than to say "I see one of Paul's footprints, and therefore he's available to help me, and if I don't see one of his footprints then I better ask for his help". Seeing his footprints doesn't necessarilly mean he's available to help me, and furthermore, he may have replaced his shoes with a new brand.

Don't construe what I just said to mean that I think a port dependency should be used in most cases. For the vast majority of cases, the bin: and lib: dependency are exactly correct, and provide compatibility with diverse operating environments. If they can be used to specify a particular requirement, they should be. All I'm saying is that (a) there is additionally a need for a straightforward dependency on a port, and (b) lib:XXX and path: (often) are simply port dependencies in disguise.

Regarding Kevin and Landon's proposal, it looks good. But I'll point out two things: (1) as far as I know it is nowhere near adoption, and (2) it actually seems to enable broad adoption of dependencies on ports, rather than strictly on their footprints. My port: dependency simply gives a way, now, to specify a dependency on a port when that's what's needed. If and when the Kevin/Landon proposal is adopted, there would be a straightforward conversion from a port: dependency to their mechanism.

[mww@…]

Some ports actually will break due to the lib:some:libsom scheme on certain environments: If they specified the exact location of the lib being in $prefix, like with

configure.args --location-of-some-lib=$prefix/lib

This actually doesnt happen, as the current ports collection is mainly used on Darwin OSs, where e. g. libsdl is not installed by default. If the configure script wont find the lib itself and it has to be pointed to the exact location, then what? Add a path-dependency to $prefix/lib/somelib? When thinking about the path-dependency, it seems more and more that the port-dependency is, what we need and that we actually could scrap the path dependency for it. Another case in favor of the path-dep would be, if the OS brings a lib that does not provide a certain functionality you need (e. g. thread safeness for some interpreter). Then the lib:whatever dependency will utterly fail, as the lib found, will not work. Only way to be sure of the given functionality would be the port-dependency.

[pguyot (Paul Guyot)]

OK. Let's forget the path: example as it indeed is like the lib:XXX hack. However, the jar example is good.

Why the hell would I have to install whatever java library from DP if it is already available in say /Library/Java/Extensions/ ?

Strictly relying on ports is what exists in other port/package systems. Most systems do so because they are on top of bare systems (e.g. there isn't much in FreeBSD's base installation).

My point is that because we're working on a package system for MacOS X, we've been developping mechanisms to avoid installing everything from DP when the host system already provides what a software needs. We'd better not install XFree86 whenever we build some program for X11.

So I prefer that we work towards fixing all the problems related to this (like the one Markus underlines here, ideally we need to tests the host libraries we depend on to decide if we need to install the DP version) rather than favoring port dependencies with your patch.

For example, we need:

• a good arithmetic on library versions
• some mechanism to get the version of binaries with an argument (e.g. -v) and a

regex.

All this requires a new syntax for port dependencies.

[jberry@…]

Paul,

jars. You gotta install them because, unfortunately, the level of flexibility in most java build files is poor. They don't have the same level of "configure" configuration that much native code has. They think they know where all their dependent jars are. So we'e got to know where they are. Which means that we typically can't readilly support a "search path" style of operation where we can accept some library that happens to be installed elsewhere. And, typically, there aren't huge numbers of libraries in pre-installed locations. Sorry, that's just how it is.

We have to recognize the difference between what would be ideal and what is real-world. In an ideal world we might not need a port dependency. In the real world we do, and always will. Rather than ignore reality, we should embrace it rather than continuing to rely on a hack which will simply be futher perpetuated.

Creating a simple port dependency syntax (as in this patch) now does not prevent us from building a fancier dependency mechansim in the future. It just puts better syntax around a hack that's being used, and provides a better transition mechanism should, or when, a better dependency mechanism be put in place.

[mww@…]

I get an error with the patch:

"Unable to open port: can't set "depends_lib": invalid depspec: port::readline"

could you check and perhaps supply a patch against the current code-base? (looks like portmgr has approved this one for port1.0 - just me being a bit slow to get it in..)

[jberry@…]

Patch to implement port dependency.

[jberry@…]

I updated the patch. I'm don't think it really changed much. Maybe you missed the change to the regular expression in portdepends.tcl?

[jberry@…]

Patch against release1 branch

[olegb@…]

commited to the release1 branch, merge with HEAD will follow.

Thanks, James.