unify the python portgroups

[jmr@…]

The current practice of having separate ports of modules for each python version is awkward, and will get far worse now that python 2.6 and 3.0 are being released. Here is a portgroup that should be able to replace the current python25 and python24 groups, and can install for any combination of the available python versions at once via variants.

[ram@…]

Cc Me!

[macsforever2000@…]

Cc Me!

[macsforever2000@…]

What will happen in the case of py-ctypes which is built into Python 2.5+?

[jmr@…]

Replying to macsforever2000@…:

What will happen in the case of py-ctypes which is built into Python 2.5+?

I suppose that ports that depend on it should only do so if a python <= 2.4 variant is set. And also py-ctypes should not declare python25+ variants (a capability yet to be added to the portgroup).

[jmr@…]

Updated with procs for setting the supported and default python versions.

[jwa@…]

Cc Me!

[myschizobuddy@…]

Cc Me!

[blb@…]

First issue is the set of "breakout" ports, those normally a part of Python but removed for dependency reasons (see ticket #12369). Since those are versioned along with Python itself, and we don't want to be setting version in a variant section, I think they'll have to remain py##- ports unless there's a better way to handle them. Or integrate back into Python and incur the heavy dependencies for them all.

[jmr@…]

Right, short of resolving #126, I guess there will need to be stuff like this in the portfiles:

foreach v {25 26 30} {
    if {[variant_isset python${v}]} {
        depends_lib-append port:py${v}-hashlib
    }
}

[blb@…]

Testing looks good so far, I have (among other ports in this test install):

$ port-test installed py*
The following ports are currently installed:
  py-mx-base @3.1.1_0+python25+python26 (active)
  py-setuptools @0.6c8_0+python25+python26 (active)
  py25-hashlib @2.5.2_0 (active)
  py25-zlib @2.5.2_0 (active)
  py26-hashlib @2.6_0 (active)
  py26-zlib @2.6_0 (active)
  python25 @2.5.2_5+darwin_9 (active)
  python26 @2.6_2+darwin_9 (active)

I used the foreach to define py25- and py26- dependencies for py-setuptools:

$ port-test info +python25 +python26
py-setuptools @0.6c8 (python, devel)
Variants:    python24, +python25, +python26
...
Build Dependencies: py25-hashlib, py26-hashlib
Library Dependencies: python25, python26, py25-zlib, py26-zlib

Now we should probably find some more esoteric modules for edge-case testing.

Obviously in this test install I have defined the py26- ports, created a python26 portgroup, and replaced the python portgroup with the one here.

[blb@…]

What's the purpose of proc python.versions and proc python.default_version? Since there are globals with the same name, which are initialized, and all use of these names is by the variable, aren't they unnecessary?

[jmr@…]

Replying to blb@…:

What's the purpose of proc python.versions and proc python.default_version? Since there are globals with the same name, which are initialized, and all use of these names is by the variable, aren't they unnecessary?

The variable and proc with the same name is just like all the regular portfile 'keywords'. They're there so that the port can specify a different set of supported python versions, and a different default version.

[blb@…]

Replying to jmr@…:

The variable and proc with the same name is just like all the regular portfile 'keywords'. They're there so that the port can specify a different set of supported python versions, and a different default version.

So it's like portmain.tcl's options ... only more directly then?

I can definitely understand the occasional need to change which python versions are supported by a module, but I would think the default version should be coming from elsewhere and not settable by a port?

[blb@…]

If #14615 gets committed, then python.default_version should come from its .current; then python_select would be the way of selecting the default version for the command line and for installing python modules.

[jmr@…]

Replying to blb@…:

If #14615 gets committed, then python.default_version should come from its .current; then python_select would be the way of selecting the default version for the command line and for installing python modules.

Then, the only problem then is what to do if the default version isn't in the list of supported versions.

[bryan@…]

Cc Me!

[mcalhoun@…]

Cc Me!

[mcalhoun@…]

Attached is a modification of first python-1.0.tcl.

The idea is to take advantage of the fact that all the pythons are
now frameworks, which handle multiple versions well.

• Nothing gets installed into ${prefix}/bin except links to ${frameworks_dir}/Python.framework/Versions/Current/bin
• Nothing gets installed into ${prefix}/share/man except links to ${frameworks_dir}/Python.framework/Versions/Current/share/man

Now, python_select automatically gives us the correct packages.

[jmr@…]

I discussed this with blb on IRC, and we concluded that this approach ultimately isn't practical without a fix for #126.

[kngspook@…]

Cc Me!

[arthur@…]

Cc Me!

[akitada@…]

Cc Me!

[tonytung@…]

Cc Me!

[eikeon@…]

Cc Me!

[stromnov@…]

Cc Me!

[jmr@…]

I have a crazy idea. Suppose we did something like this in the module ports:

name py-foo
...
variant python26 conflicts python25 python24 {
    name py26-foo
    depends_lib-delete port:py-bar
    depends_lib-append port:py26-bar
}

That is, actually change the port name in the variants. The beauty is that this takes care of the dependencies without needing #126. The horror is that this is not how variants are meant to work...

[blb@…]

The problem with using variants is the confusion of using a different PortGroup within each (wasn't there a ticket about a port trying that?) and portindex only sees one of the names (py-foo in your example, so py26-foo wouldn't exist according to PortIndex).

[jmr@…]

This would be in conjunction with the single unified python portgroup, but you're right about the index of course, which means this would still need some base support.

[jmr@…]

now with subports

[jmr@…]

altgraph ported to the new portgroup

[jmr@…]

OK, new approach, namely subports. You can try out the attached portgroup and sample portfile with current trunk.

[blb@…]

Shouldn't there be a

dist_subdir     python

in python-1.0.4.tcl as well to keep with the use of that for the distdir?

Also, for a port which has a post-destroot (eg, to install docs), that will be run with the stub py-xyz as well, so if it installs more files those will unnecessarily end up in py-xyz. The only idea I can think of so far is to set a variable when the stub stuff is setup, so it can be checked in a port's post-destroot; a bit ugly though.

[jmr@…]

I think the dist_subdir being py-xyz makes more sense than having every single python module in the same subdir.

The post-destroot would presumably fail in such a case since there would be no files in the work dir. You don't actually need an extra variable; checking if $name != $subport is sufficient. I agree it's a little unwieldy, but still less so than defining the post-destroot block once for each python version.

[blb@…]

Replying to jmr@…:

I think the dist_subdir being py-xyz makes more sense than having every single python module in the same subdir.

It'll cause some duplication for upgrades, but long-term that makes sense.

The post-destroot would presumably fail in such a case since there would be no files in the work dir. You don't actually need an extra variable; checking if $name != $subport is sufficient. I agree it's a little unwieldy, but still less so than defining the post-destroot block once for each python version.

Actually there are files in worksrcdir and the destroot ran fine, which is actually how I came across it in the first place. I'm just testing with py-bitstring; I'll attach my WIP, but isn't much different from the versioned ones.

One other issue is cleaning (for those of us who disable autoclean): 'port clean' only does py-xyz, so you also have to 'port clean subport=pyAB-xyz' to get any others. Not sure if that's a big enough issue to worry about in base.

[blb@…]

py-bitstring WIP

[jmr@…]

Ah right, distfiles isn't cleared in the stub. It should probably do that, though of course there's no guarantee it won't be repopulated afterwards.

Maybe a --subports option for clean (to make it clean all subports as well) would be handy for developers. BTW, note that the file is actually called "python-1.0.tcl" locally, it only has the extra .4 because it's been attached 4 times without overwriting.

[jmr@…]

Committed in r80976. As with all features of newly released base versions, give people a week or so to upgrade before you start using it.