Opened 13 years ago

Closed 10 years ago

#16723 closed enhancement (fixed)

unify the python portgroups

Reported by: jmroot (Joshua Root) Owned by: macports-tickets@…
Priority: Normal Milestone:
Component: ports Version:
Keywords: Cc: skymoo (Adam Mercer), mf2k (Frank Schima), jyrkiwahlstedt, myschizobuddy@…, blb@…, bryan@…, MarcusCalhoun-Lopez (Marcus Calhoun-Lopez), kngspook@…, arthur@…, akitada@…, tonytung@…, eikeon@…, stromnov (Andrew Stromnov)
Port:

Description

The current practice of having separate ports of modules for each python version is awkward, and will get far worse now that python 2.6 and 3.0 are being released. Here is a portgroup that should be able to replace the current python25 and python24 groups, and can install for any combination of the available python versions at once via variants.

Attachments (6)

python-1.0.tcl (3.5 KB) - added by jmroot (Joshua Root) 13 years ago.
python-1.0.2.tcl (8.8 KB) - added by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez) 13 years ago.
python-1.0.3.tcl (7.8 KB) - added by jmroot (Joshua Root) 10 years ago.
now with subports
Portfile (1.4 KB) - added by jmroot (Joshua Root) 10 years ago.
altgraph ported to the new portgroup
python-1.0.4.tcl (8.5 KB) - added by jmroot (Joshua Root) 10 years ago.
Portfile.2 (1.2 KB) - added by blb@… 10 years ago.
py-bitstring WIP

Download all attachments as: .zip

Change History (41)

comment:1 Changed 13 years ago by skymoo (Adam Mercer)

Cc: ram@… added

Cc Me!

comment:2 Changed 13 years ago by mf2k (Frank Schima)

Cc: macsforever2000@… added

Cc Me!

comment:3 Changed 13 years ago by mf2k (Frank Schima)

What will happen in the case of py-ctypes which is built into Python 2.5+?

comment:4 in reply to:  3 Changed 13 years ago by jmroot (Joshua Root)

Replying to macsforever2000@…:

What will happen in the case of py-ctypes which is built into Python 2.5+?

I suppose that ports that depend on it should only do so if a python <= 2.4 variant is set. And also py-ctypes should not declare python25+ variants (a capability yet to be added to the portgroup).

Changed 13 years ago by jmroot (Joshua Root)

Attachment: python-1.0.tcl added

comment:5 Changed 13 years ago by jmroot (Joshua Root)

Updated with procs for setting the supported and default python versions.

comment:6 Changed 13 years ago by jyrkiwahlstedt

Cc: jwa@… added

Cc Me!

comment:7 Changed 13 years ago by myschizobuddy@…

Cc: myschizobuddy@… added

Cc Me!

comment:8 Changed 13 years ago by blb@…

Cc: blb@… added

First issue is the set of "breakout" ports, those normally a part of Python but removed for dependency reasons (see ticket #12369). Since those are versioned along with Python itself, and we don't want to be setting version in a variant section, I think they'll have to remain py##- ports unless there's a better way to handle them. Or integrate back into Python and incur the heavy dependencies for them all.

comment:9 Changed 13 years ago by jmroot (Joshua Root)

Right, short of resolving #126, I guess there will need to be stuff like this in the portfiles:

foreach v {25 26 30} {
    if {[variant_isset python${v}]} {
        depends_lib-append port:py${v}-hashlib
    }
}

comment:10 Changed 13 years ago by blb@…

Testing looks good so far, I have (among other ports in this test install):

$ port-test installed py*
The following ports are currently installed:
  py-mx-base @3.1.1_0+python25+python26 (active)
  py-setuptools @0.6c8_0+python25+python26 (active)
  py25-hashlib @2.5.2_0 (active)
  py25-zlib @2.5.2_0 (active)
  py26-hashlib @2.6_0 (active)
  py26-zlib @2.6_0 (active)
  python25 @2.5.2_5+darwin_9 (active)
  python26 @2.6_2+darwin_9 (active)

I used the foreach to define py25- and py26- dependencies for py-setuptools:

$ port-test info +python25 +python26
py-setuptools @0.6c8 (python, devel)
Variants:    python24, +python25, +python26
...
Build Dependencies: py25-hashlib, py26-hashlib
Library Dependencies: python25, python26, py25-zlib, py26-zlib

Now we should probably find some more esoteric modules for edge-case testing.

Obviously in this test install I have defined the py26- ports, created a python26 portgroup, and replaced the python portgroup with the one here.

comment:11 Changed 13 years ago by blb@…

What's the purpose of proc python.versions and proc python.default_version? Since there are globals with the same name, which are initialized, and all use of these names is by the variable, aren't they unnecessary?

comment:12 in reply to:  11 ; Changed 13 years ago by jmroot (Joshua Root)

Replying to blb@…:

What's the purpose of proc python.versions and proc python.default_version? Since there are globals with the same name, which are initialized, and all use of these names is by the variable, aren't they unnecessary?

The variable and proc with the same name is just like all the regular portfile 'keywords'. They're there so that the port can specify a different set of supported python versions, and a different default version.

comment:13 in reply to:  12 Changed 13 years ago by blb@…

Replying to jmr@…:

The variable and proc with the same name is just like all the regular portfile 'keywords'. They're there so that the port can specify a different set of supported python versions, and a different default version.

So it's like portmain.tcl's options ... only more directly then?

I can definitely understand the occasional need to change which python versions are supported by a module, but I would think the default version should be coming from elsewhere and not settable by a port?

comment:14 Changed 13 years ago by blb@…

If #14615 gets committed, then python.default_version should come from its .current; then python_select would be the way of selecting the default version for the command line and for installing python modules.

comment:15 in reply to:  14 Changed 13 years ago by jmroot (Joshua Root)

Replying to blb@…:

If #14615 gets committed, then python.default_version should come from its .current; then python_select would be the way of selecting the default version for the command line and for installing python modules.

Then, the only problem then is what to do if the default version isn't in the list of supported versions.

comment:16 Changed 13 years ago by bryan@…

Cc: bryan@… added

Cc Me!

comment:17 Changed 13 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Cc: mcalhoun@… added

Cc Me!

comment:18 Changed 13 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Attached is a modification of first python-1.0.tcl.

The idea is to take advantage of the fact that all the pythons are
now frameworks, which handle multiple versions well.

  • Nothing gets installed into ${prefix}/bin except links to ${frameworks_dir}/Python.framework/Versions/Current/bin
  • Nothing gets installed into ${prefix}/share/man except links to ${frameworks_dir}/Python.framework/Versions/Current/share/man

Now, python_select automatically gives us the correct packages.

Changed 13 years ago by MarcusCalhoun-Lopez (Marcus Calhoun-Lopez)

Attachment: python-1.0.2.tcl added

comment:19 Changed 13 years ago by jmroot (Joshua Root)

Milestone: MacPorts base enhancementsMacPorts 1.8.0
Version: 1.7.0

I discussed this with blb on IRC, and we concluded that this approach ultimately isn't practical without a fix for #126.

comment:20 Changed 13 years ago by kngspook@…

Cc: kngspook@… added

Cc Me!

comment:21 Changed 13 years ago by arthur@…

Cc: arthur@… added

Cc Me!

comment:22 Changed 12 years ago by akitada@…

Cc: akitada@… added

Cc Me!

comment:23 Changed 12 years ago by tonytung@…

Cc: tonytung@… added

Cc Me!

comment:24 Changed 12 years ago by eikeon@…

Cc: eikeon@… added

Cc Me!

comment:25 Changed 12 years ago by blb@…

Milestone: MacPorts 1.8.0MacPorts Future

comment:26 Changed 12 years ago by stromnov (Andrew Stromnov)

Cc: stromnov@… added

Cc Me!

comment:27 Changed 11 years ago by jmroot (Joshua Root)

I have a crazy idea. Suppose we did something like this in the module ports:

name py-foo
...
variant python26 conflicts python25 python24 {
    name py26-foo
    depends_lib-delete port:py-bar
    depends_lib-append port:py26-bar
}

That is, actually change the port name in the variants. The beauty is that this takes care of the dependencies without needing #126. The horror is that this is not how variants are meant to work...

comment:28 Changed 11 years ago by blb@…

The problem with using variants is the confusion of using a different PortGroup within each (wasn't there a ticket about a port trying that?) and portindex only sees one of the names (py-foo in your example, so py26-foo wouldn't exist according to PortIndex).

comment:29 Changed 11 years ago by jmroot (Joshua Root)

This would be in conjunction with the single unified python portgroup, but you're right about the index of course, which means this would still need some base support.

Changed 10 years ago by jmroot (Joshua Root)

Attachment: python-1.0.3.tcl added

now with subports

Changed 10 years ago by jmroot (Joshua Root)

Attachment: Portfile added

altgraph ported to the new portgroup

comment:30 Changed 10 years ago by jmroot (Joshua Root)

Component: baseports
Milestone: MacPorts Future

OK, new approach, namely subports. You can try out the attached portgroup and sample portfile with current trunk.

Changed 10 years ago by jmroot (Joshua Root)

Attachment: python-1.0.4.tcl added

comment:31 Changed 10 years ago by blb@…

Shouldn't there be a

dist_subdir     python

in python-1.0.4.tcl as well to keep with the use of that for the distdir?

Also, for a port which has a post-destroot (eg, to install docs), that will be run with the stub py-xyz as well, so if it installs more files those will unnecessarily end up in py-xyz. The only idea I can think of so far is to set a variable when the stub stuff is setup, so it can be checked in a port's post-destroot; a bit ugly though.

comment:32 Changed 10 years ago by jmroot (Joshua Root)

I think the dist_subdir being py-xyz makes more sense than having every single python module in the same subdir.

The post-destroot would presumably fail in such a case since there would be no files in the work dir. You don't actually need an extra variable; checking if $name != $subport is sufficient. I agree it's a little unwieldy, but still less so than defining the post-destroot block once for each python version.

comment:33 in reply to:  32 Changed 10 years ago by blb@…

Replying to jmr@…:

I think the dist_subdir being py-xyz makes more sense than having every single python module in the same subdir.

It'll cause some duplication for upgrades, but long-term that makes sense.

The post-destroot would presumably fail in such a case since there would be no files in the work dir. You don't actually need an extra variable; checking if $name != $subport is sufficient. I agree it's a little unwieldy, but still less so than defining the post-destroot block once for each python version.

Actually there are files in worksrcdir and the destroot ran fine, which is actually how I came across it in the first place. I'm just testing with py-bitstring; I'll attach my WIP, but isn't much different from the versioned ones.

One other issue is cleaning (for those of us who disable autoclean): 'port clean' only does py-xyz, so you also have to 'port clean subport=pyAB-xyz' to get any others. Not sure if that's a big enough issue to worry about in base.

Changed 10 years ago by blb@…

Attachment: Portfile.2 added

py-bitstring WIP

comment:34 Changed 10 years ago by jmroot (Joshua Root)

Ah right, distfiles isn't cleared in the stub. It should probably do that, though of course there's no guarantee it won't be repopulated afterwards.

Maybe a --subports option for clean (to make it clean all subports as well) would be handy for developers. BTW, note that the file is actually called "python-1.0.tcl" locally, it only has the extra .4 because it's been attached 4 times without overwriting.

comment:35 Changed 10 years ago by jmroot (Joshua Root)

Resolution: fixed
Status: newclosed

Committed in r80976. As with all features of newly released base versions, give people a week or so to upgrade before you start using it.

Note: See TracTickets for help on using tickets.