Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#26585 closed enhancement (fixed)

upgrade breaks on obsolete ports

Reported by: tommyd@… Owned by: macports-tickets@…
Priority: Normal Milestone: MacPorts 1.9.2
Component: base Version: 1.9.1
Keywords: Cc: narf_tm@…, macports@…
Port: p5-io-compress-zlib

Description

I haven't upgraded my ports installation for quite some time, so recently triggered port upgrade installed unattended and thought it should be finished after a couple of hours. But unfortunately it got stuck in between, while it tried to upgrade several old (apparently removed in upstream MP) ports like p5-io-compress-zlib.

I wish upgrade would be so intelligent to find out that these ports are no longer existant and would rather mark the particular ports for removal instead of trying and failing to install them.

Attachments (1)

portupgrade.txt (65.1 KB) - added by macports@… 10 years ago.
log of port upgrade

Download all attachments as: .zip

Change History (18)

comment:1 Changed 10 years ago by macports@…

Hey, if we are going to make upgrade intelligent, maybe we should make it really smart and have it build libraries BEFORE the things that depend on them rather than AFTER. That way, when something happens like OpenSSL is updated, a port upgrade would build it first rather than last. That'd save the time of the devs who are busy bumping revs on everything to clean that mess, AND it would save the time of the users for who the upgrade built OpenSSL AFTER all the ports with bumped revs who now have to upgrade --force a whole pile of basic stuff. Cause, you know it sucks when that port upgrade fails because it upgraded OpenSSL AFTER subversion and then hits a port that wants to use the now-broken subversion to fetch the files to build the current port being upgraded.

Also, when we are in this situation, it'd be nice if macports had a function to check the version of libs linked against my installed ports and rebuild as necessary to resolve those with missing versions. That would be much better than posted a script to the mailing list that is written for one specific library and version in question and has an assumed prefix hardcoded. I know a generic and reusable solution is really pushing it but in the long run it's a win.

comment:2 in reply to:  description Changed 10 years ago by ryandesign (Ryan Schmidt)

Cc: narf_tm@… added
Port: p5-io-compress-zlib added

Replying to tommyd@…:

I haven't upgraded my ports installation for quite some time, so recently triggered port upgrade installed unattended and thought it should be finished after a couple of hours. But unfortunately it got stuck in between, while it tried to upgrade several old (apparently removed in upstream MP) ports like p5-io-compress-zlib.

I wish upgrade would be so intelligent to find out that these ports are no longer existant and would rather mark the particular ports for removal instead of trying and failing to install them.

MacPorts base already has this capability since version 1.8.0; it's called the replaced_by keyword. But it's up to individual port maintainers to remember that this capability exists and to employ it when renaming ports. In this case (see #21167), the p5-io-compress-zlib port should have been retained and marked as replaced_by p5-... (whatever it was replaced by), instead of being deleted immediately.

However, we do remove the replaced ports after awhile, when we remember to do so. I typically recommend this be a period of no less than one year; I would hope users would update their ports at least once a year, hopefully much more frequently. Unfortunately in your case it looks like your ports are over a year old, so even if we had used the replaced_by feature, it probably wouldn't have helped you now. See #21167 for recommendations on how to complete this particular upgrade, if you still need assistance.

comment:3 in reply to:  1 ; Changed 10 years ago by ryandesign (Ryan Schmidt)

Replying to macports@…:

Hey, if we are going to make upgrade intelligent, maybe we should make it really smart and have it build libraries BEFORE the things that depend on them rather than AFTER. That way, when something happens like OpenSSL is updated, a port upgrade would build it first rather than last. That'd save the time of the devs who are busy bumping revs on everything to clean that mess, AND it would save the time of the users for who the upgrade built OpenSSL AFTER all the ports with bumped revs who now have to upgrade --force a whole pile of basic stuff. Cause, you know it sucks when that port upgrade fails because it upgraded OpenSSL AFTER subversion and then hits a port that wants to use the now-broken subversion to fetch the files to build the current port being upgraded.

MacPorts already does handle upgrading dependencies first, in the correct order. I'm unclear how you arrived at the situation you describe, unless you encountered a port that did not properly declare its dependencies.

Also, when we are in this situation, it'd be nice if macports had a function to check the version of libs linked against my installed ports and rebuild as necessary to resolve those with missing versions. That would be much better than posted a script to the mailing list that is written for one specific library and version in question and has an assumed prefix hardcoded. I know a generic and reusable solution is really pushing it but in the long run it's a win.

Contributions are welcome.

comment:4 in reply to:  1 Changed 10 years ago by ryandesign (Ryan Schmidt)

Cc: macports@… added

Replying to macports@…:

I responded to your comment above. (I neglected to Cc you when I did so.)

Changed 10 years ago by macports@…

Attachment: portupgrade.txt added

log of port upgrade

comment:5 in reply to:  3 ; Changed 10 years ago by macports@…

Replying to ryandesign@…:

Replying to macports@…:

Hey, if we are going to make upgrade intelligent, maybe we should make it really smart and have it build libraries BEFORE the things that depend on them rather than AFTER. That way, when something happens like OpenSSL is updated, a port upgrade would build it first rather than last. That'd save the time of the devs who are busy bumping revs on everything to clean that mess, AND it would save the time of the users for who the upgrade built OpenSSL AFTER all the ports with bumped revs who now have to upgrade --force a whole pile of basic stuff. Cause, you know it sucks when that port upgrade fails because it upgraded OpenSSL AFTER subversion and then hits a port that wants to use the now-broken subversion to fetch the files to build the current port being upgraded.

MacPorts already does handle upgrading dependencies first, in the correct order. I'm unclear how you arrived at the situation you describe, unless you encountered a port that did not properly declare its dependencies.

Also, when we are in this situation, it'd be nice if macports had a function to check the version of libs linked against my installed ports and rebuild as necessary to resolve those with missing versions. That would be much better than posted a script to the mailing list that is written for one specific library and version in question and has an assumed prefix hardcoded. I know a generic and reusable solution is really pushing it but in the long run it's a win.

Contributions are welcome.

I have attached a file which is the log of the port upgrade in question. Notice the order in which subversion and openssl are built. Subversion comes BEFORE OpenSSL, as does serf, so both had to be rebuilt to get subversion working. Both those ports list openssl as a dependency so why does macports build them in the wrong order?

comment:6 Changed 10 years ago by jmroot (Joshua Root)

Good question. I'd need to see debug output for your upgrade since I can't reproduce this behaviour myself.

comment:7 Changed 10 years ago by jmroot (Joshua Root)

Going back to the original topic, what exactly does "got stuck" mean? Ports that are present in the registry but not the index should just produce a "Warning: No port foo found in the index."

comment:8 in reply to:  7 Changed 10 years ago by tommyd@…

Replying to jmr@…:

Going back to the original topic, what exactly does "got stuck" mean? Ports that are present in the registry but not the index should just produce a "Warning: No port foo found in the index."

That means port aborts with an error rather than a warning and the other ports aren't upgraded further. I don't have the install log handy, but I called port upgrade installed just as this without further options to ignore errors.

comment:9 Changed 10 years ago by jmroot (Joshua Root)

Well I just tried with some simple test ports and couldn't reproduce this, so it would be really handy to know what the error was.

comment:10 Changed 10 years ago by tommyd@…

I'll probably still have it in the backlog at home, so stay tuned. Since the current upgrade procedure is rather messy (also wrt Perl ports which need to be reinstalled when a new version of Perl is installed and also because I stumbled upon similar problems during the OpenSSL / Subversion upgrade) I probably just remove the mess I created with the last port upgrade and install MP and all ports from scratch. I need a working system after all...

It would be absolutely tremendous to have something like apt-get, yum or zypper for MacPorts wrt version detection and dependency resolution during upgrade and installation, but unfortunately this seems to be very way down the road. I know patches are accepted, but I don't have the knowledge nor the time to hack on this myself in tcl.

comment:11 in reply to:  6 Changed 10 years ago by macports@…

Replying to jmr@…:

Good question. I'd need to see debug output for your upgrade since I can't reproduce this behaviour myself.

Well, its a little late to run with port -d to get the debug output because the upgrade is done and the broken ports were already rebuilt to get everything working.

comment:12 in reply to:  5 ; Changed 10 years ago by jmroot (Joshua Root)

Replying to macports@…:

I have attached a file which is the log of the port upgrade in question. Notice the order in which subversion and openssl are built. Subversion comes BEFORE OpenSSL, as does serf, so both had to be rebuilt to get subversion working. Both those ports list openssl as a dependency so why does macports build them in the wrong order?

Upon reflection, you probably just need to drop the unnecessary -R.

comment:13 in reply to:  12 ; Changed 10 years ago by macports@…

Replying to jmr@…:

Replying to macports@…:

I have attached a file which is the log of the port upgrade in question. Notice the order in which subversion and openssl are built. Subversion comes BEFORE OpenSSL, as does serf, so both had to be rebuilt to get subversion working. Both those ports list openssl as a dependency so why does macports build them in the wrong order?

Upon reflection, you probably just need to drop the unnecessary -R.

According to the man page, -R upgrades dependents. Meaning, if a port is upgraded and the things that depend on it are still at the same version, those dependents would be rebuilt to ensure they are properly linked against the new version. I would expect such problems if this switch were NOT provided. With it provided, then the situation should improve, not worsen. Does this switch not work as documented?

comment:14 in reply to:  10 Changed 10 years ago by tommyd@…

Replying to tommyd@…:

I'll probably still have it in the backlog at home, so stay tuned. Since the current upgrade procedure is rather messy (also wrt Perl ports which need to be reinstalled when a new version of Perl is installed and also because I stumbled upon similar problems during the OpenSSL / Subversion upgrade) I probably just remove the mess I created with the last port upgrade and install MP and all ports from scratch. I need a working system after all...

The process stopped after trying to install p5-archive-tar:

[...] ---> Fetching p5-archive-tar ---> Attempting to fetch Archive-Tar-1.60.tar.gz from ftp://ftp.cpan.org/pub/CPAN/modules/by-module/Archive ---> Verifying checksum(s) for p5-archive-tar ---> Extracting p5-archive-tar ---> Configuring p5-archive-tar ---> Building p5-archive-tar ---> Staging p5-archive-tar into destroot ---> Computing dependencies for p5-archive-tar ---> Installing p5-archive-tar @1.60_0 ---> Deactivating p5-archive-tar @1.54_0 ---> Activating p5-archive-tar @1.60_0 ---> Cleaning p5-archive-tar Warning: No port p5-compress-zlib found in the index. To report a bug, see <http://guide.macports.org/#project.tickets>

comment:15 in reply to:  13 Changed 10 years ago by jmroot (Joshua Root)

Replying to macports@…:

According to the man page, -R upgrades dependents. Meaning, if a port is upgraded and the things that depend on it are still at the same version, those dependents would be rebuilt to ensure they are properly linked against the new version. I would expect such problems if this switch were NOT provided. With it provided, then the situation should improve, not worsen. Does this switch not work as documented?

Upgrade does nothing on ports that are not outdated (unless you force/enforce-variants). If you're already upgrading everything that is outdated, asking to do their dependents as well is not helpful. You seem to be assuming that -R upgrades dependents' dependencies, but it doesn't, it just upgrades the dependents themselves.

comment:16 Changed 10 years ago by jmroot (Joshua Root)

Resolution: fixed
Status: newclosed
Summary: Intelligent upgradeupgrade breaks on obsolete ports

comment:17 Changed 10 years ago by jmroot (Joshua Root)

Milestone: MacPorts 1.9.2
Note: See TracTickets for help on using tickets.