buildbot schedules builds for subports that have already been built

[ryandesign (Ryan Carsten Schmidt)]

When I updated php72 to 7.2.0beta3 in [85c30526dcc1de645bd47b00a263e93cb9874cc6/macports-ports], the buildbot scheduled builds for all php70-*, php71-* and php72-* subports (but, curiously, not php70 and php71 themselves), even though php70-* and php71-* were already built long ago. See e.g. ​https://build.macports.org/builders/ports-10.12_x86_64-watcher/builds/7353 and specifically ​https://build.macports.org/builders/ports-10.12_x86_64-watcher/builds/7353/steps/subports/logs/stdio

Can anyone think of a reason why this is happening? I realize the php port is the weirdest port ever but I don't see anything there that would cause it to behave differently for php70+ than it does for php5x.

[raimue (Rainer Müller)]

The only reason for that message to appear would be if this curl command in mpbb-list-subports failed: ​https://github.com/macports/mpbb/blob/17f1018819b0cc20e4ad61c5bbb9b407968519fc/mpbb-list-subports#L53-L55

Maybe we need additional logging with the archive filename/URL?

[ryandesign (Ryan Carsten Schmidt)]

According to the web server log, the right URLs are being requested, and they're returning the right 200 status code.

[ryandesign (Ryan Carsten Schmidt)]

Since both distributable and non-distributable subports are being added to the list, I think that means it thinks the archive doesn't exist on the build machine. And I think I agree, they are not there. Now why would that be?

[ryandesign (Ryan Carsten Schmidt)]

Something wrong in uninstall-old-ports.tcl maybe?

[ryandesign (Ryan Carsten Schmidt)]

I think the problem is arising when I occasionally redeliver failed GitHub webhooks.

When we first implemented the new buildbot system using the old Subversion server, we had a "problem" that when a revision was committed, the buildbot would build the port as it appeared in the HEAD of the repository, not as it appeared in that revision. I remember we had a discussion about this, and my understanding was that we had concluded that although this wasn't what we had originally intended, it worked out better that way, because our goal wasn't to build every revision; our goal was to build the latest revision. So if two commits came in for the same port within a short time, before the buildbot had a chance to start work on the first revision, it was fine if it only built the second revision.

In redelivering the GitHub webhooks, I had assumed that behavior was still in place, but it appears not to be. Now, the git clone is rolled forward or back to the commit in question before building. And of course the local portindex is updated to reflect that state. So if I were to update php70 to a new version, and then redeliver a webhook from before that php70 update, uninstall-old-ports.tcl would dutifully uninstall the latest php70—and every other port updated since the commit currently being built—because the index now contains older versions of ports.

​Here's one example where I committed an update to mongo-tools on July 24, the webhook delivery status was recorded as failed (though it probably did actually get processed the first time, buildbot just didn't respond quickly enough), and I resent the webhook when I noticed that problem on July 31. The ​cleanup step shows over 1500 ports—presumably all those that were modified between those two dates—getting uninstalled.

Ideally I would fix the buildbot so that it's not so slow, but there could be other reasons why webhook delivery would fail, such as the buildbot being down due to software update or power outage, and I would want to redeliver those webhooks. And I wonder if the same problem would occur if someone used the buildbot "Rebuild" button.

Was the buildbot behavior deliberately changed, and if so what problem were we trying to solve? Could we go back to the previous behavior where we always build from the HEAD of the repository?

[jmroot (Joshua Root)]

Seems to have happened in [588e94a585c46532585d571957634b73ce42253c/macports-infrastructure].

[ryandesign (Ryan Carsten Schmidt)]

So I guess we could just remove the --ports-commit part?

Larry, are you still around? You haven't committed anything this year...

[ryandesign (Ryan Carsten Schmidt)]

In f02854986a62259279dccf21b4d2bf3bed939762/macports-infrastructure:

master.cfg: Always check out latest ports revision

This reverts part of 588e94a585c46532585d571957634b73ce42253c.

Closes: #54648