Opened 20 months ago

Closed 18 months ago

Last modified 14 months ago

#54648 closed defect (fixed)

buildbot schedules builds for subports that have already been built

Reported by: ryandesign (Ryan Schmidt) Owned by: admin@…
Priority: Normal Milestone:
Component: buildbot/mpbb Version:
Keywords: buildbot Cc: larryv (Lawrence Velázquez), mojca (Mojca Miklavec)
Port:

Description

When I updated php72 to 7.2.0beta3 in [85c30526dcc1de645bd47b00a263e93cb9874cc6/macports-ports], the buildbot scheduled builds for all php70-*, php71-* and php72-* subports (but, curiously, not php70 and php71 themselves), even though php70-* and php71-* were already built long ago. See e.g. https://build.macports.org/builders/ports-10.12_x86_64-watcher/builds/7353 and specifically https://build.macports.org/builders/ports-10.12_x86_64-watcher/builds/7353/steps/subports/logs/stdio

Can anyone think of a reason why this is happening? I realize the php port is the weirdest port ever but I don't see anything there that would cause it to behave differently for php70+ than it does for php5x.

Change History (10)

comment:1 Changed 20 months ago by raimue (Rainer Müller)

The only reason for that message to appear would be if this curl command in mpbb-list-subports failed: https://github.com/macports/mpbb/blob/17f1018819b0cc20e4ad61c5bbb9b407968519fc/mpbb-list-subports#L53-L55

Maybe we need additional logging with the archive filename/URL?

comment:2 Changed 20 months ago by ryandesign (Ryan Schmidt)

According to the web server log, the right URLs are being requested, and they're returning the right 200 status code.

comment:3 Changed 20 months ago by ryandesign (Ryan Schmidt)

Since both distributable and non-distributable subports are being added to the list, I think that means it thinks the archive doesn't exist on the build machine. And I think I agree, they are not there. Now why would that be?

comment:4 Changed 20 months ago by ryandesign (Ryan Schmidt)

Something wrong in uninstall-old-ports.tcl maybe?

comment:5 Changed 20 months ago by ryandesign (Ryan Schmidt)

Keywords: buildbot added

I think the problem is arising when I occasionally redeliver failed GitHub webhooks.

When we first implemented the new buildbot system using the old Subversion server, we had a "problem" that when a revision was committed, the buildbot would build the port as it appeared in the HEAD of the repository, not as it appeared in that revision. I remember we had a discussion about this, and my understanding was that we had concluded that although this wasn't what we had originally intended, it worked out better that way, because our goal wasn't to build every revision; our goal was to build the latest revision. So if two commits came in for the same port within a short time, before the buildbot had a chance to start work on the first revision, it was fine if it only built the second revision.

In redelivering the GitHub webhooks, I had assumed that behavior was still in place, but it appears not to be. Now, the git clone is rolled forward or back to the commit in question before building. And of course the local portindex is updated to reflect that state. So if I were to update php70 to a new version, and then redeliver a webhook from before that php70 update, uninstall-old-ports.tcl would dutifully uninstall the latest php70—and every other port updated since the commit currently being built—because the index now contains older versions of ports.

Here's one example where I committed an update to mongo-tools on July 24, the webhook delivery status was recorded as failed (though it probably did actually get processed the first time, buildbot just didn't respond quickly enough), and I resent the webhook when I noticed that problem on July 31. The cleanup step shows over 1500 ports—presumably all those that were modified between those two dates—getting uninstalled.

Ideally I would fix the buildbot so that it's not so slow, but there could be other reasons why webhook delivery would fail, such as the buildbot being down due to software update or power outage, and I would want to redeliver those webhooks. And I wonder if the same problem would occur if someone used the buildbot "Rebuild" button.

Was the buildbot behavior deliberately changed, and if so what problem were we trying to solve? Could we go back to the previous behavior where we always build from the HEAD of the repository?

comment:6 Changed 20 months ago by jmroot (Joshua Root)

Cc: larryv added

comment:7 Changed 20 months ago by ryandesign (Ryan Schmidt)

So I guess we could just remove the --ports-commit part?

Larry, are you still around? You haven't committed anything this year...

comment:8 Changed 20 months ago by mojca (Mojca Miklavec)

Cc: mojca added

comment:9 Changed 18 months ago by ryandesign (Ryan Schmidt)

Resolution: fixed
Status: newclosed

In f02854986a62259279dccf21b4d2bf3bed939762/macports-infrastructure:

master.cfg: Always check out latest ports revision

This reverts part of 588e94a585c46532585d571957634b73ce42253c.

Closes: #54648

comment:10 Changed 14 months ago by neverpanic (Clemens Lang)

Component: server/hostingbuildbot/mpbb
Note: See TracTickets for help on using tickets.