buildbot: portwatcher should not run if portbuilder is still running

[ryandesign (Ryan Carsten Schmidt)]

portwatcher is designed to wait until all its triggered portbuilders finish. This is supposed to ensure that portwatcher does not run while any portbuilder runs, but there is one situation where that is not the case.

If the machine is unexpectedly restarted while portbuilders are running (due to power outage, kernel panic, etc.) then, when it boots back up, the portbuilders that had been scheduled will pick up where they left off, and the next scheduled portwatcher will begin running, which will include syncing the ports tree and regenerating the portindex. Doing this while portbuilders are building can cause the portbuilder to fail, for example ​https://build.macports.org/builders/ports-10.5_ppc_legacy-builder/builds/21093 :

--->  Computing dependencies for gettext
DEBUG: gettext has no conflicts
DEBUG: Searching for dependency: expat
DEBUG: Found Dependency: receipt exists for expat
DEBUG: Searching for dependency: libiconv
DEBUG: Didn't find receipt, going to depspec regex for: libiconv
Warning: It looks like your PortIndex file for file:///opt/bblocal/var/buildworker/ports/build/ports may be corrupt.
Error: Dependency 'libiconv' not found.

(The "It looks like your PortIndex file may be corrupt" message is easy to reproduce on your own system if you begin installing a port from source and, before it finishes, run sudo port sync (or just portindex in the ports directory) in a second terminal window.)

[mojca (Mojca Miklavec)]

It is probably also possible to rebuild a single job on the port builder (circumventing the watcher). But that's less of a problem than this one, I agree.

[ryandesign (Ryan Carsten Schmidt)]

In 9a7661ed0166cfb5f0fdbbd87b5a2db20f5faf24/macports-infrastructure (master):

Use floating point number for priority

Use a floating point number for priority where the integer part is the
portwatcher build number and the decimal part is the index. This ensures
that one portwatcher's builds finish before any subsequent portwatcher's
builds in situations where a portwatcher runs before a previous
portwatcher's builds have finished.

See: #53587

[ryandesign (Ryan Carsten Schmidt)]

In 7c3daba5112fcf7417c1a9d37f7fb65bace699b4/mpbb (master):

Wait before cleaning watcher if "wait" file exists

See: #53587