Opened 13 years ago

Closed 11 years ago

#17157 closed enhancement (fixed)

Update PortIndex incrementally so it is always current

Reported by: frstan@… Owned by: macports-tickets@…
Priority: Normal Milestone: MacPorts 1.9.0
Component: base Version: 1.7.0
Keywords: portindex update Cc:
Port:

Description

On Nov 9, 2008, at 9:03 AM, William Davis wrote: Commit by ryandesign@… :: r41701 /trunk/base/src/port/portindex.tcl: (link) http://trac.macports.org/changeset/41701 portindex.tcl: Generate the port index in a temporary file first, then replace the PortIndex file all at once; fixes #16234

If this is the case cant we easily update the index more than twice a day now? Perhaps even every half hour?

portindex is an annoyingly expensive operation. I don't know if the added load on the macports servers would be worth it?

If someone gets around to implementing an incremental portindex option (say we store the index in an sqlite database and we add a post-commit hook that just updates the index entries for the ports that were just committed) then we could probably have an always up-to-date index without the huge resource requirements.

-- Daniel J. Luke ticket filed by W Davis

Change History (7)

comment:1 Changed 13 years ago by wsiegrist@…

Component: server/hostingbase
Owner: changed from wsiegrist@… to macports-tickets@…

As inefficient as it is, it does not make a noticeable impact on server load. I can simply increase the frequency if thats what is desired. I'll have to see how long it takes these days to see if half-hourly is possible, but I believe hourly is definitely possible. This ticket seems to be for making indexing incremental, so I'm removing it from server/hosting and marking it as a base enhancement. If portmgr wants more frequent indexes, the can email or file a specific ticket.

comment:2 Changed 13 years ago by danielluke (Daniel J. Luke)

Cc: dluke@… removed

comment:3 Changed 13 years ago by blb@…

If #16235 gets implemented, then portindex could run for each commit. However, if that is done, then we see a commit to PortIndex every time a port is updated, doubling the number of commits to the repo.

Would it be lots of work to simply pull PortIndex out of subversion, or would it be simpler to finish the SQL index and have that sync to the MacPorts server in a port sync?

comment:4 Changed 13 years ago by raimue (Rainer Müller)

If portindex gets faster, we could even generate it on the client side only by adding it to 'port sync'.

Isn't #16235 basically the same request as this ticket?

comment:5 in reply to:  4 ; Changed 13 years ago by blb@…

Replying to raimue@…:

If portindex gets faster, we could even generate it on the client side only by adding it to 'port sync'.

Wouldn't that require us to either pay attention to what rsync is updating, or to crawl through afterwards looking for newer files? The latter works also for file:// synced via svn.

comment:6 in reply to:  5 Changed 13 years ago by raimue (Rainer Müller)

Replying to blb@…:

Replying to raimue@…:

If portindex gets faster, we could even generate it on the client side only by adding it to 'port sync'.

Wouldn't that require us to either pay attention to what rsync is updating, or to crawl through afterwards looking for newer files? The latter works also for file:// synced via svn.

Yes, my idea was to store the last modified timestamp of a Portfile in the PortIndex and only update the entry if the file is newer, by using the incremental portindex (as described in #16235). This should work for every sync method.

comment:7 Changed 11 years ago by jmroot (Joshua Root)

Milestone: MacPorts FutureMacPorts 1.9.0
Resolution: fixed
Status: newclosed

Updating indices on sync as of r66846.

Note: See TracTickets for help on using tickets.