source: users/jberry/mpwa/doc/design.txt @ 23696

Last change on this file since 23696 was 23696, checked in by jberry@…, 13 years ago

A bit of cleanup to once again sync up the mpwa submit with the design doc.

File size: 8.4 KB
1This document describes in some small amount of detail the mpwa.
5        - Develop a network-reachable port storage mechanism
7        - Allow for ad-hoc submission of new ports/revisions by any user
9        - Since users may submit ports and make them available, port availability
10          is not held up for review by committers. Committers may provide special status
11          to approved ports.
13        - All submissions and versions should always remain network reachable
14          so that old versions may be installed and/or reviewed.
16        - Allow for storage of meta information with such a submission:
17                - Submitter
18                - Comments by various users
19                - Build status
20                - (potentially binaries)
21                - etc.
23        - Allow for promotion of such a submission to into special status (stable, whatever)
24          by the port maintainer or other official personnel.
28        - We begin to implement port "submit" which allows a port-directory structure (portpkg)
29          to be submitted over the network through an http transaction.
31        - Each submitted portpkg is given a unique network-readable url and directory structure
33        - Each submitted portpkg is committed to the subversion repository.
34          (the pkgid is formed partially from the subversion revision, but that is
35          an implementation detail).
37        - The directory created for each submitted portpkg contains the originally submitted
38          compressed directory structure, an unpacked version of the same, and a meta directory
39          that might contain other meta information (comments, build reports, build status,
40          submitter information, etc).
44        - Unique portpkg submissions may be "flagged" with additional flags such as "stable".
45          (only one such flagged submission for any port? see issue below.).
46          (I'm using "flagged" to differentiate those actions which might be applied to only
47          one submission of a given port, versus "tagged" which could potentially apply
48          to multiple submissions/instances of a given port. I'm not sure this
49          differentiation is applicable, appropriate, accurate, or even useful.)
51        - We might support arbitrary tags for ports, which would be a way of escaping the
52          single category hierarchy and provide users a means of building other meaningful
53          lists of ports.
55        - We might also provide ways to query for ports based on other criteria, such as the
56          number of reported build failures for the port, generic "votes" for or against
57          the port, etc.
59        - We now begin to have the ability to search for ports by additional criteria.
60          Not just the version, but whether they are stable or not, how they are tagged,
61          and even perhaps additional things such as the reputation of the submitter or
62          how popular they are. Social ports.
64        - Note that portpkg submission is final, unequivocal, and stable. Once a submission is
65          made and assigned a pkgid/pkgurl, it may not be edited. Any changes to the
66          portpkg require a new submission. The same is not true for some of the metadata about
67          a portpkg which may be changed (comments, build status, tags, stable, etc).
71        - "port": Generically, a set of build instructions and/or resultant binaries for
72                a given piece of software. Multiple sets of such build instructions may (and
73                probably do) exist for any port, accounting for such things as changes
74                in software version, build procedures, etc.
76        - "portpkg": A particular set of build instructions for a given port. This includes
77                the bundled set of the files that this requires: Portfile + /files
78                and any other relevant metadata or contents of the port directory.
80        - "pkgid": The unique id assigned to the submission of a portpkg at
82        - "pkgurl": a url that uniquely references a portpkg.
83                This is probably formed in part from the pkgid.
85Storage layout within the macports subversion repository:
87        /portpkgs
88                /common (revisioned/shared between instances of a port)
89                        /first-letter-of-portname
90                                /portname/<directory tree> (the unbundled/uncompressed portpkg)
91                /5-low-bits-pkgid-2char-hex
92                        /5-next-bits-pkgid-2char-hex
93                                /pkgid-zero-padded-dec
94                                        /portpkg.tgz (the bundled/compressed portpkg)
95                                        /unpacked/<directory tree> (the unbundled/uncompressed portpkg)
96                                        /meta
97                                                /submission_info
98                                                /comments (user comments?)
99                                                /reports (build logs?)
100                                                /status (build status?)
101                                        /binaries (potential future expansion)
105        - It is not determined yet to what degree port meta information should be stored
106          in the database vs in the portpkg directory. It's likely that much such
107          metadata (flags, tags, version, etc) should be at least shadowed into the
108          database. Some port information (such as comments or build reports) might
109          live well only in the directory, and not in the database.
111        - The package/archive format to use for ports is not yet determined (though .tgz
112          is used currently). xar is a likely a good choice, though it will mean more work on tiger.
114        - We may need a means to build and/or fetch a local repository of the
115          port information for local use by the user, though we might be able to live
116          only with over-the-wire queries of the database. In any event, certain decisions
117          made here will effect the local repository design:
118                - The pkgurl as a unique portpkg descriptor
119                - n portpkgs for each port (even including potentially multiple
120                  portpkgs per epoch-version-revision)
121                - tags
122                - flags
124        - Perhaps the pkgid begins to supplant or replace the port revision? So that for two ports
125          at the same version, the one with the higher pkgid wins, all else being equal?
127        - What does it mean to flag a port as stable? Is there only one such designation per port?
128          Once a port instance is flagged as stable, does the previous such stable port version
129          lose that designation? Or is it possible to determine that a portpkg was previously
130          stable? And if so, that brings up the need to be able to determine the "latest stable"
131          portpkg of a port? Is stable, or whatever, simply a privileged tag, such that one
132          has to have special rights to apply it, but it can be attached to as many portpkgs
133          of a port as needed?
135        - Though we provide a network-accessible location and unique urls for portpkgs, we should not
136          restrict the ability to install an http:scheme portpkg only from this repository. The port command
137          should be able to install any properly formatted portpkg given a url to the portpkg.
138          (In other words, any url http://someurl that returns a properly formatted portpkg should
139          be installable, whether or not it is hosted at macports).
141        - Should the actions below be added to the port command, or should an additional wrapper be
142          created?
144Potential public urls for ports:
146                        // primary access to portpkg
147                        // This url returns the portpkg, not a human readable page
148      <pkgid>
150                        // A portpkg defined by a query instead of a pkgid
151                        // (will 301 redirect to<pkgid>)
152      <portname>
153      <portname>
155                        // Pages with human readable html content
156      <category>      (ports by category)
157      <tags>                       (ports by tag)
158      <portname>          (port by name)
159      <pkgid>            (portpkg by pkgid)
161Issues for urls:
163        - If a port is installed using a url that does a 301 (permanent) redirect to another port,
164          the url remembered as the installed port should be the redirected target, rather than
165          the original. This would allow the url for a stable version of a port to redirect to the
166          particular portpkg to which it resolves, and yet maintain a one-to-one mapping between
167          url and port.
172Potential port(1) Actions:
174        port submit
175                        ==> portname
176                        ==> portpkg
177                        ==> submission meta information, generically
178                        <== status
179                        <== pkgurl
180                -- Make a port submission
182        port query
183                        ==> portname
184                        <== pkgurls (with additional meta? flags, tags, submitter, date, version, revision, status?)
185                -- Query portpkgs
187        port flag_stable pkgurl
188                -- Promote a given portpkg to stable status (requires authentication)
190        port submitreport
191                        ==> pkgurl
192                        ==> report data (??? tbd)
193                -- Submit a report for a given portpkg
195        port showreports
196                        ==> pkgurl
197                        <== reports (??? tbd ...urls to the reports? or text?)
198                -- Query reports for a given port submission
200        port get
201                        ==> pkgurl
202                        <== portpkg
203                -- Bring a given portpkg into the local collection (or cache?) or portpkgs     
206Document TODO:
Note: See TracBrowser for help on using the repository browser.