source: users/jberry/mpwa/doc/design.txt @ 23679

Last change on this file since 23679 was 23679, checked in by jberry@…, 13 years ago

Minor tweaks to mpwa design doc

File size: 8.4 KB
Line 
1This document describes in some small amount of detail the mpwa.
2
3Goals:
4
5        - Develop a network-reachable port storage mechanism
6
7        - Allow for ad-hoc submission of new ports/revisions by any user
8       
9        - Since users may submit ports and make them available, port availability
10          is not held up for review by committers. Committers may provide special status
11          to approved ports.
12
13        - All submissions and versions should always remain network reachable
14          so that old versions may be installed and/or reviewed.
15
16        - Allow for storage of meta information with such a submission:
17                - Submitter
18                - Comments by various users
19                - Build status
20                - (potentially binaries)
21                - etc.
22
23        - Allow for promotion of such a submission to into special status (stable, whatever)
24          by the port maintainer or other official personnel.
25       
26Thus:
27       
28        - We begin to implement port "submit" which allows a port-directory structure (portpkg)
29          to be submitted over the network through an http transaction.
30
31        - Each submitted portpkg is given a unique network-readable url and directory structure
32
33        - Each submitted portpkg is committed to the subversion repository.
34          (the pkgid is formed partially from the subversion revision, but that is
35          an implementation detail).
36
37        - The directory created for each submitted portpkg contains the originally submitted
38          compressed directory structure, an unpacked version of the same, and a meta directory
39          that might contain other meta information (comments, build reports, build status,
40          submitter information, etc).
41       
42Additional:
43
44        - Unique portpkg submissions may be "flagged" with additional flags such as "stable".
45          (only one such flagged submission for any port? see issue below.).
46          (I'm using "flagged" to differentiate those actions which might be applied to only
47          one submission of a given port, versus "tagged" which could potentially apply
48          to multiple submissions/instances of a given port. I'm not sure this
49          differentiation is applicable, appropriate, accurate, or even useful.)
50       
51        - We might support arbitrary tags for ports, which would be a way of escaping the
52          single category hierarchy and provide users a means of building other meaningful
53          lists of ports.
54       
55        - We might also provide ways to query for ports based on other criteria, such as the
56          number of reported build failures for the port, generic "votes" for or against
57          the port, etc.
58       
59        - We now begin to have the ability to search for ports by additional criteria.
60          Not just the version, but whether they are stable or not, how they are tagged,
61          and even perhaps additional things such as the reputation of the submitter or
62          how popular they are. Social ports.
63       
64        - Note that portpkg submission is final, unequivocal, and stable. Once a submission is
65          made and assigned a pkgid/pkgurl, it may not be edited. Any changes to the
66          portpkg require a new submission. The same is not true for some of the metadata about
67          a portpkg which may be changed (comments, build status, tags, stable, etc).
68       
69Nomenclature:
70
71        - "port": Generically, a set of build instructions and/or resultant binaries for
72                a given piece of software. Multiple sets of such build instructions may (and
73                probably do) exist for any port, accounting for such things as changes
74                in software version, build procedures, etc.
75       
76        - "portpkg": A particular set of build instructions for a given port. This includes
77                the bundled set of the files that this requires: Portfile + /files
78                and any other relevant metadata or contents of the port directory.
79
80        - "pkgid": The unique id assigned to the submission of a portpkg at db.macports.org.
81               
82        - "pkgurl": a url that uniquely references a portpkg.
83                This is probably formed in part from the pkgid.
84               
85Storage layout within the macports subversion repository:
86
87        /port/common (revisioned/shared between instances of a port)
88                /first-letter-of-portname
89                        /portname/<directory tree> (the unbundled/uncompressed portpkg)
90        /port/portpkgs
91                /5-low-bits-pkgid-2char-hex
92                        /5-next-bits-pkgid-2char-hex
93                                /pkgid-zero-padded-dec
94                                        /portpkg.tgz (the bundled/compressed portpkg)
95                                        /unpacked/<directory tree> (the unbundled/uncompressed portpkg)
96                                        /meta
97                                                /submission_info
98                                                /comments (user comments?)
99                                                /reports (build logs?)
100                                                /status (build status?)
101                                        /binaries (potential future expansion)
102                       
103Issues:
104
105        - It is not determined yet to what degree port meta information should be stored
106          in the database vs in the portpkg directory. It's likely that much such
107          metadata (flags, tags, version, etc) should be at least shadowed into the
108          database. Some port information (such as comments or build reports) might
109          live well only in the directory, and not in the database.
110       
111        - The package/archive format to use for ports is not yet determined (though .tgz
112          is used currently). xar is a likely a good choice, though it will mean more work on tiger.
113       
114        - We may need a means to build and/or fetch a local repository of the
115          port information for local use by the user, though we might be able to live
116          only with over-the-wire queries of the database. In any event, certain decisions
117          made here will effect the local repository design:
118                - The pkgurl as a unique portpkg descriptor
119                - n portpkgs for each port (even including potentially multiple
120                  portpkgs per epoch-version-revision)
121                - tags
122                - flags
123               
124        - Perhaps the pkgid begins to supplant or replace the port revision? So that for two ports
125          at the same version, the one with the higher pkgid wins, all else being equal?
126               
127        - What does it mean to flag a port as stable? Is there only one such designation per port?
128          Once a port instance is flagged as stable, does the previous such stable port version
129          lose that designation? Or is it possible to determine that a portpkg was previously
130          stable? And if so, that brings up the need to be able to determine the "latest stable"
131          portpkg of a port? Is stable, or whatever, simply a privileged tag, such that one
132          has to have special rights to apply it, but it can be attached to as many portpkgs
133          of a port as needed?
134       
135        - Though we provide a network-accessible location and unique urls for portpkgs, we should not
136          restrict the ability to install an http:scheme portpkg only from this repository. The port command
137          should be able to install any properly formatted portpkg given a url to the portpkg.
138          (In other words, any url http://someurl that returns a properly formatted portpkg should
139          be installable, whether or not it is hosted at macports).
140       
141        - Should the actions below be added to the port command, or should an additional wrapper be
142          created?
143       
144Potential public urls for ports:
145
146                        // primary access to portpkg
147                        // This url returns the portpkg, not a human readable page
148                http://db.macports.org/portpkg/<pkgid>
149               
150                        // A portpkg defined by a query instead of a pkgid
151                        // (will 301 redirect to http://db.macports.org/portpkg/<pkgid>)
152                http://db.macports.org/portpkg/latest/<portname>
153                http://db.macports.org/portpkg/stable/<portname>
154               
155                        // Pages with human readable html content
156                http://db.macports.org/category/<category>      (ports by category)
157                http://db.macports.org/tag/<tags>                       (ports by tag)
158                http://db.macports.org/port/<portname>          (port by name)
159                http://db.macports.org/pkgid/<pkgid>            (portpkg by pkgid)
160               
161Issues for urls:
162
163        - If a port is installed using a url that does a 301 (permanent) redirect to another port,
164          the url remembered as the installed port should be the redirected target, rather than
165          the original. This would allow the url for a stable version of a port to redirect to the
166          particular portpkg to which it resolves, and yet maintain a one-to-one mapping between
167          url and port.
168               
169======
170
171                       
172Potential port(1) Actions:
173
174        port submit
175                        ==> portname
176                        ==> portpkg
177                        ==> submission meta information, generically
178                        <== status
179                        <== pkgurl
180                -- Make a port submission
181                       
182        port query
183                        ==> portname
184                        <== pkgurls (with additional meta? flags, tags, submitter, date, version, revision, status?)
185                -- Query portpkgs
186       
187        port flag_stable pkgurl
188                -- Promote a given portpkg to stable status (requires authentication)
189               
190        port submitreport
191                        ==> pkgurl
192                        ==> report data (??? tbd)
193                -- Submit a report for a given portpkg
194       
195        port showreports
196                        ==> pkgurl
197                        <== reports (??? tbd ...urls to the reports? or text?)
198                -- Query reports for a given port submission
199               
200        port get
201                        ==> pkgurl
202                        <== portpkg
203                -- Bring a given portpkg into the local collection (or cache?) or portpkgs     
204               
205               
206Document TODO:
Note: See TracBrowser for help on using the repository browser.