<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<chapter id="internals">
  <title>MacPorts Internals</title>

  <para>This chapter contains information to help gain a better understanding
  of MacPorts or learn how to configure it for non-default operation.</para>

  <section id="internals.hierarchy">
    <title>MacPorts File Hierarchy</title>

    <para>MacPorts port authors install files according to guidelines that
    define where ports should install the various classes of files in the
    MacPorts file hierarchy. These guidelines are listed below.</para>

    <variablelist>
      <varlistentry>
        <term><varname>${prefix}</varname></term>

        <listitem>
          <para>The base of the MacPorts filesystem hierarchy.</para>

          <para>Default: <filename>/opt/local/</filename></para>

          <variablelist>
            <varlistentry>
              <term><filename>bin/</filename></term>

              <listitem>
                <para>Common utilities, programming tools, and
                applications.</para>
              </listitem>
            </varlistentry>
          </variablelist>

          <variablelist>
            <varlistentry>
              <term><filename>etc/</filename></term>

              <listitem>
                <para>System configuration files and scripts.</para>
              </listitem>
            </varlistentry>
          </variablelist>

          <variablelist>
            <varlistentry>
              <term><filename>games/</filename></term>

              <listitem>
                <para>Useful and semi-frivolous programs.</para>
              </listitem>
            </varlistentry>
          </variablelist>

          <variablelist>
            <varlistentry>
              <term><filename>include/</filename></term>

              <listitem>
                <para>Standard C include files.</para>
              </listitem>
            </varlistentry>
          </variablelist>

          <variablelist>
            <varlistentry>
              <term><filename>java/</filename></term>

              <listitem>
                <para>Applications based on Java.</para>
              </listitem>
            </varlistentry>
          </variablelist>

          <variablelist>
            <varlistentry>
              <term><filename>libexec/</filename></term>

              <listitem>
                <para>System daemons and system utilities (executed by other
                programs).</para>
              </listitem>
            </varlistentry>
          </variablelist>

          <variablelist>
            <varlistentry>
              <term><filename>sbin/</filename></term>

              <listitem>
                <para>System programs and administration utilities.</para>
              </listitem>
            </varlistentry>
          </variablelist>

          <variablelist>
            <varlistentry>
              <term><filename>share/</filename></term>

              <listitem>
                <para>Architecture-independent files.</para>

                <variablelist>
                  <varlistentry>
                    <term><filename>doc/</filename></term>

                    <listitem>
                      <para>Miscellaneous documentation.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>examples/</filename></term>

                    <listitem>
                      <para>Examples for users and programmers.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>info/</filename></term>

                    <listitem>
                      <para>GNU Info hypertext system.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>locale/</filename></term>

                    <listitem>
                      <para>Localization files.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>man/</filename></term>

                    <listitem>
                      <para>Manual pages.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>misc/</filename></term>

                    <listitem>
                      <para>Miscellaneous system-wide ASCII text files.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>
              </listitem>
            </varlistentry>

            <varlistentry>
              <term><filename>var/</filename></term>

              <listitem>
                <para>Multi-purpose log, temporary, transient and spool
                files.</para>

                <variablelist>
                  <varlistentry>
                    <term><filename>db/</filename></term>

                    <listitem>
                      <para>Miscellaneous automatically generated
                      system-specific database files.</para>

                      <variablelist>
                        <varlistentry>
                          <term><filename>mports/</filename></term>

                          <listitem>
                            <para>MacPorts runtime data.</para>
                          </listitem>
                        </varlistentry>
                      </variablelist>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>spool/</filename></term>

                    <listitem>
                      <para>Directory containing output spool files.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>log/</filename></term>

                    <listitem>
                      <para>Miscellaneous system log files.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>

                <variablelist>
                  <varlistentry>
                    <term><filename>run/</filename></term>

                    <listitem>
                      <para>System information files describing various
                      information about the system since it was booted.</para>
                    </listitem>
                  </varlistentry>
                </variablelist>
              </listitem>
            </varlistentry>
          </variablelist>
        </listitem>
      </varlistentry>
    </variablelist>

    <variablelist>
      <varlistentry>
        <term><filename>/Applications/MacPorts/</filename></term>

        <listitem>
          <para>Native Mac OS X applications.</para>
        </listitem>
      </varlistentry>
    </variablelist>
  </section>

  <section id="internals.configuration-files">
    <title>Configuration Files</title>

    <para>The MacPorts configuration files do not need to be modified for the
    general end user. They contain options that may be of use to advanced
    users and port developers. All the MacPorts configuration files are
    located in <filename>/opt/local/etc/macports</filename>.</para>

    <section id="internals.configuration-files.sources-conf">
      <title>sources.conf</title>

      <para><filename>sources.conf</filename> is where the location(s) are set
      to scan for MacPorts Portfiles. This file is commonly modified to add a
      local Portfile repository.</para>
    </section>

    <section id="internals.configuration-files.macports-conf">
      <title>macports.conf</title>

      <para><filename>macports.conf</filename> contains advanced MacPorts
      configuration options. See the file comments for instructions.</para>
    </section>

    <section id="internals.configuration-files.variants-conf">
      <title>variants.conf</title>

      <para><filename>variants.conf</filename> is used to specify variants to
      be used for all ports where they exist.</para>
    </section>
  </section>

  <section id="internals.images">
    <title>Port Images</title>

    <para>MacPorts has a unique ability to allow for multiple versions,
    revisions, and variants of the same port installed at the same time, so
    you may test new port versions without uninstalling a previous working
    version.</para>

    <para>This capability derives from the fact that a MacPorts port by
    default is not installed into its final or "activated" location, but
    rather to an intermediate location that is only made available to other
    ports and end-users after an activation phase that makes hard links of all
    its files in <varname>${prefix}</varname> that point to the port's files
    in the image repository. Therefore deactivating a port image to install a
    different version only removes the hard links in
    <varname>${prefix}</varname> pointing to the previous port version's image
    —the deactivated port's image is not disturbed.</para>

    <note>
      <para>MacPorts image mode may be turned off in favor of direct mode by
      setting the portinstalltype variable in the
      <filename>macports.conf</filename> file "direct". But unless you know
      what you're doing, don't change it.</para>
    </note>
  </section>

  <section id="internals.apis">
    <title>MacPorts APIs and Libraries</title>

    <para>The MacPorts system is composed of three Tcl libraries:</para>

    <itemizedlist>
      <listitem>
        <para>MacPorts API - MacPorts public API for handling Portfiles,
        dependencies, and registry</para>
      </listitem>

      <listitem>
        <para>Ports API - API for Portfile parsing and execution</para>
      </listitem>

      <listitem>
        <para>pextlib - C extensions to Tcl</para>
      </listitem>
    </itemizedlist>

    <section id="internals.apis.ports">
      <title>Ports API</title>

      <para>The code for the Port API is located in
      <filename>base/src/port1.0</filename>. The Port API provides all the
      primitives required for a Portfile to be parsed, queried, and executed.
      It also provides a single procedure call that the MacPorts API uses to
      kick off execution: "eval_targets". The port Tcl library supplies these
      procedures, all of which are generated at run-time using the
      <code>options</code> procedure in portutil.tcl.</para>

      <para>The macports Tcl library loads the Portfile into a
      sub-interpreter, within which all port-specific code is run. This
      process ensures that there will never be pollution of the Tcl space of
      other ports, nor the MacPorts libraries, nor the calling
      application.</para>

      <note>
        <para>Portfiles are executed in a Tcl interpreter as Tcl code (and not
        truly parsed strictly speaking), so every Portfile option must be a
        TCL procedure.</para>
      </note>

      <para>The Ports API performs the following functions:</para>

      <itemizedlist>
        <listitem>
          <para>Manages target registrations. All targets register themselves
          with the Port API. Accordingly, the Port API creates pre-/post-/main
          overrides for each of the targets.</para>
        </listitem>

        <listitem>
          <para>Option/Default handling. All Portfile options (name, version,
          revision ...) are registered by targets. The Port API creates
          procedures for these options, and sets up the complex variable
          traces necessary to support option defaults.</para>
        </listitem>

        <listitem>
          <para>Executes target procedures, including the pre/post/main
          routines.</para>
        </listitem>

        <listitem>
          <para>Manages a state file containing information about what
          variants were specified and what targets have run
          successfully.</para>
        </listitem>

        <listitem>
          <para>Provides essential Portfile Tcl extensions (reinplace,
          xinstall, etc).</para>
        </listitem>

        <listitem>
          <para>Provides simple access to the ui_event mechanism by providing
          the various ui_ procedures (i.e., ui_msg, ui_error).</para>
        </listitem>
      </itemizedlist>
    </section>

    <section id="internals.apis.macports">
      <title>MacPorts API</title>

      <para>The code for the MacPorts API is located in
      <filename>base/src/macports1.0</filename>. The MacPorts API provides a
      public API into the MacPorts system by providing simple primitives for
      handling Portfiles, dependencies, and registry operations, and exports
      the MacPorts API for the <command>port</command> command line utility,
      or any other. The API has very little information about the contents
      Portfiles; instead, it relies entirely upon the <command>port</command>
      Tcl library. By keeping the high level API simple and generic, revisions
      to the underlying ports system will not necessarily require a revision
      of the high level MacPorts API.</para>

      <para>The MacPorts API is also responsible for loading user specified
      options into a sub-interpreter to be evaluated by the ports API. In that
      case it sets the variable name in the sub-interpreter and adds the
      option to the sub-interpreter's global array user_options(). User
      options are passed as part of the call to <code>mportopen</code>.</para>

      <para>The MacPorts API performs the following functions:</para>

      <itemizedlist>
        <listitem>
          <para>Dependency support.</para>

          <para>This is implemented in a highly generic fashion, and is used
          throughout the system. The dependency functions are exported to the
          Port API, and the Port API uses them to execute targets in the
          correct order.</para>
        </listitem>

        <listitem>
          <para>Dependency processing.</para>

          <para>Software dependencies are handled at this layer using the
          dependency support layer.</para>
        </listitem>

        <listitem>
          <para>UI abstractions.</para>

          <para>UI Abstractions are handled at this layer. Each port action is
          provided a context, and a mechanism for posting user interface
          events is exported to the Port API (ui_event).</para>
        </listitem>

        <listitem>
          <para>Registry management routines.</para>

          <para>Manages the rudimentary port registry in
          <filename>${prefix}/var/mports/receipts/</filename>.</para>

          <itemizedlist>
            <listitem>
              <para><code>mportregistry::new:</code> create a new port
              registry entry.</para>
            </listitem>

            <listitem>
              <para><code>mportregistry::exists:</code> check if a port
              registry entry exists (either versioned or not).</para>
            </listitem>

            <listitem>
              <para><code>mportregistry::delete:</code> delete an existing
              registry entry.</para>
            </listitem>

            <listitem>
              <para><code>mportregistry::close:</code> closes a new registry
              entry.</para>
            </listitem>
          </itemizedlist>
        </listitem>

        <listitem>
          <para>Exports the MacPorts API for use by client
          applications.</para>

          <para>The following routines are defined.</para>

          <itemizedlist>
            <listitem>
              <para><code>mportinit:</code> Initializes the MacPorts system.
              Should be called before trying to use any other
              procedure.</para>
            </listitem>

            <listitem>
              <para><code>mportsearch:</code> Given a regexp, searches the
              <filename>PortIndex</filename> for ports with matching
              names.</para>
            </listitem>

            <listitem>
              <para><code>mportopen:</code> Given a URI to a port, opens a
              Portfile and returns an opaque handle to it.</para>
            </listitem>

            <listitem>
              <para><code>portclose:</code> Given a port handle, closes a
              Portfile.</para>
            </listitem>

            <listitem>
              <para><code>mportexec:</code> Given a port handle, executes a
              target (i.e. install).</para>
            </listitem>

            <listitem>
              <para><code>mportinfo:</code> Given a port handle, this returns
              the PortInfo array (as a flat list of array elements). This is a
              little tricky and unstable and only used by the
              <command>portindex</command> command.</para>
            </listitem>

            <listitem>
              <para><code>mportdepends:</code> Given a port handle, returns a
              list of ports upon which the specified port depends.</para>
            </listitem>
          </itemizedlist>
        </listitem>
      </itemizedlist>

      <para>For an example of the MacPorts API, when one executes
      <command>port search cm3</command>, the port utility:</para>

      <itemizedlist>
        <listitem>
          <para>Calls the <code>mportsearch</code> function to find all ports
          containing "cm3".</para>
        </listitem>

        <listitem>
          <para>Returns Tcl array(s) containing data from the
          <filename>PortIndex</filename>: port name, version, revision,
          variants, etc.</para>
        </listitem>

        <listitem>
          <para>Formats the list of arrays in the standard viewing
          format.</para>
        </listitem>
      </itemizedlist>

      <para>For another MacPorts API example, when one executes <command>port
      install cm3</command>, the port utility:</para>

      <itemizedlist>
        <listitem>
          <para>Calls the <code>mportsearch</code> function to find the first
          port that matches the name "cm3".</para>
        </listitem>

        <listitem>
          <para>Calls the <code>mportopen</code> function to open the
          port.</para>
        </listitem>

        <listitem>
          <para>Calls the <code>mportexec</code> function to execute the
          install target in the port.</para>
        </listitem>

        <listitem>
          <para>Calls the <code>mportclose</code> function to close the
          port.</para>
        </listitem>
      </itemizedlist>
    </section>

    <section id="internals.apis.pextlib">
      <title>pextlib</title>

      <para>The pextlib TCL library provides a variety of C extensions to add
      capabilities to TCL procedures; for example, an interface to flock(2)
      and mkstemp(3).</para>
    </section>
  </section>

  <section id="internals.registry">
    <title>The MacPorts Registry</title>

    <para>This chapter provides an overview of the MacPorts registry and its
    API. The registry is queried by MacPorts utilities for information about
    installed ports related to dependencies, port images, and simple user
    information about what is installed. It provides abstraction over a
    modular receipt storage layer; this allows for flat file receipts as well
    as receipts stored in a SQLite database.</para>

    <para>The registry allows MacPorts utilities to:</para>

    <itemizedlist>
      <listitem>
        <para>Modify receipts to reflect changes made to installed ports being
        maintained by MacPorts.</para>
      </listitem>

      <listitem>
        <para>Query the global file and dependency databases for file
        conflicts between a port being installed and a port already
        installed.</para>
      </listitem>

      <listitem>
        <para>Maintain dependency trees of installed ports.</para>
      </listitem>
    </itemizedlist>

    <section id="internals.registry.files">
      <title>Registry Files</title>

      <para>The flat file registry (MacPorts default registry) files are
      contained in <filename>${portdbpath}/receipts</filename>, which by
      default is location
      <filename>${prefix}/var/macports/receipts</filename>. File mappings and
      dependency mappings are tracked in the flat file registry by two
      files:</para>

      <itemizedlist>
        <listitem>
          <para>file_map.db</para>
        </listitem>

        <listitem>
          <para>dep_map.bz2</para>
        </listitem>
      </itemizedlist>
    </section>

    <section id="internals.registry.api">
      <title>The Registry API</title>

      <para>The MacPorts registry provides a public API in the registry1.0 Tcl
      package. Using this API listed below you can access the MacPorts
      Registry using the default receipt storage mechanism chosen in
      <filename>macports.conf</filename>.</para>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::new_entry {name version {revision 0}
          {variants ""}}</computeroutput></term>

          <listitem>
            <para>Begin the creation of a new registry entry for the given
            port. Returns a reference ID to the registry entry created.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::open_entry {name {version 0}
          {revision 0} {variants ""}}</computeroutput></term>

          <listitem>
            <para>Opens an existing registry entry. Returns a reference ID to
            the registry entry that was opened.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::entry_exists {name version {revision
          0} {variants ""}}</computeroutput></term>

          <listitem>
            <para>Checks to see if a port exists in the registry. Returns 1 if
            the entry exists, 0 if not.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::write_entry
          {ref}</computeroutput></term>

          <listitem>
            <para>Writes the receipt associated with the given
            reference.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::delete_entry
          {ref}</computeroutput></term>

          <listitem>
            <para>Deletes the receipt associated with the given
            reference.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::property_store {ref property
          value}</computeroutput></term>

          <listitem>
            <para>Store the given value with the property name in the receipt
            associated with the given reference.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::property_retrieve {ref
          property}</computeroutput></term>

          <listitem>
            <para>Retrieve the property name from the receipt associated with
            the given reference. Returns the value of the property, if the
            property exists.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::installed {{name ""} {version
          ""}}</computeroutput></term>

          <listitem>
            <para>Get all installed ports, optionally all installed ports
            matching the given name, or the given name and version. Returns a
            list of the installed ports.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::location {portname
          portversion}</computeroutput></term>

          <listitem>
            <para>Returns the physical location the port is installed in on
            the disk. This is primarily useful for finding out where a port
            image is installed.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::open_file_map
          {args}</computeroutput></term>

          <listitem>
            <para>Opens the file map that contains file-port
            relationships.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::file_registered
          {file}</computeroutput></term>

          <listitem>
            <para>Returns the name of the port that owns the given file, if
            the file is registered as installed, and 0 otherwise.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::port_registered
          {name}</computeroutput></term>

          <listitem>
            <para>Returns a list of all files associated with the given port
            if that port is installed, and 0 otherwise.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::register_file {file
          port}</computeroutput></term>

          <listitem>
            <para>Registers the given file in the file map as belonging to the
            given port.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::unregister_file
          {file}</computeroutput></term>

          <listitem>
            <para>Removes the file from the file map.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::write_file_map
          {args}</computeroutput></term>

          <listitem>
            <para>Write the changes to the file map.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::open_dep_map
          {args}</computeroutput></term>

          <listitem>
            <para>Opens the dependency map that contains port dependency
            relationships.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::fileinfo_for_file
          {fname}</computeroutput></term>

          <listitem>
            <para>Returns a list for the given file name representing all data
            currently known about the file. This is a 6-tuple in the form
            of:</para>

            <orderedlist>
              <listitem>
                <para>file path</para>
              </listitem>

              <listitem>
                <para>uid</para>
              </listitem>

              <listitem>
                <para>gid</para>
              </listitem>

              <listitem>
                <para>mode</para>
              </listitem>

              <listitem>
                <para>size</para>
              </listitem>

              <listitem>
                <para>md5 checksum</para>
              </listitem>
            </orderedlist>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::fileinfo_for_index
          {flist}</computeroutput></term>

          <listitem>
            <para>Returns a list of information concerning each file in the
            given file list, if that file exists in the registry. The
            information if obtained through registry::fileinfo_for_file</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::list_depends
          {name}</computeroutput></term>

          <listitem>
            <para>Returns a list of all the ports that given port name depends
            on.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::list_dependents
          {name}</computeroutput></term>

          <listitem>
            <para>Returns a list of all the ports that depend on the given
            port name.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::register_dep {dep type
          port}</computeroutput></term>

          <listitem>
            <para>Registers the given dependency as the given type of
            dependency with the given port.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::unregister_dep {dep type
          port}</computeroutput></term>

          <listitem>
            <para>Unregister the given dependency of the given type as a
            dependency of the given port.</para>
          </listitem>
        </varlistentry>
      </variablelist>

      <variablelist>
        <varlistentry>
          <term><computeroutput>registry::write_dep_map
          {args}</computeroutput></term>

          <listitem>
            <para>Write changes to the dependency map.</para>
          </listitem>
        </varlistentry>
      </variablelist>
    </section>
  </section>
</chapter>