Changes between Initial Version and Version 1 of howto/ShareArchives2

Timestamp:

Aug 3, 2011, 4:56:47 AM (13 years ago)

Author:

fracai

Comment:

Initial dump of instructions for configuring archive sharing

howto/ShareArchives2

@@ +1 @@
+[wiki:howto <- Back to the HOWTO section]
+
+= Sharing Archives under MacPorts 2 (incomplete) =
+
+ * Audience: Users who want to share binary archives between machines
+ * Requires: MacPorts >= 2.0.0
+
+== Introduction ==
+
+Users with multiple machines can save significant compilation effort by installing pre-built ports from binary archives that have been assembled on another system. Under previous MacPorts releases this [https://trac.macports.org/wiki/howto/ShareArchives involved] enabling archive mode on all systems, manually transferring archives, and sometimes manually unpacking the archive before proceeding with the installation. MacPorts 2.0.0 introduces features which permanently enables archive mode and includes support for fetching archives from local or remote sources.
+
+== Installation ==
+
+=== Step 1: '''Generate signing keys''' ===
+
+Before MacPorts will install software from a binary archive, it will test that the package has a valid signature that is locally trusted. For now, we just need to generate the keys and sign the packages.
+
+The public and private keys are generated as described below.
+{{{
+openssl genrsa -des3 -out privkey.pem 2048
+openssl rsa -in privkey.pem -pubout -out pubkey.pem
+}}}
+
+You will be prompted to enter a passphrase to protect the private key. It is up to you to decide how complex this protection should be. Note that it is possible to store this key in your Keychain.app (using /usr/bin/ssh-add). This has the advantage of passing the key security on to the OS, though it may not be an acceptable option for automating package signing.
+
+It is also possible to create a key without a passphrase. If you are not distributing packages outside of a home network, this is likely not a problem. An unprotected private key is generated as below; the command to generate the public key is unchanged.
+
+{{{
+openssl genrsa -out privkey.pem 2048
+}}}
+
+The passphrase can also be stripped from an existing private key using the following:
+{{{
+openssl rsa -in privkey.pem -out newprivkey.pem
+}}}
+
+The public and private keys can be stored anywhere as long as they are accessible to the following steps. This tutorial has them placed on the build system at '''/usr/local/share/macports/'''.
+
+Note: While the build system need only have access to the private key, it's a good idea to store both the public and private keys in the same location. The installation system should only have access to the public key.
+
+=== Step 2: '''Let's sign some packages''' ===
+
+Now that the keys are generated, we can test signing a package. Any archive will do; the following example uses '''archive.tbz'''. Your example should specify the full path to the private key, the archive, and the archive signature that should be named identically to the input archive and suffixed with '''.rmd160'''.
+
+{{{
+openssl dgst -ripemd160 -sign privkey.pem -out archive.tbz2.rmd160 archive.tbz2
+}}}
+
+You can verify the signature as well:
+{{{
+openssl dgst -ripemd160 -verify pubkey.pem -signature archive.tbz2.rmd160 archive.tbz2
+}}}
+
+Signing an archive for the '''lighttpd''' port might look something like this:
+
+{{{
+openssl dgst -ripemd160 \
+    -sign /usr/local/share/macports/privkey.pem \
+    -out /opt/local/var/macports/software/lighttpd/lighttpd-1.4.28_0+ssl+universal.darwin_10.i386-x86_64.tbz2.rmd160 \
+    /opt/local/var/macports/software/lighttpd/lighttpd-1.4.28_0+ssl+universal.darwin_10.i386-x86_64.tbz2
+}}}
+
+Now, signing every individual archive like this would be quite time consuming. It'd be better to run a script which finds and signs every archive. Such a script is displayed below:
+
+{{{
+#!/bin/sh
+
+PRIVKEY="/usr/local/share/macports/macports-donnybrook.priv"
+PUBKEY="/usr/local/share/macports/macports-donnybrook.pub"
+SOFTWARE="/opt/local/var/macports/software"
+
+# First, clear out any outdated signatures
+for SIGNATURE in "$SOFTWARE"/*/*.rmd160
+do
+    ARCHIVE_DIR="$(dirname "$SIGNATURE")"
+    ARCHIVE="$(basename "$SIGNATURE" .rmd160)"
+
+    if [ "$SIGNATURE" -ot "$ARCHIVE_DIR"/"$ARCHIVE" -o ! -f "$ARCHIVE_DIR"/"$ARCHIVE" ]
+    then
+        /bin/echo removing outdated signature: "$SIGNATURE"
+        /bin/rm -f "$SIGNATURE"
+    fi
+done
+
+# Now, find every archive that doesn't have a signature
+for ARCHIVE in "$SOFTWARE"/*/*.{tbz2,tgz,tar,tbz,tlz,txz,xar,zip,cpgz,cpio}
+do
+    PORTNAME="$(basename "$(dirname "$ARCHIVE")")"
+    ANAME="$(basename "$ARCHIVE")"
+
+    if [ "$ARCHIVE" -nt "$ARCHIVE".rmd160 ]
+    then
+        /bin/echo -n deploying archive: "$ANAME "
+        /usr/bin/openssl dgst -ripemd160 -sign "$PRIVKEY" -out "$ARCHIVE".rmd160 "$ARCHIVE"
+        /usr/bin/openssl dgst -ripemd160 -verify "$PUBKEY" -signature "$ARCHIVE".rmd160 "$ARCHIVE"
+    fi
+done
+}}}
+
+The script can be run after upgrading, installing, or uninstalling ports to remove outdated signatures and generate updated signatures for all available archives.
+
+=== Step 3: '''Share archives''' ===
+
+In order to retrieve archives from the build system, they must first be shared. MacPorts expects this to be a basic web listing of the software directory ('''${prefix}/var/macports/software/'''). While Mac OS X ships with an installation of Apache, that's a bit heavy for what we need. Instead, the light weight web server [http://www.lighttpd.net/ lighttpd] can be used with minimal configuration.
+
+Conveniently, it's available through MacPorts, so go ahead and install that.
+{{{
+port install lighttpd
+}}}
+
+Create the configuration file and save it next to the public and private keys. (Like the keys, this file can be stored anywhere that is accessible.)
+
+{{{
+server.document-root = "/opt/local/var/macports/software/"
+
+server.username  = "_www"
+server.groupname = "_www"
+
+server.port = 6227
+
+dir-listing.activate = "enable"
+
+mimetype.assign = (
+    ".tbz2"     => "application/x-bzip-compressed-tar",
+    ".rmd160"   => "text/binary",
+
+    # make the default mime type application/octet-stream.
+    ""          => "application/octet-stream",
+)
+}}}
+
+Note: The '''server.port''' configuration is somewhat arbitrary as long as something non-standard is used so as not to collide with an existing service. The standard ports 80 or 8080 are options if you are not running another web server.
+
+You should now be able to start the service...
+{{{
+/opt/local/sbin/lighttpd -D -f /usr/local/share/macports/macports-archives-lighttpd.conf
+}}}
+and view your software archives in a browser:
+[http://localhost:6267]
+
+There is also a simple tutorial for enabling [http://redmine.lighttpd.net/wiki/1/HowToSimpleSSL SSL] support for additional security if it is desired. Be sure to install '''lighttpd''' with the '''+ssl'''.
+
+You'll also want the lighttpd service to run at all times so your client machines are not deprived of access to the port archives. A simple LaunchD task will handle this:
+
+'''/Library/LaunchDaemons/org.macports.share-archives.plist'''
+{{{
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>KeepAlive</key>
+    <true/>
+    <key>Label</key>
+    <string>org.macports.share-archives</string>
+    <key>LowPriorityIO</key>
+    <true/>
+    <key>Nice</key>
+    <integer>1</integer>
+    <key>ProgramArguments</key>
+    <array>
+        <string>/opt/local/sbin/lighttpd</string>
+        <string>-D</string>
+        <string>-f</string>
+        <string>/usr/local/share/macports/macports-archives-lighttpd.conf</string>
+    </array>
+    <key>StandardErrorPath</key>
+    <string>/var/log/macports-archives-lighttpd.out</string>
+    <key>StandardOutPath</key>
+    <string>/var/log/macports-archives-lighttpd.out</string>
+</dict>
+</plist>
+}}}
+
+Make sure your test server has been killed and then load the task with:
+{{{
+launchctl load /Library/LaunchDaemons/org.macports.share-archives.plist
+}}}
+
+Ensure that the server is running again by browsing the page as before.
+
+=== Step 4: '''Fetch archives''' ===
+
+Finally we're ready to actually fetch some archives.
+
+First, copy the public key to each client system that will be installing from archives. As before, '''/usr/local/share/macports/''' is a decent enough location.
+
+Now, MacPorts needs to be configured to fetch archives from the build system. The '''archive_site_local''' setting can be set to an IP, but it will likely be more convenient to use the Bonjour name of the local system. This name can be found in the '''Sharing''' preference pane.
+
+Add the following to '''${prefix}/etc/macports/macports.conf'''
+{{{
+archive_site_local      http://bonjour.local:6227/
+}}}
+
+A line indicating the location of the public key must also be added to '''${prefix}/etc/macports/pubkeys.conf'''. Something like:
+{{{
+/usr/local/share/macports/pubkey.pem
+}}}
+
+
+And you're done! You should now be able to compile ports once on your build system and install on as many client systems as you like (as long as the requested variants and architectures are the same). 
+
+
+[wiki:howto <- Back to the HOWTO section]