=== Intro ===

The MacPorts testing framework uses [[http://wiki.tcl.tk/1502 | tcltest]] for its unit tests as well as regression tests.
Maintainer: marius [at] macports.org

=== Running tests ===

The easiest way to run all the tests, is to use the target in the Makefile.
* make test

Each 'tests/' directory has a [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/test.tcl | test.tcl]] file, used by the make target to run all tests and format the output, making it easy to read.
The file can be used also to:
* run all tests:                ' tclsh test.tcl '
* get debug info:               ' tclsh test.tcl -debug \[0-3\] '
* list individual test files:   ' tclsh test.tcl -l '
* run specific test files:      ' tclsh test.tcl -t macports.test '
* print help message:        ' tclsh test.tcl -h '

Specific test cases can be run using the '-match' argument for the file that contains the test, from its parent directory.
* tclsh macports.test -match mportclose

Regression tests can be found in ' trunk/base/tests/ ' and can be  run just as unit tests.


=== Must know ===

* regression tests have their own directory, found in ' trunk/base/tests/ '
* each module of MacPorts (port1.0, macports1.0, package1.0) has its own ‘tests/’ directory where the test files are located and also additional files needed ([[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/Portfile | Portfile]], [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/test.tcl | test.tcl]])
* each file in a module has a corresponding test file (.test extension) in the ‘tests/’ directory
* each proc in a file has a corresponding test case (test proc_name) in the
* each test case must be independent from each other, so they can be run individually if needed
* each test must clan all auxiliary files or directories it creates and revert all port it installs
* use a single test proceduce for each tested proc; sub-test cases should be included in the same body
* when adding new regression tests, make sure to specify its name in the test_suite list of ' trunk/base/tests/test.tcl '


=== Sample file ===

{{{
# include required tcltest package and set namespace
package require tcltest 2
namespace import tcltest::*

# get absolute path to current ‘tests/’ directory
set pwd [file normalize $argv0]
set pwd [eval file join {*}[lrange [file split $pwd] 0 end-1]]

# debug options
# ports_debug and ports_verbose are commented out as default
# need to be set before ‘mportinit’
array set ui_options {}
#set ui_options(ports_debug)   yes
#set ui_options(ports_verbose) yes
mportinit ui_options

# source/require tested/needed files
# source ../../port1.0/portutil.tcl
package require portutil 1.0

# use custom macports.conf and sources.conf
# you need to provide the sources.conf (see additional files) file
makeDirectory $pwd/tmpdir
makeDirectory $pwd/tmpdir/share
makeDirectory $pwd/tmpdir/var/macports/registry
set fd [open $pwd/tmpdir/macports.conf w+]
puts $fd "portdbpath $pwd/tmpdir/var/macports"
puts $fd "prefix $pwd/tmpdir"
puts $fd "variants_conf $pwd/tmpdir/variants.conf"
puts $fd "sources_conf $pwd/sources.conf"
puts $fd "applications_dir $pwd/tmpdir/Applications"
puts $fd "frameworks_dir $pwd/tmpdir/Library/Frameworks"
close $fd
set env(PORTSRC) $pwd/tmpdir/macports.conf
file link -symbolic $pwd/tmpdir/share/macports $macports::autoconf::prefix/share/macports
close [open $pwd/tmpdir/variants.conf w+]

# if you need to use procs from macports namespace, that are just aliases, you can
# always source library.tcl (see additional files) which provides a copy macports::worker_init
# without sub-interpreters; it also sets some important environment variables like
# os.platform, os.major, os.arch, workpath, destpath, portpath
# some other option would be to get the $workername from a $mport and use it directly

# additional procs needed for testing go before the actual test cases


# test case example
# the test name must reflect the tested proc (remove namespaces if any)
# the test description should list specific values from the tested proc on which it depends
# or the partial cases it tests
test mportclose {
    Mport close unit test.
# this branch is optional and you can use other constraints too
} -constraints {
    root
# the setup branch is optional
} -setup {
    set mport [mportopen file://.]
# please make output as useful as possible (even error cases)
# all sub-test cases should be part of the body branch
} -body {
    if {[catch {mportclose $mport}] != 0} {
        return "FAIL: cannot run mportclose"
    }
    return "Mport close successful."
# the cleanup branch is optional
} -cleanup {
    file delete -force $pwd/work
} -result "Mport close successful."


# print test results
cleanupTests
}}}


=== Additional files ===

* In all tests we use this [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/Portfile | Portfile]].
* This is the [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/test.tcl | test.tcl]] file used to run and parse the output of all the tests in a module.
* A worker_init copy, without using sub-interpreters [[https://trac.macports.org/browser/branches/gsoc13-tests/src/package1.0/tests/library.tcl | library.tcl ]].
* Example of [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/sources.conf | sources.conf ]].
* An example of a test file [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/macports.test | macports.test ]].
* A [[https://trac.macports.org/browser/branches/gsoc13-tests/tests/test/library.tcl | library.tcl]] of useful procs in regression testing.


=== Resources ===

* [[http://wiki.tcl.tk/1502 | Tcltest official wiki page]]\\
* [[http://web.archive.org/web/20080617153002/www.tclscripting.com/articles/apr06/article1.html | Getting started with tcltest]]\\
* [[http://www.tcl.tk/man/tcl8.5/TclCmd/tcltest.htm | Official tcltest documentation]]\\