=== Intro ===

The MacPorts testing framework uses tcltest ![0] for its unit tests as well as regression tests.
Maintainer: marius [at] macports.org

=== Running tests ===

The easiest way to run all the tests, is to use the target in the Makefile.
* make test

Each 'tests/' directory has a 'test.tcl' file, used by the make target to run all tests and format the output, making it easy to read.
The file can be used also to:
* run all tests:                    ' tclsh test.tcl '
* get debug info:               ' tclsh test.tcl -debug \[0-3\] '
* list individual test files:   ' tclsh test.tcl -l '
* run specific test files:      ' tclsh test.tcl -t macports.test '
* print help message:        ' tclsh test.tcl -h '

Specific test cases can be run using the '-match' argument for the file that contains the test, from its parent directory.
* tclsh macports.test -match mportclose

Regression tests can be found in ' trunk/base/tests/ ' and can be  run just as unit tests.


=== Must know ===

* regression tests have their own directory, found in ' trunk/base/tests/ '
* each module of MacPorts (port1.0, macports1.0, package1.0) has its own ‘tests/’ directory where the test files are located and also additional files needed (Portfile, test.tcl)
* each file in a module has a corresponding test file (.test extension) in the ‘tests/’ directory
* each proc in a file has a corresponding test case (test proc_name) in the
* each test case must be independent from each other, so they can be run individually if needed
* each test must clan all auxiliary files or directories it creates and revert all port it installs
* use a single test proceduce for each tested proc; sub-test cases should be included in the same body
* when adding new regression tests, make sure to specify its name in the test_suite list of ' trunk/base/tests/test.tcl '


=== Sample file ===

{{{
# include required tcltest package and set namespace
package require tcltest 2
namespace import tcltest::*

# get absolute path to current ‘tests/’ directory
set pwd [file normalize $argv0]
set pwd [eval file join {*}[lrange [file split $pwd] 0 end-1]]

# debug options
# ports_debug and ports_verbose are commented out as default
# need to be set before ‘mportinit’
array set ui_options {}
#set ui_options(ports_debug)   yes
#set ui_options(ports_verbose) yes
mportinit ui_options

# source/require tested/needed files
source ../../port1.0/portutil.tcl

# additional procs needed for testing
proc registry_exists {name version {revision 0} {variants ""}} {
        global macports::registry.format
        return [${macports::registry.format}::entry_exists $name $version $revision $variants]
}


# test case example
# the test name must reflect the tested proc (remove namespaces if any)
# the test description should list specific values from the tested proc on which it depends
# or the partial cases it tests
test mportclose {
    Mport close unit test.
# this branch is optional and you can use other constraints too
} -constraints {
    root
# the setup branch is optional
} -setup {
    set mport [mportopen file://.]
# please make output as useful as possible (even error cases)
# all sub-test cases should be part of the body branch
} -body {
    if {[catch {mportclose $mport}] != 0} {
        return "FAIL: cannot run mportclose"
    }
    return "Mport close successful."
# the cleanup branch is optional
} -cleanup {
    file delete -force $pwd/work
} -result "Mport close successful."


# print test results
cleanupTests
}}}


=== Additional files ===

* In all tests we use this [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/Portfile | Portfile]].
* This is the [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/test.tcl | test.tcl]] file used to run and parse the output of all the tests in a module.
* A worker_init copy, without using sub-interpreters [[https://trac.macports.org/browser/branches/gsoc13-tests/src/package1.0/tests/library.tcl | library.tcl ]].
* Example of [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/sources.conf | sources.conf ]].
* An example of a test file [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/macports.test | macports.test ]].
* A [[https://trac.macports.org/browser/branches/gsoc13-tests/tests/test/library.tcl | library.tcl]] of useful procs in regression testing.


=== Resources ===

![0] - [[http://wiki.tcl.tk/1502 | Tcltest official wiki page]]\\
![1] - [[http://web.archive.org/web/20080617153002/www.tclscripting.com/articles/apr06/article1.html | Getting started with tcltest]]\\
![2] - [[http://www.tcl.tk/man/tcl8.5/TclCmd/tcltest.htm | Official tcltest documentation]]\\