Changes between Version 10 and Version 11 of Tests


Ignore:
Timestamp:
Sep 8, 2013, 8:19:42 PM (11 years ago)
Author:
ryandesign (Ryan Carsten Schmidt)
Comment:

use TracLinks; simplify markup; fix typos

Legend:

Unmodified
Added
Removed
Modified
  • Tests

    v10 v11  
    11=== Intro ===
    22
    3 The MacPorts testing framework uses [[http://wiki.tcl.tk/1502 | tcltest]] for its unit tests as well as regression tests. The framework has been developed during GSoC 13 by Marius Coțofană.
     3The MacPorts testing framework uses [http://wiki.tcl.tk/1502 tcltest] for its unit tests as well as regression tests. The framework has been developed during GSoC 13 by Marius Coțofană.
    44
    5 To keep things simple, each module of MacPorts ([[https://trac.macports.org/browser/trunk/base/src/macports1.0 | macports1.0]], [[https://trac.macports.org/browser/trunk/base/src/package1.0 | package1.0]], [[https://trac.macports.org/browser/trunk/base/src/port1.0 | port1.0]], [[https://trac.macports.org/browser/trunk/base/src/registry2.0 | registry2.0]]) has it's own 'tests/' directory. Each Tcl script in a module (e.g. [[https://trac.macports.org/browser/trunk/base/src/macports1.0/macports.tcl | macports.tcl]]) has its own test script located in the 'test' directory, with the same name and the '.test' extension (e.g. [[https://trac.macports.org/browser/trunk/base/src/macports1.0/tests/macports.test | macports.test]]). Every proc in a script (e.g. [[https://trac.macports.org/browser/trunk/base/src/macports1.0/macports.tcl#L334 | proc macports::findBinary]]) should have it's own test proc (e.g. [[https://trac.macports.org/browser/trunk/base/src/macports1.0/tests/macports.test#L334 | test findBinary]]) in the corresponding test file. Test procs should maintain the order in the original script and should be independent one of another.
     5To keep things simple, each module of MacPorts ([browser:trunk/base/src/macports1.0 macports1.0], [browser:trunk/base/src/package1.0 package1.0], [browser:trunk/base/src/port1.0 port1.0], [browser:trunk/base/src/registry2.0 registry2.0]) has its own 'tests/' directory. Each Tcl script in a module (e.g. [browser:trunk/base/src/macports1.0/macports.tcl macports.tcl]) has its own test script located in the 'test' directory, with the same name and the '.test' extension (e.g. [browser:trunk/base/src/macports1.0/tests/macports.test macports.test]). Every proc in a script (e.g. [browser:trunk/base/src/macports1.0/macports.tcl#L334 proc macports::findBinary]) should have its own test proc (e.g. [browser:trunk/base/src/macports1.0/tests/macports.test#L334 test findBinary]) in the corresponding test file. Test procs should maintain the order in the original script and should be independent one of another.
    66
    77Maintainer: marius [at] macports.org
     
    1414}}}
    1515
    16 Each 'tests/' directory has a [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/test.tcl | test.tcl]] file, used by the make target to run all tests and format the output, making it easy to read. The script just runs the tests individually, printing the test file name, the total number of tests, number of passed, skipped, failed as well as constraints or errors of failed tests. This is one possible output when running macports.test:
     16Each 'tests/' directory has a [browser:branches/gsoc13-tests/src/macports1.0/tests/test.tcl test.tcl] file, used by the make target to run all tests and format the output, making it easy to read. The script just runs the tests individually, printing the test file name, the total number of tests, number of passed, skipped, failed as well as constraints or errors of failed tests. This is one possible output when running macports.test:
    1717{{{
    1818Total:31 Passed:31 Failed:00 Skipped:00  macports.test
    1919}}}
    2020
    21 Many tests need root privileges to run correctly, but will be auto skipped in the other case. Constarints are printed just bellow the final result, together with the number of test cases that require it, as so:
     21Many tests need root privileges to run correctly, but will be auto skipped in the other case. Constraints are printed just below the final result, together with the number of test cases that require it, as so:
    2222{{{
    2323Total:31 Passed:24 Failed:00 Skipped:07  macports.test
     
    5454}}}
    5555
    56 Regression tests can be found in [[https://trac.macports.org/browser/trunk/base/tests/test | trunk/base/tests/test]] and can be  run just as unit tests, using 'make test' from the parent directory.
     56Regression tests can be found in [browser:trunk/base/tests/test trunk/base/tests/test] and can be  run just as unit tests, using 'make test' from the parent directory.
    5757
    5858
    5959=== Must know ===
    6060
    61 * the tests can be run only on an installed version of Macports ( so make sure you have run 'sudo make install' )
    62 * regression tests have their own directory, found in [[https://trac.macports.org/browser/trunk/base/tests/test | trunk/base/tests/test]]
    63 * each module of MacPorts (port1.0, macports1.0, package1.0) has its own ‘tests/’ directory where the test files are located and also additional files needed ([[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/Portfile | Portfile]], [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/test.tcl | test.tcl]])
     61* the tests can be run only on an installed version of MacPorts (so make sure you have run 'sudo make install')
     62* regression tests have their own directory, found in [browser:trunk/base/tests/test trunk/base/tests/test]
     63* each module of MacPorts (port1.0, macports1.0, package1.0) has its own ‘tests/’ directory where the test files are located and also additional files needed ([browser:branches/gsoc13-tests/src/macports1.0/tests/Portfile Portfile], [browser:branches/gsoc13-tests/src/macports1.0/tests/test.tcl test.tcl])
    6464* each file in a module has a corresponding test file (.test extension) in the ‘tests/’ directory
    6565* each proc in a file has a corresponding test case (test proc_name) in the
    6666* each test case must be independent from each other, so they can be run individually if needed
    67 * each test must clan all auxiliary files or directories it creates and revert all port it installs
     67* each test must clean all auxiliary files or directories it creates and revert all ports it installs
    6868* use a single test proceduce for each tested proc; sub-test cases should be included in the same body
    69 * when adding new regression tests, make sure to specify its name in the test_suite list of ' trunk/base/tests/test.tcl '
     69* when adding new regression tests, make sure to specify its name in the test_suite list of 'trunk/base/tests/test.tcl'
    7070
    7171
     
    155155=== Additional files ===
    156156
    157 * In all tests we use this [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/Portfile | Portfile]].
    158 * This is the [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/test.tcl | test.tcl]] file used to run and parse the output of all the tests in a module.
    159 * A worker_init copy, without using sub-interpreters [[https://trac.macports.org/browser/branches/gsoc13-tests/src/package1.0/tests/library.tcl | library.tcl ]].
    160 * Example of [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/sources.conf | sources.conf ]].
    161 * An example of a test file [[https://trac.macports.org/browser/branches/gsoc13-tests/src/macports1.0/tests/macports.test | macports.test ]].
    162 * A [[https://trac.macports.org/browser/branches/gsoc13-tests/tests/test/library.tcl | library.tcl]] of useful procs in regression testing.
     157* In all tests we use this [browser:branches/gsoc13-tests/src/macports1.0/tests/Portfile Portfile].
     158* This is the [browser:branches/gsoc13-tests/src/macports1.0/tests/test.tcl test.tcl] file used to run and parse the output of all the tests in a module.
     159* A worker_init copy, without using sub-interpreters [browser:branches/gsoc13-tests/src/package1.0/tests/library.tcl library.tcl].
     160* Example of [browser:branches/gsoc13-tests/src/macports1.0/tests/sources.conf sources.conf].
     161* An example of a test file [browser:branches/gsoc13-tests/src/macports1.0/tests/macports.test macports.test].
     162* A [browser:branches/gsoc13-tests/tests/test/library.tcl library.tcl] of useful procs in regression testing.
    163163
    164164
    165165=== Resources ===
    166166
    167 * [[http://wiki.tcl.tk/1502 | Tcltest official wiki page]]\\
    168 * [[http://web.archive.org/web/20080617153002/www.tclscripting.com/articles/apr06/article1.html | Getting started with tcltest]]\\
    169 * [[http://www.tcl.tk/man/tcl8.5/TclCmd/tcltest.htm | Official tcltest documentation]]\\
     167* [http://wiki.tcl.tk/1502 Tcltest official wiki page]
     168* [http://web.archive.org/web/20080617153002/www.tclscripting.com/articles/apr06/article1.html Getting started with tcltest]
     169* [http://www.tcl.tk/man/tcl8.5/TclCmd/tcltest.htm Official tcltest documentation]