The lisp-unit2 Reference Manual

Table of Contents

The lisp-unit2 Reference Manual

This is the lisp-unit2 Reference Manual, version 0.2.0, generated automatically by Declt version 2.4 "Will Decker" on Wed Jun 20 12:12:00 2018 GMT+0.

1 Introduction

lisp-unit2

lisp-unit2 is a Common Lisp library that supports unit testing. It is a new version of a library of the lisp-unit library written by Chris Riesbeck. There is a long history of testing packages in Lisp, usually called "regression" testers. More recent packages in Lisp and other languages have been inspired by JUnit for Java.

Recently longtime users at Acceleration.net felt motivated to refactor significantly, attempting to make some broad improvements to the library while maintaining its benefits and workflow

Features

• Written in portable Common Lisp
• Loadable with ASDF or Quicklisp
• Simple to define and run tests
• Redefine functions and macros without reloading tests - tests are recompiled before each run and at definition time
• Tests have source-location for going to definition
• Supports testing: return values, printed output, macro expansions, and conditions. Special support for testing floating and rational point numbers
• Running a single test returns a test-result object. Running many tests returns a test-result-db
• Tests grouped and accessed by name, tag, and package of test name
• Signals for starting and completing test runs (both individually and as a group)

Differences from lisp-unit version 1

• Simplified test retrieval / categorization.
• Tests are stored by their name's symbol-package (easing the complexity of this package vs the package argument)
• package arguments now just retrieve all tests in a given package
• dynamic variable consolidation (eg: one test database with three hashtables instead of three dynamic variables containing hashtables)
• All output goes to its own stream (and can therefore be manipulated at will). test-stream defaults to standard-output as it always has.
• Debugging is controlled by debug-hook, continue restart activated around test
• Tests are named functions that show up on the stack (so that goto-definition in a stack trace brings you to the test definition
• Contexts can be applied on a test or a run-test(s) to provide common setup/tear down locations (eg: provide a database to tests that need it). Contexts are functions that accept a thunk and apply it inside of a dynamic context eg: (lambda (body-thunk) (let ((*var* 1))(funcall body-thunk)))
• Signals used throughout (including to drive current output summaries)
• assertions passing and failing (with abort restart, to cancel assertions)
• tests starting and completing (with continue restart for resuming test run after an errors)
• test batches starting and completing
• Dynamic variables available in signal handlers and all tests unit-test and results
• Logging used throughout (to ease debugging of lisp-unit2) - compile time
• TAP output displays better messages about the error and is more consistent with the summary output
• Better job of reporting compiler warnings and errors: when defining the test (and while running it)
• Summary output prints test starting messages. Compiler / run-time errors printed into the output is obviously tied to the test in question. (Previously the results printed after the error messages and so it was textually ambiguous to which test the messages applied)
• lisp-unit2 is no longer loadable as a single file
• :lisp-unit2 feature is available for conditional compilation
• Better optimized for running both in a build-environment (jenkins etc) and from the REPL (previously there were many non-standard boolean flags for controlling output and debugging).
• test bodies compiled with (debug 3) - TODO: revisit this in light of slow tests if that becomes an issue
• Default ordering for tests and results is the definition order (instead of random or reversed)
• assert-result -> assert-passes?
• (define-test name &body) became (define-test name (&key) &body)

How to use lisp-unit2

1. Load using Quicklisp : (ql:quickload :lisp-unit2) or ASDF : (asdf:load-system :lisp-unit2).
2. Define some tests (for best luck define tests in their own package by making their name be in a specific package). By having tests in their own package, the test and the fn being tested can share the same name. (Tests are compiled to a function named after the test that runs it and an object in the test database)

(lisp-unit2:define-test my-tests::test-subtraction 
    (:tags '(my-tests::bar))
  (assert-eql 1 (- 2 1)))

3. Run all tests in your package

;; with-summary provides results while the tests are running
;;;; using the context function
(run-tests :package :my-tests
           :run-contexts #'with-summary-context)

;;;; using the context macro (does the same thing as
;;;; :run-contexts, See Contexts below for an explanation).
(with-summary () (run-tests :package :my-tests))

;; to print the results after the run is complete
(print-summary (run-tests :tags 'my-tests::bar))

;; The difference is in whether or not the output occurs while the
;; tests are running or after all the tests have run

;; to disable the debugger:
(let (*debugger-hook*)
  (run-tests :tests 'my-tests::test-subtraction))

;; to debug failed assertions with the context function
(run-tests :tests 'my-tests::test-subtraction
           :run-contexts #'with-failure-debugging-context)

;; or use the context macro
(with-failure-debugging ()
  (run-tests :tests 'my-tests::test-subtraction))

See the internal test suite for more and better examples (internal-test/*)

Defining Tests

The define-test macro is the primary way to install a test. define-test creates a function with the same name as the test, which can be called to execute the test. This is nice because there is a direct way to call every test, but also because the test has implicit source destination and thus "go-to-definition" on any printing of the test name will take you directly to the test in question. define-test also creates a unit-test object and inserts it into the test-database.

The test body is compiled at the time of definition (so that any compile warnings or errors are immediately noticeable) and also before every run of the test (so that macro expansions are never out of date).

• package: if package is provided the test name and all tags are reinterned into that package before definition. This is mostly for porting lisp-unit systems where test-names were not functions and so tests could be named the same as the function they tested.
• contexts (see Contexts below): a (single or tree of) context fn that will be applied around the test body
• tags: a list of symbols used to organize tests into groups (not hierarchical)

Undefining Tests

Because lisp-unit2 keeps a database of tests and functions, removing a tests is not as simple as deleteing the test from from your file. Instead a function uninstall-test and a macro undefine-test allow you to remove all the runtime components of a test from lisp-unit2. Uninstall-test accepts the test's symbolic name. Undefine-test is a macro whose form mimics define-test, so that you can simply add the two characters and compile the form to remove the test. This also gives you the ability to easily leave tests in your test file that are inactive currently.

Running Tests

The primary entry point for running tests is lisp-unit2:run-tests. run-tests and get-tests both accept (&key tests tags package reintern-package) as discussed below in "Test Organization". Each test can also be run individually by calling the function named for the test, and by calling lisp-unit2:run-test which each return a single test-result.

run-tests returns a test-results-db, and aside from the keys above accepts,

• run-contexts: a (single, or list of) context that will be applied around the entire body of running the tests
• test-contexts: a (single, or list of) context that will be applied around each tests body. This will be around any contexts defined on the unit test
• name: a name used for this test run. Useful for identifying what defaults to all packages being tested. We attempt to make a good default if nothing is provided

Once you have a test-result-db (or list there of) you can call rerun-tests or rerun-failures to rerun and produce new results.

Test Organization: Names, Tags, and Packages

Tests are organized by their name and by tags. Both of these are symbols in some package. Tests can be retrieved by their name, the package that their name is in, and any of the tags that reference the test.

The most common way to retrieve and run unit tests is run-tests which calls get-tests.

(lisp-unit2:run-tests &key tests tags package reintern-package) (lisp-unit2:get-tests &key tests tags package reintern-package)

Both of these functions accept:

• tests: a single or list of symbols or unit-test objects
• tags: a single or list of symbols. All tests in these tags will be returned
• package: a single or list of packages (names or objects)

If no arguments are provided lisp-unit2 will run all tests in package

In some cases, particularly when converting from lisp-unit(1) we need our tests to be in a different package (because tests are functions in their name's package). In lisp-unit these tests would not conflict with a function named the same (because tests were not functions). To ease conversion, the reintern-package argument will reintern all test names and tags provided into a different package. Define test accepts a package argument that mirrors this functionality. Suggested usage is to either have tests be named differently from the functions they test or to have tests and tags be in an explicitly referenced package, eg: (define-test my-tests::test1 (:tags '(my-tests::tag1)) ...)

Tests are organized into the *test-db* which is an instance test-database. These can be rebound if you needed to write tests about your test framework (see the internal example-tests).

Suites

While lisp-unit does not have any specific notion of a suite, it is believed that the tests are composable enough that explicit test suites are not needed.

One suggestion would be to have named functions that run you specific set of tests:

(defun run-{suite} ()
  (lisp-unit2:run-tests
    :name :{suite}
    {stuff to test} ))

(defun run-symbol-munger-and-lisp-unit2-tests () 
  (lisp-unit2:run-tests
    :name :symbol-munger-and-lisp-unit2-tests
    :package '(:symbol-munger-tests :lisp-unit2-tests)))

(defun run-error-and-basic-tests () 
  (lisp-unit2:run-tests
    :name :error-and-basic-tests
    :tests '(symbol-munger-test::test-basic)
    :tags '(:errors lisp-unit2-tests::errors)))

Another suggestion would be to define tests that call other tests:

(define-test suite-1 (:tags '(suites))
  (test-fn-1) ;; calls test-fn-1 unit-test
  (lisp-unit2::run-tests ...) ; runs an entire other set of unit tests)

Assertions

All assert-* macros signal assertion-pass and assertion-fail by comparing their expected results to the actual results during execution. All other values in the assert forms are assumed to be extra data to aid debugging.

assert-{equality-test} macros compare the actual to the expected value of each of the values returned from expected (see: multiple-values) (eg: (assert-eql exp act) => (eql act exp)) This ordering was used so that functions like typep, member, etc could be used as test.

assert-false and assert-true simply check the generic-boolean truth of their first arg (eg: null and (not null)

assert-{signal} and assert-{no-signal} macros are used for testing condition protocols. Signals that are expected/not-expected handled.

Multiple-values and assertions

Actual values are compared to all expected values, that is:

LISP-UNIT2-TESTS> (lisp-unit2:with-assertion-summary ()
                    (assert-eql (values 1 2) (values 1 2 3)))


<No Test>:Passed (ASSERT-EQL (VALUES 1 2) (VALUES 1 2 3))
T
LISP-UNIT2-TESTS> (lisp-unit2:with-assertion-summary ()
                    (assert-eql (values 1 2 3) (values 1 2)))

Failed Form: (ASSERT-EQL (VALUES 1 2 3) (VALUES 1 2))
Expected 1; 2; 3
but saw 1; 2

Debugging

Debugging is controlled by *debugger-hook* (as is usual in common-lisp). You can make lisp-unit simply record the error and move on by binding *debugger-hook* to nil around your run-tests call.

If you would like to debug failed assertions you can wrap your call in with-failure-debugging or apply the with-failure-debugging-context to the unit-test run.

Output and Results

All output is printed to *test-stream* (which by default is *standard-output*). Most forms do not output results by default, instead returning a result object. All results objects can be printed (to *test-stream*) by calling print-summary on the object in question.

print-summary prints information about passing as well as failing tests. print-failure-summary can be called to print only messages about failures, warnings, errors and empty tests (empty tests had no assertions). Care is taken to print tests with their short, but still fully packaged symbol-name (so that go-to-definition) works.

When running interactively with-summary(-context) can provide real-time output, printing start messages and result messages for each test. with-assertion-summary(-context) provides even more detailed output printing a message for each assertion passed or failed.

Test results (from one or many runs) can be captured using with-test-results. The arg: collection-place will copy all the results as they arrive into a location of your choosing (eg: variable, object slot). The arg: summarize? will print a failure summary of each test-run after all of the tests are finished running. This is useful for collecting separate results for many packages or systems (see test-asdf-system-recursive). If no args are provided summarize? is defaulted to true.

TAP Output

Lisp-unit2 provides TAP (test anything protocol) test results (printed in such a way that jenkins tap plugin can parse them).

with-tap-summary prints tap results as the tests run write-tap, write-tap-to-file accept a test-results database and write the TAP results either to test-stream or a file

ASDF

asdf:test-system is assumed to be the canonical way of testing a every system, and so lisp-unit2 makes effort to work well with test-system

Here is an example asdf:test-sytem definition, which will print the verbose summary of the tests as they run.

(defmethod asdf:perform ((o asdf:test-op) (c (eql (find-system :symbol-munger))))
  (asdf:oos 'asdf:load-op :symbol-munger-test)
  (let ((*package* (find-package :symbol-munger-test)))
    (eval (read-from-string "
            (lisp-unit2:with-summary ()
             (lisp-unit2:run-tests
              :package :symbol-munger-test
              :name :symbol-munger
              :run-context))
      "))))

Additionally, lisp-unit2 provides test-asdf-system-recursive which accepts a (list or single) system name, looks up all its dependencies and calls asdf:tests-system on each listed system. Any lisp-unit or lisp-unit2 test-results-dbs are collected and returned at the end. A failure summary is also printed for each result db (so that after running many tests you are presented with a short synopsis of what ran and what failed.

There is an interop layer for converting test results from other systems to lisp-unit2 test results, so that we can gather and summarize more information. Currently this is only implemented for lisp-unit1, but patches would be welcome to allow collecting test results from any other lisp test systems. (This is currently a bit tedious, simplification patches welcome as well, see interop.lisp).

Signals

Signals are used throughout lisp-unit2 to communicate progress and results throughout lisp unit

• missing-test: when asked to run a test that does not exist
• test-start: when a unit test starts, contains a ref to the unit-test
• test-complete: when a test completes contains a ref to results for that test (test-result)
• all-tests-start, all-tests-complete: signaled at the beginning and end of run-tests, contains a ref to the test-results-db
• collect-test-results: used in it interop to send results to with-test-results
• assertion-pass: signaled when an assertion passes
• assertion-fail: signaled when an assertion fails

All signals have an abort interrupt which simply cancels the signal. This is mostly used for meta-testing (ie: testing lisp-unit2), but there are conceivably other uses.

Contexts / Fixtures

Contexts allow you to manipulate the dynamic state of a given unit-test or test-run. These are functions of a single required argument (a thunk), that they execute inside of a new/changed dynamic environment.

A good example is the with-failure-debugging-context, which simply invokes the debugger whenever we signal a failed assertion

(defun with-failure-debugging-context (body-fn)
  "A context that invokes the debugger on failed assertions"
  (handler-bind ((assertion-fail #'invoke-debugger))
    (funcall body-fn)))

Both define-test and run-tests accept a tree of contexts that is flattened and turned into a single context function (see combine-contexts). This function then executes the body-fn for us (see: do-contexts).

This should allow any type of manipulation of the current lisp-unit2 environment through access to handling signals and setting up and tearing down dynamic environments.

Example Contexts:

• rolled-back-database-context: open a connection to the dev database and a transaction that we will abort at the end
• http-context: establish a dummy http context for use in testing web connected things
• html-output-context: establish global html building objects needed for html output functions to work
• dev-mode-context: bind any dynamic variables related to DEV/LIVE mode
• failure-debugging-context: (as above) handling signals and debugging them instead of only recording them

Example test-with-contexts defintion:

(defmacro db-render-test (name (&rest args) &body body)
  `(lisp-unit2:define-test ,name
    (:tags ',args
     :contexts
     (list #'test-context #'dom-context #'database-context )
     :package :test-objects)
    ,@body))

Data Model

• test-database: a list and some indexes of all installed tests
• unit-test: an object containing the name, code, contexts, and documentation for a single unit-test
• test-results-db: a collection of results from a single run-tests call
• test-result: the result of running a single unit test
• failure-result: information useful for debugging failures (eg: unit-test, the form that failed, the expected and actual results)

Remaining Tasks

• Expanded internal testing.

Future Features

• Benchmarking tools
• More interop with other test systems
• Database layer to store test results to a database (so that we can track bugs / test durations) over time (goes with benchmarking)

0.2.0 Acknowledgments

• Russ Tyndall - Acceleration.net

0.9.5 Acknowledgments

• [Jesse Alama][jessealama] for usability feedback. [jessealama]: https://github.com/jessealama "Jesse Alama"

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 lisp-unit2

Author

Russ Tyndall <russ@acceleration.net>

License

MIT

Description

Common Lisp library that supports unit testing.

Version

0.2.0

Dependencies

• alexandria
• cl-interpol
• iterate
• symbol-munger

Source

lisp-unit2.asd (file)

Components

• package.lisp (file)
• assert-package.lisp (file)
• vars.lisp (file)
• collectors.lisp (file)
• asserts.lisp (file)
• lisp-unit.lisp (file)
• summarize.lisp (file)
• rational.lisp (file)
• floating-point.lisp (file)
• test-anything-protocol.lisp (file)
• interop.lisp (file)

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

3.1 Lisp

3.1.1 lisp-unit2.asd

Location

lisp-unit2.asd

Systems

lisp-unit2 (system)

3.1.2 lisp-unit2/package.lisp

Parent

lisp-unit2 (system)

Location

package.lisp

Packages

lisp-unit2

3.1.3 lisp-unit2/assert-package.lisp

Dependency

package.lisp (file)

Parent

lisp-unit2 (system)

Location

assert-package.lisp

Packages

lisp-unit2-asserts

3.1.4 lisp-unit2/vars.lisp

Dependency

assert-package.lisp (file)

Parent

lisp-unit2 (system)

Location

vars.lisp

Exported Definitions

• *result* (special variable)
• *results* (special variable)
• *test-db* (special variable)
• *test-stream* (special variable)
• *unit-test* (special variable)
• all-tests-complete (condition)
• all-tests-start (condition)
• collect-test-results (condition)
• missing-test (condition)
• results (method)
• (setf results) (method)
• results (method)
• (setf results) (method)
• results (method)
• (setf results) (method)
• test-complete (condition)
• test-start (condition)
• unit-test (method)
• (setf unit-test) (method)

Internal Definitions

• *log-level* (special variable)
• *test-log-stream* (special variable)
• +statuses+ (special variable)
• name (method)
• (setf name) (method)
• result (method)
• (setf result) (method)
• test-name (method)
• (setf test-name) (method)

3.1.5 lisp-unit2/collectors.lisp

Dependency

vars.lisp (file)

Parent

lisp-unit2 (system)

Location

collectors.lisp

Internal Definitions

• %collect (generic function)
• %collect (method)
• %collect (method)
• %collect! (macro)
• %collect-new (generic function)
• %collect-new (method)
• %collect-new (method)
• %collect-new! (macro)
• %decollect (generic function)
• %decollect (method)
• %decollect (method)
• %decollect! (macro)
• head (generic function)
• head (method)
• head (method)
• (setf head) (method)
• len (generic function)
• len (method)
• len (method)
• (setf len) (method)
• list-collector (class)
• tail (generic function)
• tail (method)
• tail (method)
• (setf tail) (method)

3.1.6 lisp-unit2/asserts.lisp

Dependency

collectors.lisp (file)

Parent

lisp-unit2 (system)

Location

asserts.lisp

Exported Definitions

• assert-eq (macro)
• assert-eql (macro)
• assert-equal (macro)
• assert-equality (macro)
• assert-equalp (macro)
• assert-error (macro)
• assert-expands (macro)
• assert-false (macro)
• assert-no-error (macro)
• assert-no-signal (macro)
• assert-no-warning (macro)
• assert-prints (macro)
• assert-signal (macro)
• assert-true (macro)
• assert-typep (macro)
• assert-warning (macro)
• assertion-fail (condition)
• assertion-pass (condition)
• failure-result (class)
• logically-equal (function)
• set-equal (function)
• unit-test (method)
• (setf unit-test) (method)
• unit-test (method)
• (setf unit-test) (method)
• unit-test (method)
• (setf unit-test) (method)
• with-failure-debugging (macro)
• with-failure-debugging-context (function)

Internal Definitions

• %form-equal (function)
• actual (method)
• (setf actual) (method)
• assert-passes? (generic function)
• assert-passes? (method)
• assertion (method)
• (setf assertion) (method)
• assertion (method)
• (setf assertion) (method)
• equal-result (class)
• error-result (class)
• expand-assert (macro)
• expand-extras (macro)
• expand-macro-form (macro)
• expand-output-form (macro)
• expand-signaled-handler (function)
• expected (method)
• (setf expected) (method)
• extras (method)
• (setf extras) (method)
• failure (method)
• (setf failure) (method)
• form (method)
• (setf form) (method)
• internal-assert (function)
• macro-result (class)
• output-result (class)
• record-failure (generic function)
• record-failure (method)
• signal-result (class)
• test (method)
• (setf test) (method)

3.1.7 lisp-unit2/lisp-unit.lisp

Dependency

asserts.lisp (file)

Parent

lisp-unit2 (system)

Location

lisp-unit.lisp

Exported Definitions

• define-test (macro)
• empty (method)
• (setf empty) (method)
• errors (method)
• (setf errors) (method)
• failed (method)
• (setf failed) (method)
• get-tests (function)
• list-tags (function)
• list-tests (function)
• missing (method)
• (setf missing) (method)
• passed (method)
• (setf passed) (method)
• remove-tags (function)
• remove-tests (function)
• rerun-failures (generic function)
• rerun-failures (method)
• rerun-failures (method)
• rerun-tests (generic function)
• rerun-tests (method)
• rerun-tests (method)
• reset-test-database (function)
• results (method)
• (setf results) (method)
• run-test (generic function)
• run-test (method)
• run-test (method)
• run-test (method)
• run-tests (generic function)
• run-tests (method)
• run-tests (method)
• test-code (function)
• test-documentation (function)
• test-result (class)
• test-results-db (class)
• undefine-test (macro)
• uninstall-test (generic function)
• uninstall-test (method)
• uninstall-test (method)
• unit-test (method)
• (setf unit-test) (method)
• unit-test (class)
• warnings (method)
• (setf warnings) (method)
• with-test-signals-muffled (macro)

Internal Definitions

• %compile (generic function)
• %compile (method)
• %dequote (function)
• %has? (function)
• %in-package (function)
• %log (macro)
• %log-around (macro)
• %run-test (function)
• %run-test-name (function)
• %tests (method)
• (setf %tests) (method)
• %to-test (function)
• %ts (function)
• %uninstall-name (function)
• all-warnings (generic function)
• all-warnings (method)
• all-warnings (method)
• all-warnings (method)
• args (method)
• (setf args) (method)
• code (method)
• (setf code) (method)
• combine-contexts (function)
• contexts (method)
• (setf contexts) (method)
• data (method)
• (setf data) (method)
• do-contexts (function)
• doc (method)
• (setf doc) (method)
• end-time (method)
• (setf end-time) (method)
• eval-package (method)
• (setf eval-package) (method)
• failed-assertions (generic function)
• failed-assertions (method)
• failed-assertions (method)
• failed-assertions (method)
• install-test (generic function)
• install-test (method)
• internal-end-time (method)
• (setf internal-end-time) (method)
• internal-start-time (method)
• (setf internal-start-time) (method)
• most-recent-result (method)
• (setf most-recent-result) (method)
• name (method)
• (setf name) (method)
• name (method)
• (setf name) (method)
• name-index (method)
• (setf name-index) (method)
• null-tags-warning (condition)
• null-tags-warning-report (function)
• null-tests-warning (condition)
• null-tests-warning-report (function)
• package-index (method)
• (setf package-index) (method)
• package-names (generic function)
• package-names (method)
• passed-assertions (generic function)
• passed-assertions (method)
• passed-assertions (method)
• passed-assertions (method)
• record-result (generic function)
• record-result (method)
• record-result-context (function)
• return-value (method)
• (setf return-value) (method)
• run-time (generic function)
• run-time (method)
• short-full-name (function)
• start-time (method)
• (setf start-time) (method)
• status (function)
• tag-index (method)
• (setf tag-index) (method)
• tags (method)
• (setf tags) (method)
• tags-package-name (method)
• test-database (class)
• test-name-error (condition)
• test-name-error-report (function)
• test-results-mixin (class)
• test-signals-muffled-context (function)
• test-thunk (generic function)
• test-thunk (method)
• test-thunk-name (generic function)
• test-thunk-name (method)
• test-thunk-name (method)
• tests (generic function)
• tests (method)
• tests (method)
• (setf tests) (method)
• tests-package-name (method)
• tests-with-status (generic function)
• tests-with-status (method)
• tests-with-status (method)
• unit-test-control-mixin (class)
• valid-test-name (function)

3.1.8 lisp-unit2/summarize.lisp

Dependency

lisp-unit.lisp (file)

Parent

lisp-unit2 (system)

Location

summarize.lisp

Exported Definitions

• print-failure-summary (function)
• print-status-summary (generic function)
• print-status-summary (method)
• print-status-summary (method)
• print-status-summary (method)
• print-summary (function)
• with-assertion-summary (macro)
• with-assertion-summary-context (function)
• with-summary (macro)
• with-summary-context (function)
• with-test-results (macro)
• with-test-results-context (function)

Internal Definitions

• %out (function)
• %print-result-summary (generic function)
• %print-result-summary (method)
• %print-summary (generic function)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• pprint-test-block (macro)

3.1.9 lisp-unit2/rational.lisp

Dependency

summarize.lisp (file)

Parent

lisp-unit2 (system)

Location

rational.lisp

Exported Definitions

• assert-rational-equal (macro)
• rational-equal (generic function)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)

Internal Definitions

%seq-rational-equal (function)

3.1.10 lisp-unit2/floating-point.lisp

Dependency

rational.lisp (file)

Parent

lisp-unit2 (system)

Location

floating-point.lisp

Exported Definitions

• *epsilon* (special variable)
• *measure* (special variable)
• *significant-figures* (special variable)
• array-error (function)
• assert-float-equal (macro)
• assert-norm-equal (macro)
• assert-number-equal (macro)
• assert-numerical-equal (macro)
• assert-sigfig-equal (macro)
• complex-random (function)
• default-epsilon (generic function)
• default-epsilon (method)
• float-equal (generic function)
• float-equal (method)
• make-2d-list (function)
• make-random-2d-array (function)
• make-random-2d-list (function)
• make-random-list (function)
• norm (generic function)
• norm (method)
• norm (method)
• norm (method)
• norm-equal (generic function)
• norm-equal (method)
• norm-equal (method)
• norm-equal (method)
• norm-equal (method)
• norm-equal (method)
• number-equal (function)
• numerical-equal (generic function)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• relative-error (generic function)
• relative-error (method)
• relative-error (method)
• relative-error (method)
• relative-error (method)
• relative-error-norm (generic function)
• relative-error-norm (method)
• relative-error-norm (method)
• relative-error-norm (method)
• relative-error-norm (method)
• relative-error-norm (method)
• sigfig-equal (generic function)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sump (generic function)
• sump (method)
• sump (method)
• sump (method)
• sumsq (generic function)
• sumsq (method)
• sumsq (method)
• sumsq (method)

Internal Definitions

• %array-error (function)
• %array-indices (function)
• %complex-float-random (function)
• %complex-rational-random (function)
• %norm (generic function)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm-equal (function)
• %normalize-float (function)
• %relative-error (function)
• %relative-error-norm (function)
• %seq-sigfig-equal (function)
• %sequence-equal (function)
• %sequence-error (function)
• %sigfig-equal (function)
• sequence-error (function)

3.1.11 lisp-unit2/test-anything-protocol.lisp

Dependency

floating-point.lisp (file)

Parent

lisp-unit2 (system)

Location

test-anything-protocol.lisp

Exported Definitions

• with-tap-context (function)
• with-tap-summary (macro)
• write-tap (function)
• write-tap-to-file (function)

Internal Definitions

• %write-tap-test-result (function)
• print-tap (function)

3.1.12 lisp-unit2/interop.lisp

Dependency

test-anything-protocol.lisp (file)

Parent

lisp-unit2 (system)

Location

interop.lisp

Internal Definitions

• +interop-test-result-contexts+ (special variable)
• convert-test-result (generic function)
• convert-test-results (generic function)
• get-system-list (function)
• test-asdf-system-recursive (generic function)
• test-asdf-system-recursive (method)

4 Packages

Packages are listed by definition order.

4.1 lisp-unit2

Source

package.lisp (file)

Use List

• iterate
• common-lisp

Exported Definitions

• *epsilon* (special variable)
• *measure* (special variable)
• *result* (special variable)
• *results* (special variable)
• *significant-figures* (special variable)
• *test-db* (special variable)
• *test-stream* (special variable)
• *unit-test* (special variable)
• all-tests-complete (condition)
• all-tests-start (condition)
• array-error (function)
• assert-eq (macro)
• assert-eql (macro)
• assert-equal (macro)
• assert-equality (macro)
• assert-equalp (macro)
• assert-error (macro)
• assert-expands (macro)
• assert-false (macro)
• assert-float-equal (macro)
• assert-no-error (macro)
• assert-no-signal (macro)
• assert-no-warning (macro)
• assert-norm-equal (macro)
• assert-number-equal (macro)
• assert-numerical-equal (macro)
• assert-prints (macro)
• assert-rational-equal (macro)
• assert-sigfig-equal (macro)
• assert-signal (macro)
• assert-true (macro)
• assert-typep (macro)
• assert-warning (macro)
• assertion-fail (condition)
• assertion-pass (condition)
• collect-test-results (condition)
• complex-random (function)
• default-epsilon (generic function)
• default-epsilon (method)
• define-test (macro)
• empty (generic function)
• empty (method)
• (setf empty) (method)
• (setf empty) (generic function)
• errors (generic function)
• errors (method)
• (setf errors) (method)
• (setf errors) (generic function)
• failed (generic function)
• failed (method)
• (setf failed) (method)
• (setf failed) (generic function)
• failure-result (class)
• float-equal (generic function)
• float-equal (method)
• get-tests (function)
• list-tags (function)
• list-tests (function)
• logically-equal (function)
• make-2d-list (function)
• make-random-2d-array (function)
• make-random-2d-list (function)
• make-random-list (function)
• missing (generic function)
• missing (method)
• (setf missing) (method)
• (setf missing) (generic function)
• missing-test (condition)
• norm (generic function)
• norm (method)
• norm (method)
• norm (method)
• norm-equal (generic function)
• norm-equal (method)
• norm-equal (method)
• norm-equal (method)
• norm-equal (method)
• norm-equal (method)
• number-equal (function)
• numerical-equal (generic function)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• numerical-equal (method)
• passed (generic function)
• passed (method)
• (setf passed) (method)
• (setf passed) (generic function)
• print-failure-summary (function)
• print-status-summary (generic function)
• print-status-summary (method)
• print-status-summary (method)
• print-status-summary (method)
• print-summary (function)
• rational-equal (generic function)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• rational-equal (method)
• relative-error (generic function)
• relative-error (method)
• relative-error (method)
• relative-error (method)
• relative-error (method)
• relative-error-norm (generic function)
• relative-error-norm (method)
• relative-error-norm (method)
• relative-error-norm (method)
• relative-error-norm (method)
• relative-error-norm (method)
• remove-tags (function)
• remove-tests (function)
• rerun-failures (generic function)
• rerun-failures (method)
• rerun-failures (method)
• rerun-tests (generic function)
• rerun-tests (method)
• rerun-tests (method)
• reset-test-database (function)
• results (generic function)
• results (method)
• (setf results) (method)
• results (method)
• (setf results) (method)
• results (method)
• (setf results) (method)
• results (method)
• (setf results) (method)
• (setf results) (generic function)
• run-test (generic function)
• run-test (method)
• run-test (method)
• run-test (method)
• run-tests (generic function)
• run-tests (method)
• run-tests (method)
• set-equal (function)
• sigfig-equal (generic function)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sigfig-equal (method)
• sump (generic function)
• sump (method)
• sump (method)
• sump (method)
• sumsq (generic function)
• sumsq (method)
• sumsq (method)
• sumsq (method)
• test-code (function)
• test-complete (condition)
• test-documentation (function)
• test-result (class)
• test-results-db (class)
• test-start (condition)
• undefine-test (macro)
• uninstall-test (generic function)
• uninstall-test (method)
• uninstall-test (method)
• unit-test (generic function)
• unit-test (method)
• (setf unit-test) (method)
• unit-test (method)
• (setf unit-test) (method)
• unit-test (method)
• (setf unit-test) (method)
• unit-test (method)
• (setf unit-test) (method)
• unit-test (method)
• (setf unit-test) (method)
• (setf unit-test) (generic function)
• unit-test (class)
• warnings (generic function)
• warnings (method)
• (setf warnings) (method)
• (setf warnings) (generic function)
• with-assertion-summary (macro)
• with-assertion-summary-context (function)
• with-failure-debugging (macro)
• with-failure-debugging-context (function)
• with-summary (macro)
• with-summary-context (function)
• with-tap-context (function)
• with-tap-summary (macro)
• with-test-results (macro)
• with-test-results-context (function)
• with-test-signals-muffled (macro)
• write-tap (function)
• write-tap-to-file (function)

Internal Definitions

• %array-error (function)
• %array-indices (function)
• %collect (generic function)
• %collect (method)
• %collect (method)
• %collect! (macro)
• %collect-new (generic function)
• %collect-new (method)
• %collect-new (method)
• %collect-new! (macro)
• %compile (generic function)
• %compile (method)
• %complex-float-random (function)
• %complex-rational-random (function)
• %decollect (generic function)
• %decollect (method)
• %decollect (method)
• %decollect! (macro)
• %dequote (function)
• %form-equal (function)
• %has? (function)
• %in-package (function)
• %log (macro)
• %log-around (macro)
• %norm (generic function)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm (method)
• %norm-equal (function)
• %normalize-float (function)
• %out (function)
• %print-result-summary (generic function)
• %print-result-summary (method)
• %print-summary (generic function)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %print-summary (method)
• %relative-error (function)
• %relative-error-norm (function)
• %run-test (function)
• %run-test-name (function)
• %seq-rational-equal (function)
• %seq-sigfig-equal (function)
• %sequence-equal (function)
• %sequence-error (function)
• %sigfig-equal (function)
• %tests (generic function)
• %tests (method)
• (setf %tests) (method)
• (setf %tests) (generic function)
• %to-test (function)
• %ts (function)
• %uninstall-name (function)
• %write-tap-test-result (function)
• *log-level* (special variable)
• *test-log-stream* (special variable)
• +interop-test-result-contexts+ (special variable)
• +statuses+ (special variable)
• actual (generic function)
• actual (method)
• (setf actual) (method)
• (setf actual) (generic function)
• all-warnings (generic function)
• all-warnings (method)
• all-warnings (method)
• all-warnings (method)
• args (generic function)
• args (method)
• (setf args) (method)
• (setf args) (generic function)
• assert-passes? (generic function)
• assert-passes? (method)
• assertion (generic function)
• assertion (method)
• (setf assertion) (method)
• assertion (method)
• (setf assertion) (method)
• (setf assertion) (generic function)
• code (generic function)
• code (method)
• (setf code) (method)
• (setf code) (generic function)
• combine-contexts (function)
• contexts (generic function)
• contexts (method)
• (setf contexts) (method)
• (setf contexts) (generic function)
• convert-test-result (generic function)
• convert-test-results (generic function)
• data (generic function)
• data (method)
• (setf data) (method)
• (setf data) (generic function)
• do-contexts (function)
• doc (generic function)
• doc (method)
• (setf doc) (method)
• (setf doc) (generic function)
• end-time (generic function)
• end-time (method)
• (setf end-time) (method)
• (setf end-time) (generic function)
• equal-result (class)
• error-result (class)
• eval-package (generic function)
• eval-package (method)
• (setf eval-package) (method)
• (setf eval-package) (generic function)
• expand-assert (macro)
• expand-extras (macro)
• expand-macro-form (macro)
• expand-output-form (macro)
• expand-signaled-handler (function)
• expected (generic function)
• expected (method)
• (setf expected) (method)
• (setf expected) (generic function)
• extras (generic function)
• extras (method)
• (setf extras) (method)
• (setf extras) (generic function)
• failed-assertions (generic function)
• failed-assertions (method)
• failed-assertions (method)
• failed-assertions (method)
• failure (generic function)
• failure (method)
• (setf failure) (method)
• (setf failure) (generic function)
• form (generic function)
• form (method)
• (setf form) (method)
• (setf form) (generic function)
• get-system-list (function)
• head (generic function)
• head (method)
• head (method)
• (setf head) (generic function)
• (setf head) (method)
• install-test (generic function)
• install-test (method)
• internal-assert (function)
• internal-end-time (generic function)
• internal-end-time (method)
• (setf internal-end-time) (method)
• (setf internal-end-time) (generic function)
• internal-start-time (generic function)
• internal-start-time (method)
• (setf internal-start-time) (method)
• (setf internal-start-time) (generic function)
• len (generic function)
• len (method)
• len (method)
• (setf len) (generic function)
• (setf len) (method)
• list-collector (class)
• macro-result (class)
• most-recent-result (generic function)
• most-recent-result (method)
• (setf most-recent-result) (method)
• (setf most-recent-result) (generic function)
• name (generic function)
• name (method)
• (setf name) (method)
• name (method)
• (setf name) (method)
• name (method)
• (setf name) (method)
• (setf name) (generic function)
• name-index (generic function)
• name-index (method)
• (setf name-index) (method)
• (setf name-index) (generic function)
• null-tags-warning (condition)
• null-tags-warning-report (function)
• null-tests-warning (condition)
• null-tests-warning-report (function)
• output-result (class)
• package-index (generic function)
• package-index (method)
• (setf package-index) (method)
• (setf package-index) (generic function)
• package-names (generic function)
• package-names (method)
• passed-assertions (generic function)
• passed-assertions (method)
• passed-assertions (method)
• passed-assertions (method)
• pprint-test-block (macro)
• print-tap (function)
• record-failure (generic function)
• record-failure (method)
• record-result (generic function)
• record-result (method)
• record-result-context (function)
• result (generic function)
• result (method)
• (setf result) (method)
• (setf result) (generic function)
• return-value (generic function)
• return-value (method)
• (setf return-value) (method)
• (setf return-value) (generic function)
• run-time (generic function)
• run-time (method)
• sequence-error (function)
• short-full-name (function)
• signal-result (class)
• start-time (generic function)
• start-time (method)
• (setf start-time) (method)
• (setf start-time) (generic function)
• status (function)
• tag-index (generic function)
• tag-index (method)
• (setf tag-index) (method)
• (setf tag-index) (generic function)
• tags (generic function)
• tags (method)
• (setf tags) (method)
• (setf tags) (generic function)
• tags-package-name (generic function)
• tags-package-name (method)
• tail (generic function)
• tail (method)
• tail (method)
• (setf tail) (generic function)
• (setf tail) (method)
• test (generic function)
• test (method)
• (setf test) (method)
• (setf test) (generic function)
• test-asdf-system-recursive (generic function)
• test-asdf-system-recursive (method)
• test-database (class)
• test-name (generic function)
• test-name (method)
• (setf test-name) (method)
• (setf test-name) (generic function)
• test-name-error (condition)
• test-name-error-report (function)
• test-results-mixin (class)
• test-signals-muffled-context (function)
• test-thunk (generic function)
• test-thunk (method)
• test-thunk-name (generic function)
• test-thunk-name (method)
• test-thunk-name (method)
• tests (generic function)
• tests (method)
• tests (method)
• (setf tests) (generic function)
• (setf tests) (method)
• tests-package-name (generic function)
• tests-package-name (method)
• tests-with-status (generic function)
• tests-with-status (method)
• tests-with-status (method)
• unit-test-control-mixin (class)
• valid-test-name (function)

4.2 lisp-unit2-asserts

Source

assert-package.lisp (file)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Special variables

Special Variable: *epsilon*

Set the error epsilon if the defaults are not acceptable.

Package

lisp-unit2

Source

floating-point.lisp (file)

Special Variable: *measure*

Package

lisp-unit2

Source

floating-point.lisp (file)

Special Variable: *result*

The current test result (bound in %run-test)

Package

lisp-unit2

Source

vars.lisp (file)

Special Variable: *results*

The current results database (bound in run-tests)

Package

lisp-unit2

Source

vars.lisp (file)

Special Variable: *significant-figures*

Default to 4 significant figures.

Package

lisp-unit2

Source

floating-point.lisp (file)

Special Variable: *test-db*

The unit test database is a list of tests and some hashtable indexes

Package

lisp-unit2

Source

vars.lisp (file)

Special Variable: *test-stream*

Package

lisp-unit2

Source

vars.lisp (file)

Special Variable: *unit-test*

The currently executing unit test (bound in %run-test, ie every test function)

Package

lisp-unit2

Source

vars.lisp (file)

5.1.2 Macros

Macro: assert-eq EXPECTED FORM &rest EXTRAS

Assert whether expected and form are EQ.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-eql EXPECTED FORM &rest EXTRAS

Assert whether expected and form are EQL.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-equal EXPECTED FORM &rest EXTRAS

Assert whether expected and form are EQUAL.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-equality TEST EXPECTED FORM &rest EXTRAS

Assert whether expected and form are equal according to test.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-equalp EXPECTED FORM &rest EXTRAS

Assert whether expected and form are EQUALP.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-error CONDITION FORM &rest EXTRAS

Assert whether form signals condition.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-expands EXPANSION FORM &rest EXTRAS

Assert whether form expands to expansion.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-false FORM &rest EXTRAS

Assert whether the form is false.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-float-equal EXPECTED FORM &rest EXTRAS

Package

lisp-unit2

Source

floating-point.lisp (file)

Macro: assert-no-error CONDITION FORM &rest EXTRAS

Assert whether form signals condition.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-no-signal CONDITION FORM &rest EXTRAS

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-no-warning CONDITION FORM &rest EXTRAS

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-norm-equal EXPECTED FORM &rest EXTRAS

Package

lisp-unit2

Source

floating-point.lisp (file)

Macro: assert-number-equal EXPECTED FORM &rest EXTRAS

Package

lisp-unit2

Source

floating-point.lisp (file)

Macro: assert-numerical-equal EXPECTED FORM &rest EXTRAS

Package

lisp-unit2

Source

floating-point.lisp (file)

Macro: assert-prints OUTPUT FORM &rest EXTRAS

Assert whether printing the form generates the output.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-rational-equal EXPECTED FORM &rest EXTRAS

Package

lisp-unit2

Source

rational.lisp (file)

Macro: assert-sigfig-equal EXPECTED FORM &rest EXTRAS

Package

lisp-unit2

Source

floating-point.lisp (file)

Macro: assert-signal CONDITION FORM &rest EXTRAS

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-true FORM &rest EXTRAS

Assert whether the form is true.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-typep EXPECTED-TYPE FORM &rest EXTRAS

Assert whether expected and form are TYPEP.

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: assert-warning CONDITION FORM &rest EXTRAS

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: define-test NAME (&key TAGS CONTEXTS PACKAGE) &body BODY

Defines a new test object, test functions and installs the test
function in the test database

name: the name of the test and the test-function

contexts: a (lambda (fn)...) (or list of) that runs the fn in a dynamic context

tags: symbols that can be used to group tests

If package is provided, the name and any provided tags are reinterned into package before use. Keywords are left alone. This is useful because many times tests and the functions they test, coexist (and always could in lisp-unit v1, now that we create test functions, we dont want them overwriting the original function).

Note: This should probably not be used (rather opting for packaged::symbols), but will be useful when converting from lisp-unit 1->2. See also: run-tests, get-tests reintern-package

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Macro: undefine-test NAME (&key TAGS CONTEXTS PACKAGE) &body BODY

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Macro: with-assertion-summary () &body BODY

Package

lisp-unit2

Source

summarize.lisp (file)

Macro: with-failure-debugging () &body BODY

A context macro that invokes the debugger on failed assertions

Package

lisp-unit2

Source

asserts.lisp (file)

Macro: with-summary (&key NAME) &body BODY

Package

lisp-unit2

Source

summarize.lisp (file)

Macro: with-tap-summary () &body BODY

Package

lisp-unit2

Source

test-anything-protocol.lisp (file)

Macro: with-test-results (&key COLLECTION-PLACE SUMMARIZE?) &body BODY

see with-test-results-context

Package

lisp-unit2

Source

summarize.lisp (file)

Macro: with-test-signals-muffled () &body BODY

Package

lisp-unit2

Source

lisp-unit.lisp (file)

5.1.3 Functions

Function: array-error ARRAY1 ARRAY2 &key TEST ERROR-FUNCTION

Return a list of the indices and error between the array elements.

Package

lisp-unit2

Source

floating-point.lisp (file)

Function: complex-random LIMIT &optional STATE

Return a random complex number.

Package

lisp-unit2

Source

floating-point.lisp (file)

Function: get-tests &key TESTS TAGS PACKAGE REINTERN-PACKAGE EXCLUDE-TESTS EXCLUDE-TAGS &aux DB OUT

finds tests by names, tags, and package

if reintern-package is provided, reintern all symbols provided for tests and tags into the reintern-package. Mostly provided for easing conversion of lisp-unit1 test suites

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: list-tags ()

Return a list of the tags

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: list-tests ()

Return a list of all tests,
use get tests to find tests by package tag or name

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: logically-equal X Y

Package

lisp-unit2

Source

asserts.lisp (file)

Function: make-2d-list ROWS COLUMNS &key INITIAL-ELEMENT

Return a nested list with INITIAL-ELEMENT.

Package

lisp-unit2

Source

floating-point.lisp (file)

Function: make-random-2d-array ROWS COLUMNS &optional LIMIT

Return a 2D array of random numbers.

Package

lisp-unit2

Source

floating-point.lisp (file)

Function: make-random-2d-list ROWS COLUMNS &optional LIMIT

Return a nested list of random numbers.

Package

lisp-unit2

Source

floating-point.lisp (file)

Function: make-random-list SIZE &optional LIMIT

Return a list of random numbers.

Package

lisp-unit2

Source

floating-point.lisp (file)

Function: number-equal NUMBER1 NUMBER2 &optional EPSILON TYPE-EQ-P

Return true if the numbers are equal within some epsilon, optionally requiring the types to be identical.

Package

lisp-unit2

Source

floating-point.lisp (file)

Function: print-failure-summary O &aux *PRINT-PRETTY*

Package

lisp-unit2

Source

summarize.lisp (file)

Function: print-summary O &aux *PRINT-PRETTY*

Package

lisp-unit2

Source

summarize.lisp (file)

Function: remove-tags &optional TAGS

Remove individual tags or entire sets.

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: remove-tests &optional TESTS TAGS PACKAGE

Remove individual tests or entire sets.

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: reset-test-database ()

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: set-equal LIST1 LIST2 &rest INITARGS &key KEY TEST

Return true if every element of list1 is an element of list2 and vice versa.

Package

lisp-unit2

Source

asserts.lisp (file)

Function: test-code NAME

Returns the code stored for the test name.

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: test-documentation NAME

Return the documentation for the test.

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Function: with-assertion-summary-context BODY-FN &aux *PRINT-PRETTY* RTN

Package

lisp-unit2

Source

summarize.lisp (file)

Function: with-failure-debugging-context BODY-FN

A context that invokes the debugger on failed assertions

Package

lisp-unit2

Source

asserts.lisp (file)

Function: with-summary-context BODY-FN &key NAME &aux *PRINT-PRETTY* RTN

Package

lisp-unit2

Source

summarize.lisp (file)

Function: with-tap-context BODY-FN

Package

lisp-unit2

Source

test-anything-protocol.lisp (file)

Function: with-test-results-context BODY-FN &key COLLECTION-PLACE-SETTER SUMMARIZE? &aux RTN TEST-RESULT-DBS

A context that collects test-result-databases
if collection-place-setter (lambda (new) (setf collection-place new)) exists anytime we get a new test result we set the place to the new collection

if summarize? at the end of the block we print the results summary for all tests this defaults to T if no collection-place-setter is provided and nil otherwise

Package

lisp-unit2

Source

summarize.lisp (file)

Function: write-tap TEST-RESULTS

Write the test results to ‘stream‘ in TAP format. Returns the test results.

Package

lisp-unit2

Source

test-anything-protocol.lisp (file)

Function: write-tap-to-file TEST-RESULTS PATH

write the test results to ‘path‘ in TAP format, overwriting ‘path‘. Returns pathname to the output file

Package

lisp-unit2

Source

test-anything-protocol.lisp (file)

5.1.4 Generic functions

Generic Function: default-epsilon VALUE

Return the default epsilon for the value.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: default-epsilon VALUE

Generic Function: empty OBJECT

Generic Function: (setf empty) NEW-VALUE OBJECT

Package

lisp-unit2

Methods

Method: empty (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf empty) NEW-VALUE (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated writer method

Source

lisp-unit.lisp (file)

Generic Function: errors OBJECT

Generic Function: (setf errors) NEW-VALUE OBJECT

Package

lisp-unit2

Methods

Method: errors (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf errors) NEW-VALUE (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated writer method

Source

lisp-unit.lisp (file)

Generic Function: failed OBJECT

Generic Function: (setf failed) NEW-VALUE OBJECT

Package

lisp-unit2

Methods

Method: failed (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf failed) NEW-VALUE (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated writer method

Source

lisp-unit.lisp (file)

Generic Function: float-equal DATA1 DATA2 &optional EPSILON

Return true if the floating point data is equal.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: float-equal X Y &optional EPSILON

Generic Function: missing OBJECT

Generic Function: (setf missing) NEW-VALUE OBJECT

Package

lisp-unit2

Methods

Method: missing (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf missing) NEW-VALUE (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated writer method

Source

lisp-unit.lisp (file)

Generic Function: norm DATA &optional MEASURE

Return the element-wise norm of the data.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: norm (DATA array) &optional MEASURE

Return the entrywise norm of the array according to the measure.

Method: norm (DATA vector) &optional MEASURE

Return the norm of the vector according to the measure.

Method: norm (DATA list) &optional MEASURE

Return the norm of the list according to the measure.

Generic Function: norm-equal DATA1 DATA2 &optional EPSILON MEASURE

Return true if the norm of the data is equal.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: norm-equal (DATA1 array) (DATA2 array) &optional EPSILON MEASURE

Return true if the arrays are equal in length and the relative error norm is less than epsilon.

Method: norm-equal (DATA1 vector) (DATA2 vector) &optional EPSILON MEASURE

Return true if the vectors are equal in length and the relative error norm is less than epsilon.

Method: norm-equal (DATA1 vector) (DATA2 list) &optional EPSILON MEASURE

Return true if the vector and the list are equal in length and the relative error norm is less than epsilon.

Method: norm-equal (DATA1 list) (DATA2 vector) &optional EPSILON MEASURE

Return true if the vector and the list are equal in length and the relative error norm is less than epsilon.

Method: norm-equal (DATA1 list) (DATA2 list) &optional EPSILON MEASURE

Return true if the lists are equal in length and the relative error norm is less than epsilon.

Generic Function: numerical-equal RESULT1 RESULT2 &key TEST

Return true if the results are numerically equal according to :TEST.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: numerical-equal (RESULT1 array) (RESULT2 array) &key TEST

Return true if the arrays are equal in dimension and each element is equal according to :TEST.

Method: numerical-equal (RESULT1 vector) (RESULT2 list) &key TEST

Return true if every element of the list is equla to the corresponding element of the vector.

Method: numerical-equal (RESULT1 list) (RESULT2 vector) &key TEST

Return true if every element of the list is equal to the corresponding element of the vector.

Method: numerical-equal (RESULT1 vector) (RESULT2 vector) &key TEST

Return true if the vectors are equal in length and each element is equal according to :TEST.

Method: numerical-equal (RESULT1 list) (RESULT2 list) &key TEST

Return true if the lists are equal in length and each element is equal according to :TEST.

Method: numerical-equal (RESULT1 number) (RESULT2 number) &key TEST

Return true if the the numbers are equal according to :TEST.

Generic Function: passed OBJECT

Generic Function: (setf passed) NEW-VALUE OBJECT

Package

lisp-unit2

Methods

Method: passed (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf passed) NEW-VALUE (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated writer method

Source

lisp-unit.lisp (file)

Generic Function: print-status-summary OBJECT STATUS

Package

lisp-unit2

Source

summarize.lisp (file)

Methods

Method: print-status-summary (DB test-results-db) (STATUS list)

Method: print-status-summary (DB test-results-db) (S symbol) &aux OBJS

Method: print-status-summary (O test-result) (S symbol) &aux OBJS

Generic Function: rational-equal DATA1 DATA2

Return true if the rational data is equal.

Package

lisp-unit2

Source

rational.lisp (file)

Methods

Method: rational-equal (DATA1 array) (DATA2 array)

Return true if the arrays are equal in dimension and element-wise equal.

Method: rational-equal (DATA1 vector) (DATA2 vector)

Return true if the vectors are equal in length and element-wise equal.

Method: rational-equal (DATA1 vector) (DATA2 list)

Return true if the vector and the list are equal in length and element-wise equal.

Method: rational-equal (DATA1 list) (DATA2 vector)

Return true if the vector and the list are equal in length and element-wise equal.

Method: rational-equal (DATA1 list) (DATA2 list)

Return true if the lists are equal in length and element-wise equal.

Method: rational-equal (DATA1 complex) (DATA2 complex)

Return true if the complex parts are rational and equal.

Method: rational-equal (DATA1 rational) (DATA2 rational)

Return true if the rational numbers are equal.

Generic Function: relative-error EXACT APPROXIMATE

Return the relative-error between the 2 quantities.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: relative-error (EXACT complex) (APPROXIMATE complex)

Return the relative error of the complex numbers.

Method: relative-error (EXACT complex) (APPROXIMATE float)

Return the relative error between the float and complex number.

Method: relative-error (EXACT float) (APPROXIMATE complex)

Return the relative error between the float and complex number.

Method: relative-error (EXACT float) (APPROXIMATE float)

Return the error delta between the exact and approximate floating point value.

Generic Function: relative-error-norm EXACT APPROXIMATE &optional MEASURE

Return the relative error norm

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: relative-error-norm (EXACT array) (APPROXIMATE vector) &optional MEASURE

Return the relative error norm of the arrays.

Method: relative-error-norm (EXACT vector) (APPROXIMATE vector) &optional MEASURE

Return the relative error norm of the vectors.

Method: relative-error-norm (EXACT vector) (APPROXIMATE list) &optional MEASURE

Return the relative error norm of the list and the vector.

Method: relative-error-norm (EXACT list) (APPROXIMATE vector) &optional MEASURE

Return the relative error norm of the list and the vector.

Method: relative-error-norm (EXACT list) (APPROXIMATE list) &optional MEASURE

Return the relative error norm of the lists.

Generic Function: rerun-failures TEST-RESULTS-DB &key STATUS

reruns failed tests

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Methods

Method: rerun-failures (DBS list) &key STATUS

Method: rerun-failures (DB test-results-db) &key STATUS

Generic Function: rerun-tests TEST-RESULTS-DB

Rerun all tests from a given run (returns new results)

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Methods

Method: rerun-tests (DBS list)

Method: rerun-tests (DB test-results-db)

Generic Function: results CONDITION

Generic Function: (setf results) NEW-VALUE CONDITION

Package

lisp-unit2

Methods

Method: results (TEST-RESULTS-DB test-results-db)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf results) NEW-VALUE (TEST-RESULTS-DB test-results-db)

automatically generated writer method

Source

lisp-unit.lisp (file)

Method: results (CONDITION collect-test-results)

Method: (setf results) NEW-VALUE (CONDITION collect-test-results)

Source

vars.lisp (file)

Method: results (CONDITION all-tests-complete)

Method: (setf results) NEW-VALUE (CONDITION all-tests-complete)

Source

vars.lisp (file)

Method: results (CONDITION all-tests-start)

Method: (setf results) NEW-VALUE (CONDITION all-tests-start)

Source

vars.lisp (file)

Generic Function: run-test TEST &key TEST-CONTEXTS

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Methods

Method: run-test (N symbol) &key TEST-CONTEXTS

Method: run-test (U symbol) &key TEST-CONTEXTS around

Method: run-test (U unit-test) &key TEST-CONTEXTS

Generic Function: run-tests &key TESTS TAGS PACKAGE NAME EXCLUDE-TESTS EXCLUDE-TAGS TEST-CONTEXTS RUN-CONTEXTS REINTERN-PACKAGE

Run the specified tests.

We run all the listed tests, and all tests tagged with tags. If both test
and tags are nil (the default), then we run all tests in
package (defaulting to *package*)

name is the name of the test run. Generally expected to be the name of the system being tested. Can be defaulted with the name parameter of with-summary as well. can also be defaulted with.

reintern-package: when looking up tests, first reintern all tests and tags
in this package. In general this should probably not be used, but is provided for convenience in transitioning from lisp-unit 1 to 2 (see: define-test package)

test-contexts is a list of contexts that will be applied around
each individual test

run-contexts is a list of contexts that will be applied around the entire
suite (around signals)

exclude-tests, exclude-tags: tests / tags to remove from the
run. explicit inclusion overrides, explicit exclusion, overrides
implicit inclusion
EG: (define-test test-takes-forever (manual stuff) ...)
(find-test :tags ’stuff :exclude-tags ’manual)
will not find test-takes-forever
(find-test :tags ’(stuff manual) :exclude-tags ’manual)
(find-test :tests ’test-takes-forever :exclude-tags ’manual)
both will find test-takes-forever

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Methods

Method: run-tests &key TESTS TAGS PACKAGE NAME EXCLUDE-TESTS EXCLUDE-TAGS TEST-CONTEXTS RUN-CONTEXTS REINTERN-PACKAGE around

Method: run-tests &rest ARGS &key TESTS TAGS PACKAGE REINTERN-PACKAGE NAME EXCLUDE-TESTS EXCLUDE-TAGS TEST-CONTEXTS RUN-CONTEXTS &aux ALL-TESTS RESULTS *RESULTS*

Generic Function: sigfig-equal DATA1 DATA2 &optional SIGNIFICANT-FIGURES

Return true if the data have equal significant figures.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: sigfig-equal (DATA1 array) (DATA2 array) &optional SIGNIFICANT-FIGURES

Return true if the arrays are equal in length and the element-wise comparison is equal to significant figures.

Method: sigfig-equal (DATA1 vector) (DATA2 vector) &optional SIGNIFICANT-FIGURES

Return true if the vectors are equal in length and the element-wise comparison is equal to significant figures.

Method: sigfig-equal (DATA1 list) (DATA2 vector) &optional SIGNIFICANT-FIGURES

Return true if the list and the vector are equal in length and the element-wise comparison is equal to significant figures.

Method: sigfig-equal (DATA1 vector) (DATA2 list) &optional SIGNIFICANT-FIGURES

Return true if the vector and the list are equal in length and the element-wise comparison is equal to significant figures.

Method: sigfig-equal (DATA1 list) (DATA2 list) &optional SIGNIFICANT-FIGURES

Return true if the lists are equal in length and the element-wise comparison is equal to significant figures.

Method: sigfig-equal (DATA1 float) (DATA2 float) &optional SIGNIFICANT-FIGURES

Return true if the floating point numbers have equal significant figures.

Generic Function: sump DATA P

Return the scaling parameter and the sum of the powers of p of the ~ data.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: sump (DATA array) (P real)

Return the scaling parameter and the sum of the powers of p of the ~ array.

Method: sump (DATA vector) (P real)

Return the scaling parameter and the sum of the powers of p of the ~ vector.

Method: sump (DATA list) (P real)

Return the scaling parameter and the sum of the powers of p of the ~ data.

Generic Function: sumsq DATA

Return the scaling parameter and the sum of the squares of the ~ data.

Package

lisp-unit2

Source

floating-point.lisp (file)

Methods

Method: sumsq (DATA array)

Return the scaling parameter and the sum of the squares of the ~ array.

Method: sumsq (DATA vector)

Return the scaling parameter and the sum of the squares of the ~ vector.

Method: sumsq (DATA list)

Return the scaling parameter and the sum of the squares of the ~ list.

Generic Function: uninstall-test TEST

Uninstalls a previously defined unit test

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Methods

Method: uninstall-test (N symbol)

Method: uninstall-test (U unit-test) &aux N

Generic Function: unit-test CONDITION

Generic Function: (setf unit-test) NEW-VALUE CONDITION

Package

lisp-unit2

Methods

Method: unit-test (TEST-RESULT test-result)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf unit-test) NEW-VALUE (TEST-RESULT test-result)

automatically generated writer method

Source

lisp-unit.lisp (file)

Method: unit-test (FAILURE-RESULT failure-result)

automatically generated reader method

Source

asserts.lisp (file)

Method: (setf unit-test) NEW-VALUE (FAILURE-RESULT failure-result)

automatically generated writer method

Source

asserts.lisp (file)

Method: unit-test (CONDITION assertion-fail)

Method: (setf unit-test) NEW-VALUE (CONDITION assertion-fail)

Source

asserts.lisp (file)

Method: unit-test (CONDITION assertion-pass)

Method: (setf unit-test) NEW-VALUE (CONDITION assertion-pass)

Source

asserts.lisp (file)

Method: unit-test (CONDITION test-start)

Method: (setf unit-test) NEW-VALUE (CONDITION test-start)

Source

vars.lisp (file)

Generic Function: warnings OBJECT

Generic Function: (setf warnings) NEW-VALUE OBJECT

Package

lisp-unit2

Methods

Method: warnings (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated reader method

Source

lisp-unit.lisp (file)

Method: (setf warnings) NEW-VALUE (TEST-RESULTS-MIXIN test-results-mixin)

automatically generated writer method

Source

lisp-unit.lisp (file)

5.1.5 Conditions

Condition: all-tests-complete ()

Signaled when a test run is finished.

Package

lisp-unit2

Source

vars.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• name (method)
• name (method)
• results (method)
• results (method)

Direct slots

Slot: results

Initargs

:results

Initform

(quote nil)

Readers

results (generic function)

Writers

(setf results) (generic function)

Slot: name

Name for this test results

Initargs

:name

Initform

(quote nil)

Readers

name (generic function)

Writers

(setf name) (generic function)

Condition: all-tests-start ()

Signaled when a single test starts.

Package

lisp-unit2

Source

vars.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• results (method)
• results (method)

Direct slots

Slot: results

Initargs

:results

Initform

(quote nil)

Readers

results (generic function)

Writers

(setf results) (generic function)

Condition: assertion-fail ()

Package

lisp-unit2

Source

asserts.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• failure (method)
• failure (method)
• assertion (method)
• assertion (method)
• unit-test (method)
• unit-test (method)

Direct slots

Slot: unit-test

Initargs

:unit-test

Initform

(quote lisp-unit2:*unit-test*)

Readers

unit-test (generic function)

Writers

(setf unit-test) (generic function)

Slot: assertion

Initargs

:assertion

Initform

(quote nil)

Readers

assertion (generic function)

Writers

(setf assertion) (generic function)

Slot: failure

Initargs

:failure

Initform

(quote nil)

Readers

failure (generic function)

Writers

(setf failure) (generic function)

Condition: assertion-pass ()

Package

lisp-unit2

Source

asserts.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• assertion (method)
• assertion (method)
• unit-test (method)
• unit-test (method)

Direct slots

Slot: unit-test

Initargs

:unit-test

Initform

(quote lisp-unit2:*unit-test*)

Readers

unit-test (generic function)

Writers

(setf unit-test) (generic function)

Slot: assertion

Initargs

:assertion

Initform

(quote nil)

Readers

assertion (generic function)

Writers

(setf assertion) (generic function)

Condition: collect-test-results ()

Signaled that with-test-results should collect this test-result-database

Package

lisp-unit2

Source

vars.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• results (method)
• results (method)

Direct slots

Slot: results

Initargs

:results

Initform

(quote nil)

Readers

results (generic function)

Writers

(setf results) (generic function)

Condition: missing-test ()

Signaled when a single test is finished.

Package

lisp-unit2

Source

vars.lisp (file)

Direct superclasses

warning (condition)

Direct methods

• test-name (method)
• test-name (method)

Direct slots

Slot: test-name

Initargs

:test-name

Initform

(quote nil)

Readers

test-name (generic function)

Writers

(setf test-name) (generic function)

Condition: test-complete ()

Signaled when a single test is finished.

Package

lisp-unit2

Source

vars.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• result (method)
• result (method)

Direct slots

Slot: result

Initargs

:result

Initform

(quote nil)

Readers

result (generic function)

Writers

(setf result) (generic function)

Condition: test-start ()

Signaled when a single test starts.

Package

lisp-unit2

Source

vars.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• unit-test (method)
• unit-test (method)

Direct slots

Slot: unit-test

Initargs

:unit-test

Initform

(quote nil)

Readers

unit-test (generic function)

Writers

(setf unit-test) (generic function)

5.1.6 Classes

Class: failure-result ()

Failure details of the assertion.

Package

lisp-unit2

Source

asserts.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

• equal-result (class)
• error-result (class)
• signal-result (class)
• macro-result (class)
• output-result (class)

Direct methods

• %print-summary (method)
• %print-summary (method)
• test (method)
• test (method)
• extras (method)
• extras (method)
• expected (method)
• expected (method)
• actual (method)
• actual (method)
• form (method)
• form (method)
• unit-test (method)
• unit-test (method)

Direct slots

Slot: unit-test

Initargs

:unit-test

Initform

lisp-unit2:*unit-test*

Readers

unit-test (generic function)

Writers

(setf unit-test) (generic function)

Slot: form

Initargs

:form

Readers

form (generic function)

Writers

(setf form) (generic function)

Slot: actual

Initargs

:actual

Readers

actual (generic function)

Writers

(setf actual) (generic function)

Slot: expected

Initargs

:expected

Readers

expected (generic function)

Writers

(setf expected) (generic function)

Slot: extras

Initargs

:extras

Readers

extras (generic function)

Writers

(setf extras) (generic function)

Slot: test

Initargs

:test

Readers

test (generic function)

Writers

(setf test) (generic function)

Class: test-result ()

Package

lisp-unit2

Source

lisp-unit.lisp (file)

Direct superclasses

test-results-mixin (class)

Direct methods

• print-status-summary (method)
• %print-summary (method)
• record-result (method)
• all-warnings (method)
• failed-assertions (method)
• passed-assertions (method)
• print-object (method)
• return-value (method)
• return-value (method)
• unit-test (method)
• unit-test (method)

Direct slots

Slot: unit-test

Initargs

:unit-test

Initform

lisp-unit2:*unit-test*

Readers

unit-test (generic function)

Writers

(setf unit-test) (generic function)

Slot: return-value

Initargs

:return-value

Readers

return-value (generic function)

Writers

(setf return-value) (generic function)