The prove Reference Manual

Table of Contents

The prove Reference Manual

This is the prove Reference Manual, version 1.0.0, generated automatically by Declt version 2.4 "Will Decker" on Wed Jun 20 12:26:19 2018 GMT+0.

1 Introduction

prove

This project was originally called 'CL-TEST-MORE'.

'prove' is yet another unit testing framework for Common Lisp.

The advantages of 'prove' are:

• Various simple functions for testing and informative error messages
• ASDF integration
• Extensible test reporters
• Colorizes the report if it's available (note for SLIME)
• Reports test durations

Quickstart

1. Writing a test file

(in-package :cl-user)
(defpackage my-test
  (:use :cl
        :prove))
(in-package :my-test)

(plan 3)

(ok (not (find 4 '(1 2 3))))
(is 4 4)
(isnt 1 #\1)

(finalize)

2. Run a test file

(prove:run #P"myapp/tests/my-test.lisp")
(prove:run #P"myapp/tests/my-test.lisp" :reporter :list)

See also: ASDF integration, Reporters

3. Get a report

Installation

You can install 'prove' via Quicklisp.

(ql:quickload :prove)

Testing functions

(ok test &optional desc)

Checks if test is true (non-NIL).

(ok 1)
;->  ✓ 1 is expected to be T

(is got expected &rest test-args)

Checks if got is equivalent to expected.

(is 1 1)
;->  ✓ 1 is expected to be 1

(is #(1 2 3) #(1 2 3))
;->  × #(1 2 3) is expected to be #(1 2 3)

(is #(1 2 3) #(1 2 3) :test #'equalp)
;->  ✓ #(1 2 3) is expected to be #(1 2 3)

;; with description
(is 1 #\1 "Integer = Character ?")
;->  × Integer = Character ?

(isnt got expected &rest test-args)

Checks if got is not equivalent to expected.

(isnt 1 1)
;->  × 1 is not expected to be 1

(isnt #(1 2 3) #(1 2 3))
;->  ✓ #(1 2 3) is not expected to be #(1 2 3)

(is-values got expected &rest test-args)

Checks if the multiple values of got is equivalent to expected. This is same to (is (multiple-value-list got) expected).

(defvar *person* (make-hash-table))

(is-values (gethash :name *person*) '("Eitaro" T))
;->  × (NIL NIL) is expected to be ("Eitaro" T)

(setf (gethash :name *person*) "Eitaro")

(is-values (gethash :name *person*) '("Eitaro" T))
;->  ✓ ("Eitaro" T) is expected to be ("Eitaro" T)

(is-type got expected-type &optional desc)

Checks if got is a type of expected-type.

(is-type #(1 2 3) 'simple-vector)
;->  ✓ #(1 2 3) is expected to be a type of SIMPLE-VECTOR (got (SIMPLE-VECTOR 3))

(is-type (make-array 0 :adjustable t) 'simple-vector)
;->  × #() is expected to be a type of SIMPLE-VECTOR (got (VECTOR T 0))

(like got regex &optional desc)

Checks if got matches a regular expression regex.

(like "Hatsune 39" "\\d")
;->  ✓ "Hatsune 39" is expected to be like "\\d"

(like "初音ミク" "\\d")
;->  × "初音ミク" is expected to be like "\\d"

(is-print got expected &optional desc)

Checks if got outputs expected to *standard-output*

(is-print (princ "Hi, there") "Hi, there")
;->  ✓ (PRINC "Hi, there") is expected to output "Hi, there" (got "Hi, there")

(is-error form condition &optional desc)

Checks if form raises a condition and that is a subtype of condition.

(is-error (error "Something wrong") 'simple-error)
;->  ✓ (ERROR "Something wrong") is expected to raise a condition SIMPLE-ERROR (got #<SIMPLE-ERROR "Something wrong" {100628FE53}>)

(define-condition my-error (simple-error) ())

(is-error (error "Something wrong") 'my-error)
;->  × (ERROR "Something wrong") is expected to raise a condition MY-ERROR (got #<SIMPLE-ERROR "Something wrong" {100648E553}>)

(is-expand got expected &optional desc)

Checks if got will be macroexpanded to expected.

(is-expand (when T (princ "Hi")) (if T (progn (princ "Hi"))))
;->  ✓ (WHEN T (PRINC "Hi")) is expected to be expanded to (IF T
;                                                          (PROGN (PRINC "Hi"))) (got (IF T
;                                                                                         (PROGN
;                                                                                          (PRINC
;                                                                                           "Hi"))
;                                                                                         NIL))

If a symbol that starts with "$" is contained, it will be treated as a gensym.

(pass desc)

This will always be passed. This is convenient if the test case is complicated and hard to test with ok.

(pass "Looks good")
;->  ✓ Looks good

(fail desc)

This will always be failed. This is convenient if the test case is complicated and hard to test with ok.

(fail "Hopeless")
;->  × Hopeless

(skip how-many why)

Skip a number of how-many tests and mark them passed.

(skip 3 "No need to test these on Mac OS X")
;->  ✓ No need to test these on Mac OS X (Skipped)
;    ✓ No need to test these on Mac OS X (Skipped)
;    ✓ No need to test these on Mac OS X (Skipped)

(subtest desc &body body)

Run tests of body in a new sub test suite.

(subtest "Testing integers"
  (is 1 1)
  (is-type 1 'bit)
  (is-type 10 'fixnum))
;->      ✓ 1 is expected to be 1
;        ✓ 1 is expected to be a type of BIT (got BIT)
;        ✓ 10 is expected to be a type of FIXNUM (got (INTEGER 0 4611686018427387903))
;->  ✓ Testing integers

Other functions

(diag desc)

Outputs desc to a *test-result-output*.

(diag "Gonna run tests")
;-> # Gonna run tests

(plan num)

Declares a number of num tests are going to run. If finalize is called with no plan, a warning message will be output. num is allows to be NIL if you have no plan yet.

(finalize)

Finalizes the current test suite and outputs the test reports.

(slow-threshold milliseconds)

Set the threshold of slow test durations for the current test suite. The default threshold value is prove:*default-slow-threshold*.

(slow-threshold 150)

Reporters

You can change the test report formats by setting prove:*default-reporter* to :list, :dot, :tap or :fiveam. The default value is :list.

prove:run also takes a keyword argument :reporter.

List (Default)

The :list repoter outputs test results list as test cases pass or fail.

Dot

The :dot reporter outputs a series of dots that represent test cases, failures highlight in red, skipping in cyan.

FiveAM

The :fiveam reporter outputs test results like FiveAM does.

TAP

The :tap reporter outputs in Test Anything Protocol format.

Tips

Debugging with CL debugger

Set prove:*debug-on-error* T for invoking CL debugger whenever getting an error during running tests.

Colorize test reports on SLIME

SLIME doesn't support to color with ANSI colors in the REPL buffer officially.

You can add the feature by using slime-repl-ansi-color.el.

After installing it, set prove:*enable-colors* to T before running tests.

;; A part of my ~/.sbclrc
(ql:quickload :prove)
(setf prove:*enable-colors* t)

The following snippet is a little bit complicated, however it would be better if you don't like to load prove in all sessions.

(defmethod asdf:perform :after ((op asdf:load-op) (c (eql (asdf:find-system :prove))))
  (setf (symbol-value (intern (string :*enable-colors*) :prove)) t))

ASDF integration

Add :defsystem-depends-on (:prove-asdf) to your testing ASDF system to enable :test-file in the :components.

:test-file is same as :file except it will be loaded only when asdf:test-system.

;; Main ASDF system
(defsystem my-app

  ;; ...

  :in-order-to ((test-op (test-op my-app-test))))

;; Testing ASDF system
(defsystem my-app-test
  :depends-on (:my-app
               :prove)
  :defsystem-depends-on (:prove-asdf)
  :components
  ((:test-file "my-app"))
  :perform (test-op :after (op c)
                    (funcall (intern #.(string :run) :prove) c)))

To run tests, execute asdf:test-system or prove:run in your REPL.

(asdf:test-system :my-app)
(asdf:test-system :my-app-test)

;; Same as 'asdf:test-system' except it returns T or NIL as the result of tests.
(prove:run :my-app-test)

Changing default test function

Test functions like is uses prove:*default-test-function* for testing if no :test argument is specified. The default value is #'equal.

Changing output stream

Test reports will be output to prove:*test-result-output*. The default value is T, which means *standard-output*.

Running tests on Travis CI

Although Common Lisp isn't supported by Travis CI officially, you can run tests by using cl-travis.

Here's a list of .travis.yml from projects using prove on Travis CI:

• Clack
• CL-DBI
• Woo
• fast-http
• defclass-std

Bugs

Please report any bugs to e.arrows@gmail.com, or post an issue to GitHub.

License

Copyright (c) 2010-2014 Eitaro Fukamachi <e.arrows@gmail.com>
'prove' and CL-TEST-MORE is freely distributable under the MI

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 prove

Author

Eitaro Fukamachi

License

MIT

Version

1.0.0

Dependencies

• cl-ppcre
• cl-ansi-text
• cl-colors
• alexandria
• uiop

Source

prove.asd (file)

Component

src (module)

3 Modules

Modules are listed depth-first from the system components tree.

3.1 prove/src

Parent

prove (system)

Location

src/

Components

• prove.lisp (file)
• test.lisp (file)
• report.lisp (file)
• reporter.lisp (file)
• reporter-components (module)
• suite.lisp (file)
• asdf.lisp (file)
• color.lisp (file)
• output.lisp (file)

3.2 prove/src/reporter-components

Dependencies

• report.lisp (file)
• reporter.lisp (file)
• color.lisp (file)

Parent

src (module)

Location

src/reporter/

Components

• tap.lisp (file)
• fiveam.lisp (file)
• list.lisp (file)
• dot.lisp (file)

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.

4.1 Lisp

4.1.1 prove.asd

Location

prove.asd

Systems

prove (system)

4.1.2 prove/src/prove.lisp

Dependencies

• output.lisp (file)
• test.lisp (file)
• suite.lisp (file)
• asdf.lisp (file)
• color.lisp (file)

Parent

src (module)

Location

src/prove.lisp

Packages

prove

4.1.3 prove/src/test.lisp

Dependencies

• output.lisp (file)
• report.lisp (file)
• reporter.lisp (file)
• suite.lisp (file)

Parent

src (module)

Location

src/test.lisp

Packages

prove.test

Exported Definitions

• *debug-on-error* (special variable)
• *default-test-function* (special variable)
• *gensym-prefix* (special variable)
• deftest (macro)
• diag (function)
• fail (function)
• is (macro)
• is-condition (macro)
• is-error (macro)
• is-expand (macro)
• is-print (macro)
• is-type (macro)
• is-values (macro)
• isnt (macro)
• like (macro)
• ok (macro)
• pass (function)
• remove-test (function)
• remove-test-all (function)
• run-test (function)
• run-test-all (function)
• run-test-package (function)
• skip (function)
• subtest (macro)

Internal Definitions

• %subtest (function)
• *gensym-alist* (special variable)
• *package-tests* (special variable)
• gensym-tree-equal (generic function)
• gensym-tree-equal (method)
• gensym-tree-equal (method)
• gensymp (function)
• parse-description-and-test (function)
• test (function)
• with-catching-errors (macro)
• with-duration (macro)

4.1.4 prove/src/report.lisp

Parent

src (module)

Location

src/report.lisp

Packages

prove.report

Exported Definitions

• comment-report (class)
• composed-test-report (class)
• error-report-p (function)
• error-test-report (class)
• failed-report-p (function)
• failed-test-report (class)
• normal-test-report (class)
• passed-report-p (function)
• passed-test-report (class)
• report (class)
• skipped-report-p (function)
• skipped-test-report (class)
• test-report (class)
• test-report-p (function)

4.1.5 prove/src/reporter.lisp

Dependencies

• report.lisp (file)
• output.lisp (file)

Parent

src (module)

Location

src/reporter.lisp

Packages

prove.reporter

Exported Definitions

• *indent-level* (special variable)
• format-report (generic function)
• format-report (method)
• format-report (method)
• format/indent (function)
• print-error-report (generic function)
• print-error-report (method)
• print-error-report (method)
• print-finalize-report (generic function)
• print-finalize-report (method)
• print-plan-report (generic function)
• print-plan-report (method)
• print-plan-report (method)
• reporter (class)
• with-additional-indent (macro)

Internal Definitions

• *additional-indent* (special variable)
• *debug-indentation* (special variable)
• find-reporter (function)
• indent (function)

4.1.6 prove/src/reporter-components/tap.lisp

Parent

reporter-components (module)

Location

src/reporter/tap.lisp

Packages

prove.reporter.tap

Exported Definitions

• format-report (method)
• format-report (method)
• format-report (method)
• print-error-report (method)
• print-finalize-report (method)
• print-plan-report (method)

Internal Definitions

tap-reporter (class)

4.1.7 prove/src/reporter-components/fiveam.lisp

Parent

reporter-components (module)

Location

src/reporter/fiveam.lisp

Packages

prove.reporter.fiveam

Exported Definitions

• format-report (method)
• format-report (method)
• print-error-report (method)
• print-error-report (method)
• print-error-report (method)
• print-finalize-report (method)

Internal Definitions

fiveam-reporter (class)

4.1.8 prove/src/reporter-components/list.lisp

Parent

reporter-components (module)

Location

src/reporter/list.lisp

Packages

prove.reporter.list

Exported Definitions

• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• list-reporter (class)
• print-finalize-report (method)
• print-plan-report (method)
• report-expected-line (generic function)
• report-expected-line (method)

Internal Definitions

• escape-tildes (function)
• omit-long-value (function)
• possible-report-description (function)
• print-duration (function)

4.1.9 prove/src/reporter-components/dot.lisp

Dependency

list.lisp (file)

Parent

reporter-components (module)

Location

src/reporter/dot.lisp

Packages

prove.reporter.dot

Exported Definitions

• format-report (method)
• format-report (method)
• print-finalize-report (method)
• print-finalize-report (method)

Internal Definitions

dot-reporter (class)

4.1.10 prove/src/suite.lisp

Dependencies

• output.lisp (file)
• report.lisp (file)
• reporter.lisp (file)
• asdf.lisp (file)

Parent

src (module)

Location

src/suite.lisp

Packages

prove.suite

Exported Definitions

• *default-slow-threshold* (special variable)
• *suite* (special variable)
• add-report (function)
• current-suite (function)
• failed (method)
• (setf failed) (method)
• finalize (function)
• package-suite (class)
• plan (function)
• reports (method)
• (setf reports) (method)
• slow-threshold (function)
• suite (class)
• suite-plan (method)
• (setf suite-plan) (method)
• test-count (method)
• (setf test-count) (method)

Internal Definitions

• *defined-suites* (special variable)
• find-package-suite (function)
• reset-suite (function)

4.1.11 prove/src/asdf.lisp

Dependencies

• output.lisp (file)
• color.lisp (file)

Parent

src (module)

Location

src/asdf.lisp

Packages

prove.asdf

Exported Definitions

• *last-suite-report* (special variable)
• run (function)
• run-test-system (function)
• test-file (class)

Internal Definitions

• *system-test-files* (special variable)
• test-files-in-directory (function)

4.1.12 prove/src/color.lisp

Parent

src (module)

Location

src/color.lisp

Packages

prove.color

Exported Definitions

• *enable-colors* (special variable)
• with-color (macro)

Internal Definitions

with-gray (macro)

4.1.13 prove/src/output.lisp

Parent

src (module)

Location

src/output.lisp

Packages

prove.output

Exported Definitions

• *default-reporter* (special variable)
• *test-result-output* (special variable)

5 Packages

Packages are listed by definition order.

5.1 prove

Source

prove.lisp (file)

Nicknames

• test-more
• cl-test-more

Use List

common-lisp

5.2 prove.test

Source

test.lisp (file)

Use List

common-lisp

Exported Definitions

• *debug-on-error* (special variable)
• *default-test-function* (special variable)
• *gensym-prefix* (special variable)
• deftest (macro)
• diag (function)
• fail (function)
• is (macro)
• is-condition (macro)
• is-error (macro)
• is-expand (macro)
• is-print (macro)
• is-type (macro)
• is-values (macro)
• isnt (macro)
• like (macro)
• ok (macro)
• pass (function)
• remove-test (function)
• remove-test-all (function)
• run-test (function)
• run-test-all (function)
• run-test-package (function)
• skip (function)
• subtest (macro)

Internal Definitions

• %subtest (function)
• *gensym-alist* (special variable)
• *package-tests* (special variable)
• gensym-tree-equal (generic function)
• gensym-tree-equal (method)
• gensym-tree-equal (method)
• gensymp (function)
• parse-description-and-test (function)
• test (function)
• with-catching-errors (macro)
• with-duration (macro)

5.3 prove.report

Source

report.lisp (file)

Use List

common-lisp

Used By List

• prove.reporter.dot
• prove.reporter.list
• prove.reporter.fiveam
• prove.reporter.tap

Exported Definitions

• comment-report (class)
• composed-test-report (class)
• error-report-p (function)
• error-test-report (class)
• failed-report-p (function)
• failed-test-report (class)
• normal-test-report (class)
• passed-report-p (function)
• passed-test-report (class)
• report (class)
• skipped-report-p (function)
• skipped-test-report (class)
• test-report (class)
• test-report-p (function)

5.4 prove.reporter

Source

reporter.lisp (file)

Use List

common-lisp

Used By List

• prove.reporter.dot
• prove.reporter.list
• prove.reporter.fiveam
• prove.reporter.tap

Exported Definitions

• *indent-level* (special variable)
• format-report (generic function)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format/indent (function)
• print-error-report (generic function)
• print-error-report (method)
• print-error-report (method)
• print-error-report (method)
• print-error-report (method)
• print-error-report (method)
• print-error-report (method)
• print-finalize-report (generic function)
• print-finalize-report (method)
• print-finalize-report (method)
• print-finalize-report (method)
• print-finalize-report (method)
• print-finalize-report (method)
• print-finalize-report (method)
• print-plan-report (generic function)
• print-plan-report (method)
• print-plan-report (method)
• print-plan-report (method)
• print-plan-report (method)
• reporter (class)
• with-additional-indent (macro)

Internal Definitions

• *additional-indent* (special variable)
• *debug-indentation* (special variable)
• find-reporter (function)
• indent (function)

5.5 prove.reporter.tap

Source

tap.lisp (file)

Use List

• prove.reporter
• prove.report
• common-lisp

Internal Definitions

tap-reporter (class)

5.6 prove.reporter.fiveam

Source

fiveam.lisp (file)

Use List

• prove.reporter
• prove.report
• common-lisp

Internal Definitions

fiveam-reporter (class)

5.7 prove.reporter.list

Source

list.lisp (file)

Use List

• prove.reporter
• prove.report
• common-lisp

Used By List

prove.reporter.dot

Exported Definitions

• list-reporter (class)
• report-expected-line (generic function)
• report-expected-line (method)

Internal Definitions

• escape-tildes (function)
• omit-long-value (function)
• possible-report-description (function)
• print-duration (function)

5.8 prove.reporter.dot

Source

dot.lisp (file)

Use List

• prove.color
• prove.reporter.list
• prove.reporter
• prove.report
• common-lisp

Internal Definitions

dot-reporter (class)

5.9 prove.suite

Source

suite.lisp (file)

Use List

common-lisp

Exported Definitions

• *default-slow-threshold* (special variable)
• *suite* (special variable)
• add-report (function)
• current-suite (function)
• failed (generic function)
• failed (method)
• (setf failed) (method)
• (setf failed) (generic function)
• finalize (function)
• package-suite (class)
• plan (function)
• reports (generic function)
• reports (method)
• (setf reports) (method)
• (setf reports) (generic function)
• slow-threshold (function)
• suite (class)
• suite-plan (generic function)
• suite-plan (method)
• (setf suite-plan) (method)
• (setf suite-plan) (generic function)
• test-count (generic function)
• test-count (method)
• (setf test-count) (method)
• (setf test-count) (generic function)

Internal Definitions

• *defined-suites* (special variable)
• find-package-suite (function)
• reset-suite (function)

5.10 prove.asdf

Source

asdf.lisp (file)

Nickname

prove-asdf

Use List

• asdf/interface
• common-lisp

Exported Definitions

• *last-suite-report* (special variable)
• run (function)
• run-test-system (function)
• test-file (class)

Internal Definitions

• *system-test-files* (special variable)
• test-files-in-directory (function)

5.11 prove.color

Source

color.lisp (file)

Use List

common-lisp

Used By List

prove.reporter.dot

Exported Definitions

• *enable-colors* (special variable)
• with-color (macro)

Internal Definitions

with-gray (macro)

5.12 prove.output

Source

output.lisp (file)

Use List

common-lisp

Exported Definitions

• *default-reporter* (special variable)
• *test-result-output* (special variable)

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

6.1 Exported definitions

6.1.1 Special variables

Special Variable: *debug-on-error*

Package

prove.test

Source

test.lisp (file)

Special Variable: *default-reporter*

Package

prove.output

Source

output.lisp (file)

Special Variable: *default-slow-threshold*

Package

prove.suite

Source

suite.lisp (file)

Special Variable: *default-test-function*

Package

prove.test

Source

test.lisp (file)

Special Variable: *enable-colors*

Flag whether colorize a test report. The default is T except on Emacs (SLIME).

Package

prove.color

Source

color.lisp (file)

Special Variable: *gensym-prefix*

Package

prove.test

Source

test.lisp (file)

Special Variable: *indent-level*

Level for nested test-cases output.
Number of spaces, added for each indentation level
is described in reporter’s indent-space slot.

Also, macro shift-indent could be used to slightly
indent content inside the main indentation level.

full-indent = indent-space * indent-level + additional-indent

Here is an example of the output:

1| x Blah minor.
2| Next line of description:
3|
4| x Nested test.
5| Also has multiline description.

In this example, indent-space is 4, that is why
text on lines 1 and 4 have 4 spaces between the ’x’ horizontally.

Outputting the first line " x ", reporter sets *additional-indent* to 4. That is why these additional
4 lines are prepended to the rest lines of the main
test case description.

When inner testcase runs, it increments *indent-level*, which shifts output to another 4 spaces (indent-space)
to the right, simultaneously resetting *additional-indent* to zero.

For nested test, reporter writes " x " and again,
sets *additional-indent* to 4 and every other lines now shifted by 1 * 4 + 4 = 8 spaces.

Package

prove.reporter

Source

reporter.lisp (file)

Special Variable: *last-suite-report*

Package

prove.asdf

Source

asdf.lisp (file)

Special Variable: *suite*

Package

prove.suite

Source

suite.lisp (file)

Special Variable: *test-result-output*

Package

prove.output

Source

output.lisp (file)

6.1.2 Macros

Macro: deftest NAME &body TEST-FORMS

Package

prove.test

Source

test.lisp (file)

Macro: is GOT EXPECTED &rest ARGS

Package

prove.test

Source

test.lisp (file)

Macro: is-condition FORM CONDITION &optional DESC

Package

prove.test

Source

test.lisp (file)

Macro: is-error FORM CONDITION &optional DESC

Package

prove.test

Source

test.lisp (file)

Macro: is-expand GOT EXPECTED &optional DESC

Package

prove.test

Source

test.lisp (file)

Macro: is-print GOT EXPECTED &optional DESC

Package

prove.test

Source

test.lisp (file)

Macro: is-type GOT EXPECTED-TYPE &optional DESC

Package

prove.test

Source

test.lisp (file)

Macro: is-values GOT EXPECTED &rest ARGS

Package

prove.test

Source

test.lisp (file)

Macro: isnt GOT EXPECTED &rest ARGS

Package

prove.test

Source

test.lisp (file)

Macro: like GOT REGEX &optional DESC

Package

prove.test

Source

test.lisp (file)

Macro: ok TEST &optional DESC

Package

prove.test

Source

test.lisp (file)

Macro: subtest DESC &body BODY

Package

prove.test

Source

test.lisp (file)

Macro: with-additional-indent (REPORTER STREAM CONTROL-STRING &rest FORMAT-ARGUMENTS) &body BODY

Package

prove.reporter

Source

reporter.lisp (file)

Macro: with-color (COLOR &rest ARGS) &body BODY

Package

prove.color

Source

color.lisp (file)

6.1.3 Functions

Function: add-report REPORT SUITE

Package

prove.suite

Source

suite.lisp (file)

Function: current-suite ()

Package

prove.suite

Source

suite.lisp (file)

Function: diag DESC

Package

prove.test

Source

test.lisp (file)

Function: error-report-p REPORT

Package

prove.report

Source

report.lisp (file)

Function: fail DESC

Package

prove.test

Source

test.lisp (file)

Function: failed-report-p REPORT

Package

prove.report

Source

report.lisp (file)

Function: finalize &optional SUITE

Package

prove.suite

Source

suite.lisp (file)

Function: format/indent REPORTER STREAM CONTROL-STRING &rest FORMAT-ARGUMENTS

Writes a text to given stream with indentation, dictated by *indent-level* and *additional-indent*.

If first line start with ~&, then output will start from a fresh line. Otherwise, all lines except the first one are indented.

Package

prove.reporter

Source

reporter.lisp (file)

Function: pass DESC

Package

prove.test

Source

test.lisp (file)

Function: passed-report-p REPORT

Package

prove.report

Source

report.lisp (file)

Function: plan NUM

Package

prove.suite

Source

suite.lisp (file)

Function: remove-test NAME

Package

prove.test

Source

test.lisp (file)

Function: remove-test-all ()

Package

prove.test

Source

test.lisp (file)

Function: run OBJECT &key REPORTER

Runs a test. OBJECT can be one of a file pathname, a directory pathname or an ASDF system name.
Returns 3 multiple-values, a flag if the tests passed as T or NIL, passed test files as a list and failed test files also as a list.

Example:
(prove:run :myapp-test)
(prove:run #P"myapp/tests/")
(prove:run #P"myapp/tests/01-main.lisp")

Package

prove.asdf

Source

asdf.lisp (file)

Function: run-test NAME

Package

prove.test

Source

test.lisp (file)

Function: run-test-all ()

Package

prove.test

Source

test.lisp (file)

Function: run-test-package PACKAGE-DESIGNATOR

Package

prove.test

Source

test.lisp (file)

Function: run-test-system SYSTEM-DESIGNATOR

Runs a testing ASDF system.

Package

prove.asdf

Source

asdf.lisp (file)

Function: skip HOW-MANY WHY &rest FORMAT-ARGS

Package

prove.test

Source

test.lisp (file)

Function: skipped-report-p REPORT

Package

prove.report

Source

report.lisp (file)

Function: slow-threshold &optional NEW-THRESHOLD

Package

prove.suite

Source

suite.lisp (file)

Function: test-report-p REPORT

Package

prove.report

Source

report.lisp (file)

6.1.4 Generic functions

Generic Function: failed OBJECT

Generic Function: (setf failed) NEW-VALUE OBJECT

Package

prove.suite

Methods

Method: failed (SUITE suite)

automatically generated reader method

Source

suite.lisp (file)

Method: (setf failed) NEW-VALUE (SUITE suite)

automatically generated writer method

Source

suite.lisp (file)

Generic Function: format-report STREAM REPORTER REPORT &rest ARGS &key COUNT

Package

prove.reporter

Source

reporter.lisp (file)

Methods

Method: format-report STREAM (REPORTER dot-reporter) (REPORT test-report) &rest ARGS

Source

dot.lisp (file)

Method: format-report STREAM (REPORTER dot-reporter) (REPORT comment-report) &rest ARGS

Source

dot.lisp (file)

Method: format-report STREAM (REPORTER list-reporter) (REPORT composed-test-report) &rest ARGS

Source

list.lisp (file)

Method: format-report STREAM (REPORTER list-reporter) (REPORT error-test-report) &rest ARGS

Source

list.lisp (file)

Method: format-report STREAM (REPORTER list-reporter) (REPORT failed-test-report) &rest ARGS

Source

list.lisp (file)

Method: format-report STREAM (REPORTER list-reporter) (REPORT skipped-test-report) &rest ARGS

Source

list.lisp (file)

Method: format-report STREAM (REPORTER list-reporter) (REPORT normal-test-report) &rest ARGS

Source

list.lisp (file)

Method: format-report STREAM (REPORTER list-reporter) (REPORT comment-report) &rest ARGS

Source

list.lisp (file)

Method: format-report STREAM (REPORTER fiveam-reporter) (REPORT test-report) &rest ARGS

Source

fiveam.lisp (file)

Method: format-report STREAM (REPORTER fiveam-reporter) (REPORT comment-report) &rest ARGS

Source

fiveam.lisp (file)

Method: format-report STREAM (REPORTER tap-reporter) (REPORT skipped-test-report) &key COUNT

Source

tap.lisp (file)

Method: format-report STREAM (REPORTER tap-reporter) (REPORT test-report) &key COUNT

Source

tap.lisp (file)

Method: format-report STREAM (REPORTER tap-reporter) (REPORT comment-report) &rest ARGS

Source

tap.lisp (file)

Method: format-report STREAM (REPORTER null) (REPORT report) &rest ARGS

Method: format-report STREAM (REPORTER reporter) (REPORT report) &rest ARGS

Generic Function: print-error-report REPORTER REPORT STREAM

Package

prove.reporter

Source

reporter.lisp (file)

Methods

Method: print-error-report (REPORTER fiveam-reporter) (REPORT comment-report) STREAM

Source

fiveam.lisp (file)

Method: print-error-report (REPORTER fiveam-reporter) (REPORT composed-test-report) STREAM

Source

fiveam.lisp (file)

Method: print-error-report (REPORTER fiveam-reporter) (REPORT failed-test-report) STREAM

Source

fiveam.lisp (file)

Method: print-error-report (REPORTER tap-reporter) (REPORT failed-test-report) STREAM

Source

tap.lisp (file)

Method: print-error-report (REPORTER reporter) (REPORT report) STREAM

Method: print-error-report (REPORTER null) (REPORT test-report) STREAM

Generic Function: print-finalize-report REPORTER PLAN REPORTS STREAM

Package

prove.reporter

Source

reporter.lisp (file)

Methods

Method: print-finalize-report (REPORTER dot-reporter) PLAN REPORTS STREAM after

Source

dot.lisp (file)

Method: print-finalize-report (REPORTER dot-reporter) PLAN REPORTS STREAM before

Source

dot.lisp (file)

Method: print-finalize-report (REPORTER list-reporter) PLAN REPORTS STREAM

Source

list.lisp (file)

Method: print-finalize-report (REPORTER fiveam-reporter) PLAN REPORTS STREAM

Source

fiveam.lisp (file)

Method: print-finalize-report (REPORTER tap-reporter) PLAN REPORTS STREAM

Source

tap.lisp (file)

Method: print-finalize-report (REPORTER null) PLAN REPORTS STREAM

Generic Function: print-plan-report REPORTER NUM STREAM

Package

prove.reporter

Source

reporter.lisp (file)

Methods

Method: print-plan-report (REPORTER list-reporter) NUM STREAM

Source

list.lisp (file)

Method: print-plan-report (REPORTER tap-reporter) NUM STREAM

Source

tap.lisp (file)

Method: print-plan-report (REPORTER null) NUM STREAM

Method: print-plan-report REPORTER NUM STREAM

Generic Function: report-expected-line REPORT

Reports about failed or passed test.
Should return a string with description of what have happened.

Package

prove.reporter.list

Source

list.lisp (file)

Methods

Method: report-expected-line (REPORT normal-test-report)

Generic Function: reports OBJECT

Generic Function: (setf reports) NEW-VALUE OBJECT

Package

prove.suite

Methods

Method: reports (SUITE suite)

automatically generated reader method

Source

suite.lisp (file)

Method: (setf reports) NEW-VALUE (SUITE suite)

automatically generated writer method

Source

suite.lisp (file)

Generic Function: suite-plan OBJECT

Generic Function: (setf suite-plan) NEW-VALUE OBJECT

Package

prove.suite

Methods

Method: suite-plan (SUITE suite)

automatically generated reader method

Source

suite.lisp (file)

Method: (setf suite-plan) NEW-VALUE (SUITE suite)

automatically generated writer method

Source

suite.lisp (file)

Generic Function: test-count OBJECT

Generic Function: (setf test-count) NEW-VALUE OBJECT

Package

prove.suite

Methods

Method: test-count (SUITE suite)

automatically generated reader method

Source

suite.lisp (file)

Method: (setf test-count) NEW-VALUE (SUITE suite)

automatically generated writer method

Source

suite.lisp (file)

6.1.5 Classes

Class: comment-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

report (class)

Direct methods

• format-report (method)
• format-report (method)
• print-error-report (method)
• format-report (method)
• format-report (method)

Class: composed-test-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

test-report (class)

Direct methods

• format-report (method)
• print-error-report (method)

Direct slots

Slot: plan

Initargs

:plan

Slot: children

Initargs

:children

Initform

(make-array 0 :adjustable t :fill-pointer 0)

Class: error-test-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

failed-test-report (class)

Direct methods

format-report (method)

Class: failed-test-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

normal-test-report (class)

Direct subclasses

error-test-report (class)

Direct methods

• format-report (method)
• print-error-report (method)
• print-error-report (method)

Class: list-reporter ()

Package

prove.reporter.list

Source

list.lisp (file)

Direct superclasses

reporter (class)

Direct subclasses

dot-reporter (class)

Direct methods

• print-finalize-report (method)
• print-plan-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)
• format-report (method)

Class: normal-test-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

test-report (class)

Direct subclasses

• passed-test-report (class)
• failed-test-report (class)
• skipped-test-report (class)

Direct methods

• format-report (method)
• report-expected-line (method)
• print-object (method)

Direct slots

Slot: test-function

Type

(or function symbol)

Initargs

:test-function

Slot: notp

Type

boolean

Initargs

:notp

Slot: got

Initargs

:got

Initform

(error ":got is required")

Slot: got-form

Initargs

:got-form

Initform

(quote #:unbound)

Slot: expected

Initargs

:expected

Initform

(error ":expected is required")

Slot: report-expected-label

Type

(or null string)

Initargs

:report-expected-label

Class: package-suite ()

Package

prove.suite

Source

suite.lisp (file)

Direct superclasses

suite (class)

Class: passed-test-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

normal-test-report (class)

Class: report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

• comment-report (class)
• test-report (class)

Direct methods

• print-error-report (method)
• format-report (method)
• format-report (method)

Direct slots

Slot: description

Type

(or null string)

Initargs

:description

Class: reporter ()

Package

prove.reporter

Source

reporter.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

• tap-reporter (class)
• fiveam-reporter (class)
• list-reporter (class)

Direct methods

• print-error-report (method)
• format-report (method)

Direct slots

Slot: indent-space

Initform

2

Class: skipped-test-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

normal-test-report (class)

Direct methods

• format-report (method)
• format-report (method)

Class: suite ()

Package

prove.suite

Source

suite.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

package-suite (class)

Direct methods

• reports (method)
• reports (method)
• failed (method)
• failed (method)
• test-count (method)
• test-count (method)
• suite-plan (method)
• suite-plan (method)

Direct slots

Slot: plan

Initargs

:plan

Initform

:unspecified

Readers

suite-plan (generic function)

Writers

(setf suite-plan) (generic function)

Slot: slow-threshold

Initargs

:slow-threshold

Initform

prove.suite:*default-slow-threshold*

Slot: test-count

Initform

0

Readers

test-count (generic function)

Writers

(setf test-count) (generic function)

Slot: failed

Initform

0

Readers

failed (generic function)

Writers

(setf failed) (generic function)

Slot: reports

Initform

(make-array 0 :adjustable t :fill-pointer 0)

Readers

reports (generic function)

Writers

(setf reports) (generic function)

Class: test-file ()

Package

prove.asdf

Source

asdf.lisp (file)

Direct superclasses

cl-source-file (class)

Direct methods

• perform (method)
• compute-action-stamp (method)
• perform (method)

Class: test-report ()

Package

prove.report

Source

report.lisp (file)

Direct superclasses

report (class)

Direct subclasses

• normal-test-report (class)
• composed-test-report (class)

Direct methods

• format-report (method)
• format-report (method)
• format-report (method)
• print-error-report (method)

Direct slots

Slot: duration

Initargs

:duration

Slot: slow-threshold

Initargs

:slow-threshold

Slot: print-error-detail

Type

boolean

Initargs

:print-error-detail

Initform

t

6.2 Internal definitions

6.2.1 Special variables

Special Variable: *additional-indent*

Number of spaces to add to each line. see *indent-level* docstring for full description.

Package

prove.reporter

Source

reporter.lisp (file)

Special Variable: *debug-indentation*

If True, then indentation will have ’=’ and ’-’ symbols for main indentaion and additional, instead of spaces.

Package

prove.reporter

Source

reporter.lisp (file)

Special Variable: *defined-suites*

Package

prove.suite

Source

suite.lisp (file)

Special Variable: *gensym-alist*

Package

prove.test

Source

test.lisp (file)

Special Variable: *package-tests*

Package

prove.test

Source

test.lisp (file)

Special Variable: *system-test-files*

Package

prove.asdf

Source

asdf.lisp (file)

6.2.2 Macros

Macro: with-catching-errors (&key DESCRIPTION EXPECTED) &body BODY

Package

prove.test

Source

test.lisp (file)

Macro: with-duration ((DURATION RESULT) FORM) &body BODY

Package

prove.test

Source

test.lisp (file)

Macro: with-gray STREAM &body BODY

Package

prove.color

Source

color.lisp (file)

6.2.3 Functions

Function: %subtest DESC BODY-FN

Package

prove.test

Source

test.lisp (file)

Function: escape-tildes TEXT

Package

prove.reporter.list

Source

list.lisp (file)

Function: find-package-suite PACKAGE-DESIGNATOR

Package

prove.suite

Source

suite.lisp (file)

Function: find-reporter NAME

Package

prove.reporter

Source

reporter.lisp (file)

Function: gensymp VAL

Package

prove.test

Source

test.lisp (file)

Function: indent SPACE &optional COUNT

Creates a string with a number of spaces to indent new line of a test report.

Package

prove.reporter

Source

reporter.lisp (file)

Function: omit-long-value VALUE

Package

prove.reporter.list

Source

list.lisp (file)

Function: parse-description-and-test ARGS

Package

prove.test

Source

test.lisp (file)

Function: possible-report-description REPORT

Package

prove.reporter.list

Source

list.lisp (file)

Function: print-duration STREAM DURATION &optional SLOW-THRESHOLD

Package

prove.reporter.list

Source

list.lisp (file)

Function: reset-suite SUITE

Package

prove.suite

Source

suite.lisp (file)

Function: test GOT EXPECTED ARGS &key NOTP DURATION GOT-FORM TEST-FN PASSED-REPORT-CLASS FAILED-REPORT-CLASS REPORT-EXPECTED-LABEL PRINT-ERROR-DETAIL OUTPUT

Package

prove.test

Source

test.lisp (file)

Function: test-files-in-directory DIRECTORY

Package

prove.asdf

Source

asdf.lisp (file)

6.2.4 Generic functions

Generic Function: gensym-tree-equal X Y

Package

prove.test

Source

test.lisp (file)

Methods

Method: gensym-tree-equal X Y

Method: gensym-tree-equal (X cons) (Y cons)

6.2.5 Classes

Class: dot-reporter ()

Package

prove.reporter.dot

Source

dot.lisp (file)

Direct superclasses

list-reporter (class)

Direct methods

• print-finalize-report (method)
• print-finalize-report (method)
• format-report (method)
• format-report (method)

Class: fiveam-reporter ()

Package

prove.reporter.fiveam

Source

fiveam.lisp (file)

Direct superclasses

reporter (class)

Direct methods

• print-finalize-report (method)
• print-error-report (method)
• print-error-report (method)
• print-error-report (method)
• format-report (method)
• format-report (method)

Class: tap-reporter ()

Package

prove.reporter.tap

Source

tap.lisp (file)

Direct superclasses

reporter (class)

Direct methods

• print-finalize-report (method)
• print-plan-report (method)
• print-error-report (method)
• format-report (method)
• format-report (method)
• format-report (method)

Direct slots

Slot: indent-space

Initform

4

Appendix A Indexes

A.1 Concepts

A.2 Functions