This is the nst Reference Manual, version 4.1.2, generated automatically by Declt version 4.0 beta 2 "William Riker" on Sun Dec 15 05:12:25 2024 GMT+0.
nst/nst.asd
nst/core/package.lisp
nst/core/utils.lisp
nst/core/errors.lisp
nst/core/interrupt.lisp
nst/core/permuter.lisp
nst/core/globals.lisp
nst/core/context.lisp
nst/core/artifacts.lisp
nst/core/group.lisp
nst/core/check.lisp
nst/core/fixture.lisp
nst/core/test-def.lisp
nst/core/status.lisp
nst/core/runner.lisp
nst/core/criteria.lisp
nst/core/process.lisp
nst/core/command.lisp
nst/core/xml.lisp
nst/core/junit.lisp
nst/core/sampling.lisp
nst/core/method.lisp
nst/core/interfaces.lisp
The main system appears first, followed by any subsystem dependency.
nst
The NST unit/regression testing system
John Maraist <lisper@maraist.org>
LLGPL 3.latest
4.1.2
closer-mop
(system).
org-sampler
(system).
core
(module).
Modules are listed depth-first from the system components tree.
nst/core
nst
(system).
package.lisp
(file).
utils.lisp
(file).
errors.lisp
(file).
interrupt.lisp
(file).
permuter.lisp
(file).
globals.lisp
(file).
context.lisp
(file).
artifacts.lisp
(file).
group.lisp
(file).
check.lisp
(file).
fixture.lisp
(file).
test-def.lisp
(file).
status.lisp
(file).
runner.lisp
(file).
criteria.lisp
(file).
process.lisp
(file).
command.lisp
(file).
xml.lisp
(file).
junit.lisp
(file).
sampling.lisp
(file).
method.lisp
(file).
interfaces.lisp
(file).
Files are sorted by type and then listed depth-first from the systems components trees.
nst/nst.asd
nst/core/package.lisp
nst/core/utils.lisp
nst/core/errors.lisp
nst/core/interrupt.lisp
nst/core/permuter.lisp
nst/core/globals.lisp
nst/core/context.lisp
nst/core/artifacts.lisp
nst/core/group.lisp
nst/core/check.lisp
nst/core/fixture.lisp
nst/core/test-def.lisp
nst/core/status.lisp
nst/core/runner.lisp
nst/core/criteria.lisp
nst/core/process.lisp
nst/core/command.lisp
nst/core/xml.lisp
nst/core/junit.lisp
nst/core/sampling.lisp
nst/core/method.lisp
nst/core/interfaces.lisp
nst/nst.asd
nst/core/utils.lisp
package.lisp
(file).
core
(module).
cdr-or-nil
(function).
def-hashtable-fns
(macro).
eql-for-sigdigits
(function).
log10
(macro).
named-function
(macro).
no-effect
(function).
sig-place
(function).
symbol-or-car
(function).
nst/core/errors.lisp
package.lisp
(file).
core
(module).
actual-value
(reader method).
criterion-missing-mandatory-argument
(condition).
criterion-name
(reader method).
debug-for-fail
(condition).
define-nst-error
(macro).
expected-form
(reader method).
group
(reader method).
group
(reader method).
no-nst-groups-in-package
(condition).
no-such-nst-group
(condition).
no-such-nst-test
(condition).
not-expected-form
(condition).
nst-deprecation-warning-mixin
(condition).
nst-error
(condition).
nst-hard-deprecation
(condition).
nst-soft-deprecation
(condition).
nst-soft-keyarg-deprecation
(condition).
old-name
(reader method).
package-of
(reader method).
replacement
(reader method).
required-keyarg
(reader method).
soft-dep-warning
(function).
test
(reader method).
with-nst-control-handlers
(macro).
with-retry
(macro).
nst/core/interrupt.lisp
package.lisp
(file).
core
(module).
+keyboard-interrupt-class+
(constant).
handler-bind-interruptable
(macro).
nst/core/permuter.lisp
package.lisp
(file).
core
(module).
initialize-instance
(method).
print-object
(method).
fmt-permuter
(generic function).
get-unassigned
(function).
has-next
(reader method).
next-permutation
(function).
permuter
(class).
permuter-frame
(class).
qw
(special variable).
qwerty
(special variable).
relax-stack
(function).
tighten-stack
(function).
whittlable
(function).
whittle-relaxed-stack
(function).
nst/core/globals.lisp
errors.lisp
(file).
utils.lisp
(file).
core
(module).
*debug-on-error*
(special variable).
*debug-on-fail*
(special variable).
*default-report-verbosity*
(special variable).
*nst-output-stream*
(special variable).
*default-debug-config*
(special variable).
*default-debug-protect*
(special variable).
*generate-backtraces*
(special variable).
*nst-check-internal-name*
(special variable).
*nst-check-user-name*
(special variable).
*nst-context*
(special variable).
*nst-context-evaluable*
(special variable).
*nst-debug*
(special variable).
*nst-group-name*
(special variable).
*nst-group-shown*
(special variable).
*nst-report-driver*
(special variable).
*nst-stack*
(special variable).
*nst-verbosity*
(special variable).
apply-debug-options
(macro).
apply-default-debug-options
(macro).
assemble-protected-option-values
(function).
at-verbosity
(macro).
base-name
(generic function).
format-at-verbosity
(macro).
nst-repl-property-display
(generic function).
nst-repl-property-doc
(generic function).
nst-repl-property-encode
(generic function).
princ-filled-text
(function).
protect-nst-config
(macro).
restore-protected-option-values
(function).
run-debug-options
(function).
set-nst-property
(generic function).
show-nst-property
(generic function).
with-output-for-verbosity
(macro).
nst/core/context.lisp
package.lisp
(file).
globals.lisp
(file).
core
(module).
context-layer
(class).
criterion
(reader method).
(setf criterion)
(writer method).
criterion-args
(reader method).
(setf criterion-args)
(writer method).
criterion-context-layer
(class).
get-display-context-layers
(function).
get-local-criterion-context
(function).
given-stack
(reader method).
(setf given-stack)
(writer method).
show-context-layer
(generic function).
with-context-layer
(macro).
with-criterion-context-layer
(macro).
nst/core/artifacts.lisp
globals.lisp
(file).
core
(module).
+name-uses+
(special variable).
+results-record+
(special variable).
all-uses
(function).
artifact-recorder
(function).
blurb-one-name-use
(function).
copy-name-use
(function).
copy-result-stats
(function).
executable-uses
(function).
make-name-use
(function).
make-result-stats
(function).
name-use
(function).
name-use
(structure).
name-use-fixtures
(reader).
(setf name-use-fixtures)
(writer).
name-use-groups
(reader).
(setf name-use-groups)
(writer).
name-use-last-use
(reader).
(setf name-use-last-use)
(writer).
name-use-name
(reader).
(setf name-use-name)
(writer).
name-use-p
(function).
name-use-package-p
(reader).
(setf name-use-package-p)
(writer).
name-use-tests
(reader).
(setf name-use-tests)
(writer).
note-name-invocation
(function).
record-name-use
(function).
record-package-use
(function).
result-stats
(structure).
result-stats-elapsed-time
(reader).
(setf result-stats-elapsed-time)
(writer).
result-stats-erring
(reader).
(setf result-stats-erring)
(writer).
result-stats-failing
(reader).
(setf result-stats-failing)
(writer).
result-stats-p
(function).
result-stats-passing
(reader).
(setf result-stats-passing)
(writer).
result-stats-tests
(reader).
(setf result-stats-tests)
(writer).
result-stats-timestamp
(reader).
(setf result-stats-timestamp)
(writer).
result-stats-warning
(reader).
(setf result-stats-warning)
(writer).
trace-group
(generic function).
trace-results
(function).
trace-test
(generic function).
nst/core/group.lisp
artifacts.lisp
(file).
globals.lisp
(file).
core
(module).
def-test-group
(macro).
*group-records*
(special variable).
base-name
(method).
copy-group-record
(function).
group-record
(function).
(setf group-record)
(function).
group-record
(structure).
group-record-anon-fixture-forms
(reader).
(setf group-record-anon-fixture-forms)
(writer).
group-record-aspirational
(reader).
(setf group-record-aspirational)
(writer).
group-record-documentation
(reader).
(setf group-record-documentation)
(writer).
group-record-eachtest-cleanup-thunk
(reader).
(setf group-record-eachtest-cleanup-thunk)
(writer).
group-record-eachtest-setup-thunk
(reader).
(setf group-record-eachtest-setup-thunk)
(writer).
group-record-fixtures-cleanup-thunk
(reader).
(setf group-record-fixtures-cleanup-thunk)
(writer).
group-record-fixtures-setup-thunk
(reader).
(setf group-record-fixtures-setup-thunk)
(writer).
group-record-given-fixtures
(reader).
(setf group-record-given-fixtures)
(writer).
group-record-include-groups
(reader).
(setf group-record-include-groups)
(writer).
group-record-name
(reader).
(setf group-record-name)
(writer).
group-record-p
(function).
group-record-tests
(reader).
(setf group-record-tests)
(writer).
group-record-withfixtures-cleanup-thunk
(reader).
(setf group-record-withfixtures-cleanup-thunk)
(writer).
group-record-withfixtures-setup-thunk
(reader).
(setf group-record-withfixtures-setup-thunk)
(writer).
make-group-record
(function).
package-groups
(function).
pull-test-name-list
(function).
separate-group-subforms
(function).
test-record
(function).
(setf test-record)
(function).
trace-group
(method).
nst/core/check.lisp
context.lisp
(file).
interrupt.lisp
(file).
core
(module).
check-criterion-on-form
(function).
check-criterion-on-value
(function).
def-criterion
(macro).
def-criterion-alias
(macro).
def-criterion-unevaluated
(macro).
def-form-criterion
(macro).
def-values-criterion
(macro).
apply-criterion
(generic function).
build-continue-check-expr
(function).
continue-check
(function).
decompose-arg-values-lambda-list
(function).
def-check-alias
(macro).
def-control-check
(macro).
def-value-check
(macro).
extract-parameters
(function).
returning-criterion-config-error
(macro).
returning-test-error
(macro).
nst/core/fixture.lisp
errors.lisp
(file).
globals.lisp
(file).
core
(module).
def-fixtures
(macro).
with-fixtures
(macro).
*fixture-bindings*
(special variable).
*fixture-letlist*
(special variable).
*fixture-record*
(special variable).
apply-fixture
(function).
apply-fixtures
(function).
base-name
(method).
copy-fixture-record
(function).
decode-caching-decls
(function).
decode-fixture-syntax
(function).
fixture-bindings
(function).
(setf fixture-bindings)
(function).
fixture-letlist
(function).
(setf fixture-letlist)
(function).
fixture-record
(function).
(setf fixture-record)
(function).
fixture-record
(structure).
fixture-record-bindings-list
(reader).
(setf fixture-record-bindings-list)
(writer).
fixture-record-bound-names
(reader).
(setf fixture-record-bound-names)
(writer).
fixture-record-cache-flush
(reader).
(setf fixture-record-cache-flush)
(writer).
fixture-record-documentation
(reader).
(setf fixture-record-documentation)
(writer).
fixture-record-function
(reader).
(setf fixture-record-function)
(writer).
fixture-record-name
(reader).
(setf fixture-record-name)
(writer).
fixture-record-p
(function).
flush-fixture-cache
(function).
make-fixture-record
(function).
open-fixture
(function).
nst/core/test-def.lisp
errors.lisp
(file).
globals.lisp
(file).
group.lisp
(file).
core
(module).
def-test
(macro).
base-name
(method).
copy-test-record
(function).
decode-defcheck-name-and-args
(function).
def-check
(macro).
make-test-record
(function).
test-is-aspirational
(function).
test-record
(structure).
test-record-aspirational
(reader).
(setf test-record-aspirational)
(writer).
test-record-aspirational-supp
(reader).
(setf test-record-aspirational-supp)
(writer).
test-record-cleanup
(reader).
(setf test-record-cleanup)
(writer).
test-record-criterion
(reader).
(setf test-record-criterion)
(writer).
test-record-documentation
(reader).
(setf test-record-documentation)
(writer).
test-record-finish
(reader).
(setf test-record-finish)
(writer).
test-record-fixtures
(reader).
(setf test-record-fixtures)
(writer).
test-record-forms
(reader).
(setf test-record-forms)
(writer).
test-record-group
(reader).
(setf test-record-group)
(writer).
test-record-name
(reader).
(setf test-record-name)
(writer).
test-record-p
(function).
test-record-results
(reader).
(setf test-record-results)
(writer).
test-record-setup
(reader).
(setf test-record-setup)
(writer).
test-record-special-fixture-names
(reader).
(setf test-record-special-fixture-names)
(writer).
test-record-startup
(reader).
(setf test-record-startup)
(writer).
trace-test
(method).
nst/core/status.lisp
globals.lisp
(file).
artifacts.lisp
(file).
check.lisp
(file).
core
(module).
add-error
(function).
add-failure
(function).
add-info
(function).
add-warning
(generic function).
emit-failure
(macro).
emit-success
(macro).
emit-warning
(macro).
make-error-report
(function).
make-failure-report
(function).
make-success-report
(function).
make-warning-report
(function).
wrap-thrown-lisp-warning
(function).
%make-check-result
(function).
add-thrown-error
(function).
all-groups-report
(function).
all-package-report
(function).
all-tests-report
(function).
apply-check-note-formatter
(function).
apply-formatter
(function).
calibrate-check-result
(function).
check-note
(structure).
check-note-args
(reader).
(setf check-note-args)
(writer).
check-note-context
(reader).
(setf check-note-context)
(writer).
check-note-format
(reader).
(setf check-note-format)
(writer).
check-note-p
(function).
check-note-stack
(reader).
(setf check-note-stack)
(writer).
check-note-type-string
(generic function).
check-result
(macro).
check-result
(structure).
check-result-check-name
(reader).
(setf check-result-check-name)
(writer).
check-result-elapsed-time
(function).
(setf check-result-elapsed-time)
(function).
check-result-erring
(function).
(setf check-result-erring)
(function).
check-result-errors
(reader).
(setf check-result-errors)
(writer).
check-result-failing
(function).
(setf check-result-failing)
(function).
check-result-failures
(reader).
(setf check-result-failures)
(writer).
check-result-group-name
(reader).
(setf check-result-group-name)
(writer).
check-result-info
(reader).
(setf check-result-info)
(writer).
check-result-p
(function).
check-result-passing
(function).
(setf check-result-passing)
(function).
check-result-skipped
(reader).
(setf check-result-skipped)
(writer).
check-result-tests
(reader).
(setf check-result-tests)
(writer).
check-result-timestamp
(function).
(setf check-result-timestamp)
(function).
check-result-warning
(function).
(setf check-result-warning)
(function).
check-result-warnings
(reader).
(setf check-result-warnings)
(writer).
copy-check-note
(function).
copy-check-result
(function).
copy-error-check-note
(function).
copy-group-result
(function).
copy-multi-results
(function).
copy-package-result
(function).
count-nonnulls
(macro).
emit-error
(macro).
error-check-note
(structure).
error-check-note-args
(function).
(setf error-check-note-args)
(function).
error-check-note-context
(function).
(setf error-check-note-context)
(function).
error-check-note-error
(reader).
(setf error-check-note-error)
(writer).
error-check-note-format
(function).
(setf error-check-note-format)
(function).
error-check-note-p
(function).
error-check-note-stack
(function).
(setf error-check-note-stack)
(function).
finish-multiple-report
(function).
fixture-binding-error-note
(function).
format-for-warning
(generic function).
get-report-from-names
(function).
group-report
(function).
group-result
(structure).
group-result-check-results
(reader).
(setf group-result-check-results)
(writer).
group-result-elapsed-time
(function).
(setf group-result-elapsed-time)
(function).
group-result-erring
(function).
(setf group-result-erring)
(function).
group-result-failing
(function).
(setf group-result-failing)
(function).
group-result-group-name
(reader).
(setf group-result-group-name)
(writer).
group-result-p
(function).
group-result-passing
(function).
(setf group-result-passing)
(function).
group-result-tests
(function).
(setf group-result-tests)
(function).
group-result-timestamp
(function).
(setf group-result-timestamp)
(function).
group-result-warning
(function).
(setf group-result-warning)
(function).
interesting-result-p
(function).
make-and-calibrate-check-result
(function).
make-check-note
(function).
make-check-result
(function).
make-config-error
(function).
make-error-check-note
(function).
make-error-note
(function).
make-group-result
(function).
make-multi-results
(function).
make-package-result
(function).
make-skipped-report
(function).
multi-results
(structure).
multi-results-elapsed-time
(function).
(setf multi-results-elapsed-time)
(function).
multi-results-erring
(function).
(setf multi-results-erring)
(function).
multi-results-failing
(function).
(setf multi-results-failing)
(function).
multi-results-group-reports
(reader).
(setf multi-results-group-reports)
(writer).
multi-results-p
(function).
multi-results-package-reports
(reader).
(setf multi-results-package-reports)
(writer).
multi-results-passing
(function).
(setf multi-results-passing)
(function).
multi-results-stats-source
(reader).
(setf multi-results-stats-source)
(writer).
multi-results-system
(reader).
(setf multi-results-system)
(writer).
multi-results-test-reports
(reader).
(setf multi-results-test-reports)
(writer).
multi-results-tests
(function).
(setf multi-results-tests)
(function).
multi-results-timestamp
(function).
(setf multi-results-timestamp)
(function).
multi-results-warning
(function).
(setf multi-results-warning)
(function).
multiple-report
(function).
nst-dump
(function).
package-report
(function).
package-result
(structure).
package-result-elapsed-time
(function).
(setf package-result-elapsed-time)
(function).
package-result-erring
(function).
(setf package-result-erring)
(function).
package-result-failing
(function).
(setf package-result-failing)
(function).
package-result-group-results
(reader).
(setf package-result-group-results)
(writer).
package-result-p
(function).
package-result-package-name
(reader).
(setf package-result-package-name)
(writer).
package-result-passing
(function).
(setf package-result-passing)
(function).
package-result-tests
(function).
(setf package-result-tests)
(function).
package-result-timestamp
(function).
(setf package-result-timestamp)
(function).
package-result-warning
(function).
(setf package-result-warning)
(function).
report-details
(function).
report-group
(function).
report-interesting
(function).
report-multiple
(function).
report-package
(function).
report-summary
(function).
report-test
(function).
result-summary
(generic function).
test-report
(function).
use-stats-from
(function).
nst/core/runner.lisp
interrupt.lisp
(file).
test-def.lisp
(file).
group.lisp
(file).
status.lisp
(file).
core
(module).
*implicit-group-choice*
(special variable).
core-run-test
(function).
get-test-record-specials
(function).
run-group
(function).
run-group-record
(function).
run-group-test
(function).
run-group-tests
(function).
run-group-tests-fixtures-thunk
(function).
run-package
(function).
run-test
(function).
run-test-fixtures-thunk
(function).
run-test-inst
(function).
wrap-fixture-name-specials
(function).
nst/core/criteria.lisp
errors.lisp
(file).
interrupt.lisp
(file).
permuter.lisp
(file).
check.lisp
(file).
status.lisp
(file).
core
(module).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
apply-criterion
(method).
refine-package-symbol-desigs
(function).
nst/core/process.lisp
errors.lisp
(file).
interrupt.lisp
(file).
check.lisp
(file).
status.lisp
(file).
core
(module).
*assert-eq-format-string*
(special variable).
*assert-eql-format-string*
(special variable).
*assert-equal-format-string*
(special variable).
*assert-equalp-format-string*
(special variable).
*assert-nonnil-format-string*
(special variable).
*assert-not-eq-format-string*
(special variable).
*assert-not-eql-format-string*
(special variable).
*assert-not-equal-format-string*
(special variable).
*assert-not-equalp-format-string*
(special variable).
*assert-null-format-string*
(special variable).
*assert-zero-format-string*
(special variable).
assert-criterion
(macro).
assert-eq
(function).
assert-eql
(function).
assert-equal
(function).
assert-equalp
(function).
assert-non-nil
(function).
assert-not-eq
(function).
assert-not-eql
(function).
assert-not-equal
(function).
assert-not-equalp
(function).
assert-null
(function).
assert-zero
(function).
def-binary-negated-predicate-assert
(macro).
def-binary-predicate-assert
(macro).
def-eval-test
(macro).
def-unary-negated-predicate-assert
(macro).
def-unary-predicate-assert
(macro).
apply-criterion
(method).
apply-criterion
(method).
assert-criterion-fn
(function).
assertion-failure-args
(reader method).
assertion-failure-fatal
(reader method).
assertion-failure-formatter
(reader method).
nst-assertion-condition
(condition).
nst-assertion-failure
(condition).
nst-assertion-result
(reader method).
nst-assertion-result-check
(condition).
nst/core/command.lisp
globals.lisp
(file).
status.lisp
(file).
core
(module).
nst-cmd
(macro).
*last-repl-call*
(special variable).
+nst-repl-commands+
(special variable).
+nst-repl-properties+
(special variable).
consider-repl-call-save
(generic function).
def-nst-interactive-command
(macro).
def-nst-property
(macro).
flag-filter
(function).
nst-arg-names
(generic function).
nst-long-help
(generic function).
nst-repl-property-display
(method).
nst-repl-property-display
(method).
nst-repl-property-display
(method).
nst-repl-property-doc
(method).
nst-repl-property-doc
(method).
nst-repl-property-doc
(method).
nst-repl-property-encode
(method).
nst-repl-property-encode
(method).
nst-repl-property-encode
(method).
nst-short-help
(generic function).
prep-arg-names-help
(generic function).
run-command-actual
(generic function).
run-nst-command
(function).
set-nst-property
(method).
set-nst-property
(method).
set-nst-property
(method).
show-nst-property
(method).
show-nst-property
(method).
show-nst-property
(method).
nst/core/xml.lisp
status.lisp
(file).
core
(module).
*default-xml-pprint-dispatch*
(special variable).
check-note-type-string
(method).
elapsed-time-to-string
(function).
junit-header
(function).
pprint-cdata-string
(function).
pprint-xml
(function).
symbol-to-junit-name
(function).
timestamp-to-string
(function).
with-pprint-cdata
(macro).
with-xml-tagged-pprint-logical-block
(macro).
nst/core/junit.lisp
xml.lisp
(file).
core
(module).
junit-results-by-group
(function).
nst-junit-dump
(generic function).
*char-escapes*
(special variable).
*entities*
(special variable).
junit-group-result
(generic function).
nst-xml-dump
(function).
string-escaped
(function).
nst/core/sampling.lisp
check.lisp
(file).
core
(module).
*max-compound-structure-depth*
(special variable).
arbitrary
(generic function).
compound-structure
(macro).
def-arbitrary-instance-type
(macro).
*current-compound-structure-depth*
(special variable).
*default-character-range*
(special variable).
*max-auto-array-rank*
(special variable).
+arbitrary-generable-types+
(special variable).
+scalar-generable-types+
(special variable).
apply-criterion
(method).
arbitrary-by-spec
(generic function).
arbitrary-generable-types
(function).
arbitrary-grounded-type
(generic function).
coin-flip
(macro).
domain
(reader method).
generate-sample
(function).
keyword-args
(reader method).
low-biased-natnum
(function).
mid-biased-natnum
(function).
pick-from-sequence
(function).
unknown-arbitrary-domain
(condition).
nst/core/method.lisp
status.lisp
(file).
check.lisp
(file).
core
(module).
(setf class-name)
(writer method).
class-name
(reader method).
def-test-generic
(macro).
def-test-method
(macro).
def-test-method-criterion
(macro).
nst-results
(method combination).
*test-methods*
(special variable).
apply-criterion
(method).
check-result-union
(function).
collect-test-generics
(function).
decode-def-test-generic-body
(function).
invoke-test-methods
(function).
method-context-layer
(class).
method-name
(reader method).
(setf method-name)
(writer method).
object
(reader method).
(setf object)
(writer method).
show-context-layer
(method).
with-method-context-layer
(macro).
nst/core/interfaces.lisp
check.lisp
(file).
status.lisp
(file).
core
(module).
def-bundle-package
(macro).
Packages are listed by definition order.
nst
Unit and regression testing for Common Lisp
common-lisp
.
*assert-eq-format-string*
(special variable).
*assert-eql-format-string*
(special variable).
*assert-equal-format-string*
(special variable).
*assert-equalp-format-string*
(special variable).
*assert-nonnil-format-string*
(special variable).
*assert-not-eq-format-string*
(special variable).
*assert-not-eql-format-string*
(special variable).
*assert-not-equal-format-string*
(special variable).
*assert-not-equalp-format-string*
(special variable).
*assert-null-format-string*
(special variable).
*assert-zero-format-string*
(special variable).
*debug-on-error*
(special variable).
*debug-on-fail*
(special variable).
*default-report-verbosity*
(special variable).
*max-compound-structure-depth*
(special variable).
*nst-output-stream*
(special variable).
add-error
(function).
add-failure
(function).
add-info
(function).
add-warning
(generic function).
arbitrary
(generic function).
assert-criterion
(macro).
assert-eq
(function).
assert-eql
(function).
assert-equal
(function).
assert-equalp
(function).
assert-non-nil
(function).
assert-not-eq
(function).
assert-not-eql
(function).
assert-not-equal
(function).
assert-not-equalp
(function).
assert-null
(function).
assert-zero
(function).
check-criterion-on-form
(function).
check-criterion-on-value
(function).
compound-structure
(macro).
def-arbitrary-instance-type
(macro).
def-binary-negated-predicate-assert
(macro).
def-binary-predicate-assert
(macro).
def-criterion
(macro).
def-criterion-alias
(macro).
def-criterion-unevaluated
(macro).
def-eval-test
(macro).
def-fixtures
(macro).
def-form-criterion
(macro).
def-test
(macro).
def-test-generic
(macro).
def-test-group
(macro).
def-test-method
(macro).
def-test-method-criterion
(macro).
def-unary-negated-predicate-assert
(macro).
def-unary-predicate-assert
(macro).
def-values-criterion
(macro).
emit-failure
(macro).
emit-success
(macro).
emit-warning
(macro).
junit-results-by-group
(function).
make-error-report
(function).
make-failure-report
(function).
make-success-report
(function).
make-warning-report
(function).
nst-cmd
(macro).
nst-junit-dump
(generic function).
nst-results
(method combination).
with-fixtures
(macro).
wrap-thrown-lisp-warning
(function).
%make-check-result
(function).
*char-escapes*
(special variable).
*current-compound-structure-depth*
(special variable).
*default-character-range*
(special variable).
*default-debug-config*
(special variable).
*default-debug-protect*
(special variable).
*default-xml-pprint-dispatch*
(special variable).
*entities*
(special variable).
*fixture-bindings*
(special variable).
*fixture-letlist*
(special variable).
*fixture-record*
(special variable).
*generate-backtraces*
(special variable).
*group-records*
(special variable).
*implicit-group-choice*
(special variable).
*last-repl-call*
(special variable).
*max-auto-array-rank*
(special variable).
*nst-check-internal-name*
(special variable).
*nst-check-user-name*
(special variable).
*nst-context*
(special variable).
*nst-context-evaluable*
(special variable).
*nst-debug*
(special variable).
*nst-group-name*
(special variable).
*nst-group-shown*
(special variable).
*nst-report-driver*
(special variable).
*nst-stack*
(special variable).
*nst-verbosity*
(special variable).
*test-methods*
(special variable).
+arbitrary-generable-types+
(special variable).
+keyboard-interrupt-class+
(constant).
+name-uses+
(special variable).
+nst-repl-commands+
(special variable).
+nst-repl-properties+
(special variable).
+results-record+
(special variable).
+scalar-generable-types+
(special variable).
actual-value
(generic reader).
add-thrown-error
(function).
all-groups-report
(function).
all-package-report
(function).
all-tests-report
(function).
all-uses
(function).
apply-check-note-formatter
(function).
apply-criterion
(generic function).
apply-debug-options
(macro).
apply-default-debug-options
(macro).
apply-fixture
(function).
apply-fixtures
(function).
apply-formatter
(function).
arbitrary-by-spec
(generic function).
arbitrary-generable-types
(function).
arbitrary-grounded-type
(generic function).
artifact-recorder
(function).
assemble-protected-option-values
(function).
assert-criterion-fn
(function).
assertion-failure-args
(generic reader).
assertion-failure-fatal
(generic reader).
assertion-failure-formatter
(generic reader).
at-verbosity
(macro).
base-name
(generic function).
blurb-one-name-use
(function).
build-continue-check-expr
(function).
calibrate-check-result
(function).
cdr-or-nil
(function).
check-note
(structure).
check-note-args
(reader).
(setf check-note-args)
(writer).
check-note-context
(reader).
(setf check-note-context)
(writer).
check-note-format
(reader).
(setf check-note-format)
(writer).
check-note-p
(function).
check-note-stack
(reader).
(setf check-note-stack)
(writer).
check-note-type-string
(generic function).
check-result
(macro).
check-result
(structure).
check-result-check-name
(reader).
(setf check-result-check-name)
(writer).
check-result-elapsed-time
(function).
(setf check-result-elapsed-time)
(function).
check-result-erring
(function).
(setf check-result-erring)
(function).
check-result-errors
(reader).
(setf check-result-errors)
(writer).
check-result-failing
(function).
(setf check-result-failing)
(function).
check-result-failures
(reader).
(setf check-result-failures)
(writer).
check-result-group-name
(reader).
(setf check-result-group-name)
(writer).
check-result-info
(reader).
(setf check-result-info)
(writer).
check-result-p
(function).
check-result-passing
(function).
(setf check-result-passing)
(function).
check-result-skipped
(reader).
(setf check-result-skipped)
(writer).
check-result-tests
(reader).
(setf check-result-tests)
(writer).
check-result-timestamp
(function).
(setf check-result-timestamp)
(function).
check-result-union
(function).
check-result-warning
(function).
(setf check-result-warning)
(function).
check-result-warnings
(reader).
(setf check-result-warnings)
(writer).
coin-flip
(macro).
collect-test-generics
(function).
consider-repl-call-save
(generic function).
context-layer
(class).
continue-check
(function).
copy-check-note
(function).
copy-check-result
(function).
copy-error-check-note
(function).
copy-fixture-record
(function).
copy-group-record
(function).
copy-group-result
(function).
copy-multi-results
(function).
copy-name-use
(function).
copy-package-result
(function).
copy-result-stats
(function).
copy-test-record
(function).
core-run-test
(function).
count-nonnulls
(macro).
criterion
(generic reader).
(setf criterion)
(generic writer).
criterion-args
(generic reader).
(setf criterion-args)
(generic writer).
criterion-context-layer
(class).
criterion-missing-mandatory-argument
(condition).
criterion-name
(generic reader).
debug-for-fail
(condition).
decode-caching-decls
(function).
decode-def-test-generic-body
(function).
decode-defcheck-name-and-args
(function).
decode-fixture-syntax
(function).
decompose-arg-values-lambda-list
(function).
def-bundle-package
(macro).
def-check
(macro).
def-check-alias
(macro).
def-control-check
(macro).
def-hashtable-fns
(macro).
def-nst-interactive-command
(macro).
def-nst-property
(macro).
def-value-check
(macro).
define-nst-error
(macro).
domain
(generic reader).
elapsed-time-to-string
(function).
emit-error
(macro).
eql-for-sigdigits
(function).
error-check-note
(structure).
error-check-note-args
(function).
(setf error-check-note-args)
(function).
error-check-note-context
(function).
(setf error-check-note-context)
(function).
error-check-note-error
(reader).
(setf error-check-note-error)
(writer).
error-check-note-format
(function).
(setf error-check-note-format)
(function).
error-check-note-p
(function).
error-check-note-stack
(function).
(setf error-check-note-stack)
(function).
executable-uses
(function).
expected-form
(generic reader).
extract-parameters
(function).
finish-multiple-report
(function).
fixture-binding-error-note
(function).
fixture-bindings
(function).
(setf fixture-bindings)
(function).
fixture-letlist
(function).
(setf fixture-letlist)
(function).
fixture-record
(function).
(setf fixture-record)
(function).
fixture-record
(structure).
fixture-record-bindings-list
(reader).
(setf fixture-record-bindings-list)
(writer).
fixture-record-bound-names
(reader).
(setf fixture-record-bound-names)
(writer).
fixture-record-cache-flush
(reader).
(setf fixture-record-cache-flush)
(writer).
fixture-record-documentation
(reader).
(setf fixture-record-documentation)
(writer).
fixture-record-function
(reader).
(setf fixture-record-function)
(writer).
fixture-record-name
(reader).
(setf fixture-record-name)
(writer).
fixture-record-p
(function).
flag-filter
(function).
flush-fixture-cache
(function).
fmt-permuter
(generic function).
format-at-verbosity
(macro).
format-for-warning
(generic function).
generate-sample
(function).
get-display-context-layers
(function).
get-local-criterion-context
(function).
get-report-from-names
(function).
get-test-record-specials
(function).
get-unassigned
(function).
given-stack
(generic reader).
(setf given-stack)
(generic writer).
group
(generic reader).
group-record
(function).
(setf group-record)
(function).
group-record
(structure).
group-record-anon-fixture-forms
(reader).
(setf group-record-anon-fixture-forms)
(writer).
group-record-aspirational
(reader).
(setf group-record-aspirational)
(writer).
group-record-documentation
(reader).
(setf group-record-documentation)
(writer).
group-record-eachtest-cleanup-thunk
(reader).
(setf group-record-eachtest-cleanup-thunk)
(writer).
group-record-eachtest-setup-thunk
(reader).
(setf group-record-eachtest-setup-thunk)
(writer).
group-record-fixtures-cleanup-thunk
(reader).
(setf group-record-fixtures-cleanup-thunk)
(writer).
group-record-fixtures-setup-thunk
(reader).
(setf group-record-fixtures-setup-thunk)
(writer).
group-record-given-fixtures
(reader).
(setf group-record-given-fixtures)
(writer).
group-record-include-groups
(reader).
(setf group-record-include-groups)
(writer).
group-record-name
(reader).
(setf group-record-name)
(writer).
group-record-p
(function).
group-record-tests
(reader).
(setf group-record-tests)
(writer).
group-record-withfixtures-cleanup-thunk
(reader).
(setf group-record-withfixtures-cleanup-thunk)
(writer).
group-record-withfixtures-setup-thunk
(reader).
(setf group-record-withfixtures-setup-thunk)
(writer).
group-report
(function).
group-result
(structure).
group-result-check-results
(reader).
(setf group-result-check-results)
(writer).
group-result-elapsed-time
(function).
(setf group-result-elapsed-time)
(function).
group-result-erring
(function).
(setf group-result-erring)
(function).
group-result-failing
(function).
(setf group-result-failing)
(function).
group-result-group-name
(reader).
(setf group-result-group-name)
(writer).
group-result-p
(function).
group-result-passing
(function).
(setf group-result-passing)
(function).
group-result-tests
(function).
(setf group-result-tests)
(function).
group-result-timestamp
(function).
(setf group-result-timestamp)
(function).
group-result-warning
(function).
(setf group-result-warning)
(function).
handler-bind-interruptable
(macro).
has-next
(generic reader).
interesting-result-p
(function).
invoke-test-methods
(function).
junit-group-result
(generic function).
junit-header
(function).
keyword-args
(generic reader).
log10
(macro).
low-biased-natnum
(function).
make-and-calibrate-check-result
(function).
make-check-note
(function).
make-check-result
(function).
make-config-error
(function).
make-error-check-note
(function).
make-error-note
(function).
make-fixture-record
(function).
make-group-record
(function).
make-group-result
(function).
make-multi-results
(function).
make-name-use
(function).
make-package-result
(function).
make-result-stats
(function).
make-skipped-report
(function).
make-test-record
(function).
method-context-layer
(class).
method-name
(generic reader).
(setf method-name)
(generic writer).
mid-biased-natnum
(function).
multi-results
(structure).
multi-results-elapsed-time
(function).
(setf multi-results-elapsed-time)
(function).
multi-results-erring
(function).
(setf multi-results-erring)
(function).
multi-results-failing
(function).
(setf multi-results-failing)
(function).
multi-results-group-reports
(reader).
(setf multi-results-group-reports)
(writer).
multi-results-p
(function).
multi-results-package-reports
(reader).
(setf multi-results-package-reports)
(writer).
multi-results-passing
(function).
(setf multi-results-passing)
(function).
multi-results-stats-source
(reader).
(setf multi-results-stats-source)
(writer).
multi-results-system
(reader).
(setf multi-results-system)
(writer).
multi-results-test-reports
(reader).
(setf multi-results-test-reports)
(writer).
multi-results-tests
(function).
(setf multi-results-tests)
(function).
multi-results-timestamp
(function).
(setf multi-results-timestamp)
(function).
multi-results-warning
(function).
(setf multi-results-warning)
(function).
multiple-report
(function).
name-use
(function).
name-use
(structure).
name-use-fixtures
(reader).
(setf name-use-fixtures)
(writer).
name-use-groups
(reader).
(setf name-use-groups)
(writer).
name-use-last-use
(reader).
(setf name-use-last-use)
(writer).
name-use-name
(reader).
(setf name-use-name)
(writer).
name-use-p
(function).
name-use-package-p
(reader).
(setf name-use-package-p)
(writer).
name-use-tests
(reader).
(setf name-use-tests)
(writer).
named-function
(macro).
next-permutation
(function).
no-effect
(function).
no-nst-groups-in-package
(condition).
no-such-nst-group
(condition).
no-such-nst-test
(condition).
not-expected-form
(condition).
note-name-invocation
(function).
nst-arg-names
(generic function).
nst-assertion-condition
(condition).
nst-assertion-failure
(condition).
nst-assertion-result
(generic reader).
nst-assertion-result-check
(condition).
nst-deprecation-warning-mixin
(condition).
nst-dump
(function).
nst-error
(condition).
nst-hard-deprecation
(condition).
nst-long-help
(generic function).
nst-repl-property-display
(generic function).
nst-repl-property-doc
(generic function).
nst-repl-property-encode
(generic function).
nst-short-help
(generic function).
nst-soft-deprecation
(condition).
nst-soft-keyarg-deprecation
(condition).
nst-xml-dump
(function).
object
(generic reader).
(setf object)
(generic writer).
old-name
(generic reader).
open-fixture
(function).
package-groups
(function).
package-of
(generic reader).
package-report
(function).
package-result
(structure).
package-result-elapsed-time
(function).
(setf package-result-elapsed-time)
(function).
package-result-erring
(function).
(setf package-result-erring)
(function).
package-result-failing
(function).
(setf package-result-failing)
(function).
package-result-group-results
(reader).
(setf package-result-group-results)
(writer).
package-result-p
(function).
package-result-package-name
(reader).
(setf package-result-package-name)
(writer).
package-result-passing
(function).
(setf package-result-passing)
(function).
package-result-tests
(function).
(setf package-result-tests)
(function).
package-result-timestamp
(function).
(setf package-result-timestamp)
(function).
package-result-warning
(function).
(setf package-result-warning)
(function).
permuter
(class).
permuter-frame
(class).
pick-from-sequence
(function).
pprint-cdata-string
(function).
pprint-xml
(function).
prep-arg-names-help
(generic function).
princ-filled-text
(function).
protect-nst-config
(macro).
pull-test-name-list
(function).
qw
(special variable).
qwerty
(special variable).
record-name-use
(function).
record-package-use
(function).
refine-package-symbol-desigs
(function).
relax-stack
(function).
replacement
(generic reader).
report-details
(function).
report-group
(function).
report-interesting
(function).
report-multiple
(function).
report-package
(function).
report-summary
(function).
report-test
(function).
required-keyarg
(generic reader).
restore-protected-option-values
(function).
result-stats
(structure).
result-stats-elapsed-time
(reader).
(setf result-stats-elapsed-time)
(writer).
result-stats-erring
(reader).
(setf result-stats-erring)
(writer).
result-stats-failing
(reader).
(setf result-stats-failing)
(writer).
result-stats-p
(function).
result-stats-passing
(reader).
(setf result-stats-passing)
(writer).
result-stats-tests
(reader).
(setf result-stats-tests)
(writer).
result-stats-timestamp
(reader).
(setf result-stats-timestamp)
(writer).
result-stats-warning
(reader).
(setf result-stats-warning)
(writer).
result-summary
(generic function).
returning-criterion-config-error
(macro).
returning-test-error
(macro).
run-command-actual
(generic function).
run-debug-options
(function).
run-group
(function).
run-group-record
(function).
run-group-test
(function).
run-group-tests
(function).
run-group-tests-fixtures-thunk
(function).
run-nst-command
(function).
run-package
(function).
run-test
(function).
run-test-fixtures-thunk
(function).
run-test-inst
(function).
separate-group-subforms
(function).
set-nst-property
(generic function).
show-context-layer
(generic function).
show-nst-property
(generic function).
sig-place
(function).
soft-dep-warning
(function).
string-escaped
(function).
symbol-or-car
(function).
symbol-to-junit-name
(function).
test
(generic reader).
test-is-aspirational
(function).
test-record
(function).
(setf test-record)
(function).
test-record
(structure).
test-record-aspirational
(reader).
(setf test-record-aspirational)
(writer).
test-record-aspirational-supp
(reader).
(setf test-record-aspirational-supp)
(writer).
test-record-cleanup
(reader).
(setf test-record-cleanup)
(writer).
test-record-criterion
(reader).
(setf test-record-criterion)
(writer).
test-record-documentation
(reader).
(setf test-record-documentation)
(writer).
test-record-finish
(reader).
(setf test-record-finish)
(writer).
test-record-fixtures
(reader).
(setf test-record-fixtures)
(writer).
test-record-forms
(reader).
(setf test-record-forms)
(writer).
test-record-group
(reader).
(setf test-record-group)
(writer).
test-record-name
(reader).
(setf test-record-name)
(writer).
test-record-p
(function).
test-record-results
(reader).
(setf test-record-results)
(writer).
test-record-setup
(reader).
(setf test-record-setup)
(writer).
test-record-special-fixture-names
(reader).
(setf test-record-special-fixture-names)
(writer).
test-record-startup
(reader).
(setf test-record-startup)
(writer).
test-report
(function).
tighten-stack
(function).
timestamp-to-string
(function).
trace-group
(generic function).
trace-results
(function).
trace-test
(generic function).
unknown-arbitrary-domain
(condition).
use-stats-from
(function).
whittlable
(function).
whittle-relaxed-stack
(function).
with-context-layer
(macro).
with-criterion-context-layer
(macro).
with-method-context-layer
(macro).
with-nst-control-handlers
(macro).
with-output-for-verbosity
(macro).
with-pprint-cdata
(macro).
with-retry
(macro).
with-xml-tagged-pprint-logical-block
(macro).
wrap-fixture-name-specials
(function).
Definitions are sorted by export status, category, package, and then by lexicographic order.
User variable =*debug-on-error*=: if non-nil, will break into the Lisp REPL debugger upon encountering an unexpected error. If nil, will record the error and continue with other tests.
nst
.
User variable =*debug-on-fail*=: if non-nil, will break into the Lisp REPL debugger upon encountering a test which fails. If nil, will record the failure and continue with other tests. This variable is useful inspecting the dynamic environment under which a test was evaluated.
nst
.
User variable =*default-report-verbosity*= determines the default value for =*nst-verbosity*= when printing reports (2 by default).
nst
.
The =*max-compound-structure-depth*= variable sets the maximum nesting depth of compound data structures: beyond that depth, =scalar= rather than =t= is the default element generator. This restriction does not apply to explicitly specified element types, only to the use of defaults.
nst
.
Determines the output stream to which NST should print its output (=*standard-output*= by default).
nst
.
The =assert-criterion= macro asserts that an NST criterion should pass.
#+begin_example
(assert-criterion ( [ :msg-format FORMAT-STRING ]
[ :msg-args FORMAT-ARGUMENTS ]
[ :fatal FLAG [
[ :fail-on-warning FLAG ] )
FORM
...
FORM)
#+end_example
- msg-format :: Format string used to build the label of the restart point.
- msg-args :: Format arguments used to build the label of the restart point.
- fatal :: If non-nil, a failure of this assertion indicates that execution of
the test forms should be aborted.
- fail-on-warning :: If non-nil, then an NST result which includes a warning
indicates failure of this assertion.
nst
.
The =compound-structure= macro wraps substructure which should be considered compound for the limits set by =*max-compound-structure-depth*=.
nst
.
New type specifications for invariant-testing. are defined with the
=def-arbitrary-instance-type= macro.
#+begin_example
(def-arbitrary-instance-type ( SPEC-NAME [ :params FORMALS ]
[ :scalar BOOL ]
[ :key KEY ] )
FORM
FORM
...
FORM)
#+end_example
- =formals= :: Formal parameter definition used to pass subcomponent types.
- =scalar= :: When a non-null value is provided for the =:scalar= argument, the
new specifier is taken to be generable by the =:scalar=
specification.
#+begin_example
(def-arbitrary-instance-type (ratio :scalar t)
(/ (arbitrary ’integer)
(let ((raw (arbitrary (find-class ’integer))))
(cond
((< raw 0) raw)
(t (+ 1 raw))))))
#+end_example
- =key= :: The =:key= argument gives a list of keyword arguments which may
accompany the new specification. For the =cons= type, keyword
arguments allow specifications for the left and right components:
#+begin_example
(def-arbitrary-instance-type
(cons :key ((car t car-supp-p) (cdr t cdr-supp-p)))
(compound-structure
(when (and (not car-supp-p)
(>= *current-compound-structure-depth*
*max-compound-structure-depth*))
(setf car ’scalar))
(when (and (not cdr-supp-p)
(>= *current-compound-structure-depth*
*max-compound-structure-depth*))
(setf cdr ’scalar))
(cons (arbitrary car) (arbitrary cdr))))
#+end_example
- =form= :: Construct and return (as if through =progn= the arbtrary instance.
nst
.
Macro =def-binary-negated-predicate-assert= uses the negated result of a binary predicate as the basis for an assertion function just as =def-unary-negated-predicate-assert= uses the negated result of a unary predicate. This macro’s arguments are just as for =def-unary-predicate-assert=.
nst
.
Macro =def-binary-predicate-assert= uses a binary predicate as the basis for an assertion function just as =def-unary-predicate-assert= uses a unary predicate. This macro’s arguments are just as for =def-unary-predicate-assert=.
nst
.
The =def-criterion= macro defines a new criterion for use in NST tests.
These criteria definitions are like generic function method definitions with two
sets of formal parameters: the forms provided as the actual parameters of the
criterion itself, and the values arising from the evaluation of the forms under
test.
#+begin_example
(def-criterion (name criterion-lambda-list values-lambda-list)
[ doc-string )
form
form
...
form)
#+end_example
- name :: Name of the criterion.
- criterion-lambda-list :: Lambda list for the arguments to the criterion. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: =:values= for call-by-value, or =:forms for call-by-name (the default). The list may include the keywords =&key=, =&optional=, =&body= and =&rest= but may not use =&whole= or =&environment=. Apart from this restriction, in the former case the list may be any ordinary lambda list as for =defun=, and in the latter case the list may be any macro lambda list as for =defmacro=.
- values-lambda-list :: Lambda list for the forms under test. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: :values for call-by-value (the default), or :form for call-by-name. In the former case, the list may include the keywords =&key=, =&optional=, =&body= and =&rest=, but not =&whole= or =&environment=; apart from that restriction, list may be any ordinary lambda list as for =defun=. In the latter case, the remainder of the list must contain exactly one symbol, to which a form which would evaluate to the values under test will be bound.
If the criterion ignores the values, then instead of a lambda list, this argument may be the symbol =:ignore=. On many platforms, listing a dummy parameter which is then =declare=d =ignore= or =ignorable= will produce a style warning: the body of a =def-criterion= should not be assumed to correspond directly to the body of a =defmethod=; in general there will be surrounding =destructuring-bind=s.
- documentation :: An optional documentation string for the criterion.
- form :: The body of the criterion definition should return a test result report contructed with the =make-success-report=, etc. functions.
Examples:
#+begin_example
(def-criterion (:true () (bool))
(if bool
(make-success-report)
(make-failure-report :format "Expected non-null, got: ~s"
:args (list bool))))
(def-criterion (:eql (target) (actual))
(if (eql (eval target) actual)
(make-success-report)
(make-failure-report :format "Not eql to value of ~s"
:args (list target))))
#+end_example
nst
.
The simplest mechanism for defining a new criterion involves simply
defining one criterion to rewrite as another using =def-criterion-alias=.
#+begin_example
(def-criterion-alias (name (:seq arg))
[ doc-string ]
expansion)
#+end_example
The body of the expansion should be a Lisp form which, when evaluated, returns
an S-expression quoting the new criterion which the rewrite should produce. The
=arg= are passed as for Lisp macros: they are not evaluated and are most
typically comma-inserted into a backquoted result. For example:
#+begin_example
(def-criterion-alias (:forms-eq) ‘(:predicate eq))
(def-criterion-alias (:symbol name) ‘(:eq ’,name))
#+end_example
nst
.
The =def-criterion-unevaluated= macro is deprecated as of NST 2.1.2. It was
consolidated into the =def-criterion= macro.
Replace:
#+begin_example
(def-criterion-unevaluated name (pattern ... pattern) name
BODY)
#+end_example
with:
#+begin_example
(def-criterion name (:forms pattern ... pattern)
(:form name)
BODY)
#+end_example
nst
.
The =def-eval-test= macro abbreviates a call to =def-test= with a single
=:eval= criterion. Its arguments are just as for =def-test= and =:eval=.
#+begin_example
(def-eval-test (NAME [ :group GROUP-NAME ]
[ :setup FORM ]
[ :cleanup FORM ]
[ :startup FORM ]
[ :finish FORM ]
[ :fixtures (FIXTURE ... FIXTURE) ]
[ :documentation STRING ]
[ :check-warnings FLAG ]
[ :muffle-warnings FLAG ]
[ :attempt-continue FLAG ]
[ :force-continue FLAG ] )
FORM
...
FORM)
(def-eval-test NAME
FORM
...
FORM)
#+end_example
nst
.
Fixtures are data structures and values which may be referred to by name
during testing. NST provides the ability to use fixtures across multiple tests
and test groups, and to inject fixtures into the runtime namespace for
debugging. A set of fixtures is defined using the =def-fixtures= macro:
#+begin_example
(def-fixtures fixture-name ( [ :outer FORM ]
[ :inner FORM ]
[ :setup FORM ]
[ :cleanup FORM ]
[ :startup FORM ]
[ :finish FORM ]
[ :documentation STRING ]
[ :cache FLAG ]
[ :export-names FLAG ]
[ :export-fixture-name FLAG ]
[ :export-bound-names FLAG ] )
( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] )
...
( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] ) )
#+end_example
- fixture-name :: The name to be associated with this set of fixtures.
- inner :: List of declarations to be made inside the let-binding of
names of any use of this fixture. Do not include the "declare"
keyword here; NST adds these declarations to others, including a
special declaration of all bound names.
- outer :: List of declarations to be made outside the let-binding of
names of any use of this fixture.
- documentation :: A documentation string for the fixture set.
- special :: Specifies a list of names which should be declared
=special= in the scope within which this set’s fixtures are
evaluated. The individual names are taken to be single variable
names. Each =(:fixture NAME)= specifies all of the names of the given
fixture set. This declaration is generally optional under most
platforms, but can help supress spurious warnings. Note that
multiple =NAMEs= may be listed, and these lists and the bare
names may be intermixed. If only one name or fixture is
specified, it need not be placed in a list
- export-fixture-name :: When non-nil, the fixture name will be added
to the list of symbols exported by the current package.
- export-bound-names :: When non-nil, the names bound by this fixture
will be added to the list of symbols exported by the current
package.
- export-names :: When non-nil, sets the default value to t for the
two options above.
- cache :: If specified with the group options, when non-nil, the
fixture values are cached at their first use, and re-applied at
subsequent fixture application rather than being recalculated.
When a fixture is attached to a test or test group, each =NAME=
defined in that fixture becomes available in the body of that test or
group as if =let*= bound to the corresponding =FORM=. A fixture in
one set may refer back to other fixtures in the same set (again as
=let*=) but forward references are not allowed.
The four arguments =startup=, =finish=, =setup= and =cleanup= specify
forms which are run everytime the fixture is applied to a group or
test. The =startup= (respectively =finish=) form is run before
fixtures are bound (after their bindings are released). These forms
are useful, for example, to initialize a database connection from
which the fixture values are drawn. The =setup= form is run after
inclusion of names from fixture sets, but before any tests from the
group. The =cleanup= form is normally run after the test completes,
but while the fixtures are still in scope. Normally, the =cleanup=
form will not be run if the =setup= form raises an error, and the
=finish= form will not be run if the =startup= form raises an error;
although the user is able to select (perhaps unwisely) a restart which
disregards the error.
The names of a fixture and the names it binds can be exported from the
package where the fixture is defined using the =export-bound-names=
and =export-fixture-name= arguments. The default value of both is the
value of =export-names=, whose default value is =nil=.
The =cache= option, if non-nil, directs NST to evaluate a fixture’s
form one single time, and re-use the resulting value on subsequent
applications of the fixture. Note that if this value is mutated by
the test cases, test behavior may become unpredictable! However this
option can considerably improve performance when constant-valued
fixtures are applied repeatedly. Caching may be set on or off (the
default is off) for the entire fixture set, and the setting may vary
for individual fixtures.
Examples of fixture definitions:
#+begin_example
(def-fixtures f1 ()
(c 3)
(d ’asdfg))
(def-fixtures f2 (:special ((:fixture f1)))
(d 4)
(e ’asdfg)
(f c))
(def-fixtures f3 ()
((:cache t) g (ackermann 1 2))
((:cache nil) h (factorial 5)))
#+end_example
To cause a side-effect among the evaluation of a fixture’s name definitions, =nil= can be provided as a fixture name. In uses of the fixture, NST will replace =nil= with a non-interned symbol; in documentation such as form =:whatis=, any =nil=s are omitted.
nst
.
The =def-form-criterion= macro was deprecated as of NST 1.3.0. /Code using =def-form-criterion= in any but the simplest ways is very likely to fail./ Use =def-criterion= instead.
nst
.
Individual unit tests are encoded with the =def-test= form:
#+begin_example
(def-test NAME ( [ :group GROUP-NAME ]
[ :setup FORM ]
[ :cleanup FORM ]
[ :startup FORM ]
[ :finish FORM ]
[ :fixtures (FIXTURE FIXTURE ... FIXTURE) ]
[ :aspirational FLAG ]
[ :documentation STRING ] )
criterion
FORM ... FORM)
(def-test NAME criterion &body (:seq FORM))
#+end_example
The =SETUP=, =CLEANUP=, =STARTUP=, =FINISH= and =FIXTURES= are just as for
fixtures and test groups, but apply only to the one test. The =CRITERION=
is a list or symbol specifying the properties which should hold for
the =FORM=s.
When a test is not enclosed within a group body, a group name must be
provided by the =GROUP= option. When a test is enclosed within
a group body, the =GROUP= option is not required, but if
provided it must agree with the group name.
When there are no =SETUP=, =CLEANUP=, =STARTUP=, =FINISH= or =FIXTURES=
arguments, the =NAME= may be given without parentheses. Likewise, any
criterion consisting of a single symbol, e.g. =(:pass)=, may be abbreviated
as just the symbol without the parentheses, e.g. =:pass=.
The =:documentation= form provides a documentation string in
the standard Lisp sense. Since documentation strings are stored
against names, and since the same name can be used for several tests
(so long as they are all in different packages), documentation strings
on tests may not be particularly useful.
An =aspirational= test is one which verifies some part of an API or code
contract which may not yet be implemented. When a group is marked
aspirational, all tests within the group are taken to be aspirational as well.
At this point, there is no particular processing for aspirational tests and
groups, but we hope to implement it at some point in the future.
The =def-check= form is a deprecated synonym for =def-test=.
nst
.
The =def-test-generic= declares a generic test function.
#+begin_example
(def-test-generic FUNCTION-NAME)
#+end_example
For example,
#+begin_example
(nst:def-test-generic for-clses)
#+end_example
nst
.
The =def-test-group= form defines a group of the
given name, providing one instantiation of the bindings of the given fixtures
to each test. Groups can be associated with fixture sets, stateful
initialization, and stateful cleanup.
#+begin_example
(def-test-group NAME (FIXTURE FIXTURE ...)
(:aspirational FLAG)
(:setup FORM FORM ... FORM)
(:cleanup FORM FORM ... FORM)
(:startup FORM FORM ... FORM)
(:finish FORM FORM ... FORM)
(:each-setup FORM FORM ... FORM)
(:each-cleanup FORM FORM ... FORM)
(:include-groups GROUP GROUP ... GROUP)
(:documentation STRING)
TEST
...
TEST)
#+end_example
Arguments:
- group-name :: Name of the test group being defined
- given-fixtures :: List of the names of fixtures and anonymous fixtures to be used with the tests in this group.
- aspirational :: An aspirational test is one which verifies some part of an API or code contract which may not yet be implemented. When a group is marked aspirational, all tests within the group are taken to be aspirational as well.
At this point, there is no particular processing for aspirational tests and groups, but we hope to implement it at some point in the future.
- forms :: Zero or more test forms, given by =def-check=.
- setup :: These forms are run once, before any of the individual tests, but after the fixture names are bound.
- cleanup :: These forms are run once, after all of the individual tests, but while the fixture names are still bound.
- startup :: These forms are run once, before any of the individual tests and before the fixture names are bound.
- finish :: These forms are run once, after all of the individual tests, and after the scope of the bindings to fixture names.
- each-setup :: These forms are run before each individual test.
- each-cleanup :: These forms are run after each individual test.
- include-group :: The test groups named in this form will be run (respectively reported) anytime this group is run (reported).
- documentation :: Docstring for the class.
nst
.
The =def-test-method= defines a general method for a generic test function.
#+begin_example
(def-test-method FUNCTION-NAME (TEST-VALUE CLASS-NAME)
FORM
...
FORM)
#+end_example
- function-name :: The name of the test function for which we are defining a
method.
- test-value :: Formal parameter to which the value under test will be bound.
- class-name :: The class for which we are defining a method.
The method body should return a test result report, constructed with
=make-success-result=, etc.
For example:
#+begin_example
(nst:def-test-method for-clses (o mid-cls)
(with-slots (mc1 mc2) o
(cond
((< mc1 mc2) (make-success-report))
(t (make-failure-report :format "~d not < ~d" :args (list mc1 mc2))))))
(nst:def-test-method for-clses (o side-cls)
(with-slots (sc1 sc2) o
(cond
((eql sc1 sc2) (make-success-report))
(t (make-failure-report :format "~d not eql ~d" :args (list sc1 sc2))))))
#+end_example
nst
.
The =def-test-method-criterion= macro provides a simple facility for defining a generic test function method in terms of an NST criterion.
#+begin_example
(def-test-method-criterion FUNCTION-NAME CLASS-NAME [ CRITERION ] )
#+end_example
- function-name :: The name of the test function for which we are defining a
method.
- class-name :: The class for which we are defining a method.
- criterion :: The criterion to be applied to members of the class.
For example:
#+begin_example
(nst:def-test-method-criterion for-clses top-cls
(:predicate (lambda (tc) (< (tc1 tc) (tc2 tc)))))
#+end_example
nst
.
Macro =def-unary-negated-predicate-assert= uses the negated result of a unary predicate as the basis of an assertion function. This macro’s arguments are just as for =def-unary-predicate-assert=.
nst
.
Macro =macro def-unary-predicate-assert= creates an assertion function using
the result of a call to a unary predicate. A non-nil result from the predicate
corresponds to a successful assertion.
#+begin_example
(def-unary-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE
[ :message-defvar NAME ]
[ :pred-name NAME ]
[ :doc-state-flag BOOL ] )
#+end_example
- assert-fn :: The name of the assertion function being defined.
- predicate :: The predicate used to define the assertion function. It should take a single argument.
- default-message :: Format string used by default for reporting failures of this assertion. It should expect to be used in a call to =format= with one additional argument, the value being tested.
- message-defvar :: The name of a global variable into which the default message will be stored. If this argument is omitted, the result of a call to =gensym= is used.
- pred-name :: This argument is used only for documenting the underlying predicate in the assertion function’s docstring. By default, it is the same as the predicate.
nst
.
The =def-values-criterion= macro was deprecated as of NST 1.3.0. For new criteria, use =def-criterion= instead. In the short term, code using =def-values-criterion= should continue to work as before.
nst
.
The =emit-failure= function is deprecated; use =make-failure-report= instead.
nst
.
The =emit-success= function is deprecated; use =make-success-report= instead.
nst
.
The =emit-warning= function is deprecated; use =make-warning-report= instead.
nst
.
User-level NST operations are accessible from the REPL via the =nst-cmd=
macro.
#+begin_example
(nst-cmd NST-COMMAND ARG ... ARG)
#+end_example
Where a particular system supports the facility (Currently Allegro, and SBCL
under ACL-REPL) the top-level alias =:nst= provides a shorthand to this
function.
nst
.
The =with-fixtures= macro faciliates debugging and other non-NST uses of
fixtures sets:
#+begin_example
(with-fixtures (FIXTURE FIXTURE ... FIXTURE)
FORM
FORM
...
FORM)
#+end_example
This macro evaluates the forms in a namespace expanded with the bindings
provided by the fixtures.
nst
.
For use within user-defined NST criteria: add an error to a result.
#+begin_example
(add-error RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ] )
#+end_example
nst
.
For use within user-defined NST criteria: add a failure to a result.
#+begin_example
(add-failure RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ])
#+end_example
nst
.
For use within user-defined NST criteria: add an info note to a result.
#+begin_example
(add-info RESULT-REPORT INFO-ITEM)
#+end_example
nst
.
The =ASSERT-EQ= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQ=, succeeds whenever that call returns non-nil.
nst
.
The =ASSERT-EQL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQL=, succeeds whenever that call returns non-nil.
nst
.
The =ASSERT-EQUAL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUAL=, succeeds whenever that call returns non-nil.
nst
.
The =ASSERT-EQUALP= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUALP=, succeeds whenever that call returns non-nil.
nst
.
The =ASSERT-NON-NIL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =NULL= function returns =nil=.
nst
.
The =ASSERT-NOT-EQ= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQ=, succeeds whenever that call returns =nil=.
nst
.
The =ASSERT-NOT-EQL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQL=, succeeds whenever that call returns =nil=.
nst
.
The =ASSERT-NOT-EQUAL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUAL=, succeeds whenever that call returns =nil=.
nst
.
The =ASSERT-NOT-EQUALP= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUALP=, succeeds whenever that call returns =nil=.
nst
.
The =ASSERT-NULL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =NULL= function returns non-nil.
nst
.
The =ASSERT-ZERO= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =ZEROP= function returns non-nil.
nst
.
This function verifies that the values return by evaluating the form adheres
to the criterion.
#+begin_example
(check-criterion-on-form CRITERION FORM)
#+end_example
nst
.
The =check-criterion-on-value= function can be called from within a criterion
body to verify that a value adheres to a criterion.
#+begin_example
(check-criterion-on-value CRITERION VALUE)
#+end_example
nst
.
The \texttt{junit-results-by-group} function writes the NST test results in JUnit XML format, organized by group, aligning test groups with Java classes, and individual tests with \texttt{@Test} methods.
#+begin_example
(junit-results-by-group [ :verbose FLAG ]
[ :dir DIRECTORY ]
[ :file FILESPEC ]
[ :stream STREAM ]
[ :if-dir-does-not-exist BOOL ]
[ :if-file-exists BOOL ] )
#+end_example
Either =:dir= and =:file= options, or the =:stream= option, but not both, should
e used to specify the target for XML output; if none of the three options are
given, the function will write to =*standard-output*=.
nst
.
Function =make-error-report= produces a report of an error during test execution.
#+begin_example
(make-error-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ] )
#+end_example
nst
.
The =make-failure-report= function returns a report of test failure.
#+begin_example
(make-failure-report FORMAT ARGS)
#+end_example
The =format-string= and =args= are as to the Common Lisp function =format=. The
=emit-failure= function is an older, deprecated version of this function.
nst
.
The =make-success-report= function indicates a successful test result.
#+begin_example
(make-success-report)
#+end_example
Note that some older examples show =(make-check-result)=, =(emit-success)= or
=(check-result)=. The former is an internal function and should not be used
from outside the core NST files. The latter two are deprecated.
nst
.
Function =make-warning-report= is like =make-failure-report=, but provides
supplimentary information as a warning.
#+begin_example
(make-warning-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ])
#+end_example
The =emit-warning= function is an older, deprecated version of this function.
nst
.
The helper function =wrap-thrown-lisp-warning= creates an NST =check-note= object from a standard Lisp =warning=.
nst
.
For use within user-defined NST criteria: add a warning to a
result.
The =add-warning= function adds an warning to a result record. The item can be
any of a Lisp warning, an NST check-note or a format string; in the first two
cases, no additional arguments should be provided.
#+begin_example
(add-warning RESULT-REPORT [ :format ITEM ] [ :args ARGUMENT-LIST ] )
#+end_example
nst
.
string
) &rest args) ¶check-note
) &rest args) ¶warning
) &rest args) ¶This function takes a single argument, which determines the
type of the value to be generated. For simple types, the name of the type (or
the class object, such as returned by =find-class= by itself is a complete
specification. For more complicated types, =arbitrary= can also take a list
argument, where the first element gives the type and the remaining elements are
keyword argument providing additional requirements for the generated value.
Function =nst-junit-dump= pushes the entire NST state to a JUnit XML file whose stream is specified by its argument.
method-context-layer
)) ¶automatically generated writer method
method-context-layer
)) ¶automatically generated reader method
NST defines a method combination =nst-results= as the default
method combination for functions defined by =def-test-generic=. This
combination runs /all/ applicable methods, and combines all of their results
into a single NST result record.
This default can be overridden by specifying =t= as the method combination in
the intial declaration.
#+begin_example
(nst:def-test-generic overridden (:method-combination t))
(nst:def-test-method-criterion overridden mid-cls
(:slots (mc1 (:eql 0))
(mc2 (:eql 2))))
(nst:def-test-method-criterion overridden bot-cls
(:slots (sc1 (:eql 1))
(sc2 (:eql 1))))
nst
.
check-result-union
(function).
nil
Name of the class of error thrown when a keyboard interrupt is received, or nil if this implementation of Lisp has no such class.
nst
.
User variable: the default setting applied by default. Should be a list of
alternating keyword/forms matching:
- nst-set - list of lists, each with arguments to :nst :set
- progn - list of forms to be evaluated
nst
.
Map implementing FIXTURE-BINDINGS: from fixture name to the list of local variable names bound that fixture
nst
.
Map implementing FIXTURE-LETLIST: from fixture name to the list of local variable names bound that fixture
nst
.
Map implementing FIXTURE-RECORD: from fixture name to its =fixture-record=
nst
.
User variable: if non-null, will attempt to capture the Lisp backtrace of errors in tests.
nst
.
Primary storage of group and test records; this level of map translates from group name to an instance of =group-record=.
nst
.
Dynamic variable used to set the name of a test in its result report.
nst
.
Dynamic variable used to set the name of a test in its result report.
nst
.
Dynamic-scoped variable recording the values under test - a list of context-layer instances.
nst
.
Dynamic-scoped variable tracking whether the values under test should be asusmed evaluated. Used in preparing context expressions.
nst
.
User variable: apply customizable debugging settings.
nst
.
Dynamic variable used to set the name of the group in a test result report.
nst
.
Dynamic-scoped variable tracking whether the name of a group has been printed, so that tests need not repeat it.
nst
.
Dynamic-scoped variable - one of :multiple, :package, :group or :test to determine the top-level form of a report. Used as a control parameter for printing reports.
nst
.
Dynamic-scoped variable - the stack of values under test by the current criterion.
nst
.
User variable determining how verbose NST’s output to the REPL should be. Internally, this variable takes an integer value: 0 and below are silent, 1 is the default, 2 and 3 are more verbose, and 4 is for full tracing. The command-line interpreter assigns keywords to these values, from most terse to most verbose: :silent, nil, :quiet, :default, t, :verbose, :vverbose (:quiet and :default are the same, and are the initial setting).
nst
.
Global variable for programmer-defined NST test methods.
nst
.
Map from a symbol in the =:nst-name-usage= package to a =name-use= instance decribing how a symbol of the same name is used for NST artifacts.
nst
.
Deprecated: use def-criterion-alias instead
nst
.
nst
.
Like handler-bind (and the arguments are similar), but cancel any error- handling if the error is related to a keyboard interrupt.
nst
.
For use within criteria definitions only — catch errors and process them as errors in the "glue" among criteria and forms.
nst
.
For use within criteria definitions only — catch errors and process them as errors arising from within the .
nst
.
Macro for inserting restarts at points in the NST test-running stack.
nst
.
nst
.
Return the list of group- and test-records, and possibly also a Lisp package object, corresponding to possible intended targets of an NST run.
nst
.
nst
.
nst
.
Return the cdr given a list, or return nil if given a symbol.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
Process a list of binding specifications to extract both the bindings, and the local variables needed for caching.
nst
.
This function unpacks the information inside the first form of a def-test block, which can be either a single symbol naming the test, or a list whose first element is that symbol and whose remaining elements are options.
nst
.
Assemble syntactic elements of a fixture into the fixture function and the list of names defined by the fixture.
nst
.
Test whether two numbers are eql to the given number of significant digits.
nst
.
nst
.
Return the list of group- and test-records, and possibly also a Lisp package object, corresponding to possible intended targets of an NST run.
nst
.
Reader function: from fixture name to the list of local variable names bound that fixture
nst
.
Writer function: from fixture name to the list of local variable names bound that fixture
nst
.
Reader function: from fixture name to the list of local variable names bound that fixture
nst
.
Writer function: from fixture name to the list of local variable names bound that fixture
nst
.
Reader function: from fixture name to its =fixture-record=
nst
.
Writer function: from fixture name to its =fixture-record=
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
Top-level function for reporting the results of a group.
nst
.
nst
.
nst
.
Functional wrapper around the constructor for =check-result= structure, permitting the use of apply.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
Function =make-skipped-report= is like =make-failure-report=, but provides
supplimentary information as a skipped.
#+begin_example
(make-skipped-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ])
#+end_example
The =emit-skipped= function is an older, deprecated version of this function.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
Push the bindings from a fixture into a package’s namespace.
nst
.
Return the groups whose name is a symbol in the given package.
nst
.
Top-level function for reporting the results of a package.
nst
.
nst
.
nst
.
Given syntax of the form (DEF-CHECK ...) or (DEF-TEST ...), pull the name of the test from the syntax.
nst
.
Store the use of a name (as given by a fixture-, group-, or test-record) in the index.
nst
.
Top-level function for reporting the results of the tests in a group.
nst
.
Top-level function for reporting the results of several tests.
nst
.
Top-level function for reporting the results of the tests in a package.
nst
.
Top-level function for reporting the results of a test.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
Run a group by its user-given name. Note that this is /not/ an interactive function — certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.
nst
.
Programmatic entry point for running all tests in a group.
nst
.
Run all groups in a package. Note that this is /not/ an interactive function — certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.
nst
.
Run a test standalone by its user-given name (and its group’s name). Note that this is /not/ an interactive function — certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.
nst
.
Given a list of forms, separate and decode the option specifiers from any
test forms. Returns many results:
- The list of test forms, in the same order as originally given.
- The =setup= option forms, and a flag set if the option was present.
- The =cleanup= option forms, and a flag set if the option was present.
- The =startup= option forms, and a flag set if the option was present.
- The =finish= option forms, and a flag set if the option was present.
- The =each= option forms, and a flag set if the option was present.
- The =each-cleanup= option forms, and a flag set if the option was present.
- The =docstring=, or nil if none was given.
- The =aspirational= flag, and a flag set if it was present.
- The =include-groups= list, and a flag set if it was present.
nst
.
Return the first element given a list, or return a symbol.
nst
.
Check both the test and group record to see if a test should be considered aspirational.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
nst
.
Top-level function for reporting the results of a test.
nst
.
Internal debugging function: dump the results hash.
nst
.
nst
.
not-expected-form
)) ¶Internal generic function whose methods are the translations of criteria.
nst
.
(eql :methods)
) args0 values-form1) ¶(eql :sample)
) args0 g1) ¶Invariants to be tested, and the domains over which they range, are specified
with the =:sample= criterion:
#+begin_example
(def-criterion ( [ :verify FORM ]
[ :value LAMBDA-LIST ]
[ :domains ((NAME SPEC) ... (NAME SPEC)) ]
[ :where FORM ]
[ :where-ignore (NAME ... NAME) ]
[ :where-declare (DECLARATION ... DECLARATION) ]
[ :sample-size NUMBER ]
[ :qualifying-sample NUMBER ]
[ :max-tries NUMBER ] ) )
#+end_example
- =:verify= :: The expression to be (repeatedly) evaluated, which is expected
always to return a non-null value. This is the sole required
argument, although in any particular use it is unlikely to be
the only argument given.
- =:domains= :: The variables in the =verify= expression which are to be given
multiple randomized values. The default value is =nil=,
denoting an empty list.
- =:value= :: A lambda list to which the values given by the argument form
should be applied. The default value is =nil=, denoting no such
arguments.
- =:where= :: A condition which determines the validity of the input argument.
For example, the condition would assert that a number is positive
in an application where a negative value would be known to cause
a failure. The default value is =t=, allowing any values.
- =:where-ignore= :: List of domain variables which are not mentioned in the
=where= clause. These names will be declared as ignored
in appropriate bindings, suppressing warnings under Lisps
which check for such things in interpreted expressions.
This list need not be given explicitly when no =where=
argument is given. Similarly, the =where-declare= argument
accepts a list of declarations to be associated with the
=where= form.
- =:sample-size= :: Gives the base specification of the number of value sets
which will be generated. Two further arguments have some
bearing on the number of generation attempts when the
=where= argument is non =t=. The =qualifying-sample=
argument gives the minimum acceptable size of actual tested
values, not counting sets rejected via the =where=
expression. The =max-tries= argument gives the maximum
number of value sets to be generated.
Examples:
#+begin_example
(:sample :sample-size 10
:domains ((x (list :elem symbol)))
:verify (equal x (reverse (reverse x))))
#+end_example
#+begin_example
(:sample :domains ((x real))
:where (> x 1)
:verify (< (sqrt x) x)
:sample-size 10
:max-tries 12)
#+end_example
(eql :process)
) args0 g1) ¶(eql :eval)
) args0 forms-list) ¶(eql :with-common-criterion)
) args1 form0) ¶(eql :applying-common-criterion)
) args0 g1) ¶(eql :package-internal)
) args0 values-form1) ¶(eql :package-exports)
) args0 values-form1) ¶(eql :slots)
) args0 values-form1) ¶(eql :across)
) args0 values-form1) ¶(eql :permute)
) args0 values-form1) ¶(eql :alist)
) args1 form0) ¶(eql :alist*)
) args0 values-form1) ¶(eql :seq)
) args0 values-form1) ¶(eql :each)
) args0 values-form1) ¶(eql :values)
) args0 form) ¶(eql :proj)
) args0 values-form1) ¶(eql :progn)
) args0 forms) ¶(eql :check-err)
) args0 forms) ¶(eql :apply)
) args0 exprs-form) ¶(eql :any)
) args0 expr-list-form) ¶(eql :all)
) args0 expr-list-form) ¶(eql :not)
) args0 expr-list-form) ¶(eql :perf)
) args0 expr-form) ¶(eql :err)
) args0 expr-form) ¶(eql :info)
) args0 expr-list-form) ¶(eql :dump-forms)
) args0 values-form1) ¶(eql :drop-values)
) args1 form0) ¶(eql :predicate)
) args0 values-form1) ¶(eql :value-list)
) args1 form0) ¶(eql :forms-equal)
) args1 form0) ¶(eql :forms-eql)
) args1 form0) ¶(eql :forms-eq)
) args1 form0) ¶(eql :equalp)
) args0 values-form1) ¶(eql :equal)
) args0 values-form1) ¶(eql :eql)
) args0 values-form1) ¶(eql :symbol)
) args1 form0) ¶(eql :eq)
) args0 values-form1) ¶(eql :true)
) args0 values-form1) ¶(eql :true-form)
) args0 g1) ¶(eql :warn)
) args0 g1) ¶(eql :skip)
) args0 g1) ¶(eql :fail)
) args0 g1) ¶(eql :pass)
) args0 g1) ¶Return an arbitrary value according to a spec
nst
.
(eql #<structure-class common-lisp:hash-table>)
) &key test size key val) ¶(eql #<built-in-class common-lisp:symbol>)
) &key existing gensym exported nonnull package) ¶(eql #<built-in-class common-lisp:array>)
) &key rank dimens elem) ¶(eql #<built-in-class common-lisp:cons>)
) &key car cdr) ¶(eql #<built-in-class common-lisp:list>)
) &key elem length) ¶(eql #<built-in-class common-lisp:string>)
) &key noncontrol range) ¶(eql #<built-in-class common-lisp:vector>)
) &key elem length) ¶(eql #<built-in-class common-lisp:character>)
) &key noncontrol range) ¶(eql #<built-in-class common-lisp:complex>)
) &key) ¶(eql long-float)
) &key) ¶(eql #<built-in-class common-lisp:double-float>)
) &key) ¶(eql #<built-in-class common-lisp:single-float>)
) &key) ¶(eql short-float)
) &key) ¶(eql #<built-in-class common-lisp:ratio>)
) &key) ¶(eql #<built-in-class common-lisp:bignum>)
) &key) ¶(eql #<built-in-class common-lisp:fixnum>)
) &key) ¶(eql nst::scalar)
) &key) ¶(eql #<built-in-class common-lisp:float>)
) &key) ¶(eql #<built-in-class common-lisp:integer>)
) &key) ¶(eql #<built-in-class common-lisp:rational>)
) &key) ¶(eql #<built-in-class common-lisp:real>)
) &key) ¶(eql #<built-in-class common-lisp:number>)
) &key) ¶symbol
) &rest keyword-args &key &allow-other-keys) ¶(eql #<sb-pcl:system-class common-lisp:t>)
) &key) ¶Return an arbitrary type bounded by the given type.
nst
.
symbol
)) ¶(eql #<sb-pcl:system-class common-lisp:t>)
)) ¶(eql #<built-in-class common-lisp:symbol>)
)) ¶(eql #<built-in-class common-lisp:number>)
)) ¶(eql #<built-in-class common-lisp:real>)
)) ¶(eql #<built-in-class common-lisp:rational>)
)) ¶(eql #<built-in-class common-lisp:integer>)
)) ¶(eql #<built-in-class common-lisp:fixnum>)
)) ¶(eql bignum)
)) ¶(eql #<built-in-class common-lisp:ratio>)
)) ¶(eql #<built-in-class common-lisp:float>)
)) ¶(eql #<built-in-class common-lisp:single-float>)
)) ¶(eql #<built-in-class common-lisp:double-float>)
)) ¶(eql #<built-in-class common-lisp:complex>)
)) ¶nst
.
nst-assertion-failure
)) ¶args
.
nst
.
nst-assertion-condition
)) ¶nst
.
nst-assertion-failure
)) ¶Return a symbol naming the record
nst
.
test-record
)) ¶fixture-record
)) ¶group-record
)) ¶By default, return =nil= for a non-indexable artifact.
package
)) ¶nst
.
(eql :run)
) args) ¶(eql :run-test)
) args) ¶(eql :run-group)
) args) ¶(eql :run-package)
) args) ¶(eql :open)
) args) ¶nst
.
criterion-context-layer
)) ¶automatically generated reader method
nst
.
criterion-context-layer
)) ¶automatically generated writer method
nst
.
criterion-context-layer
)) ¶automatically generated reader method
nst
.
criterion-context-layer
)) ¶automatically generated writer method
nst
.
criterion-missing-mandatory-argument
)) ¶nst
.
unknown-arbitrary-domain
)) ¶nst
.
not-expected-form
)) ¶nst
.
permuter-frame
)) ¶Hook allowing us to sometimes do better than the usual pretty-printer for warnings.
nst
.
criterion-context-layer
)) ¶automatically generated reader method
nst
.
criterion-context-layer
)) ¶automatically generated writer method
nst
.
no-such-nst-test
)) ¶no-such-nst-group
)) ¶nst
.
group-result
) &rest args &key verbose dir file stream if-dir-does-not-exist if-file-exists) ¶symbol
) &rest args &key dir file verbose stream if-dir-does-not-exist if-file-exists) ¶nst
.
unknown-arbitrary-domain
)) ¶nst
.
method-context-layer
)) ¶automatically generated reader method
nst
.
method-context-layer
)) ¶automatically generated writer method
Return a string giving help for the argument names for an ~ NST REPL command.
nst
.
(eql nst-artifact-lookup-package::apply)
)) ¶(eql nst-artifact-lookup-package::whatis)
)) ¶(eql nst-artifact-lookup-package::unset)
)) ¶(eql nst-artifact-lookup-package::set)
)) ¶(eql nst-artifact-lookup-package::clear)
)) ¶(eql nst-artifact-lookup-package::detail)
)) ¶(eql nst-artifact-lookup-package::report)
)) ¶(eql nst-artifact-lookup-package::run)
)) ¶(eql nst-artifact-lookup-package::run-test)
)) ¶(eql nst-artifact-lookup-package::run-group)
)) ¶(eql nst-artifact-lookup-package::run-package)
)) ¶(eql nst-artifact-lookup-package::open)
)) ¶(eql nst-artifact-lookup-package::debug)
)) ¶(eql nst-artifact-lookup-package::help)
)) ¶nst
.
nst-assertion-result-check
)) ¶Return the long help message for an NST REPL command.
nst
.
(eql nst-artifact-lookup-package::apply)
)) ¶(eql nst-artifact-lookup-package::whatis)
)) ¶(eql nst-artifact-lookup-package::unset)
)) ¶(eql nst-artifact-lookup-package::set)
)) ¶(eql nst-artifact-lookup-package::clear)
)) ¶(eql nst-artifact-lookup-package::detail)
)) ¶(eql nst-artifact-lookup-package::report)
)) ¶(eql nst-artifact-lookup-package::run)
)) ¶(eql nst-artifact-lookup-package::run-test)
)) ¶(eql nst-artifact-lookup-package::run-group)
)) ¶(eql nst-artifact-lookup-package::run-package)
)) ¶(eql nst-artifact-lookup-package::open)
)) ¶(eql nst-artifact-lookup-package::debug)
)) ¶(eql nst-artifact-lookup-package::help)
)) ¶Return the display value of an NST property’s internal value.
Return the documentation string of an NST property.
Encode an NST property’s display value as an internal value.
Return the short help message for an NST REPL command.
nst
.
(eql nst-artifact-lookup-package::apply)
)) ¶(eql nst-artifact-lookup-package::whatis)
)) ¶(eql nst-artifact-lookup-package::unset)
)) ¶(eql nst-artifact-lookup-package::set)
)) ¶(eql nst-artifact-lookup-package::clear)
)) ¶(eql nst-artifact-lookup-package::detail)
)) ¶(eql nst-artifact-lookup-package::report)
)) ¶(eql nst-artifact-lookup-package::run)
)) ¶(eql nst-artifact-lookup-package::run-test)
)) ¶(eql nst-artifact-lookup-package::run-group)
)) ¶(eql nst-artifact-lookup-package::run-package)
)) ¶(eql nst-artifact-lookup-package::open)
)) ¶(eql nst-artifact-lookup-package::debug)
)) ¶(eql nst-artifact-lookup-package::help)
)) ¶nst
.
method-context-layer
)) ¶automatically generated reader method
nst
.
method-context-layer
)) ¶automatically generated writer method
nst
.
nst-deprecation-warning-mixin
)) ¶nst
.
no-nst-groups-in-package
)) ¶This function formats the lambda list of an
NST REPL command for display in the online-help system. The macro
def-nst-interactive-command that expands a REPL command definition
into the underlying Lisp defmethods uses this function to generate
the bodies of the help-related methods, so this function must be
available from compile-time forward.
nst
.
nst-deprecation-warning-mixin
)) ¶nst
.
criterion-missing-mandatory-argument
)) ¶Receives a reporting structure (or list of them); returns a
six-value summary of the results:
- A symbol, one of: :error :fail :warn :info :clear
- The total number of named checks.
- The number passed.
- The number raising an error.
- The number failing.
- The number giving a warning.
nst
.
null
) &optional code total passed erred failed warned other) ¶cons
) &optional code total passed erred failed warned other) ¶multi-results
) &optional code total passed erred failed warned other) ¶package-result
) &optional code total passed erred failed warned other) ¶group-result
) &optional code total passed erred failed warned other) ¶check-result
) &optional code total passed erred failed warned other) ¶Top-level command interpreter for the NST tester
nst
.
(eql nst-artifact-lookup-package::apply)
) &rest g0) ¶(eql nst-artifact-lookup-package::whatis)
) &rest g0) ¶(eql nst-artifact-lookup-package::unset)
) &rest g0) ¶(eql nst-artifact-lookup-package::set)
) &rest g0) ¶(eql nst-artifact-lookup-package::clear)
) &rest g0) ¶(eql nst-artifact-lookup-package::detail)
) &rest g0) ¶(eql nst-artifact-lookup-package::report)
) &rest g0) ¶(eql nst-artifact-lookup-package::run)
) &rest g0) ¶(eql nst-artifact-lookup-package::run-test)
) &rest g0) ¶(eql nst-artifact-lookup-package::run-group)
) &rest g0) ¶(eql nst-artifact-lookup-package::run-package)
) &rest g0) ¶(eql nst-artifact-lookup-package::open)
) &rest g0) ¶(eql nst-artifact-lookup-package::debug)
) &rest g0) ¶(eql nst-artifact-lookup-package::help)
) &rest g0) ¶Used to determine whether a context-layer should be displayed. Methods may safely refer and mutate the hashtable stored in the special variable -context-display-state- .
nst
.
method-context-layer
)) ¶criterion-context-layer
)) ¶Display a property value (presumably) to the REPL.
nst
.
no-such-nst-test
)) ¶test
.
Provide debugging information about a group.
nst
.
group-record
)) ¶Provide debugging information about a test.
nst
.
group-record
) (ts test-record
)) ¶Return non-nil if an item is a group record.
Mixin of field used in deprecation warnings
nst
.
nst-deprecation-warning-mixin
.
warning
.
nst
.
nst-deprecation-warning-mixin
.
style-warning
.
nst
.
nst-deprecation-warning-mixin
.
style-warning
.
A single note issued in criteria checking.
context - the surrounding criteria structure, a list of context-layer structs
stack - the stack of values at the note point
format, args - further details; this string and list may e.g. be provided to
cl:format
** IMPORTANT ** Do not use check-note-format or check-note-args! Do not assume that the format slot is a string! The API for these slots is apply-check-note-formatter, which checks for function values (or other types allowed as a formatter since the time when this docstring was written).
Overall check result structure, containing notes of four distinct types. A
note is an instance of the check-note structure below. The four note types are:
warnings - generated warnings
failures - criteria which are not met, but not a Lisp error
errors - Lisp errors
skipped - skipped information if the test is skipped.
info - supplimentary information
Each of these fields is a list; warnings, failures and errors are check-note
instances, and the info field is of any value.
nst
.
1
nst::*nst-group-name*
nst::*nst-check-user-name*
A note issued in regards to a thrown error.
Information associated with a fixture definition
nst
.
structure-object
.
common-lisp
.
common-lisp
.
Superclass of NST group definitions.
nst
.
structure-object
.
common-lisp
.
(make-hash-table :test (quote eq))
Overall group result structure, mapping checks to results by name.
Multiple results structure.
Record for tracking the artifacts which NST associates with a name.
Overall package result structure, mapping groups to results by name.
Statistics common to the different result summaries.
nst
.
structure-object
.
0
0
0
0
common-lisp
.
0
0
(multiple-value-list (get-decoded-time))
Structure storing parsed NST test records.
- =group= :: The group-record structure (not the symbolic name) of the group
containing this test.
nst
.
structure-object
.
common-lisp
.
Superclass of records of test criterion.
nst
.
A record of test criterion
criterion - the criterion symbol itself
criterion-args - arguments to the criterion
given-stack - the stack of values assessed by the criterion
A record of method-based test invocation.
Jump to: | %
(
A B C D E F G H I J K L M N O P R S T U W |
---|
Jump to: | %
(
A B C D E F G H I J K L M N O P R S T U W |
---|
Jump to: | *
+
A B C D E F G H I K L M N O P Q R S T W |
---|
Jump to: | *
+
A B C D E F G H I K L M N O P Q R S T W |
---|
Jump to: | A C D E F G I J M N P R S T U X |
---|
Jump to: | A C D E F G I J M N P R S T U X |
---|