The nst Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The nst Reference Manual

This is the nst Reference Manual, version 4.1.0, generated automatically by Declt version 2.3 "Robert April" on Tue Feb 20 09:07:47 2018 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

This document is the manual and users' guide to the 4.1.(x) series of the NST test framework, last updated for 4.1.0. NST is a unit test system for Common Lisp which provides support for test fixture data, stateful setup and cleanup of tests, grouping of tests, and (we think!) a useful runtime interface. Suggestions and comments are welcome. The files in the NST distribution's self-test directory, especially self-test/core/builtin-checks.lisp, holds the NST tests for NST and contain many examples (some of which we have adapted for this manual). Known bugs and infelicities, platform-specific release notes, and other technical materials are available via the link on NST's CLiki page, cliki.net/NST.

Contributors. The primary author of both NST and this manual is John Maraist (Smart Information Flow Technologies, 211 North First Street, Suite 300, Minneapolis, MN 55401; jmaraist at sift.info). Robert P. Goldman provided guidance, comments and suggestions through the development. Other contributors include Michael J.S. Pelican, Steven A. Harp, Michael Atighetchi and Patrick Stein.

Fixtures

Fixtures are data structures and values which may be referred to by name during testing. NST provides the ability to use fixtures across multiple tests and test groups, and to inject fixtures into the runtime namespace for debugging. A set of fixtures is defined using the def-fixtures macro:

(def-fixtures fixture-name ( [ :outer FORM ]
                             [ :inner FORM ]
                             [ :setup FORM ]
                             [ :cleanup FORM ]
                             [ :startup FORM ]
                             [ :finish FORM ]
                             [ :documentation STRING ]
                             [ :cache FLAG ]
                             [ :export-names FLAG ]
                             [ :export-fixture-name FLAG ]
                             [ :export-bound-names FLAG ] )
  ( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] )
  ...
  ( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] ) )

When a fixture is attached to a test or test group, each NAME defined in that fixture becomes available in the body of that test or group as if let* bound to the corresponding FORM. A fixture in one set may refer back to other fixtures in the same set (again as let*) but forward references are not allowed.

The four arguments startup, finish, setup and cleanup specify forms which are run everytime the fixture is applied to a group or test. The startup (respectively finish) form is run before fixtures are bound (after their bindings are released). These forms are useful, for example, to initialize a database connection from which the fixture values are drawn. The setup form is run after inclusion of names from fixture sets, but before any tests from the group. The cleanup form is normally run after the test completes, but while the fixtures are still in scope. Normally, the cleanup form will not be run if the setup form raises an error, and the finish form will not be run if the startup form raises an error; although the user is able to select (perhaps unwisely) a restart which disregards the error.

The names of a fixture and the names it binds can be exported from the package where the fixture is defined using the export-bound-names and export-fixture-name arguments. The default value of both is the value of export-names, whose default value is nil.

The cache option, if non-nil, directs NST to evaluate a fixture's form one single time, and re-use the resulting value on subsequent applications of the fixture. Note that if this value is mutated by the test cases, test behavior may become unpredictable! However this option can considerably improve performance when constant-valued fixtures are applied repeatedly. Caching may be set on or off (the default is off) for the entire fixture set, and the setting may vary for individual fixtures.

Examples of fixture definitions:

(def-fixtures f1 ()
  (c 3)
  (d 'asdfg))
(def-fixtures f2 (:special ((:fixture f1)))
  (d 4)
  (e 'asdfg)
  (f c))
(def-fixtures f3 ()
  ((:cache t)   g (ackermann 1 2))
  ((:cache nil) h (factorial 5)))

To cause a side-effect among the evaluation of a fixture's name definitions, nil can be provided as a fixture name. In uses of the fixture, NST will replace nil with a non-interned symbol; in documentation such as form :whatis, any =nil=s are omitted.

The with-fixtures macro faciliates debugging and other non-NST uses of fixtures sets:

(with-fixtures (FIXTURE FIXTURE ... FIXTURE)
  FORM
  FORM
  ...
  FORM)

This macro evaluates the forms in a namespace expanded with the bindings provided by the fixtures.

Test groups

The def-test-group form defines a group of the given name, providing one instantiation of the bindings of the given fixtures to each test. Groups can be associated with fixture sets, stateful initialization, and stateful cleanup.

(def-test-group NAME (FIXTURE FIXTURE ...)
  (:aspirational FLAG)
  (:setup FORM FORM ... FORM)
  (:cleanup FORM FORM ... FORM)
  (:startup FORM FORM ... FORM)
  (:finish FORM FORM ... FORM)
  (:each-setup FORM FORM ... FORM)
  (:each-cleanup FORM FORM ... FORM)
  (:include-groups GROUP GROUP ... GROUP)
  (:documentation STRING)
  TEST
  ...
  TEST)

Arguments:

Testing values

Individual unit tests are encoded with the def-test form:

(def-test NAME ( [ :group GROUP-NAME ]
                  [ :setup FORM ]
                  [ :cleanup FORM ]
                  [ :startup FORM ]
                  [ :finish FORM ]
                  [ :fixtures (FIXTURE FIXTURE ... FIXTURE) ]
                  [ :aspirational FLAG ]
                  [ :documentation STRING ] )
    criterion
 FORM ... FORM)

(def-test NAME criterion &body (:seq FORM))

The SETUP, CLEANUP, STARTUP, FINISH and FIXTURES are just as for fixtures and test groups, but apply only to the one test. The CRITERION is a list or symbol specifying the properties which should hold for the =FORM=s.

When a test is not enclosed within a group body, a group name must be provided by the GROUP option. When a test is enclosed within a group body, the GROUP option is not required, but if provided it must agree with the group name.

When there are no SETUP, CLEANUP, STARTUP, FINISH or FIXTURES arguments, the NAME may be given without parentheses. Likewise, any criterion consisting of a single symbol, e.g. (:pass), may be abbreviated as just the symbol without the parentheses, e.g. :pass.

The :documentation form provides a documentation string in the standard Lisp sense. Since documentation strings are stored against names, and since the same name can be used for several tests (so long as they are all in different packages), documentation strings on tests may not be particularly useful.

An aspirational test is one which verifies some part of an API or code contract which may not yet be implemented. When a group is marked aspirational, all tests within the group are taken to be aspirational as well. At this point, there is no particular processing for aspirational tests and groups, but we hope to implement it at some point in the future.

The def-check form is a deprecated synonym for def-test.

Basic criteria

Criterion true

The :true criterion expects one form, which is evaluated at testing time; the criterion requires the result to be non-nil.

Criterion eq

The :eq criterion checks a form using eq. The criterion argument and the form under test are both evaluated at testing time.

Usage:

(:eq TARGET)

Example:

(def-test eq1 (:eq 'b) (cadr '(a b c)))

Criterion symbol

The :symbol criterion checks that its form under test evaluates to a symbol which is eq to the symbol name given as the criterion argument.

Usage:

(:symbol NAME)

Example:

(def-test sym1  (:symbol a) (car '(a b c)))

Example of a test which fails:

(def-test sym1x (:symbol a) (cadr '(a b c)))

Criterion eql

The :eql criterion checks a form using eql. The criterion argument and the form under test are both evaluated at testing time.

Usage:

(:eql TARGET)

Example:

(def-test eql1 (:eql 2) (cadr '(1 2 3)))

Criterion equal

The :equal criterion checks a form using eql. The criterion argument and the form under test are both evaluated at testing time.

Criterion equalp

The :equalp criterion checks a form using equalp. The criterion argument and the form under test are both evaluated at testing time.

Usage:

(:equalp TARGET)

Criterion forms-eq

The :forms-eq criterion compares its two forms under test using eq. The forms are both evaluated at testing time.

Usage:

(:forms-eq)

Example:

(def-test eqforms1 :forms-eq (cadr '(a b c)) (caddr '(a c b)))

Criterion forms-eql

The :forms-eql criterion compares its two forms under test using eql. The two forms under test are both evaluated at testing time.

Usage:

(:forms-eql)

Example:

(def-test eqlforms1 :forms-eql (cadr '(a 3 c)) (caddr '(a c 3)))

Criterion forms-equal

The :forms-equal criterion compares its two forms under test using equal. The forms are both evaluated at testing time.

Usage:

(:forms-equal)

Criterion predicate

The :predicate criterion applies a predicate to the result of evaluating its form under test. The criterion argument is a symbol (unquoted) or a lambda expression; at testing time, the forms under test are evaluated and passed to the denoted function. The criterion expects that the result of the function is non-nil.

Usage:

(:predicate PRED)

Example:

(def-test pred1 (:predicate numberp) 3)

Example of a test which fails:

(def-test pred2 (:predicate eql) (+ 1 2) 3)

Criterion err

The :err criterion evaluates the form under test, expecting the evaluation to raise some condition. If the class argument is supplied, the criterion expects the raised condition to be a subclass. Note that the name of the type should not be quoted; it is not evaluated.

Usage:

(:err &KEY (TYPE CLASS))

Examples:

(def-test err1 (:err :type error) (error "this should be caught"))
(def-test err2 (:err) (error "this should be caught"))

Criterion perf

The :perf criterion evaluates the forms under test at testing time, checking that the evaluation completes within the given time limit.

Usage:

(:perf &KEY (MS MILLISECS) (SEC SECONDS) (MIN MINUTES))

Example:

(def-test perf1 (:perf :min 2) (ack 3 5))

Compound criteria

Criterion not

The :not criterion passes when testing according to subcriterion fails (but does not throw an error).

Usage:

(:not SUBCRITERION)

Example:

(def-test not1 (:not (:symbol b)) 'a)

Criterion all

The :all criterion brings several other criteria under one check, and verifies that they all pass.

Usage:

(:all SUBCRITERION ... SUBCRITERION)

Example:

(def-check not1 ()
    (:all (:predicate even-p)
          (:predicate prime-p))
  2)

Criterion any

The :any criterion passes when any of the subordinate criteria pass.

Usage:

(:any SUBCRITERION ... SUBCRITERION)

Example:

(def-check not1 ()
    (:any (:predicate even-p)
          (:predicate prime-p))
  5)

Criterion apply

The :apply criterion first evaluates the forms under test, applying function to them. The overall criterion passes or fails exactly when the subordinate criterion with the application's multiple result values.

Usage:

(:apply FUNCTION CRITERION)

Example:

(def-test applycheck (:apply cadr (:eql 10)) '(0 10 20))

Criterion check-err

The :check-err criterion is like :err, but proceeds according to the subordinate criterion rather than simply evaluating the input forms.

Usage:

(:check-err CRITERION)

Example:

(def-test check-err1
    (:check-err :forms-eq)
  'asdfgh (error "this should be caught"))

Criterion progn

The :progn criterion first evaluates the =FORM=s in order, and then proceeds with evaluation of the forms under test according to the subordinate criterion.

Usage:

(:progn FORM ... FORM SUBCRITERION)

Example:

(def-test form1 (:progn (setf zz 3) (:eql 3)) zz)

Criterion proj

The :proj criterion rearranges the forms under test by selecting a new list according to the index numbers into the old list. Checking of the reorganized forms continues according to the subordinate criterion.

Usage:

(:proj INDICES CRITERION)

Example:

(def-test proj-1
    (:proj (0 2) :forms-eq)
  'a 3 (car '(a b)))

Criterion with-common-criterion

The :with-common-criterion criterion applies one criterion to several data forms.

Usage:

(:with-common-criterion [ CRITERION | (CRITERION ARG ... ARG) ]
                        (FORM ...  FORM) ... (FORM ...  FORM) )

Criterion applying-common-criterion

The :applying-common-criterion criterion applies one criterion to several pairs of criterion arguments and data forms.

Usage:

(:applying-common-criterion [ CRITERION | (CRITERION ARG ... ARG) ]
                            ( ((ARG ... ARG) (FORM ... FORM))
                              ...
                              ((ARG ... ARG) (FORM ... FORM)) )
                            ...
                            ( ((ARG ... ARG) (FORM ... FORM))
                              ...
                              ((ARG ... ARG) (FORM ... FORM)) ) )

Criteria for multiple values

Criterion values

The :values criterion checks each of the forms under test according to the respective subordinate criterion.

Usage:

(:values SUBCRITERION ... SUBCRITERION)

Criterion value-list

The :value-list criterion converts multiple values into a single list value.

Usage:

(:value-list FURTHER)

Criterion drop-values

The :drop-values criterion checks the primary value according to the subordinate criterion, ignoring any additional returned values from the evaluation of the form under test.

Usage:

(:drop-values CRITERION)

Criteria for lists

Criterion seq

The :seq criterion evaluates its input form, checks each of its elements according to the respective subordinate criterion, and passes when all of them pass.

Usage:

(:seq SUBCRITERION ... SUBCRITERION)

Example:

(def-check seqcheck
    (:seq (:predicate symbolp) (:eql 1) (:symbol d))
  '(a 1 d))

Criterion each

The :each criterion evaluates the form under test, expecting to find a list as a result. Expects that each argument of the list according to the subordinate criterion, and passes when all of these checks pass.

Usage:

(:each CRITERION)

Example:

(def-test each1 (:each (:symbol a)) '(a a a a a))

Criterion permute

The :permute criterion evaluates the form under test, expecting to find a list as a result. The criterion expects to find that some permutation of this list will satisfy the subordinate criterion.

Usage:

(:permute CRITERION)

Example:

(def-test permute1 (:permute (:each (:eq 'a))) '(a a))
(def-check permute2
    (:permute (:seq (:symbol b)
                    (:predicate symbolp)
                    (:predicate numberp)))
  '(1 a b))

Criterion alist

The :alist criterion evaluates the form under test, expecting to find an association list as a result. Using the two given function specs to test the keys (during retrieval, via assoc) and the values, the criterion enforces that the association lists contains exactly equivalent keys, mapping to respective equivalent values. Implemented using :alist* plus a check of the list length, which could be incorrect if the criterion lists duplicate keys.

Usage:

(:alist KEY-TEST-FN VALUE-TEST-FN (KEY VALUE) ... (KEY VALUE))

Criteria for vectors

Criterion across

The :across criterion is like :seq, but for a vector instead of a list.

Usage:

(:across SUBCRITERION ... SUBCRITERION)

Example:

(def-check across1
    (:across (:predicate symbolp) (:eql 1))
  (vector 'a 1))

Criteria for classes

Criterion slots

The :slots criterion evaluates its input form, and passes when the value at each given slot satisfies the corresponding subordinate constraint.

Usage:

(:slots (SLOT-NAME SUBCRITERION) ... (SLOT-NAME SUBCRITERION))

Example:

(defclass classcheck ()
  ((s1 :initarg :s1 :reader get-s1)
   (s2 :initarg :s2)
   (s3 :initarg :s3)))
 (def-test slot1
     (:slots (s1 (:eql 10))
             (s2 (:symbol zz))
             (s3 (:seq (:symbol q) (:symbol w)
                       (:symbol e) (:symbol r))))
   (make-instance 'classcheck
     :s1 10 :s2 'zz :s3 '(q w e r)))

Programmatic and debugging criteria

Criterion pass

The :pass is a trivial test which always passes.

Example:

(def-test passing-test :pass 3 4 "sd")

Criterion warn

The :warn criterion issues a warning. The format string and arguments should be suitable for the Lisp format function.

Usage:

(:warn FORMAT-STRING FORM ... FORM)

Example:

(:warn "~{}d is not a perfect square" 5)

Criterion info

The :info criterion adds an informational note to the check result.

Usage:

(:info STRING SUBCRITERION)

Example:

(def-test known-bug (:info "Known bug" (:eql 3)) 4)

Testing processes

The test criteria of the previous section all examined the result of evaluating the forms under test. This section presents NST's criteria for validating the process of a computation, specifying assertions which should hold at the initial, intermediate and final points of the process.

Criterion eval

(def-criterion ( [ :check-warnings FLAG ] [ :muffle-warnings FLAG ]
                 [ :attempt-continue FLAG ] [ :force-continue FLAG ] )
  FORM
  ...
  FORM)

The :eval criterion executes its forms, expecting calls to various assertion functions to check intermediate states of an arbitrarily-long process.

Defining tests

The def-eval-test macro abbreviates a call to def-test with a single :eval criterion. Its arguments are just as for def-test and :eval.

(def-eval-test (NAME [ :group GROUP-NAME ]
                     [ :setup FORM ]
                     [ :cleanup FORM ]
                     [ :startup FORM ]
                     [ :finish FORM ]
                     [ :fixtures (FIXTURE ... FIXTURE) ]
                     [ :documentation STRING ]
                     [ :check-warnings FLAG ]
                     [ :muffle-warnings FLAG ]
                     [ :attempt-continue FLAG ]
                     [ :force-continue FLAG ] )
  FORM
  ...
  FORM)

(def-eval-test NAME
  FORM
  ...
  FORM)

Asserting properties

The assert-non-nil function is a unary predicate for use within the forms evaluated for an :eval criterion. It succeeds whenever the null function returns nil.

The assert-not-eq function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using eq, succeeds whenever that call returns nil.

The assert-not-equalp function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using equalp, succeeds whenever that call returns nil.

The assert-null function is a unary predicate for use within the forms evaluated for an :eval criterion. It succeeds whenever the null function returns non-nil.

The assert-criterion macro asserts that an NST criterion should pass.

(assert-criterion ( [ :msg-format FORMAT-STRING ]
                    [ :msg-args FORMAT-ARGUMENTS ]
                    [ :fatal FLAG [
                    [ :fail-on-warning FLAG ] )
  FORM
  ...
  FORM)

The assert-equal function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using equal, succeeds whenever that call returns non-nil.

The assert-zero function is a unary predicate for use within the forms evaluated for an :eval criterion. It succeeds whenever the zerop function returns non-nil.

The assert-equalp function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using equalp, succeeds whenever that call returns non-nil.

The assert-not-equal function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using equal, succeeds whenever that call returns nil.

The assert-eq function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using eq, succeeds whenever that call returns non-nil.

The assert-eql function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using eql, succeeds whenever that call returns non-nil.

The assert-not-eql function is a unary predicate for use within the forms evaluated for an :eval criterion. It compares the expected and tested values using eql, succeeds whenever that call returns nil.

Defining new assertion functions

Macro macro def-unary-predicate-assert creates an assertion function using the result of a call to a unary predicate. A non-nil result from the predicate corresponds to a successful assertion.

(def-unary-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE
                            [ :message-defvar NAME ]
                            [ :pred-name NAME ]
                            [ :doc-state-flag BOOL ] )

Macro def-binary-predicate-assert uses a binary predicate as the basis for an assertion function just as def-unary-predicate-assert uses a unary predicate. This macro's arguments are just as for def-unary-predicate-assert.

Macro def-unary-negated-predicate-assert uses the negated result of a unary predicate as the basis of an assertion function. This macro's arguments are just as for def-unary-predicate-assert.

Macro def-binary-negated-predicate-assert uses the negated result of a binary predicate as the basis for an assertion function just as def-unary-negated-predicate-assert uses the negated result of a unary predicate. This macro's arguments are just as for def-unary-predicate-assert.

A simpler process checker

Criterion process

The :process criterion allows simple interleaving of Lisp function calls and NST checks, to allow checking of intermediate states of an arbitrarily-long process.

This criterion takes as its body a list of forms. The first element of each form should be a symbol:

The :process criterion takes no value arguments in a def-test.

For example:

(def-test process-1
    (:process (:eval (setf zzz 0))
              (:check (:true-form (eql zzz 0)))
              (:eval (incf zzz))
              (:check (:true-form (eql zzz 1)))
              (:eval (incf zzz))
              (:check (:true-form (eql zzz 2)))))

Testing invariants against sampled data

The sample criterion provides random generation of data for validating program properties. Our approach is based on Claessen and Hughes's Quickcheck (Koen Claessen and John Hughes, ``QuickCheck: a lightweight tool for random testing of Haskell programs,'' from Proceedings of the International Conference on Functional Programming, 2000. QuickCheck papers, code and other resources are available at www.cs.chalmers.se/~rjmh/QuickCheck).

This style of testing is somewhat more complicated than specific tests on single, bespoke forms. There are two distinct efforts, which we address in the next two sections: describing how the sample data is to be generated, and specifying the test itself.

Generating sample data

Data generation is centered around the generic function arbitrary.

This function takes a single argument, which determines the type of the value to be generated. For simple types, the name of the type (or the class object, such as returned by find-class by itself is a complete specification. For more complicated types, arbitrary can also take a list argument, where the first element gives the type and the remaining elements are keyword argument providing additional requirements for the generated value.

NST provides methods of arbitrary for many standard Lisp types. Non-scalar types are associated with additional keyword arguments for constraints on value generation.

Beyond those standard Lisp types, NST provides the type scalar as a supertype of the numeric types plus character, string and symbol. Users may extend this definition to include additional type specifications, as we discuss below. Types are not associated with scalar are referred to as compound (although there is no corresponding type specification). To avoid generating structures too large to hold in memory, NST provides the global variable *max-compound-structure-depth* and the macro compound-structure.

The *max-compound-structure-depth* variable sets the maximum nesting depth of compound data structures: beyond that depth, scalar rather than t is the default element generator. This restriction does not apply to explicitly specified element types, only to the use of defaults.

The compound-structure macro wraps substructure which should be considered compound for the limits set by *max-compound-structure-depth*.

New type specifications for invariant-testing. are defined with the def-arbitrary-instance-type macro.

(def-arbitrary-instance-type ( SPEC-NAME [ :params FORMALS ]
                                         [ :scalar BOOL ]
                                         [ :key KEY ] )
  FORM
  FORM
  ...
  FORM)

Invariants as tests

Defining test criteria

The criteria used in test forms decide whether, when and how to use the forms under test and the forms and subcriteria provided to each test criterion. Criteria receive their arguments as forms, and may examine them with or without evaluation, as the particular criterion requires. NST provides two mechanisms for defining new criteria, and a number of support functions for use within these definitions. The simpler, but more limited, way to define a new criterion is by specifying how it should be rewritten to another criterion. The def-criterion-alias macro provides this mechanism, which we discuss in Section \ref{def-criterion-alias-section}. The def-criterion macro provides the more general mechanism for criteria definition, where Lisp code produces a result report from the forms under test and criterion's forms and subcriteria. We discuss def-criterion in Section \ref{def-criterion-section}. We discuss the NST API for creating these result reports in Section \ref{criteria-forms-report-section}, and for recursive processing of subcriteria in Section \ref{subcriteria-section}.

Aliases over criteria

The simplest mechanism for defining a new criterion involves simply defining one criterion to rewrite as another using def-criterion-alias.

(def-criterion-alias (name (:seq arg))
  [ doc-string ]
  expansion)

The body of the expansion should be a Lisp form which, when evaluated, returns an S-expression quoting the new criterion which the rewrite should produce. The arg are passed as for Lisp macros: they are not evaluated and are most typically comma-inserted into a backquoted result. For example:

(def-criterion-alias (:forms-eq) `(:predicate eq))
(def-criterion-alias (:symbol name) `(:eq ',name))

Reporting forms

NST provides functions both for building test reports, and for adding information to a report.

The make-success-report function indicates a successful test result.

(make-success-report)

Note that some older examples show (make-check-result), (emit-success) or (check-result). The former is an internal function and should not be used from outside the core NST files. The latter two are deprecated.

The make-failure-report function returns a report of test failure.

(make-failure-report FORMAT ARGS)

The format-string and args are as to the Common Lisp function format. The emit-failure function is an older, deprecated version of this function.

Function make-warning-report is like make-failure-report, but provides supplimentary information as a warning.

(make-warning-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ])

The emit-warning function is an older, deprecated version of this function.

Function make-error-report produces a report of an error during test execution.

(make-error-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ] )

For use within user-defined NST criteria: add an error to a result.

(add-error RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ] )

For use within user-defined NST criteria: add a failure to a result.

(add-failure RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ])

For use within user-defined NST criteria: add an info note to a result.

(add-info RESULT-REPORT INFO-ITEM)

For use within user-defined NST criteria: add a warning to a result.

The add-warning function adds an warning to a result record. The item can be any of a Lisp warning, an NST check-note or a format string; in the first two cases, no additional arguments should be provided.

(add-warning RESULT-REPORT [ :format ITEM ] [ :args ARGUMENT-LIST ] )

The helper function wrap-thrown-lisp-warning creates an NST check-note object from a standard Lisp warning.

Processing subcriteria

The criterion itself can contain subcriteria which can be incorporated into the main criterion's assessment. NST provides two functions which trigger testing by a subcriterion, each returning the check's result report.

The check-criterion-on-value function can be called from within a criterion body to verify that a value adheres to a criterion.

(check-criterion-on-value CRITERION VALUE)

This function verifies that the values return by evaluating the form adheres to the criterion.

(check-criterion-on-form CRITERION FORM)

General criteria definitions

The def-criterion macro defines a new criterion for use in NST tests. These criteria definitions are like generic function method definitions with two sets of formal parameters: the forms provided as the actual parameters of the criterion itself, and the values arising from the evaluation of the forms under test.

(def-criterion (name criterion-lambda-list values-lambda-list)
  [ doc-string )
  form
  form
  ...
  form)

Examples:

(def-criterion (:true () (bool))
  (if bool
      (make-success-report)
      (make-failure-report :format "Expected non-null, got: ~s"
                    :args (list bool))))

(def-criterion (:eql (target) (actual))
  (if (eql (eval target) actual)
      (make-success-report)
      (make-failure-report :format "Not eql to value of ~s"
                    :args (list target))))

The runtime system

The runtime system provides several operations for scheduling and running tests, and debugging failing and erring tests.

User-level NST operations are accessible from the REPL via the nst-cmd macro.

(nst-cmd NST-COMMAND ARG ... ARG)

Where a particular system supports the facility (Currently Allegro, and SBCL under ACL-REPL) the top-level alias :nst provides a shorthand to this function. Calling nst or nst-cmd without a command argument repeats the last test-executing command. For the sake of brevity we use the :nst shorthand below.

Command help

The :help command gives a complete inventory of runtime system commands.

Usage:

:nst :help

There are a number of commands for running tests, but most of the time only one will be needed:

Command run

The :run command executes all tests in the named package, or in the named group, or runs the named test. It is not necessary to prefix the name with a package prefix. The name does not need to be prefix-qualified, but if the name is ambiguous then :run will simply report the possible interpretations.

Usage:

:nst :run NAME

Command run-package

The :run-package command executes all tests associated with groups in the named packages, and reports the test results afterwards.

Usage:

:nst :run-package (:SEQ PACKAGE)

Command run-group

The :run-group command executes all tests associated with the name groups, and reports the test results afterwards. The group name should be package-qualified.

Usage:

:nst :run-group (:SEQ GROUP)

Command run-test

The :run-test command executes the given test. Both the group and test name should be package-qualified.

Usage:

:nst :run-test GROUP TEST

One further command for running a test is useful when writing and debugging the tests themselves:

Command apply

The :apply command assesses whether a test criterion prints the uses to which a particular name has been applied in an NST session.

Usage:

:nst :apply NAME

There are two commands for (re)printing the results of tests:

Command report

The :report command summarizes successes, failures and errors in tests. It reports either for the named artifact, or for all recently-run tests.

Usage:

:nst :run
:nst :run PACKAGE
:nst :run GROUP
:nst :run GROUP TEST

Command detail

The :detail command gives detailed information about individual test results.

Usage:

:nst :detail
:nst :detail PACKAGE
:nst :detail GROUP
:nst :detail GROUP TEST

The set and unset display and adjust NST's configuration.

Command set

The :set command assigns or displays the values of NST runtime switches.

Usage:

:nst :set PROPERTY
:nst :set PROPERTY VALUE

There are currently three properties which can be manipulated by set and unset:

The :verbosity switch controls the level of NST's output.

Usage:

:nst :verbose SETTING

Valid settings are:

The :report and :detail commands operate by setting minimum levels of verbosity.

The :debug-on-error switch controls NST's behavior on errors. When non-nil, NST will break into the debugger when it encounters an error.

Usage:

:nst :debug-on-error FLAG

The :debug command is a short-cut for setting this property.

The :debug-on-fail switch controls NST's behavior when a test fails When non-nil, NST will break into the debugger when it encounters a failing test.

Usage:

:nst :debug-on-fail FLAG

This behavior is less useful than it may seem; by the time the results of the test are examined for failure, the stack from the actual form evaluation will usually have been released. Still, this switch is useful for inspecting the environment in which a failing test was run.

Note that both :debug-on-error and :debug-on-fail apply in the case of an error; if the latter is set but the former is not, then the debugger will be entered after an erring test completes.

The :debug command is a short-cut for setting this property.

Command unset

The :unset command clears the values of NST runtime switches.

Usage:

:nst :unset PROPERTY

The :backtraces switch, when non-=nil=, directs NST to attempt to capture the Lisp backtrace of errors in tests.

Usage:

:nst :backtraces FLAG

This property is only available on platform which allow programmatic examination of backtraces, which is not standardized in Common Lisp; currently we have implemented this feature on Allegro only.

This property has a complicated default setting. Firstly, if the symbol ='common-lisp-user::*nst-generate-backtraces*= is bound when NST loads, NST will use its value as the initial value for this property. Otherwise by default, on MacOS systems the property initializes to nil because of a known error on that system, but this setting can be overriden by the property :nst-unsafe-allegro-backtraces. Finally, if none of these issues apply, the initial value is t.

This property is only available on platform which allow programmatic examination of backtraces, which is not standardized in Common Lisp; currently we have implemented this feature on Allegro only.This property has a complicated default setting. Firstly, if the symbol ='common-lisp-user::*nst-generate-backtraces*= is bound when NST loads, NST will use its value as the initial value for this property. Otherwise by default, on MacOS systems the property initializes to nil because of a known error on that system, but this setting can be overriden by the property :nst-unsafe-allegro-backtraces. Finally, if none of these issues apply, the initial value is ==.

The above NST commands are governed by a number of global variables. In general, interactive use of NST should not require direct access to these variables, but when automating NST operations may require changing, or creating a new dynamic scope for, their settings.

User variable *debug-on-error*: if non-nil, will break into the Lisp REPL debugger upon encountering an unexpected error. If nil, will record the error and continue with other tests.

User variable *debug-on-fail*: if non-nil, will break into the Lisp REPL debugger upon encountering a test which fails. If nil, will record the failure and continue with other tests. This variable is useful inspecting the dynamic environment under which a test was evaluated.

User variable *default-report-verbosity* determines the default value for *nst-verbosity* when printing reports (2 by default).

Determines the output stream to which NST should print its output (*standard-output* by default).

Command open

The :open command injects the binding given by the fixture set into the current package.

Usage:

:nst :open (:SEQ FIXTURE)

Example:

CL-USER(75): (nst:def-fixtures small-fixture ()
                (fix-var1 3)
                (fix-var2 'asdfg))
NIL
CL-USER(76): (boundp 'fix-var1)
NIL
CL-USER(77): :nst :open small-fixture
Opened fixture SMALL-FIXTURE.
CL-USER(78): fix-var1
3
CL-USER(79):

Fixtures can be opened into a different package than where they were first defined, but these bindings are in addition to the bindings in the original package, and are made by a symbol import to the additional package.

Integration with ASDF

NST's integration with ASDF is a work in progress. This section described the current integration, the ways we expect it to change, and a less-flexible and lower-level, but likely more stable, alternative integration technique.

NST's ASDF systems

From version 1.2.2, the system :asdf-nst provides two classes for ASDF system definitions, asdf:nst-test-runner and asdf:nst-test-holder.

Up to NST 1.2.1 :asdf-nst provided a single class asdf:nst-testable, and in the future we plan to reunify the current two classes into a single class again. However our first implementation required NST to be loaded even when a system was not being tested, because we had no way to distinguish the source code associated with testing from production code. We plan to solve this problem with a new file type nst-file in a future version of NST. This file type not be compiled or loaded for the compile-op or load-op of the system, only for its test-op.

Test-running systems

ASDF systems of the asdf:nst-test-runner class do not themselves contain NST declarations in their source code, but may identify other systems which do, and which should be tested as a part of testing the given system. These systems also allow local definitions of NST's configuration for the execution of their tests.

Specify that a system runs NST tests by providing :class asdf:nst-test-runner argument to asdf:defsystem. Use the :nst-systems argument to name the systems which house the actual unit tests:

Another optional argument to an nst-test-runner system definition is:

Test-containing systems

The asdf:nst-test-holder class is a subclass of nst-test-runner for systems which are not only tested via NST, but also contains NST tests in their source code.

Specify that a system defines NST tests by providing :class asdf:nst-test-holder to asdf:defsystem. The arguments for asdf:nst-test-runner may be used for asdf:nst-test-holder, as well as the following:

The next three arguments to an nst-testable system are mutually exclusive, and moreover exclude any of the above group or :nst-systems:

Examples of nst-testable ASDF system definitions:

;; NST and its ASDF interface must be loaded
;; before we can process the defsystem form.
(asdf:oos 'asdf:load-op :asdf-nst)
#+LaTeX:
(defsystem :mnst
    :class nst-test-holder
    :description "The NST test suite's self-test."
    :serial t
    :nst-systems (:masdfnst)
    :nst-groups ((:mnst-simple . g1)
                 (:mnst-simple . g1a)
                 (:mnst-simple . g1a1)
                 (:mnst-simple . core-checks))
    :depends-on (:nst)
    :in-order-to ((test-op (load-op :mnst)))
    :components ((:module "core"
                  :components ((:file "byhand")
                               (:file "builtin-checks")))))

An alternate ASDF integration technique

We plan to deprecate and then remove asdf:nst-test-holder and nst-test-runner once we have implemented a unified replacement for them. To avoid the possibility of a bit-rotted test scheme, the link between a system and its unit tests can be made explicit by providing methods for ASDF generic functions which make calls to the NST API. Specifically:

When providing an explicit asdf:perform method, it is also necessary to explicitly list system dependencies to NST and to the other systems which contain the tested system's unit test definitions.

Inheritance-based test methods

/This feature is in-progress. It currently does not work under Lispworks or Clisp, and details of the API may change in subsequent versions./

For testing objects in a class hierarchy NST offers xUnit-style test methods dispatching on different classes. The idea is that an object should have all relevant tests applied to it without requiring that the tests be explicitly enumerated in the test definition: all tests applicable to an object's class, or to any of its superclasses, should be discovered and run.

Our running examples of this section are tests on objects of these four classes:

(defclass top-cls ()
     ((tc1 :initarg :tc1 :reader tc1)
      (tc2 :initarg :tc2 :reader tc2)))

(defclass mid-cls (top-cls)
     ((mc1 :initarg :mc1 :reader mc1)
      (mc2 :initarg :mc2 :reader mc2)))

(defclass side-cls ()
     ((sc1 :initarg :sc1 :reader sc1)
      (sc2 :initarg :sc2 :reader sc2)))

(defclass bot-cls (mid-cls side-cls)
     ((bc1 :initarg :bc1 :reader bc1)
      (bc2 :initarg :bc2 :reader bc2)))

Declaring methods

There are two macros which define a particular method of a generic test function.

The def-test-method-criterion macro provides a simple facility for defining a generic test function method in terms of an NST criterion.

(def-test-method-criterion function-name class-name
  criterion)

For example:

(nst:def-test-method-criterion for-clses top-cls
      (:predicate (lambda (tc) (< (tc1 tc) (tc2 tc)))))

The def-test-generic declares a generic test function.

(def-test-generic function-name)

For example,

(nst:def-test-generic for-clses)

The def-test-method defines a general method for a generic test function.

(def-test-method function-name
                 ( test-value class-name )
  form
  ...
  form)

The method body should return a test result report, constructed with make-success-result, etc.For example:

(nst:def-test-method for-clses (o mid-cls)
  (with-slots (mc1 mc2) o
    (cond
      ((< mc1 mc2) (make-success-report))
      (t (make-failure-report :format "~d not < ~d" :args (list mc1 mc2))))))
(nst:def-test-method for-clses (o side-cls)
  (with-slots (sc1 sc2) o
    (cond
      ((eql sc1 sc2) (make-success-report))
      (t (make-failure-report :format "~d not eql ~d" :args (list sc1 sc2))))))

Invoking methods

Criterion methods

The :methods criterion runs the test functions applicable to the value under test.

For example:

(def-test-group method-tests ()
  (def-test t-p :methods (make-instance 'top-cls :tc1 0 :tc2 2))
  (def-test m-p :methods (make-instance 'mid-cls :tc1 0 :tc2 2 :mc1 0 :mc2 2))
  (def-test s-p :methods (make-instance 'side-cls :sc1 1 :sc2 1))
  (def-test b-p :methods (make-instance 'bot-cls
                           :tc1 0 :tc2 2 :mc1 0 :mc2 2 :sc1 1 :sc2 1))
  (def-test t-f :methods (make-instance 'top-cls :tc1 4 :tc2 2))
  (def-test m-f-t  :methods (make-instance 'mid-cls
                              :tc1 4 :tc2 2 :mc1 0 :mc2 2))
  (def-test m-f-m  :methods (make-instance 'mid-cls
                              :tc1 0 :tc2 2 :mc1 4 :mc2 2))
  (def-test m-f-mt :methods (make-instance 'mid-cls
                              :tc1 4 :tc2 2 :mc1 4 :mc2 2))
  (def-test s-f :methods (make-instance 'side-cls :sc1 1 :sc2 3))
  (def-test b-f-t :methods (make-instance 'bot-cls
                             :tc1 4 :tc2 2 :mc1 0 :mc2 2 :sc1 1 :sc2 1))
  (def-test b-f-m :methods (make-instance 'bot-cls
                             :tc1 0 :tc2 2 :mc1 4 :mc2 2 :sc1 1 :sc2 1))
  (def-test b-f-s :methods (make-instance 'bot-cls
                             :tc1 0 :tc2 2 :mc1 0 :mc2 2 :sc1 1 :sc2 3))
  (def-test b-f-mt :methods (make-instance 'bot-cls
                              :tc1 4 :tc2 2 :mc1 4 :mc2 2 :sc1 1 :sc2 1))
  (def-test b-f-ms :methods (make-instance 'bot-cls
                              :tc1 0 :tc2 2 :mc1 4 :mc2 2 :sc1 1 :sc2 3))
  (def-test b-f-ts :methods (make-instance 'bot-cls
                              :tc1 4 :tc2 2 :mc1 0 :mc2 2 :sc1 1 :sc2 3))
  (def-test b-f-mts :methods (make-instance 'bot-cls
                               :tc1 4 :tc2 2 :mc1 4 :mc2 2 :sc1 1 :sc2 3)))

Method combinations

NST defines a method combination nst-results as the default method combination for functions defined by def-test-generic. This combination runs all applicable methods, and combines all of their results into a single NST result record.

This default can be overridden by specifying t as the method combination in the intial declaration.

(nst:def-test-generic overridden
    (:method-combination t))
(nst:def-test-method-criterion overridden mid-cls
  (:slots (mc1 (:eql 0))
          (mc2 (:eql 2))))
(nst:def-test-method-criterion overridden bot-cls
  (:slots (sc1 (:eql 1))
          (sc2 (:eql 1))))

Deprecated forms

The macros, functions and variables documented in this section are all deprecated. Some continue to be exported from the NST API; others have already been removed. This section describes how code using these forms should be ported to the active NST API.

Older criteria-defining macros

The def-criterion-unevaluated macro is deprecated as of NST 2.1.2. It was consolidated into the def-criterion macro.

Replace:

(def-criterion-unevaluated name (pattern ... pattern) name
  BODY)

with:

(def-criterion name (:forms pattern ... pattern)
                    (:form name)
                    BODY)

The def-values-criterion macro was deprecated as of NST 1.3.0. For new criteria, use def-criterion instead. In the short term, code using def-values-criterion should continue to work as before.

The def-form-criterion macro was deprecated as of NST 1.3.0. Code using def-form-criterion in any but the simplest ways is very likely to fail. Use def-criterion instead.

Old test result generators

The emit-failure function is deprecated; use make-failure-report instead.

The emit-success function is deprecated; use make-success-report instead.

The emit-warning function is deprecated; use make-warning-report instead.


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 nst

Author

John Maraist <lisper@maraist.org>

License

LLGPL 3.latest

Description

The NST unit/regression testing system

Version

4.1.0

Dependencies
Source

nst.asd (file)

Component

core (module)


Next: , Previous: , Up: Top   [Contents][Index]

3 Modules

Modules are listed depth-first from the system components tree.


Previous: , Up: Modules   [Contents][Index]

3.1 nst/core

Parent

nst (system)

Location

core/

Components

Next: , Previous: , Up: Top   [Contents][Index]

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

4.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.1 nst.asd

Location

nst.asd

Systems

nst (system)

Packages

nst-asd

Internal Definitions

nst-doc (function)


Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.2 nst/core/package.lisp

Parent

core (module)

Location

core/package.lisp

Packages

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.3 nst/core/utils.lisp

Dependency

package.lisp (file)

Parent

core (module)

Location

core/utils.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.4 nst/core/errors.lisp

Dependency

package.lisp (file)

Parent

core (module)

Location

core/errors.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.5 nst/core/interrupt.lisp

Dependency

package.lisp (file)

Parent

core (module)

Location

core/interrupt.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.6 nst/core/permuter.lisp

Dependency

package.lisp (file)

Parent

core (module)

Location

core/permuter.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.7 nst/core/globals.lisp

Dependencies
Parent

core (module)

Location

core/globals.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.8 nst/core/context.lisp

Dependencies
Parent

core (module)

Location

core/context.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.9 nst/core/artifacts.lisp

Dependency

globals.lisp (file)

Parent

core (module)

Location

core/artifacts.lisp

Packages

nst-name-usage

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.10 nst/core/group.lisp

Dependencies
Parent

core (module)

Location

core/group.lisp

Exported Definitions

def-test-group (macro)

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.11 nst/core/check.lisp

Dependencies
Parent

core (module)

Location

core/check.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.12 nst/core/fixture.lisp

Dependencies
Parent

core (module)

Location

core/fixture.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.13 nst/core/test-def.lisp

Dependencies
Parent

core (module)

Location

core/test-def.lisp

Exported Definitions

def-test (macro)

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.14 nst/core/status.lisp

Dependencies
Parent

core (module)

Location

core/status.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.15 nst/core/runner.lisp

Dependencies
Parent

core (module)

Location

core/runner.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.16 nst/core/criteria.lisp

Dependencies
Parent

core (module)

Location

core/criteria.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.17 nst/core/process.lisp

Dependencies
Parent

core (module)

Location

core/process.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.18 nst/core/command.lisp

Dependencies
Parent

core (module)

Location

core/command.lisp

Packages

nst-artifact-lookup-package

Exported Definitions

nst-cmd (macro)

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.19 nst/core/xml.lisp

Dependency

status.lisp (file)

Parent

core (module)

Location

core/xml.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.20 nst/core/junit.lisp

Dependency

xml.lisp (file)

Parent

core (module)

Location

core/junit.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.21 nst/core/sampling.lisp

Dependency

check.lisp (file)

Parent

core (module)

Location

core/sampling.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

4.1.22 nst/core/method.lisp

Dependencies
Parent

core (module)

Location

core/method.lisp

Exported Definitions
Internal Definitions

Previous: , Up: Lisp files   [Contents][Index]

4.1.23 nst/core/interfaces.lisp

Dependencies
Parent

core (module)

Location

core/interfaces.lisp

Packages

nst-control-api

Internal Definitions

def-bundle-package (macro)


Next: , Previous: , Up: Top   [Contents][Index]

5 Packages

Packages are listed by definition order.


Next: , Previous: , Up: Packages   [Contents][Index]

5.1 nst-asd

Source

nst.asd

Use List
Internal Definitions

nst-doc (function)


Next: , Previous: , Up: Packages   [Contents][Index]

5.2 nst-name-use-in-packages

Internal package for names’ record-keeping.

Source

package.lisp (file)


Next: , Previous: , Up: Packages   [Contents][Index]

5.3 nst

Unit and regression testing for Common Lisp

Source

package.lisp (file)

Use List

common-lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Packages   [Contents][Index]

5.4 nst-name-usage

Source

artifacts.lisp (file)


Next: , Previous: , Up: Packages   [Contents][Index]

5.5 nst-artifact-lookup-package

Auxiliary package for canonical NST command names.

Source

command.lisp (file)


Previous: , Up: Packages   [Contents][Index]

5.6 nst-control-api

API-level control of test execution and reporting.

Source

interfaces.lisp (file)


Next: , Previous: , Up: Top   [Contents][Index]

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

6.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

6.1.1 Special variables

Special Variable: *assert-eq-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-eql-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-equal-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-equalp-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-nonnil-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-not-eq-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-not-eql-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-not-equal-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-not-equalp-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-null-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *assert-zero-format-string*
Package

nst

Source

process.lisp (file)

Special Variable: *debug-on-error*

User variable =*debug-on-error*=: if non-nil, will break into the Lisp REPL debugger upon encountering an unexpected error. If nil, will record the error and continue with other tests.

Package

nst

Source

globals.lisp (file)

Special Variable: *debug-on-fail*

User variable =*debug-on-fail*=: if non-nil, will break into the Lisp REPL debugger upon encountering a test which fails. If nil, will record the failure and continue with other tests. This variable is useful inspecting the dynamic environment under which a test was evaluated.

Package

nst

Source

globals.lisp (file)

Special Variable: *default-report-verbosity*

User variable =*default-report-verbosity*= determines the default value for =*nst-verbosity*= when printing reports (2 by default).

Package

nst

Source

globals.lisp (file)

Special Variable: *max-compound-structure-depth*

The =*max-compound-structure-depth*= variable sets the maximum nesting depth of compound data structures: beyond that depth, =scalar= rather than =t= is the default element generator. This restriction does not apply to explicitly specified element types, only to the use of defaults.

Package

nst

Source

sampling.lisp (file)

Special Variable: *nst-output-stream*

Determines the output stream to which NST should print its output (=*standard-output*= by default).

Package

nst

Source

globals.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

6.1.2 Macros

Macro: assert-criterion KEY-ARGS CRITERION-EXPR &rest VALUE-EXPRS

The =assert-criterion= macro asserts that an NST criterion should pass. #+begin_example
(assert-criterion ( [ :msg-format FORMAT-STRING ]
[ :msg-args FORMAT-ARGUMENTS ]
[ :fatal FLAG [
[ :fail-on-warning FLAG ] )
FORM
...
FORM)
#+end_example
- msg-format :: Format string used to build the label of the restart point. - msg-args :: Format arguments used to build the label of the restart point. - fatal :: If non-nil, a failure of this assertion indicates that execution of the test forms should be aborted.
- fail-on-warning :: If non-nil, then an NST result which includes a warning indicates failure of this assertion.

Package

nst

Source

process.lisp (file)

Macro: compound-structure &body FORMS

The =compound-structure= macro wraps substructure which should be considered compound for the limits set by =*max-compound-structure-depth*=.

Package

nst

Source

sampling.lisp (file)

Macro: def-arbitrary-instance-type TYPE-EXPR &body TECHNIQUE

New type specifications for invariant-testing. are defined with the =def-arbitrary-instance-type= macro.
#+begin_example
(def-arbitrary-instance-type ( SPEC-NAME [ :params FORMALS ]
[ :scalar BOOL ]
[ :key KEY ] )
FORM
FORM
...
FORM)
#+end_example
- =formals= :: Formal parameter definition used to pass subcomponent types. - =scalar= :: When a non-null value is provided for the =:scalar= argument, the new specifier is taken to be generable by the =:scalar= specification.
#+begin_example
(def-arbitrary-instance-type (ratio :scalar t)
(/ (arbitrary ’integer)
(let ((raw (arbitrary (find-class ’integer)))) (cond
((< raw 0) raw)
(t (+ 1 raw))))))
#+end_example
- =key= :: The =:key= argument gives a list of keyword arguments which may accompany the new specification. For the =cons= type, keyword arguments allow specifications for the left and right components: #+begin_example
(def-arbitrary-instance-type
(cons :key ((car t car-supp-p) (cdr t cdr-supp-p))) (compound-structure
(when (and (not car-supp-p)
(>= *current-compound-structure-depth* *max-compound-structure-depth*))
(setf car ’scalar))
(when (and (not cdr-supp-p)
(>= *current-compound-structure-depth* *max-compound-structure-depth*))
(setf cdr ’scalar))
(cons (arbitrary car) (arbitrary cdr)))) #+end_example
- =form= :: Construct and return (as if through =progn= the arbtrary instance.

Package

nst

Source

sampling.lisp (file)

Macro: def-binary-negated-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE &rest KEYARGS &key MESSAGE-DEFVAR

Macro =def-binary-negated-predicate-assert= uses the negated result of a binary predicate as the basis for an assertion function just as =def-unary-negated-predicate-assert= uses the negated result of a unary predicate. This macro’s arguments are just as for =def-unary-predicate-assert=.

Package

nst

Source

process.lisp (file)

Macro: def-binary-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE &key MESSAGE-DEFVAR DOC-STATE-FLAG PRED-NAME

Macro =def-binary-predicate-assert= uses a binary predicate as the basis for an assertion function just as =def-unary-predicate-assert= uses a unary predicate. This macro’s arguments are just as for =def-unary-predicate-assert=.

Package

nst

Source

process.lisp (file)

Macro: def-criterion (NAME ARGS-FORMALS VALUES-FORMALS) &body FORMS

The =def-criterion= macro defines a new criterion for use in NST tests.
These criteria definitions are like generic function method definitions with two
sets of formal parameters: the forms provided as the actual parameters of the
criterion itself, and the values arising from the evaluation of the forms under
test.
#+begin_example
(def-criterion (name criterion-lambda-list values-lambda-list)
[ doc-string )
form
form
...
form)
#+end_example

- name :: Name of the criterion.
- criterion-lambda-list :: Lambda list for the arguments to the criterion. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: =:values= for call-by-value, or =:forms for call-by-name (the default). The list may include the keywords =&key=, =&optional=, =&body= and =&rest= but may not use =&whole= or =&environment=. Apart from this restriction, in the former case the list may be any ordinary lambda list as for =defun=, and in the latter case the list may be any macro lambda list as for =defmacro=.
- values-lambda-list :: Lambda list for the forms under test. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: :values for call-by-value (the default), or :form for call-by-name. In the former case, the list may include the keywords =&key=, =&optional=, =&body= and =&rest=, but not =&whole= or =&environment=; apart from that restriction, list may be any ordinary lambda list as for =defun=. In the latter case, the remainder of the list must contain exactly one symbol, to which a form which would evaluate to the values under test will be bound. If the criterion ignores the values, then instead of a lambda list, this argument may be the symbol =:ignore=. On many platforms, listing a dummy parameter which is then =declare=d =ignore= or =ignorable= will produce a style warning: the body of a =def-criterion= should not be assumed to correspond directly to the body of a =defmethod=; in general there will be surrounding =destructuring-bind=s.
- documentation :: An optional documentation string for the criterion.
- form :: The body of the criterion definition should return a test result report contructed with the =make-success-report=, etc. functions.
Examples:
#+begin_example
(def-criterion (:true () (bool))
(if bool
(make-success-report)
(make-failure-report :format "Expected non-null, got: ~s"
:args (list bool))))

(def-criterion (:eql (target) (actual))
(if (eql (eval target) actual)
(make-success-report)
(make-failure-report :format "Not eql to value of ~s"
:args (list target))))
#+end_example

Package

nst

Source

check.lisp (file)

Macro: def-criterion-alias (NAME . ARGS-FORMALS) DOCSTRING-OR-FORM &optional FORM

The simplest mechanism for defining a new criterion involves simply
defining one criterion to rewrite as another using =def-criterion-alias=. #+begin_example
(def-criterion-alias (name (:seq arg))
[ doc-string ]
expansion)
#+end_example
The body of the expansion should be a Lisp form which, when evaluated, returns an S-expression quoting the new criterion which the rewrite should produce. The =arg= are passed as for Lisp macros: they are not evaluated and are most typically comma-inserted into a backquoted result. For example: #+begin_example
(def-criterion-alias (:forms-eq) ‘(:predicate eq))
(def-criterion-alias (:symbol name) ‘(:eq ’,name))
#+end_example

Package

nst

Source

check.lisp (file)

Macro: def-criterion-unevaluated (NAME ARGS-FORMALS FORMS-FORMAL &key IGNORE-FORMS) &body FORMS

The =def-criterion-unevaluated= macro is deprecated as of NST 2.1.2. It was consolidated into the =def-criterion= macro.

Replace:
#+begin_example
(def-criterion-unevaluated name (pattern ... pattern) name
BODY)
#+end_example
with:
#+begin_example
(def-criterion name (:forms pattern ... pattern)
(:form name)
BODY)
#+end_example

Package

nst

Source

check.lisp (file)

Macro: def-eval-test NAME-OR-NAME-AND-ARGS &rest FORMS

The =def-eval-test= macro abbreviates a call to =def-test= with a single =:eval= criterion. Its arguments are just as for =def-test= and =:eval=. #+begin_example
(def-eval-test (NAME [ :group GROUP-NAME ]
[ :setup FORM ]
[ :cleanup FORM ]
[ :startup FORM ]
[ :finish FORM ]
[ :fixtures (FIXTURE ... FIXTURE) ]
[ :documentation STRING ]
[ :check-warnings FLAG ]
[ :muffle-warnings FLAG ]
[ :attempt-continue FLAG ]
[ :force-continue FLAG ] )
FORM
...
FORM)

(def-eval-test NAME
FORM
...
FORM)
#+end_example

Package

nst

Source

process.lisp (file)

Macro: def-fixtures NAME (&key USES ASSUMES SPECIAL OUTER INNER DOCUMENTATION CACHE SETUP CLEANUP STARTUP FINISH EXPORT-NAMES EXPORT-BOUND-NAMES EXPORT-FIXTURE-NAME) &body BINDINGS

Fixtures are data structures and values which may be referred to by name during testing. NST provides the ability to use fixtures across multiple tests and test groups, and to inject fixtures into the runtime namespace for debugging. A set of fixtures is defined using the =def-fixtures= macro: #+begin_example
(def-fixtures fixture-name ( [ :outer FORM ]
[ :inner FORM ]
[ :setup FORM ]
[ :cleanup FORM ]
[ :startup FORM ]
[ :finish FORM ]
[ :documentation STRING ]
[ :cache FLAG ]
[ :export-names FLAG ]
[ :export-fixture-name FLAG ]
[ :export-bound-names FLAG ] )
( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] )
...
( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] ) )
#+end_example
- fixture-name :: The name to be associated with this set of fixtures.
- inner :: List of declarations to be made inside the let-binding of
names of any use of this fixture. Do not include the "declare" keyword here; NST adds these declarations to others, including a special declaration of all bound names.
- outer :: List of declarations to be made outside the let-binding of
names of any use of this fixture.
- documentation :: A documentation string for the fixture set.
- special :: Specifies a list of names which should be declared
=special= in the scope within which this set’s fixtures are
evaluated. The individual names are taken to be single variable names. Each =(:fixture NAME)= specifies all of the names of the given fixture set. This declaration is generally optional under most platforms, but can help supress spurious warnings. Note that
multiple =NAMEs= may be listed, and these lists and the bare
names may be intermixed. If only one name or fixture is
specified, it need not be placed in a list
- export-fixture-name :: When non-nil, the fixture name will be added
to the list of symbols exported by the current package.
- export-bound-names :: When non-nil, the names bound by this fixture
will be added to the list of symbols exported by the current package.
- export-names :: When non-nil, sets the default value to t for the
two options above.
- cache :: If specified with the group options, when non-nil, the
fixture values are cached at their first use, and re-applied at subsequent fixture application rather than being recalculated.

When a fixture is attached to a test or test group, each =NAME=
defined in that fixture becomes available in the body of that test or
group as if =let*= bound to the corresponding =FORM=. A fixture in
one set may refer back to other fixtures in the same set (again as
=let*=) but forward references are not allowed.

The four arguments =startup=, =finish=, =setup= and =cleanup= specify
forms which are run everytime the fixture is applied to a group or
test. The =startup= (respectively =finish=) form is run before
fixtures are bound (after their bindings are released). These forms
are useful, for example, to initialize a database connection from
which the fixture values are drawn. The =setup= form is run after inclusion of names from fixture sets, but before any tests from the
group. The =cleanup= form is normally run after the test completes,
but while the fixtures are still in scope. Normally, the =cleanup=
form will not be run if the =setup= form raises an error, and the
=finish= form will not be run if the =startup= form raises an error; although the user is able to select (perhaps unwisely) a restart which disregards the error.

The names of a fixture and the names it binds can be exported from the package where the fixture is defined using the =export-bound-names=
and =export-fixture-name= arguments. The default value of both is the value of =export-names=, whose default value is =nil=.

The =cache= option, if non-nil, directs NST to evaluate a fixture’s
form one single time, and re-use the resulting value on subsequent applications of the fixture. Note that if this value is mutated by
the test cases, test behavior may become unpredictable! However this option can considerably improve performance when constant-valued
fixtures are applied repeatedly. Caching may be set on or off (the
default is off) for the entire fixture set, and the setting may vary
for individual fixtures.

Examples of fixture definitions:
#+begin_example
(def-fixtures f1 ()
(c 3)
(d ’asdfg))
(def-fixtures f2 (:special ((:fixture f1)))
(d 4)
(e ’asdfg)
(f c))
(def-fixtures f3 ()
((:cache t) g (ackermann 1 2))
((:cache nil) h (factorial 5)))
#+end_example

To cause a side-effect among the evaluation of a fixture’s name definitions, =nil= can be provided as a fixture name. In uses of the fixture, NST will replace =nil= with a non-interned symbol; in documentation such as form =:whatis=, any =nil=s are omitted.

Package

nst

Source

fixture.lisp (file)

Macro: def-form-criterion (NAME ARGS-FORMALS FORM-FORMAL) &rest FORMS

The =def-form-criterion= macro was deprecated as of NST 1.3.0. /Code using =def-form-criterion= in any but the simplest ways is very likely to fail./ Use =def-criterion= instead.

Package

nst

Source

check.lisp (file)

Macro: def-test NAME-OR-NAME-AND-ARGS CRITERION &rest FORMS

Individual unit tests are encoded with the =def-test= form: #+begin_example
(def-test NAME ( [ :group GROUP-NAME ]
[ :setup FORM ]
[ :cleanup FORM ]
[ :startup FORM ]
[ :finish FORM ]
[ :fixtures (FIXTURE FIXTURE ... FIXTURE) ]
[ :aspirational FLAG ]
[ :documentation STRING ] )
criterion
FORM ... FORM)

(def-test NAME criterion &body (:seq FORM))
#+end_example
The =SETUP=, =CLEANUP=, =STARTUP=, =FINISH= and =FIXTURES= are just as for fixtures and test groups, but apply only to the one test. The =CRITERION= is a list or symbol specifying the properties which should hold for
the =FORM=s.

When a test is not enclosed within a group body, a group name must be provided by the =GROUP= option. When a test is enclosed within
a group body, the =GROUP= option is not required, but if
provided it must agree with the group name.

When there are no =SETUP=, =CLEANUP=, =STARTUP=, =FINISH= or =FIXTURES= arguments, the =NAME= may be given without parentheses. Likewise, any criterion consisting of a single symbol, e.g. =(:pass)=, may be abbreviated as just the symbol without the parentheses, e.g. =:pass=.

The =:documentation= form provides a documentation string in
the standard Lisp sense. Since documentation strings are stored
against names, and since the same name can be used for several tests
(so long as they are all in different packages), documentation strings
on tests may not be particularly useful.

An =aspirational= test is one which verifies some part of an API or code contract which may not yet be implemented. When a group is marked aspirational, all tests within the group are taken to be aspirational as well. At this point, there is no particular processing for aspirational tests and groups, but we hope to implement it at some point in the future.

The =def-check= form is a deprecated synonym for =def-test=.

Package

nst

Source

test-def.lisp (file)

Macro: def-test-generic FUNCTION-NAME &body FORMS

The =def-test-generic= declares a generic test function. #+begin_example
(def-test-generic FUNCTION-NAME)
#+end_example
For example,
#+begin_example
(nst:def-test-generic for-clses)
#+end_example

Package

nst

Source

method.lisp (file)

Macro: def-test-group GROUP-NAME GIVEN-FIXTURES &body FORMS

The =def-test-group= form defines a group of the
given name, providing one instantiation of the bindings of the given fixtures
to each test. Groups can be associated with fixture sets, stateful
initialization, and stateful cleanup.
#+begin_example
(def-test-group NAME (FIXTURE FIXTURE ...)
(:aspirational FLAG)
(:setup FORM FORM ... FORM)
(:cleanup FORM FORM ... FORM)
(:startup FORM FORM ... FORM)
(:finish FORM FORM ... FORM)
(:each-setup FORM FORM ... FORM)
(:each-cleanup FORM FORM ... FORM)
(:include-groups GROUP GROUP ... GROUP)
(:documentation STRING)
TEST
...
TEST)
#+end_example
Arguments:
- group-name :: Name of the test group being defined
- given-fixtures :: List of the names of fixtures and anonymous fixtures to be used with the tests in this group.
- aspirational :: An aspirational test is one which verifies some part of an API or code contract which may not yet be implemented. When a group is marked aspirational, all tests within the group are taken to be aspirational as well. At this point, there is no particular processing for aspirational tests and groups, but we hope to implement it at some point in the future.
- forms :: Zero or more test forms, given by =def-check=.
- setup :: These forms are run once, before any of the individual tests, but after the fixture names are bound.
- cleanup :: These forms are run once, after all of the individual tests, but while the fixture names are still bound.
- startup :: These forms are run once, before any of the individual tests and before the fixture names are bound.
- finish :: These forms are run once, after all of the individual tests, and after the scope of the bindings to fixture names.
- each-setup :: These forms are run before each individual test.
- each-cleanup :: These forms are run after each individual test.
- include-group :: The test groups named in this form will be run (respectively reported) anytime this group is run (reported).
- documentation :: Docstring for the class.

Package

nst

Source

group.lisp (file)

Macro: def-test-method FUNCTION-NAME (ARG CLASS) &body BODY

The =def-test-method= defines a general method for a generic test function. #+begin_example
(def-test-method FUNCTION-NAME (TEST-VALUE CLASS-NAME)
FORM
...
FORM)
#+end_example
- function-name :: The name of the test function for which we are defining a method.
- test-value :: Formal parameter to which the value under test will be bound. - class-name :: The class for which we are defining a method.
The method body should return a test result report, constructed with =make-success-result=, etc.
For example:
#+begin_example
(nst:def-test-method for-clses (o mid-cls)
(with-slots (mc1 mc2) o
(cond
((< mc1 mc2) (make-success-report))
(t (make-failure-report :format "~d not < ~d" :args (list mc1 mc2)))))) (nst:def-test-method for-clses (o side-cls)
(with-slots (sc1 sc2) o
(cond
((eql sc1 sc2) (make-success-report))
(t (make-failure-report :format "~d not eql ~d" :args (list sc1 sc2)))))) #+end_example

Package

nst

Source

method.lisp (file)

Macro: def-test-method-criterion FUNCTION-NAME CLASS DOCUMENTATION &optional CRITERION

The =def-test-method-criterion= macro provides a simple facility for defining a generic test function method in terms of an NST criterion. #+begin_example
(def-test-method-criterion FUNCTION-NAME CLASS-NAME [ CRITERION ] )
#+end_example
- function-name :: The name of the test function for which we are defining a
method.
- class-name :: The class for which we are defining a method.
- criterion :: The criterion to be applied to members of the class.
For example:
#+begin_example
(nst:def-test-method-criterion for-clses top-cls
(:predicate (lambda (tc) (< (tc1 tc) (tc2 tc)))))
#+end_example

Package

nst

Source

method.lisp (file)

Macro: def-unary-negated-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE &rest KEYARGS &key MESSAGE-DEFVAR

Macro =def-unary-negated-predicate-assert= uses the negated result of a unary predicate as the basis of an assertion function. This macro’s arguments are just as for =def-unary-predicate-assert=.

Package

nst

Source

process.lisp (file)

Macro: def-unary-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE &key MESSAGE-DEFVAR DOC-STATE-FLAG PRED-NAME &allow-other-keys

Macro =macro def-unary-predicate-assert= creates an assertion function using
the result of a call to a unary predicate. A non-nil result from the predicate
corresponds to a successful assertion.
#+begin_example
(def-unary-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE
[ :message-defvar NAME ]
[ :pred-name NAME ]
[ :doc-state-flag BOOL ] )
#+end_example
- assert-fn :: The name of the assertion function being defined.
- predicate :: The predicate used to define the assertion function. It should take a single argument.
- default-message :: Format string used by default for reporting failures of this assertion. It should expect to be used in a call to =format= with one additional argument, the value being tested. - message-defvar :: The name of a global variable into which the default message will be stored. If this argument is omitted, the result of a call to =gensym= is used.
- pred-name :: This argument is used only for documenting the underlying predicate in the assertion function’s docstring. By default, it is the same as the predicate.

Package

nst

Source

process.lisp (file)

Macro: def-values-criterion (NAME ARGS-FORMALS FORMS-FORMALS &key DECLARE) &body FORMS

The =def-values-criterion= macro was deprecated as of NST 1.3.0. For new criteria, use =def-criterion= instead. In the short term, code using =def-values-criterion= should continue to work as before.

Package

nst

Source

check.lisp (file)

Macro: emit-failure &rest ARGS

The =emit-failure= function is deprecated; use =make-failure-report= instead.

Package

nst

Source

status.lisp (file)

Macro: emit-success &rest ARGS

The =emit-success= function is deprecated; use =make-success-report= instead.

Package

nst

Source

status.lisp (file)

Macro: emit-warning &rest ARGS

The =emit-warning= function is deprecated; use =make-warning-report= instead.

Package

nst

Source

status.lisp (file)

Macro: nst-cmd &rest ARGS

User-level NST operations are accessible from the REPL via the =nst-cmd= macro.
#+begin_example
(nst-cmd NST-COMMAND ARG ... ARG)
#+end_example
Where a particular system supports the facility (Currently Allegro, and SBCL under ACL-REPL) the top-level alias =:nst= provides a shorthand to this function.

Package

nst

Source

command.lisp (file)

Macro: with-fixtures (&rest FIXTURES) &body FORMS

The =with-fixtures= macro faciliates debugging and other non-NST uses of fixtures sets:
#+begin_example
(with-fixtures (FIXTURE FIXTURE ... FIXTURE)
FORM
FORM
...
FORM)
#+end_example
This macro evaluates the forms in a namespace expanded with the bindings provided by the fixtures.

Package

nst

Source

fixture.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

6.1.3 Functions

Function: add-error RESULT &key FORMAT ARGS

For use within user-defined NST criteria: add an error to a result. #+begin_example
(add-error RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ] ) #+end_example

Package

nst

Source

status.lisp (file)

Function: add-failure RESULT &key FORMAT ARGS

For use within user-defined NST criteria: add a failure to a result. #+begin_example
(add-failure RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ]) #+end_example

Package

nst

Source

status.lisp (file)

Function: add-info RESULT ITEM

For use within user-defined NST criteria: add an info note to a result. #+begin_example
(add-info RESULT-REPORT INFO-ITEM)
#+end_example

Package

nst

Source

status.lisp (file)

Function: assert-eq G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-EQ= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQ=, succeeds whenever that call returns non-nil.

Package

nst

Source

process.lisp (file)

Function: assert-eql G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-EQL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQL=, succeeds whenever that call returns non-nil.

Package

nst

Source

process.lisp (file)

Function: assert-equal G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-EQUAL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUAL=, succeeds whenever that call returns non-nil.

Package

nst

Source

process.lisp (file)

Function: assert-equalp G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-EQUALP= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUALP=, succeeds whenever that call returns non-nil.

Package

nst

Source

process.lisp (file)

Function: assert-non-nil G0 &rest G1 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-NON-NIL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =NULL= function returns =nil=.

Package

nst

Source

process.lisp (file)

Function: assert-not-eq G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-NOT-EQ= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQ=, succeeds whenever that call returns =nil=.

Package

nst

Source

process.lisp (file)

Function: assert-not-eql G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-NOT-EQL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQL=, succeeds whenever that call returns =nil=.

Package

nst

Source

process.lisp (file)

Function: assert-not-equal G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-NOT-EQUAL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUAL=, succeeds whenever that call returns =nil=.

Package

nst

Source

process.lisp (file)

Function: assert-not-equalp G0 G1 &rest G2 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-NOT-EQUALP= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUALP=, succeeds whenever that call returns =nil=.

Package

nst

Source

process.lisp (file)

Function: assert-null G0 &rest G1 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-NULL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =NULL= function returns non-nil.

Package

nst

Source

process.lisp (file)

Function: assert-zero G0 &rest G1 &key FORMAT FORMAT-ARGS FATAL &allow-other-keys

The =ASSERT-ZERO= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =ZEROP= function returns non-nil.

Package

nst

Source

process.lisp (file)

Function: check-criterion-on-form CRITERION FORM

This function verifies that the values return by evaluating the form adheres to the criterion.
#+begin_example
(check-criterion-on-form CRITERION FORM)
#+end_example

Package

nst

Source

check.lisp (file)

Function: check-criterion-on-value CRITERION VAL

The =check-criterion-on-value= function can be called from within a criterion body to verify that a value adheres to a criterion.
#+begin_example
(check-criterion-on-value CRITERION VALUE)
#+end_example

Package

nst

Source

check.lisp (file)

Function: junit-results-by-group &rest ARGS &key VERBOSE &allow-other-keys

The \texttt{junit-results-by-group} function writes the NST test results in JUnit XML format, organized by group, aligning test groups with Java classes, and individual tests with \texttt{@Test} methods. #+begin_example
(junit-results-by-group [ :verbose FLAG ]
[ :dir DIRECTORY ]
[ :file FILESPEC ]
[ :stream STREAM ]
[ :if-dir-does-not-exist BOOL ]
[ :if-file-exists BOOL ] )
#+end_example
Either =:dir= and =:file= options, or the =:stream= option, but not both, should
e used to specify the target for XML output; if none of the three options are
given, the function will write to =*standard-output*=.

Package

nst

Source

junit.lisp (file)

Function: make-error-report E &rest FORMAT-ARGS

Function =make-error-report= produces a report of an error during test execution. #+begin_example
(make-error-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ] ) #+end_example

Package

nst

Source

status.lisp (file)

Function: make-failure-report &key FORMAT ARGS INFO

The =make-failure-report= function returns a report of test failure. #+begin_example
(make-failure-report FORMAT ARGS)
#+end_example
The =format-string= and =args= are as to the Common Lisp function =format=. The =emit-failure= function is an older, deprecated version of this function.

Package

nst

Source

status.lisp (file)

Function: make-success-report &rest ARGS &key WARNINGS INFO

The =make-success-report= function indicates a successful test result. #+begin_example
(make-success-report)
#+end_example
Note that some older examples show =(make-check-result)=, =(emit-success)= or =(check-result)=. The former is an internal function and should not be used from outside the core NST files. The latter two are deprecated.

Package

nst

Source

status.lisp (file)

Function: make-warning-report &key FORMAT ARGS

Function =make-warning-report= is like =make-failure-report=, but provides supplimentary information as a warning.
#+begin_example
(make-warning-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ]) #+end_example
The =emit-warning= function is an older, deprecated version of this function.

Package

nst

Source

status.lisp (file)

Function: wrap-thrown-lisp-warning W

The helper function =wrap-thrown-lisp-warning= creates an NST =check-note= object from a standard Lisp =warning=.

Package

nst

Source

status.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

6.1.4 Generic functions

Generic Function: add-warning RESULT W &rest ARGS

For use within user-defined NST criteria: add a warning to a
result.

The =add-warning= function adds an warning to a result record. The item can be any of a Lisp warning, an NST check-note or a format string; in the first two cases, no additional arguments should be provided.
#+begin_example
(add-warning RESULT-REPORT [ :format ITEM ] [ :args ARGUMENT-LIST ] ) #+end_example

Package

nst

Source

status.lisp (file)

Methods
Method: add-warning RESULT (FORMAT-STRING string) &rest ARGS
Method: add-warning RESULT (W check-note) &rest ARGS
Method: add-warning RESULT (W warning) &rest ARGS
Generic Function: arbitrary TYP

This function takes a single argument, which determines the
type of the value to be generated. For simple types, the name of the type (or the class object, such as returned by =find-class= by itself is a complete specification. For more complicated types, =arbitrary= can also take a list argument, where the first element gives the type and the remaining elements are keyword argument providing additional requirements for the generated value.

Package

nst

Source

sampling.lisp (file)

Methods
Method: arbitrary (SPEC cons)
Method: arbitrary OTHER
Generic Function: nst-junit-dump STREAM

Function =nst-junit-dump= pushes the entire NST state to a JUnit XML file whose stream is specified by its argument.

Package

nst

Source

junit.lisp (file)

Methods
Method: nst-junit-dump (FILENAME string)

Write the results of this session’s NST testing in JUnit XML format.

Method: nst-junit-dump (STREAM stream)

Previous: , Up: Exported definitions   [Contents][Index]

6.1.5 Method combinations

Short Method Combination: nst-results ()

NST defines a method combination =nst-results= as the default
method combination for functions defined by =def-test-generic=. This combination runs /all/ applicable methods, and combines all of their results into a single NST result record.

This default can be overridden by specifying =t= as the method combination in the intial declaration.
#+begin_example
(nst:def-test-generic overridden (:method-combination t)) (nst:def-test-method-criterion overridden mid-cls
(:slots (mc1 (:eql 0))
(mc2 (:eql 2))))
(nst:def-test-method-criterion overridden bot-cls
(:slots (sc1 (:eql 1))
(sc2 (:eql 1))))

Package

nst

Source

method.lisp (file)

Operator

check-result-union (function)

Indentity with one argument

nil


Previous: , Up: Definitions   [Contents][Index]

6.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

6.2.1 Constants

Constant: +keyboard-interrupt-class+

Name of the class of error thrown when a keyboard interrupt is received, or nil if this implementation of Lisp has no such class.

Package

nst

Source

interrupt.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

6.2.2 Special variables

Special Variable: *char-escapes*
Package

nst

Source

junit.lisp (file)

Special Variable: *current-compound-structure-depth*
Package

nst

Source

sampling.lisp (file)

Special Variable: *default-character-range*
Package

nst

Source

sampling.lisp (file)

Special Variable: *default-debug-config*

User variable: the default setting applied by default. Should be a list of alternating keyword/forms matching:
- nst-set - list of lists, each with arguments to :nst :set
- progn - list of forms to be evaluated

Package

nst

Source

globals.lisp (file)

Special Variable: *default-debug-protect*
Package

nst

Source

globals.lisp (file)

Special Variable: *default-xml-pprint-dispatch*
Package

nst

Source

xml.lisp (file)

Special Variable: *entities*
Package

nst

Source

junit.lisp (file)

Special Variable: *fixture-bindings*

Map implementing FIXTURE-BINDINGS: from fixture name to the list of local variable names bound that fixture

Package

nst

Source

fixture.lisp (file)

Special Variable: *fixture-letlist*

Map implementing FIXTURE-LETLIST: from fixture name to the list of local variable names bound that fixture

Package

nst

Source

fixture.lisp (file)

Special Variable: *fixture-record*

Map implementing FIXTURE-RECORD: from fixture name to its =fixture-record=

Package

nst

Source

fixture.lisp (file)

Special Variable: *generate-backtraces*

User variable: if non-null, will attempt to capture the Lisp backtrace of errors in tests.

Package

nst

Source

globals.lisp (file)

Special Variable: *group-records*

Primary storage of group and test records; this level of map translates from group name to an instance of =group-record=.

Package

nst

Source

group.lisp (file)

Special Variable: *implicit-group-choice*
Package

nst

Source

runner.lisp (file)

Special Variable: *last-repl-call*
Package

nst

Source

command.lisp (file)

Special Variable: *max-auto-array-rank*
Package

nst

Source

sampling.lisp (file)

Special Variable: *nst-check-internal-name*

Dynamic variable used to set the name of a test in its result report.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-check-user-name*

Dynamic variable used to set the name of a test in its result report.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-context*

Dynamic-scoped variable recording the values under test - a list of context-layer instances.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-context-evaluable*

Dynamic-scoped variable tracking whether the values under test should be asusmed evaluated. Used in preparing context expressions.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-debug*

User variable: apply customizable debugging settings.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-group-name*

Dynamic variable used to set the name of the group in a test result report.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-group-shown*

Dynamic-scoped variable tracking whether the name of a group has been printed, so that tests need not repeat it.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-report-driver*

Dynamic-scoped variable - one of :multiple, :package, :group or :test to determine the top-level form of a report. Used as a control parameter for printing reports.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-stack*

Dynamic-scoped variable - the stack of values under test by the current criterion.

Package

nst

Source

globals.lisp (file)

Special Variable: *nst-verbosity*

User variable determining how verbose NST’s output to the REPL should be. Internally, this variable takes an integer value: 0 and below are silent, 1 is the default, 2 and 3 are more verbose, and 4 is for full tracing. The command-line interpreter assigns keywords to these values, from most terse to most verbose: :silent, nil, :quiet, :default, t, :verbose, :vverbose (:quiet and :default are the same, and are the initial setting).

Package

nst

Source

globals.lisp (file)

Special Variable: *test-methods*

Global variable for programmer-defined NST test methods.

Package

nst

Source

method.lisp (file)

Special Variable: +arbitrary-generable-types+
Package

nst

Source

sampling.lisp (file)

Special Variable: +name-uses+

Map from a symbol in the =:nst-name-usage= package to a =name-use= instance decribing how a symbol of the same name is used for NST artifacts.

Package

nst

Source

artifacts.lisp (file)

Special Variable: +nst-repl-commands+
Package

nst

Source

command.lisp (file)

Special Variable: +nst-repl-properties+
Package

nst

Source

command.lisp (file)

Special Variable: +results-record+

Results of test runs.

Package

nst

Source

artifacts.lisp (file)

Special Variable: +scalar-generable-types+
Package

nst

Source

sampling.lisp (file)

Special Variable: qw
Package

nst

Source

permuter.lisp (file)

Special Variable: qwerty
Package

nst

Source

permuter.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

6.2.3 Macros

Macro: apply-debug-options FORMS-SPEC PROTECT-VARS &body FORMS
Package

nst

Source

globals.lisp (file)

Macro: apply-default-debug-options &body FORMS
Package

nst

Source

globals.lisp (file)

Macro: at-verbosity LV &body FORMS
Package

nst

Source

globals.lisp (file)

Macro: check-result &rest ARGS
Package

nst

Source

status.lisp (file)

Macro: coin-flip &rest OUTCOMES
Package

nst

Source

sampling.lisp (file)

Macro: count-nonnulls &rest BOOLS
Package

nst

Source

status.lisp (file)

Macro: def-bundle-package NAME REPACKAGING &body FORMS
Package

nst

Source

interfaces.lisp (file)

Macro: def-check &rest ARGS
Package

nst

Source

test-def.lisp (file)

Macro: def-check-alias &rest ARGS

Deprecated: use def-criterion-alias instead

Package

nst

Source

check.lisp (file)

Macro: def-control-check &rest ARGS

Deprecated: use def-criterion instead

Package

nst

Source

check.lisp (file)

Macro: def-hashtable-fns FN (&key TEST GLOBAL) &optional DOCSTRING
Package

nst

Source

utils.lisp (file)

Macro: def-nst-interactive-command (NAME &key SHORT-HELP LONG-HELP LONG-HELP-SPECIAL ARGS REPEATABLE) &body FORMS
Package

nst

Source

command.lisp (file)

Macro: def-nst-property NAME VARIABLE &key DOC FILTER UNFILTER
Package

nst

Source

command.lisp (file)

Macro: def-value-check &rest ARGS

Deprecated: use def-criterion instead

Package

nst

Source

check.lisp (file)

Macro: define-nst-error NAME FIELDS (STREAM EXP) &body PRINTER
Package

nst

Source

errors.lisp (file)

Macro: emit-error &rest ARGS

Deprecated; use make-success-report.

Package

nst

Source

status.lisp (file)

Macro: format-at-verbosity LV FORMAT &rest ARGS
Package

nst

Source

globals.lisp (file)

Macro: handler-bind-interruptable HANDLERS &body FORMS

Like handler-bind (and the arguments are similar), but cancel any error- handling if the error is related to a keyboard interrupt.

Package

nst

Source

interrupt.lisp (file)

Macro: log10 V
Package

nst

Source

utils.lisp (file)

Macro: named-function NAME LAMBDA-EXPRESSION
Package

nst

Source

utils.lisp (file)

Macro: protect-nst-config &body FORMS
Package

nst

Source

globals.lisp (file)

Macro: returning-criterion-config-error (MSG) &body FORMS

For use within criteria definitions only — catch errors and process them as errors in the "glue" among criteria and forms.

Package

nst

Source

check.lisp (file)

Macro: returning-test-error &body FORMS

For use within criteria definitions only — catch errors and process them as errors arising from within the .

Package

nst

Source

check.lisp (file)

Macro: with-context-layer LAYER-EXPR &body BODY
Package

nst

Source

context.lisp (file)

Macro: with-criterion-context-layer (&rest LAYER-ARGS) &body BODY
Package

nst

Source

context.lisp (file)

Macro: with-method-context-layer (METHOD-NAME CLASS-NAME OBJECT) &body BODY
Package

nst

Source

method.lisp (file)

Macro: with-nst-control-handlers ((FORMAL FLAG &key FOR-FAIL FOR-ERROR CERROR-LABEL-VAR CERROR-LABEL WITH-RETRY POST-CERROR GROUP TESTS FAIL-TEST-MSG LOG-LOCATION HANDLER-RETURN-TO HANDLER-RETURN) &body HANDLER) &body BODY

Macro for inserting restarts at points in the NST test-running stack.

Package

nst

Source

errors.lisp (file)

Macro: with-output-for-verbosity (LV STREAM-NAME) &body FORMS
Package

nst

Source

globals.lisp (file)

Macro: with-pprint-cdata (STREAM-NAME) &body FORMS
Package

nst

Source

xml.lisp (file)

Macro: with-retry (CONTINUATION-LABEL) &body FORMS
Package

nst

Source

errors.lisp (file)

Macro: with-xml-tagged-pprint-logical-block (STREAM TAG &key PROPERTIES) &body FORMS
Package

nst

Source

xml.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

6.2.4 Functions

Function: %make-check-result &key (TESTS TESTS) (PASSING PASSING) (ERRING ERRING) (FAILING FAILING) (WARNING WARNING) (ELAPSED-TIME ELAPSED-TIME) (TIMESTAMP TIMESTAMP) (GROUP-NAME GROUP-NAME) (CHECK-NAME CHECK-NAME) (WARNINGS WARNINGS) (FAILURES FAILURES) (ERRORS ERRORS) (INFO INFO)
Package

nst

Source

status.lisp (file)

Function: add-thrown-error RESULT E &rest FORMAT-ARGS
Package

nst

Source

status.lisp (file)

Function: all-groups-report ()
Package

nst

Source

status.lisp (file)

Function: all-package-report ()
Package

nst

Source

status.lisp (file)

Function: all-tests-report ()
Package

nst

Source

status.lisp (file)

Function: all-uses NAME

Return the list of group- and test-records, and possibly also a Lisp package object, corresponding to possible intended targets of an NST run.

Package

nst

Source

artifacts.lisp (file)

Function: apply-check-note-formatter STREAM CHECK-NOTE
Package

nst

Source

status.lisp (file)

Function: apply-fixture FIXTURE NEXT-FIXTURES GROUP-RECORD TEST-RECORDS BODY-THUNK
Package

nst

Source

fixture.lisp (file)

Function: apply-fixtures FIXTURES GROUP-RECORD TEST-RECORDS BODY-THUNK
Package

nst

Source

fixture.lisp (file)

Function: apply-formatter STREAM FORMATTER ARGS
Package

nst

Source

status.lisp (file)

Function: arbitrary-generable-types ()
Package

nst

Source

sampling.lisp (file)

Function: artifact-recorder SYMBOL
Package

nst

Source

artifacts.lisp (file)

Function: assemble-protected-option-values OTHER-VARS
Package

nst

Source

globals.lisp (file)

Function: assert-criterion-fn CRITERION-EXPR VALUES-FORM &key MSG-FORMAT MSG-ARGS FATAL FAIL-ON-WARNING
Package

nst

Source

process.lisp (file)

Function: blurb-one-name-use STREAM NAME ENTITY ITEM ENUMERATE
Package

nst

Source

artifacts.lisp (file)

Function: build-continue-check-expr CRITERION FORM
Package

nst

Source

check.lisp (file)

Function: calibrate-check-result R
Package

nst

Source

status.lisp (file)

Function: cdr-or-nil NAME-OR-NAME-AND-ARGS

Return the cdr given a list, or return nil if given a symbol.

Package

nst

Source

utils.lisp (file)

Function: check-note-args INSTANCE
Function: (setf check-note-args) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-note-context INSTANCE
Function: (setf check-note-context) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-note-format INSTANCE
Function: (setf check-note-format) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-note-p OBJECT
Package

nst

Source

status.lisp (file)

Function: check-note-stack INSTANCE
Function: (setf check-note-stack) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-check-name INSTANCE
Function: (setf check-result-check-name) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-elapsed-time INSTANCE
Function: (setf check-result-elapsed-time) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-erring INSTANCE
Function: (setf check-result-erring) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-errors INSTANCE
Function: (setf check-result-errors) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-failing INSTANCE
Function: (setf check-result-failing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-failures INSTANCE
Function: (setf check-result-failures) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-group-name INSTANCE
Function: (setf check-result-group-name) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-info INSTANCE
Function: (setf check-result-info) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-p OBJECT
Package

nst

Source

status.lisp (file)

Function: check-result-passing INSTANCE
Function: (setf check-result-passing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-tests INSTANCE
Function: (setf check-result-tests) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-timestamp INSTANCE
Function: (setf check-result-timestamp) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-union &rest REPORTS
Package

nst

Source

method.lisp (file)

Function: check-result-warning INSTANCE
Function: (setf check-result-warning) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: check-result-warnings INSTANCE
Function: (setf check-result-warnings) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: collect-test-generics OBJ
Package

nst

Source

method.lisp (file)

Function: continue-check CRITERION FORM
Package

nst

Source

check.lisp (file)

Function: copy-check-note INSTANCE
Package

nst

Source

status.lisp (file)

Function: copy-check-result INSTANCE
Package

nst

Source

status.lisp (file)

Function: copy-error-check-note INSTANCE
Package

nst

Source

status.lisp (file)

Function: copy-fixture-record INSTANCE
Package

nst

Source

fixture.lisp (file)

Function: copy-group-record INSTANCE
Package

nst

Source

group.lisp (file)

Function: copy-group-result INSTANCE
Package

nst

Source

status.lisp (file)

Function: copy-multi-results INSTANCE
Package

nst

Source

status.lisp (file)

Function: copy-name-use INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: copy-package-result INSTANCE
Package

nst

Source

status.lisp (file)

Function: copy-result-stats INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: copy-test-record INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: core-run-test TEST

Capture the result of the test.

Package

nst

Source

runner.lisp (file)

Function: decode-caching-decls FIXTURE-NAME BINDING-SPECS ALL-CACHE

Process a list of binding specifications to extract both the bindings, and the local variables needed for caching.

Package

nst

Source

fixture.lisp (file)

Function: decode-def-test-generic-body FORMS
Package

nst

Source

method.lisp (file)

Function: decode-defcheck-name-and-args NAME-OR-NAME-AND-ARGS

This function unpacks the information inside the first form of a def-test block, which can be either a single symbol naming the test, or a list whose first element is that symbol and whose remaining elements are options.

Package

nst

Source

test-def.lisp (file)

Function: decode-fixture-syntax FIXTURE-NAME BINDING-SPECS DECLS OUTER-DECLS SETUP CLEANUP STARTUP FINISH CACHE SPECIALS USE-FIXTURES SPECIAL-VARIABLES

Assemble syntactic elements of a fixture into the fixture function and the list of names defined by the fixture.

Package

nst

Source

fixture.lisp (file)

Function: decompose-arg-values-lambda-list ARGS-FORMALS
Package

nst

Source

check.lisp (file)

Function: elapsed-time-to-string ELAPSED-TIME
Package

nst

Source

xml.lisp (file)

Function: eql-for-sigdigits DIGITS N1 N2

Test whether two numbers are eql to the given number of significant digits.

Package

nst

Source

utils.lisp (file)

Function: error-check-note-args INSTANCE
Function: (setf error-check-note-args) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: error-check-note-context INSTANCE
Function: (setf error-check-note-context) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: error-check-note-error INSTANCE
Function: (setf error-check-note-error) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: error-check-note-format INSTANCE
Function: (setf error-check-note-format) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: error-check-note-p OBJECT
Package

nst

Source

status.lisp (file)

Function: error-check-note-stack INSTANCE
Function: (setf error-check-note-stack) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: executable-uses NAME

Return the list of group- and test-records, and possibly also a Lisp package object, corresponding to possible intended targets of an NST run.

Package

nst

Source

artifacts.lisp (file)

Function: extract-parameters X
Package

nst

Source

check.lisp (file)

Function: finish-multiple-report RESULT
Package

nst

Source

status.lisp (file)

Function: fixture-binding-error-note FIXTURE-NAME VARIABLE-NAME ERROR
Package

nst

Source

status.lisp (file)

Function: fixture-bindings NAME

Reader function: from fixture name to the list of local variable names bound that fixture

Package

nst

Source

fixture.lisp (file)

Writer

(setf fixture-bindings) (function)

Function: (setf fixture-bindings) VALUE NAME

Writer function: from fixture name to the list of local variable names bound that fixture

Package

nst

Source

fixture.lisp (file)

Reader

fixture-bindings (function)

Function: fixture-letlist NAME

Reader function: from fixture name to the list of local variable names bound that fixture

Package

nst

Source

fixture.lisp (file)

Writer

(setf fixture-letlist) (function)

Function: (setf fixture-letlist) VALUE NAME

Writer function: from fixture name to the list of local variable names bound that fixture

Package

nst

Source

fixture.lisp (file)

Reader

fixture-letlist (function)

Function: fixture-record NAME

Reader function: from fixture name to its =fixture-record=

Package

nst

Source

fixture.lisp (file)

Writer

(setf fixture-record) (function)

Function: (setf fixture-record) VALUE NAME

Writer function: from fixture name to its =fixture-record=

Package

nst

Source

fixture.lisp (file)

Reader

fixture-record (function)

Function: fixture-record-bindings-list INSTANCE
Function: (setf fixture-record-bindings-list) VALUE INSTANCE
Package

nst

Source

fixture.lisp (file)

Function: fixture-record-bound-names INSTANCE
Function: (setf fixture-record-bound-names) VALUE INSTANCE
Package

nst

Source

fixture.lisp (file)

Function: fixture-record-cache-flush INSTANCE
Function: (setf fixture-record-cache-flush) VALUE INSTANCE
Package

nst

Source

fixture.lisp (file)

Function: fixture-record-documentation INSTANCE
Function: (setf fixture-record-documentation) VALUE INSTANCE
Package

nst

Source

fixture.lisp (file)

Function: fixture-record-function INSTANCE
Function: (setf fixture-record-function) VALUE INSTANCE
Package

nst

Source

fixture.lisp (file)

Function: fixture-record-name INSTANCE
Function: (setf fixture-record-name) VALUE INSTANCE
Package

nst

Source

fixture.lisp (file)

Function: fixture-record-p OBJECT
Package

nst

Source

fixture.lisp (file)

Function: flag-filter X
Package

nst

Source

command.lisp (file)

Function: flush-fixture-cache NAME
Package

nst

Source

fixture.lisp (file)

Function: generate-sample DOMAINS
Package

nst

Source

sampling.lisp (file)

Function: get-display-context-layers LAYERS
Package

nst

Source

context.lisp (file)

Function: get-local-criterion-context CONTEXT-LAYER-LIST
Package

nst

Source

context.lisp (file)

Function: get-report-from-names GROUP-OR-PACKAGE GP-SUPP-P TEST TEST-SUPP-P
Package

nst

Source

status.lisp (file)

Function: get-test-record-specials TEST-RECORD
Package

nst

Source

runner.lisp (file)

Function: get-unassigned P
Package

nst

Source

permuter.lisp (file)

Function: group-record NAME
Function: (setf group-record) GROUP-RECORD NAME
Package

nst

Source

group.lisp (file)

Function: group-record-anon-fixture-forms INSTANCE
Function: (setf group-record-anon-fixture-forms) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-aspirational INSTANCE
Function: (setf group-record-aspirational) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-documentation INSTANCE
Function: (setf group-record-documentation) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-eachtest-cleanup-thunk INSTANCE
Function: (setf group-record-eachtest-cleanup-thunk) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-eachtest-setup-thunk INSTANCE
Function: (setf group-record-eachtest-setup-thunk) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-fixtures-cleanup-thunk INSTANCE
Function: (setf group-record-fixtures-cleanup-thunk) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-fixtures-setup-thunk INSTANCE
Function: (setf group-record-fixtures-setup-thunk) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-given-fixtures INSTANCE
Function: (setf group-record-given-fixtures) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-include-groups INSTANCE
Function: (setf group-record-include-groups) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-name INSTANCE
Function: (setf group-record-name) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-p OBJECT
Package

nst

Source

group.lisp (file)

Function: group-record-tests INSTANCE
Function: (setf group-record-tests) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-withfixtures-cleanup-thunk INSTANCE
Function: (setf group-record-withfixtures-cleanup-thunk) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-record-withfixtures-setup-thunk INSTANCE
Function: (setf group-record-withfixtures-setup-thunk) VALUE INSTANCE
Package

nst

Source

group.lisp (file)

Function: group-report GROUP

Top-level function for reporting the results of a group.

Package

nst

Source

status.lisp (file)

Function: group-result-check-results INSTANCE
Function: (setf group-result-check-results) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-elapsed-time INSTANCE
Function: (setf group-result-elapsed-time) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-erring INSTANCE
Function: (setf group-result-erring) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-failing INSTANCE
Function: (setf group-result-failing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-group-name INSTANCE
Function: (setf group-result-group-name) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-p OBJECT
Package

nst

Source

status.lisp (file)

Function: group-result-passing INSTANCE
Function: (setf group-result-passing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-tests INSTANCE
Function: (setf group-result-tests) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-timestamp INSTANCE
Function: (setf group-result-timestamp) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: group-result-warning INSTANCE
Function: (setf group-result-warning) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: interesting-result-p RESULT
Package

nst

Source

status.lisp (file)

Function: invoke-test-methods OBJ
Package

nst

Source

method.lisp (file)

Function: junit-header STREAM
Package

nst

Source

xml.lisp (file)

Function: low-biased-natnum &optional BIAS
Package

nst

Source

sampling.lisp (file)

Function: make-and-calibrate-check-result &rest ARGS
Package

nst

Source

status.lisp (file)

Function: make-check-note &key (CONTEXT CONTEXT) (STACK STACK) (FORMAT FORMAT) (ARGS ARGS)
Package

nst

Source

status.lisp (file)

Function: make-check-result &key GROUP-NAME CHECK-NAME WARNINGS FAILURES ERRORS INFO TESTS PASSING ERRING FAILING WARNING ELAPSED-TIME TIMESTAMP

Functional wrapper around the constructor for =check-result= structure, permitting the use of apply.

Package

nst

Source

status.lisp (file)

Function: make-config-error ERROR GROUP-RECORD TEST-RECORD MSG
Package

nst

Source

status.lisp (file)

Function: make-error-check-note &key (CONTEXT CONTEXT) (STACK STACK) (FORMAT FORMAT) (ARGS ARGS) (ERROR ERROR)
Package

nst

Source

status.lisp (file)

Function: make-error-note E &rest FORMAT-ARGS &aux FORMAT ARGS
Package

nst

Source

status.lisp (file)

Function: make-fixture-record &key (NAME NAME) (FUNCTION FUNCTION) (BOUND-NAMES BOUND-NAMES) (BINDINGS-LIST BINDINGS-LIST) (DOCUMENTATION DOCUMENTATION) (CACHE-FLUSH CACHE-FLUSH)
Package

nst

Source

fixture.lisp (file)

Function: make-group-record &key (NAME NAME) (ANON-FIXTURE-FORMS ANON-FIXTURE-FORMS) (ASPIRATIONAL ASPIRATIONAL) (GIVEN-FIXTURES GIVEN-FIXTURES) (DOCUMENTATION DOCUMENTATION) (TESTS TESTS) (FIXTURES-SETUP-THUNK FIXTURES-SETUP-THUNK) (FIXTURES-CLEANUP-THUNK FIXTURES-CLEANUP-THUNK) (WITHFIXTURES-SETUP-THUNK WITHFIXTURES-SETUP-THUNK) (WITHFIXTURES-CLEANUP-THUNK WITHFIXTURES-CLEANUP-THUNK) (EACHTEST-SETUP-THUNK EACHTEST-SETUP-THUNK) (EACHTEST-CLEANUP-THUNK EACHTEST-CLEANUP-THUNK) (INCLUDE-GROUPS INCLUDE-GROUPS)
Package

nst

Source

group.lisp (file)

Function: make-group-result &key (TESTS TESTS) (PASSING PASSING) (ERRING ERRING) (FAILING FAILING) (WARNING WARNING) (ELAPSED-TIME ELAPSED-TIME) (TIMESTAMP TIMESTAMP) (GROUP-NAME GROUP-NAME) (CHECK-RESULTS CHECK-RESULTS)
Package

nst

Source

status.lisp (file)

Function: make-multi-results &key (TESTS TESTS) (PASSING PASSING) (ERRING ERRING) (FAILING FAILING) (WARNING WARNING) (ELAPSED-TIME ELAPSED-TIME) (TIMESTAMP TIMESTAMP) (PACKAGE-REPORTS PACKAGE-REPORTS) (GROUP-REPORTS GROUP-REPORTS) (TEST-REPORTS TEST-REPORTS) (SYSTEM SYSTEM) (STATS-SOURCE STATS-SOURCE)
Package

nst

Source

status.lisp (file)

Function: make-name-use &key (NAME NAME) (FIXTURES FIXTURES) (GROUPS GROUPS) (TESTS TESTS) (PACKAGE-P PACKAGE-P) (LAST-USE LAST-USE)
Package

nst

Source

artifacts.lisp (file)

Function: make-package-result &key (TESTS TESTS) (PASSING PASSING) (ERRING ERRING) (FAILING FAILING) (WARNING WARNING) (ELAPSED-TIME ELAPSED-TIME) (TIMESTAMP TIMESTAMP) (PACKAGE-NAME PACKAGE-NAME) (GROUP-RESULTS GROUP-RESULTS)
Package

nst

Source

status.lisp (file)

Function: make-result-stats &key (TESTS TESTS) (PASSING PASSING) (ERRING ERRING) (FAILING FAILING) (WARNING WARNING) (ELAPSED-TIME ELAPSED-TIME) (TIMESTAMP TIMESTAMP)
Package

nst

Source

artifacts.lisp (file)

Function: make-test-record &key (NAME NAME) (GROUP GROUP) (FIXTURES FIXTURES) (CRITERION CRITERION) (FORMS FORMS) (SPECIAL-FIXTURE-NAMES SPECIAL-FIXTURE-NAMES) (DOCUMENTATION DOCUMENTATION) (SETUP SETUP) (CLEANUP CLEANUP) (STARTUP STARTUP) (FINISH FINISH) (RESULTS RESULTS) (ASPIRATIONAL ASPIRATIONAL) (ASPIRATIONAL-SUPP ASPIRATIONAL-SUPP)
Package

nst

Source

test-def.lisp (file)

Function: mid-biased-natnum MID &optional BIAS
Package

nst

Source

sampling.lisp (file)

Function: multi-results-elapsed-time INSTANCE
Function: (setf multi-results-elapsed-time) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-erring INSTANCE
Function: (setf multi-results-erring) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-failing INSTANCE
Function: (setf multi-results-failing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-group-reports INSTANCE
Function: (setf multi-results-group-reports) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-p OBJECT
Package

nst

Source

status.lisp (file)

Function: multi-results-package-reports INSTANCE
Function: (setf multi-results-package-reports) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-passing INSTANCE
Function: (setf multi-results-passing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-stats-source INSTANCE
Function: (setf multi-results-stats-source) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-system INSTANCE
Function: (setf multi-results-system) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-test-reports INSTANCE
Function: (setf multi-results-test-reports) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-tests INSTANCE
Function: (setf multi-results-tests) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-timestamp INSTANCE
Function: (setf multi-results-timestamp) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multi-results-warning INSTANCE
Function: (setf multi-results-warning) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: multiple-report PACKAGES GROUPS TESTS &key SYSTEM
Package

nst

Source

status.lisp (file)

Function: name-use SYMBOL &optional NO-CREATE
Package

nst

Source

artifacts.lisp (file)

Function: name-use-fixtures INSTANCE
Function: (setf name-use-fixtures) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: name-use-groups INSTANCE
Function: (setf name-use-groups) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: name-use-last-use INSTANCE
Function: (setf name-use-last-use) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: name-use-name INSTANCE
Function: (setf name-use-name) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: name-use-p OBJECT
Package

nst

Source

artifacts.lisp (file)

Function: name-use-package-p INSTANCE
Function: (setf name-use-package-p) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: name-use-tests INSTANCE
Function: (setf name-use-tests) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: next-permutation P
Package

nst

Source

permuter.lisp (file)

Function: no-effect ()
Package

nst

Source

utils.lisp (file)

Function: note-name-invocation WHAT
Package

nst

Source

artifacts.lisp (file)

Function: nst-doc ()
Package

nst-asd

Source

nst.asd

Function: nst-dump &key STREAM VERBOSITY

Spit out the full NST state.

Package

nst

Source

status.lisp (file)

Function: nst-xml-dump STREAM
Package

nst

Source

junit.lisp (file)

Function: open-fixture NAME &optional IN-PACKAGE

Push the bindings from a fixture into a package’s namespace.

Package

nst

Source

fixture.lisp (file)

Function: package-groups PACKAGE-OR-SYMBOL

Return the groups whose name is a symbol in the given package.

Package

nst

Source

group.lisp (file)

Function: package-report &optional PACKAGE

Top-level function for reporting the results of a package.

Package

nst

Source

status.lisp (file)

Function: package-result-elapsed-time INSTANCE
Function: (setf package-result-elapsed-time) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-erring INSTANCE
Function: (setf package-result-erring) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-failing INSTANCE
Function: (setf package-result-failing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-group-results INSTANCE
Function: (setf package-result-group-results) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-p OBJECT
Package

nst

Source

status.lisp (file)

Function: package-result-package-name INSTANCE
Function: (setf package-result-package-name) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-passing INSTANCE
Function: (setf package-result-passing) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-tests INSTANCE
Function: (setf package-result-tests) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-timestamp INSTANCE
Function: (setf package-result-timestamp) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: package-result-warning INSTANCE
Function: (setf package-result-warning) VALUE INSTANCE
Package

nst

Source

status.lisp (file)

Function: pick-from-sequence SEQ
Package

nst

Source

sampling.lisp (file)

Function: pprint-cdata-string STREAM STRING
Package

nst

Source

xml.lisp (file)

Function: pprint-xml OBJECT &optional STREAM
Package

nst

Source

xml.lisp (file)

Function: princ-filled-text STRING &optional STREAM
Package

nst

Source

globals.lisp (file)

Function: pull-test-name-list FORM

Given syntax of the form (DEF-CHECK ...) or (DEF-TEST ...), pull the name of the test from the syntax.

Package

nst

Source

group.lisp (file)

Function: record-name-use RECORD

Store the use of a name (as given by a fixture-, group-, or test-record) in the index.

Package

nst

Source

artifacts.lisp (file)

Function: record-package-use BASE-NAME
Package

nst

Source

artifacts.lisp (file)

Function: refine-package-symbol-desigs PACKAGE-DESIG SYMBOL-DESIG
Package

nst

Source

criteria.lisp (file)

Function: relax-stack P
Package

nst

Source

permuter.lisp (file)

Function: report-details GROUP-OR-PACKAGE GP-SUPP-P TEST TEST-SUPP-P
Package

nst

Source

status.lisp (file)

Function: report-group GROUP &optional STREAM *NST-VERBOSITY*

Top-level function for reporting the results of the tests in a group.

Package

nst

Source

status.lisp (file)

Function: report-interesting ()
Package

nst

Source

status.lisp (file)

Function: report-multiple PACKAGES GROUPS TESTS &key STREAM VERBOSITY SYSTEM

Top-level function for reporting the results of several tests.

Package

nst

Source

status.lisp (file)

Function: report-package &optional PACKAGE STREAM *NST-VERBOSITY*

Top-level function for reporting the results of the tests in a package.

Package

nst

Source

status.lisp (file)

Function: report-summary GROUP-OR-PACKAGE GP-SUPP-P TEST TEST-SUPP-P
Package

nst

Source

status.lisp (file)

Function: report-test GROUP TEST &optional STREAM *NST-VERBOSITY*

Top-level function for reporting the results of a test.

Package

nst

Source

status.lisp (file)

Function: restore-protected-option-values STORED-VALUES
Package

nst

Source

globals.lisp (file)

Function: result-stats-elapsed-time INSTANCE
Function: (setf result-stats-elapsed-time) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: result-stats-erring INSTANCE
Function: (setf result-stats-erring) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: result-stats-failing INSTANCE
Function: (setf result-stats-failing) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: result-stats-p OBJECT
Package

nst

Source

artifacts.lisp (file)

Function: result-stats-passing INSTANCE
Function: (setf result-stats-passing) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: result-stats-tests INSTANCE
Function: (setf result-stats-tests) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: result-stats-timestamp INSTANCE
Function: (setf result-stats-timestamp) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: result-stats-warning INSTANCE
Function: (setf result-stats-warning) VALUE INSTANCE
Package

nst

Source

artifacts.lisp (file)

Function: run-debug-options OPTIONS-FORM
Package

nst

Source

globals.lisp (file)

Function: run-group GROUP-NAME

Run a group by its user-given name. Note that this is /not/ an interactive function — certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.

Package

nst

Source

runner.lisp (file)

Function: run-group-record GROUP-RECORD
Package

nst

Source

runner.lisp (file)

Function: run-group-test GROUP-RECORD TEST-RECORD
Package

nst

Source

runner.lisp (file)

Function: run-group-tests GROUP-RECORD TEST-RECORDS

Programmatic entry point for running all tests in a group.

Package

nst

Source

runner.lisp (file)

Function: run-group-tests-fixtures-thunk GROUP-RECORD TEST-RECORDS
Package

nst

Source

runner.lisp (file)

Function: run-nst-command &rest ARGS
Package

nst

Source

command.lisp (file)

Function: run-package &optional PACKAGE-OR-NAME

Run all groups in a package. Note that this is /not/ an interactive function — certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.

Package

nst

Source

runner.lisp (file)

Function: run-test GROUP-NAME TEST-NAME

Run a test standalone by its user-given name (and its group’s name). Note that this is /not/ an interactive function — certain behaviors provided by e.g. the ASDF extension or REPL macros require the dynamic configuration provided by those wrappers.

Package

nst

Source

runner.lisp (file)

Function: run-test-fixtures-thunk GROUP-RECORD TEST-RECORD
Package

nst

Source

runner.lisp (file)

Function: run-test-inst TEST-RECORD
Package

nst

Source

runner.lisp (file)

Function: separate-group-subforms FORMS

Given a list of forms, separate and decode the option specifiers from any test forms. Returns many results:
- The list of test forms, in the same order as originally given.
- The =setup= option forms, and a flag set if the option was present.
- The =cleanup= option forms, and a flag set if the option was present. - The =startup= option forms, and a flag set if the option was present. - The =finish= option forms, and a flag set if the option was present.
- The =each= option forms, and a flag set if the option was present.
- The =each-cleanup= option forms, and a flag set if the option was present. - The =docstring=, or nil if none was given.
- The =aspirational= flag, and a flag set if it was present.
- The =include-groups= list, and a flag set if it was present.

Package

nst

Source

group.lisp (file)

Function: sig-place N VALUE

Returns the n-th significant place of value

Package

nst

Source

utils.lisp (file)

Function: soft-dep-warning PREFIX CND STREAM
Package

nst

Source

errors.lisp (file)

Function: string-escaped STRING
Package

nst

Source

junit.lisp (file)

Function: symbol-or-car NAME-OR-NAME-AND-ARGS

Return the first element given a list, or return a symbol.

Package

nst

Source

utils.lisp (file)

Function: symbol-to-junit-name SYMBOL
Package

nst

Source

xml.lisp (file)

Function: test-is-aspirational TEST-RECORD

Check both the test and group record to see if a test should be considered aspirational.

Package

nst

Source

test-def.lisp (file)

Function: test-record GROUP TEST
Function: (setf test-record) TEST-RECORD GROUP TEST
Package

nst

Source

group.lisp (file)

Function: test-record-aspirational INSTANCE
Function: (setf test-record-aspirational) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-aspirational-supp INSTANCE
Function: (setf test-record-aspirational-supp) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-cleanup INSTANCE
Function: (setf test-record-cleanup) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-criterion INSTANCE
Function: (setf test-record-criterion) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-documentation INSTANCE
Function: (setf test-record-documentation) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-finish INSTANCE
Function: (setf test-record-finish) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-fixtures INSTANCE
Function: (setf test-record-fixtures) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-forms INSTANCE
Function: (setf test-record-forms) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-group INSTANCE
Function: (setf test-record-group) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-name INSTANCE
Function: (setf test-record-name) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-p OBJECT
Package

nst

Source

test-def.lisp (file)

Function: test-record-results INSTANCE
Function: (setf test-record-results) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-setup INSTANCE
Function: (setf test-record-setup) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-special-fixture-names INSTANCE
Function: (setf test-record-special-fixture-names) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-record-startup INSTANCE
Function: (setf test-record-startup) VALUE INSTANCE
Package

nst

Source

test-def.lisp (file)

Function: test-report GROUP TEST

Top-level function for reporting the results of a test.

Package

nst

Source

status.lisp (file)

Function: tighten-stack P SLOTTABLE
Package

nst

Source

permuter.lisp (file)

Function: timestamp-to-string TIMESTAMP
Package

nst

Source

xml.lisp (file)

Function: trace-results ()

Internal debugging function: dump the results hash.

Package

nst

Source

artifacts.lisp (file)

Function: use-stats-from STATS-SOURCE STATS-DEST
Package

nst

Source

status.lisp (file)

Function: whittlable PF
Package

nst

Source

permuter.lisp (file)

Function: whittle-relaxed-stack P
Package

nst

Source

permuter.lisp (file)

Function: wrap-fixture-name-specials FIXTURE-NAMES-SPECIAL FORM
Package

nst

Source

runner.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

6.2.5 Generic functions

Generic Function: actual-value CONDITION
Package

nst

Methods
Method: actual-value (CONDITION not-expected-form)
Source

errors.lisp (file)

Generic Function: apply-criterion TOP ARGS FORM

Internal generic function whose methods are the translations of criteria.

Package

nst

Source

check.lisp (file)

Methods
Method: apply-criterion (TOP (eql methods)) args0 values-form1
Source

method.lisp (file)

Method: apply-criterion (TOP (eql sample)) args0 G1

Invariants to be tested, and the domains over which they range, are specified with the =:sample= criterion:
#+begin_example
(def-criterion ( [ :verify FORM ]
[ :value LAMBDA-LIST ]
[ :domains ((NAME SPEC) ... (NAME SPEC)) ]
[ :where FORM ]
[ :where-ignore (NAME ... NAME) ]
[ :where-declare (DECLARATION ... DECLARATION) ]
[ :sample-size NUMBER ]
[ :qualifying-sample NUMBER ]
[ :max-tries NUMBER ] ) )
#+end_example
- =:verify= :: The expression to be (repeatedly) evaluated, which is expected always to return a non-null value. This is the sole required argument, although in any particular use it is unlikely to be the only argument given.
- =:domains= :: The variables in the =verify= expression which are to be given multiple randomized values. The default value is =nil=, denoting an empty list.
- =:value= :: A lambda list to which the values given by the argument form should be applied. The default value is =nil=, denoting no such arguments.
- =:where= :: A condition which determines the validity of the input argument. For example, the condition would assert that a number is positive in an application where a negative value would be known to cause a failure. The default value is =t=, allowing any values.
- =:where-ignore= :: List of domain variables which are not mentioned in the =where= clause. These names will be declared as ignored in appropriate bindings, suppressing warnings under Lisps which check for such things in interpreted expressions. This list need not be given explicitly when no =where= argument is given. Similarly, the =where-declare= argument accepts a list of declarations to be associated with the =where= form.
- =:sample-size= :: Gives the base specification of the number of value sets which will be generated. Two further arguments have some bearing on the number of generation attempts when the =where= argument is non =t=. The =qualifying-sample= argument gives the minimum acceptable size of actual tested values, not counting sets rejected via the =where= expression. The =max-tries= argument gives the maximum number of value sets to be generated.
Examples:
#+begin_example
(:sample :sample-size 10
:domains ((x (list :elem symbol)))
:verify (equal x (reverse (reverse x))))
#+end_example
#+begin_example
(:sample :domains ((x real))
:where (> x 1)
:verify (< (sqrt x) x)
:sample-size 10
:max-tries 12)
#+end_example

Source

sampling.lisp (file)

Method: apply-criterion (TOP (eql process)) args0 G1
Source

process.lisp (file)

Method: apply-criterion (TOP (eql eval)) args0 FORMS-LIST
Source

process.lisp (file)

Method: apply-criterion (TOP (eql with-common-criterion)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql applying-common-criterion)) args0 G1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql package-internal)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql package-exports)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql slots)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql across)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql permute)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql alist)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql alist*)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql seq)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql each)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql values)) args0 FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql proj)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql progn)) args0 FORMS
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql check-err)) args0 FORMS
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql apply)) args0 EXPRS-FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql any)) args0 EXPR-LIST-FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql all)) args0 EXPR-LIST-FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql not)) args0 EXPR-LIST-FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql perf)) args0 EXPR-FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql err)) args0 EXPR-FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql info)) args0 EXPR-LIST-FORM
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql dump-forms)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql drop-values)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql predicate)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql value-list)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql forms-equal)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql forms-eql)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql forms-eq)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql equalp)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql equal)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql eql)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql symbol)) args1 form0
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql eq)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql true)) args0 values-form1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql true-form)) args0 G1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql warn)) args0 G1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql fail)) args0 G1
Source

criteria.lisp (file)

Method: apply-criterion (TOP (eql pass)) args0 G1
Source

criteria.lisp (file)

Method: apply-criterion TOP ARGS FORM around
Generic Function: arbitrary-by-spec NAME &key TEST SIZE KEY VAL EXISTING GENSYM EXPORTED NONNULL PACKAGE RANK DIMENS ELEM CAR CDR LENGTH NONCONTROL RANGE &allow-other-keys

Return an arbitrary value according to a spec

Package

nst

Source

sampling.lisp (file)

Methods
Method: arbitrary-by-spec (G0 (eql #<structure-class common-lisp:hash-table>)) &key TEST SIZE KEY VAL
Method: arbitrary-by-spec (L (eql #<built-in-class common-lisp:symbol>)) &key EXISTING GENSYM EXPORTED NONNULL PACKAGE
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:array>)) &key RANK DIMENS ELEM
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:cons>)) &key CAR CDR
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:list>)) &key ELEM LENGTH
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:string>)) &key NONCONTROL RANGE
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:vector>)) &key ELEM LENGTH
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:character>)) &key NONCONTROL RANGE
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:complex>)) &key
Method: arbitrary-by-spec (G0 (eql long-float)) &key
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:double-float>)) &key
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:single-float>)) &key
Method: arbitrary-by-spec (G0 (eql short-float)) &key
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:ratio>)) &key
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:bignum>)) &key
Method: arbitrary-by-spec (G0 (eql #<built-in-class common-lisp:fixnum>)) &key
Method: arbitrary-by-spec (G0 (eql scalar)) &key
Method: arbitrary-by-spec (N (eql #<built-in-class common-lisp:float>)) &key
Method: arbitrary-by-spec (N (eql #<built-in-class common-lisp:integer>)) &key
Method: arbitrary-by-spec (N (eql #<built-in-class common-lisp:rational>)) &key
Method: arbitrary-by-spec (N (eql #<built-in-class common-lisp:real>)) &key
Method: arbitrary-by-spec (N (eql #<built-in-class common-lisp:number>)) &key
Method: arbitrary-by-spec UNKNOWN &rest KEYWORD-ARGS
Method: arbitrary-by-spec (OTHER-NAMED-SPEC symbol) &rest KEYWORD-ARGS &key &allow-other-keys
Method: arbitrary-by-spec (ANY (eql #<system-class common-lisp:t>)) &key
Generic Function: arbitrary-grounded-type BOUND

Return an arbitrary type bounded by the given type.

Package

nst

Source

sampling.lisp (file)

Methods
Method: arbitrary-grounded-type (S symbol)
Method: arbitrary-grounded-type (S (eql #<system-class common-lisp:t>))
Method: arbitrary-grounded-type (S (eql #<built-in-class common-lisp:symbol>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:number>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:real>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:rational>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:integer>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:fixnum>))
Method: arbitrary-grounded-type (N (eql bignum))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:ratio>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:float>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:single-float>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:double-float>))
Method: arbitrary-grounded-type (N (eql #<built-in-class common-lisp:complex>))
Generic Function: assertion-failure-args CONDITION
Package

nst

Methods
Method: assertion-failure-args (CONDITION nst-assertion-failure)
Source

process.lisp (file)

Generic Function: assertion-failure-fatal CONDITION
Package

nst

Methods
Method: assertion-failure-fatal (CONDITION nst-assertion-condition)
Source

process.lisp (file)

Generic Function: assertion-failure-formatter CONDITION
Package

nst

Methods
Method: assertion-failure-formatter (CONDITION nst-assertion-failure)
Source

process.lisp (file)

Generic Function: base-name RECORD

Return a symbol naming the record

Package

nst

Source

globals.lisp (file)

Methods
Method: base-name (TEST-RECORD test-record)
Source

test-def.lisp (file)

Method: base-name (FIXTURE-RECORD fixture-record)
Source

fixture.lisp (file)

Method: base-name (GROUP-RECORD group-record)
Source

group.lisp (file)

Method: base-name RECORD

By default, return =nil= for a non-indexable artifact.

Method: base-name (PACKAGE package)
Generic Function: check-note-type-string TYPE CONTEXT
Package

nst

Source

status.lisp (file)

Methods
Method: check-note-type-string TYPE (CONTEXT (eql xml))
Source

xml.lisp (file)

Method: check-note-type-string TYPE CONTEXT
Generic Function: consider-repl-call-save NAME ARGS
Package

nst

Source

command.lisp (file)

Methods
Method: consider-repl-call-save (CMD (eql run)) ARGS
Method: consider-repl-call-save (CMD (eql run-test)) ARGS
Method: consider-repl-call-save (CMD (eql run-group)) ARGS
Method: consider-repl-call-save (CMD (eql run-package)) ARGS
Method: consider-repl-call-save (CMD (eql open)) ARGS
Method: consider-repl-call-save NAME ARGS
Generic Function: criterion OBJECT
Generic Function: (setf criterion) NEW-VALUE OBJECT
Package

nst

Methods
Method: criterion (CRITERION-CONTEXT-LAYER criterion-context-layer)

automatically generated reader method

Source

context.lisp (file)

Method: (setf criterion) NEW-VALUE (CRITERION-CONTEXT-LAYER criterion-context-layer)

automatically generated writer method

Source

context.lisp (file)

Generic Function: criterion-args OBJECT
Generic Function: (setf criterion-args) NEW-VALUE OBJECT
Package

nst

Methods
Method: criterion-args (CRITERION-CONTEXT-LAYER criterion-context-layer)

automatically generated reader method

Source

context.lisp (file)

Method: (setf criterion-args) NEW-VALUE (CRITERION-CONTEXT-LAYER criterion-context-layer)

automatically generated writer method

Source

context.lisp (file)

Generic Function: criterion-name CONDITION
Package

nst

Methods
Method: criterion-name (CONDITION criterion-missing-mandatory-argument)
Source

errors.lisp (file)

Generic Function: domain CONDITION
Package

nst

Methods
Method: domain (CONDITION unknown-arbitrary-domain)
Source

sampling.lisp (file)

Generic Function: expected-form CONDITION
Package

nst

Methods
Method: expected-form (CONDITION not-expected-form)
Source

errors.lisp (file)

Generic Function: fmt-permuter STREAM P
Package

nst

Source

permuter.lisp (file)

Methods
Method: fmt-permuter STREAM (P permuter)
Method: fmt-permuter STREAM (PF permuter-frame)
Generic Function: format-for-warning STREAM ITEM COLON AT-SIGN &rest PARAMS

Hook allowing us to sometimes do better than the usual pretty-printer for warnings.

Package

nst

Source

status.lisp (file)

Methods
Method: format-for-warning STREAM ITEM COLON AT-SIGN &rest PARAMS
Generic Function: given-stack OBJECT
Generic Function: (setf given-stack) NEW-VALUE OBJECT
Package

nst

Methods
Method: given-stack (CRITERION-CONTEXT-LAYER criterion-context-layer)

automatically generated reader method

Source

context.lisp (file)

Method: (setf given-stack) NEW-VALUE (CRITERION-CONTEXT-LAYER criterion-context-layer)

automatically generated writer method

Source

context.lisp (file)

Generic Function: group CONDITION
Package

nst

Methods
Method: group (CONDITION no-such-nst-test)
Source

errors.lisp (file)

Method: group (CONDITION no-such-nst-group)
Source

errors.lisp (file)

Generic Function: has-next OBJECT
Package

nst

Methods
Method: has-next (PERMUTER permuter)

automatically generated reader method

Source

permuter.lisp (file)

Generic Function: junit-group-result GROUP &rest ARGS &key DIR FILE VERBOSE STREAM IF-DIR-DOES-NOT-EXIST IF-FILE-EXISTS
Package

nst

Source

junit.lisp (file)

Methods
Method: junit-group-result (GROUP group-result) &rest ARGS &key VERBOSE DIR FILE STREAM IF-DIR-DOES-NOT-EXIST IF-FILE-EXISTS
Method: junit-group-result (GROUP symbol) &rest ARGS &key DIR FILE VERBOSE STREAM IF-DIR-DOES-NOT-EXIST IF-FILE-EXISTS
Generic Function: keyword-args CONDITION
Package

nst

Methods
Method: keyword-args (CONDITION unknown-arbitrary-domain)
Source

sampling.lisp (file)

Generic Function: method-name OBJECT
Generic Function: (setf method-name) NEW-VALUE OBJECT
Package

nst

Methods
Method: method-name (METHOD-CONTEXT-LAYER method-context-layer)

automatically generated reader method

Source

method.lisp (file)

Method: (setf method-name) NEW-VALUE (METHOD-CONTEXT-LAYER method-context-layer)

automatically generated writer method

Source

method.lisp (file)

Generic Function: nst-arg-names COMMAND

Return a string giving help for the argument names for an ~ NST REPL command.

Package

nst

Source

command.lisp (file)

Methods
Method: nst-arg-names (CMD (eql apply))
Method: nst-arg-names (CMD (eql whatis))
Method: nst-arg-names (CMD (eql unset))
Method: nst-arg-names (CMD (eql set))
Method: nst-arg-names (CMD (eql clear))
Method: nst-arg-names (CMD (eql detail))
Method: nst-arg-names (CMD (eql report))
Method: nst-arg-names (CMD (eql run))
Method: nst-arg-names (CMD (eql run-test))
Method: nst-arg-names (CMD (eql run-group))
Method: nst-arg-names (CMD (eql run-package))
Method: nst-arg-names (CMD (eql open))
Method: nst-arg-names (CMD (eql debug))
Method: nst-arg-names (CMD (eql help))
Method: nst-arg-names COMMAND
Generic Function: nst-assertion-result CONDITION
Package

nst

Methods
Method: nst-assertion-result (CONDITION nst-assertion-result-check)
Source

process.lisp (file)

Generic Function: nst-long-help COMMAND

Return the long help message for an NST REPL command.

Package

nst

Source

command.lisp (file)

Methods
Method: nst-long-help (CMD (eql apply))
Method: nst-long-help (CMD (eql whatis))
Method: nst-long-help (CMD (eql unset))
Method: nst-long-help (CMD (eql set))
Method: nst-long-help (CMD (eql clear))
Method: nst-long-help (CMD (eql detail))
Method: nst-long-help (CMD (eql report))
Method: nst-long-help (CMD (eql run))
Method: nst-long-help (CMD (eql run-test))
Method: nst-long-help (CMD (eql run-group))
Method: nst-long-help (CMD (eql run-package))
Method: nst-long-help (CMD (eql open))
Method: nst-long-help (CMD (eql debug))
Method: nst-long-help (CMD (eql help))
Method: nst-long-help COMMAND
Generic Function: nst-repl-property-display N

Return the display value of an NST property’s internal value.

Package

nst

Source

globals.lisp (file)

Methods
Method: nst-repl-property-display (N (eql verbose))
Source

command.lisp (file)

Method: nst-repl-property-display (N (eql debug-on-fail))
Source

command.lisp (file)

Method: nst-repl-property-display (N (eql debug-on-error))
Source

command.lisp (file)

Generic Function: nst-repl-property-doc N

Return the documentation string of an NST property.

Package

nst

Source

globals.lisp (file)

Methods
Method: nst-repl-property-doc (N (eql verbose))
Source

command.lisp (file)

Method: nst-repl-property-doc (N (eql debug-on-fail))
Source

command.lisp (file)

Method: nst-repl-property-doc (N (eql debug-on-error))
Source

command.lisp (file)

Generic Function: nst-repl-property-encode PROP VAL

Encode an NST property’s display value as an internal value.

Package

nst

Source

globals.lisp (file)

Methods
Method: nst-repl-property-encode (N (eql verbose)) VALUE
Source

command.lisp (file)

Method: nst-repl-property-encode (N (eql debug-on-fail)) VALUE
Source

command.lisp (file)

Method: nst-repl-property-encode (N (eql debug-on-error)) VALUE
Source

command.lisp (file)

Generic Function: nst-short-help COMMAND

Return the short help message for an NST REPL command.

Package

nst

Source

command.lisp (file)

Methods
Method: nst-short-help (CMD (eql apply))
Method: nst-short-help (CMD (eql whatis))
Method: nst-short-help (CMD (eql unset))
Method: nst-short-help (CMD (eql set))
Method: nst-short-help (CMD (eql clear))
Method: nst-short-help (CMD (eql detail))
Method: nst-short-help (CMD (eql report))
Method: nst-short-help (CMD (eql run))
Method: nst-short-help (CMD (eql run-test))
Method: nst-short-help (CMD (eql run-group))
Method: nst-short-help (CMD (eql run-package))
Method: nst-short-help (CMD (eql open))
Method: nst-short-help (CMD (eql debug))
Method: nst-short-help (CMD (eql help))
Method: nst-short-help COMMAND
Generic Function: object OBJECT
Generic Function: (setf object) NEW-VALUE OBJECT
Package

nst

Methods
Method: object (METHOD-CONTEXT-LAYER method-context-layer)

automatically generated reader method

Source

method.lisp (file)

Method: (setf object) NEW-VALUE (METHOD-CONTEXT-LAYER method-context-layer)

automatically generated writer method

Source

method.lisp (file)

Generic Function: old-name CONDITION
Package

nst

Methods
Method: old-name (CONDITION nst-deprecation-warning-mixin)
Source

errors.lisp (file)

Generic Function: package-of CONDITION
Package

nst

Methods
Method: package-of (CONDITION no-nst-groups-in-package)
Source

errors.lisp (file)

Generic Function: prep-arg-names-help ARG-LIST

This function formats the lambda list of an
NST REPL command for display in the online-help system. The macro def-nst-interactive-command that expands a REPL command definition into the underlying Lisp defmethods uses this function to generate the bodies of the help-related methods, so this function must be available from compile-time forward.

Package

nst

Source

command.lisp (file)

Methods
Method: prep-arg-names-help ARG-LIST &aux IN-MACROLIST IN-KEYLIST
Generic Function: replacement CONDITION
Package

nst

Methods
Method: replacement (CONDITION nst-deprecation-warning-mixin)
Source

errors.lisp (file)

Generic Function: required-keyarg CONDITION
Package

nst

Methods
Method: required-keyarg (CONDITION criterion-missing-mandatory-argument)
Source

errors.lisp (file)

Generic Function: result-summary REPORT &optional CODE TOTAL PASSED ERRED FAILED WARNED OTHER

Receives a reporting structure (or list of them); returns a six-value summary of the results:
- A symbol, one of: :error :fail :warn :info :clear
- The total number of named checks.
- The number passed.
- The number raising an error.
- The number failing.
- The number giving a warning.

Package

nst

Source

status.lisp (file)

Methods
Method: result-summary (RS null) &optional CODE TOTAL PASSED ERRED FAILED WARNED OTHER
Method: result-summary (RS cons) &optional CODE TOTAL PASSED ERRED FAILED WARNED OTHER
Method: result-summary (R multi-results) &optional CODE TOTAL PASSED ERRED FAILED WARNED OTHER
Method: result-summary (R package-result) &optional CODE TOTAL PASSED ERRED FAILED WARNED OTHER
Method: result-summary (R group-result) &optional CODE TOTAL PASSED ERRED FAILED WARNED OTHER
Method: result-summary (R check-result) &optional CODE TOTAL PASSED ERRED FAILED WARNED OTHER
Generic Function: run-command-actual COMMAND &rest ARGS

Top-level command interpreter for the NST tester

Package

nst

Source

command.lisp (file)

Methods
Method: run-command-actual (CMD (eql apply)) &rest G0
Method: run-command-actual (CMD (eql whatis)) &rest G0
Method: run-command-actual (CMD (eql unset)) &rest G0
Method: run-command-actual (CMD (eql set)) &rest G0
Method: run-command-actual (CMD (eql clear)) &rest G0
Method: run-command-actual (CMD (eql detail)) &rest G0
Method: run-command-actual (CMD (eql report)) &rest G0
Method: run-command-actual (CMD (eql run)) &rest G0
Method: run-command-actual (CMD (eql run-test)) &rest G0
Method: run-command-actual (CMD (eql run-group)) &rest G0
Method: run-command-actual (CMD (eql run-package)) &rest G0
Method: run-command-actual (CMD (eql open)) &rest G0
Method: run-command-actual (CMD (eql debug)) &rest G0
Method: run-command-actual (CMD (eql help)) &rest G0
Method: run-command-actual COMMAND &rest ARGS
Method: run-command-actual COMMAND &rest ARGS before
Generic Function: set-nst-property NAME VALUE
Package

nst

Source

globals.lisp (file)

Methods
Method: set-nst-property (NAME (eql verbose)) VALUE
Source

command.lisp (file)

Method: set-nst-property (NAME (eql debug-on-fail)) VALUE
Source

command.lisp (file)

Method: set-nst-property (NAME (eql debug-on-error)) VALUE
Source

command.lisp (file)

Method: set-nst-property NAME VALUE
Generic Function: show-context-layer LAYER

Used to determine whether a context-layer should be displayed. Methods may safely refer and mutate the hashtable stored in the special variable -context-display-state- .

Package

nst

Source

context.lisp (file)

Methods
Method: show-context-layer (LAYER method-context-layer)
Source

method.lisp (file)

Method: show-context-layer (LAYER criterion-context-layer)
Method: show-context-layer L
Generic Function: show-nst-property NAME

Display a property value (presumably) to the REPL.

Package

nst

Source

globals.lisp (file)

Methods
Method: show-nst-property (NAME (eql verbose))
Source

command.lisp (file)

Method: show-nst-property (NAME (eql debug-on-fail))
Source

command.lisp (file)

Method: show-nst-property (NAME (eql debug-on-error))
Source

command.lisp (file)

Generic Function: test CONDITION
Package

nst

Methods
Method: test (CONDITION no-such-nst-test)
Source

errors.lisp (file)

Generic Function: trace-group GR

Provide debugging information about a group.

Package

nst

Source

artifacts.lisp (file)

Methods
Method: trace-group (G group-record)
Source

group.lisp (file)

Method: trace-group GR
Generic Function: trace-test GR TS

Provide debugging information about a test.

Package

nst

Source

artifacts.lisp (file)

Methods
Method: trace-test (GR group-record) (TS test-record)

Return non-nil if an item is a group record.

Source

test-def.lisp (file)

Method: trace-test GR TS

Next: , Previous: , Up: Internal definitions   [Contents][Index]

6.2.6 Conditions

Condition: criterion-missing-mandatory-argument ()
Package

nst

Source

errors.lisp (file)

Direct superclasses

nst-error (condition)

Direct methods
Direct slots
Slot: criterion-name
Initargs

:criterion-name

Readers

criterion-name (generic function)

Slot: required-keyarg
Initargs

:required-keyarg

Readers

required-keyarg (generic function)

Condition: debug-for-fail ()

Condition raised

Package

nst

Source

errors.lisp (file)

Direct superclasses

condition (condition)

Condition: no-nst-groups-in-package ()
Package

nst

Source

errors.lisp (file)

Direct superclasses

nst-error (condition)

Direct methods

package-of (method)

Direct slots
Slot: package
Initargs

:package

Readers

package-of (generic function)

Condition: no-such-nst-group ()
Package

nst

Source

errors.lisp (file)

Direct superclasses

nst-error (condition)

Direct methods

group (method)

Direct slots
Slot: group
Initargs

:group

Readers

group (generic function)

Condition: no-such-nst-test ()
Package

nst

Source

errors.lisp (file)

Direct superclasses

nst-error (condition)

Direct methods
Direct slots
Slot: group
Initargs

:group

Readers

group (generic function)

Slot: test
Initargs

:test

Readers

test (generic function)

Condition: not-expected-form ()
Package

nst

Source

errors.lisp (file)

Direct superclasses

nst-error (condition)

Direct methods
Direct slots
Slot: expected-form
Initargs

:expected-form

Readers

expected-form (generic function)

Slot: actual-value
Initargs

:actual-value

Readers

actual-value (generic function)

Condition: nst-assertion-condition ()
Package

nst

Source

process.lisp (file)

Direct superclasses

condition (condition)

Direct subclasses
Direct methods

assertion-failure-fatal (method)

Direct slots
Slot: fatal
Initargs

:fatal

Readers

assertion-failure-fatal (generic function)

Condition: nst-assertion-failure ()
Package

nst

Source

process.lisp (file)

Direct superclasses

nst-assertion-condition (condition)

Direct methods
Direct slots
Slot: formatter
Initargs

:formatter

Readers

assertion-failure-formatter (generic function)

Slot: args
Initargs

:args

Readers

assertion-failure-args (generic function)

Condition: nst-assertion-result-check ()
Package

nst

Source

process.lisp (file)

Direct superclasses

nst-assertion-condition (condition)

Direct methods

nst-assertion-result (method)

Direct slots
Slot: result
Initargs

:result

Readers

nst-assertion-result (generic function)

Condition: nst-deprecation-warning-mixin ()

Mixin of field used in deprecation warnings

Package

nst

Source

errors.lisp (file)

Direct superclasses

condition (condition)

Direct subclasses
Direct methods
Direct slots
Slot: old-name
Initargs

:old-name

Readers

old-name (generic function)

Slot: replacement
Initargs

:replacement

Readers

replacement (generic function)

Condition: nst-error ()
Package

nst

Source

errors.lisp (file)

Direct superclasses

condition (condition)

Direct subclasses
Condition: nst-hard-deprecation ()
Package

nst

Source

errors.lisp (file)

Direct superclasses
Condition: nst-soft-deprecation ()
Package

nst

Source

errors.lisp (file)

Direct superclasses
Condition: nst-soft-keyarg-deprecation ()
Package

nst

Source

errors.lisp (file)

Direct superclasses
Condition: unknown-arbitrary-domain ()
Package

nst

Source

sampling.lisp (file)

Direct superclasses

nst-error (condition)

Direct methods
Direct slots
Slot: domain
Initargs

:domain

Readers

domain (generic function)

Slot: keyword-args
Initargs

:keyword-args

Readers

keyword-args (generic function)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

6.2.7 Structures

Structure: check-note ()

A single note issued in criteria checking.
context - the surrounding criteria structure, a list of context-layer structs stack - the stack of values at the note point
format, args - further details; this string and list may e.g. be provided to cl:format

** IMPORTANT ** Do not use check-note-format or check-note-args! Do not assume that the format slot is a string! The API for these slots is apply-check-note-formatter, which checks for function values (or other types allowed as a formatter since the time when this docstring was written).

Package

nst

Source

status.lisp (file)

Direct superclasses

structure-object (structure)

Direct subclasses

error-check-note (structure)

Direct methods

add-warning (method)

Direct slots
Slot: context
Readers

check-note-context (function)

Writers

(setf check-note-context) (function)

Slot: stack
Readers

check-note-stack (function)

Writers

(setf check-note-stack) (function)

Slot: format
Readers

check-note-format (function)

Writers

(setf check-note-format) (function)

Slot: args
Readers

check-note-args (function)

Writers

(setf check-note-args) (function)

Structure: check-result ()

Overall check result structure, containing notes of four distinct types. A note is an instance of the check-note structure below. The four note types are: warnings - generated warnings
failures - criteria which are not met, but not a Lisp error
errors - Lisp errors
info - supplimentary information
Each of these fields is a list; warnings, failures and errors are check-note instances, and the info field is of any value.

Package

nst

Source

status.lisp (file)

Direct superclasses

result-stats (structure)

Direct methods

result-summary (method)

Direct slots
Slot: tests
Initform

1

Readers

check-result-tests (function)

Writers

(setf check-result-tests) (function)

Slot: group-name
Initform

nst::*nst-group-name*

Readers

check-result-group-name (function)

Writers

(setf check-result-group-name) (function)

Slot: check-name
Initform

nst::*nst-check-user-name*

Readers

check-result-check-name (function)

Writers

(setf check-result-check-name) (function)

Slot: warnings
Readers

check-result-warnings (function)

Writers

(setf check-result-warnings) (function)

Slot: failures
Readers

check-result-failures (function)

Writers

(setf check-result-failures) (function)

Slot: errors
Readers

check-result-errors (function)

Writers

(setf check-result-errors) (function)

Slot: info
Readers

check-result-info (function)

Writers

(setf check-result-info) (function)

Structure: error-check-note ()

A note issued in regards to a thrown error.

Package

nst

Source

status.lisp (file)

Direct superclasses

check-note (structure)

Direct slots
Slot: error
Readers

error-check-note-error (function)

Writers

(setf error-check-note-error) (function)

Structure: fixture-record ()

Information associated with a fixture definition

Package

nst

Source

fixture.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

base-name (method)

Direct slots
Slot: name
Readers

fixture-record-name (function)

Writers

(setf fixture-record-name) (function)

Slot: function
Readers

fixture-record-function (function)

Writers

(setf fixture-record-function) (function)

Slot: bound-names
Readers

fixture-record-bound-names (function)

Writers

(setf fixture-record-bound-names) (function)

Slot: bindings-list
Readers

fixture-record-bindings-list (function)

Writers

(setf fixture-record-bindings-list) (function)

Slot: documentation
Readers

fixture-record-documentation (function)

Writers

(setf fixture-record-documentation) (function)

Slot: cache-flush
Readers

fixture-record-cache-flush (function)

Writers

(setf fixture-record-cache-flush) (function)

Structure: group-record ()

Superclass of NST group definitions.

Package

nst

Source

group.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods
Direct slots
Slot: name
Readers

group-record-name (function)

Writers

(setf group-record-name) (function)

Slot: anon-fixture-forms
Readers

group-record-anon-fixture-forms (function)

Writers

(setf group-record-anon-fixture-forms) (function)

Slot: aspirational
Readers

group-record-aspirational (function)

Writers

(setf group-record-aspirational) (function)

Slot: given-fixtures
Readers

group-record-given-fixtures (function)

Writers

(setf group-record-given-fixtures) (function)

Slot: documentation
Readers

group-record-documentation (function)

Writers

(setf group-record-documentation) (function)

Slot: tests
Initform

(make-hash-table :test (quote eq))

Readers

group-record-tests (function)

Writers

(setf group-record-tests) (function)

Slot: fixtures-setup-thunk
Readers

group-record-fixtures-setup-thunk (function)

Writers

(setf group-record-fixtures-setup-thunk) (function)

Slot: fixtures-cleanup-thunk
Readers

group-record-fixtures-cleanup-thunk (function)

Writers

(setf group-record-fixtures-cleanup-thunk) (function)

Slot: withfixtures-setup-thunk
Readers

group-record-withfixtures-setup-thunk (function)

Writers

(setf group-record-withfixtures-setup-thunk) (function)

Slot: withfixtures-cleanup-thunk
Readers

group-record-withfixtures-cleanup-thunk (function)

Writers

(setf group-record-withfixtures-cleanup-thunk) (function)

Slot: eachtest-setup-thunk
Readers

group-record-eachtest-setup-thunk (function)

Writers

(setf group-record-eachtest-setup-thunk) (function)

Slot: eachtest-cleanup-thunk
Readers

group-record-eachtest-cleanup-thunk (function)

Writers

(setf group-record-eachtest-cleanup-thunk) (function)

Slot: include-groups
Readers

group-record-include-groups (function)

Writers

(setf group-record-include-groups) (function)

Structure: group-result ()

Overall group result structure, mapping checks to results by name.

Package

nst

Source

status.lisp (file)

Direct superclasses

result-stats (structure)

Direct methods
Direct slots
Slot: group-name
Readers

group-result-group-name (function)

Writers

(setf group-result-group-name) (function)

Slot: check-results
Initform

(make-hash-table :test (quote eq))

Readers

group-result-check-results (function)

Writers

(setf group-result-check-results) (function)

Structure: multi-results ()

Multiple results structure.

Package

nst

Source

status.lisp (file)

Direct superclasses

result-stats (structure)

Direct methods

result-summary (method)

Direct slots
Slot: package-reports
Readers

multi-results-package-reports (function)

Writers

(setf multi-results-package-reports) (function)

Slot: group-reports
Readers

multi-results-group-reports (function)

Writers

(setf multi-results-group-reports) (function)

Slot: test-reports
Readers

multi-results-test-reports (function)

Writers

(setf multi-results-test-reports) (function)

Slot: system
Readers

multi-results-system (function)

Writers

(setf multi-results-system) (function)

Slot: stats-source
Readers

multi-results-stats-source (function)

Writers

(setf multi-results-stats-source) (function)

Structure: name-use ()

Record for tracking the artifacts which NST associates with a name.

Package

nst

Source

artifacts.lisp (file)

Direct superclasses

structure-object (structure)

Direct slots
Slot: name
Readers

name-use-name (function)

Writers

(setf name-use-name) (function)

Slot: fixtures
Readers

name-use-fixtures (function)

Writers

(setf name-use-fixtures) (function)

Slot: groups
Readers

name-use-groups (function)

Writers

(setf name-use-groups) (function)

Slot: tests
Readers

name-use-tests (function)

Writers

(setf name-use-tests) (function)

Slot: package-p
Readers

name-use-package-p (function)

Writers

(setf name-use-package-p) (function)

Slot: last-use
Readers

name-use-last-use (function)

Writers

(setf name-use-last-use) (function)

Structure: package-result ()

Overall package result structure, mapping groups to results by name.

Package

nst

Source

status.lisp (file)

Direct superclasses

result-stats (structure)

Direct methods

result-summary (method)

Direct slots
Slot: package-name
Readers

package-result-package-name (function)

Writers

(setf package-result-package-name) (function)

Slot: group-results
Initform

(make-hash-table :test (quote eq))

Readers

package-result-group-results (function)

Writers

(setf package-result-group-results) (function)

Structure: result-stats ()

Statistics common to the different result summaries.

Package

nst

Source

artifacts.lisp (file)

Direct superclasses

structure-object (structure)

Direct subclasses
Direct slots
Slot: tests
Initform

0

Readers

result-stats-tests (function)

Writers

(setf result-stats-tests) (function)

Slot: passing
Initform

0

Readers

result-stats-passing (function)

Writers

(setf result-stats-passing) (function)

Slot: erring
Initform

0

Readers

result-stats-erring (function)

Writers

(setf result-stats-erring) (function)

Slot: failing
Initform

0

Readers

result-stats-failing (function)

Writers

(setf result-stats-failing) (function)

Slot: warning
Initform

0

Readers

result-stats-warning (function)

Writers

(setf result-stats-warning) (function)

Slot: elapsed-time
Initform

0

Readers

result-stats-elapsed-time (function)

Writers

(setf result-stats-elapsed-time) (function)

Slot: timestamp
Initform

(multiple-value-list (get-decoded-time))

Readers

result-stats-timestamp (function)

Writers

(setf result-stats-timestamp) (function)

Structure: test-record ()

Structure storing parsed NST test records.
- =group= :: The group-record structure (not the symbolic name) of the group containing this test.

Package

nst

Source

test-def.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods
Direct slots
Slot: name
Readers

test-record-name (function)

Writers

(setf test-record-name) (function)

Slot: group
Readers

test-record-group (function)

Writers

(setf test-record-group) (function)

Slot: fixtures
Readers

test-record-fixtures (function)

Writers

(setf test-record-fixtures) (function)

Slot: criterion
Readers

test-record-criterion (function)

Writers

(setf test-record-criterion) (function)

Slot: forms
Readers

test-record-forms (function)

Writers

(setf test-record-forms) (function)

Slot: special-fixture-names
Readers

test-record-special-fixture-names (function)

Writers

(setf test-record-special-fixture-names) (function)

Slot: documentation
Readers

test-record-documentation (function)

Writers

(setf test-record-documentation) (function)

Slot: setup
Readers

test-record-setup (function)

Writers

(setf test-record-setup) (function)

Slot: cleanup
Readers

test-record-cleanup (function)

Writers

(setf test-record-cleanup) (function)

Slot: startup
Readers

test-record-startup (function)

Writers

(setf test-record-startup) (function)

Slot: finish
Readers

test-record-finish (function)

Writers

(setf test-record-finish) (function)

Slot: results
Readers

test-record-results (function)

Writers

(setf test-record-results) (function)

Slot: aspirational
Readers

test-record-aspirational (function)

Writers

(setf test-record-aspirational) (function)

Slot: aspirational-supp
Readers

test-record-aspirational-supp (function)

Writers

(setf test-record-aspirational-supp) (function)


Previous: , Up: Internal definitions   [Contents][Index]

6.2.8 Classes

Class: context-layer ()

Superclass of records of test criterion.

Package

nst

Source

context.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses
Class: criterion-context-layer ()

A record of test criterion
criterion - the criterion symbol itself
criterion-args - arguments to the criterion
given-stack - the stack of values assessed by the criterion

Package

nst

Source

context.lisp (file)

Direct superclasses

context-layer (class)

Direct methods
Direct slots
Slot: criterion
Initargs

:criterion

Readers

criterion (generic function)

Writers

(setf criterion) (generic function)

Slot: criterion-args
Initargs

:criterion-args

Readers

criterion-args (generic function)

Writers

(setf criterion-args) (generic function)

Slot: given-stack
Initargs

:given-stack

Readers

given-stack (generic function)

Writers

(setf given-stack) (generic function)

Class: method-context-layer ()

A record of method-based test invocation.

Package

nst

Source

method.lisp (file)

Direct superclasses

context-layer (class)

Direct methods
Direct slots
Slot: method-name
Initargs

:method-name

Readers

method-name (generic function)

Writers

(setf method-name) (generic function)

Slot: class-name
Initargs

:class-name

Readers

class-name (generic function)

Writers

(setf class-name) (generic function)

Slot: object
Initargs

:object

Readers

object (generic function)

Writers

(setf object) (generic function)

Class: permuter ()
Package

nst

Source

permuter.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
Direct slots
Slot: next-permutation
Type

(or null cons)

Slot: perm-stack
Type

(or null (cons nst::permuter-frame))

Slot: has-next
Initform

t

Readers

has-next (generic function)

Slot: degenerate
Class: permuter-frame ()
Package

nst

Source

permuter.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

fmt-permuter (method)

Direct slots
Slot: next-firsts
Type

(or null cons)

Initargs

:next

Slot: prev-firsts
Type

(or null cons)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   M   N  
Index Entry  Section

F
File, Lisp, nst.asd: The nst<dot>asd file
File, Lisp, nst/core/artifacts.lisp: The nst/core/artifacts<dot>lisp file
File, Lisp, nst/core/check.lisp: The nst/core/check<dot>lisp file
File, Lisp, nst/core/command.lisp: The nst/core/command<dot>lisp file
File, Lisp, nst/core/context.lisp: The nst/core/context<dot>lisp file
File, Lisp, nst/core/criteria.lisp: The nst/core/criteria<dot>lisp file
File, Lisp, nst/core/errors.lisp: The nst/core/errors<dot>lisp file
File, Lisp, nst/core/fixture.lisp: The nst/core/fixture<dot>lisp file
File, Lisp, nst/core/globals.lisp: The nst/core/globals<dot>lisp file
File, Lisp, nst/core/group.lisp: The nst/core/group<dot>lisp file
File, Lisp, nst/core/interfaces.lisp: The nst/core/interfaces<dot>lisp file
File, Lisp, nst/core/interrupt.lisp: The nst/core/interrupt<dot>lisp file
File, Lisp, nst/core/junit.lisp: The nst/core/junit<dot>lisp file
File, Lisp, nst/core/method.lisp: The nst/core/method<dot>lisp file
File, Lisp, nst/core/package.lisp: The nst/core/package<dot>lisp file
File, Lisp, nst/core/permuter.lisp: The nst/core/permuter<dot>lisp file
File, Lisp, nst/core/process.lisp: The nst/core/process<dot>lisp file
File, Lisp, nst/core/runner.lisp: The nst/core/runner<dot>lisp file
File, Lisp, nst/core/sampling.lisp: The nst/core/sampling<dot>lisp file
File, Lisp, nst/core/status.lisp: The nst/core/status<dot>lisp file
File, Lisp, nst/core/test-def.lisp: The nst/core/test-def<dot>lisp file
File, Lisp, nst/core/utils.lisp: The nst/core/utils<dot>lisp file
File, Lisp, nst/core/xml.lisp: The nst/core/xml<dot>lisp file

L
Lisp File, nst.asd: The nst<dot>asd file
Lisp File, nst/core/artifacts.lisp: The nst/core/artifacts<dot>lisp file
Lisp File, nst/core/check.lisp: The nst/core/check<dot>lisp file
Lisp File, nst/core/command.lisp: The nst/core/command<dot>lisp file
Lisp File, nst/core/context.lisp: The nst/core/context<dot>lisp file
Lisp File, nst/core/criteria.lisp: The nst/core/criteria<dot>lisp file
Lisp File, nst/core/errors.lisp: The nst/core/errors<dot>lisp file
Lisp File, nst/core/fixture.lisp: The nst/core/fixture<dot>lisp file
Lisp File, nst/core/globals.lisp: The nst/core/globals<dot>lisp file
Lisp File, nst/core/group.lisp: The nst/core/group<dot>lisp file
Lisp File, nst/core/interfaces.lisp: The nst/core/interfaces<dot>lisp file
Lisp File, nst/core/interrupt.lisp: The nst/core/interrupt<dot>lisp file
Lisp File, nst/core/junit.lisp: The nst/core/junit<dot>lisp file
Lisp File, nst/core/method.lisp: The nst/core/method<dot>lisp file
Lisp File, nst/core/package.lisp: The nst/core/package<dot>lisp file
Lisp File, nst/core/permuter.lisp: The nst/core/permuter<dot>lisp file
Lisp File, nst/core/process.lisp: The nst/core/process<dot>lisp file
Lisp File, nst/core/runner.lisp: The nst/core/runner<dot>lisp file
Lisp File, nst/core/sampling.lisp: The nst/core/sampling<dot>lisp file
Lisp File, nst/core/status.lisp: The nst/core/status<dot>lisp file
Lisp File, nst/core/test-def.lisp: The nst/core/test-def<dot>lisp file
Lisp File, nst/core/utils.lisp: The nst/core/utils<dot>lisp file
Lisp File, nst/core/xml.lisp: The nst/core/xml<dot>lisp file

M
Module, nst/core: The nst/core module

N
nst.asd: The nst<dot>asd file
nst/core: The nst/core module
nst/core/artifacts.lisp: The nst/core/artifacts<dot>lisp file
nst/core/check.lisp: The nst/core/check<dot>lisp file
nst/core/command.lisp: The nst/core/command<dot>lisp file
nst/core/context.lisp: The nst/core/context<dot>lisp file
nst/core/criteria.lisp: The nst/core/criteria<dot>lisp file
nst/core/errors.lisp: The nst/core/errors<dot>lisp file
nst/core/fixture.lisp: The nst/core/fixture<dot>lisp file
nst/core/globals.lisp: The nst/core/globals<dot>lisp file
nst/core/group.lisp: The nst/core/group<dot>lisp file
nst/core/interfaces.lisp: The nst/core/interfaces<dot>lisp file
nst/core/interrupt.lisp: The nst/core/interrupt<dot>lisp file
nst/core/junit.lisp: The nst/core/junit<dot>lisp file
nst/core/method.lisp: The nst/core/method<dot>lisp file
nst/core/package.lisp: The nst/core/package<dot>lisp file
nst/core/permuter.lisp: The nst/core/permuter<dot>lisp file
nst/core/process.lisp: The nst/core/process<dot>lisp file
nst/core/runner.lisp: The nst/core/runner<dot>lisp file
nst/core/sampling.lisp: The nst/core/sampling<dot>lisp file
nst/core/status.lisp: The nst/core/status<dot>lisp file
nst/core/test-def.lisp: The nst/core/test-def<dot>lisp file
nst/core/utils.lisp: The nst/core/utils<dot>lisp file
nst/core/xml.lisp: The nst/core/xml<dot>lisp file

Jump to:   F   L   M   N  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   %   (  
A   B   C   D   E   F   G   H   I   J   K   L   M   N   O   P   R   S   T   U   W  
Index Entry  Section

%
%make-check-result: Internal functions

(
(setf check-note-args): Internal functions
(setf check-note-context): Internal functions
(setf check-note-format): Internal functions
(setf check-note-stack): Internal functions
(setf check-result-check-name): Internal functions
(setf check-result-elapsed-time): Internal functions
(setf check-result-erring): Internal functions
(setf check-result-errors): Internal functions
(setf check-result-failing): Internal functions
(setf check-result-failures): Internal functions
(setf check-result-group-name): Internal functions
(setf check-result-info): Internal functions
(setf check-result-passing): Internal functions
(setf check-result-tests): Internal functions
(setf check-result-timestamp): Internal functions
(setf check-result-warning): Internal functions
(setf check-result-warnings): Internal functions
(setf criterion): Internal generic functions
(setf criterion): Internal generic functions
(setf criterion-args): Internal generic functions
(setf criterion-args): Internal generic functions
(setf error-check-note-args): Internal functions
(setf error-check-note-context): Internal functions
(setf error-check-note-error): Internal functions
(setf error-check-note-format): Internal functions
(setf error-check-note-stack): Internal functions
(setf fixture-bindings): Internal functions
(setf fixture-letlist): Internal functions
(setf fixture-record): Internal functions
(setf fixture-record-bindings-list): Internal functions
(setf fixture-record-bound-names): Internal functions
(setf fixture-record-cache-flush): Internal functions
(setf fixture-record-documentation): Internal functions
(setf fixture-record-function): Internal functions
(setf fixture-record-name): Internal functions
(setf given-stack): Internal generic functions
(setf given-stack): Internal generic functions
(setf group-record): Internal functions
(setf group-record-anon-fixture-forms): Internal functions
(setf group-record-aspirational): Internal functions
(setf group-record-documentation): Internal functions
(setf group-record-eachtest-cleanup-thunk): Internal functions
(setf group-record-eachtest-setup-thunk): Internal functions
(setf group-record-fixtures-cleanup-thunk): Internal functions
(setf group-record-fixtures-setup-thunk): Internal functions
(setf group-record-given-fixtures): Internal functions
(setf group-record-include-groups): Internal functions
(setf group-record-name): Internal functions
(setf group-record-tests): Internal functions
(setf group-record-withfixtures-cleanup-thunk): Internal functions
(setf group-record-withfixtures-setup-thunk): Internal functions
(setf group-result-check-results): Internal functions
(setf group-result-elapsed-time): Internal functions
(setf group-result-erring): Internal functions
(setf group-result-failing): Internal functions
(setf group-result-group-name): Internal functions
(setf group-result-passing): Internal functions
(setf group-result-tests): Internal functions
(setf group-result-timestamp): Internal functions
(setf group-result-warning): Internal functions
(setf method-name): Internal generic functions
(setf method-name): Internal generic functions
(setf multi-results-elapsed-time): Internal functions
(setf multi-results-erring): Internal functions
(setf multi-results-failing): Internal functions
(setf multi-results-group-reports): Internal functions
(setf multi-results-package-reports): Internal functions
(setf multi-results-passing): Internal functions
(setf multi-results-stats-source): Internal functions
(setf multi-results-system): Internal functions
(setf multi-results-test-reports): Internal functions
(setf multi-results-tests): Internal functions
(setf multi-results-timestamp): Internal functions
(setf multi-results-warning): Internal functions
(setf name-use-fixtures): Internal functions
(setf name-use-groups): Internal functions
(setf name-use-last-use): Internal functions
(setf name-use-name): Internal functions
(setf name-use-package-p): Internal functions
(setf name-use-tests): Internal functions
(setf object): Internal generic functions
(setf object): Internal generic functions
(setf package-result-elapsed-time): Internal functions
(setf package-result-erring): Internal functions
(setf package-result-failing): Internal functions
(setf package-result-group-results): Internal functions
(setf package-result-package-name): Internal functions
(setf package-result-passing): Internal functions
(setf package-result-tests): Internal functions
(setf package-result-timestamp): Internal functions
(setf package-result-warning): Internal functions
(setf result-stats-elapsed-time): Internal functions
(setf result-stats-erring): Internal functions
(setf result-stats-failing): Internal functions
(setf result-stats-passing): Internal functions
(setf result-stats-tests): Internal functions
(setf result-stats-timestamp): Internal functions
(setf result-stats-warning): Internal functions
(setf test-record): Internal functions
(setf test-record-aspirational): Internal functions
(setf test-record-aspirational-supp): Internal functions
(setf test-record-cleanup): Internal functions
(setf test-record-criterion): Internal functions
(setf test-record-documentation): Internal functions
(setf test-record-finish): Internal functions
(setf test-record-fixtures): Internal functions
(setf test-record-forms): Internal functions
(setf test-record-group): Internal functions
(setf test-record-name): Internal functions
(setf test-record-results): Internal functions
(setf test-record-setup): Internal functions
(setf test-record-special-fixture-names): Internal functions
(setf test-record-startup): Internal functions

A
actual-value: Internal generic functions
actual-value: Internal generic functions
add-error: Exported functions
add-failure: Exported functions
add-info: Exported functions
add-thrown-error: Internal functions
add-warning: Exported generic functions
add-warning: Exported generic functions
add-warning: Exported generic functions
add-warning: Exported generic functions
all-groups-report: Internal functions
all-package-report: Internal functions
all-tests-report: Internal functions
all-uses: Internal functions
apply-check-note-formatter: Internal functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-criterion: Internal generic functions
apply-debug-options: Internal macros
apply-default-debug-options: Internal macros
apply-fixture: Internal functions
apply-fixtures: Internal functions
apply-formatter: Internal functions
arbitrary: Exported generic functions
arbitrary: Exported generic functions
arbitrary: Exported generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-by-spec: Internal generic functions
arbitrary-generable-types: Internal functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
arbitrary-grounded-type: Internal generic functions
artifact-recorder: Internal functions
assemble-protected-option-values: Internal functions
assert-criterion: Exported macros
assert-criterion-fn: Internal functions
assert-eq: Exported functions
assert-eql: Exported functions
assert-equal: Exported functions
assert-equalp: Exported functions
assert-non-nil: Exported functions
assert-not-eq: Exported functions
assert-not-eql: Exported functions
assert-not-equal: Exported functions
assert-not-equalp: Exported functions
assert-null: Exported functions
assert-zero: Exported functions
assertion-failure-args: Internal generic functions
assertion-failure-args: Internal generic functions
assertion-failure-fatal: Internal generic functions
assertion-failure-fatal: Internal generic functions
assertion-failure-formatter: Internal generic functions
assertion-failure-formatter: Internal generic functions
at-verbosity: Internal macros

B
base-name: Internal generic functions
base-name: Internal generic functions
base-name: Internal generic functions
base-name: Internal generic functions
base-name: Internal generic functions
base-name: Internal generic functions
blurb-one-name-use: Internal functions
build-continue-check-expr: Internal functions

C
calibrate-check-result: Internal functions
cdr-or-nil: Internal functions
check-criterion-on-form: Exported functions
check-criterion-on-value: Exported functions
check-note-args: Internal functions
check-note-context: Internal functions
check-note-format: Internal functions
check-note-p: Internal functions
check-note-stack: Internal functions
check-note-type-string: Internal generic functions
check-note-type-string: Internal generic functions
check-note-type-string: Internal generic functions
check-result: Internal macros
check-result-check-name: Internal functions
check-result-elapsed-time: Internal functions
check-result-erring: Internal functions
check-result-errors: Internal functions
check-result-failing: Internal functions
check-result-f