Next: Introduction, Previous: (dir), Up: (dir) [Contents][Index]
The trivial-benchmark Reference Manual
This is the trivial-benchmark Reference Manual, version 2.0.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Aug 15 06:02:26 2022 GMT+0.
Table of Contents
Next: Systems, Previous: The trivial-benchmark Reference Manual, Up: The trivial-benchmark Reference Manual [Contents][Index]
1 Introduction
About Trivial-Benchmark
Frequently I want to do a quick benchmark comparison of my functions. TIME is nice to get some data, but it's limited to a single run so there isn't really much of a statistical value in it. Trivial-Benchmark runs a block of code many times and outputs some statistical data for it. On SBCL this includes the data from TIME, for all other implementations just the REAL- and RUN-TIME data. However, you can extend the system by adding your own metrics to it, or even by adding additional statistical computeations.
Measurement How-To
For basic throwaway benchmarking, the with-timing macro should suffice:
(benchmark:with-timing (1000)
(+ 1 1))
However, you can also do more complex timing using make-timer and with-sampling. The former creates a new timer object (with an optional list of metrics to sample) and the latter collects one sample for each metric of the timer for the duration of the body forms.
(defvar *timer* (benchmark:make-timer))
(benchmark:with-sampling (*timer*)
(+ 1 1))
(benchmark:with-sampling (*timer*)
(expt 10 100))
(benchmark:report *timer*)
(benchmark:reset *timer*)
(benchmark:report *timer*)
Sample Output:
Extending Trivial-Benchmark
If you're interested in adding additional metrics, you'll want to take a look at the metric class, as well as the related methods, start stop discard commit take-sample samples sample-size condense reduce-samples compute report reset. For a basic metric, you only need to implement start, stop, commit, samples, and take-sample. The other functions have standard methods that will do their computations based on those five.
If you have a function that returns a current state of your sample and simply want to have the metric show the delta between start and stop, you can use define-delta-metric.
(define-delta-metric (run-time internal-time-units-per-second)
(get-internal-run-time))
You can also implement a new computation type that is used to display the different metrics in the table. For that, simply write a method on compute with the name of your computation as an eql-specializer and then push the name onto *default-computations* so that it's always displayed when using report. As an example, the maximum is simply calculated as follows:
(defmethod compute ((x (eql :maximum)) (metric metric))
(reduce-samples metric #'max))
Benchmark Suites
Analogous to unit testing, performance testing or benchmark testing is a useful way to measure performance regressions in a code base. Trivial-Benchmark has preliminary support for defining and running benchmarks.
First, a benchmark package needs to be defined, which is essentially a normal Common Lisp package with some additional bookkeeping.
(define-benchmark-package #:my-app-benchmarks
(:use #:my-app)
(:export #:run-benchmarks))
The macro define-benchmark-package automatically uses the COMMON-LISP and the TRIVIAL-BENCHMARK packages.
After the package has been defined, you can define benchmarks like normal Lisp function definitions using the define-benchmark macro. Within the define-benchmark macro, you use with-benchmark-sampling around forms that you wish to measure.
(define-benchmark measure-trig ()
(declare (optimize speed))
(loop :repeat 100000
:for r := (random 1.0d0)
:do (with-benchmark-sampling
(+ (sin r)
(cos r)
(tan r)))))
The macro with-benchmark-sampling can be called many times throughout the body of the define-benchmark, and even called within functions called within the body. In other words, you're able to write something like this:
(defun trig-trial (r)
(with-benchmark-sampling
(+ (sin r) (cos r) (tan r))))
(defun inv-trig-trial (r)
(with-benchmark-sampling
(+ (asin r) (acos r) (atan r))))
(define-benchmark measure-all-trig ()
(declare (optimize speed))
(loop :repeat 100000
:for r := (random 1.0d0)
:do (trig-trial r)
(inv-trig-trial r)))
You can run the benchmarks of a package by doing:
(run-package-benchmarks :package ':my-app-benchmarks
:verbose t)
You can elide the :package argument if you're in the package already. Running this will give:
> (run-package-benchmarks :package ':my-app-benchmarks
:verbose t)
Benchmarking MEASURE-ALL-TRIG...done.
Benchmarking BENCH-TRIG...done.
#<HASH-TABLE :TEST EQ :COUNT 2 {1002FD8243}>
The returned hash table contains a summary of all of the statistics, which you can further process as needed (e.g., record, plot, etc.). The table maps the benchmark names to an alist of metrics and their computed statistics. We can inspect this easily enough:
> (alexandria:hash-table-alist *)
((BENCH-TRIG
(REAL-TIME :SAMPLES 100000 :TOTAL 52/125 :MINIMUM 0 :MAXIMUM 3/1000 :MEDIAN 0
:AVERAGE 13/3125000 :DEVIATION 6.482819e-5)
(RUN-TIME :SAMPLES 100000 :TOTAL 383/1000 :MINIMUM 0 :MAXIMUM 1/1000 :MEDIAN
0 :AVERAGE 383/100000000 :DEVIATION 6.176837e-5)
(USER-RUN-TIME :SAMPLES 100000 :TOTAL 52663/250000 :MINIMUM 1/1000000
:MAXIMUM 53/1000000 :MEDIAN 1/500000 :AVERAGE 52663/25000000000 :DEVIATION
9.770944e-7)
(SYSTEM-RUN-TIME :SAMPLES 100000 :TOTAL 94299/500000 :MINIMUM 1/1000000
:MAXIMUM 87/1000000 :MEDIAN 1/500000 :AVERAGE 94299/50000000000 :DEVIATION
1.1286627e-6)
(PAGE-FAULTS :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
:AVERAGE 0 :DEVIATION 0.0)
(GC-RUN-TIME :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
:AVERAGE 0 :DEVIATION 0.0)
(BYTES-CONSED :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
:AVERAGE 0 :DEVIATION 0.0)
(EVAL-CALLS :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0 :AVERAGE
0 :DEVIATION 0.0))
(MEASURE-ALL-TRIG
(REAL-TIME :SAMPLES 200000 :TOTAL 102/125 :MINIMUM 0 :MAXIMUM 1/200 :MEDIAN 0
:AVERAGE 51/12500000 :DEVIATION 6.4601496e-5)
(RUN-TIME :SAMPLES 200000 :TOTAL 77/100 :MINIMUM 0 :MAXIMUM 1/1000 :MEDIAN 0
:AVERAGE 77/20000000 :DEVIATION 6.192881e-5)
(USER-RUN-TIME :SAMPLES 200000 :TOTAL 41371/100000 :MINIMUM 1/1000000
:MAXIMUM 1/12500 :MEDIAN 1/500000 :AVERAGE 41371/20000000000 :DEVIATION
1.2130914e-6)
(SYSTEM-RUN-TIME :SAMPLES 200000 :TOTAL 363543/1000000 :MINIMUM 1/1000000
:MAXIMUM 11/100000 :MEDIAN 1/500000 :AVERAGE 363543/200000000000 :DEVIATION
1.4713012e-6)
(PAGE-FAULTS :SAMPLES 200000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
:AVERAGE 0 :DEVIATION 0.0)
(GC-RUN-TIME :SAMPLES 200000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
:AVERAGE 0 :DEVIATION 0.0)
(BYTES-CONSED :SAMPLES 200000 :TOTAL 15106048 :MINIMUM 0 :MAXIMUM 131072
:MEDIAN 0 :AVERAGE 236032/3125 :DEVIATION 1595.1276)
(EVAL-CALLS :SAMPLES 200000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0 :AVERAGE
0 :DEVIATION 0.0)))
Next: Files, Previous: Introduction, Up: The trivial-benchmark Reference Manual [Contents][Index]
2 Systems
The main system appears first, followed by any subsystem dependency.
2.1 trivial-benchmark
An easy to use benchmarking system.
- Maintainer
Nicolas Hafner <shinmera@tymoon.eu>
- Author
Nicolas Hafner <shinmera@tymoon.eu>
- Home Page
- Source Control
(GIT https://github.com/Shinmera/trivial-benchmark.git)
- Bug Tracker
- License
zlib
- Version
2.0.0
- Dependency
alexandria (system).
- Source
- Child Components
- package.lisp (file).
- toolkit.lisp (file).
- timer.lisp (file).
- samples.lisp (file).
- suite.lisp (file).
Next: Packages, Previous: Systems, Up: The trivial-benchmark Reference Manual [Contents][Index]
3 Files
Files are sorted by type and then listed depth-first from the systems components trees.
3.1 Lisp
- trivial-benchmark/trivial-benchmark.asd
- trivial-benchmark/package.lisp
- trivial-benchmark/toolkit.lisp
- trivial-benchmark/timer.lisp
- trivial-benchmark/samples.lisp
- trivial-benchmark/suite.lisp
Next: trivial-benchmark/package.lisp, Previous: Lisp, Up: Lisp [Contents][Index]
3.1.1 trivial-benchmark/trivial-benchmark.asd
- Source
- Parent Component
trivial-benchmark (system).
- ASDF Systems
Next: trivial-benchmark/toolkit.lisp, Previous: trivial-benchmark/trivial-benchmark.asd, Up: Lisp [Contents][Index]
3.1.2 trivial-benchmark/package.lisp
- Source
- Parent Component
trivial-benchmark (system).
- Packages
Next: trivial-benchmark/timer.lisp, Previous: trivial-benchmark/package.lisp, Up: Lisp [Contents][Index]
3.1.3 trivial-benchmark/toolkit.lisp
- Dependency
package.lisp (file).
- Source
- Parent Component
trivial-benchmark (system).
- Public Interface
- print-table (function).
- round-to (function).
- type= (function).
Next: trivial-benchmark/samples.lisp, Previous: trivial-benchmark/toolkit.lisp, Up: Lisp [Contents][Index]
3.1.4 trivial-benchmark/timer.lisp
- Dependency
toolkit.lisp (file).
- Source
- Parent Component
trivial-benchmark (system).
- Public Interface
- *default-computations* (special variable).
- *default-metrics* (special variable).
- commit (generic function).
- compute (generic function).
- condense (generic function).
- describe-object (method).
- discard (generic function).
- do-metrics (macro).
- initialize-instance (method).
- make-timer (function).
- map-metrics (function).
- metric (generic function).
- (setf metric) (generic function).
- metric (class).
- metric-types (generic function).
- metrics (generic reader).
- (setf metrics) (writer method).
- print-object (method).
- print-object (method).
- reduce-samples (generic function).
- report (generic function).
- reset (generic function).
- running (generic reader).
- (setf running) (writer method).
- sample-size (generic function).
- samples (generic reader).
- start (generic function).
- take-sample (generic function).
- timer (class).
- with-sampling (macro).
- with-timing (macro).
- Internals
- format-timer-stats (function).
- report-to (generic function).
- stop (generic function).
Next: trivial-benchmark/suite.lisp, Previous: trivial-benchmark/timer.lisp, Up: Lisp [Contents][Index]
3.1.5 trivial-benchmark/samples.lisp
- Dependency
timer.lisp (file).
- Source
- Parent Component
trivial-benchmark (system).
- Public Interface
- bytes-consed (class).
- commit (method).
- commit (method).
- cpu-cycles (class).
- define-delta-metric (macro).
- delta-metric (class).
- discard (method).
- discard (method).
- eval-calls (class).
- gc-run-time (class).
- page-faults (class).
- real-time (class).
- reset (method).
- reset (method).
- run-time (class).
- samples (reader method).
- (setf samples) (writer method).
- start (method).
- start (method).
- starting-value (generic reader).
- (setf starting-value) (writer method).
- system-run-time (class).
- take-sample (method).
- take-sample (method).
- take-sample (method).
- take-sample (method).
- take-sample (method).
- take-sample (method).
- take-sample (method).
- take-sample (method).
- user-run-time (class).
- vector-metric (class).
- Internals
- cpu-cycles-h0 (reader method).
- (setf cpu-cycles-h0) (writer method).
- cpu-cycles-h1 (reader method).
- (setf cpu-cycles-h1) (writer method).
- cpu-cycles-l0 (reader method).
- (setf cpu-cycles-l0) (writer method).
- cpu-cycles-l1 (reader method).
- (setf cpu-cycles-l1) (writer method).
- stop (method).
- stop (method).
- stopping-value (reader method).
- (setf stopping-value) (writer method).
- units (reader method).
- units (reader method).
- units (reader method).
- units (reader method).
- units (reader method).
- units (reader method).
- units (reader method).
- units (reader method).
- units (reader method).
- (setf units) (writer method).
- (setf units) (writer method).
- (setf units) (writer method).
- (setf units) (writer method).
- (setf units) (writer method).
- (setf units) (writer method).
- (setf units) (writer method).
- (setf units) (writer method).
- (setf units) (writer method).
Previous: trivial-benchmark/samples.lisp, Up: Lisp [Contents][Index]
3.1.6 trivial-benchmark/suite.lisp
- Dependency
samples.lisp (file).
- Source
- Parent Component
trivial-benchmark (system).
- Public Interface
- define-benchmark (macro).
- define-benchmark-package (macro).
- find-benchmark-package (function).
- run-package-benchmarks (function).
- with-benchmark-sampling (macro).
- with-suite-report (macro).
- Internals
- *benchmark-packages* (special variable).
- *current-benchmark* (special variable).
- *current-suite-report* (special variable).
- *current-timer* (special variable).
- make-suite-report-table (function).
- metrics-alist (function).
- print-verbose-after-message (function).
- print-verbose-before-message (function).
- report-to (method).
- run-one-benchmark (function).
Next: Definitions, Previous: Files, Up: The trivial-benchmark Reference Manual [Contents][Index]
4 Packages
Packages are listed by definition order.
4.1 trivial-benchmark
- Source
- Nicknames
- benchmark
- org.shirakumo.trivial-benchmark
- Use List
common-lisp.
- Public Interface
- *default-computations* (special variable).
- *default-metrics* (special variable).
- bytes-consed (class).
- commit (generic function).
- compute (generic function).
- condense (generic function).
- cpu-cycles (class).
- define-benchmark (macro).
- define-benchmark-package (macro).
- define-delta-metric (macro).
- delta-metric (class).
- discard (generic function).
- do-metrics (macro).
- eval-calls (class).
- find-benchmark-package (function).
- gc-run-time (class).
- make-timer (function).
- map-metrics (function).
- metric (generic function).
- (setf metric) (generic function).
- metric (class).
- metric-types (generic function).
- metrics (generic reader).
- (setf metrics) (generic writer).
- page-faults (class).
- print-table (function).
- real-time (class).
- reduce-samples (generic function).
- report (generic function).
- reset (generic function).
- round-to (function).
- run-package-benchmarks (function).
- run-time (class).
- running (generic reader).
- (setf running) (generic writer).
- sample-size (generic function).
- samples (generic reader).
- (setf samples) (generic writer).
- start (generic function).
- starting-value (generic reader).
- (setf starting-value) (generic writer).
- system-run-time (class).
- take-sample (generic function).
- timer (class).
- type= (function).
- user-run-time (class).
- vector-metric (class).
- with-benchmark-sampling (macro).
- with-sampling (macro).
- with-suite-report (macro).
- with-timing (macro).
- Internals
- *benchmark-packages* (special variable).
- *current-benchmark* (special variable).
- *current-suite-report* (special variable).
- *current-timer* (special variable).
- cpu-cycles-h0 (generic reader).
- (setf cpu-cycles-h0) (generic writer).
- cpu-cycles-h1 (generic reader).
- (setf cpu-cycles-h1) (generic writer).
- cpu-cycles-l0 (generic reader).
- (setf cpu-cycles-l0) (generic writer).
- cpu-cycles-l1 (generic reader).
- (setf cpu-cycles-l1) (generic writer).
- format-timer-stats (function).
- make-suite-report-table (function).
- metrics-alist (function).
- print-verbose-after-message (function).
- print-verbose-before-message (function).
- report-to (generic function).
- run-one-benchmark (function).
- stop (generic function).
- stopping-value (generic reader).
- (setf stopping-value) (generic writer).
- units (generic reader).
- (setf units) (generic writer).
Next: Indexes, Previous: Packages, Up: The trivial-benchmark Reference Manual [Contents][Index]
5 Definitions
Definitions are sorted by export status, category, package, and then by lexicographic order.
Next: Internals, Previous: Definitions, Up: Definitions [Contents][Index]
5.1 Public Interface
Next: Macros, Previous: Public Interface, Up: Public Interface [Contents][Index]
5.1.1 Special variables
- Special Variable: *default-computations* ¶
-
The list of computation-names used to print the REPORT table.
- Package
- Source
- Special Variable: *default-metrics* ¶
-
The list of class-names used to populate a TIMER by default.
- Package
- Source
Next: Ordinary functions, Previous: Special variables, Up: Public Interface [Contents][Index]
5.1.2 Macros
- Macro: define-benchmark (name args &body body) ¶
-
Define a benchmark named NAME. The structure of this macro is that of DEFUN, except that WITH-BENCHMARK-SAMPLING can be called to collect metrics on the enclosed forms.
ARGS must be a lambda list that does not require arguments present at the call site.
- Package
- Source
- Macro: define-benchmark-package (name &body package-options) ¶
-
Define a package as if by DEFPACKAGE, except that both the COMMON-LISP and TRIVIAL-BENCHMARK packages are used, and that this package is designated as one containing benchmarks.
- Package
- Source
- Macro: define-delta-metric (name &body sample-point-forms) ¶
-
Shortcut to define a DELTA-METRIC.
The SAMPLE-POINT-FORMS should return a number to use to calculate a delta. NAME can be either just a NAME or a list of the form (NAME UNITS) where UNITS is the number the sample points are divided by when saving them. For example, if you sample a function of milliseconds, but want to report in seconds, set UNITS to 1000. Doing this instead of dividing directly in the form avoids potential consing when running into bignums.
See DELTA-METRIC
- Package
- Source
- Macro: do-metrics ((metric-var timer &optional result-form) &body forms) ¶
-
Binds METRIC-VAR to each metric of TIMER and then evaluates FORMS. Returns the value of RESULT-FORM after the loop.
- Package
- Source
- Macro: with-benchmark-sampling (&body forms) ¶
-
Benchmark the execution of the forms FORMS within the context of a DEFINE-BENCH macro. Acts like PROGN.
- Package
- Source
- Macro: with-sampling ((timer-form) &body forms) ¶
-
Takes a sample for the evaluation time of FORMS and stores it in the timer given by TIMER-FORM. Acts like a PROGN.
Specifically, START is called, then FORMS are evaluated. If an error occurs within the body, DISCARD is called on the timer, otherwise COMMIT is called once the body exits.
See START
See DISCARD
See COMMIT- Package
- Source
- Macro: with-suite-report ((suite) &body body) ¶
-
- Package
- Source
- Macro: with-timing ((n &optional timer-form stream computations) &body forms) ¶
-
Evaluates FORMS N times, using WITH-SAMPLING on the value of TIMER-FORM each iteration. At the end, runs REPORT on the timer with STREAM and COMPUTATIONS.
See WITH-SAMPLING
- Package
- Source
Next: Generic functions, Previous: Macros, Up: Public Interface [Contents][Index]
5.1.3 Ordinary functions
- Function: find-benchmark-package (package-designator) ¶
-
Find the benchmark package designated by PACKAGE-DESIGNATOR and return it, otherwise return NIL if it doesn’t exist.
- Package
- Source
- Function: make-timer (&optional metric-types) ¶
-
Creates a TIMER object using the given METRIC-TYPES
- Package
- Source
- Function: map-metrics (timer function) ¶
-
Maps the metrics in TIMER, calling FUNCTION with each.
- Package
- Source
- Function: print-table (table &key stream padding) ¶
-
Prints a table (each list in TABLE being a row) with proper spacing.
- Package
- Source
- Function: round-to (num n) ¶
-
Rounds NUM to N digits after the dot.
- Package
- Source
- Function: run-package-benchmarks (&key package verbose) ¶
-
Run all of the benchmarks associated with the benchmark package PACKAGE, the current one by default.
Return a hash table mapping benchmark names to their metrics represented as lists. In particular, it will be an alist whose CARs are metrics and whose CDRs are plists of computations.
VERBOSE is a generalized boolean that controls output verbosity while testing.
- Package
- Source
- Function: type= (a b) ¶
-
Returns T if A and B denote the same type (are subtypes of one another)
If one of the arguments does not denote a type, the result of TYPE-OF is used in their place.- Package
- Source
Next: Standalone methods, Previous: Ordinary functions, Up: Public Interface [Contents][Index]
5.1.4 Generic functions
- Generic Function: commit (metric) ¶
-
Commit the current sample of METRIC. If the metric is running, call STOP first.
- Package
- Source
- Methods
- Method: commit ((metric cpu-cycles)) ¶
-
- Source
- Method: commit ((metric delta-metric)) ¶
-
- Source
- Generic Function: compute (thing metric) ¶
-
Compute a value of the statistical computation THING for METRIC based on its current samples.
- Package
- Source
- Methods
- Generic Function: condense (sample) ¶
-
Turn the SAMPLE value into a usable number.
- Package
- Source
- Methods
- Method: condense (thing) ¶
- Generic Function: discard (metric) ¶
-
Discard the current sample of METRIC. If the metric is running, call STOP first.
- Package
- Source
- Methods
- Method: discard ((metric cpu-cycles)) ¶
-
- Source
- Method: discard ((metric delta-metric)) ¶
-
- Source
- Generic Function: metric (type timer) ¶
-
Returns the metric of TYPE in TIMER if any. The metric must match the type by TYPE=
- Package
- Source
- Methods
- Generic Function: (setf metric) (timer) ¶
-
Sets the METRIC in TIMER.
The metric is replaced if it is found in the timer by TYPE= comparison.- Package
- Source
- Methods
- Generic Function: metric-types (timer) ¶
-
Returns the types of metrics in TIMER.
- Package
- Source
- Methods
- Generic Reader: metrics (timer) ¶
-
Returns a list of metrics stored in TIMER.
- Package
- Source
- Methods
- Generic Writer: (setf metrics) (object) ¶
- Generic Function: reduce-samples (metric function) ¶
-
Apply FUNCTION to the samples stored in METRIC in a REDUCE fashion.
- Package
- Source
- Methods
- Method: reduce-samples (metric function) ¶
- Generic Function: report (thing &key stream computations) ¶
-
Print a report of all COMPUTATIONS for THING to STREAM
STREAM can be one of the following:
T — Print to *standard-output*
NIL — Print to a string and return it. STREAM — Print to the stream- Package
- Source
- Methods
- Method: report (thing &key stream computations) ¶
- Generic Function: reset (metric) ¶
-
Reset the METRIC and remove all its samples.
- Package
- Source
- Methods
- Method: reset :after ((metric vector-metric)) ¶
-
- Source
- Method: reset ((metric vector-metric)) ¶
-
- Source
- Generic Reader: running (metric) ¶
-
Returns T if the metric is currently sampling.
See START
See STOP- Package
- Source
- Methods
- Generic Writer: (setf running) (object) ¶
- Generic Function: sample-size (metric) ¶
-
Return the number of samples stored in METRIC.
- Package
- Source
- Methods
- Method: sample-size (metric) ¶
- Generic Reader: samples (metric) ¶
-
Return a sequence of committed samples stored in METRIC.
- Package
- Source
- Methods
- Reader Method: samples ((vector-metric vector-metric)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Generic Writer: (setf samples) (object) ¶
-
- Package
- Methods
- Writer Method: (setf samples) ((vector-metric vector-metric)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Generic Function: start (metric) ¶
-
Begin a sample for METRIC. Sets RUNNING to T.
- Package
- Source
- Methods
- Method: start ((metric cpu-cycles)) ¶
-
- Source
- Method: start ((metric delta-metric)) ¶
-
- Source
- Generic Reader: starting-value (delta-metric) ¶
-
Returns the value stored when START was called on the DELTA-METRIC.
- Package
- Source
- Methods
- Reader Method: starting-value ((delta-metric delta-metric)) ¶
-
automatically generated reader method
- Target Slot
- Generic Writer: (setf starting-value) (object) ¶
-
- Package
- Methods
- Writer Method: (setf starting-value) ((delta-metric delta-metric)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Generic Function: take-sample (metric) ¶
-
Return a current sampling value for METRIC.
Note that not all metrics must implement a method for this function. It is perfectly plausible for a metric to skip this method if it cannot provide a sample value at any point in time.
- Package
- Source
- Methods
- Method: take-sample ((metric eval-calls)) ¶
-
Samples SB-IMPL::*EVAL-CALLS*.
- Source
- Method: take-sample ((metric bytes-consed)) ¶
-
Samples SB-IMPL::GET-BYTES-CONSED.
- Source
- Method: take-sample ((metric gc-run-time)) ¶
-
Samples SB-IMPL::*GC-RUN-TIME* in seconds.
- Source
- Method: take-sample ((metric page-faults)) ¶
-
Samples the third value (page-faults) of SB-SYS:GET-SYSTEM-INFO in seconds.
- Source
- Method: take-sample ((metric system-run-time)) ¶
-
Samples the second value (system-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.
- Source
- Method: take-sample ((metric user-run-time)) ¶
-
Samples the first value (user-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.
- Source
- Method: take-sample ((metric run-time)) ¶
-
Samples the results of GET-INTERNAL-RUN-TIME in seconds.
- Source
Next: Classes, Previous: Generic functions, Up: Public Interface [Contents][Index]
5.1.5 Standalone methods
Previous: Standalone methods, Up: Public Interface [Contents][Index]
5.1.6 Classes
- Class: bytes-consed ¶
-
Samples SB-IMPL::GET-BYTES-CONSED.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
1
- Readers
- Writers
- Class: cpu-cycles ¶
-
Samples SB-IMPL::ELAPSED-CYCLES.
- Class: delta-metric ¶
-
A LISTED-METRIC that calculates a sample point according to the delta between the START and COMMIT. Sub-classes of this must implement the TAKE-SAMPLE method.
See LISTED-METRIC
- Class: eval-calls ¶
-
Samples SB-IMPL::*EVAL-CALLS*.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
1
- Readers
- Writers
- Class: gc-run-time ¶
-
Samples SB-IMPL::*GC-RUN-TIME* in seconds.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
1000
- Readers
- Writers
- Class: metric ¶
-
A class container for sampling information.
- Package
- Source
- Direct subclasses
- Direct methods
- Direct slots
- Slot: running ¶
-
- Initargs
:running
- Readers
- Writers
- Class: page-faults ¶
-
Samples the third value (page-faults) of SB-SYS:GET-SYSTEM-INFO in seconds.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
1
- Readers
- Writers
- Class: real-time ¶
-
Samples the results of GET-INTERNAL-REAL-TIME in seconds.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
internal-time-units-per-second
- Readers
- Writers
- Class: run-time ¶
-
Samples the results of GET-INTERNAL-RUN-TIME in seconds.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
internal-time-units-per-second
- Readers
- Writers
- Class: system-run-time ¶
-
Samples the second value (system-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
1000000
- Readers
- Writers
- Class: timer ¶
-
Class container for a set of METRICS.
- Package
- Source
- Direct methods
- Direct slots
- Slot: metrics ¶
-
- Initargs
:metrics
- Readers
- Writers
- Class: user-run-time ¶
-
Samples the first value (user-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.
- Package
- Source
- Direct superclasses
- Direct methods
- Direct slots
- Slot: units ¶
-
- Initform
1000000
- Readers
- Writers
- Class: vector-metric ¶
-
A METRIC that implements its sample storage as a vector. SAMPLES is SETF-able for this class.
See METRIC
- Package
- Source
- Direct superclasses
- Direct subclasses
- Direct methods
- Direct slots
- Slot: samples ¶
-
- Initform
(make-array 1000 :adjustable t :fill-pointer 0)
- Initargs
:samples
- Readers
- Writers
Previous: Public Interface, Up: Definitions [Contents][Index]
5.2 Internals
Next: Ordinary functions, Previous: Internals, Up: Internals [Contents][Index]
5.2.1 Special variables
- Special Variable: *benchmark-packages* ¶
-
A table mapping package objects (which are benchmark packages) to a list of fbound symbols.
- Package
- Source
- Special Variable: *current-benchmark* ¶
-
- Package
- Source
- Special Variable: *current-suite-report* ¶
-
- Package
- Source
- Special Variable: *current-timer* ¶
-
- Package
- Source
Next: Generic functions, Previous: Special variables, Up: Internals [Contents][Index]
5.2.2 Ordinary functions
- Function: format-timer-stats (stream timer &optional computations) ¶
-
- Package
- Source
- Function: make-suite-report-table () ¶
-
- Package
- Source
- Function: metrics-alist (timer &optional computations) ¶
-
- Package
- Source
- Function: print-verbose-after-message (fn &optional out report) ¶
-
- Package
- Source
- Function: print-verbose-before-message (fn &optional out) ¶
-
- Package
- Source
- Function: run-one-benchmark (fn &optional verbose) ¶
-
- Package
- Source
Previous: Ordinary functions, Up: Internals [Contents][Index]
5.2.3 Generic functions
- Generic Reader: cpu-cycles-h0 (object) ¶
-
- Package
- Methods
- Reader Method: cpu-cycles-h0 ((cpu-cycles cpu-cycles)) ¶
-
automatically generated reader method
- Source
- Target Slot
h0.
- Generic Writer: (setf cpu-cycles-h0) (object) ¶
-
- Package
- Methods
- Writer Method: (setf cpu-cycles-h0) ((cpu-cycles cpu-cycles)) ¶
-
automatically generated writer method
- Source
- Target Slot
h0.
- Generic Reader: cpu-cycles-h1 (object) ¶
-
- Package
- Methods
- Reader Method: cpu-cycles-h1 ((cpu-cycles cpu-cycles)) ¶
-
automatically generated reader method
- Source
- Target Slot
h1.
- Generic Writer: (setf cpu-cycles-h1) (object) ¶
-
- Package
- Methods
- Writer Method: (setf cpu-cycles-h1) ((cpu-cycles cpu-cycles)) ¶
-
automatically generated writer method
- Source
- Target Slot
h1.
- Generic Reader: cpu-cycles-l0 (object) ¶
-
- Package
- Methods
- Reader Method: cpu-cycles-l0 ((cpu-cycles cpu-cycles)) ¶
-
automatically generated reader method
- Source
- Target Slot
l0.
- Generic Writer: (setf cpu-cycles-l0) (object) ¶
-
- Package
- Methods
- Writer Method: (setf cpu-cycles-l0) ((cpu-cycles cpu-cycles)) ¶
-
automatically generated writer method
- Source
- Target Slot
l0.
- Generic Reader: cpu-cycles-l1 (object) ¶
-
- Package
- Methods
- Reader Method: cpu-cycles-l1 ((cpu-cycles cpu-cycles)) ¶
-
automatically generated reader method
- Source
- Target Slot
l1.
- Generic Writer: (setf cpu-cycles-l1) (object) ¶
-
- Package
- Methods
- Writer Method: (setf cpu-cycles-l1) ((cpu-cycles cpu-cycles)) ¶
-
automatically generated writer method
- Source
- Target Slot
l1.
- Generic Function: report-to (stream thing &key computations) ¶
- Generic Function: stop (metric) ¶
-
Stop the sample for METRIC. Sets RUNNING to NIL.
- Package
- Source
- Methods
- Method: stop ((metric cpu-cycles)) ¶
-
- Source
- Method: stop ((metric delta-metric)) ¶
-
- Source
- Generic Reader: stopping-value (object) ¶
-
- Package
- Methods
- Reader Method: stopping-value ((delta-metric delta-metric)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Generic Writer: (setf stopping-value) (object) ¶
-
- Package
- Methods
- Writer Method: (setf stopping-value) ((delta-metric delta-metric)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Generic Reader: units (object) ¶
-
- Package
- Methods
- Reader Method: units ((eval-calls eval-calls)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((bytes-consed bytes-consed)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((gc-run-time gc-run-time)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((page-faults page-faults)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((system-run-time system-run-time)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((user-run-time user-run-time)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((run-time run-time)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((real-time real-time)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Reader Method: units ((delta-metric delta-metric)) ¶
-
automatically generated reader method
- Source
- Target Slot
- Generic Writer: (setf units) (object) ¶
-
- Package
- Methods
- Writer Method: (setf units) ((eval-calls eval-calls)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((bytes-consed bytes-consed)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((gc-run-time gc-run-time)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((page-faults page-faults)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((system-run-time system-run-time)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((user-run-time user-run-time)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((run-time run-time)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((real-time real-time)) ¶
-
automatically generated writer method
- Source
- Target Slot
- Writer Method: (setf units) ((delta-metric delta-metric)) ¶
-
automatically generated writer method
- Source
- Target Slot
Previous: Definitions, Up: The trivial-benchmark Reference Manual [Contents][Index]
Appendix A Indexes
A.2 Functions
| Jump to: | (
C D F G I M P R S T U W |
|---|
| Jump to: | (
C D F G I M P R S T U W |
|---|
Next: Data types, Previous: Functions, Up: Indexes [Contents][Index]
A.3 Variables
| Jump to: | *
H L M R S U |
|---|
| Jump to: | *
H L M R S U |
|---|
A.4 Data types
| Jump to: | B C D E F G M P R S T U V |
|---|
| Jump to: | B C D E F G M P R S T U V |
|---|