The trivial-benchmark Reference Manual

Next: , Previous: , Up: (dir)   [Contents][Index]

The trivial-benchmark Reference Manual

This is the trivial-benchmark Reference Manual, version 2.0.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Aug 15 06:02:26 2022 GMT+0.

Table of Contents


1 Introduction

About Trivial-Benchmark

Frequently I want to do a quick benchmark comparison of my functions. TIME is nice to get some data, but it's limited to a single run so there isn't really much of a statistical value in it. Trivial-Benchmark runs a block of code many times and outputs some statistical data for it. On SBCL this includes the data from TIME, for all other implementations just the REAL- and RUN-TIME data. However, you can extend the system by adding your own metrics to it, or even by adding additional statistical computeations.

Measurement How-To

For basic throwaway benchmarking, the with-timing macro should suffice:

(benchmark:with-timing (1000)
  (+ 1 1))

However, you can also do more complex timing using make-timer and with-sampling. The former creates a new timer object (with an optional list of metrics to sample) and the latter collects one sample for each metric of the timer for the duration of the body forms.

(defvar *timer* (benchmark:make-timer))

(benchmark:with-sampling (*timer*)
  (+ 1 1))

(benchmark:with-sampling (*timer*)
  (expt 10 100))

(benchmark:report *timer*)

(benchmark:reset *timer*)

(benchmark:report *timer*)

Sample Output:
Output Screenshot

Extending Trivial-Benchmark

If you're interested in adding additional metrics, you'll want to take a look at the metric class, as well as the related methods, start stop discard commit take-sample samples sample-size condense reduce-samples compute report reset. For a basic metric, you only need to implement start, stop, commit, samples, and take-sample. The other functions have standard methods that will do their computations based on those five.

If you have a function that returns a current state of your sample and simply want to have the metric show the delta between start and stop, you can use define-delta-metric.

(define-delta-metric (run-time internal-time-units-per-second)
  (get-internal-run-time))

You can also implement a new computation type that is used to display the different metrics in the table. For that, simply write a method on compute with the name of your computation as an eql-specializer and then push the name onto *default-computations* so that it's always displayed when using report. As an example, the maximum is simply calculated as follows:

(defmethod compute ((x (eql :maximum)) (metric metric))
  (reduce-samples metric #'max))

Benchmark Suites

Analogous to unit testing, performance testing or benchmark testing is a useful way to measure performance regressions in a code base. Trivial-Benchmark has preliminary support for defining and running benchmarks.

First, a benchmark package needs to be defined, which is essentially a normal Common Lisp package with some additional bookkeeping.

(define-benchmark-package #:my-app-benchmarks
  (:use #:my-app)
  (:export #:run-benchmarks))

The macro define-benchmark-package automatically uses the COMMON-LISP and the TRIVIAL-BENCHMARK packages.

After the package has been defined, you can define benchmarks like normal Lisp function definitions using the define-benchmark macro. Within the define-benchmark macro, you use with-benchmark-sampling around forms that you wish to measure.

(define-benchmark measure-trig ()
  (declare (optimize speed))
  (loop :repeat 100000
        :for r := (random 1.0d0)
        :do (with-benchmark-sampling
              (+ (sin r)
                 (cos r)
                 (tan r)))))

The macro with-benchmark-sampling can be called many times throughout the body of the define-benchmark, and even called within functions called within the body. In other words, you're able to write something like this:

(defun trig-trial (r)
  (with-benchmark-sampling
    (+ (sin r) (cos r) (tan r))))

(defun inv-trig-trial (r)
  (with-benchmark-sampling
    (+ (asin r) (acos r) (atan r))))

(define-benchmark measure-all-trig ()
  (declare (optimize speed))
  (loop :repeat 100000
        :for r := (random 1.0d0)
        :do (trig-trial r)
            (inv-trig-trial r)))

You can run the benchmarks of a package by doing:

(run-package-benchmarks :package ':my-app-benchmarks
                        :verbose t)

You can elide the :package argument if you're in the package already. Running this will give:

> (run-package-benchmarks :package ':my-app-benchmarks 
                          :verbose t)
Benchmarking MEASURE-ALL-TRIG...done.
Benchmarking BENCH-TRIG...done.
#<HASH-TABLE :TEST EQ :COUNT 2 {1002FD8243}>

The returned hash table contains a summary of all of the statistics, which you can further process as needed (e.g., record, plot, etc.). The table maps the benchmark names to an alist of metrics and their computed statistics. We can inspect this easily enough:

> (alexandria:hash-table-alist *)
((BENCH-TRIG
  (REAL-TIME :SAMPLES 100000 :TOTAL 52/125 :MINIMUM 0 :MAXIMUM 3/1000 :MEDIAN 0
   :AVERAGE 13/3125000 :DEVIATION 6.482819e-5)
  (RUN-TIME :SAMPLES 100000 :TOTAL 383/1000 :MINIMUM 0 :MAXIMUM 1/1000 :MEDIAN
   0 :AVERAGE 383/100000000 :DEVIATION 6.176837e-5)
  (USER-RUN-TIME :SAMPLES 100000 :TOTAL 52663/250000 :MINIMUM 1/1000000
   :MAXIMUM 53/1000000 :MEDIAN 1/500000 :AVERAGE 52663/25000000000 :DEVIATION
   9.770944e-7)
  (SYSTEM-RUN-TIME :SAMPLES 100000 :TOTAL 94299/500000 :MINIMUM 1/1000000
   :MAXIMUM 87/1000000 :MEDIAN 1/500000 :AVERAGE 94299/50000000000 :DEVIATION
   1.1286627e-6)
  (PAGE-FAULTS :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
   :AVERAGE 0 :DEVIATION 0.0)
  (GC-RUN-TIME :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
   :AVERAGE 0 :DEVIATION 0.0)
  (BYTES-CONSED :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
   :AVERAGE 0 :DEVIATION 0.0)
  (EVAL-CALLS :SAMPLES 100000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0 :AVERAGE
   0 :DEVIATION 0.0))
 (MEASURE-ALL-TRIG
  (REAL-TIME :SAMPLES 200000 :TOTAL 102/125 :MINIMUM 0 :MAXIMUM 1/200 :MEDIAN 0
   :AVERAGE 51/12500000 :DEVIATION 6.4601496e-5)
  (RUN-TIME :SAMPLES 200000 :TOTAL 77/100 :MINIMUM 0 :MAXIMUM 1/1000 :MEDIAN 0
   :AVERAGE 77/20000000 :DEVIATION 6.192881e-5)
  (USER-RUN-TIME :SAMPLES 200000 :TOTAL 41371/100000 :MINIMUM 1/1000000
   :MAXIMUM 1/12500 :MEDIAN 1/500000 :AVERAGE 41371/20000000000 :DEVIATION
   1.2130914e-6)
  (SYSTEM-RUN-TIME :SAMPLES 200000 :TOTAL 363543/1000000 :MINIMUM 1/1000000
   :MAXIMUM 11/100000 :MEDIAN 1/500000 :AVERAGE 363543/200000000000 :DEVIATION
   1.4713012e-6)
  (PAGE-FAULTS :SAMPLES 200000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
   :AVERAGE 0 :DEVIATION 0.0)
  (GC-RUN-TIME :SAMPLES 200000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0
   :AVERAGE 0 :DEVIATION 0.0)
  (BYTES-CONSED :SAMPLES 200000 :TOTAL 15106048 :MINIMUM 0 :MAXIMUM 131072
   :MEDIAN 0 :AVERAGE 236032/3125 :DEVIATION 1595.1276)
  (EVAL-CALLS :SAMPLES 200000 :TOTAL 0 :MINIMUM 0 :MAXIMUM 0 :MEDIAN 0 :AVERAGE
   0 :DEVIATION 0.0)))

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 trivial-benchmark

An easy to use benchmarking system.

Maintainer

Nicolas Hafner <shinmera@tymoon.eu>

Author

Nicolas Hafner <shinmera@tymoon.eu>

Home Page

https://shinmera.github.io/trivial-benchmark/

Source Control

(GIT https://github.com/Shinmera/trivial-benchmark.git)

Bug Tracker

https://github.com/Shinmera/trivial-benchmark/issues

License

zlib

Version

2.0.0

Dependency

alexandria (system).

Source

trivial-benchmark.asd.

Child Components

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp   [Contents][Index]

3.1.1 trivial-benchmark/trivial-benchmark.asd

Source

trivial-benchmark.asd.

Parent Component

trivial-benchmark (system).

ASDF Systems

trivial-benchmark.


3.1.2 trivial-benchmark/package.lisp

Source

trivial-benchmark.asd.

Parent Component

trivial-benchmark (system).

Packages

trivial-benchmark.


3.1.3 trivial-benchmark/toolkit.lisp

Dependency

package.lisp (file).

Source

trivial-benchmark.asd.

Parent Component

trivial-benchmark (system).

Public Interface

3.1.4 trivial-benchmark/timer.lisp

Dependency

toolkit.lisp (file).

Source

trivial-benchmark.asd.

Parent Component

trivial-benchmark (system).

Public Interface
Internals

3.1.5 trivial-benchmark/samples.lisp

Dependency

timer.lisp (file).

Source

trivial-benchmark.asd.

Parent Component

trivial-benchmark (system).

Public Interface
Internals

3.1.6 trivial-benchmark/suite.lisp

Dependency

samples.lisp (file).

Source

trivial-benchmark.asd.

Parent Component

trivial-benchmark (system).

Public Interface
Internals

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 trivial-benchmark

Source

package.lisp.

Nicknames
  • benchmark
  • org.shirakumo.trivial-benchmark
Use List

common-lisp.

Public Interface
Internals

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Public Interface


Next: , Previous: , Up: Public Interface   [Contents][Index]

5.1.1 Special variables

Special Variable: *default-computations*

The list of computation-names used to print the REPORT table.

Package

trivial-benchmark.

Source

timer.lisp.

Special Variable: *default-metrics*

The list of class-names used to populate a TIMER by default.

Package

trivial-benchmark.

Source

timer.lisp.


5.1.2 Macros

Macro: define-benchmark (name args &body body)

Define a benchmark named NAME. The structure of this macro is that of DEFUN, except that WITH-BENCHMARK-SAMPLING can be called to collect metrics on the enclosed forms.

ARGS must be a lambda list that does not require arguments present at the call site.

Package

trivial-benchmark.

Source

suite.lisp.

Macro: define-benchmark-package (name &body package-options)

Define a package as if by DEFPACKAGE, except that both the COMMON-LISP and TRIVIAL-BENCHMARK packages are used, and that this package is designated as one containing benchmarks.

Package

trivial-benchmark.

Source

suite.lisp.

Macro: define-delta-metric (name &body sample-point-forms)

Shortcut to define a DELTA-METRIC.
The SAMPLE-POINT-FORMS should return a number to use to calculate a delta. NAME can be either just a NAME or a list of the form (NAME UNITS) where UNITS is the number the sample points are divided by when saving them. For example, if you sample a function of milliseconds, but want to report in seconds, set UNITS to 1000. Doing this instead of dividing directly in the form avoids potential consing when running into bignums.

See DELTA-METRIC

Package

trivial-benchmark.

Source

samples.lisp.

Macro: do-metrics ((metric-var timer &optional result-form) &body forms)

Binds METRIC-VAR to each metric of TIMER and then evaluates FORMS. Returns the value of RESULT-FORM after the loop.

Package

trivial-benchmark.

Source

timer.lisp.

Macro: with-benchmark-sampling (&body forms)

Benchmark the execution of the forms FORMS within the context of a DEFINE-BENCH macro. Acts like PROGN.

Package

trivial-benchmark.

Source

suite.lisp.

Macro: with-sampling ((timer-form) &body forms)

Takes a sample for the evaluation time of FORMS and stores it in the timer given by TIMER-FORM. Acts like a PROGN.

Specifically, START is called, then FORMS are evaluated. If an error occurs within the body, DISCARD is called on the timer, otherwise COMMIT is called once the body exits.

See START
See DISCARD
See COMMIT

Package

trivial-benchmark.

Source

timer.lisp.

Macro: with-suite-report ((suite) &body body)
Package

trivial-benchmark.

Source

suite.lisp.

Macro: with-timing ((n &optional timer-form stream computations) &body forms)

Evaluates FORMS N times, using WITH-SAMPLING on the value of TIMER-FORM each iteration. At the end, runs REPORT on the timer with STREAM and COMPUTATIONS.

See WITH-SAMPLING

Package

trivial-benchmark.

Source

timer.lisp.


5.1.3 Ordinary functions

Function: find-benchmark-package (package-designator)

Find the benchmark package designated by PACKAGE-DESIGNATOR and return it, otherwise return NIL if it doesn’t exist.

Package

trivial-benchmark.

Source

suite.lisp.

Function: make-timer (&optional metric-types)

Creates a TIMER object using the given METRIC-TYPES

Package

trivial-benchmark.

Source

timer.lisp.

Function: map-metrics (timer function)

Maps the metrics in TIMER, calling FUNCTION with each.

Package

trivial-benchmark.

Source

timer.lisp.

Function: print-table (table &key stream padding)

Prints a table (each list in TABLE being a row) with proper spacing.

Package

trivial-benchmark.

Source

toolkit.lisp.

Function: round-to (num n)

Rounds NUM to N digits after the dot.

Package

trivial-benchmark.

Source

toolkit.lisp.

Function: run-package-benchmarks (&key package verbose)

Run all of the benchmarks associated with the benchmark package PACKAGE, the current one by default.

Return a hash table mapping benchmark names to their metrics represented as lists. In particular, it will be an alist whose CARs are metrics and whose CDRs are plists of computations.

VERBOSE is a generalized boolean that controls output verbosity while testing.

Package

trivial-benchmark.

Source

suite.lisp.

Function: type= (a b)

Returns T if A and B denote the same type (are subtypes of one another)
If one of the arguments does not denote a type, the result of TYPE-OF is used in their place.

Package

trivial-benchmark.

Source

toolkit.lisp.


5.1.4 Generic functions

Generic Function: commit (metric)

Commit the current sample of METRIC. If the metric is running, call STOP first.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: commit ((metric cpu-cycles))
Source

samples.lisp.

Method: commit ((metric delta-metric))
Source

samples.lisp.

Method: commit ((timer timer))
Method: commit :before ((metric metric))
Generic Function: compute (thing metric)

Compute a value of the statistical computation THING for METRIC based on its current samples.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: compute ((x (eql :count)) (metric metric))
Method: compute ((x (eql :samples)) (metric metric))
Method: compute ((x (eql :total)) (metric metric))
Method: compute ((x (eql :minimum)) (metric metric))
Method: compute ((x (eql :maximum)) (metric metric))
Method: compute ((x (eql :median)) (metric metric))
Method: compute ((x (eql :average)) (metric metric))
Method: compute ((x (eql :deviation)) (metric metric))
Method: compute ((computations list) (metric metric))
Generic Function: condense (sample)

Turn the SAMPLE value into a usable number.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: condense (thing)
Generic Function: discard (metric)

Discard the current sample of METRIC. If the metric is running, call STOP first.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: discard ((metric cpu-cycles))
Source

samples.lisp.

Method: discard ((metric delta-metric))
Source

samples.lisp.

Method: discard ((timer timer))
Method: discard :before ((metric metric))
Generic Function: metric (type timer)

Returns the metric of TYPE in TIMER if any. The metric must match the type by TYPE=

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: metric ((type symbol) (timer timer))
Generic Function: (setf metric) (timer)

Sets the METRIC in TIMER.
The metric is replaced if it is found in the timer by TYPE= comparison.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: (setf metric) ((timer timer))
Generic Function: metric-types (timer)

Returns the types of metrics in TIMER.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: metric-types ((timer timer))
Generic Reader: metrics (timer)

Returns a list of metrics stored in TIMER.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Reader Method: metrics ((timer timer))

automatically generated reader method

Target Slot

metrics.

Generic Writer: (setf metrics) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf metrics) ((timer timer))

automatically generated writer method

Source

timer.lisp.

Target Slot

metrics.

Generic Function: reduce-samples (metric function)

Apply FUNCTION to the samples stored in METRIC in a REDUCE fashion.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: reduce-samples (metric function)
Generic Function: report (thing &key stream computations)

Print a report of all COMPUTATIONS for THING to STREAM

STREAM can be one of the following:
T — Print to *standard-output*
NIL — Print to a string and return it. STREAM — Print to the stream

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: report (thing &key stream computations)
Generic Function: reset (metric)

Reset the METRIC and remove all its samples.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: reset :after ((metric vector-metric))
Source

samples.lisp.

Method: reset ((metric vector-metric))
Source

samples.lisp.

Method: reset ((timer timer))
Generic Reader: running (metric)

Returns T if the metric is currently sampling.

See START
See STOP

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Reader Method: running ((metric metric))

automatically generated reader method

Target Slot

running.

Generic Writer: (setf running) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf running) ((metric metric))

automatically generated writer method

Source

timer.lisp.

Target Slot

running.

Generic Function: sample-size (metric)

Return the number of samples stored in METRIC.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: sample-size (metric)
Generic Reader: samples (metric)

Return a sequence of committed samples stored in METRIC.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Reader Method: samples ((vector-metric vector-metric))

automatically generated reader method

Source

samples.lisp.

Target Slot

samples.

Generic Writer: (setf samples) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf samples) ((vector-metric vector-metric))

automatically generated writer method

Source

samples.lisp.

Target Slot

samples.

Generic Function: start (metric)

Begin a sample for METRIC. Sets RUNNING to T.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: start ((metric cpu-cycles))
Source

samples.lisp.

Method: start ((metric delta-metric))
Source

samples.lisp.

Method: start ((timer timer))
Method: start :around ((metric metric))
Generic Reader: starting-value (delta-metric)

Returns the value stored when START was called on the DELTA-METRIC.

Package

trivial-benchmark.

Source

samples.lisp.

Methods
Reader Method: starting-value ((delta-metric delta-metric))

automatically generated reader method

Target Slot

starting-value.

Generic Writer: (setf starting-value) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf starting-value) ((delta-metric delta-metric))

automatically generated writer method

Source

samples.lisp.

Target Slot

starting-value.

Generic Function: take-sample (metric)

Return a current sampling value for METRIC.

Note that not all metrics must implement a method for this function. It is perfectly plausible for a metric to skip this method if it cannot provide a sample value at any point in time.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: take-sample ((metric eval-calls))

Samples SB-IMPL::*EVAL-CALLS*.

Source

samples.lisp.

Method: take-sample ((metric bytes-consed))

Samples SB-IMPL::GET-BYTES-CONSED.

Source

samples.lisp.

Method: take-sample ((metric gc-run-time))

Samples SB-IMPL::*GC-RUN-TIME* in seconds.

Source

samples.lisp.

Method: take-sample ((metric page-faults))

Samples the third value (page-faults) of SB-SYS:GET-SYSTEM-INFO in seconds.

Source

samples.lisp.

Method: take-sample ((metric system-run-time))

Samples the second value (system-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.

Source

samples.lisp.

Method: take-sample ((metric user-run-time))

Samples the first value (user-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.

Source

samples.lisp.

Method: take-sample ((metric run-time))

Samples the results of GET-INTERNAL-RUN-TIME in seconds.

Source

samples.lisp.

Method: take-sample ((metric real-time))

Samples the results of GET-INTERNAL-REAL-TIME in seconds.

Source

samples.lisp.

Method: take-sample ((timer timer))

5.1.5 Standalone methods

Method: describe-object ((timer timer) stream)
Source

timer.lisp.

Method: initialize-instance :after ((timer timer) &key)
Source

timer.lisp.

Method: print-object ((timer timer) stream)
Source

timer.lisp.

Method: print-object ((metric metric) stream)
Source

timer.lisp.


5.1.6 Classes

Class: bytes-consed

Samples SB-IMPL::GET-BYTES-CONSED.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

1

Readers

units.

Writers

(setf units).

Class: cpu-cycles

Samples SB-IMPL::ELAPSED-CYCLES.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

vector-metric.

Direct methods
Direct slots
Slot: h0
Readers

cpu-cycles-h0.

Writers

(setf cpu-cycles-h0).

Slot: l0
Readers

cpu-cycles-l0.

Writers

(setf cpu-cycles-l0).

Slot: h1
Readers

cpu-cycles-h1.

Writers

(setf cpu-cycles-h1).

Slot: l1
Readers

cpu-cycles-l1.

Writers

(setf cpu-cycles-l1).

Class: delta-metric

A LISTED-METRIC that calculates a sample point according to the delta between the START and COMMIT. Sub-classes of this must implement the TAKE-SAMPLE method.

See LISTED-METRIC

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

vector-metric.

Direct subclasses
Direct methods
Direct slots
Slot: starting-value
Readers

starting-value.

Writers

(setf starting-value).

Slot: stopping-value
Readers

stopping-value.

Writers

(setf stopping-value).

Slot: units
Initform

1

Readers

units.

Writers

(setf units).

Class: eval-calls

Samples SB-IMPL::*EVAL-CALLS*.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

1

Readers

units.

Writers

(setf units).

Class: gc-run-time

Samples SB-IMPL::*GC-RUN-TIME* in seconds.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

1000

Readers

units.

Writers

(setf units).

Class: metric

A class container for sampling information.

Package

trivial-benchmark.

Source

timer.lisp.

Direct subclasses

vector-metric.

Direct methods
Direct slots
Slot: running
Initargs

:running

Readers

running.

Writers

(setf running).

Class: page-faults

Samples the third value (page-faults) of SB-SYS:GET-SYSTEM-INFO in seconds.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

1

Readers

units.

Writers

(setf units).

Class: real-time

Samples the results of GET-INTERNAL-REAL-TIME in seconds.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

internal-time-units-per-second

Readers

units.

Writers

(setf units).

Class: run-time

Samples the results of GET-INTERNAL-RUN-TIME in seconds.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

internal-time-units-per-second

Readers

units.

Writers

(setf units).

Class: system-run-time

Samples the second value (system-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

1000000

Readers

units.

Writers

(setf units).

Class: timer

Class container for a set of METRICS.

Package

trivial-benchmark.

Source

timer.lisp.

Direct methods
Direct slots
Slot: metrics
Initargs

:metrics

Readers

metrics.

Writers

(setf metrics).

Class: user-run-time

Samples the first value (user-run-time) of SB-SYS:GET-SYSTEM-INFO in seconds.

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

delta-metric.

Direct methods
Direct slots
Slot: units
Initform

1000000

Readers

units.

Writers

(setf units).

Class: vector-metric

A METRIC that implements its sample storage as a vector. SAMPLES is SETF-able for this class.

See METRIC

Package

trivial-benchmark.

Source

samples.lisp.

Direct superclasses

metric.

Direct subclasses
Direct methods
Direct slots
Slot: samples
Initform

(make-array 1000 :adjustable t :fill-pointer 0)

Initargs

:samples

Readers

samples.

Writers

(setf samples).


5.2 Internals


Next: , Previous: , Up: Internals   [Contents][Index]

5.2.1 Special variables

Special Variable: *benchmark-packages*

A table mapping package objects (which are benchmark packages) to a list of fbound symbols.

Package

trivial-benchmark.

Source

suite.lisp.

Special Variable: *current-benchmark*
Package

trivial-benchmark.

Source

suite.lisp.

Special Variable: *current-suite-report*
Package

trivial-benchmark.

Source

suite.lisp.

Special Variable: *current-timer*
Package

trivial-benchmark.

Source

suite.lisp.


5.2.2 Ordinary functions

Function: format-timer-stats (stream timer &optional computations)
Package

trivial-benchmark.

Source

timer.lisp.

Function: make-suite-report-table ()
Package

trivial-benchmark.

Source

suite.lisp.

Function: metrics-alist (timer &optional computations)
Package

trivial-benchmark.

Source

suite.lisp.

Function: print-verbose-after-message (fn &optional out report)
Package

trivial-benchmark.

Source

suite.lisp.

Function: print-verbose-before-message (fn &optional out)
Package

trivial-benchmark.

Source

suite.lisp.

Function: run-one-benchmark (fn &optional verbose)
Package

trivial-benchmark.

Source

suite.lisp.


5.2.3 Generic functions

Generic Reader: cpu-cycles-h0 (object)
Package

trivial-benchmark.

Methods
Reader Method: cpu-cycles-h0 ((cpu-cycles cpu-cycles))

automatically generated reader method

Source

samples.lisp.

Target Slot

h0.

Generic Writer: (setf cpu-cycles-h0) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf cpu-cycles-h0) ((cpu-cycles cpu-cycles))

automatically generated writer method

Source

samples.lisp.

Target Slot

h0.

Generic Reader: cpu-cycles-h1 (object)
Package

trivial-benchmark.

Methods
Reader Method: cpu-cycles-h1 ((cpu-cycles cpu-cycles))

automatically generated reader method

Source

samples.lisp.

Target Slot

h1.

Generic Writer: (setf cpu-cycles-h1) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf cpu-cycles-h1) ((cpu-cycles cpu-cycles))

automatically generated writer method

Source

samples.lisp.

Target Slot

h1.

Generic Reader: cpu-cycles-l0 (object)
Package

trivial-benchmark.

Methods
Reader Method: cpu-cycles-l0 ((cpu-cycles cpu-cycles))

automatically generated reader method

Source

samples.lisp.

Target Slot

l0.

Generic Writer: (setf cpu-cycles-l0) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf cpu-cycles-l0) ((cpu-cycles cpu-cycles))

automatically generated writer method

Source

samples.lisp.

Target Slot

l0.

Generic Reader: cpu-cycles-l1 (object)
Package

trivial-benchmark.

Methods
Reader Method: cpu-cycles-l1 ((cpu-cycles cpu-cycles))

automatically generated reader method

Source

samples.lisp.

Target Slot

l1.

Generic Writer: (setf cpu-cycles-l1) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf cpu-cycles-l1) ((cpu-cycles cpu-cycles))

automatically generated writer method

Source

samples.lisp.

Target Slot

l1.

Generic Function: report-to (stream thing &key computations)
Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: report-to ((stream stream) (hash-table hash-table) &key computations)
Source

suite.lisp.

Method: report-to ((stream stream) (timer timer) &key computations)
Method: report-to ((stream (eql t)) thing &key computations)
Method: report-to ((string (eql nil)) thing &key computations)
Method: report-to ((stream stream) (metric metric) &key computations)
Generic Function: stop (metric)

Stop the sample for METRIC. Sets RUNNING to NIL.

Package

trivial-benchmark.

Source

timer.lisp.

Methods
Method: stop ((metric cpu-cycles))
Source

samples.lisp.

Method: stop ((metric delta-metric))
Source

samples.lisp.

Method: stop ((timer timer))
Method: stop :around ((metric metric))
Method: stop ((metric metric))
Generic Reader: stopping-value (object)
Package

trivial-benchmark.

Methods
Reader Method: stopping-value ((delta-metric delta-metric))

automatically generated reader method

Source

samples.lisp.

Target Slot

stopping-value.

Generic Writer: (setf stopping-value) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf stopping-value) ((delta-metric delta-metric))

automatically generated writer method

Source

samples.lisp.

Target Slot

stopping-value.

Generic Reader: units (object)
Package

trivial-benchmark.

Methods
Reader Method: units ((eval-calls eval-calls))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((bytes-consed bytes-consed))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((gc-run-time gc-run-time))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((page-faults page-faults))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((system-run-time system-run-time))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((user-run-time user-run-time))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((run-time run-time))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((real-time real-time))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Reader Method: units ((delta-metric delta-metric))

automatically generated reader method

Source

samples.lisp.

Target Slot

units.

Generic Writer: (setf units) (object)
Package

trivial-benchmark.

Methods
Writer Method: (setf units) ((eval-calls eval-calls))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((bytes-consed bytes-consed))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((gc-run-time gc-run-time))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((page-faults page-faults))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((system-run-time system-run-time))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((user-run-time user-run-time))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((run-time run-time))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((real-time real-time))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.

Writer Method: (setf units) ((delta-metric delta-metric))

automatically generated writer method

Source

samples.lisp.

Target Slot

units.


Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts


Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   (  
C   D   F   G   I   M   P   R   S   T   U   W  
Index Entry  Section

(
(setf cpu-cycles-h0): Private generic functions
(setf cpu-cycles-h0): Private generic functions
(setf cpu-cycles-h1): Private generic functions
(setf cpu-cycles-h1): Private generic functions
(setf cpu-cycles-l0): Private generic functions
(setf cpu-cycles-l0): Private generic functions
(setf cpu-cycles-l1): Private generic functions
(setf cpu-cycles-l1): Private generic functions
(setf metric): Public generic functions
(setf metric): Public generic functions
(setf metrics): Public generic functions
(setf metrics): Public generic functions
(setf running): Public generic functions
(setf running): Public generic functions
(setf samples): Public generic functions
(setf samples): Public generic functions
(setf starting-value): Public generic functions
(setf starting-value): Public generic functions
(setf stopping-value): Private generic functions
(setf stopping-value): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions
(setf units): Private generic functions

C
commit: Public generic functions
commit: Public generic functions
commit: Public generic functions
commit: Public generic functions
commit: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
compute: Public generic functions
condense: Public generic functions
condense: Public generic functions
cpu-cycles-h0: Private generic functions
cpu-cycles-h0: Private generic functions
cpu-cycles-h1: Private generic functions
cpu-cycles-h1: Private generic functions
cpu-cycles-l0: Private generic functions
cpu-cycles-l0: Private generic functions
cpu-cycles-l1: Private generic functions
cpu-cycles-l1: Private generic functions

D
define-benchmark: Public macros
define-benchmark-package: Public macros
define-delta-metric: Public macros
describe-object: Public standalone methods
discard: Public generic functions
discard: Public generic functions
discard: Public generic functions
discard: Public generic functions
discard: Public generic functions
do-metrics: Public macros

F
find-benchmark-package: Public ordinary functions
format-timer-stats: Private ordinary functions
Function, find-benchmark-package: Public ordinary functions
Function, format-timer-stats: Private ordinary functions
Function, make-suite-report-table: Private ordinary functions
Function, make-timer: Public ordinary functions
Function, map-metrics: Public ordinary functions
Function, metrics-alist: Private ordinary functions
Function, print-table: Public ordinary functions
Function, print-verbose-after-message: Private ordinary functions
Function, print-verbose-before-message: Private ordinary functions
Function, round-to: Public ordinary functions
Function, run-one-benchmark: Private ordinary functions
Function, run-package-benchmarks: Public ordinary functions
Function, type=: Public ordinary functions

G
Generic Function, (setf cpu-cycles-h0): Private generic functions
Generic Function, (setf cpu-cycles-h1): Private generic functions
Generic Function, (setf cpu-cycles-l0): Private generic functions
Generic Function, (setf cpu-cycles-l1): Private generic functions
Generic Function, (setf metric): Public generic functions
Generic Function, (setf metrics): Public generic functions
Generic Function, (setf running): Public generic functions
Generic Function, (setf samples): Public generic functions
Generic Function, (setf starting-value): Public generic functions
Generic Function, (setf stopping-value): Private generic functions
Generic Function, (setf units): Private generic functions
Generic Function, commit: Public generic functions
Generic Function, compute: Public generic functions
Generic Function, condense: Public generic functions
Generic Function, cpu-cycles-h0: Private generic functions
Generic Function, cpu-cycles-h1: Private generic functions
Generic Function, cpu-cycles-l0: Private generic functions
Generic Function, cpu-cycles-l1: Private generic functions
Generic Function, discard: Public generic functions
Generic Function, metric: Public generic functions
Generic Function, metric-types: Public generic functions
Generic Function, metrics: Public generic functions
Generic Function, reduce-samples: Public generic functions
Generic Function, report: Public generic functions
Generic Function, report-to: Private generic functions
Generic Function, reset: Public generic functions
Generic Function, running: Public generic functions
Generic Function, sample-size: Public generic functions
Generic Function, samples: Public generic functions
Generic Function, start: Public generic functions
Generic Function, starting-value: Public generic functions
Generic Function, stop: Private generic functions
Generic Function, stopping-value: Private generic functions
Generic Function, take-sample: Public generic functions
Generic Function, units: Private generic functions

I
initialize-instance: Public standalone methods

M
Macro, define-benchmark: Public macros
Macro, define-benchmark-package: Public macros
Macro, define-delta-metric: Public macros
Macro, do-metrics: Public macros
Macro, with-benchmark-sampling: Public macros
Macro, with-sampling: Public macros
Macro, with-suite-report: Public macros
Macro, with-timing: Public macros
make-suite-report-table: Private ordinary functions
make-timer: Public ordinary functions
map-metrics: Public ordinary functions
Method, (setf cpu-cycles-h0): Private generic functions
Method, (setf cpu-cycles-h1): Private generic functions
Method, (setf cpu-cycles-l0): Private generic functions
Method, (setf cpu-cycles-l1): Private generic functions
Method, (setf metric): Public generic functions
Method, (setf metrics): Public generic functions
Method, (setf running): Public generic functions
Method, (setf samples): Public generic functions
Method, (setf starting-value): Public generic functions
Method, (setf stopping-value): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, (setf units): Private generic functions
Method, commit: Public generic functions
Method, commit: Public generic functions
Method, commit: Public generic functions
Method, commit: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, compute: Public generic functions
Method, condense: Public generic functions
Method, cpu-cycles-h0: Private generic functions
Method, cpu-cycles-h1: Private generic functions
Method, cpu-cycles-l0: Private generic functions
Method, cpu-cycles-l1: Private generic functions
Method, describe-object: Public standalone methods
Method, discard: Public generic functions
Method, discard: Public generic functions
Method, discard: Public generic functions
Method, discard: Public generic functions
Method, initialize-instance: Public standalone methods
Method, metric: Public generic functions
Method, metric-types: Public generic functions
Method, metrics: Public generic functions
Method, print-object: Public standalone methods
Method, print-object: Public standalone methods
Method, reduce-samples: Public generic functions
Method, report: Public generic functions
Method, report-to: Private generic functions
Method, report-to: Private generic functions
Method, report-to: Private generic functions
Method, report-to: Private generic functions
Method, report-to: Private generic functions
Method, reset: Public generic functions
Method, reset: Public generic functions
Method, reset: Public generic functions
Method, running: Public generic functions
Method, sample-size: Public generic functions
Method, samples: Public generic functions
Method, start: Public generic functions
Method, start: Public generic functions
Method, start: Public generic functions
Method, start: Public generic functions
Method, starting-value: Public generic functions
Method, stop: Private generic functions
Method, stop: Private generic functions
Method, stop: Private generic functions
Method, stop: Private generic functions
Method, stop: Private generic functions
Method, stopping-value: Private generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, take-sample: Public generic functions
Method, units: Private generic functions
Method, units: Private generic functions
Method, units: Private generic functions
Method, units: Private generic functions
Method, units: Private generic functions
Method, units: Private generic functions
Method, units: Private generic functions
Method, units: Private generic functions
Method, units: Private generic functions
metric: Public generic functions
metric: Public generic functions
metric-types: Public generic functions
metric-types: Public generic functions
metrics: Public generic functions
metrics: Public generic functions
metrics-alist: Private ordinary functions

P
print-object: Public standalone methods
print-object: Public standalone methods
print-table: Public ordinary functions
print-verbose-after-message: Private ordinary functions
print-verbose-before-message: Private ordinary functions

R
reduce-samples: Public generic functions
reduce-samples: Public generic functions
report: Public generic functions
report: Public generic functions
report-to: Private generic functions
report-to: Private generic functions
report-to: Private generic functions
report-to: Private generic functions
report-to: Private generic functions
report-to: Private generic functions
reset: Public generic functions
reset: Public generic functions
reset: Public generic functions
reset: Public generic functions
round-to: Public ordinary functions
run-one-benchmark: Private ordinary functions
run-package-benchmarks: Public ordinary functions
running: Public generic functions
running: Public generic functions

S
sample-size: Public generic functions
sample-size: Public generic functions
samples: Public generic functions
samples: Public generic functions
start: Public generic functions
start: Public generic functions
start: Public generic functions
start: Public generic functions
start: Public generic functions
starting-value: Public generic functions
starting-value: Public generic functions
stop: Private generic functions
stop: Private generic functions
stop: Private generic functions
stop: Private generic functions
stop: Private generic functions
stop: Private generic functions
stopping-value: Private generic functions
stopping-value: Private generic functions

T
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
take-sample: Public generic functions
type=: Public ordinary functions

U
units: Private generic functions
units: Private generic functions
units: Private generic functions
units: Private generic functions
units: Private generic functions
units: Private generic functions
units: Private generic functions
units: Private generic functions
units: Private generic functions
units: Private generic functions

W
with-benchmark-sampling: Public macros
with-sampling: Public macros
with-suite-report: Public macros
with-timing: Public macros

Jump to:   (  
C   D   F   G   I   M   P   R   S   T   U   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *  
H   L   M   R   S   U  
Index Entry  Section

*
*benchmark-packages*: Private special variables
*current-benchmark*: Private special variables
*current-suite-report*: Private special variables
*current-timer*: Private special variables
*default-computations*: Public special variables
*default-metrics*: Public special variables

H
h0: Public classes
h1: Public classes

L
l0: Public classes
l1: Public classes

M
metrics: Public classes

R
running: Public classes

S
samples: Public classes
Slot, h0: Public classes
Slot, h1: Public classes
Slot, l0: Public classes
Slot, l1: Public classes
Slot, metrics: Public classes
Slot, running: Public classes
Slot, samples: Public classes
Slot, starting-value: Public classes
Slot, stopping-value: Public classes
Slot, units: Public classes
Slot, units: Public classes
Slot, units: Public classes
Slot, units: Public classes
Slot, units: Public classes
Slot, units: Public classes
Slot, units: Public classes
Slot, units: Public classes
Slot, units: Public classes
Special Variable, *benchmark-packages*: Private special variables
Special Variable, *current-benchmark*: Private special variables
Special Variable, *current-suite-report*: Private special variables
Special Variable, *current-timer*: Private special variables
Special Variable, *default-computations*: Public special variables
Special Variable, *default-metrics*: Public special variables
starting-value: Public classes
stopping-value: Public classes

U
units: Public classes
units: Public classes
units: Public classes
units: Public classes
units: Public classes
units: Public classes
units: Public classes
units: Public classes
units: Public classes

Jump to:   *  
H   L   M   R   S   U  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   B   C   D   E   F   G   M   P   R   S   T   U   V  
Index Entry  Section

B
bytes-consed: Public classes

C
Class, bytes-consed: Public classes
Class, cpu-cycles: Public classes
Class, delta-metric: Public classes
Class, eval-calls: Public classes
Class, gc-run-time: Public classes
Class, metric: Public classes
Class, page-faults: Public classes
Class, real-time: Public classes
Class, run-time: Public classes
Class, system-run-time: Public classes
Class, timer: Public classes
Class, user-run-time: Public classes
Class, vector-metric: Public classes
cpu-cycles: Public classes

D
delta-metric: Public classes

E
eval-calls: Public classes

F
File, package.lisp: The trivial-benchmark/package․lisp file
File, samples.lisp: The trivial-benchmark/samples․lisp file
File, suite.lisp: The trivial-benchmark/suite․lisp file
File, timer.lisp: The trivial-benchmark/timer․lisp file
File, toolkit.lisp: The trivial-benchmark/toolkit․lisp file
File, trivial-benchmark.asd: The trivial-benchmark/trivial-benchmark․asd file

G
gc-run-time: Public classes

M
metric: Public classes

P
Package, trivial-benchmark: The trivial-benchmark package
package.lisp: The trivial-benchmark/package․lisp file
page-faults: Public classes

R
real-time: Public classes
run-time: Public classes

S
samples.lisp: The trivial-benchmark/samples․lisp file
suite.lisp: The trivial-benchmark/suite․lisp file
System, trivial-benchmark: The trivial-benchmark system
system-run-time: Public classes

T
timer: Public classes
timer.lisp: The trivial-benchmark/timer․lisp file
toolkit.lisp: The trivial-benchmark/toolkit․lisp file
trivial-benchmark: The trivial-benchmark system
trivial-benchmark: The trivial-benchmark package
trivial-benchmark.asd: The trivial-benchmark/trivial-benchmark․asd file

U
user-run-time: Public classes

V
vector-metric: Public classes

Jump to:   B   C   D   E   F   G   M   P   R   S   T   U   V