The the-cost-of-nothing Reference Manual

Table of Contents

The the-cost-of-nothing Reference Manual

This is the the-cost-of-nothing Reference Manual, generated automatically by Declt version 3.0 "Montgomery Scott" on Tue Dec 22 15:14:57 2020 GMT+0.

1 Introduction

#+BEGIN_QUOTE
A LISP programmer knows the value of everything, but the cost of nothing.

--- Alan Perlis
#+END_QUOTE

This library is a toolbox for measuring the run time of Common Lisp code.
It provides macros and functions for accurate benchmarking and lightweight
monitoring.  Furthermore, it provides predefined benchmarks to determine
the cost of certain actions on a given platform and implementation.

I hope you find it useful. If you have new ideas for benchmarks, or
suggestions on how to improve existing ones, feel free to contact me.

** Benchmarking
To run all predefined benchmarks, simply execute

#+BEGIN_SRC lisp :results output
(asdf:test-system :the-cost-of-nothing)
#+END_SRC

To obtain the execution time of an expression as a double-float in seconds,
execute
#+BEGIN_SRC lisp
(benchmark EXPRESSION)
#+END_SRC

To have said execution time immediately and human-readably written to
*trace-output* (much like CL:TIME), execute
#+BEGIN_SRC lisp
(bench EXPRESSION)
#+END_SRC

** Monitoring

Coming soon.

** Remember

#+BEGIN_QUOTE
There are lies, damned lies, and benchmarks.
#+END_QUOTE

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 the-cost-of-nothing

Author

Marco Heisig <marco.heisig@fau.de>

License

MIT

Description

Determine the cost of things in Common Lisp.

Long Description

This library provides portable and sophisticated benchmark functions. It comes bundled with an extensive test suite that describes the performance of the currently used Lisp implementation, e.g. with respect to garbage collection, sequence traversal, CLOS and floating-point performance.

Dependencies

• alexandria
• closer-mop
• local-time
• trivial-garbage

Source

the-cost-of-nothing.asd (file)

Components

• core (module)
• benchmarks (module)

3 Modules

Modules are listed depth-first from the system components tree.

3.1 the-cost-of-nothing/core

Parent

the-cost-of-nothing (system)

Location

core/

Components

• packages.lisp (file)
• utilities.lisp (file)
• timestamps.lisp (file)
• monitoring.lisp (file)
• benchmarking.lisp (file)
• time-series.lisp (file)

3.2 the-cost-of-nothing/benchmarks

Parent

the-cost-of-nothing (system)

Location

benchmarks/

Components

• memory-management.lisp (file)
• functions.lisp (file)
• numerics.lisp (file)
• report.lisp (file)

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.

4.1 Lisp

4.1.1 the-cost-of-nothing.asd

Location

the-cost-of-nothing.asd

Systems

the-cost-of-nothing (system)

4.1.2 the-cost-of-nothing/core/packages.lisp

Parent

core (module)

Location

core/packages.lisp

Packages

the-cost-of-nothing

4.1.3 the-cost-of-nothing/core/utilities.lisp

Dependency

packages.lisp (file)

Parent

core (module)

Location

core/utilities.lisp

Exported Definitions

• print-time (function)
• write-si-unit (function)

Internal Definitions

• *si-prefix-alist* (special variable)
• call-with-memoization (function)
• compiled-lambda (function)
• define-memo-function (macro)
• lambda-list-bindings (function)
• make-argument-list (function)
• make-lambda-list (function)
• nth-keyword-name (function)
• with-memoization (macro)

4.1.4 the-cost-of-nothing/core/timestamps.lisp

Dependency

utilities.lisp (file)

Parent

core (module)

Location

core/timestamps.lisp

Internal Definitions

• *timestamp-function* (special variable)
• call-with-fuzzy-timestamps (function)
• make-timestamp (function)
• with-fuzzy-timestamps (macro)

4.1.5 the-cost-of-nothing/core/monitoring.lisp

Dependency

timestamps.lisp (file)

Parent

core (module)

Location

core/monitoring.lisp

Exported Definitions

• make-measurement (function)
• measurement (structure)
• measurement-context (function)
• (setf measurement-context) (function)
• measurement-name (function)
• (setf measurement-name) (function)
• measurement-timestamp (function)
• (setf measurement-timestamp) (function)
• measurement-value (function)
• (setf measurement-value) (function)
• measurementp (function)
• monitor (macro)
• monitoring-region-end (structure)
• monitoring-region-start (structure)
• with-monitoring-region (macro)

Internal Definitions

• %monitor (function)
• *context* (special variable)
• *measurements* (special variable)
• call-with-monitoring-region (function)
• copy-measurement (function)
• copy-monitoring-region-end (function)
• copy-monitoring-region-start (function)
• make-monitoring-region-end (function)
• make-monitoring-region-start (function)
• monitoring-region-end-context (function)
• (setf monitoring-region-end-context) (function)
• monitoring-region-end-name (function)
• (setf monitoring-region-end-name) (function)
• monitoring-region-end-p (function)
• monitoring-region-end-timestamp (function)
• (setf monitoring-region-end-timestamp) (function)
• monitoring-region-end-value (function)
• (setf monitoring-region-end-value) (function)
• monitoring-region-start-context (function)
• (setf monitoring-region-start-context) (function)
• monitoring-region-start-name (function)
• (setf monitoring-region-start-name) (function)
• monitoring-region-start-p (function)
• monitoring-region-start-timestamp (function)
• (setf monitoring-region-start-timestamp) (function)
• monitoring-region-start-value (function)
• (setf monitoring-region-start-value) (function)
• store-measurement (generic function)
• store-measurement (method)

4.1.6 the-cost-of-nothing/core/benchmarking.lisp

Dependency

monitoring.lisp (file)

Parent

core (module)

Location

core/benchmarking.lisp

Exported Definitions

• bench (macro)
• benchmark (macro)
• benchmark-thunk (function)
• touch (function)

Internal Definitions

• *default-min-sample-time* (special variable)
• *default-overhead* (special variable)
• execution-time-of-thunk (function)
• sample-execution-time-of-thunk (function)

4.1.7 the-cost-of-nothing/core/time-series.lisp

Dependency

benchmarking.lisp (file)

Parent

core (module)

Location

core/time-series.lisp

Internal Definitions

• measurements (method)
• (setf measurements) (method)
• time-series (class)

4.1.8 the-cost-of-nothing/benchmarks/memory-management.lisp

Parent

benchmarks (module)

Location

benchmarks/memory-management.lisp

Exported Definitions

• cons-cost (function)
• gc-cost (function)
• make-list-cost (function)
• make-sequence-cost (function)

Internal Definitions

• make-instance-cost (function)
• make-struct-cost (function)
• print-memory-management-report (function)

4.1.9 the-cost-of-nothing/benchmarks/functions.lisp

Dependency

memory-management.lisp (file)

Parent

benchmarks (module)

Location

benchmarks/functions.lisp

Exported Definitions

funcall-cost (function)

Internal Definitions

print-functions-report (function)

4.1.10 the-cost-of-nothing/benchmarks/numerics.lisp

Dependency

functions.lisp (file)

Parent

benchmarks (module)

Location

benchmarks/numerics.lisp

Exported Definitions

flops (function)

Internal Definitions

print-numerics-report (function)

4.1.11 the-cost-of-nothing/benchmarks/report.lisp

Dependency

numerics.lisp (file)

Parent

benchmarks (module)

Location

benchmarks/report.lisp

Exported Definitions

print-report (function)

Internal Definitions

y-intersection-and-slope (function)

5 Packages

Packages are listed by definition order.

5.1 the-cost-of-nothing

Source

packages.lisp (file)

Use List

closer-common-lisp

Exported Definitions

• bench (macro)
• benchmark (macro)
• benchmark-thunk (function)
• cons-cost (function)
• flops (function)
• funcall-cost (function)
• gc-cost (function)
• make-list-cost (function)
• make-measurement (function)
• make-sequence-cost (function)
• measurement (structure)
• measurement-context (function)
• (setf measurement-context) (function)
• measurement-name (function)
• (setf measurement-name) (function)
• measurement-timestamp (function)
• (setf measurement-timestamp) (function)
• measurement-value (function)
• (setf measurement-value) (function)
• measurementp (function)
• monitor (macro)
• monitoring-region-end (structure)
• monitoring-region-start (structure)
• print-report (function)
• print-time (function)
• touch (function)
• with-monitoring-region (macro)
• write-si-unit (function)

Internal Definitions

• %monitor (function)
• *context* (special variable)
• *default-min-sample-time* (special variable)
• *default-overhead* (special variable)
• *measurements* (special variable)
• *si-prefix-alist* (special variable)
• *timestamp-function* (special variable)
• call-with-fuzzy-timestamps (function)
• call-with-memoization (function)
• call-with-monitoring-region (function)
• compiled-lambda (function)
• copy-measurement (function)
• copy-monitoring-region-end (function)
• copy-monitoring-region-start (function)
• define-memo-function (macro)
• execution-time-of-thunk (function)
• lambda-list-bindings (function)
• make-argument-list (function)
• make-instance-cost (function)
• make-lambda-list (function)
• make-monitoring-region-end (function)
• make-monitoring-region-start (function)
• make-struct-cost (function)
• make-timestamp (function)
• measurements (generic function)
• measurements (method)
• (setf measurements) (method)
• (setf measurements) (generic function)
• monitoring-region-end-context (function)
• (setf monitoring-region-end-context) (function)
• monitoring-region-end-name (function)
• (setf monitoring-region-end-name) (function)
• monitoring-region-end-p (function)
• monitoring-region-end-timestamp (function)
• (setf monitoring-region-end-timestamp) (function)
• monitoring-region-end-value (function)
• (setf monitoring-region-end-value) (function)
• monitoring-region-start-context (function)
• (setf monitoring-region-start-context) (function)
• monitoring-region-start-name (function)
• (setf monitoring-region-start-name) (function)
• monitoring-region-start-p (function)
• monitoring-region-start-timestamp (function)
• (setf monitoring-region-start-timestamp) (function)
• monitoring-region-start-value (function)
• (setf monitoring-region-start-value) (function)
• nth-keyword-name (function)
• print-functions-report (function)
• print-memory-management-report (function)
• print-numerics-report (function)
• sample-execution-time-of-thunk (function)
• store-measurement (generic function)
• store-measurement (method)
• time-series (class)
• with-fuzzy-timestamps (macro)
• with-memoization (macro)
• y-intersection-and-slope (function)

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

6.1 Exported definitions

6.1.1 Macros

Macro: bench FORM &rest ARGS &key MAX-SAMPLES MIN-SAMPLE-TIME TIMEOUT OVERHEAD

Evaluate FORM multiple times and print the averaged execution time to *TRACE-OUTPUT*.

Examples:
(bench nil) => 0.00 nanoseconds
(bench (make-hash-table)) => 247.03 nanoseconds

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Macro: benchmark FORM &rest ARGS &key MAX-SAMPLES MIN-SAMPLE-TIME TIMEOUT OVERHEAD

Execute BODY multiple times to accurately measure its execution time in seconds. The returned values are literally the same as those from an invocation of MEASURE-EXECUTION-TIME with suitable lambdas.

Examples:
(benchmark (cons nil nil)) -> 3.3d-9 1.0 36995264
(benchmark (gc :full t)) -> 0.031 0.9 90

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Macro: monitor FORM &key NAME

Monitor VALUE by storing a measurement in *MEASUREMENTS*. If *MEASUREMENTS* is NIL, no measurement is recorded.

The keyword argument NAME can be used to describe the nature of the measurement. The default name of an measurement is the VALUE form that is the first argument of this macro.

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Macro: with-monitoring-region (NAME) &body BODY

Execute BODY in a region monitoring NAME. This entails the following things:

1. The special variable *MEASUREMENTS* is bound to itself, to ensure that
it has the same value throughout the entire monitoring block.

1. An MONITORING-REGION-START measurement is stored right before executing BODY. Its value is NIL initially, but later set to the corresponding MONITORING-REGION-END measurement.

2. For the dynamic extent of BODY, NAME is prepended to the context of all measurements.

3. A MONITORING-REGION-END measurement is stored once control is transferred outside of BODY. Its value is the corresponding MONITORING-REGION-START measurement.

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

6.1.2 Functions

Function: benchmark-thunk THUNK &key TIMEOUT MIN-SAMPLE-TIME MAX-SAMPLES OVERHEAD

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Function: cons-cost ()

Package

the-cost-of-nothing

Source

memory-management.lisp (file)

Function: flops &key ELEMENT-TYPE

Package

the-cost-of-nothing

Source

numerics.lisp (file)

Function: funcall-cost &key MANDATORY-ARGUMENTS OPTIONAL-ARGUMENTS KEYWORD-ARGUMENTS REST-ARGUMENTS

Package

the-cost-of-nothing

Source

functions.lisp (file)

Function: gc-cost &key FULL

Package

the-cost-of-nothing

Source

memory-management.lisp (file)

Function: make-list-cost SIZE &key INITIAL-ELEMENT

Package

the-cost-of-nothing

Source

memory-management.lisp (file)

Function: make-measurement VALUE NAME

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: make-sequence-cost RESULT-TYPE LENGTH &key INITIAL-ELEMENT

Package

the-cost-of-nothing

Source

memory-management.lisp (file)

Function: measurement-context INSTANCE

Function: (setf measurement-context) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: measurement-name INSTANCE

Function: (setf measurement-name) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: measurement-timestamp INSTANCE

Function: (setf measurement-timestamp) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: measurement-value INSTANCE

Function: (setf measurement-value) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: measurementp OBJECT

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: print-report &optional STREAM

Package

the-cost-of-nothing

Source

report.lisp (file)

Function: print-time TIME &optional STREAM

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Function: touch OBJECT

Protect OBJECT from compiler optimization.

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Function: write-si-unit QUANTITY UNIT STREAM

Package

the-cost-of-nothing

Source

utilities.lisp (file)

6.1.3 Structures

Structure: measurement ()

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Direct superclasses

structure-object (structure)

Direct subclasses

• monitoring-region-start (structure)
• monitoring-region-end (structure)

Direct methods

print-object (method)

Direct slots

Slot: value

Readers

measurement-value (function)

Writers

(setf measurement-value) (function)

Slot: name

Readers

measurement-name (function)

Writers

(setf measurement-name) (function)

Slot: context

Initform

the-cost-of-nothing::*context*

Readers

measurement-context (function)

Writers

(setf measurement-context) (function)

Slot: timestamp

Initform

(the-cost-of-nothing::make-timestamp)

Readers

measurement-timestamp (function)

Writers

(setf measurement-timestamp) (function)

Structure: monitoring-region-end ()

A measurement that is only emitted when leaving a monitoring region. Its value is the corresponding monitoring region start.

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Direct superclasses

measurement (structure)

Direct methods

print-object (method)

Structure: monitoring-region-start ()

A measurement that is only emitted when entering a monitoring
region. Its value is the corresponding monitoring region end, or NIL, when the region is still active.

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Direct superclasses

measurement (structure)

Direct methods

print-object (method)

6.2 Internal definitions

6.2.1 Special variables

Special Variable: *context*

A list of the names of all surrounding monitoring regions.

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Special Variable: *default-min-sample-time*

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Special Variable: *default-overhead*

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Special Variable: *measurements*

An object suitable as a second argument to STORE-MEASUREMENT.

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Special Variable: *si-prefix-alist*

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Special Variable: *timestamp-function*

Package

the-cost-of-nothing

Source

timestamps.lisp (file)

6.2.2 Macros

Macro: define-memo-function NAME LAMBDA-LIST &body BODY

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Macro: with-fuzzy-timestamps (&key FUZZINESS) &body BODY

Set the timestamp fuzziness for the dynamic extent of BODY. This can be useful for profiling performance critical regions of code, where the cost of measuring the time would otherwise dominate the execution.

A timestamp fuzziness of zero means that each call to MAKE-TIMESTAMP produces a new timestamp. Values bigger than zero mean that MAKE-TIMESTAMP returns the same timestamp for the given number of consecutive calls.

Package

the-cost-of-nothing

Source

timestamps.lisp (file)

Macro: with-memoization KEY-FORM &body BODY

Package

the-cost-of-nothing

Source

utilities.lisp (file)

6.2.3 Functions

Function: %monitor VALUE NAME

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: call-with-fuzzy-timestamps THUNK FUZZINESS

Package

the-cost-of-nothing

Source

timestamps.lisp (file)

Function: call-with-memoization KEY TABLE THUNK

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Function: call-with-monitoring-region NAME THUNK

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: compiled-lambda LAMBDA-LIST &rest BODY

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Function: copy-measurement INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: copy-monitoring-region-end INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: copy-monitoring-region-start INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: execution-time-of-thunk THUNK

Execute THUNK and return the execution time of THUNK in seconds as a double-float.

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Function: lambda-list-bindings LAMBDA-LIST

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Function: make-argument-list &key MANDATORY-ARGUMENTS OPTIONAL-ARGUMENTS KEYWORD-ARGUMENTS REST-ARGUMENTS

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Function: make-instance-cost &key NUMBER-OF-SLOTS

Package

the-cost-of-nothing

Source

memory-management.lisp (file)

Function: make-lambda-list &key MANDATORY-ARGUMENTS OPTIONAL-ARGUMENTS KEYWORD-ARGUMENTS REST-ARGUMENTS

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Function: make-monitoring-region-end NAME VALUE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: make-monitoring-region-start NAME

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: make-struct-cost &key NUMBER-OF-SLOTS

Package

the-cost-of-nothing

Source

memory-management.lisp (file)

Function: make-timestamp ()

Package

the-cost-of-nothing

Source

timestamps.lisp (file)

Function: monitoring-region-end-context INSTANCE

Function: (setf monitoring-region-end-context) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-end-name INSTANCE

Function: (setf monitoring-region-end-name) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-end-p OBJECT

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-end-timestamp INSTANCE

Function: (setf monitoring-region-end-timestamp) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-end-value INSTANCE

Function: (setf monitoring-region-end-value) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-start-context INSTANCE

Function: (setf monitoring-region-start-context) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-start-name INSTANCE

Function: (setf monitoring-region-start-name) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-start-p OBJECT

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-start-timestamp INSTANCE

Function: (setf monitoring-region-start-timestamp) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: monitoring-region-start-value INSTANCE

Function: (setf monitoring-region-start-value) VALUE INSTANCE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Function: nth-keyword-name N

Package

the-cost-of-nothing

Source

utilities.lisp (file)

Function: print-functions-report &optional STREAM

Package

the-cost-of-nothing

Source

functions.lisp (file)

Function: print-memory-management-report &optional STREAM

Package

the-cost-of-nothing

Source

memory-management.lisp (file)

Function: print-numerics-report &optional STREAM

Package

the-cost-of-nothing

Source

numerics.lisp (file)

Function: sample-execution-time-of-thunk THUNK MIN-SAMPLE-TIME

Measure the execution time of invoking THUNK more and more often, until the execution time exceeds MIN-SAMPLE-TIME.

Package

the-cost-of-nothing

Source

benchmarking.lisp (file)

Function: y-intersection-and-slope X0 Y0 X1 Y1

Package

the-cost-of-nothing

Source

report.lisp (file)

6.2.4 Generic functions

Generic Function: measurements OBJECT

Generic Function: (setf measurements) NEW-VALUE OBJECT

Package

the-cost-of-nothing

Methods

Method: measurements (TIME-SERIES time-series)

automatically generated reader method

Source

time-series.lisp (file)

Method: (setf measurements) NEW-VALUE (TIME-SERIES time-series)

automatically generated writer method

Source

time-series.lisp (file)

Generic Function: store-measurement MEASUREMENT STORAGE

Package

the-cost-of-nothing

Source

monitoring.lisp (file)

Methods

Method: store-measurement MEASUREMENT (NULL null)

6.2.5 Classes

Class: time-series ()

Package

the-cost-of-nothing

Source

time-series.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• measurements (method)
• measurements (method)

Direct slots

Slot: %measurements

Initform

(quote nil)

Readers

measurements (generic function)

Writers

(setf measurements) (generic function)

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables

A.4 Data types