The cl-statsd Reference Manual

Table of Contents

The cl-statsd Reference Manual

This is the cl-statsd Reference Manual, version 0.2.1, generated automatically by Declt version 3.0 "Montgomery Scott" on Mon Apr 19 15:21:53 2021 GMT+0.

1 Introduction

StatsD client in Common Lisp

If you are not familiar with statsd read this. To learn about statsd metric types read this.

Speed

CL-STATSD is relatively fast:

(let ((statsd::*client* (statsd:make-sync-client)))
  (loop for i from 0 to 999999 do
    (statsd:counter "self-test" 1)))

Using this Docker image with defaults on busy VM.

Metrics API

• counter (key value &key (rate) (client *client*))
• inc (key &key (rate) (client *client*)) - shortcut for (counter key 1 ...)
• dec (key &key (rate) (client *client*)) - shortcut for (counter key -1 ...)
• timing (key value &key (rate) (client *client*))
• with-timing ((key &optional (client '*client*)) &body body) - executes body and collects execution time
• gauge (key value &key (rate) (client *client*))
• set (key value &key (rate) (client *client*))

Sampling rate can be controlled using *random-range* parameter (default is 100). If set to 0 turns off sampling completely (equivalent of constant rate 1)

Pipelining

Given statsd:*client* is bound you can write something like this:

(statsd:pipeline ()
  (statsd:inc "qwe")
  (statsd:inc "ewq"))

Pipelining implemented using 'wrapper' client that gathers metrics, concatenates them and sends directly to the wrapped client transport. Currently CL-STATSD makes no effort to split pipelined data or otherwise respect/detect MTU.

Error handling

By default all errors simply ignored. You can customize this behaviour by providing :error-handler strategy:

(let ((statsd:*client* (statsd:make-sync-client :error-handler :throw)))
  (loop for i from 0 to 3 do
    (statsd:counter "example" (random 100))
    (statsd::transport.close (statsd::client-transport statsd:*client*))))
; Evaluation aborted on #<CL-STATSD:TRANSPORT-ERROR {1005445653}>.

New error handling strategies can be created by specializing handler-handle-error/2

Clients

null

Like /dev/null

capture

Queues metrics. Useful for debugging, testing

(let ((statsd::*client* (statsd:make-capture-client)))
  (statsd:counter "example" (random 100))
  (statsd:capture-client.recv))
"example:62|c"

sync

Calls transport synchronously

async

More like 'connection-as-a-service', runs in separate thread, all metrics queued first. To prevent queue from overgrowing async client understands throttling threshold (i.e. max queue length):

(let ((statsd:*client* (statsd:make-async-client :error-handler :throw
                                                 :throttle-threshold 5)))
  (loop for i from 0 to 999999 do
        (statsd:counter "example" (random 100))))
; Evaluation aborted on #<CL-STATSD:THROTTLE-THRESHOLD-REACHED {100414CEF3}>.

Throttle threshold can be set globally using *throttle-threshold* value or per async client (:throttle-threshold parameter).

Transports

CL-STATSD comes with the following transport:

• usocket - sends meterics to statsd

New transports can be created by specializing

• make-transport
• transport.connect
• transport.send
• transport.close

TODO

• [x] Async client throttling
• [ ] Travis CI integration
• [ ] Pipelines

License

MIT

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 cl-statsd

Author

Ilya Khaprov <ilya.khaprov@publitechs.com>

License

MIT

Description

Statsd client in Common Lisp

Version

0.2.1

Dependencies

• alexandria
• cl-interpol
• local-time
• safe-queue
• trivial-utf-8
• usocket
• log4cl
• bordeaux-threads

Source

cl-statsd.asd (file)

Component

src (module)

3 Modules

Modules are listed depth-first from the system components tree.

3.1 cl-statsd/src

Parent

cl-statsd (system)

Location

src/

Components

• package.lisp (file)
• clients/base.lisp (file)
• statsd.lisp (file)
• transports (module)
• clients (module)

3.2 cl-statsd/src/transports

Dependency

statsd.lisp (file)

Parent

src (module)

Location

src/transports/

Components

• base.lisp (file)
• usocket.lisp (file)

3.3 cl-statsd/src/clients

Dependency

transports (module)

Parent

src (module)

Location

src/clients/

Components

• capture.lisp (file)
• null.lisp (file)
• sync.lisp (file)
• async.lisp (file)

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.

4.1 Lisp

4.1.1 cl-statsd.asd

Location

cl-statsd.asd

Systems

cl-statsd (system)

4.1.2 cl-statsd/src/package.lisp

Parent

src (module)

Location

src/package.lisp

Packages

cl-statsd

4.1.3 cl-statsd/src/clients/base.lisp

Dependency

package.lisp (file)

Parent

src (module)

Location

src/clients/base.lisp

Exported Definitions

• *random-range* (special variable)
• handler-handle-error (generic function)
• handler-handle-error (method)
• handler-handle-error (method)
• stop-client (function)

Internal Definitions

• client-error-handler (method)
• client-handler-error (function)
• client-prefix (method)
• client-transport (method)
• maybe-send (macro)
• send (generic function)
• send (method)
• serialize-metric (generic function)
• serialize-metric (method)
• serialize-metric (method)
• serialize-metric (method)
• serialize-metric (method)
• serialize-metric% (function)
• statsd-client-base (class)
• statsd-client-with-prefix (class)
• statsd-client-with-transport (class)
• stop-client% (generic function)
• stop-client% (method)
• stop-client% (method)
• with-smart?-error-handling (macro)

4.1.4 cl-statsd/src/statsd.lisp

Dependency

clients/base.lisp (file)

Parent

src (module)

Location

src/statsd.lisp

Exported Definitions

• *client* (special variable)
• counter (function)
• dec (function)
• gauge (function)
• inc (function)
• set (function)
• timing (function)
• with-timing (macro)

Internal Definitions

with-timing% (function)

4.1.5 cl-statsd/src/transports/base.lisp

Parent

transports (module)

Location

src/transports/base.lisp

Exported Definitions

• make-transport (generic function)
• socket-transport (class)
• transport-base (class)
• transport-error (condition)
• transport.close (generic function)
• transport.connect (generic function)
• transport.send (generic function)

Internal Definitions

• transport-error-error (method)
• transport-host (method)
• transport-port (method)
• transport-socket (method)
• transport-tcp-p (method)

4.1.6 cl-statsd/src/transports/usocket.lisp

Dependency

base.lisp (file)

Parent

transports (module)

Location

src/transports/usocket.lisp

Exported Definitions

• make-transport (method)
• transport.close (method)
• transport.connect (method)
• transport.send (method)

Internal Definitions

usocket-transport (class)

4.1.7 cl-statsd/src/clients/capture.lisp

Parent

clients (module)

Location

src/clients/capture.lisp

Exported Definitions

• capture-client.recv (function)
• capture-client.reset (function)
• make-capture-client (function)
• transport.close (method)
• transport.send (method)

Internal Definitions

• capture-client (class)
• capture-client-queue (method)
• (setf capture-client-queue) (method)
• client-transport (method)
• send (method)

4.1.8 cl-statsd/src/clients/null.lisp

Dependency

capture.lisp (file)

Parent

clients (module)

Location

src/clients/null.lisp

Exported Definitions

make-null-client (function)

Internal Definitions

• null-client (class)
• send (method)

4.1.9 cl-statsd/src/clients/sync.lisp

Dependency

null.lisp (file)

Parent

clients (module)

Location

src/clients/sync.lisp

Exported Definitions

make-sync-client (function)

Internal Definitions

• send (method)
• sync-client (class)

4.1.10 cl-statsd/src/clients/async.lisp

Dependency

sync.lisp (file)

Parent

clients (module)

Location

src/clients/async.lisp

Exported Definitions

• *throttle-threshold* (special variable)
• make-async-client (function)
• start-async-client (function)
• throttle-threshold-reached (condition)

Internal Definitions

• +async-client-reconnects-default+ (constant)
• async-client (class)
• async-client-mailbox (method)
• async-client-reconnects (method)
• async-client-state (method)
• async-client-thread (method)
• async-client-thread-fun (function)
• async-client-throttle-threshold (method)
• send (method)
• stop-client% (method)
• throttle-threshold (method)

5 Packages

Packages are listed by definition order.

5.1 cl-statsd

Source

package.lisp (file)

Nickname

statsd

Use List

• alexandria
• common-lisp

Exported Definitions

• *client* (special variable)
• *random-range* (special variable)
• *throttle-threshold* (special variable)
• capture-client.recv (function)
• capture-client.reset (function)
• counter (function)
• dec (function)
• gauge (function)
• handler-handle-error (generic function)
• handler-handle-error (method)
• handler-handle-error (method)
• inc (function)
• make-async-client (function)
• make-capture-client (function)
• make-null-client (function)
• make-sync-client (function)
• make-transport (generic function)
• make-transport (method)
• set (function)
• socket-transport (class)
• start-async-client (function)
• stop-client (function)
• throttle-threshold-reached (condition)
• timing (function)
• transport-base (class)
• transport-error (condition)
• transport.close (generic function)
• transport.close (method)
• transport.close (method)
• transport.connect (generic function)
• transport.connect (method)
• transport.send (generic function)
• transport.send (method)
• transport.send (method)
• with-timing (macro)

Internal Definitions

• +async-client-reconnects-default+ (constant)
• async-client (class)
• async-client-mailbox (generic function)
• async-client-mailbox (method)
• async-client-reconnects (generic function)
• async-client-reconnects (method)
• async-client-state (generic function)
• async-client-state (method)
• async-client-thread (generic function)
• async-client-thread (method)
• async-client-thread-fun (function)
• async-client-throttle-threshold (generic function)
• async-client-throttle-threshold (method)
• capture-client (class)
• capture-client-queue (generic function)
• capture-client-queue (method)
• (setf capture-client-queue) (method)
• (setf capture-client-queue) (generic function)
• client-error-handler (generic function)
• client-error-handler (method)
• client-handler-error (function)
• client-prefix (generic function)
• client-prefix (method)
• client-transport (generic function)
• client-transport (method)
• client-transport (method)
• maybe-send (macro)
• null-client (class)
• send (generic function)
• send (method)
• send (method)
• send (method)
• send (method)
• send (method)
• serialize-metric (generic function)
• serialize-metric (method)
• serialize-metric (method)
• serialize-metric (method)
• serialize-metric (method)
• serialize-metric% (function)
• statsd-client-base (class)
• statsd-client-with-prefix (class)
• statsd-client-with-transport (class)
• stop-client% (generic function)
• stop-client% (method)
• stop-client% (method)
• stop-client% (method)
• sync-client (class)
• throttle-threshold (generic function)
• throttle-threshold (method)
• transport-error-error (generic function)
• transport-error-error (method)
• transport-host (generic function)
• transport-host (method)
• transport-port (generic function)
• transport-port (method)
• transport-socket (generic function)
• transport-socket (method)
• transport-tcp-p (generic function)
• transport-tcp-p (method)
• usocket-transport (class)
• with-smart?-error-handling (macro)
• with-timing% (function)

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

6.1 Exported definitions

6.1.1 Special variables

Special Variable: *client*

Package

cl-statsd

Source

statsd.lisp (file)

Special Variable: *random-range*

Package

cl-statsd

Source

clients/base.lisp (file)

Special Variable: *throttle-threshold*

Package

cl-statsd

Source

async.lisp (file)

6.1.2 Macros

Macro: with-timing (KEY &optional CLIENT) &body BODY

Package

cl-statsd

Source

statsd.lisp (file)

6.1.3 Functions

Function: capture-client.recv &optional CLIENT

Package

cl-statsd

Source

capture.lisp (file)

Function: capture-client.reset &optional CLIENT

Package

cl-statsd

Source

capture.lisp (file)

Function: counter KEY VALUE &key RATE CLIENT

Package

cl-statsd

Source

statsd.lisp (file)

Function: dec KEY &key RATE CLIENT

Package

cl-statsd

Source

statsd.lisp (file)

Function: gauge KEY VALUE &key RATE CLIENT

Package

cl-statsd

Source

statsd.lisp (file)

Function: inc KEY &key RATE CLIENT

Package

cl-statsd

Source

statsd.lisp (file)

Function: make-async-client &key PREFIX THROTTLE-THRESHOLD ERROR-HANDLER TRANSPORT HOST PORT RECONNECTS TCP-P

Package

cl-statsd

Source

async.lisp (file)

Function: make-capture-client &key PREFIX

Package

cl-statsd

Source

capture.lisp (file)

Function: make-null-client ()

Package

cl-statsd

Source

null.lisp (file)

Function: make-sync-client &key PREFIX ERROR-HANDLER TRANSPORT HOST PORT TCP-P

Package

cl-statsd

Source

sync.lisp (file)

Function: set KEY VALUE &key RATE CLIENT

Package

cl-statsd

Source

statsd.lisp (file)

Function: start-async-client &optional CLIENT

Package

cl-statsd

Source

async.lisp (file)

Function: stop-client &key CLIENT TIMEOUT

Package

cl-statsd

Source

clients/base.lisp (file)

Function: timing KEY VALUE &key RATE CLIENT

Package

cl-statsd

Source

statsd.lisp (file)

6.1.4 Generic functions

Generic Function: handler-handle-error HANDLER E

Package

cl-statsd

Source

clients/base.lisp (file)

Methods

Method: handler-handle-error (HANDLER (eql ignore)) E

Method: handler-handle-error (HANDLER (eql throw)) E

Generic Function: make-transport TRANSPORT HOST PORT PROTOCOL

Package

cl-statsd

Source

base.lisp (file)

Methods

Method: make-transport (TRANSPORT (eql usocket)) HOST PORT TCP-P

Source

usocket.lisp (file)

Generic Function: transport.close TRANSPORT

Package

cl-statsd

Source

base.lisp (file)

Methods

Method: transport.close (TRANSPORT capture-client)

Source

capture.lisp (file)

Method: transport.close (TRANSPORT usocket-transport)

Source

usocket.lisp (file)

Generic Function: transport.connect TRANSPORT

Package

cl-statsd

Source

base.lisp (file)

Methods

Method: transport.connect (TRANSPORT usocket-transport)

Source

usocket.lisp (file)

Generic Function: transport.send TRANSPORT METRICS

Package

cl-statsd

Source

base.lisp (file)

Methods

Method: transport.send (TRANSPORT capture-client) METRICS

Source

capture.lisp (file)

Method: transport.send (TRANSPORT usocket-transport) METRICS

Source

usocket.lisp (file)

6.1.5 Conditions

Condition: throttle-threshold-reached ()

Package

cl-statsd

Source

async.lisp (file)

Direct superclasses

error (condition)

Direct methods

throttle-threshold (method)

Direct slots

Slot: threshold

Initargs

:threshold

Readers

throttle-threshold (generic function)

Condition: transport-error ()

Package

cl-statsd

Source

base.lisp (file)

Direct superclasses

error (condition)

Direct methods

transport-error-error (method)

Direct slots

Slot: error

Initargs

:error

Readers

transport-error-error (generic function)

6.1.6 Classes

Class: socket-transport ()

Package

cl-statsd

Source

base.lisp (file)

Direct superclasses

transport-base (class)

Direct subclasses

usocket-transport (class)

Direct methods

• transport-tcp-p (method)
• transport-port (method)
• transport-host (method)
• transport-socket (method)

Direct slots

Slot: socket

Initargs

:socket

Readers

transport-socket (generic function)

Slot: host

Initargs

:host

Readers

transport-host (generic function)

Slot: port

Initargs

:port

Readers

transport-port (generic function)

Slot: tcp

Initargs

:tcp-p

Readers

transport-tcp-p (generic function)

Class: transport-base ()

Package

cl-statsd

Source

base.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

• socket-transport (class)
• capture-client (class)

6.2 Internal definitions

6.2.1 Constants

Constant: +async-client-reconnects-default+

Package

cl-statsd

Source

async.lisp (file)

6.2.2 Macros

Macro: maybe-send RATE &body BODY

Package

cl-statsd

Source

clients/base.lisp (file)

Macro: with-smart?-error-handling CLIENT &body BODY

Package

cl-statsd

Source

clients/base.lisp (file)

6.2.3 Functions

Function: async-client-thread-fun CLIENT

Package

cl-statsd

Source

async.lisp (file)

Function: client-handler-error CLIENT E

Package

cl-statsd

Source

clients/base.lisp (file)

Function: serialize-metric% METRIC KEY VALUE RATE

Package

cl-statsd

Source

clients/base.lisp (file)

Function: with-timing% CLIENT KEY LAMBDA

Package

cl-statsd

Source

statsd.lisp (file)

6.2.4 Generic functions

Generic Function: async-client-mailbox OBJECT

Package

cl-statsd

Methods

Method: async-client-mailbox (ASYNC-CLIENT async-client)

automatically generated reader method

Source

async.lisp (file)

Generic Function: async-client-reconnects OBJECT

Package

cl-statsd

Methods

Method: async-client-reconnects (ASYNC-CLIENT async-client)

automatically generated reader method

Source

async.lisp (file)

Generic Function: async-client-state OBJECT

Package

cl-statsd

Methods

Method: async-client-state (ASYNC-CLIENT async-client)

automatically generated reader method

Source

async.lisp (file)

Generic Function: async-client-thread OBJECT

Package

cl-statsd

Methods

Method: async-client-thread (ASYNC-CLIENT async-client)

automatically generated reader method

Source

async.lisp (file)

Generic Function: async-client-throttle-threshold OBJECT

Package

cl-statsd

Methods

Method: async-client-throttle-threshold (ASYNC-CLIENT async-client)

automatically generated reader method

Source

async.lisp (file)

Generic Function: capture-client-queue OBJECT

Generic Function: (setf capture-client-queue) NEW-VALUE OBJECT

Package

cl-statsd

Methods

Method: capture-client-queue (CAPTURE-CLIENT capture-client)

automatically generated reader method

Source

capture.lisp (file)

Method: (setf capture-client-queue) NEW-VALUE (CAPTURE-CLIENT capture-client)

automatically generated writer method

Source

capture.lisp (file)

Generic Function: client-error-handler OBJECT

Package

cl-statsd

Methods

Method: client-error-handler (STATSD-CLIENT-BASE statsd-client-base)

automatically generated reader method

Source

clients/base.lisp (file)

Generic Function: client-prefix OBJECT

Package

cl-statsd

Methods

Method: client-prefix (STATSD-CLIENT-WITH-PREFIX statsd-client-with-prefix)

automatically generated reader method

Source

clients/base.lisp (file)

Generic Function: client-transport OBJECT

Package

cl-statsd

Methods

Method: client-transport (CLIENT capture-client)

Source

capture.lisp (file)

Method: client-transport (STATSD-CLIENT-WITH-TRANSPORT statsd-client-with-transport)

automatically generated reader method

Source

clients/base.lisp (file)

Generic Function: send CLIENT METRIC KEY VALUE RATE

Package

cl-statsd

Source

clients/base.lisp (file)

Methods

Method: send (CLIENT async-client) METRIC KEY VALUE RATE

Source

async.lisp (file)

Method: send (CLIENT sync-client) METRIC KEY VALUE RATE

Source

sync.lisp (file)

Method: send (CLIENT null-client) METRIC KEY VALUE RATE

Source

null.lisp (file)

Method: send (CLIENT capture-client) METRIC KEY VALUE RATE

Source

capture.lisp (file)

Method: send (CLIENT statsd-client-with-prefix) METRIC KEY VALUE RATE around

Generic Function: serialize-metric METRIC KEY VALUE RATE

Package

cl-statsd

Source

clients/base.lisp (file)

Methods

Method: serialize-metric (METRIC (eql set)) KEY VALUE RATE

Method: serialize-metric (METRIC (eql gauge)) KEY VALUE RATE

Method: serialize-metric (METRIC (eql timing)) KEY VALUE RATE

Method: serialize-metric (METRIC (eql counter)) KEY VALUE RATE

Generic Function: stop-client% CLIENT TIMEOUT

Package

cl-statsd

Source

clients/base.lisp (file)

Methods

Method: stop-client% (CLIENT async-client) TIMEOUT

Source

async.lisp (file)

Method: stop-client% (CLIENT statsd-client-with-transport) TIMEOUT

Method: stop-client% CLIENT TIMEOUT

Generic Function: throttle-threshold CONDITION

Package

cl-statsd

Methods

Method: throttle-threshold (CONDITION throttle-threshold-reached)

Source

async.lisp (file)

Generic Function: transport-error-error CONDITION

Package

cl-statsd

Methods

Method: transport-error-error (CONDITION transport-error)

Source

base.lisp (file)

Generic Function: transport-host OBJECT

Package

cl-statsd

Methods

Method: transport-host (SOCKET-TRANSPORT socket-transport)

automatically generated reader method

Source

base.lisp (file)

Generic Function: transport-port OBJECT

Package

cl-statsd

Methods

Method: transport-port (SOCKET-TRANSPORT socket-transport)

automatically generated reader method

Source

base.lisp (file)

Generic Function: transport-socket OBJECT

Package

cl-statsd

Methods

Method: transport-socket (SOCKET-TRANSPORT socket-transport)

automatically generated reader method

Source

base.lisp (file)

Generic Function: transport-tcp-p OBJECT

Package

cl-statsd

Methods

Method: transport-tcp-p (SOCKET-TRANSPORT socket-transport)

automatically generated reader method

Source

base.lisp (file)

6.2.5 Classes

Class: async-client ()

Package

cl-statsd

Source

async.lisp (file)

Direct superclasses

statsd-client-with-transport (class)

Direct methods

• send (method)
• stop-client% (method)
• async-client-mailbox (method)
• async-client-thread (method)
• async-client-throttle-threshold (method)
• async-client-state (method)
• async-client-reconnects (method)

Direct slots

Slot: reconnects

Initargs

:reconnects

Initform

cl-statsd::+async-client-reconnects-default+

Readers

async-client-reconnects (generic function)

Slot: state

Initform

:created

Readers

async-client-state (generic function)

Slot: throttle-threshold

Initargs

:throttle-threshold

Readers

async-client-throttle-threshold (generic function)

Slot: thread

Readers

async-client-thread (generic function)

Slot: mailbox

Initform

(safe-queue:make-mailbox)

Readers

async-client-mailbox (generic function)

Class: capture-client ()

Package

cl-statsd

Source

capture.lisp (file)

Direct superclasses

• transport-base (class)
• statsd-client-with-prefix (class)

Direct methods

• transport.close (method)
• transport.send (method)
• client-transport (method)
• send (method)
• capture-client-queue (method)
• capture-client-queue (method)

Direct slots

Slot: queue

Initform

(safe-queue:make-queue)

Readers

capture-client-queue (generic function)

Writers

(setf capture-client-queue) (generic function)

Class: null-client ()

Package

cl-statsd

Source

null.lisp (file)

Direct superclasses

statsd-client-base (class)

Direct methods

send (method)

Class: statsd-client-base ()

Package

cl-statsd

Source

clients/base.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

• statsd-client-with-prefix (class)
• null-client (class)

Direct methods

client-error-handler (method)

Direct slots

Slot: error-handler

Initargs

:error-handler

Initform

:ignore

Readers

client-error-handler (generic function)

Class: statsd-client-with-prefix ()

Package

cl-statsd

Source

clients/base.lisp (file)

Direct superclasses

statsd-client-base (class)

Direct subclasses

• statsd-client-with-transport (class)
• capture-client (class)

Direct methods

• send (method)
• client-prefix (method)

Direct slots

Slot: prefix

Initargs

:prefix

Readers

client-prefix (generic function)

Class: statsd-client-with-transport ()

Package

cl-statsd

Source

clients/base.lisp (file)

Direct superclasses

statsd-client-with-prefix (class)

Direct subclasses

• sync-client (class)
• async-client (class)

Direct methods

• stop-client% (method)
• client-transport (method)

Direct slots

Slot: transport

Initargs

:transport

Readers

client-transport (generic function)

Class: sync-client ()

Package

cl-statsd

Source

sync.lisp (file)

Direct superclasses

statsd-client-with-transport (class)

Direct methods

send (method)

Class: usocket-transport ()

Package

cl-statsd

Source

usocket.lisp (file)

Direct superclasses

socket-transport (class)

Direct methods

• transport.close (method)
• transport.send (method)
• transport.connect (method)

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables

A.4 Data types