The cl-etcd Reference Manual

Next: , Previous: , Up: (dir)   [Contents][Index]

The cl-etcd Reference Manual

This is the cl-etcd Reference Manual, version 0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Thu Sep 15 03:44:06 2022 GMT+0.

Table of Contents


1 Introduction

cl-etcd

Build Status

The purpose of cl-etcd is to simplify the writing of distributed applications in Common Lisp. etcd is a strongly consistent, distributed key-value store. At it's core is an implementation of the raft consensus algorithm. GC pauses in my lisp implementation of choice make it a poor choice for a lisp-based raft implementation, and so we run etcd as an asynchronous child process under lisp.

The cl-etcd package includes basic get-etcd and watch functions, as well as the convenience macro with-etcd to make it easy to start-up and shut-down your "embedded" etcd node. get-etcd is setf-able so you can easily modify the key-value store the Lisp way.

If you want to be notified about leader state changes on your node, supply the optional :on-leader and :on-follower lambdas, and they will be called when the node becomes a cluster leader or follower respectively.

Etcd is configured for auto-TLS communication between peers, meaning that the inter-node traffic will be encrypted with self-signed certificates. No form of authentication or encryption is currently performed between etcd and the client (the cl-etcd library code), however etcd is configured to only allow connections from localhost.

Here's a trivial example of a single node:

(with-etcd (etcd nil)
  (setf (get-etcd "hello" etcd) "world")
  (get-etcd "hello" etcd))

To be notified on state changes to leader or follower, do this:

(defun become-leader (etcd)
  (print "I'm the leader!"))

(defun become-follower (etcd)
  (print "I'm a follower!"))

(with-etcd (etcd nil :on-leader #'become-leader :on-follower #'become-follower)
  (setf (get-etcd "hello" etcd) "world")
  (get-etcd "hello" etcd))

A single instance is pretty useless, so let's make a 3-node cluster! The second argument to with-etcd is a hashtable of etcd arguments. The easiest way to populate this is through TOML config files. Let's make three config files like so...

config1.ini:

[etcd]
name = "infra0"
initial-advertise-peer-urls = "http://127.0.0.1:2380"
listen-peer-urls = "http://127.0.0.1:2380"
listen-client-urls = "http://127.0.0.1:2379"
advertise-client-urls = "http://127.0.0.1:2379"
initial-cluster = "infra0=http://127.0.0.1:2380,infra1=http://127.0.0.1:2480,infra2=http://127.0.0.1:2580"
host-whitelist = "127.0.0.1"

config2.ini:

[etcd]
name = "infra1"
initial-advertise-peer-urls = "http://127.0.0.1:2480"
listen-peer-urls = "http://127.0.0.1:2480"
listen-client-urls = "http://127.0.0.1:2479"
advertise-client-urls = "http://127.0.0.1:2479"
initial-cluster = "infra0=http://127.0.0.1:2380,infra1=http://127.0.0.1:2480,infra2=http://127.0.0.1:2580"
host-whitelist = "127.0.0.1"

config2.ini:

[etcd]
name = "infra2"
initial-advertise-peer-urls = "http://127.0.0.1:2580"
listen-peer-urls = "http://127.0.0.1:2580"
listen-client-urls = "http://127.0.0.1:2579"
advertise-client-urls = "http://127.0.0.1:2579"
initial-cluster = "infra0=http://127.0.0.1:2380,infra1=http://127.0.0.1:2480,infra2=http://127.0.0.1:2580"
host-whitelist = "127.0.0.1"

Now, in each process, load the appropriate config file:

(defun become-leader (etcd)
  (setf (get-etcd "hello" etcd) "world"))

(let ((config (cdr (assoc :etcd (cl-toml:parse-file "config1.ini")))))
  (with-etcd (etcd config :on-leader #'become-leader :on-follower #'become-follower)
    (sleep 3)
    (get-etcd "hello" etcd)))

As this is a work-in-progress, details may change, and feedback is always welcome.

AG


2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 cl-etcd

Run etcd as an asynchronous inferior process.

Author

Anthony Green <green@moxielogic.com>

License

AGPL3

Version

0

Dependencies
  • alexandria (system).
  • async-process (system).
  • bordeaux-threads (system).
  • cl-base64 (system).
  • cl-json (system).
  • cl-ppcre (system).
  • str (system).
  • drakma (system).
  • flexi-streams (system).
  • split-sequence (system).
Source

cl-etcd.asd.

Child Components

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp   [Contents][Index]

3.1.1 cl-etcd/cl-etcd.asd

Source

cl-etcd.asd.

Parent Component

cl-etcd (system).

ASDF Systems

cl-etcd.


3.1.2 cl-etcd/package.lisp

Source

cl-etcd.asd.

Parent Component

cl-etcd (system).

Packages

cl-etcd.


Previous: , Up: Lisp   [Contents][Index]

3.1.3 cl-etcd/etcd.lisp

Dependency

package.lisp (file).

Source

cl-etcd.asd.

Parent Component

cl-etcd (system).

Public Interface
Internals

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 cl-etcd

Source

package.lisp.

Use List

common-lisp.

Public Interface
Internals

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Public Interface


5.1.1 Macros

Macro: with-etcd ((etcd config &key on-leader on-follower) &body body)

Create an etcd subprocess, ETCD. CONFIG is a hashtable of etcd config options: name, initial-advertise-peer-urls, listen-peer-urls, listen-client-urls, advertise-client-urls, initial-cluster, initial-cluster-state, initial-cluster-token. Otherwise, CONFIG is nil and we are creating a non-clustered etcd instance.

Package

cl-etcd.

Source

etcd.lisp.


5.1.2 Ordinary functions

Function: delete-dir-etcd (dir etcd &key error-on-missing)

Delete the directory DIR recursively from ETCD.

Package

cl-etcd.

Source

etcd.lisp.

Function: delete-etcd (key etcd &key error-on-missing)

Delete the KEY from ETCD.

Package

cl-etcd.

Source

etcd.lisp.

Function: get-etcd (key etcd)

Get the value of KEY from ETCD. Returns NIL if KEY not found. Throws an error on unexpected errors.

Package

cl-etcd.

Source

etcd.lisp.

Function: (setf get-etcd) (key etcd)

Put the KEY/VALUE pair into ETCD.

Package

cl-etcd.

Source

etcd.lisp.

Function: watch-etcd (etcd key)

Like GET-ETCD, but waits until value changes.

Package

cl-etcd.

Source

etcd.lisp.


5.1.3 Generic functions

Generic Reader: id (object)
Package

cl-etcd.

Methods
Reader Method: id ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

id.


5.1.4 Standalone methods

Method: initialize-instance :after ((etcd etcd) &key)
Source

etcd.lisp.


5.1.5 Classes

Class: etcd
Package

cl-etcd.

Source

etcd.lisp.

Direct methods
Direct slots
Slot: config
Initargs

:config

Readers

config.

Writers

This slot is read-only.

Slot: process
Readers

process.

Writers

This slot is read-only.

Slot: get-put-uri
Readers

get-put-uri.

Writers

This slot is read-only.

Slot: id
Readers

id.

Writers

This slot is read-only.

Slot: on-leader
Initargs

:on-leader

Readers

on-leader.

Writers

This slot is read-only.

Slot: on-follower
Initargs

:on-follower

Readers

on-follower.

Writers

This slot is read-only.

Slot: ready
Readers

ready.

Writers

This slot is read-only.

Slot: version
Readers

version.

Writers

This slot is read-only.

Slot: start-semaphore
Initform

(bordeaux-threads:make-semaphore)

Readers

start-semaphore.

Writers

This slot is read-only.

Slot: role
Readers

role.

Writers

This slot is read-only.


5.2 Internals


Next: , Previous: , Up: Internals   [Contents][Index]

5.2.1 Special variables

Special Variable: +became-regex+
Package

cl-etcd.

Source

etcd.lisp.

Special Variable: +etcd-member-id-regex+
Package

cl-etcd.

Source

etcd.lisp.


5.2.2 Ordinary functions

Function: %get-etcd (key etcd wait)

Get the value of KEY from ETCD, but wait for a change first if WAIT. Returns NIL if KEY not found. Throws an error on unexpected errors.

Package

cl-etcd.

Source

etcd.lisp.

Function: delete-process (process)
Package

cl-etcd.

Source

etcd.lisp.

Function: get-process-output-string (process)
Package

cl-etcd.

Source

etcd.lisp.

Function: monitor-etcd-output (etcd s)
Package

cl-etcd.

Source

etcd.lisp.

Function: process-send-input (process string)
Package

cl-etcd.

Source

etcd.lisp.

Function: run-process (command &key name output-callback output-callback-type directory)
Package

cl-etcd.

Source

etcd.lisp.

Function: write-to-buffer (process string)
Package

cl-etcd.

Source

etcd.lisp.


Next: , Previous: , Up: Internals   [Contents][Index]

5.2.3 Generic functions

Generic Reader: config (object)
Package

cl-etcd.

Methods
Reader Method: config ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

config.

Generic Reader: get-put-uri (object)
Package

cl-etcd.

Methods
Reader Method: get-put-uri ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

get-put-uri.

Generic Reader: on-follower (object)
Package

cl-etcd.

Methods
Reader Method: on-follower ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

on-follower.

Generic Reader: on-leader (object)
Package

cl-etcd.

Methods
Reader Method: on-leader ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

on-leader.

Generic Reader: process (object)
Package

cl-etcd.

Methods
Reader Method: process ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

process.

Generic Reader: process-buffer-stream (object)
Package

cl-etcd.

Methods
Reader Method: process-buffer-stream ((process process))

automatically generated reader method

Source

etcd.lisp.

Target Slot

buffer-stream.

Generic Reader: process-command (object)
Package

cl-etcd.

Methods
Reader Method: process-command ((process process))

automatically generated reader method

Source

etcd.lisp.

Target Slot

command.

Generic Reader: process-name (object)
Package

cl-etcd.

Methods
Reader Method: process-name ((process process))

automatically generated reader method

Source

etcd.lisp.

Target Slot

name.

Generic Reader: process-output-callback (object)
Package

cl-etcd.

Methods
Reader Method: process-output-callback ((process process))

automatically generated reader method

Source

etcd.lisp.

Target Slot

output-callback.

Generic Reader: process-output-callback-type (object)
Package

cl-etcd.

Methods
Reader Method: process-output-callback-type ((process process))

automatically generated reader method

Source

etcd.lisp.

Target Slot

callback-type.

Generic Reader: process-pointer (object)
Package

cl-etcd.

Methods
Reader Method: process-pointer ((process process))

automatically generated reader method

Source

etcd.lisp.

Target Slot

pointer.

Generic Reader: process-read-thread (object)
Package

cl-etcd.

Methods
Reader Method: process-read-thread ((process process))

automatically generated reader method

Source

etcd.lisp.

Target Slot

read-thread.

Generic Reader: ready (object)
Package

cl-etcd.

Methods
Reader Method: ready ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

ready.

Generic Reader: role (object)
Package

cl-etcd.

Methods
Reader Method: role ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

role.

Generic Writer: set-process-read-thread (new-value object)
Package

cl-etcd.

Methods
Writer Method: set-process-read-thread ((new-value process) process)

automatically generated writer method

Source

etcd.lisp.

Target Slot

read-thread.

Generic Reader: start-semaphore (object)
Package

cl-etcd.

Methods
Reader Method: start-semaphore ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

start-semaphore.

Generic Reader: version (object)
Package

cl-etcd.

Methods
Reader Method: version ((etcd etcd))

automatically generated reader method

Source

etcd.lisp.

Target Slot

version.


Previous: , Up: Internals   [Contents][Index]

5.2.4 Classes

Class: process
Package

cl-etcd.

Source

etcd.lisp.

Direct methods
Direct slots
Slot: pointer
Initargs

:pointer

Readers

process-pointer.

Writers

This slot is read-only.

Slot: name
Initargs

:name

Readers

process-name.

Writers

This slot is read-only.

Slot: command
Initargs

:command

Readers

process-command.

Writers

This slot is read-only.

Slot: buffer-stream
Initargs

:buffer-stream

Readers

process-buffer-stream.

Writers

This slot is read-only.

Slot: read-thread
Initargs

:read-thread

Readers

process-read-thread.

Writers

set-process-read-thread.

Slot: output-callback
Initargs

:output-callback

Readers

process-output-callback.

Writers

This slot is read-only.

Slot: callback-type
Initargs

:output-callback-type

Readers

process-output-callback-type.

Writers

This slot is read-only.


Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts


Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   %   (  
C   D   F   G   I   M   O   P   R   S   V   W  
Index Entry  Section

%
%get-etcd: Private ordinary functions

(
(setf get-etcd): Public ordinary functions

C
config: Private generic functions
config: Private generic functions

D
delete-dir-etcd: Public ordinary functions
delete-etcd: Public ordinary functions
delete-process: Private ordinary functions

F
Function, %get-etcd: Private ordinary functions
Function, (setf get-etcd): Public ordinary functions
Function, delete-dir-etcd: Public ordinary functions
Function, delete-etcd: Public ordinary functions
Function, delete-process: Private ordinary functions
Function, get-etcd: Public ordinary functions
Function, get-process-output-string: Private ordinary functions
Function, monitor-etcd-output: Private ordinary functions
Function, process-send-input: Private ordinary functions
Function, run-process: Private ordinary functions
Function, watch-etcd: Public ordinary functions
Function, write-to-buffer: Private ordinary functions

G
Generic Function, config: Private generic functions
Generic Function, get-put-uri: Private generic functions
Generic Function, id: Public generic functions
Generic Function, on-follower: Private generic functions
Generic Function, on-leader: Private generic functions
Generic Function, process: Private generic functions
Generic Function, process-buffer-stream: Private generic functions
Generic Function, process-command: Private generic functions
Generic Function, process-name: Private generic functions
Generic Function, process-output-callback: Private generic functions
Generic Function, process-output-callback-type: Private generic functions
Generic Function, process-pointer: Private generic functions
Generic Function, process-read-thread: Private generic functions
Generic Function, ready: Private generic functions
Generic Function, role: Private generic functions
Generic Function, set-process-read-thread: Private generic functions
Generic Function, start-semaphore: Private generic functions
Generic Function, version: Private generic functions
get-etcd: Public ordinary functions
get-process-output-string: Private ordinary functions
get-put-uri: Private generic functions
get-put-uri: Private generic functions

I
id: Public generic functions
id: Public generic functions
initialize-instance: Public standalone methods

M
Macro, with-etcd: Public macros
Method, config: Private generic functions
Method, get-put-uri: Private generic functions
Method, id: Public generic functions
Method, initialize-instance: Public standalone methods
Method, on-follower: Private generic functions
Method, on-leader: Private generic functions
Method, process: Private generic functions
Method, process-buffer-stream: Private generic functions
Method, process-command: Private generic functions
Method, process-name: Private generic functions
Method, process-output-callback: Private generic functions
Method, process-output-callback-type: Private generic functions
Method, process-pointer: Private generic functions
Method, process-read-thread: Private generic functions
Method, ready: Private generic functions
Method, role: Private generic functions
Method, set-process-read-thread: Private generic functions
Method, start-semaphore: Private generic functions
Method, version: Private generic functions
monitor-etcd-output: Private ordinary functions

O
on-follower: Private generic functions
on-follower: Private generic functions
on-leader: Private generic functions
on-leader: Private generic functions

P
process: Private generic functions
process: Private generic functions
process-buffer-stream: Private generic functions
process-buffer-stream: Private generic functions
process-command: Private generic functions
process-command: Private generic functions
process-name: Private generic functions
process-name: Private generic functions
process-output-callback: Private generic functions
process-output-callback: Private generic functions
process-output-callback-type: Private generic functions
process-output-callback-type: Private generic functions
process-pointer: Private generic functions
process-pointer: Private generic functions
process-read-thread: Private generic functions
process-read-thread: Private generic functions
process-send-input: Private ordinary functions

R
ready: Private generic functions
ready: Private generic functions
role: Private generic functions
role: Private generic functions
run-process: Private ordinary functions

S
set-process-read-thread: Private generic functions
set-process-read-thread: Private generic functions
start-semaphore: Private generic functions
start-semaphore: Private generic functions

V
version: Private generic functions
version: Private generic functions

W
watch-etcd: Public ordinary functions
with-etcd: Public macros
write-to-buffer: Private ordinary functions

Jump to:   %   (  
C   D   F   G   I   M   O   P   R   S   V   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   +  
B   C   G   I   N   O   P   R   S   V  
Index Entry  Section

+
+became-regex+: Private special variables
+etcd-member-id-regex+: Private special variables

B
buffer-stream: Private classes

C
callback-type: Private classes
command: Private classes
config: Public classes

G
get-put-uri: Public classes

I
id: Public classes

N
name: Private classes

O
on-follower: Public classes
on-leader: Public classes
output-callback: Private classes

P
pointer: Private classes
process: Public classes

R
read-thread: Private classes
ready: Public classes
role: Public classes

S
Slot, buffer-stream: Private classes
Slot, callback-type: Private classes
Slot, command: Private classes
Slot, config: Public classes
Slot, get-put-uri: Public classes
Slot, id: Public classes
Slot, name: Private classes
Slot, on-follower: Public classes
Slot, on-leader: Public classes
Slot, output-callback: Private classes
Slot, pointer: Private classes
Slot, process: Public classes
Slot, read-thread: Private classes
Slot, ready: Public classes
Slot, role: Public classes
Slot, start-semaphore: Public classes
Slot, version: Public classes
Special Variable, +became-regex+: Private special variables
Special Variable, +etcd-member-id-regex+: Private special variables
start-semaphore: Public classes

V
version: Public classes

Jump to:   +  
B   C   G   I   N   O   P   R   S   V