The cl-beanstalk Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The cl-beanstalk Reference Manual

This is the cl-beanstalk Reference Manual, generated automatically by Declt version 2.3 "Robert April" on Tue Feb 20 07:53:22 2018 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

CL-BEANSTALK, an Implementation of the beanstalk v1.4.2 protocol in Common Lisp

CL-BEANSTALK implements the version 1.4.2 of the Beanstalk queuing service protocol. It allows splitting programs into "worker" parts that execute units of work asynchronously, by fetching them from the beanstalk daemon.

Installing CL-BEANSTALK

CL-BEANSTALK requires three CL libraries to run:

  1. usocket
  2. flexi-streams
  3. split-sequence

All of these libraries are available via quicklisp.

Getting cl-beanstalk from quicklisp

On the lisp REPL, with quicklisp loaded, run:

    (ql:quickload :cl-beanstalk)

And you're done!

Getting cl-beanstalk from source

git clone git://github.com/antifuchs/cl-beanstalk.git

Then, add the directory to your asdf system definition directory. After this, asdf:load-system should be able to find and load cl-beanstalk.

Using CL-BEANSTALK

First, all protocol functions reside in the BEANSTALK: package. Their names are the same as in the beanstalk protocol documentation.

Second, there are several error conditions specified in the protocol document, and all of these map to a specific lisp condition. All error conditions are subtypes of BEANSTALK:BEANSTALK-ERROR. There are several such error conditions, for an exhaustive list see the file package.lisp.

Connecting to the queue server and a little example

To start using CL-BEANSTALK, you need to connect to a beanstalk daemon. To do this, it provides functions BEANSTALK:CONNECT and BEANSTALK:DISCONNECT, and the macro BEANSTALK:WITH-BEANSTALK-CONNECTION:

(beanstalk:with-beanstalk-connection (conn "localhost" 11300)
  ;; Put jobs into a specific tube:
  (beanstalk:use conn "URLs-to-fetch")
  ;; Put two pieces of work into the tube, both at priority 100, 
  ;; no delay, with 180 seconds maximum running time:
  (beanstalk:put conn 100 0 180 "http://boinkor.net/")
  (beanstalk:put conn 100 0 180 "http://planet.lisp.org/"))

Getting work from the queue server

Workers need to get their units of work from somewhere, and they need to tell the queue server that they have performed (or failed to perform that work). Here's a simple (meaning, untested) worker implementation that fetches URLs from the tube in the last example and handles them in some way:

(beanstalk:with-beanstalk-connection (conn "localhost" 11300)
	  ;; Get jobs from a specific tube:
	  (beanstalk:watch conn "URLs-to-fetch")
	  (loop 
	    ;; reserve the next job:
	    (multiple-value-bind (url id status) (beanstalk:reserve conn :timeout 1)
              ;; If no new jobs arrived within 1 second, quit:
              (when (eql status :timed-out) (return))
              (handle-response (drakma:http-request url))
              ;; The job is done, remove it from the queue:
              (beanstalk:delete conn id))))

Note that this code can run on several worker processes (indeed, worker machines) at once, and perform several thousands units of work in parallel.

For a complete reference, see the beanstalk protocol documentation.

Conditions and errors

Whenever it makes sense to do so, a function signals a non-error condition to highlight an untypical event: For example, if a job is buried instead of put on the "ready" queue because the server can't reverve enough memory, that results in a condition being signaled via WARN or SIGNAL.

These functions will signal out-of-band information from the queue server:


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 cl-beanstalk

Description

An interface to the beanstalk queue server

Dependencies
Source

cl-beanstalk.asd (file)

Components

Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 cl-beanstalk.asd

Location

cl-beanstalk.asd

Systems

cl-beanstalk (system)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 cl-beanstalk/package.lisp

Parent

cl-beanstalk (system)

Location

package.lisp

Packages

Previous: , Up: Lisp files   [Contents][Index]

3.1.3 cl-beanstalk/beanstalk.lisp

Dependency

package.lisp (file)

Parent

cl-beanstalk (system)

Location

beanstalk.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Next: , Previous: , Up: Packages   [Contents][Index]

4.1 beanstalk-internal

Source

package.lisp (file)

Use List

common-lisp

Internal Definitions

Previous: , Up: Packages   [Contents][Index]

4.2 beanstalk

Source

package.lisp (file)

Exported Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Macros

Macro: with-beanstalk-connection (CONNECTION HOST PORT &rest INITARGS) &body BODY

Open a beanstalk connection to HOST:PORT for as long as control is inside BODY.

Package

beanstalk

Source

beanstalk.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.2 Functions

Function: bury CONNECTION ID PRI
Package

beanstalk

Source

beanstalk.lisp (file)

Function: connect HOST PORT &rest INITARGS

Connect to a beanstalk daemon running on HOST:PORT.

Package

beanstalk

Source

beanstalk.lisp (file)

Function: delete CONNECTION ID
Package

beanstalk

Source

beanstalk.lisp (file)

Function: disconnect CONNECTION

Disconnect the beanstalk connection.

Package

beanstalk

Source

beanstalk.lisp (file)

Function: ignore CONNECTION TUBE
Package

beanstalk

Source

beanstalk.lisp (file)

Function: kick CONNECTION BOUND
Package

beanstalk

Source

beanstalk.lisp (file)

Function: list-tube-used CONNECTION
Package

beanstalk

Source

beanstalk.lisp (file)

Function: list-tubes CONNECTION
Package

beanstalk

Source

beanstalk.lisp (file)

Function: list-tubes-watched CONNECTION
Package

beanstalk

Source

beanstalk.lisp (file)

Function: peek CONNECTION ID &key EXTERNAL-FORMAT
Package

beanstalk

Source

beanstalk.lisp (file)

Function: peek-buried CONNECTION &key EXTERNAL-FORMAT
Package

beanstalk

Source

beanstalk.lisp (file)

Function: peek-delayed CONNECTION &key EXTERNAL-FORMAT
Package

beanstalk

Source

beanstalk.lisp (file)

Function: peek-ready CONNECTION &key EXTERNAL-FORMAT
Package

beanstalk

Source

beanstalk.lisp (file)

Function: put CONNECTION PRI DELAY TTR MESSAGE &key EXTERNAL-FORMAT
Package

beanstalk

Source

beanstalk.lisp (file)

Function: quit CONNECTION

Alias for BEANSTALK:DISCONNECT.

Package

beanstalk

Source

beanstalk.lisp (file)

Function: release CONNECTION ID PRI DELAY
Package

beanstalk

Source

beanstalk.lisp (file)

Function: reserve CONNECTION &key TIMEOUT EXTERNAL-FORMAT
Package

beanstalk

Source

beanstalk.lisp (file)

Function: stats CONNECTION
Package

beanstalk

Source

beanstalk.lisp (file)

Function: stats-job CONNECTION ID
Package

beanstalk

Source

beanstalk.lisp (file)

Function: stats-tube CONNECTION TUBE
Package

beanstalk

Source

beanstalk.lisp (file)

Function: touch CONNECTION ID
Package

beanstalk

Source

beanstalk.lisp (file)

Function: use CONNECTION NAME
Package

beanstalk

Source

beanstalk.lisp (file)

Function: watch CONNECTION TUBE
Package

beanstalk

Source

beanstalk.lisp (file)


Previous: , Up: Exported definitions   [Contents][Index]

5.1.3 Conditions

Condition: yaml-parsing-failed ()
Package

beanstalk

Source

beanstalk.lisp (file)

Direct superclasses

beanstalk-error (condition)

Direct methods

document (method)

Direct slots
Slot: document
Initargs

:document

Readers

document (generic function)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Special variables

Special Variable: *debug-protocol*
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Special Variable: *yaml-external-format*
Package

beanstalk-internal

Source

beanstalk.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Macros

Macro: command-reply-case (REPLY CONNECTION COMMAND DATA-PRODUCER &rest ARGS) &body CLAUSES
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Macro: proto-debug DATUM &rest ARGS
Package

beanstalk-internal

Source

beanstalk.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Functions

Function: converse CONNECTION COMMAND DATA-PRODUCER &rest ARGS
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Function: length-below-200 STRING
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Function: parse-yaml-data N-BYTES STREAM STRING-FIELDS
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Function: proto-debug* DATUM &rest ARGS
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Function: read-data-reply BYTES-AS-STRING STREAM EXTERNAL-FORMAT
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Function: read-reply CONNECTION
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Function: send-command CONNECTION NAME &rest ARGS
Package

beanstalk-internal

Source

beanstalk.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.4 Generic functions

Generic Function: buried-job-id CONDITION
Package

beanstalk-internal

Methods
Method: buried-job-id (CONDITION buried-job)
Source

beanstalk.lisp (file)

Generic Function: convert-message-out MESSAGE CONNECTION EXTERNAL-FORMAT
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Methods
Method: convert-message-out (MESSAGE string) CONN EXTERNAL-FORMAT
Method: convert-message-out (MESSAGE vector) CONN EXTERNAL-FORMAT
Generic Function: document CONDITION
Package

beanstalk-internal

Methods
Method: document (CONDITION yaml-parsing-failed)
Source

beanstalk.lisp (file)

Generic Function: name-of OBJECT
Generic Function: (setf name-of) NEW-VALUE OBJECT
Package

beanstalk-internal

Methods
Method: name-of (BEANSTALK-CONNECTION beanstalk-connection)

automatically generated reader method

Source

beanstalk.lisp (file)

Method: (setf name-of) NEW-VALUE (BEANSTALK-CONNECTION beanstalk-connection)

automatically generated writer method

Source

beanstalk.lisp (file)

Generic Function: reply CONDITION
Package

beanstalk-internal

Methods
Method: reply (CONDITION bad-reply)
Source

beanstalk.lisp (file)

Generic Function: socket-of OBJECT
Generic Function: (setf socket-of) NEW-VALUE OBJECT
Package

beanstalk-internal

Methods
Method: socket-of (BEANSTALK-CONNECTION beanstalk-connection)

automatically generated reader method

Source

beanstalk.lisp (file)

Method: (setf socket-of) NEW-VALUE (BEANSTALK-CONNECTION beanstalk-connection)

automatically generated writer method

Source

beanstalk.lisp (file)

Generic Function: stream-of OBJECT
Generic Function: (setf stream-of) NEW-VALUE OBJECT
Package

beanstalk-internal

Methods
Method: stream-of (BEANSTALK-CONNECTION beanstalk-connection)

automatically generated reader method

Source

beanstalk.lisp (file)

Method: (setf stream-of) NEW-VALUE (BEANSTALK-CONNECTION beanstalk-connection)

automatically generated writer method

Source

beanstalk.lisp (file)

Generic Function: unknown-command CONDITION
Package

beanstalk-internal

Methods
Method: unknown-command (CONDITION unknown-command)
Source

beanstalk.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.5 Conditions

Condition: bad-message-format ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

beanstalk-error (condition)

Direct subclasses

expected-crlf (condition)

Condition: bad-reply ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

condition (condition)

Direct methods

reply (method)

Direct slots
Slot: reply
Initargs

:reply

Readers

reply (generic function)

Condition: beanstalk-error ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

condition (condition)

Direct subclasses
Condition: beanstalkd-draining ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

beanstalk-error (condition)

Condition: beanstalkd-internal-error ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

beanstalk-error (condition)

Condition: beanstalkd-out-of-memory ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

beanstalk-error (condition)

Condition: buried-job ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

warning (condition)

Direct methods

buried-job-id (method)

Direct slots
Slot: id
Initargs

:id

Readers

buried-job-id (generic function)

Condition: deadline-soon ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

warning (condition)

Condition: expected-crlf ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

bad-message-format (condition)

Condition: unknown-command ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

beanstalk-error (condition)

Direct methods

unknown-command (method)

Direct slots
Slot: unknown-command
Initargs

beanstalk-internal::command

Readers

unknown-command (generic function)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.6 Classes

Class: beanstalk-connection ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
  • print-object (method)
  • name-of (method)
  • name-of (method)
  • stream-of (method)
  • stream-of (method)
  • socket-of (method)
  • socket-of (method)
Direct slots
Slot: socket
Initargs

:socket

Readers

socket-of (generic function)

Writers

(setf socket-of) (generic function)

Slot: stream
Initargs

:stream

Readers

stream-of (generic function)

Writers

(setf stream-of) (generic function)

Slot: name
Initargs

:name

Readers

name-of (generic function)

Writers

(setf name-of) (generic function)


Previous: , Up: Internal definitions   [Contents][Index]

5.2.7 Types

Type: pri ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Type: time-spec ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)

Type: tube-name ()
Package

beanstalk-internal

Source

beanstalk.lisp (file)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   C   F   L  
Index Entry  Section

C
cl-beanstalk.asd: The cl-beanstalk<dot>asd file
cl-beanstalk/beanstalk.lisp: The cl-beanstalk/beanstalk<dot>lisp file
cl-beanstalk/package.lisp: The cl-beanstalk/package<dot>lisp file

F
File, Lisp, cl-beanstalk.asd: The cl-beanstalk<dot>asd file
File, Lisp, cl-beanstalk/beanstalk.lisp: The cl-beanstalk/beanstalk<dot>lisp file
File, Lisp, cl-beanstalk/package.lisp: The cl-beanstalk/package<dot>lisp file

L
Lisp File, cl-beanstalk.asd: The cl-beanstalk<dot>asd file
Lisp File, cl-beanstalk/beanstalk.lisp: The cl-beanstalk/beanstalk<dot>lisp file
Lisp File, cl-beanstalk/package.lisp: The cl-beanstalk/package<dot>lisp file

Jump to:   C   F   L  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   (  
B   C   D   F   G   I   K   L   M   N   P   Q   R   S   T   U   W  
Index Entry  Section

(
(setf name-of): Internal generic functions
(setf name-of): Internal generic functions
(setf socket-of): Internal generic functions
(setf socket-of): Internal generic functions
(setf stream-of): Internal generic functions
(setf stream-of): Internal generic functions

B
buried-job-id: Internal generic functions
buried-job-id: Internal generic functions
bury: Exported functions

C
command-reply-case: Internal macros
connect: Exported functions
converse: Internal functions
convert-message-out: Internal generic functions
convert-message-out: Internal generic functions
convert-message-out: Internal generic functions

D
delete: Exported functions
disconnect: Exported functions
document: Internal generic functions
document: Internal generic functions

F
Function, bury: Exported functions
Function, connect: Exported functions
Function, converse: Internal functions
Function, delete: Exported functions
Function, disconnect: Exported functions
Function, ignore: Exported functions
Function, kick: Exported functions
Function, length-below-200: Internal functions
Function, list-tube-used: Exported functions
Function, list-tubes: Exported functions
Function, list-tubes-watched: Exported functions
Function, parse-yaml-data: Internal functions
Function, peek: Exported functions
Function, peek-buried: Exported functions
Function, peek-delayed: Exported functions
Function, peek-ready: Exported functions
Function, proto-debug*: Internal functions
Function, put: Exported functions
Function, quit: Exported functions
Function, read-data-reply: Internal functions
Function, read-reply: Internal functions
Function, release: Exported functions
Function, reserve: Exported functions
Function, send-command: Internal functions
Function, stats: Exported functions
Function, stats-job: Exported functions
Function, stats-tube: Exported functions
Function, touch: Exported functions
Function, use: Exported functions
Function, watch: Exported functions

G
Generic Function, (setf name-of): Internal generic functions
Generic Function, (setf socket-of): Internal generic functions
Generic Function, (setf stream-of): Internal generic functions
Generic Function, buried-job-id: Internal generic functions
Generic Function, convert-message-out: Internal generic functions
Generic Function, document: Internal generic functions
Generic Function, name-of: Internal generic functions
Generic Function, reply: Internal generic functions
Generic Function, socket-of: Internal generic functions
Generic Function, stream-of: Internal generic functions
Generic Function, unknown-command: Internal generic functions

I
ignore: Exported functions

K
kick: Exported functions

L
length-below-200: Internal functions
list-tube-used: Exported functions
list-tubes: Exported functions
list-tubes-watched: Exported functions

M
Macro, command-reply-case: Internal macros
Macro, proto-debug: Internal macros
Macro, with-beanstalk-connection: Exported macros
Method, (setf name-of): Internal generic functions
Method, (setf socket-of): Internal generic functions
Method, (setf stream-of): Internal generic functions
Method, buried-job-id: Internal generic functions
Method, convert-message-out: Internal generic functions
Method, convert-message-out: Internal generic functions
Method, document: Internal generic functions
Method, name-of: Internal generic functions
Method, reply: Internal generic functions
Method, socket-of: Internal generic functions
Method, stream-of: Internal generic functions
Method, unknown-command: Internal generic functions

N
name-of: Internal generic functions
name-of: Internal generic functions

P
parse-yaml-data: Internal functions
peek: Exported functions
peek-buried: Exported functions
peek-delayed: Exported functions
peek-ready: Exported functions
proto-debug: Internal macros
proto-debug*: Internal functions
put: Exported functions

Q
quit: Exported functions

R
read-data-reply: Internal functions
read-reply: Internal functions
release: Exported functions
reply: Internal generic functions
reply: Internal generic functions
reserve: Exported functions

S
send-command: Internal functions
socket-of: Internal generic functions
socket-of: Internal generic functions
stats: Exported functions
stats-job: Exported functions
stats-tube: Exported functions
stream-of: Internal generic functions
stream-of: Internal generic functions

T
touch: Exported functions

U
unknown-command: Internal generic functions
unknown-command: Internal generic functions
use: Exported functions

W
watch: Exported functions
with-beanstalk-connection: Exported macros

Jump to:   (  
B   C   D   F   G   I   K   L   M   N   P   Q   R   S   T   U   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *  
D   I   N   R   S   U  
Index Entry  Section

*
*debug-protocol*: Internal special variables
*yaml-external-format*: Internal special variables

D
document: Exported conditions

I
id: Internal conditions

N
name: Internal classes

R
reply: Internal conditions

S
Slot, document: Exported conditions
Slot, id: Internal conditions
Slot, name: Internal classes
Slot, reply: Internal conditions
Slot, socket: Internal classes
Slot, stream: Internal classes
Slot, unknown-command: Internal conditions
socket: Internal classes
Special Variable, *debug-protocol*: Internal special variables
Special Variable, *yaml-external-format*: Internal special variables
stream: Internal classes

U
unknown-command: Internal conditions

Jump to:   *  
D   I   N   R   S   U  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   B   C   D   E   P   S   T   U   Y  
Index Entry  Section

B
bad-message-format: Internal conditions
bad-reply: Internal conditions
beanstalk: The beanstalk package
beanstalk-connection: Internal classes
beanstalk-error: Internal conditions
beanstalk-internal: The beanstalk-internal package
beanstalkd-draining: Internal conditions
beanstalkd-internal-error: Internal conditions
beanstalkd-out-of-memory: Internal conditions
buried-job: Internal conditions

C
cl-beanstalk: The cl-beanstalk system
Class, beanstalk-connection: Internal classes
Condition, bad-message-format: Internal conditions
Condition, bad-reply: Internal conditions
Condition, beanstalk-error: Internal conditions
Condition, beanstalkd-draining: Internal conditions
Condition, beanstalkd-internal-error: Internal conditions
Condition, beanstalkd-out-of-memory: Internal conditions
Condition, buried-job: Internal conditions
Condition, deadline-soon: Internal conditions
Condition, expected-crlf: Internal conditions
Condition, unknown-command: Internal conditions
Condition, yaml-parsing-failed: Exported conditions

D
deadline-soon: Internal conditions

E
expected-crlf: Internal conditions

P
Package, beanstalk: The beanstalk package
Package, beanstalk-internal: The beanstalk-internal package
pri: Internal types

S
System, cl-beanstalk: The cl-beanstalk system

T
time-spec: Internal types
tube-name: Internal types
Type, pri: Internal types
Type, time-spec: Internal types
Type, tube-name: Internal types

U
unknown-command: Internal conditions

Y
yaml-parsing-failed: Exported conditions

Jump to:   B   C   D   E   P   S   T   U   Y