Next: Introduction, Previous: (dir), Up: (dir) [Contents][Index]
This is the cl-beanstalk Reference Manual, generated automatically by Declt version 3.0 "Montgomery Scott" on Mon Apr 19 14:32:15 2021 GMT+0.
• Introduction | What cl-beanstalk is all about | |
• Systems | The systems documentation | |
• Files | The files documentation | |
• Packages | The packages documentation | |
• Definitions | The symbols documentation | |
• Indexes | Concepts, functions, variables and data types |
CL-BEANSTALK implements the version 1.4.2 of the Beanstalk queuing service protocol. It allows splitting programs into "worker" parts that execute units of work asynchronously, by fetching them from the beanstalk daemon.
CL-BEANSTALK requires three CL libraries to run:
All of these libraries are available via quicklisp.
On the lisp REPL, with quicklisp loaded, run:
(ql:quickload :cl-beanstalk)
And you're done!
git clone git://github.com/antifuchs/cl-beanstalk.git
Then, add the directory to your asdf system definition directory. After this, asdf:load-system should be able to find and load cl-beanstalk.
First, all protocol functions reside in the BEANSTALK: package. Their names are the same as in the beanstalk protocol documentation.
Second, there are several error conditions specified in the protocol document, and all of these map to a specific lisp condition. All error conditions are subtypes of BEANSTALK:BEANSTALK-ERROR
. There are several such error conditions, for an exhaustive list see the file package.lisp.
To start using CL-BEANSTALK, you need to connect to a beanstalk daemon. To do this, it provides functions BEANSTALK:CONNECT
and BEANSTALK:DISCONNECT
, and the macro BEANSTALK:WITH-BEANSTALK-CONNECTION
:
(beanstalk:with-beanstalk-connection (conn "localhost" 11300)
;; Put jobs into a specific tube:
(beanstalk:use conn "URLs-to-fetch")
;; Put two pieces of work into the tube, both at priority 100,
;; no delay, with 180 seconds maximum running time:
(beanstalk:put conn 100 0 180 "http://boinkor.net/")
(beanstalk:put conn 100 0 180 "http://planet.lisp.org/"))
Workers need to get their units of work from somewhere, and they need to tell the queue server that they have performed (or failed to perform that work). Here's a simple (meaning, untested) worker implementation that fetches URLs from the tube in the last example and handles them in some way:
(beanstalk:with-beanstalk-connection (conn "localhost" 11300)
;; Get jobs from a specific tube:
(beanstalk:watch conn "URLs-to-fetch")
(loop
;; reserve the next job:
(multiple-value-bind (url id status) (beanstalk:reserve conn :timeout 1)
;; If no new jobs arrived within 1 second, quit:
(when (eql status :timed-out) (return))
(handle-response (drakma:http-request url))
;; The job is done, remove it from the queue:
(beanstalk:delete conn id))))
Note that this code can run on several worker processes (indeed, worker machines) at once, and perform several thousands units of work in parallel.
For a complete reference, see the beanstalk protocol documentation.
Whenever it makes sense to do so, a function signals a non-error condition to highlight an untypical event: For example, if a job is buried instead of put on the "ready" queue because the server can't reverve enough memory, that results in a condition being signaled via WARN or SIGNAL.
These functions will signal out-of-band information from the queue server:
BEANSTALK:PUT
and BEANSTALK:RELEASE
will signal beanstalk:buried-job
if the queue server ran out of memory while allocating a bigger priority queue. This (along with the third return value, :buried) indicates that the job was not put on the "ready" queue, but is now buried.BEANSTALK:RESERVE
will signal beanstalk:deadline-soon
indicating that a job previously reserved in this worker process (a job that was neither deleted nor released) job is within 1 second of its allocated time to run deadline. The worker can choose to extend the deadline by applying BEANSTALK:TOUCH
to the reserved job ID.Next: Files, Previous: Introduction, Up: Top [Contents][Index]
The main system appears first, followed by any subsystem dependency.
• The cl-beanstalk system |
An interface to the beanstalk queue server
cl-beanstalk.asd (file)
Files are sorted by type and then listed depth-first from the systems components trees.
• Lisp files |
• The cl-beanstalk.asd file | ||
• The cl-beanstalk/package.lisp file | ||
• The cl-beanstalk/beanstalk.lisp file |
Next: The cl-beanstalk/package․lisp file, Previous: Lisp files, Up: Lisp files [Contents][Index]
cl-beanstalk.asd
cl-beanstalk (system)
Next: The cl-beanstalk/beanstalk․lisp file, Previous: The cl-beanstalk․asd file, Up: Lisp files [Contents][Index]
cl-beanstalk (system)
package.lisp
Previous: The cl-beanstalk/package․lisp file, Up: Lisp files [Contents][Index]
package.lisp (file)
cl-beanstalk (system)
beanstalk.lisp
Next: Definitions, Previous: Files, Up: Top [Contents][Index]
Packages are listed by definition order.
• The beanstalk-internal package | ||
• The beanstalk package |
Next: The beanstalk package, Previous: Packages, Up: Packages [Contents][Index]
package.lisp (file)
common-lisp
Previous: The beanstalk-internal package, Up: Packages [Contents][Index]
package.lisp (file)
Definitions are sorted by export status, category, package, and then by lexicographic order.
• Exported definitions | ||
• Internal definitions |
Next: Internal definitions, Previous: Definitions, Up: Definitions [Contents][Index]
• Exported macros | ||
• Exported functions | ||
• Exported conditions |
Next: Exported functions, Previous: Exported definitions, Up: Exported definitions [Contents][Index]
Open a beanstalk connection to HOST:PORT for as long as control is inside BODY.
beanstalk.lisp (file)
Next: Exported conditions, Previous: Exported macros, Up: Exported definitions [Contents][Index]
beanstalk.lisp (file)
Connect to a beanstalk daemon running on HOST:PORT.
beanstalk.lisp (file)
beanstalk.lisp (file)
Disconnect the beanstalk connection.
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
Alias for BEANSTALK:DISCONNECT.
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
Previous: Exported functions, Up: Exported definitions [Contents][Index]
beanstalk.lisp (file)
beanstalk-error (condition)
document (method)
:document
document (generic function)
Previous: Exported definitions, Up: Definitions [Contents][Index]
• Internal special variables | ||
• Internal macros | ||
• Internal functions | ||
• Internal generic functions | ||
• Internal conditions | ||
• Internal classes | ||
• Internal types |
Next: Internal macros, Previous: Internal definitions, Up: Internal definitions [Contents][Index]
beanstalk.lisp (file)
beanstalk.lisp (file)
Next: Internal functions, Previous: Internal special variables, Up: Internal definitions [Contents][Index]
beanstalk.lisp (file)
beanstalk.lisp (file)
Next: Internal generic functions, Previous: Internal macros, Up: Internal definitions [Contents][Index]
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
Next: Internal conditions, Previous: Internal functions, Up: Internal definitions [Contents][Index]
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
automatically generated reader method
beanstalk.lisp (file)
automatically generated writer method
beanstalk.lisp (file)
beanstalk.lisp (file)
automatically generated reader method
beanstalk.lisp (file)
automatically generated writer method
beanstalk.lisp (file)
automatically generated reader method
beanstalk.lisp (file)
automatically generated writer method
beanstalk.lisp (file)
beanstalk.lisp (file)
Next: Internal classes, Previous: Internal generic functions, Up: Internal definitions [Contents][Index]
beanstalk.lisp (file)
beanstalk-error (condition)
expected-crlf (condition)
beanstalk.lisp (file)
condition (condition)
reply (method)
:reply
reply (generic function)
beanstalk.lisp (file)
condition (condition)
beanstalk.lisp (file)
beanstalk-error (condition)
beanstalk.lisp (file)
beanstalk-error (condition)
beanstalk.lisp (file)
beanstalk-error (condition)
beanstalk.lisp (file)
warning (condition)
buried-job-id (method)
:id
buried-job-id (generic function)
beanstalk.lisp (file)
warning (condition)
beanstalk.lisp (file)
bad-message-format (condition)
beanstalk.lisp (file)
beanstalk-error (condition)
unknown-command (method)
beanstalk-internal::command
unknown-command (generic function)
Next: Internal types, Previous: Internal conditions, Up: Internal definitions [Contents][Index]
beanstalk.lisp (file)
standard-object (class)
:socket
socket-of (generic function)
(setf socket-of) (generic function)
:stream
stream-of (generic function)
(setf stream-of) (generic function)
:name
name-of (generic function)
(setf name-of) (generic function)
Previous: Internal classes, Up: Internal definitions [Contents][Index]
beanstalk.lisp (file)
beanstalk.lisp (file)
beanstalk.lisp (file)
Previous: Definitions, Up: Top [Contents][Index]
• Concept index | ||
• Function index | ||
• Variable index | ||
• Data type index |
Next: Function index, Previous: Indexes, Up: Indexes [Contents][Index]
Jump to: | C F L |
---|
Jump to: | C F L |
---|
Next: Variable index, Previous: Concept index, Up: Indexes [Contents][Index]
Jump to: | (
B C D F G I K L M N P Q R S T U W |
---|
Jump to: | (
B C D F G I K L M N P Q R S T U W |
---|
Next: Data type index, Previous: Function index, Up: Indexes [Contents][Index]
Jump to: | *
D I N R S U |
---|
Jump to: | *
D I N R S U |
---|
Previous: Variable index, Up: Indexes [Contents][Index]
Jump to: | B C D E P S T U Y |
---|
Jump to: | B C D E P S T U Y |
---|