The legion Reference Manual

Table of Contents

The legion Reference Manual

This is the legion Reference Manual, version 0.1.1, generated automatically by Declt version 2.4 "Will Decker" on Wed Jun 20 12:04:41 2018 GMT+0.

1 Introduction

Legion

Jesus asked him, "What is your name?" And he said, "My name is Legion, for we are many." ── Gospel of Mark chapter 5, verse 9

Usage

Worker

(defparameter *worker*
  (make-worker
    (lambda (job)
      (format t "Processed: ~S~%" job))))

(start *worker*)

(add-job *worker* 10)
(add-job *worker* "Hi")

(stop *worker*)

Cluster

(defparameter *cluster*
  (make-cluster 4
    (lambda (job)
      (format t "Processed: ~S~%" job))))

(start *cluster*)

(add-job *cluster* 10)
(add-job *cluster* "Hi")

(stop *cluster*)

NOTE: Cluster doesn't guarantee the order of processing jobs.

Functions

[Class] worker

Base class of workers.

You can specify the value by specifying :queue.

[Function] (worker-status worker)

Return the worker's status which is specifically one of :running, :idle, :shutting and :shutdown.

[Function] (worker-queue-count worker)

Return the number of outstanding jobs of the worker.

[Method] (start worker-or-cluster)

Start the given worker or cluster to process jobs.

[Method] (stop worker-or-cluster)

Stop the given worker or cluster after processing its queued jobs.

[Method] (kill worker-or-cluster)

Stop the given worker or cluster immediately (outstanding jobs will be remained in its queue).

[Method] (add-job worker-or-cluster val)

Enqueue a new job val which will be passed to a function specified for make-worker.

[Method] (fetch-job worker) => val, existsp

Dequeue a job from worker's queue.

[Class] cluster

Base class of clusters.

[Function] (cluster-status cluster)

Return the cluster's status which is one of :running, :shutting and :shutdown.

[Function] (start-cluster cluster)

Start workers of cluster.

[Function] (cluster-workers cluster) => workers-array

Return workers of cluster in simple-array.

Author

• Eitaro Fukamachi (e.arrows@gmail.com)

Copyright

Copyright (c) 2015 Eitaro Fukamachi (e.arrows@gmail.com)

License

Licensed under the BSD 3-Clause License.

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 legion

Author

Eitaro Fukamachi

License

BSD 3-Clause

Description

Simple worker threads with a queue.

Version

0.1.1

Dependencies

• vom
• legion/main (system)

Source

legion.asd (file)

2.2 legion/main

Dependencies

• legion/worker (system)
• legion/cluster (system)
• legion/error (system)

Source

legion.asd (file)

Component

lisp.lisp (file)

2.3 legion/cluster

Dependencies

• legion/worker (system)
• legion/queue (system)
• legion/error (system)
• bordeaux-threads

Source

legion.asd (file)

Component

lisp.lisp (file)

2.4 legion/worker

Dependencies

• legion/queue (system)
• bordeaux-threads

Source

legion.asd (file)

Component

lisp.lisp (file)

2.5 legion/queue

Dependencies

• cl-speedy-queue
• bordeaux-threads

Source

legion.asd (file)

Component

lisp.lisp (file)

2.6 legion/error

Source

legion.asd (file)

Component

lisp.lisp (file)

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

3.1 Lisp

3.1.1 legion.asd

Location

legion.asd

Systems

• legion (system)
• legion/main (system)
• legion/cluster (system)
• legion/worker (system)
• legion/queue (system)
• legion/error (system)

3.1.2 legion/main/lisp.lisp

Parent

legion/main (system)

Location

main.lisp

Packages

legion

Exported Definitions

• make-cluster (function)
• make-worker (function)

3.1.3 legion/cluster/lisp.lisp

Parent

legion/cluster (system)

Location

cluster.lisp

Packages

legion/cluster

Exported Definitions

• add-job (method)
• cluster (class)
• cluster-status (method)
• (setf cluster-status) (method)
• cluster-workers (method)
• kill (method)
• start (method)
• stop (method)

Internal Definitions

• cluster-queue (method)
• (setf cluster-queue) (method)

3.1.4 legion/worker/lisp.lisp

Parent

legion/worker (system)

Location

worker.lisp

Packages

legion/worker

Exported Definitions

• add-job (generic function)
• add-job (method)
• fetch-job (generic function)
• fetch-job (method)
• kill (generic function)
• kill (method)
• process-job (generic function)
• process-job (method)
• start (generic function)
• start (method)
• stop (generic function)
• stop (method)
• wakenup (function)
• worker (class)
• worker-queue (method)
• (setf worker-queue) (method)
• worker-queue-count (function)
• worker-status (method)
• (setf worker-status) (method)

Internal Definitions

• run (generic function)
• run (method)
• run (method)
• worker-process-fn (method)
• worker-thread (method)
• worker-wait-cond (method)
• worker-wait-lock (method)

3.1.5 legion/queue/lisp.lisp

Parent

legion/queue (system)

Location

queue.lisp

Packages

legion/queue

Exported Definitions

• dequeue (function)
• enqueue (function)
• make-queue (function)
• queue (structure)
• queue-count (function)
• queue-empty-p (function)

Internal Definitions

• %make-queue (function)
• copy-queue (function)
• extend (function)
• need-to-extend-p (function)
• need-to-tidy-p (function)
• queue-buffered (function)
• (setf queue-buffered) (function)
• queue-lock (function)
• (setf queue-lock) (function)
• queue-p (function)
• queue-primary (function)
• (setf queue-primary) (function)
• queue-to-enqueue (function)
• tidy (function)

3.1.6 legion/error/lisp.lisp

Parent

legion/error (system)

Location

error.lisp

Packages

legion/error

Exported Definitions

legion-error (condition)

4 Packages

Packages are listed by definition order.

4.1 legion

Source

lisp.lisp (file)

Nickname

legion/main

Use List

common-lisp

Exported Definitions

• make-cluster (function)
• make-worker (function)

4.2 legion/cluster

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions

• cluster (class)
• cluster-status (generic function)
• cluster-status (method)
• (setf cluster-status) (method)
• (setf cluster-status) (generic function)
• cluster-workers (generic function)
• cluster-workers (method)

Internal Definitions

• cluster-queue (generic function)
• cluster-queue (method)
• (setf cluster-queue) (method)
• (setf cluster-queue) (generic function)

4.3 legion/worker

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions

• add-job (generic function)
• add-job (method)
• add-job (method)
• fetch-job (generic function)
• fetch-job (method)
• kill (generic function)
• kill (method)
• kill (method)
• process-job (generic function)
• process-job (method)
• start (generic function)
• start (method)
• start (method)
• stop (generic function)
• stop (method)
• stop (method)
• wakenup (function)
• worker (class)
• worker-queue (generic function)
• worker-queue (method)
• (setf worker-queue) (method)
• (setf worker-queue) (generic function)
• worker-queue-count (function)
• worker-status (generic function)
• worker-status (method)
• (setf worker-status) (method)
• (setf worker-status) (generic function)

Internal Definitions

• run (generic function)
• run (method)
• run (method)
• worker-process-fn (generic function)
• worker-process-fn (method)
• worker-thread (generic function)
• worker-thread (method)
• worker-wait-cond (generic function)
• worker-wait-cond (method)
• worker-wait-lock (generic function)
• worker-wait-lock (method)

4.4 legion/queue

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions

• dequeue (function)
• enqueue (function)
• make-queue (function)
• queue (structure)
• queue-count (function)
• queue-empty-p (function)

Internal Definitions

• %make-queue (function)
• copy-queue (function)
• extend (function)
• need-to-extend-p (function)
• need-to-tidy-p (function)
• queue-buffered (function)
• (setf queue-buffered) (function)
• queue-lock (function)
• (setf queue-lock) (function)
• queue-p (function)
• queue-primary (function)
• (setf queue-primary) (function)
• queue-to-enqueue (function)
• tidy (function)

4.5 legion/error

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions

legion-error (condition)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Functions

Function: dequeue QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: enqueue JOB QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: make-cluster WORKER-NUM PROCESS-FN &rest INITARGS

Package

legion

Source

lisp.lisp (file)

Function: make-queue &optional INITIAL-SIZE

Package

legion/queue

Source

lisp.lisp (file)

Function: make-worker PROCESS-FN &rest INITARGS

Package

legion

Source

lisp.lisp (file)

Function: queue-count QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: queue-empty-p QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: wakenup WORKER

Package

legion/worker

Source

lisp.lisp (file)

Function: worker-queue-count WORKER

Return the number of outstanding jobs.

Package

legion/worker

Source

lisp.lisp (file)

5.1.2 Generic functions

Generic Function: add-job WORKER VAL

Enqueue VAL to WORKER’s queue. This returns WORKER when the queueing has been succeeded; otherwise NIL is returned.

Package

legion/worker

Source

lisp.lisp (file)

Methods

Method: add-job (CLUSTER cluster) JOB

Source

lisp.lisp (file)

Method: add-job (WORKER worker) VAL

Generic Function: cluster-status OBJECT

Generic Function: (setf cluster-status) NEW-VALUE OBJECT

Package

legion/cluster

Methods

Method: cluster-status (CLUSTER cluster)

automatically generated reader method

Source

lisp.lisp (file)

Method: (setf cluster-status) NEW-VALUE (CLUSTER cluster)

automatically generated writer method

Source

lisp.lisp (file)

Generic Function: cluster-workers OBJECT

Package

legion/cluster

Methods

Method: cluster-workers (CLUSTER cluster)

automatically generated reader method

Source

lisp.lisp (file)

Generic Function: fetch-job WORKER

Dequeue a value from WORKER’s queue. This returns multiple values – the job and a successed flag.

Package

legion/worker

Source

lisp.lisp (file)

Methods

Method: fetch-job (WORKER worker)

Generic Function: kill WORKER

Stop the given WORKER immediately.
It raises an error if the WORKER is not running.

Package

legion/worker

Source

lisp.lisp (file)

Methods

Method: kill (CLUSTER cluster)

Source

lisp.lisp (file)

Method: kill (WORKER worker)

Generic Function: process-job WORKER JOB

Package

legion/worker

Source

lisp.lisp (file)

Methods

Method: process-job (WORKER worker) JOB

Generic Function: start WORKER

Start the given WORKER.
It raises an error if the WORKER is already running.

Package

legion/worker

Source

lisp.lisp (file)

Methods

Method: start (CLUSTER cluster)

Source

lisp.lisp (file)

Method: start (WORKER worker)

Generic Function: stop WORKER

Stop the given WORKER after processing its queued jobs. It raises an error if the WORKER is not running.

Package

legion/worker

Source

lisp.lisp (file)

Methods

Method: stop (CLUSTER cluster)

Source

lisp.lisp (file)

Method: stop (WORKER worker)

Generic Function: worker-queue OBJECT

Generic Function: (setf worker-queue) NEW-VALUE OBJECT

Package

legion/worker

Methods

Method: worker-queue (WORKER worker)

automatically generated reader method

Source

lisp.lisp (file)

Method: (setf worker-queue) NEW-VALUE (WORKER worker)

automatically generated writer method

Source

lisp.lisp (file)

Generic Function: worker-status OBJECT

Generic Function: (setf worker-status) NEW-VALUE OBJECT

Package

legion/worker

Methods

Method: worker-status (WORKER worker)

automatically generated reader method

Source

lisp.lisp (file)

Method: (setf worker-status) NEW-VALUE (WORKER worker)

automatically generated writer method

Source

lisp.lisp (file)

5.1.3 Conditions

Condition: legion-error ()

Package

legion/error

Source

lisp.lisp (file)

Direct superclasses

error (condition)

5.1.4 Structures

Structure: queue ()

Package

legion/queue

Source

lisp.lisp (file)

Direct superclasses

structure-object (structure)

Direct slots

Slot: primary

Readers

queue-primary (function)

Writers

(setf queue-primary) (function)

Slot: buffered

Initform

(quote nil)

Readers

queue-buffered (function)

Writers

(setf queue-buffered) (function)

Slot: lock

Initform

(bordeaux-threads:make-lock "queue lock")

Readers

queue-lock (function)

Writers

(setf queue-lock) (function)

5.1.5 Classes

Class: cluster ()

Package

legion/cluster

Source

lisp.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• add-job (method)
• kill (method)
• stop (method)
• start (method)
• print-object (method)
• initialize-instance (method)
• cluster-workers (method)
• cluster-queue (method)
• cluster-queue (method)
• cluster-status (method)
• cluster-status (method)

Direct slots

Slot: status

Initform

:shutdown

Readers

cluster-status (generic function)

Writers

(setf cluster-status) (generic function)

Slot: queue

Initargs

:queue

Initform

(legion/queue:make-queue)

Readers

cluster-queue (generic function)

Writers

(setf cluster-queue) (generic function)

Slot: workers

Initargs

:workers

Initform

(quote nil)

Readers

cluster-workers (generic function)

Class: worker ()

Package

legion/worker

Source

lisp.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• add-job (method)
• kill (method)
• stop (method)
• start (method)
• run (method)
• run (method)
• process-job (method)
• fetch-job (method)
• print-object (method)
• worker-wait-cond (method)
• worker-wait-lock (method)
• worker-thread (method)
• worker-process-fn (method)
• worker-queue (method)
• worker-queue (method)
• worker-status (method)
• worker-status (method)

Direct slots

Slot: status

Initform

:shutdown

Readers

worker-status (generic function)

Writers

(setf worker-status) (generic function)

Slot: queue

Initargs

:queue

Initform

(legion/queue:make-queue)

Readers

worker-queue (generic function)

Writers

(setf worker-queue) (generic function)

Slot: process-fn

Initargs

:process-fn

Readers

worker-process-fn (generic function)

Slot: thread

Readers

worker-thread (generic function)

Slot: wait-lock

Initform

(bordeaux-threads:make-recursive-lock "wait-lock")

Readers

worker-wait-lock (generic function)

Slot: wait-cond

Initform

(bordeaux-threads:make-condition-variable)

Readers

worker-wait-cond (generic function)

5.2 Internal definitions

5.2.1 Functions

Function: %make-queue &key (PRIMARY PRIMARY) (BUFFERED BUFFERED) (LOCK LOCK)

Package

legion/queue

Source

lisp.lisp (file)

Function: copy-queue INSTANCE

Package

legion/queue

Source

lisp.lisp (file)

Function: extend QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: need-to-extend-p QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: need-to-tidy-p QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: queue-buffered INSTANCE

Function: (setf queue-buffered) VALUE INSTANCE

Package

legion/queue

Source

lisp.lisp (file)

Function: queue-lock INSTANCE

Function: (setf queue-lock) VALUE INSTANCE

Package

legion/queue

Source

lisp.lisp (file)

Function: queue-p OBJECT

Package

legion/queue

Source

lisp.lisp (file)

Function: queue-primary INSTANCE

Function: (setf queue-primary) VALUE INSTANCE

Package

legion/queue

Source

lisp.lisp (file)

Function: queue-to-enqueue QUEUE

Package

legion/queue

Source

lisp.lisp (file)

Function: tidy QUEUE

Package

legion/queue

Source

lisp.lisp (file)

5.2.2 Generic functions

Generic Function: cluster-queue OBJECT

Generic Function: (setf cluster-queue) NEW-VALUE OBJECT

Package

legion/cluster

Methods

Method: cluster-queue (CLUSTER cluster)

automatically generated reader method

Source

lisp.lisp (file)

Method: (setf cluster-queue) NEW-VALUE (CLUSTER cluster)

automatically generated writer method

Source

lisp.lisp (file)

Generic Function: run WORKER

Package

legion/worker

Source

lisp.lisp (file)

Methods

Method: run (WORKER worker)

Method: run (WORKER worker) around

Generic Function: worker-process-fn OBJECT

Package

legion/worker

Methods

Method: worker-process-fn (WORKER worker)

automatically generated reader method

Source

lisp.lisp (file)

Generic Function: worker-thread OBJECT

Package

legion/worker

Methods

Method: worker-thread (WORKER worker)

automatically generated reader method

Source

lisp.lisp (file)

Generic Function: worker-wait-cond OBJECT

Package

legion/worker

Methods

Method: worker-wait-cond (WORKER worker)

automatically generated reader method

Source

lisp.lisp (file)

Generic Function: worker-wait-lock OBJECT

Package

legion/worker

Methods

Method: worker-wait-lock (WORKER worker)

automatically generated reader method

Source

lisp.lisp (file)

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables

A.4 Data types