The swank-crew Reference Manual

Next: , Previous: , Up: (dir)   [Contents][Index]

The swank-crew Reference Manual

This is the swank-crew Reference Manual, version 1.4, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Aug 15 05:56:11 2022 GMT+0.

Table of Contents


1 Introduction

Swank Crew

Swank Crew is a distributed computation framework written in Common Lisp that uses Swank Client to evaluate Lisp expressions on remote hosts.

Swank Crew works with GNU Emacs and Slime to make developing distributed applications easier. Typically, you interactively develop code on a master machine using Slime. Code running on the master uses the Swank Crew API to farm out work to a gang of worker machines. If code on a worker throws an exception, a Slime stack trace window pops up and you can interactively debug the problem using your existing Slime connection to the master.

The Swank Crew API

connect-workers host/port-alist master-host-name master-swank-port

Makes Swank connections to all the workers in HOST/PORT-ALIST and returns a
WORKER-POOL containing them.  HOST/PORT-ALIST is a list of (host-name . port)
pairs.  MASTER-HOST-NAME and MASTER-SWANK-PORT are a host name and Swank port
that workers can use to return results to the master.

disconnect-workers worker-pool

Closes the Swank connections of all connected workers in WORKER-POOL.

eval-form-all-workers worker-pool form &key result-done (replay-required t)

Evaluates FORM on all workers in WORKER-POOL.  When RESULT-DONE is non-NIL, it
must be a function of two arguments.  In this case RESULT-DONE is called as
each worker returns a result with two arguments, a non-negative integer
representing the order in which the work was dispatched and the worker's
result.  If REPLAY-REQUIRED is true, which is the default, FORM will be
remembered and evaluated again for side effects on any new worker that joins
POOL.

eval-form-repeatedly worker-pool result-count form &key (worker-count (worker-count worker-pool))

Evaluates FORM, which must return a function of no arguments, on WORKER-COUNT
workers in WORKER-POOL, then arranges for the workers to repeatedly call the
function to create RESULT-COUNT results, which are returned in a list.
WORKER-COUNT defaults to the number of workers in WORKER-POOL.

eval-repeatedly-async-state worker-pool form initial-state update-state &key (worker-count (worker-count worker-pool))

Evaluates FORM, which must return a function of one argument, on WORKER-COUNT
workers in WORKER-POOL, then arranges for the workers to repeatedly call the
work function and send its results back to the master.  The work function is
passed a state argument, initially set to INITIAL-STATE, that the master can
update asynchronously as it receives new results.

On the master, the work state is initialized to INITIAL-STATE and UPDATE-STATE
is called repeatedly to process results received from the workers.
UPDATE-STATE is called with two arguments, the current work state and a list
containing all unprocessed work results.  UPDATE-STATE should return three
values: the new work state, a boolean indicating whether the computation should
end, and a boolean indicating whether the latest work state should be
distributed to workers.  When UPDATE-STATE's second return value is true,
EVAL-REPEATEDLY-ASYNC-STATE tells the workers to stop and returns the latest
work state.

parallel-mapcar worker-pool make-work list &optional result-done

Traverses LIST, calling MAKE-WORK on each element to create a form that is then
passed to a remote worker in WORKER-POOL for evaluation.  Results of evaluating
each form are collected into a list, which is returned when every remote worker
is done.  If RESULT-DONE is provided, then it must be a function of two
arguments.  In this case RESULT-DONE is called on the master as each worker
returns a result.  The arguments to RESULT-DONE are the position of the work in
LIST and the worker's result.

parallel-reduce worker-pool make-work list initial-value reducer

Traverses LIST, calling MAKE-WORK on each element to create a form that is
then passed to a remote worker in WORKER-POOL for evaluation.  As results are
returned, REDUCER, a binary function, is used to combine successive results in
the manner of REDUCE.  INITIAL-VALUE is used as the starting value for the
reduction computation.  The form

  (parallel-reduce worker-pool make-work list initial-value reducer)

is equivalent to

  (reduce reducer
          (mapcar (lambda (x) (eval (funcall make-work x))) list)
          :initial-value initial-value)

For more information, see the documentation strings for the above functions in master.lisp and the example code in swank-crew-test.lisp.

Installation

Swank Crew functions best when used with Slime. In order to implement the distributed Slime debugging features described above, you must patch swank.lisp in the directory where you installed Slime:

patch path/to/slime/swank.lisp < swank.lisp-patch

The change to swank.lisp implements the forwarding of Swank protocol messages from worker machines through the master's Swank server back to your Slime client. It's needed to debug problems on worker machines using your Slime connection to the master.


2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 swank-crew

Distributed master/worker computing framework.

Author

Robert Brown <robert.brown@gmail.com>

License

New BSD license. See the copyright messages in individual files.

Long Description

Swank Crew is a framework for developing distributed master/worker applications. It uses Slime’s Swank protocol to transport data between machines, making the debugging of distributed applications easier.

Version

1.4

Dependencies
  • bordeaux-threads (system).
  • com.google.base (system).
  • swank-client (system).
Source

swank-crew.asd.

Child Components

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp   [Contents][Index]

3.1.1 swank-crew/swank-crew.asd

Source

swank-crew.asd.

Parent Component

swank-crew (system).

ASDF Systems

swank-crew.


3.1.2 swank-crew/package.lisp

Source

swank-crew.asd.

Parent Component

swank-crew (system).

Packages

swank-crew.


3.1.3 swank-crew/master.lisp

Dependency

package.lisp (file).

Source

swank-crew.asd.

Parent Component

swank-crew (system).

Public Interface
Internals

Previous: , Up: Lisp   [Contents][Index]

3.1.4 swank-crew/worker.lisp

Dependency

package.lisp (file).

Source

swank-crew.asd.

Parent Component

swank-crew (system).

Internals

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 swank-crew

Evaluate expressions on remote Lisps using the Swank protocol.

Source

package.lisp.

Use List
  • com.google.base.
  • common-lisp.
Public Interface
Internals

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Public Interface


5.1.1 Ordinary functions

Function: connect-workers (host/port-alist master-host-name master-swank-port)

Makes Swank connections to all the workers in HOST/PORT-ALIST and returns a WORKER-POOL containing them. HOST/PORT-ALIST is a list of (host-name . port) pairs. MASTER-HOST-NAME and MASTER-SWANK-PORT are a host name and Swank port that workers can use to return results to the master.

Package

swank-crew.

Source

master.lisp.

Function: disconnect-workers (worker-pool)

Closes the Swank connections of all connected workers in WORKER-POOL.

Package

swank-crew.

Source

master.lisp.

Function: eval-form-all-workers (worker-pool form &key result-done replay-required)

Evaluates FORM on all workers in WORKER-POOL. When RESULT-DONE is non-NIL, it must be a function of two arguments. In this case RESULT-DONE is called as each worker returns a result with two arguments, a non-negative integer representing the order in which the work was dispatched and the worker’s result. If REPLAY-REQUIRED is true, which is the default, FORM will be remembered and evaluated again for side effects on any new worker that joins POOL.

Package

swank-crew.

Source

master.lisp.

Function: eval-form-repeatedly (worker-pool result-count form &key worker-count)

Evaluates FORM, which must return a function of no arguments, on WORKER-COUNT workers in WORKER-POOL, then arranges for the workers to repeatedly call the function to create RESULT-COUNT results, which are returned in a list. WORKER-COUNT defaults to the number of workers in WORKER-POOL.

Package

swank-crew.

Source

master.lisp.

Function: eval-repeatedly-async-state (worker-pool form initial-state update-state &key worker-count)

Evaluates FORM, which must return a function of one argument, on WORKER-COUNT workers in WORKER-POOL, then arranges for the workers to repeatedly call the work function and send its results back to the master. The work function is passed a state argument, initially set to INITIAL-STATE, that the master can update asynchronously as it receives new results.

On the master, the work state is initialized to INITIAL-STATE and UPDATE-STATE is called repeatedly to process results received from the workers. UPDATE-STATE is called with two arguments, the current work state and a list containing all unprocessed work results. UPDATE-STATE should return three values: the new work state, a boolean indicating whether the computation should end, and a boolean indicating whether the latest work state should be distributed to workers. When UPDATE-STATE’s second return value is true, EVAL-REPEATEDLY-ASYNC-STATE tells the workers to stop and returns the latest work state.

Package

swank-crew.

Source

master.lisp.

Function: parallel-mapcar (worker-pool make-work list &optional result-done)

Traverses LIST, calling MAKE-WORK on each element to create a form that is then passed to a remote worker in WORKER-POOL for evaluation. Results of evaluating each form are collected into a list, which is returned when every remote worker is done. If RESULT-DONE is provided, then it must be a function of two arguments. In this case RESULT-DONE is called on the master as each worker returns a result. The arguments to RESULT-DONE are the position of the work in LIST and the worker’s result.

Package

swank-crew.

Source

master.lisp.

Function: parallel-reduce (worker-pool make-work list initial-value reducer)

Traverses LIST, calling MAKE-WORK on each element to create a form that is then passed to a remote worker in WORKER-POOL for evaluation. As results are returned, REDUCER, a binary function, is used to combine successive results in the manner of REDUCE. INITIAL-VALUE is used as the starting value for the reduction computation. The form

(parallel-reduce worker-pool make-work list initial-value reducer)

is equivalent to

(reduce reducer
(mapcar (lambda (x) (eval (funcall make-work x))) list) :initial-value initial-value)

Package

swank-crew.

Source

master.lisp.

Function: worker-count (worker-pool)

Returns the total number of workers in WORKER-POOL.

Package

swank-crew.

Source

master.lisp.


5.1.2 Standalone methods

Method: initialize-instance :after ((worker-pool worker-pool) &key)
Source

master.lisp.

Method: print-object ((worker-pool worker-pool) stream)

Prints WORKER-POOL to STREAM. This function runs without locking WORKER-POOL, so it may output inconsistent information.

Source

master.lisp.


5.1.3 Classes

Class: worker-pool

A pool of Swank workers to which Lisp forms can be sent for evaluation.

Package

swank-crew.

Source

master.lisp.

Direct methods
Direct slots
Slot: id

The worker pool’s ID.

Type

symbol

Initargs

:id

Readers

id.

Writers

This slot is read-only.

Slot: master-host-name

Host name of the master. Used by workers to return results to the master.

Type

string

Initargs

:master-host-name

Readers

master-host-name.

Writers

This slot is read-only.

Slot: master-swank-port

Port on the master on which a Swank server is listening. Used by workers to return results to the master.

Type

swank-crew::port

Initargs

:master-swank-port

Readers

master-swank-port.

Writers

This slot is read-only.

Slot: workers

Vector containing all workers in the worker pool.

Type

vector

Initform

#()

Readers

workers.

Writers

(setf %workers).

Slot: lock

Lock protecting IDLE-WORKERS, WORKER-AVAILABLE, and DISCONNECTING.

Initform

(bordeaux-threads:make-lock "worker pool")

Readers

lock.

Writers

This slot is read-only.

Slot: idle-workers

List of currently idle workers.

Type

list

Initform

(quote nil)

Readers

idle-workers.

Writers

(setf idle-workers).

Slot: worker-available

Condition signaled when a worker becomes idle.

Initform

(bordeaux-threads:make-condition-variable)

Readers

worker-available.

Writers

This slot is read-only.

Slot: disconnecting

Set non-NIL when the worker pool is being torn down to tell the reconnector thread it should exit.

Readers

disconnecting.

Writers

(setf disconnecting).

Slot: replay-forms-lock

Lock that protects REPLAY-FORMS.

Initform

(bordeaux-threads:make-lock "replay forms lock")

Readers

replay-forms-lock.

Writers

This slot is read-only.

Slot: replay-forms

List containing all forms passed to EVAL-FORM-ALL-WORKERS that need to be replayed on new workers when they join the pool.

Type

list

Initform

(quote nil)

Readers

replay-forms.

Writers

(setf replay-forms).


5.2 Internals


Next: , Previous: , Up: Internals   [Contents][Index]

5.2.1 Constants

Constant: +worker-reconnect-interval+

Seconds between attempts to reconnect workers.

Package

swank-crew.

Source

master.lisp.


Next: , Previous: , Up: Internals   [Contents][Index]

5.2.2 Special variables

Special Variable: *evals*

List containing an EVALUATION instance for each running computation.

Package

swank-crew.

Source

master.lisp.

Special Variable: *evals-lock*

Lock protecting access to *EVALS*.

Package

swank-crew.

Source

master.lisp.

Special Variable: *evaluation-id*

Counter used to generate a unique ID for EVALUATION instances.

Package

swank-crew.

Source

master.lisp.

Special Variable: *evaluation-id-lock*

Lock protecting access to *EVALUATION-ID*.

Package

swank-crew.

Source

master.lisp.

Special Variable: *last-form-evaled*

The last form evaluated by the worker.

Package

swank-crew.

Source

worker.lisp.

Special Variable: *last-random-state*

The value of *RANDOM-STATE* right before the worker started evaluating *LAST-FORM-EVALED*.

Package

swank-crew.

Source

worker.lisp.

Special Variable: *last-repeated-eval-work-function*

Thunk created by the current invocation of REPEATEDLY-EVALUATE to produce results for an EVAL-FORM-REPEATEDLY request on the master. This variable is useful for debugging.

Package

swank-crew.

Source

worker.lisp.

Special Variable: *replay-forms-counts*

Mapping from worker pool IDs to the number of replay forms we have evaluated on this client for that pool.

Package

swank-crew.

Source

worker.lisp.

Special Variable: *replay-forms-counts-lock*

Lock protecting access to *REPLAY-FORMS-COUNTS*.

Package

swank-crew.

Source

worker.lisp.

Special Variable: *worker-pools*

Mapping from worker pool IDs to active worker pools.

Package

swank-crew.

Source

master.lisp.

Special Variable: *worker-pools-lock*

Lock protecting access to *WORKER-POOLS*.

Package

swank-crew.

Source

master.lisp.


5.2.3 Ordinary functions

Function: add-eval (eval)

Adds EVAL to the list of in-progress computations.

Package

swank-crew.

Source

master.lisp.

Function: add-evaluated-form (worker-pool form)

Adds FORM to the set of forms that need to be evaluated on a new worker when it joins WORKER-POOL.

Package

swank-crew.

Source

master.lisp.

Function: allocate-worker (worker-pool &key worker-to-avoid)

Allocates and returns an idle worker from WORKER-POOL that is connected. If WORKER-POOL is being shut down, then NIL is returned.

Package

swank-crew.

Source

master.lisp.

Function: allocate-worker-and-evaluate (worker-pool form set-worker worker-done)

Allocates a connected worker from WORKER-POOL and sends it FORM to evaluate. Returns the worker that was successfully allocated, or NIL if WORKER-POOL is shutting down. SET-WORKER is a function that is called to tell the WORKER-DONE continuation what worker is evaluating FORM.

Package

swank-crew.

Source

master.lisp.

Function: async-evaluate (form initial-state id master-host-name master-swank-port)

Evaluates FORM, which must return a work function of one argument, then calls that function repeatedly to produce results, each time passing it the current computation state. At first this state is INITIAL-STATE, but the master may update the state asynchronously. Each work result is sent to host MASTER-HOST-NAME by making a Swank connection to port MASTER-SWANK-PORT and remotely evaluating an expression that records the result. ID is used on the master machine to process results and on the worker to update the state.

Package

swank-crew.

Source

worker.lisp.

Function: async-results-loop (async-eval update-state)

Handles incoming results for ASYNC-EVAL by calling UPDATE-STATE whenever ASYNC-EVAL holds unprocessed results. UPDATE-STATE is called with two arguments, the work state and a list of the unprocessed results.

Package

swank-crew.

Source

master.lisp.

Function: async-work-form (form initial-state worker-pool id)

Returns a form for evaluation on a client of WORKER-POOL that ensures the client is caught up and then evaluates FORM for the async evaluation request identified by ID.

Package

swank-crew.

Source

master.lisp.

Function: catch-up-if-necessary (master-host-name master-swank-port worker-pool-id pool-count)

Ensures that the current client is up to date, that it has evaluated all POOL-COUNT replay forms associated with the pool identified by WORKER-POOL-ID. If it is necessary to evaluate forms in order to catch up, they are fetched by making a Swank connection to host MASTER-HOST-NAME on port MASTER-SWANK-PORT.

Package

swank-crew.

Source

worker.lisp.

Function: clear-debugging-info ()

Sets all debugging global variables to NIL.

Package

swank-crew.

Source

worker.lisp.

Function: debugging-form (form)

Returns an expression that when evaluated returns the result of evaluating FORM. In addition, the returned expression arranges to update the values of *LAST-FORM-EVALED* and *LAST-RANDOM-STATE* for ease of debugging.

Package

swank-crew.

Source

worker.lisp.

Function: dispatch-work (worker-pool make-work list result-done retain-workers replay-required)

Traverses LIST, calling MAKE-WORK on each element to create a form that is then passed to a remote worker in WORKER-POOL for evaluation. When RETAIN-WORKERS is true, each form is evaluated on a unique worker. Otherwise, workers are reused and may process multiple forms. In either case, the results of evaluating each form are collected into a list, which is returned when every remote worker is done. If RESULT-DONE is not NIL, then it must be a function of two arguments. In this case RESULT-DONE is called on the master as each worker returns a result. The arguments to RESULT-DONE are an integer counter, indicating the work form’s position in LIST, and the result of evaluating the form. REPLAY-REQUIRED indicates whether new workers will have to perform the work generated by MAKE-WORK before being considered to be caught up.

Package

swank-crew.

Source

master.lisp.

Function: ensure-caught-up-then-evaluate (form worker-pool replay-required)

Returns a form that when evaluated on a worker in WORKER-POOL ensures that the worker is caught up then evaluates FORM and returns its result. REPLAY-REQUIRED indicates whether new workers must evaluate FORM before being considered to be caught up.

Package

swank-crew.

Source

master.lisp.

Function: eval-on-master (make-work list result-done)

Iterates over the members of LIST calling MAKE-WORK on each one. If RESULT-DONE is not NIL, it must be a function of two arguments. In this case, after each application of MAKE-WORK to a LIST member, RESULT-DONE is called with the member’s position in LIST and MAKE-WORK’s result.

Package

swank-crew.

Source

master.lisp.

Function: evaluate-form (form master-host-name master-swank-port worker-pool-id pool-count replay-required)

Evaluates FORM, but first ensures that this worker has evaluated all POOL-COUNT replay forms associated with the worker pool identified by WORKER-POOL-ID on the master. When catching up is required, fetches forms by making a Swank connection to host MASTER-HOST-NAME on port MASTER-SWANK-PORT. REPLAY-REQUIRED indicates whether FORM may need to be replayed in order to bring a worker up to date.

Package

swank-crew.

Source

worker.lisp.

Function: fetch-and-evaluate (master-host-name master-swank-port worker-pool-id local-count)

Fetches from the master and then evaluates all forms required to catch up with other workers in the pool identified by WORKER-POOL-ID on the master. All replay forms after the first LOCAL-COUNT forms are fetched by making a Swank connection to host MASTER-HOST-NAME on port MASTER-SWANK-PORT.

Package

swank-crew.

Source

worker.lisp.

Function: find-eval (id)

Returns the running EVAL instance identified by ID, or NIL if no computation with that ID is currently running.

Package

swank-crew.

Source

master.lisp.

Function: find-worker-pool (worker-pool-id)

Returns the worker pool identified by WORKER-POOL-ID.

Package

swank-crew.

Source

master.lisp.

Function: free-worker (worker worker-pool)

Changes the state of WORKER to idle, so that it’s available for allocation.

Package

swank-crew.

Source

master.lisp.

Function: handle-connection-closed (disconnected-worker worker-pool)

Called when the connection to DISCONNECTED-WORKER, a member of WORKER-POOL, is closed because of a call to SLIME-CLOSE, the death of the worker, or because of a communications error. Moves all uncompleted work intended for DISCONNECTED-WORKER to another idle connected worker in WORKER-POOL.

Package

swank-crew.

Source

master.lisp.

Function: make-worker-pool (master-host-name master-swank-port connect-infos connect-worker)
Package

swank-crew.

Source

master.lisp.

Function: no-workers-p (worker-pool)

Returns T if WORKER-POOL is NIL or contains no workers.

Package

swank-crew.

Source

master.lisp.

Function: reconnect-workers (worker-pool)

Reconnects disconnected workers in WORKER-POOL.

Package

swank-crew.

Source

master.lisp.

Function: record-async-result (id result worker-state-counter)

Stores RESULT as one of the values produced by the async evaluation identified by ID. Returns a boolean indicating whether the worker should continue to call ASYNC-RESULT with additional results.

Package

swank-crew.

Source

master.lisp.

Function: record-repeated-result (id result)

Stores RESULT as one of the values produced by the repeated evaluation identified by ID. Returns a boolean indicating whether the worker should continue to call RECORD-REPEATED-RESULT with additional results.

Package

swank-crew.

Source

master.lisp.

Function: remove-eval (eval)

Removes EVAL from the list of in-progress computations.

Package

swank-crew.

Source

master.lisp.

Function: repeated-work-form (form worker-pool id)

Returns a form for evaluation on a client of WORKER-POOL that ensures the client is caught up and then evaluates FORM for the repeated evaluation request identified by ID.

Package

swank-crew.

Source

master.lisp.

Function: repeatedly-evaluate (form id master-host-name master-swank-port)

Evaluates FORM, which must return a function of no arguments, then calls that function repeatedly to produce results. Each result is sent to host MASTER-HOST-NAME by making a Swank connection on port MASTER-SWANK-PORT and remotely evaluating an expression that records the result. ID is used on the master machine to correctly record the result.

Package

swank-crew.

Source

worker.lisp.

Function: send-many-results (send-result master-host-name master-swank-port)

Creates a Slime connection to host MASTER-HOST-NAME on port MASTER-SWANK-PORT, then repeatedly calls SEND-RESULT with the new connection as argument. Returns when SEND-RESULT returns NIL.

Package

swank-crew.

Source

worker.lisp.

Function: unevaluated-replay-forms (worker-pool-id evaluated-count)

For a worker that has executed EVALUATED-COUNT of the replay forms associated with the worker-pool identified by WORKER-POOL-ID, returns a list of the forms the worker needs to evaluate in order to be completely up to date.

Package

swank-crew.

Source

master.lisp.

Function: worker-counts (worker-pool)

Returns the number of idle, busy, and disconnected workers in WORKER-POOL. This function executes without locking WORKER-POOL, so it may return inconsistent information.

Package

swank-crew.

Source

master.lisp.


Next: , Previous: , Up: Internals   [Contents][Index]

5.2.4 Generic functions

Generic Writer: (setf %workers) (object)
Package

swank-crew.

Methods
Writer Method: (setf %workers) ((worker-pool worker-pool))

Vector containing all workers in the worker pool.

Source

master.lisp.

Target Slot

workers.

Generic Reader: connect-info (object)
Package

swank-crew.

Methods
Reader Method: connect-info ((worker worker))

Information used when connecting to this worker.

Source

master.lisp.

Target Slot

connect-info.

Generic Function: connect-worker (connect-info close-handler)

Creates a connection to a worker’s Swank server using information in CONNECT-INFO. Passes the thunk CLOSE-HANDLER to SWANK-CLIENT:SLIME-CONNECT, so that it is invoked when the connection closes.

Package

swank-crew.

Source

master.lisp.

Methods
Method: connect-worker ((connect-info connect-info) close-handler)
Generic Reader: connection (object)
Generic Writer: (setf connection) (object)
Package

swank-crew.

Methods
Reader Method: connection ((worker worker))
Writer Method: (setf connection) ((worker worker))

When non-NIL, an open Swank connection to the worker.

Source

master.lisp.

Target Slot

connection.

Generic Reader: disconnecting (object)
Generic Writer: (setf disconnecting) (object)
Package

swank-crew.

Methods
Reader Method: disconnecting ((worker-pool worker-pool))
Writer Method: (setf disconnecting) ((worker-pool worker-pool))

Set non-NIL when the worker pool is being torn down to tell the reconnector thread it should exit.

Source

master.lisp.

Target Slot

disconnecting.

Generic Reader: done (object)
Generic Writer: (setf done) (object)
Package

swank-crew.

Methods
Reader Method: done ((evaluation evaluation))
Writer Method: (setf done) ((evaluation evaluation))

Is the computation done?

Source

master.lisp.

Target Slot

done.

Generic Reader: done-condition (object)
Package

swank-crew.

Methods
Reader Method: done-condition ((evaluation evaluation))

Condition variable notified when computation is done.

Source

master.lisp.

Target Slot

done-condition.

Generic Reader: host-name (object)
Generic Writer: (setf host-name) (object)
Package

swank-crew.

Methods
Reader Method: host-name ((connect-info connect-info))
Writer Method: (setf host-name) ((connect-info connect-info))

Host the worker is running on.

Source

master.lisp.

Target Slot

host-name.

Generic Reader: id (object)
Package

swank-crew.

Methods
Reader Method: id ((evaluation evaluation))

Unique ID for this running evaluation request.

Source

master.lisp.

Target Slot

id.

Reader Method: id ((worker-pool worker-pool))

The worker pool’s ID.

Source

master.lisp.

Target Slot

id.

Generic Reader: idle-workers (object)
Generic Writer: (setf idle-workers) (object)
Package

swank-crew.

Methods
Reader Method: idle-workers ((worker-pool worker-pool))
Writer Method: (setf idle-workers) ((worker-pool worker-pool))

List of currently idle workers.

Source

master.lisp.

Target Slot

idle-workers.

Generic Reader: lock (object)
Package

swank-crew.

Methods
Reader Method: lock ((evaluation evaluation))

Lock protecting the access to this instance.

Source

master.lisp.

Target Slot

lock.

Reader Method: lock ((worker-pool worker-pool))

Lock protecting IDLE-WORKERS, WORKER-AVAILABLE, and DISCONNECTING.

Source

master.lisp.

Target Slot

lock.

Reader Method: lock ((worker worker))

Lock protecting the mutable data of this worker.

Source

master.lisp.

Target Slot

lock.

Generic Reader: master-host-name (object)
Package

swank-crew.

Methods
Reader Method: master-host-name ((worker-pool worker-pool))

Host name of the master. Used by workers to return results to the master.

Source

master.lisp.

Target Slot

master-host-name.

Generic Reader: master-swank-port (object)
Package

swank-crew.

Methods
Reader Method: master-swank-port ((worker-pool worker-pool))

Port on the master on which a Swank server is listening. Used by workers to return results to the master.

Source

master.lisp.

Target Slot

master-swank-port.

Generic Reader: port (object)
Generic Writer: (setf port) (object)
Package

swank-crew.

Methods
Reader Method: port ((connect-info connect-info))
Writer Method: (setf port) ((connect-info connect-info))

Port on which the worker’s Swank server is listening for connections.

Source

master.lisp.

Target Slot

port.

Generic Function: reconnect-worker (connect-info close-handler)

Reconnects to a Swank server using information in CONNECT-INFO. Passes the thunk CLOSE-HANDLER to SWANK-CLIENT:SLIME-CONNECT, so that it is invoked when the connection closes.

Package

swank-crew.

Source

master.lisp.

Methods
Method: reconnect-worker ((connect-info connect-info) close-handler)
Generic Reader: replay-forms (object)
Generic Writer: (setf replay-forms) (object)
Package

swank-crew.

Methods
Reader Method: replay-forms ((worker-pool worker-pool))
Writer Method: (setf replay-forms) ((worker-pool worker-pool))

List containing all forms passed to EVAL-FORM-ALL-WORKERS that need to be replayed on new workers when they join the pool.

Source

master.lisp.

Target Slot

replay-forms.

Generic Reader: replay-forms-lock (object)
Package

swank-crew.

Methods
Reader Method: replay-forms-lock ((worker-pool worker-pool))

Lock that protects REPLAY-FORMS.

Source

master.lisp.

Target Slot

replay-forms-lock.

Generic Reader: results (object)
Generic Writer: (setf results) (object)
Package

swank-crew.

Methods
Reader Method: results ((async-eval async-eval))
Writer Method: (setf results) ((async-eval async-eval))

List holding returned, but unprocessed, results.

Source

master.lisp.

Target Slot

results.

Reader Method: results ((repeated-eval repeated-eval))

Vector holding returned results.

Source

master.lisp.

Target Slot

results.

Generic Reader: results-available (object)
Package

swank-crew.

Methods
Reader Method: results-available ((async-eval async-eval))

Condition notified when new results are available.

Source

master.lisp.

Target Slot

results-available.

Generic Reader: results-position (object)
Generic Writer: (setf results-position) (object)
Package

swank-crew.

Methods
Reader Method: results-position ((repeated-eval repeated-eval))
Writer Method: (setf results-position) ((repeated-eval repeated-eval))

Position where the next result will be recorded.

Source

master.lisp.

Target Slot

results-position.

Generic Reader: set-worker (object)
Generic Writer: (setf set-worker) (object)
Package

swank-crew.

Methods
Reader Method: set-worker ((worker worker))
Writer Method: (setf set-worker) ((worker worker))

Function called to record which worker is evaluating a form.

Source

master.lisp.

Target Slot

set-worker.

Generic Reader: state (object)
Generic Writer: (setf state) (object)
Package

swank-crew.

Methods
Reader Method: state ((async-eval async-eval))
Writer Method: (setf state) ((async-eval async-eval))

State of the asynchronous computation, updated from results.

Source

master.lisp.

Target Slot

state.

Generic Reader: state-counter (object)
Generic Writer: (setf state-counter) (object)
Package

swank-crew.

Methods
Reader Method: state-counter ((async-eval async-eval))
Writer Method: (setf state-counter) ((async-eval async-eval))

Counter incremented each time STATE is updated.

Source

master.lisp.

Target Slot

state-counter.

Generic Reader: worker-available (object)
Package

swank-crew.

Methods
Reader Method: worker-available ((worker-pool worker-pool))

Condition signaled when a worker becomes idle.

Source

master.lisp.

Target Slot

worker-available.

Generic Reader: workers (object)
Package

swank-crew.

Methods
Reader Method: workers ((worker-pool worker-pool))

Vector containing all workers in the worker pool.

Source

master.lisp.

Target Slot

workers.


Next: , Previous: , Up: Internals   [Contents][Index]

5.2.5 Classes

Class: async-eval

Data needed to process incoming asynchronous evaluation results.

Package

swank-crew.

Source

master.lisp.

Direct superclasses

evaluation.

Direct methods
Direct slots
Slot: results

List holding returned, but unprocessed, results.

Type

list

Initform

(quote nil)

Readers

results.

Writers

(setf results).

Slot: results-available

Condition notified when new results are available.

Initform

(bordeaux-threads:make-condition-variable)

Readers

results-available.

Writers

This slot is read-only.

Slot: state

State of the asynchronous computation, updated from results.

Initargs

:state

Readers

state.

Writers

(setf state).

Slot: state-counter

Counter incremented each time STATE is updated.

Type

(integer 0)

Initform

0

Readers

state-counter.

Writers

(setf state-counter).

Class: connect-info

Information needed when connecting to a worker’s Swank server.

Package

swank-crew.

Source

master.lisp.

Direct methods
Direct slots
Slot: host-name

Host the worker is running on.

Type

string

Initargs

:host-name

Readers

host-name.

Writers

(setf host-name).

Slot: port

Port on which the worker’s Swank server is listening for connections.

Type

swank-crew::port

Initargs

:port

Readers

port.

Writers

(setf port).

Class: evaluation

Stores the data needed to process incoming evaluation results.

Package

swank-crew.

Source

master.lisp.

Direct subclasses
Direct methods
Direct slots
Slot: id

Unique ID for this running evaluation request.

Type

integer

Initform

(bordeaux-threads:with-lock-held (swank-crew::*evaluation-id-lock*) (incf swank-crew::*evaluation-id*))

Initargs

:id

Readers

id.

Writers

This slot is read-only.

Slot: lock

Lock protecting the access to this instance.

Initform

(bordeaux-threads:make-lock "evaluation")

Readers

lock.

Writers

This slot is read-only.

Slot: done

Is the computation done?

Type

boolean

Readers

done.

Writers

(setf done).

Slot: done-condition

Condition variable notified when computation is done.

Initform

(bordeaux-threads:make-condition-variable)

Readers

done-condition.

Writers

This slot is read-only.

Class: repeated-eval

Stores the data needed to process an incoming repeated eval result.

Package

swank-crew.

Source

master.lisp.

Direct superclasses

evaluation.

Direct methods
Direct slots
Slot: results

Vector holding returned results.

Type

simple-vector

Initargs

:results

Readers

results.

Writers

This slot is read-only.

Slot: results-position

Position where the next result will be recorded.

Type

com.google.base:vector-index

Initform

0

Readers

results-position.

Writers

(setf results-position).

Class: worker

A remote Lisp running a Swank server.

Package

swank-crew.

Source

master.lisp.

Direct methods
Direct slots
Slot: lock

Lock protecting the mutable data of this worker.

Initform

(bordeaux-threads:make-lock "worker")

Readers

lock.

Writers

This slot is read-only.

Slot: connect-info

Information used when connecting to this worker.

Type

swank-crew::connect-info

Initargs

:connect-info

Readers

connect-info.

Writers

This slot is read-only.

Slot: set-worker

Function called to record which worker is evaluating a form.

Type

function

Initform

(function identity)

Readers

set-worker.

Writers

(setf set-worker).

Slot: connection

When non-NIL, an open Swank connection to the worker.

Type

(or null swank-client:swank-connection)

Readers

connection.

Writers

(setf connection).


Previous: , Up: Internals   [Contents][Index]

5.2.6 Types

Type: port ()

A non-privileged TCP/IP port number.

Package

swank-crew.

Source

master.lisp.


Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts


Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   (  
A   C   D   E   F   G   H   I   L   M   N   P   R   S   U   W  
Index Entry  Section

(
(setf %workers): Private generic functions
(setf %workers): Private generic functions
(setf connection): Private generic functions
(setf connection): Private generic functions
(setf disconnecting): Private generic functions
(setf disconnecting): Private generic functions
(setf done): Private generic functions
(setf done): Private generic functions
(setf host-name): Private generic functions
(setf host-name): Private generic functions
(setf idle-workers): Private generic functions
(setf idle-workers): Private generic functions
(setf port): Private generic functions
(setf port): Private generic functions
(setf replay-forms): Private generic functions
(setf replay-forms): Private generic functions
(setf results): Private generic functions
(setf results): Private generic functions
(setf results-position): Private generic functions
(setf results-position): Private generic functions
(setf set-worker): Private generic functions
(setf set-worker): Private generic functions
(setf state): Private generic functions
(setf state): Private generic functions
(setf state-counter): Private generic functions
(setf state-counter): Private generic functions

A
add-eval: Private ordinary functions
add-evaluated-form: Private ordinary functions
allocate-worker: Private ordinary functions
allocate-worker-and-evaluate: Private ordinary functions
async-evaluate: Private ordinary functions
async-results-loop: Private ordinary functions
async-work-form: Private ordinary functions

C
catch-up-if-necessary: Private ordinary functions
clear-debugging-info: Private ordinary functions
connect-info: Private generic functions
connect-info: Private generic functions
connect-worker: Private generic functions
connect-worker: Private generic functions
connect-workers: Public ordinary functions
connection: Private generic functions
connection: Private generic functions

D
debugging-form: Private ordinary functions
disconnect-workers: Public ordinary functions
disconnecting: Private generic functions
disconnecting: Private generic functions
dispatch-work: Private ordinary functions
done: Private generic functions
done: Private generic functions
done-condition: Private generic functions
done-condition: Private generic functions

E
ensure-caught-up-then-evaluate: Private ordinary functions
eval-form-all-workers: Public ordinary functions
eval-form-repeatedly: Public ordinary functions
eval-on-master: Private ordinary functions
eval-repeatedly-async-state: Public ordinary functions
evaluate-form: Private ordinary functions

F
fetch-and-evaluate: Private ordinary functions
find-eval: Private ordinary functions
find-worker-pool: Private ordinary functions
free-worker: Private ordinary functions
Function, add-eval: Private ordinary functions
Function, add-evaluated-form: Private ordinary functions
Function, allocate-worker: Private ordinary functions
Function, allocate-worker-and-evaluate: Private ordinary functions
Function, async-evaluate: Private ordinary functions
Function, async-results-loop: Private ordinary functions
Function, async-work-form: Private ordinary functions
Function, catch-up-if-necessary: Private ordinary functions
Function, clear-debugging-info: Private ordinary functions
Function, connect-workers: Public ordinary functions
Function, debugging-form: Private ordinary functions
Function, disconnect-workers: Public ordinary functions
Function, dispatch-work: Private ordinary functions
Function, ensure-caught-up-then-evaluate: Private ordinary functions
Function, eval-form-all-workers: Public ordinary functions
Function, eval-form-repeatedly: Public ordinary functions
Function, eval-on-master: Private ordinary functions
Function, eval-repeatedly-async-state: Public ordinary functions
Function, evaluate-form: Private ordinary functions
Function, fetch-and-evaluate: Private ordinary functions
Function, find-eval: Private ordinary functions
Function, find-worker-pool: Private ordinary functions
Function, free-worker: Private ordinary functions
Function, handle-connection-closed: Private ordinary functions
Function, make-worker-pool: Private ordinary functions
Function, no-workers-p: Private ordinary functions
Function, parallel-mapcar: Public ordinary functions
Function, parallel-reduce: Public ordinary functions
Function, reconnect-workers: Private ordinary functions
Function, record-async-result: Private ordinary functions
Function, record-repeated-result: Private ordinary functions
Function, remove-eval: Private ordinary functions
Function, repeated-work-form: Private ordinary functions
Function, repeatedly-evaluate: Private ordinary functions
Function, send-many-results: Private ordinary functions
Function, unevaluated-replay-forms: Private ordinary functions
Function, worker-count: Public ordinary functions
Function, worker-counts: Private ordinary functions

G
Generic Function, (setf %workers): Private generic functions
Generic Function, (setf connection): Private generic functions
Generic Function, (setf disconnecting): Private generic functions
Generic Function, (setf done): Private generic functions
Generic Function, (setf host-name): Private generic functions
Generic Function, (setf idle-workers): Private generic functions
Generic Function, (setf port): Private generic functions
Generic Function, (setf replay-forms): Private generic functions
Generic Function, (setf results): Private generic functions
Generic Function, (setf results-position): Private generic functions
Generic Function, (setf set-worker): Private generic functions
Generic Function, (setf state): Private generic functions
Generic Function, (setf state-counter): Private generic functions
Generic Function, connect-info: Private generic functions
Generic Function, connect-worker: Private generic functions
Generic Function, connection: Private generic functions
Generic Function, disconnecting: Private generic functions
Generic Function, done: Private generic functions
Generic Function, done-condition: Private generic functions
Generic Function, host-name: Private generic functions
Generic Function, id: Private generic functions
Generic Function, idle-workers: Private generic functions
Generic Function, lock: Private generic functions
Generic Function, master-host-name: Private generic functions
Generic Function, master-swank-port: Private generic functions
Generic Function, port: Private generic functions
Generic Function, reconnect-worker: Private generic functions
Generic Function, replay-forms: Private generic functions
Generic Function, replay-forms-lock: Private generic functions
Generic Function, results: Private generic functions
Generic Function, results-available: Private generic functions
Generic Function, results-position: Private generic functions
Generic Function, set-worker: Private generic functions
Generic Function, state: Private generic functions
Generic Function, state-counter: Private generic functions
Generic Function, worker-available: Private generic functions
Generic Function, workers: Private generic functions

H
handle-connection-closed: Private ordinary functions
host-name: Private generic functions
host-name: Private generic functions

I
id: Private generic functions
id: Private generic functions
id: Private generic functions
idle-workers: Private generic functions
idle-workers: Private generic functions
initialize-instance: Public standalone methods

L
lock: Private generic functions
lock: Private generic functions
lock: Private generic functions
lock: Private generic functions

M
make-worker-pool: Private ordinary functions
master-host-name: Private generic functions
master-host-name: Private generic functions
master-swank-port: Private generic functions
master-swank-port: Private generic functions
Method, (setf %workers): Private generic functions
Method, (setf connection): Private generic functions
Method, (setf disconnecting): Private generic functions
Method, (setf done): Private generic functions
Method, (setf host-name): Private generic functions
Method, (setf idle-workers): Private generic functions
Method, (setf port): Private generic functions
Method, (setf replay-forms): Private generic functions
Method, (setf results): Private generic functions
Method, (setf results-position): Private generic functions
Method, (setf set-worker): Private generic functions
Method, (setf state): Private generic functions
Method, (setf state-counter): Private generic functions
Method, connect-info: Private generic functions
Method, connect-worker: Private generic functions
Method, connection: Private generic functions
Method, disconnecting: Private generic functions
Method, done: Private generic functions
Method, done-condition: Private generic functions
Method, host-name: Private generic functions
Method, id: Private generic functions
Method, id: Private generic functions
Method, idle-workers: Private generic functions
Method, initialize-instance: Public standalone methods
Method, lock: Private generic functions
Method, lock: Private generic functions
Method, lock: Private generic functions
Method, master-host-name: Private generic functions
Method, master-swank-port: Private generic functions
Method, port: Private generic functions
Method, print-object: Public standalone methods
Method, reconnect-worker: Private generic functions
Method, replay-forms: Private generic functions
Method, replay-forms-lock: Private generic functions
Method, results: Private generic functions
Method, results: Private generic functions
Method, results-available: Private generic functions
Method, results-position: Private generic functions
Method, set-worker: Private generic functions
Method, state: Private generic functions
Method, state-counter: Private generic functions
Method, worker-available: Private generic functions
Method, workers: Private generic functions

N
no-workers-p: Private ordinary functions

P
parallel-mapcar: Public ordinary functions
parallel-reduce: Public ordinary functions
port: Private generic functions
port: Private generic functions
print-object: Public standalone methods

R
reconnect-worker: Private generic functions
reconnect-worker: Private generic functions
reconnect-workers: Private ordinary functions
record-async-result: Private ordinary functions
record-repeated-result: Private ordinary functions
remove-eval: Private ordinary functions
repeated-work-form: Private ordinary functions
repeatedly-evaluate: Private ordinary functions
replay-forms: Private generic functions
replay-forms: Private generic functions
replay-forms-lock: Private generic functions
replay-forms-lock: Private generic functions
results: Private generic functions
results: Private generic functions
results: Private generic functions
results-available: Private generic functions
results-available: Private generic functions
results-position: Private generic functions
results-position: Private generic functions

S
send-many-results: Private ordinary functions
set-worker: Private generic functions
set-worker: Private generic functions
state: Private generic functions
state: Private generic functions
state-counter: Private generic functions
state-counter: Private generic functions

U
unevaluated-replay-forms: Private ordinary functions

W
worker-available: Private generic functions
worker-available: Private generic functions
worker-count: Public ordinary functions
worker-counts: Private ordinary functions
workers: Private generic functions
workers: Private generic functions

Jump to:   (  
A   C   D   E   F   G   H   I   L   M   N   P   R   S   U   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *   +  
C   D   H   I   L   M   P   R   S   W  
Index Entry  Section

*
*evals*: Private special variables
*evals-lock*: Private special variables
*evaluation-id*: Private special variables
*evaluation-id-lock*: Private special variables
*last-form-evaled*: Private special variables
*last-random-state*: Private special variables
*last-repeated-eval-work-function*: Private special variables
*replay-forms-counts*: Private special variables
*replay-forms-counts-lock*: Private special variables
*worker-pools*: Private special variables
*worker-pools-lock*: Private special variables

+
+worker-reconnect-interval+: Private constants

C
connect-info: Private classes
connection: Private classes
Constant, +worker-reconnect-interval+: Private constants

D
disconnecting: Public classes
done: Private classes
done-condition: Private classes

H
host-name: Private classes

I
id: Public classes
id: Private classes
idle-workers: Public classes

L
lock: Public classes
lock: Private classes
lock: Private classes

M
master-host-name: Public classes
master-swank-port: Public classes

P
port: Private classes

R
replay-forms: Public classes
replay-forms-lock: Public classes
results: Private classes
results: Private classes
results-available: Private classes
results-position: Private classes

S
set-worker: Private classes
Slot, connect-info: Private classes
Slot, connection: Private classes
Slot, disconnecting: Public classes
Slot, done: Private classes
Slot, done-condition: Private classes
Slot, host-name: Private classes
Slot, id: Public classes
Slot, id: Private classes
Slot, idle-workers: Public classes
Slot, lock: Public classes
Slot, lock: Private classes
Slot, lock: Private classes
Slot, master-host-name: Public classes
Slot, master-swank-port: Public classes
Slot, port: Private classes
Slot, replay-forms: Public classes
Slot, replay-forms-lock: Public classes
Slot, results: Private classes
Slot, results: Private classes
Slot, results-available: Private classes
Slot, results-position: Private classes
Slot, set-worker: Private classes
Slot, state: Private classes
Slot, state-counter: Private classes
Slot, worker-available: Public classes
Slot, workers: Public classes
Special Variable, *evals*: Private special variables
Special Variable, *evals-lock*: Private special variables
Special Variable, *evaluation-id*: Private special variables
Special Variable, *evaluation-id-lock*: Private special variables
Special Variable, *last-form-evaled*: Private special variables
Special Variable, *last-random-state*: Private special variables
Special Variable, *last-repeated-eval-work-function*: Private special variables
Special Variable, *replay-forms-counts*: Private special variables
Special Variable, *replay-forms-counts-lock*: Private special variables
Special Variable, *worker-pools*: Private special variables
Special Variable, *worker-pools-lock*: Private special variables
state: Private classes
state-counter: Private classes

W
worker-available: Public classes
workers: Public classes

Jump to:   *   +  
C   D   H   I   L   M   P   R   S   W