The py4cl Reference Manual

This is the py4cl Reference Manual, generated automatically by Declt version 4.0 beta 2 "William Riker" on Sun Sep 15 06:25:51 2024 GMT+0.

Table of Contents


1 Introduction


2 Systems

The main system appears first, followed by any subsystem dependency.


2.1 py4cl

Call Python libraries from Common Lisp

Author

Ben Dudson <>

License

MIT

Dependencies
  • trivial-garbage (system).
  • uiop (system).
  • cl-json (system).
  • numpy-file-format (system).
Source

py4cl.asd.

Child Components

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


3.1 Lisp


3.1.1 py4cl/py4cl.asd

Source

py4cl.asd.

Parent Component

py4cl (system).

ASDF Systems

py4cl.

Packages

py4cl/config.

Public Interface

*base-directory* (special variable).


3.1.2 py4cl/package.lisp

Source

py4cl.asd.

Parent Component

py4cl (system).

Packages

py4cl.


3.1.3 py4cl/config.lisp

Dependency

package.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).

Public Interface
Internals

take-input (function).


3.1.4 py4cl/reader.lisp

Dependency

config.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).

Internals

3.1.5 py4cl/writer.lisp

Dependency

reader.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).

Internals

3.1.6 py4cl/python-process.lisp

Dependency

writer.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).

Public Interface
Internals

3.1.7 py4cl/lisp-classes.lisp

Dependency

python-process.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).

Public Interface

3.1.8 py4cl/callpython.lisp

Dependency

lisp-classes.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).

Public Interface
Internals

3.1.9 py4cl/import-export.lisp

Dependency

callpython.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).

Public Interface

3.1.10 py4cl/do-after-load.lisp

Dependency

import-export.lisp (file).

Source

py4cl.asd.

Parent Component

py4cl (system).


4 Packages

Packages are listed by definition order.


4.1 py4cl

Source

package.lisp.

Use List

common-lisp.

Public Interface
Internals

4.2 py4cl/config

Source

py4cl.asd.

Public Interface

*base-directory* (special variable).


5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


5.1 Public Interface


5.1.1 Special variables

Special Variable: *base-directory*
Package

py4cl/config.

Source

py4cl.asd.

Special Variable: *config*

Used for storing configuration at a centralized location.

Package

py4cl.

Source

config.lisp.

Special Variable: *python-command*

String, the Python executable to launch e.g. "python" or "python3"

Package

py4cl.

Source

python-process.lisp.


5.1.2 Macros

Macro: chain (target &rest chain)

Chain method calls, member access, and indexing operations
on objects. The operations in CHAIN are applied in order from
first to last to the TARGET object.

TARGET can be
cons – a python function to call, returning an object to operate on otherwise – a value, to be converted to a python value

CHAIN can consist of
cons – a method to call
symbol – a member data variable
otherwise – a value put between [] brackets to access an index

Keywords inside python function calls are converted to python keywords.

Functions can be specified using a symbol or a string. If a symbol is used then it is converted to python using STRING-DOWNCASE.

Examples:

(chain "hello {0}" (format "world") (capitalize))
=> python: "hello {0}".format("world").capitalize()
=> "Hello world"

(chain (range 3) stop)
=> python: range(3).stop
=> 3

(chain "hello" 4)
=> python: "hello"[4]
=> "o"

Package

py4cl.

Source

callpython.lisp.

Macro: import-function (fun-name &key docstring as from)

Define a function which calls python
Example
(py4cl:python-exec "import math")
(py4cl:import-function "math.sqrt")
(math.sqrt 42)
-> 6.4807405

Keywords:

AS specifies the symbol to be used in Lisp. This can be a symbol
or a string. If a string is given then it is read using READ-FROM-STRING.

DOCSTRING is a string which becomes the function docstring

FROM specifies a module to load the function from. This will cause the python module to be imported into the python session.

Package

py4cl.

Source

import-export.lisp.

Macro: import-module (module-name &key as reload)

Import a python module as a Lisp package. The module name should be
a string.

Example:
(py4cl:import-module "math")
(math:sqrt 4) ; => 2.0

or using
Keywords:
AS specifies the name to be used for both the Lisp package and python module. It should be a string, and if not supplied then the module name is used.

RELOAD specifies that the package should be deleted and reloaded.
By default if the package already exists then a string is returned.

Package

py4cl.

Source

import-export.lisp.

Macro: remote-objects (&body body)

Ensures that all values returned by python functions
and methods are kept in python, and only handles returned to lisp. This is useful if performing operations on large datasets.

Package

py4cl.

Source

callpython.lisp.

Macro: remote-objects* (&body body)

Ensures that all values returned by python functions
and methods are kept in python, and only handles returned to lisp.
This is useful if performing operations on large datasets.

This version evaluates the result, returning it as a lisp value if possible.

Package

py4cl.

Source

callpython.lisp.


5.1.3 Ordinary functions

Function: config-var (var)
Package

py4cl.

Source

config.lisp.

Function: (setf config-var) (var)
Package

py4cl.

Source

config.lisp.

Function: export-function (function python-name)

Makes a lisp FUNCTION available in python process as PYTHON-NAME

Package

py4cl.

Source

import-export.lisp.

Function: initialize ()

Intended to be called first upon installation. Sets up default python command, and numpy pickle file and lower bounds.

Package

py4cl.

Source

config.lisp.

Function: load-config ()
Package

py4cl.

Source

config.lisp.

Function: py-cd (path)
Package

py4cl.

Source

config.lisp.

Function: python-alive-p (&optional process)

Returns non-NIL if the python process is alive
(e.g. SBCL -> T, CCL -> RUNNING).
Optionally pass the process object returned by PYTHON-START

Package

py4cl.

Source

python-process.lisp.

Function: python-call (fun-name &rest args)

Call a python function, given the function name as a string
and additional arguments. Keywords are converted to keyword arguments.

Package

py4cl.

Source

callpython.lisp.

Function: python-call-async (fun-name &rest args)

Call a python function asynchronously.
Returns a lambda which when called returns the result.

Package

py4cl.

Source

callpython.lisp.

Function: python-eval (&rest args)

Evaluate an expression in python, returning the result Arguments ARGS can be strings, or other objects. Anything which is not a string is converted to a python value

Examples:

(python-eval "[i**2 for i in range(" 4 ")]") => #(0 1 4 9)

(let ((a 10) (b 2))
(py4cl:python-eval a

Package

py4cl.

Source

callpython.lisp.

Function: (setf python-eval) (&rest args)

Set an expression to a value. Just adds "=" and the value to the end of the expression. Note that the result is evaluated with exec rather than eval.

Examples:

(setf (python-eval "a") 2) ; python "a=2"

Package

py4cl.

Source

callpython.lisp.

Function: python-exec (&rest args)

Execute (using exec) an expression in python. This is used for statements rather than expressions.

Package

py4cl.

Source

callpython.lisp.

Function: python-generator (function stop-value)
Package

py4cl.

Source

callpython.lisp.

Function: python-interrupt (&optional process-info)
Package

py4cl.

Source

python-process.lisp.

Function: python-method (obj method-name &rest args)

Call a given method on an object OBJ. METHOD-NAME can be a symbol (converted to lower case) or a string.

Examples:

(python-method "hello {0}" ’format "world")
; => "hello world"

(python-method ’(1 2 3) ’__len__)
; => 3

Package

py4cl.

Source

callpython.lisp.

Function: python-setf (&rest args)

Set python variables in ARGS ("var1" value1 "var2" value2 ...)

Package

py4cl.

Source

callpython.lisp.

Function: python-start (&optional command)

Start a new python subprocess
This sets the global variable *python* to the process handle,
in addition to returning it.
COMMAND is a string with the python executable to launch e.g. "python" By default this is is set to *PYTHON-COMMAND*

Package

py4cl.

Source

python-process.lisp.

Function: python-start-if-not-alive ()

If no python process is running, tries to start it. If still not alive, raises a condition.

Package

py4cl.

Source

python-process.lisp.

Function: python-stop (&optional process)
Package

py4cl.

Source

python-process.lisp.

Function: python-version-info ()

Return a list, using the result of python’s sys.version_info.

Package

py4cl.

Source

python-process.lisp.

Function: save-config ()
Package

py4cl.

Source

config.lisp.


5.1.4 Generic functions

Generic Function: python-getattr (object slot-name)

Called when python accesses an object’s slot (__getattr__)

Package

py4cl.

Source

lisp-classes.lisp.

Generic Function: python-setattr (object slot-name value)

Called when python sets an object’s slot (__setattr__)

Package

py4cl.

Source

lisp-classes.lisp.


5.1.5 Conditions

Condition: python-error
Package

py4cl.

Source

callpython.lisp.

Direct superclasses

error.

Direct methods

text.

Direct slots
Slot: text
Initargs

:text

Readers

text.

Writers

This slot is read-only.


5.2 Internals


5.2.1 Special variables

Special Variable: *current-python-process-id*

A number which changes when python is started. This
is used to prevent garbage collection from deleting objects in the wrong python session

Package

py4cl.

Source

python-process.lisp.

Special Variable: *freed-python-objects*

A list of handles to be freed. This is used because garbage collection may occur in parallel with the main thread.

Package

py4cl.

Source

reader.lisp.

Special Variable: *handle-counter*
Package

py4cl.

Source

writer.lisp.

Special Variable: *lisp-objects*
Package

py4cl.

Source

writer.lisp.

Special Variable: *numpy-pickle-index*

Used for transferring multiple numpy-pickled arrays in one pyeval/exec/etc

Package

py4cl.

Source

writer.lisp.

Special Variable: *py4cl-tests*

Set nil for something like py4cl/tests::interrupt test, for unknown reasons.

Package

py4cl.

Source

python-process.lisp.

Special Variable: *python*

Most recently started python subprocess

Package

py4cl.

Source

python-process.lisp.


5.2.2 Ordinary functions

Function: clear-lisp-objects ()

Clear the *lisp-objects* object store, allowing them to be GC’d

Package

py4cl.

Source

writer.lisp.

Function: copy-python-object (instance)
Package

py4cl.

Source

reader.lisp.

Function: delete-freed-python-objects ()
Package

py4cl.

Source

reader.lisp.

Function: delete-numpy-pickle-arrays ()

Delete pickled arrays, to free space.

Package

py4cl.

Source

reader.lisp.

Function: dispatch-messages (process)

Read response from python, loop to handle any callbacks

Package

py4cl.

Source

callpython.lisp.

Function: dispatch-reply (stream value)
Package

py4cl.

Source

callpython.lisp.

Function: free-handle (handle)

Remove an object with HANDLE from the hash table

Package

py4cl.

Source

writer.lisp.

Function: free-python-object (python-id handle)
Package

py4cl.

Source

reader.lisp.

Function: function-args (args)

Internal function, intended to be called by the CHAIN macro. Converts function arguments to a list of strings and (pythonize ) function calls. Handles keywords and insertion of commas. Returns a list which can be passed to PYTHON-EVAL.

Examples:

(py4cl::function-args ’(1 :test 2))
=> ((PY4CL::PYTHONIZE 1) "," "test" "=" (PY4CL::PYTHONIZE 2))

Package

py4cl.

Source

callpython.lisp.

Function: lisp-object (handle)

Get the lisp object corresponding to HANDLE

Package

py4cl.

Source

writer.lisp.

Function: make-python-object (&key type handle)
Package

py4cl.

Source

reader.lisp.

Function: make-python-object-finalize (&key type handle)

Make a PYTHON-OBJECT struct with a finalizer.
This deletes the object from the dict store in python.

Uses trivial-garbage (public domain)

Package

py4cl.

Source

reader.lisp.

Function: object-handle (object)

Store OBJECT and return a handle

Package

py4cl.

Source

writer.lisp.

Function: python-eval* (cmd-char &rest args)

Internal function, which converts ARGS into a string to be evaluated This handles both EVAL and EXEC calls with CMD-CHAR being different in the two cases.

Anything in ARGS which is not a string is passed through PYTHONIZE

Package

py4cl.

Source

callpython.lisp.

Reader: python-object-handle (instance)
Writer: (setf python-object-handle) (instance)
Package

py4cl.

Source

reader.lisp.

Target Slot

handle.

Function: python-object-p (object)
Package

py4cl.

Source

reader.lisp.

Reader: python-object-type (instance)
Writer: (setf python-object-type) (instance)
Package

py4cl.

Source

reader.lisp.

Target Slot

type.

Function: stream-read-string (stream)

Reads a string from a stream
Expects a line containing the number of chars following e.g. ’5~%hello’
Returns the string or nil on error

Package

py4cl.

Source

reader.lisp.

Function: stream-read-value (stream)

Get a value from a stream
Currently works by reading a string then using read-from-string

Package

py4cl.

Source

reader.lisp.

Function: stream-write-string (str stream)

Write a string to a stream, putting the length first

Package

py4cl.

Source

writer.lisp.

Function: stream-write-value (value stream)

Write a value to a stream, in a format which can be read by the python subprocess as the corresponding python type

Package

py4cl.

Source

writer.lisp.

Function: take-input (prompt default)
Package

py4cl.

Source

config.lisp.


5.2.3 Generic functions

Generic Function: pythonize (obj)

Convert an object into a string which can be written to stream. Default implementation creates a handle to an unknown Lisp object.

Package

py4cl.

Source

writer.lisp.

Methods
Method: pythonize ((obj ratio))

Handle ratios, using Python’s Fraction if available

Method: pythonize ((obj python-object))

A handle for a python object, stored in a dict in Python

Method: pythonize ((obj function))

Handle a function by converting to a callback object
The lisp function is stored in the same object store as other objects.

Method: pythonize ((obj hash-table))

Convert hash-table to python map. Produces a string {key1:value1, key2:value2,}

Method: pythonize ((obj symbol))

Handle symbols. Need to handle NIL,
converting it to Python None, and convert T to True.

Method: pythonize ((obj string))
Method: pythonize ((obj cons))

Convert a list. This leaves a trailing comma so that python evals a list with a single element as a tuple

Method: pythonize ((obj array))
Method: pythonize ((obj complex))

Create string of the form "(1+2j".
If imaginary part is negative the output is of form "(1+-2j" which is interpreted correctly by python (3.7.2).

Method: pythonize ((obj real))

Write a real number.
Note that python doesn’t handle ’d’,’f’, ’s’ or ’L’ exponent markers

Method: pythonize (obj)
Generic Reader: text (condition)
Package

py4cl.

Methods
Reader Method: text ((condition python-error))
Source

callpython.lisp.

Target Slot

text.


5.2.4 Structures

Structure: python-object

A handle for a python object
which couldn’t be translated into a Lisp value.
TYPE slot is the python type string
HANDLE slot is a unique key used to refer to a value in python.

Package

py4cl.

Source

reader.lisp.

Direct superclasses

structure-object.

Direct methods

pythonize.

Direct slots
Slot: type
Package

common-lisp.

Type

string

Initform

""

Readers

python-object-type.

Writers

(setf python-object-type).

Slot: handle
Readers

python-object-handle.

Writers

(setf python-object-handle).


Appendix A Indexes


A.1 Concepts


A.2 Functions

Jump to:   (  
C   D   E   F   G   I   L   M   O   P   R   S   T  
Index Entry  Section

(
(setf config-var): Public ordinary functions
(setf python-eval): Public ordinary functions
(setf python-object-handle): Private ordinary functions
(setf python-object-type): Private ordinary functions

C
chain: Public macros
clear-lisp-objects: Private ordinary functions
config-var: Public ordinary functions
copy-python-object: Private ordinary functions

D
delete-freed-python-objects: Private ordinary functions
delete-numpy-pickle-arrays: Private ordinary functions
dispatch-messages: Private ordinary functions
dispatch-reply: Private ordinary functions

E
export-function: Public ordinary functions

F
free-handle: Private ordinary functions
free-python-object: Private ordinary functions
Function, (setf config-var): Public ordinary functions
Function, (setf python-eval): Public ordinary functions
Function, (setf python-object-handle): Private ordinary functions
Function, (setf python-object-type): Private ordinary functions
Function, clear-lisp-objects: Private ordinary functions
Function, config-var: Public ordinary functions
Function, copy-python-object: Private ordinary functions
Function, delete-freed-python-objects: Private ordinary functions
Function, delete-numpy-pickle-arrays: Private ordinary functions
Function, dispatch-messages: Private ordinary functions
Function, dispatch-reply: Private ordinary functions
Function, export-function: Public ordinary functions
Function, free-handle: Private ordinary functions
Function, free-python-object: Private ordinary functions
Function, function-args: Private ordinary functions
Function, initialize: Public ordinary functions
Function, lisp-object: Private ordinary functions
Function, load-config: Public ordinary functions
Function, make-python-object: Private ordinary functions
Function, make-python-object-finalize: Private ordinary functions
Function, object-handle: Private ordinary functions
Function, py-cd: Public ordinary functions
Function, python-alive-p: Public ordinary functions
Function, python-call: Public ordinary functions
Function, python-call-async: Public ordinary functions
Function, python-eval: Public ordinary functions
Function, python-eval*: Private ordinary functions
Function, python-exec: Public ordinary functions
Function, python-generator: Public ordinary functions
Function, python-interrupt: Public ordinary functions
Function, python-method: Public ordinary functions
Function, python-object-handle: Private ordinary functions
Function, python-object-p: Private ordinary functions
Function, python-object-type: Private ordinary functions
Function, python-setf: Public ordinary functions
Function, python-start: Public ordinary functions
Function, python-start-if-not-alive: Public ordinary functions
Function, python-stop: Public ordinary functions
Function, python-version-info: Public ordinary functions
Function, save-config: Public ordinary functions
Function, stream-read-string: Private ordinary functions
Function, stream-read-value: Private ordinary functions
Function, stream-write-string: Private ordinary functions
Function, stream-write-value: Private ordinary functions
Function, take-input: Private ordinary functions
function-args: Private ordinary functions

G
Generic Function, python-getattr: Public generic functions
Generic Function, python-setattr: Public generic functions
Generic Function, pythonize: Private generic functions
Generic Function, text: Private generic functions

I
import-function: Public macros
import-module: Public macros
initialize: Public ordinary functions

L
lisp-object: Private ordinary functions
load-config: Public ordinary functions

M
Macro, chain: Public macros
Macro, import-function: Public macros
Macro, import-module: Public macros
Macro, remote-objects: Public macros
Macro, remote-objects*: Public macros
make-python-object: Private ordinary functions
make-python-object-finalize: Private ordinary functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, pythonize: Private generic functions
Method, text: Private generic functions

O
object-handle: Private ordinary functions

P
py-cd: Public ordinary functions
python-alive-p: Public ordinary functions
python-call: Public ordinary functions
python-call-async: Public ordinary functions
python-eval: Public ordinary functions
python-eval*: Private ordinary functions
python-exec: Public ordinary functions
python-generator: Public ordinary functions
python-getattr: Public generic functions
python-interrupt: Public ordinary functions
python-method: Public ordinary functions
python-object-handle: Private ordinary functions
python-object-p: Private ordinary functions
python-object-type: Private ordinary functions
python-setattr: Public generic functions
python-setf: Public ordinary functions
python-start: Public ordinary functions
python-start-if-not-alive: Public ordinary functions
python-stop: Public ordinary functions
python-version-info: Public ordinary functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions
pythonize: Private generic functions

R
remote-objects: Public macros
remote-objects*: Public macros

S
save-config: Public ordinary functions
stream-read-string: Private ordinary functions
stream-read-value: Private ordinary functions
stream-write-string: Private ordinary functions
stream-write-value: Private ordinary functions

T
take-input: Private ordinary functions
text: Private generic functions
text: Private generic functions


A.4 Data types

Jump to:   C   D   F   I   L   P   R   S   W  
Index Entry  Section

C
callpython.lisp: The py4cl/callpython․lisp file
Condition, python-error: Public conditions
config.lisp: The py4cl/config․lisp file

D
do-after-load.lisp: The py4cl/do-after-load․lisp file

F
File, callpython.lisp: The py4cl/callpython․lisp file
File, config.lisp: The py4cl/config․lisp file
File, do-after-load.lisp: The py4cl/do-after-load․lisp file
File, import-export.lisp: The py4cl/import-export․lisp file
File, lisp-classes.lisp: The py4cl/lisp-classes․lisp file
File, package.lisp: The py4cl/package․lisp file
File, py4cl.asd: The py4cl/py4cl․asd file
File, python-process.lisp: The py4cl/python-process․lisp file
File, reader.lisp: The py4cl/reader․lisp file
File, writer.lisp: The py4cl/writer․lisp file

I
import-export.lisp: The py4cl/import-export․lisp file

L
lisp-classes.lisp: The py4cl/lisp-classes․lisp file

P
Package, py4cl: The py4cl package
Package, py4cl/config: The py4cl/config package
package.lisp: The py4cl/package․lisp file
py4cl: The py4cl system
py4cl: The py4cl package
py4cl.asd: The py4cl/py4cl․asd file
py4cl/config: The py4cl/config package
python-error: Public conditions
python-object: Private structures
python-process.lisp: The py4cl/python-process․lisp file

R
reader.lisp: The py4cl/reader․lisp file

S
Structure, python-object: Private structures
System, py4cl: The py4cl system

W
writer.lisp: The py4cl/writer․lisp file