This is the py4cl2 Reference Manual, version 2.9.2, generated automatically by Declt version 4.0 beta 2 "William Riker" on Fri Sep 15 06:21:05 2023 GMT+0.
py4cl2/py4cl2.asdpy4cl2/package.lisppy4cl2/config.lisppy4cl2/features.lisppy4cl2/python-process.lisppy4cl2/reader.lisppy4cl2/writer.lisppy4cl2/lisp-classes.lisppy4cl2/callpython.lisppy4cl2/arg-list.lisppy4cl2/import-export.lisppy4cl2/do-after-load.lispThe main system appears first, followed by any subsystem dependency.
py4cl2Some improvements over py4cl. py4cl is a library for interfacing with python libraries from common lisp, using streams to communicate with the python process.
Report the issues at https://github.com/digikar99/py4cl2/issues
(More) Documentation is available at https://digikar99.github.io/py4cl2/
MIT
2.9.2
alexandria (system).
bordeaux-threads (system).
cl-json (system).
float-features (system).
iterate (system).
numpy-file-format (system).
parse-number (system).
trivial-garbage (system).
uiop (system).
python-code (file).
package.lisp (file).
config.lisp (file).
features.lisp (file).
python-process.lisp (file).
reader.lisp (file).
writer.lisp (file).
lisp-classes.lisp (file).
callpython.lisp (file).
arg-list.lisp (file).
import-export.lisp (file).
do-after-load.lisp (file).
.config (file).
Files are sorted by type and then listed depth-first from the systems components trees.
py4cl2/py4cl2.asdpy4cl2/package.lisppy4cl2/config.lisppy4cl2/features.lisppy4cl2/python-process.lisppy4cl2/reader.lisppy4cl2/writer.lisppy4cl2/lisp-classes.lisppy4cl2/callpython.lisppy4cl2/arg-list.lisppy4cl2/import-export.lisppy4cl2/do-after-load.lisppy4cl2/package.lisppython-code (file).
py4cl2 (system).
py4cl2/config.lisppackage.lisp (file).
py4cl2 (system).
*config* (special variable).
*lispifiers* (special variable).
*pythonizers* (special variable).
config-var (function).
(setf config-var) (function).
initialize (function).
load-config (function).
py-cd (function).
save-config (function).
with-lispifiers (macro).
with-pythonizers (macro).
%pythonize (function).
+py4cl2-config-path+ (constant).
customize (function).
no-lispifier-found (condition).
take-input (function).
py4cl2/features.lisppackage.lisp (file).
config.lisp (file).
py4cl2 (system).
*internal-features* (special variable).
*warn-on-unavailable-feature-usage* (special variable).
*feature-exclusion-alist* (special variable).
numpy-installed-p (function).
py4cl2/python-process.lisppackage.lisp (file).
features.lisp (file).
py4cl2 (system).
*additional-init-codes* (special variable).
pyinterrupt (function).
pystart (function).
pystop (function).
python-alive-p (function).
python-process-startup-error (condition).
python-start-if-not-alive (function).
with-python-output (macro).
*current-python-process-id* (special variable).
*in-with-python-output* (special variable).
*py4cl-tests* (special variable).
*python* (special variable).
*python-code* (special variable).
*python-output-semaphore* (special variable).
*python-output-thread* (special variable).
*python-process-busy-p* (special variable).
*python-startup-error* (special variable).
command (reader method).
py4cl2/reader.lisppackage.lisp (file).
python-process.lisp (file).
py4cl2 (system).
*print-python-object* (special variable).
print-object (method).
python-object (structure).
python-object-type (reader).
(setf python-object-type) (writer).
*freed-python-objects* (special variable).
copy-python-object (function).
delete-freed-python-objects (function).
delete-numpy-pickle-arrays (function).
free-python-object (function).
make-python-object (function).
make-python-object-finalize (function).
python-object-handle (reader).
(setf python-object-handle) (writer).
python-object-p (function).
stream-read-string (function).
stream-read-value (function).
py4cl2/writer.lisppackage.lisp (file).
features.lisp (file).
reader.lisp (file).
py4cl2 (system).
pythonize (generic function).
*handle-counter* (special variable).
*lisp-objects* (special variable).
*lisp-to-python-alist* (special variable).
*numpy-pickle-index* (special variable).
clear-lisp-objects (function).
free-handle (function).
lisp-object (function).
object-handle (function).
stream-write-string (function).
stream-write-value (function).
py4cl2/lisp-classes.lisppackage.lisp (file).
writer.lisp (file).
py4cl2 (system).
python-getattr (generic function).
python-setattr (generic function).
py4cl2/callpython.lispreader.lisp (file).
writer.lisp (file).
python-process.lisp (file).
lisp-classes.lisp (file).
py4cl2 (system).
chain (macro).
chain* (function).
(setf chain*) (function).
pycall (function).
pyerror (condition).
pyeval (function).
(setf pyeval) (function).
pyexec (function).
pygenerator (function).
pyhelp (function).
pymethod (function).
pyslot-value (function).
(setf pyslot-value) (function).
python-eof-and-dead (condition).
python-eof-but-alive (condition).
pyversion-info (function).
raw-pyeval (function).
(setf raw-pyeval) (function).
raw-pyexec (function).
with-remote-objects (macro).
with-remote-objects* (macro).
%chain* (function).
%pycall-args (function).
*python-lock* (special variable).
dispatch-messages (function).
dispatch-reply (function).
python-process (reader method).
python-process (reader method).
raw-py (function).
text (reader method).
py4cl2/arg-list.lisppackage.lisp (file).
callpython.lisp (file).
py4cl2 (system).
%get-arg-list (method).
%get-arg-list (method).
%get-arg-list (method).
%get-arg-list (method).
both-positional-and-keyword-rest (condition).
break-into-words (function).
collect-first-word (function).
default-arg-list-necessary (condition).
default-is-python-object (condition).
get-arg-list (function).
get-unique-symbol (function).
lispify-name (function).
numeric-char-p (function).
pythonize-kwargs (function).
split-point-p (function).
py4cl2/import-export.lisppython-process.lisp (file).
callpython.lisp (file).
arg-list.lisp (file).
py4cl2 (system).
*defpymodule-silent-p* (special variable).
defpyfun (macro).
defpyfuns (macro).
defpymodule (macro).
export-function (function).
pymethod-list (function).
pyslot-list (function).
*called-from-defpymodule* (special variable).
*defpymodule-cache* (special variable).
*function-reload-string* (special variable).
*is-submodule* (special variable).
*lisp-package-supplied-p* (special variable).
builtin-p (function).
defpyfun* (function).
defpymodule* (function).
defpysubmodules (function).
defvar-doc (macro).
ensure-package (function).
fun-symbol (function).
function-reload-string (function).
pymodule-import-string (function).
py4cl2/do-after-load.lispimport-export.lisp (file).
py4cl2 (system).
Packages are listed by definition order.
py4cl2alexandria.
common-lisp.
iterate.
*additional-init-codes* (special variable).
*config* (special variable).
*defpymodule-silent-p* (special variable).
*internal-features* (special variable).
*lispifiers* (special variable).
*print-python-object* (special variable).
*pythonizers* (special variable).
*warn-on-unavailable-feature-usage* (special variable).
chain (macro).
chain* (function).
(setf chain*) (function).
config-var (function).
(setf config-var) (function).
defpyfun (macro).
defpyfuns (macro).
defpymodule (macro).
export-function (function).
initialize (function).
load-config (function).
py-cd (function).
pycall (function).
pyerror (condition).
pyeval (function).
(setf pyeval) (function).
pyexec (function).
pygenerator (function).
pyhelp (function).
pyinterrupt (function).
pymethod (function).
pymethod-list (function).
pyslot-list (function).
pyslot-value (function).
(setf pyslot-value) (function).
pystart (function).
pystop (function).
python-alive-p (function).
python-eof-and-dead (condition).
python-eof-but-alive (condition).
python-getattr (generic function).
python-object (structure).
python-object-type (reader).
(setf python-object-type) (writer).
python-process-startup-error (condition).
python-setattr (generic function).
python-start-if-not-alive (function).
pythonize (generic function).
pyversion-info (function).
raw-pyeval (function).
(setf raw-pyeval) (function).
raw-pyexec (function).
save-config (function).
with-lispifiers (macro).
with-python-output (macro).
with-pythonizers (macro).
with-remote-objects (macro).
with-remote-objects* (macro).
%chain* (function).
%get-arg-list (generic function).
%pycall-args (function).
%pythonize (function).
*called-from-defpymodule* (special variable).
*current-python-process-id* (special variable).
*defpymodule-cache* (special variable).
*feature-exclusion-alist* (special variable).
*freed-python-objects* (special variable).
*function-reload-string* (special variable).
*handle-counter* (special variable).
*in-with-python-output* (special variable).
*is-submodule* (special variable).
*lisp-objects* (special variable).
*lisp-package-supplied-p* (special variable).
*lisp-to-python-alist* (special variable).
*numpy-pickle-index* (special variable).
*py4cl-tests* (special variable).
*python* (special variable).
*python-code* (special variable).
*python-lock* (special variable).
*python-output-semaphore* (special variable).
*python-output-thread* (special variable).
*python-process-busy-p* (special variable).
*python-startup-error* (special variable).
+py4cl2-config-path+ (constant).
both-positional-and-keyword-rest (condition).
break-into-words (function).
builtin-p (function).
clear-lisp-objects (function).
collect-first-word (function).
command (generic reader).
copy-python-object (function).
customize (function).
default-arg-list-necessary (condition).
default-is-python-object (condition).
defpyfun* (function).
defpymodule* (function).
defpysubmodules (function).
defvar-doc (macro).
delete-freed-python-objects (function).
delete-numpy-pickle-arrays (function).
dispatch-messages (function).
dispatch-reply (function).
ensure-package (function).
free-handle (function).
free-python-object (function).
fun-symbol (function).
function-reload-string (function).
get-arg-list (function).
get-unique-symbol (function).
lisp-object (function).
lispify-name (function).
make-python-object (function).
make-python-object-finalize (function).
no-lispifier-found (condition).
numeric-char-p (function).
numpy-installed-p (function).
object-handle (function).
pymodule-import-string (function).
python-object-handle (reader).
(setf python-object-handle) (writer).
python-object-p (function).
python-process (generic reader).
pythonize-kwargs (function).
raw-py (function).
split-point-p (function).
stream-read-string (function).
stream-read-value (function).
stream-write-string (function).
stream-write-value (function).
take-input (function).
text (generic reader).
Definitions are sorted by export status, category, package, and then by lexicographic order.
A list of strings each of which should be python code. All the code
will be executed by PYSTART. The code should not contain single-quotation marks.
Configuration variable used to store configuration values for PY4CL2. This variable should be manipulated using CONFIG-VAR and (SETF CONFIG-VAR).
DEFPYMODULE avoids printing progress if this is T.
A list of PY4CL2 features available on the system. (Support for :ARRAYS requires numpy and is only determinable after python process has started.)
The list can include one or more of:
:WITH-PYTHON-OUTPUT
:TYPED-ARRAYS
:FAST-LARGE-ARRAY-TRANSFER
:INTERRUPT
Each entry in the alist *LISPIFIERS* maps from a lisp-type to
a single-argument lisp function. This function takes as input the "default" lisp
objects and is expected to appropriately parse it to the corresponding lisp object.
NOTE: This is a new feature and hence unstable; recommended to avoid in production code.
If non-NIL, python’s ’str’ is called on the python-object before printing.
Each entry in the alist *PYTHONIZERS* maps from a lisp-type to
a single-argument PYTHON-FUNCTION-DESIGNATOR. This python function takes as input the
"default" python objects and is expected to appropriately convert it to the corresponding
python object.
NOTE: This is a new feature and hence unstable; recommended to avoid in production code.
If non-NIL, then issues a WARNING on use of a feature unavailable on the platform. See PY4CL2:*INTERNAL-FEATURES* for the list of available features.
Defines a function which calls python
Example
(py4cl:pyexec "import math")
(py4cl:defpyfun "math.sqrt")
(math.sqrt 42) -> 6.4807405
Arguments:
FUN-NAME: name of the function in python, before import
PYMODULE-NAME: name of the module containing FUN-NAME
AS: name of the function in python, after import
CACHE: if non-NIL, constructs the function body at macroexpansion time
LISP-FUN-NAME: name of the lisp symbol to which the function is bound*
LISP-PACKAGE: package (not its name) in which LISP-FUN-NAME will be interned
SAFETY: if T, adds an additional line in the function asking to import the
package or function, so that the function works even after PYSTOP is called.
However, this increases the overhead of stream communication, and therefore,
can reduce speed.
Each ARG is supposed to be a 2-3 element list of
(pyfun-name pymodule-name) or (pyfun-name pymodule-name lisp-fun-name).
In addition, when ARG is a 2-element list, then, the first element can be
a list of python function names.
Import a python module (and its submodules) as a lisp-package(s).
Example:
(py4cl:defpymodule "math" :lisp-package "M")
(m:sqrt 4) ; => 2.0
Arguments:
PYMODULE-NAME: name of the module in python, before importing
IMPORT-SUBMODULES: leave nil for purposes of speed, if you won’t use the
submodules
CACHE: if non-NIL, produces the DEFPACKAGE and DEFUN forms at macroexpansion time
to speed-up future reloads of the system
LISP-PACKAGE: lisp package, in which to intern (and export) the callables
RECOMPILE-ON-CHANGE: the name of the ASDF system to recompile if the python version of
PYMODULE-NAME changes; this only has effect if CACHE is non-NIL
RELOAD: redefine the LISP-PACKAGE if T
SAFETY: value of safety to pass to defpyfun; see defpyfun
SILENT: prints "status" lines when NIL
Each entry of OVERRIDING-LISPIFIERS is a two-element list of the form
(TYPE LISPIFIER)
Here, TYPE is unevaluated, while LISPIFIER will be evaluated; the LISPIFIER is expected
to take a default-lispified object (see lisp-python types translation table in docs)
and return the appropriate object user expects.
For example,
(PYEVAL "[1, 2, 3]") ;=> #(1 2 3) ; the default lispified object
(with-lispifiers ((vector (lambda (x) (coerce (print x) ’list))))
(print (pyeval "[1,2,3]"))
(print (pyeval 5)))
; #(1 2 3) ; default lispified object
; (1 2 3) ; coerced to LIST by the lispifier
; 5 ; lispifier uncalled for non-VECTOR
5
NOTE: This is a new feature and hence unstable; recommended to avoid in production code.
Gets the output of the python program executed in FORMS-DECL in the form a string.
Each entry of OVERRIDING-PYTHONIZERS is a two-element list of the form
(TYPE PYTHONIZER)
Here, TYPE is unevaluated, while PYTHONIZER will be evaluated; the PYTHONIZER is expected
to take a default-pythonized object (see lisp-python types translation table in docs)
and return the appropriate object user expects.
For example,
(pyeval "[1, 2, 3]") ;=> #(1 2 3) ; the default object
(with-pythonizers ((vector "tuple"))
(print (pyeval "[1,2,3]"))
(print (pyeval 5)))
; #(1 2 3) ; default object
; (1 2 3) ; coerced to tuple by the pythonizer, which then translates to list
; 5 ; pythonizer uncalled for non-VECTOR
5
NOTE: This is a new feature and hence unstable; recommended to avoid in production code.
Ensures that all values returned by python functions
and methods are kept in python, and only handles returned to lisp.
This is useful if performing operations on large datasets.
Ensures that all values returned by python functions
and methods are kept in python, and only handles returned to lisp.
This is useful if performing operations on large datasets. Unlike
with-remote-objects, evaluates the last result and returns not just a handle.
Returns the value associated with VAR in *CONFIG*.
Configuration variables include (all in PY4CL2 package):
- PYCMD: Path to the python binary to be used
- NUMPY-PICKLE-LOCATION: PY4CL2 uses pickled files to transfer large arrays between lisp
and python efficiently. These can have sizes exceeding 100MB. It is recommended that this
be set to path on a ram-disk. See [this](https://unix.stackexchange.com/questions/66329/creating-a-ram-disk-on-linux) for
instructions on creating a ram-disk on linux-based systems.
- NUMPY-PICKLE-LOWER-BOUND: The minimum size of the array for which PY4CL2 should use pickled files.
- PRINT-PYTHON-TRACEBACK: If non-NIL, the traceback on python’s side gets printed in
case of a python error. (NOTE: This is a new-feature, hence unstable)
Sets the value of VAR to NEW-VALUE in *CONFIG*. For all but PYCMD, the values are saved to a configuration-file for persistence whenever they are changed. To persist PYCMD, call SAVE-CONFIG.
Makes a lisp FUNCTION available in python process as PYTHON-NAME
Intended to be called first upon installation. Sets up default python command, and numpy pickle file and lower bounds.
Load to *CONFIG* from /home/quickref/quicklisp/dists/quicklisp/software/py4cl2-v2.9.2/src/../.config
Calls FUN-NAME with ARGS as arguments. Arguments can be keyword based, or otherwise.
Calls python eval on args; PYTHONIZEs arg if it satisfies PYTHONIZEP.
Eg.
> (let ((a 5)) (pyeval a "*" a))
25
Set an expression to a value. Just adds "=" and the value
to the end of the expression. Note that the result is evaluated
with exec rather than eval.
Example:
(setf (pyeval "a") 2) ; python "a=2"
Can be useful for modifying a value directly in python.
Calls python exec on args; PYTHONIZEs arg if it satisfies PYTHONIZEP.
Issues a SIGINT command to the python process
PYCALLs METHOD of OBJECT with ARGS
Examples:
> (pymethod "’hello {0}’" ’format "world")
"hello world"
> (pymethod ’(1 2 3) ’–len–)
3
Note: FUN-NAME is NOT PYTHONIZEd if it is a string.
Start a new python subprocess
This sets the global variable *python* to the process handle,
in addition to returning it.
COMMAND is a string with the python executable to launch e.g. "python"
By default this is is set to (CONFIG-VAR ’PYCMD)
Stop (Quit) the python process PROCESS
Returns non-NIL if the python process is alive
(e.g. SBCL -> T, CCL -> RUNNING).
Optionally pass the process object returned by PYTHON-START
type.
If no python process is running, tries to start it. If still not alive, raises a condition.
Return a list, using the result of python’s sys.version_info.
Calls python eval on the concatenation of strings, as they are, without any pythonization or modification.
Calls python exec on the concatenation of strings, as they are, without any
pythonization or modification.
NOTE: Like usual, there are peculiarities to exec commands.
For instance,
import sys
def foo:
sys.stdout.write(’hello’)
foo()
will result in ’sys’ name not defined PYERROR.
Save to /home/quickref/quicklisp/dists/quicklisp/software/py4cl2-v2.9.2/src/../.config from *CONFIG*
Called when python accesses an object’s slot (__getattr__)
Called when python sets an object’s slot (__setattr__)
Convert an object into a string which can be written to stream. Default implementation creates a handle to an unknown Lisp object.
ratio)) ¶Handle ratios, using Python’s Fraction if available
python-object)) ¶A handle for a python object, stored in a dict in Python
function)) ¶Handle a function by converting to a callback object
The lisp function is stored in the same object store as other objects.
hash-table)) ¶Convert hash-table to python map. Produces a string {key1:value1, key2:value2,}
symbol)) ¶Mapping from lisp symbol names to python names:
symbol : symbol-name : python-name
’foo-bar : "FOO-BAR" : foo_bar
’|Foo-Bar| : "Foo-Bar" : Foo_Bar
’*foo-bar* : "*FOO-BAR*" : FOO_BAR
t : "T" : True
nil : "NIL" : False
string)) ¶cons)) ¶Convert a list. This leaves a trailing comma so that python evals a list with a single element as a tuple
array)) ¶complex)) ¶Create string of the form "(1+2j".
If imaginary part is negative the output is of form "(1+-2j"
which is interpreted correctly by python (3.7.2).
real)) ¶Write a real number.
Note that python doesn’t handle ’d’,’f’, ’s’ or ’L’ exponent markers
end-of-file.
:python-process
This slot is read-only.
end-of-file.
:python-process
This slot is read-only.
error.
A handle for a python object
which couldn’t be translated into a Lisp value.
TYPE slot is the python type string
HANDLE slot is a unique key used to refer to a value in python.
Internal variable used for coordinating between DEFPYMODULE and DEFPYFUN.
A number which changes when python is started. This
is used to prevent garbage collection from deleting objects in the wrong
python session
If non-NIL, DEFPYMODULE produces the expansion during macroexpansion time. Use intended for DEFPYSUBMODULES.
A list of handles to be freed. This is used because garbage collection may occur in parallel with the main thread.
String pyexec-ed at the start of a DEFPYFUN when SAFETY is T.
Used for coordinating import statements from defpymodule while calling recursively
Internal variable used by PYMODULE-IMPORT-STRING to determine the import string.
Used for transferring multiple numpy-pickled arrays in one pyeval/exec/etc
Most recently started python subprocess
Used by pyinterrupt to determine if python process is waiting for input, or
is busy processing.
A possible workaround is to use strace.
See https://askubuntu.com/questions/1118109/how-do-i-tell-if-a-command-is-running-or-waiting-for-user-input
A wrapper around PYTHONIZE to take custom *PYTHONIZERS* into account.
Returns ((h e l l o) (W o r l d)), given (h e l l o W o r l d).
Some builtin functions like ’sum’ do not take keyword args.
Clear the *lisp-objects* object store, allowing them to be GC’d
Returns ((h e l l o) W o r l d), given (h e l l o W o r l d).
Returns multiple values:
- a DEFVAR form to capture the existence of package before ensuring it
- an ENSURE-PACKAGE form
- the actual form that defines the package and functions
Delete pickled arrays, to free space.
Read response from python, loop to handle any callbacks
Remove an object with HANDLE from the hash table
Returns a list of two lists: PARAMETER-LIST and PASS_LIST
Get the lisp object corresponding to HANDLE
Converts NAME to a lisp-like name. Specifically:
1. Replaces underscores with hyphens.
2. CamelCase is converted to CAMEL-CASE
Make a PYTHON-OBJECT struct with a finalizer.
This deletes the object from the dict store in python.
Uses trivial-garbage (public domain)
Store OBJECT and return a handle
Intended as an abstraction to RAW-PYEVAL and RAW_PYEXEC. Passes strings as they are, without any ’pythonize’ation.
Reads a string from a stream
Expects a line containing the number of chars following
e.g. ’5~%hello’
Returns the string or nil on error
Get a value from a stream
Currently works by reading a string then using read-from-string
Write a string to a stream, putting the length first
Write a value to a stream, in a format which can be read by the python subprocess as the corresponding python type
(eql :|type|)) fullname lisp-package) ¶(eql :|builtin_function_or_method|)) fullname lisp-package) ¶(eql :|numpy.ufunc|)) fullname lisp-package) ¶python-process-startup-error)) ¶python-eof-and-dead)) ¶python-eof-but-alive)) ¶condition.
condition.
| Jump to: | %
(
B C D E F G I L M N O P R S T W |
|---|
| Jump to: | %
(
B C D E F G I L M N O P R S T W |
|---|
| Jump to: | *
+
C H P S T |
|---|
| Jump to: | *
+
C H P S T |
|---|
| Jump to: | .
A B C D F I L N P R S W |
|---|
| Jump to: | .
A B C D F I L N P R S W |
|---|