The cxx-jit Reference Manual

This is the cxx-jit Reference Manual, version 1.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Sun Sep 15 03:52:15 2024 GMT+0.

Table of Contents


1 Introduction


2 Systems

The main system appears first, followed by any subsystem dependency.


2.1 cxx-jit

Common Lisp Cxx Interoperation

Author

Islam Omar

License

MIT

Long Description

#+OPTIONS: toc:3 (only include two levels in TOC)
* CL-CXX-JIT - Common Lisp C++ JIT for exposing C++ functions

This library provides an interface to C++ from lisp. It compiles C++ code, then loads it into lisp. It was inspired by [[https://github.com/onqtam/rcrl][RCRL]] and the older project [[https://github.com/Islam0mar/cl-cxx][CL-CXX]].

** API
=(from ’("list of string headers") ’import "normal string is inserted as it is" ’("function/function-pointer/mem-fun-pointer/lambda" . "name-used-in-lisp"))=

** Examples
*** SDL2 Example
[[sdl2.gif]]
#+begin_src lisp
(ql:quickload :cxx-jit)
(setf cxx-jit:*cxx-compiler-link-libs* "-lGL -lSDL2 -lSDL2main")

(cxx-jit:from ’(
"<SDL2/SDL.h>")
’import
’("[](){return SDL_Init(SDL_Init(SDL_INIT_VIDEO));}" . "init")
’("SDL_CreateWindow" . "create-window")
’("SDL_CreateRenderer" . "create-renderer")
’("SDL_SetRenderDrawColor" . "set-color")
’("SDL_DestroyWindow" . "destroy-window")
’("SDL_RenderClear" . "clear-renderer")
’("SDL_RenderPresent" . "renderer-render")
’("SDL_Quit" . "sdl-quit"))

(init)
(setf wind (create-window "create-window" 0 0 600 700 0))
(setf rend (create-renderer wind -1 0))
(loop for x to (* 255 3)
for r = (if (> x 255) 255 x)
for g = (if (> x 255) (if (> x (* 2 255)) 255 (rem x 256)) 0)
for b = (if (> x (* 2 255)) (rem x 256) 0)
do
(print x)
(set-color rend r g b 255)
(clear-renderer rend)
(renderer-render rend)
(sleep 0.01))

(destroy-window wind)
(sdl-quit)
#+end_src

*** Basic Example
Start with =(ql:quickload :cxx-jit) (in-package cxx-jit)=
#+begin_src lisp
(from ’("<string>") ’import ’("[](std::string x){return \"Hi, \"+x;}" . "hi"))
(hi "there!")

(from ’("<cmath>") ’import ’("static_cast<double(*)(double)>(std::sin)" . "cpp-sin"))
(cpp-sin 0d0)
(cpp-sin pi)
(cpp-sin (/ pi 2))

(from nil ’import "struct C{ auto hi(){return \"Hello, World\\n\";} auto bye(){return \"Bye\";} };" ’("&C::bye" . "bye") ’("&C::hi" . "hi") ’("[](){static C x; return x;}" . "cc"))
(cc)
(hi *)
(bye **)
;;; structure definition could be written in a header file then be used as the following:
(from ’("c.hpp") ’import ’("&C::bye" . "bye") ’("&C::hi" . "hi") ’("[](){static C x; return x;}" . "cc"))
#+end_src

*** Eigen Library Example
Start with =(ql:quickload :cxx-jit) (in-package cxx-jit)=
#+begin_src lisp
(from ’("<Eigen/Core>" "<Eigen/Core>") ’import ’("[](Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> x){
std::stringstream s;
s << x;
return s.str();}" . "print-matrix"))

(from ’("<Eigen/Core>" "<Eigen/Core>") ’import ’("static_cast<const Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> (*)()> (&Eigen::Matrix3d::Identity)" . "identity-matrix"))

(print-matrix (identity-matrix))
#+end_src

** Prerequisites
- common lisp supporting [[https://www.common-lisp.net/project/cffi/][CFFI]]
- working C++17 compiler
- the following variables should be set according to OS and compiler used
|———————————————+————————————————————————————————|
| variable | default value | |———————————————+————————————————————————————————|
| =*cxx-compiler-executable-path*= | "~/usr/bin/g++~" |
| =*cxx-compiler-flags*= | "~-std=c++17 -Wall -Wextra -I/usr/include/eigen3~" |
| =*cxx-compiler-working-directory*= | "~/tmp/~" #\/ ’/’ should be the last character |
| =+cxx-compiler-lib-name+= | (intern "~plugin~") |
| =+cxx-compiler-wrap-cxx-path+= | shouldn’t be changed "~path to wrap-cxx.cpp~" |
| =*cxx-compiler-internal-flags*= | "~-shared -fPIC -Wl,–no-undefined -Wl,–no-allow-shlib-undefined~" |
| | for g++ and for clang++ "~-shared -fPIC -Wl,-undefined,error -Wl,-flat_namespace~" |
| =*cxx-compiler-link-libs*= | "~-lm~" these flags are added after "~-o output~" to link correctly |
| =*cxx-type-name-to-cffi-type-symbol-alist*= | alist to map c++ type name to cffi type example:"~(push (cons "long unsigned int" :ulong) *)~" | |———————————————+————————————————————————————————|

** Installation
Clone into home/common-lisp directory. Then =(ql:quickload :cxx-jit-test)=

** Supported Types
|——————+——————|
| C++ type | Lisp cffi type |
|——————+——————|
| fundamental | same |
| string | :string |
| class | :pointer |
| std::is_function | :pointer |
| other | not implemented! |
|——————+——————|

** Under The Hood
+ function/lambda/member_function/function_pointer is wrapped into a dummy lambda class to have a unique template specialization.
#+begin_src C++
Import([&]() { return __VA_ARGS__; });
#+end_src
+ =Import= function calls =DecayThenResolve= with function pointer as the template specialization so thunk pointer is omitted and we only return the direct function pointer which will be used from lisp side.
+ =InvocableTypeName= returns a vector contains: [return type, class type for class function member, args]. It resolves C++ types as follows:
* Fundamental types and pointers are passed directly
* String is converted to char* with new[] operator, should be cleared with =ClCxxDeleteObject(ptr, true)=
* Class/std::is_function is converted to void* with new[] operator, should be cleared with =ClCxxDeleteObject(ptr, false)=
* rest report an issue for other cases
+ Meta data for each function defined is passed through a lisp callback with this data:
#+begin_src C++
typedef struct {
// could be void*
void (*thunk_ptr)();
bool method_p;
const char **type; // memory handled in C++
std::uint8_t type_size;
} MetaData;
#+end_src

** NOTE
Tested on:
- SBCL 2.0.1 on debian

** Todo List
*** TODO Add redirect stdout : =freopen("/tmp/tmp.txt", "w", stdout);= @apemangr
*** TODO Use trivial-garbage with =ClCxxDeleteObject=
*** TODO Add non-polling =from=
*** TODO Test functions
*** TODO Benchmark
*** TODO Better class interface

** Copyright

Copyright (c) 2021 Islam Omar (io1131@fayoum.edu.eg)

** License

Licensed under the MIT License.

Version

1.0

Dependencies
  • cffi (system).
  • uiop (system).
  • trivial-garbage (system).
Source

cxx-jit.asd.

Child Components

3 Modules

Modules are listed depth-first from the system components tree.


3.1 cxx-jit/src

Source

cxx-jit.asd.

Parent Component

cxx-jit (system).

Child Component

cxx-jit.lisp (file).


4 Files

Files are sorted by type and then listed depth-first from the systems components trees.


4.1 Lisp


4.1.1 cxx-jit/cxx-jit.asd

Source

cxx-jit.asd.

Parent Component

cxx-jit (system).

ASDF Systems

cxx-jit.

Packages

cxx-jit/system.


4.1.2 cxx-jit/package.lisp

Source

cxx-jit.asd.

Parent Component

cxx-jit (system).

Packages

cxx-jit.


4.1.3 cxx-jit/src/cxx-jit.lisp

Source

cxx-jit.asd.

Parent Component

src (module).

Public Interface
Internals

5 Packages

Packages are listed by definition order.


5.1 cxx-jit

Source

package.lisp.

Use List
  • cffi.
  • common-lisp.
  • trivial-garbage.
  • uiop/driver.
Public Interface
Internals

5.2 cxx-jit/system

Source

cxx-jit.asd.

Use List
  • asdf/interface.
  • common-lisp.

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


6.1 Public Interface


6.1.1 Constants

Constant: +cxx-compiler-lib-name+
Package

cxx-jit.

Source

cxx-jit.lisp.

Constant: +cxx-compiler-wrap-cxx-path+
Package

cxx-jit.

Source

cxx-jit.lisp.


6.1.2 Special variables

Special Variable: *cxx-compiler-flags*
Package

cxx-jit.

Source

cxx-jit.lisp.

Special Variable: *cxx-compiler-internal-flags*
Package

cxx-jit.

Source

cxx-jit.lisp.

Package

cxx-jit.

Source

cxx-jit.lisp.

Special Variable: *cxx-compiler-packages*
Package

cxx-jit.

Source

cxx-jit.lisp.

Special Variable: *cxx-compiler-packages-number*
Package

cxx-jit.

Source

cxx-jit.lisp.

Special Variable: *cxx-compiler-process*
Package

cxx-jit.

Source

cxx-jit.lisp.

Special Variable: *cxx-compiler-working-directory*
Package

cxx-jit.

Source

cxx-jit.lisp.

Special Variable: *cxx-type-name-to-cffi-type-symbol-alist*
Package

cxx-jit.

Source

cxx-jit.lisp.


6.1.3 Ordinary functions

Function: from (header-names import &rest body)

import cxx functions/methods from the header

Package

cxx-jit.

Source

cxx-jit.lisp.

Function: try-get-cxx-compiler-output ()

returns nil if compiler process is compiling else returns the exit value from the process

Package

cxx-jit.

Source

cxx-jit.lisp.


6.2 Internals


6.2.1 Special variables

Special Variable: *cxx--fun-names*
Package

cxx-jit.

Source

cxx-jit.lisp.

Special Variable: *cxx-compiler-executable-path*
Package

cxx-jit.

Source

cxx-jit.lisp.


6.2.2 Ordinary functions

Function: cffi-type (type)

Returns cffi-type as a keyword

Package

cxx-jit.

Source

cxx-jit.lisp.

Function: compile-code (code)

compile aync. code string with cxx compiler

Package

cxx-jit.

Source

cxx-jit.lisp.

Function: copy-and-load-new-library ()

if compilation suceceded copy plugin.so to plugin_x.so ,where x = 0,1,...
then load the library

Package

cxx-jit.

Source

cxx-jit.lisp.

Function: parse-input-args (arg-types)

return argument types (with variables if they are inputs) in a proper list

Package

cxx-jit.

Source

cxx-jit.lisp.

Function: symbols-list (arg-types &optional method-p)

Return a list of symbols ’(V0 V1 V2 V3 ...) representing the number of args

Package

cxx-jit.

Source

cxx-jit.lisp.


6.2.3 Classes

Class: meta-data-tclass
Package

cxx-jit.

Source

cxx-jit.lisp.

Direct superclasses
  • foreign-struct-type.
  • translatable-foreign-type.

Appendix A Indexes


A.1 Concepts


A.3 Variables

Jump to:   *   +  
C   S  
Index Entry  Section

*
*cxx--fun-names*: Private special variables
*cxx-compiler-executable-path*: Private special variables
*cxx-compiler-flags*: Public special variables
*cxx-compiler-internal-flags*: Public special variables
*cxx-compiler-link-libs*: Public special variables
*cxx-compiler-packages*: Public special variables
*cxx-compiler-packages-number*: Public special variables
*cxx-compiler-process*: Public special variables
*cxx-compiler-working-directory*: Public special variables
*cxx-type-name-to-cffi-type-symbol-alist*: Public special variables

+
+cxx-compiler-lib-name+: Public constants
+cxx-compiler-wrap-cxx-path+: Public constants

C
Constant, +cxx-compiler-lib-name+: Public constants
Constant, +cxx-compiler-wrap-cxx-path+: Public constants

S
Special Variable, *cxx--fun-names*: Private special variables
Special Variable, *cxx-compiler-executable-path*: Private special variables
Special Variable, *cxx-compiler-flags*: Public special variables
Special Variable, *cxx-compiler-internal-flags*: Public special variables
Special Variable, *cxx-compiler-link-libs*: Public special variables
Special Variable, *cxx-compiler-packages*: Public special variables
Special Variable, *cxx-compiler-packages-number*: Public special variables
Special Variable, *cxx-compiler-process*: Public special variables
Special Variable, *cxx-compiler-working-directory*: Public special variables
Special Variable, *cxx-type-name-to-cffi-type-symbol-alist*: Public special variables