The serializable-object Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The serializable-object Reference Manual

This is the serializable-object Reference Manual, version 0.1, generated automatically by Declt version 3.0 "Montgomery Scott" on Fri Jun 26 12:12:47 2020 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction


* Serializable-Object [[https://github.com/guicho271828/serializable-object][https://travis-ci.org/guicho271828/serializable-object.svg?branch=master]]

This library provides an abstract class for serializing CLOS objects in a FASL file.

+ It exposes =serializable-object= class, =save= generic function and =load-instance= function.
  An instance of =serializable-object= has a slot =pathname=.
+ The implementation focuses on simplicity and conformance rather than speed.
+ It carefully combines the behavior of =make-load-form= and the file compiler specified in CLHS.
  + This guarantees the correct behavior for storing and loading the references (e.g. cyclic reference, displaced arrays)
+ Should be thread-safe.
+ =:compression t= compresses the fasl file, similar to the numpy =.npz= format.

*TODO:* The current implementation stores an object into a fasl file, whose format is
implementation-dependent (and also version dependent).
Future work could instead use an existing CLOS object serializer.

We do not use them currently for various reasons:
First, the existing serializers are incomplete and unreliable; They are not as thoroughly tested
as the default serialization performed in =compile-file= and =load=,
which is required by the standard and is provided by each Lisp implementation.
# Relying on this feature minimizes the risk of encountering the corner cases,
#  e.g. how the non-CLOS objects are encoded in a binary format through references.
The strongest selling point of this library is this reliability,
with the cost of being restricted to the FASL-compatible lisp versions.
# One way to address this restriction would be to write a FASL converter script between the 
# different =lisp-implementation-version= and =lisp-implementation-type= using
# lisp environment switcher such as [[https://github.com/roswell/roswell][Roswell]] and write a implementation-independent communication system.


*TODO:* This library was originally written as a MOP abstract class.
However, the initial implementation with MOP was not working as intended 
and I decided to switch to the easier solution due to the timeframe.
In the future, we will pursue the API where =make-instance= can take
=:path= and other keywords which are used for loading an instance if necessary.


** Usage

#+begin_src lisp
(ql:quickload :serializable-object)
(in-package :serializable-object)


(defclass my (serializable-object)
  ((value :initarg :value :initform nil)))

(defparameter *my-instance* (make-instance 'my :value 5))

(save *my-instance* :verbose t :pathname "/tmp/tmp")
;; => Saving object # to /tmp/tmp 
;; => Writing a magic code to /tmp/tmpAAURSO1.tmp 
;; => ; compiling file "/tmp/tmpAAURSO1.tmp" (written 12 MAR 2019 12:54:04 PM):
;; => ; compiling (INITIALIZATION-FORM)
;; => 
;; => ; wrote /tmp/tmp.fasl
;; -> ; compilation finished in 0:00:00.004
;; -> #P"/tmp/tmp.fasl"
;; -> ()

(defparameter *another-instance* (load-instance 'my :verbose t :pathname "/tmp/tmp"))
;; => ; loading #P"/tmp/tmp.fasl"
;; -> *ANOTHER-INSTANCE*

(print (slot-value *another-instance* 'value))
;; => 5

#+end_src

** API

: Class SERIALIZABLE-OBJECT

Abstract class.

:  Generic function (save instance &key pathname store verbose parents)
:  Methods:
:    (save (serializable-object))
:    (save :around (serializable-object))

Save an instance to a FASL file using the value of PATHNAME slot in the instance.
When PATHNAME is given as an argument,

+ the object is stored in the file specified by PATHNAME,
+ the slot value is _temporarily_ set to this value while saving the instance,
+ and the value of PATHNAME is used to save the PATHNAME slot in the saved object.

If STORE is non-nil when PATHNAME is given, PATHNAME also overwrites the slot value in the runtime object.
Otherwise the PATHNAME slot value is restored to the original value after returning from this function.

If PARENTS is non-nil (default: t), ENSURE-DIRECTORIES-EXIST is called to
ensure that the path exists.

When an error occurs during the call to SAVE (e.g. nonexisting directory or permission error),
the path is reverted to the original value.

How it works:

1. SAVE generic-function stores a single line of macro (initialization-form) to
   a temporary file and compiles a file under the dynamic environment where a
   special variable *instance* is bound to the object to be stored.

2. The file compiler expands the macro to the code that sets *instance* to the
   value of *instance*.  MAKE-LOAD-FORM expands the value into a loadable form.

3. To load the instance, LOAD-INSTANCE function sets up a dynamic binding for
   *instance* and load the compiled file in this dynamic environment. The
   compiled code sets the newly created object (by evaluating the form produced
   by make-load-form) to *instance*. LOAD-INSTANCE retrieves this value.


: Function (load-instance class &key pathname (if-does-not-exist t) verbose &allow-other-keys)

Load an instance from a file.

+ When IF-DOES-NOT-EXIST is non-nil (default), it checks the file existence.
+ When IF-DOES-NOT-EXIST is nil and the file does not exist, it calls MAKE-INSTANCE with the specified arguments.
+ When verbose is non-nil, it writes messages to the standard output.
+ It always checks if the loaded object is of the specified class.

** Related Work

*** hu.dwim.serializer

http://quickdocs.org/hu.dwim.serializer/api

+ documentation : poor
+ tutorial : none
+ performance : unknown
+ scope / usability : all CLOS objects
  + cyclic references?
  + displaced arrays?

*** trivial-hashtable-serialize

http://quickdocs.org/trivial-hashtable-serialize/

+ documentation : poor
+ tutorial : good
+ performance : unknown
+ scope / usability : hash table only

*** cl-store

http://quickdocs.org/cl-store/

+ documentation : minimal
+ tutorial : none
+ performance : unknown. to a 32bit int stream
+ scope / usability : All CLOS class.
  + arrays?
  + cyclic references?
  + Exported slots can be customized, all slots by default.
  + Consideres the class slots.

*** cl-marshall

http://quickdocs.org/cl-marshall/

+ documentation : minimal (source code) good (tutorial)
+ tutorial : good
+ performance : unknown. to a string that consists of a list
+ scope / usability :
  + needs to specify class-persistent-slots.
  + cyclic refernces?
  + displaced arrays?
  + Exported slots can be customized, needs to be specified for each class.
  + no consideration for class slots.

*** persistent-variables

http://quickdocs.org/persistent-variables/

+ documentation : 
+ tutorial : 
+ performance : 
+ scope / usability :
  + very specific. needs to be declared as defpvar

*** userial

http://quickdocs.org/userial/

+ documentation : 
+ tutorial : 
+ performance : 
+ scope / usability :
  + offers the versioning system
  + but heavily depends on ContextL.

*** specialization-store

http://quickdocs.org/specialization-store/

+ documentation : 
+ tutorial : there is an extensive tutorial, but the idea seems too complicated.
+ performance : 
+ scope / usability : 

** Dependencies
This library is at least tested on implementation listed below:

+ SBCL 1.4.12 on X86-64 Linux 4.4.0-142-generic (author's environment)

Also, it depends on the following libraries:

+ alexandria by *Nikodemus Siivola , and others.* :
    Alexandria is a collection of portable public domain utilities.
+ closer-mop
+ bordeaux-threads

** Author, License, Copyright

Licensed under LGPL v3.

Copyright (c) 2019 Masataro Asai (guicho2.71828@gmail.com)
Copyright (c) 2019 IBM Corporation


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 serializable-object

Author

Masataro Asai

Contact

guicho2.71828@gmail.com

License

LGPL

Description

Provides a simple class and API for the objects serializable in a FASL file

Version

0.1

Dependency

alexandria

Source

serializable-object.asd (file)

Component

package.lisp (file)


Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 serializable-object.asd

Location

/home/quickref/quicklisp/dists/quicklisp/software/serializable-object-20191227-git/serializable-object.asd

Systems

serializable-object (system)


Previous: , Up: Lisp files   [Contents][Index]

3.1.2 serializable-object/package.lisp

Parent

serializable-object (system)

Location

package.lisp

Packages

serializable-object

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 serializable-object

Source

package.lisp (file)

Use List
Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Functions

Function: load-instance PATHNAME &rest ARGS &key CLASS IF-DOES-NOT-EXIST VERBOSE &allow-other-keys

Load an instance from PATHNAME which was used when the object was saved.
The loaded instance should be of type CLASS.

+ When VERBOSE is non-nil, it writes messages to the standard output.

+ When IF-DOES-NOT-EXIST is :error (default), it signals an error when the file is missing.

+ When IF-DOES-NOT-EXIST is nil and the file does not exist, it calls
MAKE-INSTANCE with CLASS, with :pathname parameter and other parameters
specified in ARGS except :if-does-not-exist, :class, :verbose.

Package

serializable-object

Source

package.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.2 Generic functions

Generic Function: save INSTANCE &key PATHNAME STORE VERBOSE PARENTS COMPRESSION OVERWRITE &allow-other-keys

Save an instance to a FASL file using the value of PATHNAME slot in the instance.
When PATHNAME is given as an argument,

+ the object is stored in the file specified by PATHNAME,
+ the slot value is _temporarily_ set to this value while saving the instance,
+ and the value of PATHNAME is used to save the PATHNAME slot in the saved object.

If STORE is non-nil when PATHNAME is given, PATHNAME also overwrites the slot value in the runtime object. Otherwise the PATHNAME slot value is restored to the original value after returning from this function.

If PARENTS is non-nil (default: t), ENSURE-DIRECTORIES-EXIST is called to
ensure that the path exists.

When an error occurs during the call to SAVE (e.g. nonexisting directory or permission error),
the path is reverted to the original value.

How it works:

1. SAVE generic-function stores a single line of macro (initialization-form) to
a temporary file and compiles a file under the dynamic environment where a
special variable *instance* is bound to the object to be stored.

2. The file compiler expands the macro to the code that sets *instance* to the
value of *instance*. MAKE-LOAD-FORM expands the value into a loadable form.

3. To load the instance, LOAD-INSTANCE function sets up a dynamic binding for
*instance* and load the compiled file in this dynamic environment. The
compiled code sets the newly created object (by evaluating the form produced
by make-load-form) to *instance*. LOAD-INSTANCE retrieves this value.

Package

serializable-object

Source

package.lisp (file)

Methods
Method: save (INSTANCE serializable-object) &key VERBOSE PARENTS COMPRESSION OVERWRITE &allow-other-keys
Method: save (INSTANCE serializable-object) &key PATHNAME STORE &allow-other-keys around

Previous: , Up: Exported definitions   [Contents][Index]

5.1.3 Classes

Class: serializable-object ()
Package

serializable-object

Source

package.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
  • save (method)
  • save (method)
  • make-load-form (method)
Direct slots
Slot: pathname
Initargs

:pathname


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Special variables

Special Variable: *instance*
Package

serializable-object

Source

package.lisp (file)


Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Macros

Macro: initialization-form ()
Package

serializable-object

Source

package.lisp (file)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   S  
Index Entry  Section

F
File, Lisp, serializable-object.asd: The serializable-object․asd file
File, Lisp, serializable-object/package.lisp: The serializable-object/package․lisp file

L
Lisp File, serializable-object.asd: The serializable-object․asd file
Lisp File, serializable-object/package.lisp: The serializable-object/package․lisp file

S
serializable-object.asd: The serializable-object․asd file
serializable-object/package.lisp: The serializable-object/package․lisp file

Jump to:   F   L   S  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   F   G   I   L   M   S  
Index Entry  Section

F
Function, load-instance: Exported functions

G
Generic Function, save: Exported generic functions

I
initialization-form: Internal macros

L
load-instance: Exported functions

M
Macro, initialization-form: Internal macros
Method, save: Exported generic functions
Method, save: Exported generic functions

S
save: Exported generic functions
save: Exported generic functions
save: Exported generic functions

Jump to:   F   G   I   L   M   S  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *  
P   S  
Index Entry  Section

*
*instance*: Internal special variables

P
pathname: Exported classes

S
Slot, pathname: Exported classes
Special Variable, *instance*: Internal special variables

Jump to:   *  
P   S  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   C   P   S  
Index Entry  Section

C
Class, serializable-object: Exported classes

P
Package, serializable-object: The serializable-object package

S
serializable-object: The serializable-object system
serializable-object: The serializable-object package
serializable-object: Exported classes
System, serializable-object: The serializable-object system

Jump to:   C   P   S