Next: Introduction, Previous: (dir), Up: (dir) [Contents][Index]
This is the marshal Reference Manual, version 1.3, generated automatically by Declt version 3.0 "Montgomery Scott" on Tue Dec 22 12:31:24 2020 GMT+0.
• Introduction | What marshal is all about | |
• Systems | The systems documentation | |
• Files | The files documentation | |
• Packages | The packages documentation | |
• Definitions | The symbols documentation | |
• Indexes | Concepts, functions, variables and data types |
Simple and fast marshalling of Lisp datastructures. Convert any object into a string representation, put it on a stream an revive it from there. Only minimal changes required to make your CLOS objects serializable. Actually you only need to add 1 method per baseclass.
Just to make that clear: project is not dead - the functionality is simply finished and working.
MIT License. See included LICENSE file.
For this ASDF needs to be installed and has to have access to marshal.asd. Otherwise do a load *.lisp/tests-.lisp.
(require :marshal)
Serialization of simple examples:
$ (ms:marshal (list 1 2 3 "Foo" "Bar" (make-array '(3) :initial-contents '(a b c))))
--> (:PCODE 1
(:LIST 1 1 2 3 (:SIMPLE-STRING 2 "Foo") (:SIMPLE-STRING 3 "Bar")
(:ARRAY 4 (3) T (A B C))))
Deserialization:
$ (ms:unmarshal '(:PCODE 1
(:LIST 1 1 2 3 (:SIMPLE-STRING 2 "Foo") (:SIMPLE-STRING 3 "Bar")
(:ARRAY 4 (3) T (A B C)))))
--> (1 2 3 "Foo" "Bar" #(A B C))
That means that a
(ms:unmarshal (ms:marshal myobject))
returns a deep clone of myobject.
Definition of a class
(defclass ship ()
((name :initform "" :initarg :name :accessor name)
(dimensions :initform '(:width 0 :length 0) :initarg :dimensions :accessor dimensions)
(course :initform 0 :initarg :course :accessor course)
(cruise :initform 0 :initarg :cruise :accessor cruise) ; shall be transient
(dinghy :initform NIL :initarg :dinghy :accessor dinghy)) ; another ship -> ref
(:documentation "A democlass. Some 'persistent slots', one transient.
Some numbers, string, lists and object references."))
(defparameter ark (make-instance 'ship :name "Ark" :course 360
:dimensions '(:width 30 :length 90)))
Let's try to serialize this:
$ (ms:marshal ark)
--> (:PCODE 1 NIL)
Actually nothing happens.
Next we define the method class-persistent-slots
for this class. The method has to be defined
in the package :marshal
.
(defmethod ms:class-persistent-slots ((self ship))
'(name dimensions course dinghy))
--> #<STANDARD-METHOD MARSHAL:CLASS-PERSISTENT-SLOTS (SHIP) {1002B16B31}
Note that the slot cruise
is not listed. Therefore it will not be serialized.
$ (ms:marshal ark)
--> (:PCODE 1
(:OBJECT 1 SHIP (:SIMPLE-STRING 2 "Ark") (:LIST 3 :WIDTH 30 :LENGTH 90) 360
(:LIST 4)))
Fine. Try a (ms:unmarshal (ms:marshal ark))
and you will get a clone of the object ark.
Let's define some subclasses (yes, it's Lisp, we use multiple inheritance here).
(defclass sailingship (ship)
((sailarea :initform 0 :initarg :sailarea :accessor sailarea))
)
(defclass motorship (ship)
((enginepower :initform 0 :initarg :enginepower :accessor enginepower))
)
(defclass motorsailor (motorship sailingship)
()
)
Some instances:
(defparameter ship2 (make-instance 'sailingship :name "Pinta" :course 270 :cruise 9
:dimensions '(:width 7 :length 21) :sailarea 400))
(defparameter ship3 (make-instance 'motorship :name "Titanic" :course 320 :cruise 21
:dimensions '(:width 28 :length 269) :enginepower 51000))
(defparameter ship4 (make-instance 'motorsailor :name "Krusenstern" :course 180
:cruise 17.4 :dimensions '(:width 12 :length 82)
:sailarea 3400 :enginepower 2000))
Let's try
$ (ms:marshal ship4)
--> (:PCODE 1
(:OBJECT 1 MOTORSAILOR (:SIMPLE-STRING 2 "Krusenstern")
(:LIST 3 :WIDTH 12 :LENGTH 82) 180 (:LIST 4)))
Note that the slots to be marshalled are determined by the function ms:class-peristant-slots
of
the baseclass ship
.
One last class and an instance. please note the backreference to another ship. That's a circular reference.
(defclass dinghy (ship)
((aboard :initform NIL :initarg :aboard :accessor aboard)) ; another ship -> circular ref
)
(defparameter ship5 (make-instance 'dinghy :name "Gig" :course 320 :cruise 5
:dimensions '(:width 2 :length 6) :aboard ship4))
(setf (dinghy ship4) ship5)
$ (ms:marshal ship4)
--> (:PCODE 1
(:OBJECT 1 MOTORSAILOR (:SIMPLE-STRING 2 "Krusenstern")
(:LIST 3 :WIDTH 12 :LENGTH 82) 180
(:OBJECT 4 DINGHY (:SIMPLE-STRING 5 "Gig") (:LIST 6 :WIDTH 2 :LENGTH 6)
320 (:LIST 7))))
We see the reference to the dhingy (the "sub ship"), but not the
backreference. Simply, because so far the back link aboard
is still transient.
(defmethod ms:class-persistent-slots ((self dinghy))
(append (call-next-method) '(aboard)))
$ (marshall ship4)
--> (:PCODE 1
(:OBJECT 1 MOTORSAILOR (:SIMPLE-STRING 2 "Krusenstern")
(:LIST 3 :WIDTH 12 :LENGTH 82) 180
(:OBJECT 4 DINGHY (:SIMPLE-STRING 5 "Gig") (:LIST 6 :WIDTH 2 :LENGTH 6)
320 (:LIST 7) (:REFERENCE 1))))
Brilliant! References, circles et. are working regardless these are references from and to lists, objects, hashtables, array etc.
Everything is in the package :marshal, nickname :ms.
To enable the serialization of a class, you need to specialise the
method ms:class-persistent-slots
for this class, or one of its
baseclasses. This method must reside in the package :marshal! It has
to return a list of slotnames. These slots will be serialized.
A call to ms:marshal
on an object will generate the string (actually
a sexp) representation. It will try to find the method ms:class-persistent-slots
to see which slots have to be serialized.
A call to ms:unmarshal
with sexp generated by a ms:marshal
will
revive an object.
You may have different implementations of classes to be serialized
and deserialized. For examples different classes on the endpoints of
a net work connections. Or simply different classes as time passes
between the persistent storage of a serialization and its retrieval.
It is important to understand that the classes that are serialized and
the one of the object that will be deserialized need to have the same
name and need to have the same slotnames as listed in ms:class-persistent-slots
.
If you define a method ms:initialize-unmarshalled-instance
for your
class, then this method will be called in the end of the
deserialization process. This gives you the chance to initialize
transient slots, that were not serialized, or to do other
initialization tricks.
There is a function called 'coding-idiom', that defines the language of the
marshalling. The default vocabulary is quite verbose. In case you are
going to send the objects through a network, you may want to change
that to a shorter set of verbs. Well, I think there are better ways
to speed that up, e.g. by adding a nginx proxy with automaic gzip
compression in front of your lisp webserver. Anyway, you will find an
alternative, shorter implementation in coding-idiom.lisp
, it is fairly
straight-forward.
The most simple and recommended way to install cl-marshal is by using Quicklisp. If you installed Quicklisp a simple
(ql:quickload :marshal)
will download the package and actually load it. You only need to do this once per machine. Later a
(require :marshal)
will be enough.
Alternatively you may get the code with:
git clone git://github.com/wlbr/cl-marshal.git
Either you add this to your asdf repository, then you will only need
to do a (require :marshal)
in your source.
Or, you may put the source in a subdirectory of your project and add
the file marshal.asd
wih its full path to your own asdf definition.
Or, you may put the source in a subdirectory of your project and load
the file "marshal.asd" directly. After that a (asdf:load-system "marshal")
should be sufficient.
Or, as a kind of worst case, you simply do a direct (load <file>)
of the files package.lisp
, coding-idiom.lisp
, marshal.lisp
,
and unmarshal.lisp
.
None except for asdf.
xlunit for the unit tests only (the tests are not included in the asdf system).
Tested with SBCL and CCL. No rocket science required, should run in any environment.
A set of unit tests is included in tests.lisp.
If you run into any trouble or find bugs, please report them via the Github issue tracker.
First written as encode/decode for CLOS objects only during a diploma thesis in '95. Major rework/enhancements during a research project in the end of the '90s. Refactoring in 2011 when revisiting Lisp.
Written by Michael Wolber. Major fixes and enhancements by Christoph Oechslein.
Next: Files, Previous: Introduction, Up: Top [Contents][Index]
The main system appears first, followed by any subsystem dependency.
• The marshal system |
Michael Wolber <mwolber@gmx.de>
MIT
marshal: Simple (de)serialization of Lisp datastructures.
1.3
marshal.asd (file)
Files are sorted by type and then listed depth-first from the systems components trees.
• Lisp files |
Next: The marshal/package․lisp file, Previous: Lisp files, Up: Lisp files [Contents][Index]
marshal.asd
marshal (system)
Next: The marshal/utils․lisp file, Previous: The marshal․asd file, Up: Lisp files [Contents][Index]
marshal (system)
package.lisp
Next: The marshal/serialization-format․lisp file, Previous: The marshal/package․lisp file, Up: Lisp files [Contents][Index]
package.lisp (file)
marshal (system)
utils.lisp
Next: The marshal/coding-idiom․lisp file, Previous: The marshal/utils․lisp file, Up: Lisp files [Contents][Index]
utils.lisp (file)
marshal (system)
serialization-format.lisp
Next: The marshal/marshal․lisp file, Previous: The marshal/serialization-format․lisp file, Up: Lisp files [Contents][Index]
serialization-format.lisp (file)
marshal (system)
coding-idiom.lisp
*idiom-table* (special variable)
Next: The marshal/unmarshal․lisp file, Previous: The marshal/coding-idiom․lisp file, Up: Lisp files [Contents][Index]
coding-idiom.lisp (file)
marshal (system)
marshal.lisp
Previous: The marshal/marshal․lisp file, Up: Lisp files [Contents][Index]
marshal.lisp (file)
marshal (system)
unmarshal.lisp
Next: Definitions, Previous: Files, Up: Top [Contents][Index]
Packages are listed by definition order.
• The marshal package | ||
• The utils package | ||
• The serialization-format package |
Next: The utils package, Previous: Packages, Up: Packages [Contents][Index]
package.lisp (file)
ms
common-lisp
Next: The serialization-format package, Previous: The marshal package, Up: Packages [Contents][Index]
package.lisp (file)
common-lisp
Previous: The utils package, Up: Packages [Contents][Index]
package.lisp (file)
fmt
common-lisp
Definitions are sorted by export status, category, package, and then by lexicographic order.
• Exported definitions | ||
• Internal definitions |
Next: Internal definitions, Previous: Definitions, Up: Definitions [Contents][Index]
• Exported special variables | ||
• Exported functions | ||
• Exported generic functions |
Next: Exported functions, Previous: Exported definitions, Up: Exported definitions [Contents][Index]
Definition of the vocabulary of the generated serialization. You can
increase verbosity or compactness by choosing your own ’language’.
key= access code used inside the programs source code
value= generated identifier.
coding-idiom.lisp (file)
Next: Exported generic functions, Previous: Exported special variables, Up: Exported definitions [Contents][Index]
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
utils.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
utils.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
utils.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
serialization-format.lisp (file)
Previous: Exported functions, Up: Exported definitions [Contents][Index]
Defines the slots that will be serialized. Has to return list of valid slotnames.
If this is a nested list, then the elements of the second level
need to be pairs of slot and accessors.
marshal.lisp (file)
Defines the slots that will be serialized. Has to return list of valid slotnames.
If this is a nested list, then the elements of the second level
need to be pairs of slot and accessors.
marshal.lisp (file)
Called as the last step of the deserialization of an object. !Must return the object!!
unmarshal.lisp (file)
Called as the last step of the deserialization of an object. !Must return the object!!
Generates an sexp when called with an object. The sexp can be used to send it over a network or to store it in a database etc.
marshal.lisp (file)
Returns an object when called with a wellformed marshal sexp.
unmarshal.lisp (file)
Previous: Exported definitions, Up: Definitions [Contents][Index]
• Internal constants | ||
• Internal functions | ||
• Internal generic functions | ||
• Internal classes |
Next: Internal functions, Previous: Internal definitions, Up: Internal definitions [Contents][Index]
coding-idiom.lisp (file)
Next: Internal generic functions, Previous: Internal constants, Up: Internal definitions [Contents][Index]
marshal.lisp (file)
Definition of the vocabulary of the generated serialization.You can increase verbosity or compactness by choosing your own ’language’. Simply define your own vocabulary and redefine the variable ms:*idiom-table*.
coding-idiom.lisp (file)
unmarshal.lisp (file)
marshal.lisp (file)
unmarshal.lisp (file)
unmarshal.lisp (file)
Next: Internal classes, Previous: Internal functions, Up: Internal definitions [Contents][Index]
marshal.lisp (file)
marshal.lisp (file)
automatically generated reader method
marshal.lisp (file)
automatically generated writer method
marshal.lisp (file)
marshal.lisp (file)
automatically generated reader method
marshal.lisp (file)
automatically generated writer method
marshal.lisp (file)
marshal.lisp (file)
unmarshal.lisp (file)
Previous: Internal generic functions, Up: Internal definitions [Contents][Index]
marshal.lisp (file)
standard-object (class)
hashtable (generic function)
(setf hashtable) (generic function)
0
next-key (generic function)
(setf next-key) (generic function)
Previous: Definitions, Up: Top [Contents][Index]
• Concept index | ||
• Function index | ||
• Variable index | ||
• Data type index |
Next: Function index, Previous: Indexes, Up: Indexes [Contents][Index]
Jump to: | F L M |
---|
Jump to: | F L M |
---|
Next: Variable index, Previous: Concept index, Up: Indexes [Contents][Index]
Jump to: | %
(
A C D F G H I L M N O P R S T U |
---|
Jump to: | %
(
A C D F G H I L M N O P R S T U |
---|
Next: Data type index, Previous: Function index, Up: Indexes [Contents][Index]
Jump to: | *
+
C H N S |
---|
Jump to: | *
+
C H N S |
---|
Previous: Variable index, Up: Indexes [Contents][Index]
Jump to: | C M P S U |
---|
Jump to: | C M P S U |
---|