This is the cl-binary-store Reference Manual, version 1.0.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Sun Jul 13 18:53:12 2025 GMT+0.
cl-binary-store/cl-binary-store.asdcl-binary-store/features.lispcl-binary-store/cl-binary-store.lispcl-binary-store/cl-binary-store-user.lispcl-binary-store/codespaces.lispcl-binary-store/sbcl-special-hash-tables.lispcl-binary-store/type-discrimination.lispcl-binary-store/object-info.lispcl-binary-store/basic-codespace-codes.lispcl-binary-store/reference-coding.lispcl-binary-store/actions.lispcl-binary-store/basic-codespace.lispcl-binary-store/sap-ref.lispcl-binary-store/storage.lispcl-binary-store/unsigned-bytes.lispcl-binary-store/referrers-and-fixup.lispcl-binary-store/numbers.lispcl-binary-store/reference-count.lispcl-binary-store/magic-numbers.lispcl-binary-store/end-action.lispcl-binary-store/cons.lispcl-binary-store/sbcl-utilities.lispcl-binary-store/simple-array.lispcl-binary-store/simple-vector.lispcl-binary-store/symbols.lispcl-binary-store/array.lispcl-binary-store/pathname.lispcl-binary-store/hash-table.lispcl-binary-store/objects.lispcl-binary-store/user.lispThe main system appears first, followed by any subsystem dependency.
cl-binary-storeFast serialization / deserialization library
Fast serialization / deserialization library
Andrew J. Berkley <ajberkley@gmail.com>
BSD-3
1.0.0
flexi-streams (system).
babel (system).
static-vectors (system).
alexandria (system).
features.lisp (file).
cl-binary-store.lisp (file).
cl-binary-store-user.lisp (file).
codespaces.lisp (file).
sbcl-special-hash-tables.lisp (file).
type-discrimination.lisp (file).
object-info.lisp (file).
basic-codespace-codes.lisp (file).
reference-coding.lisp (file).
actions.lisp (file).
basic-codespace.lisp (file).
sap-ref.lisp (file).
storage.lisp (file).
unsigned-bytes.lisp (file).
referrers-and-fixup.lisp (file).
numbers.lisp (file).
reference-count.lisp (file).
magic-numbers.lisp (file).
end-action.lisp (file).
cons.lisp (file).
sbcl-utilities.lisp (file).
simple-array.lisp (file).
simple-vector.lisp (file).
symbols.lisp (file).
array.lisp (file).
pathname.lisp (file).
hash-table.lisp (file).
objects.lisp (file).
user.lisp (file).
Files are sorted by type and then listed depth-first from the systems components trees.
cl-binary-store/cl-binary-store.asdcl-binary-store/features.lispcl-binary-store/cl-binary-store.lispcl-binary-store/cl-binary-store-user.lispcl-binary-store/codespaces.lispcl-binary-store/sbcl-special-hash-tables.lispcl-binary-store/type-discrimination.lispcl-binary-store/object-info.lispcl-binary-store/basic-codespace-codes.lispcl-binary-store/reference-coding.lispcl-binary-store/actions.lispcl-binary-store/basic-codespace.lispcl-binary-store/sap-ref.lispcl-binary-store/storage.lispcl-binary-store/unsigned-bytes.lispcl-binary-store/referrers-and-fixup.lispcl-binary-store/numbers.lispcl-binary-store/reference-count.lispcl-binary-store/magic-numbers.lispcl-binary-store/end-action.lispcl-binary-store/cons.lispcl-binary-store/sbcl-utilities.lispcl-binary-store/simple-array.lispcl-binary-store/simple-vector.lispcl-binary-store/symbols.lispcl-binary-store/array.lispcl-binary-store/pathname.lispcl-binary-store/hash-table.lispcl-binary-store/objects.lispcl-binary-store/user.lispcl-binary-store/cl-binary-store.asdcl-binary-store (system).
cl-binary-store/cl-binary-store.lispcl-binary-store (system).
invalid-input-data (condition).
maybe-expected-error (condition).
unexpected-data (function).
cl-binary-store/cl-binary-store-user.lispcl-binary-store.lisp (file).
cl-binary-store (system).
cl-binary-store/codespaces.lispfeatures.lisp (file).
unsigned-bytes.lisp (file).
cl-binary-store (system).
*codespaces* (special variable).
*current-codespace* (special variable).
*output-end-marker* (special variable).
*track-references* (special variable).
*version-being-read* (special variable).
define-codespace (macro).
defrestore (macro).
defstore (macro).
delete-codespace (function).
delete-restore (macro).
delete-store (macro).
*current-codespace/compile-time* (special variable).
analyze-references-hash-table (function).
build-global-state-declarations (function).
build-global-state-let-bindings (function).
build-map-reference-tables (function).
build-reference-tables (function).
build-restore-objects (function).
build-source-code (function).
build-store-objects (function).
codespace (structure).
codespace-magic-number (reader).
(setf codespace-magic-number) (writer).
codespace-name (reader).
(setf codespace-name) (writer).
codespace-p (function).
codespace-ref-tables (reader).
(setf codespace-ref-tables) (writer).
codespace-restore-global-state-info (reader).
(setf codespace-restore-global-state-info) (writer).
codespace-restore-infos (reader).
(setf codespace-restore-infos) (writer).
codespace-restore-objects (reader).
(setf codespace-restore-objects) (writer).
codespace-restore-objects-source-code (reader).
(setf codespace-restore-objects-source-code) (writer).
codespace-store-global-state-info (reader).
(setf codespace-store-global-state-info) (writer).
codespace-store-infos (reader).
(setf codespace-store-infos) (writer).
codespace-store-objects (reader).
(setf codespace-store-objects) (writer).
codespace-store-objects-source-code (reader).
(setf codespace-store-objects-source-code) (writer).
compile-codespace (function).
compiled-p (function).
copy-codespace (function).
copy-global-state (function).
copy-ref-table (function).
copy-restore-info (function).
copy-store-info (function).
deep-copy-codespace (function).
global-state (structure).
global-state-construction-code (reader).
(setf global-state-construction-code) (writer).
global-state-dynamic-extent (reader).
(setf global-state-dynamic-extent) (writer).
global-state-name (reader).
(setf global-state-name) (writer).
global-state-p (function).
global-state-type (reader).
(setf global-state-type) (writer).
invalid (function).
make-codespace (function).
make-global-state (function).
make-read-dispatch-table (function).
make-ref-table (function).
make-restore-info (function).
make-store-info (function).
ref-table (structure).
ref-table-construction-code (reader).
(setf ref-table-construction-code) (writer).
ref-table-name (reader).
(setf ref-table-name) (writer).
ref-table-p (function).
register-global-state (macro).
register-references (macro).
register-references& (function).
register-restore-state& (function).
register-store-state& (function).
replacing-reference-tables (function).
restore-info (structure).
restore-info-p (function).
restore-info-restore-function-dispatch-code (reader).
(setf restore-info-restore-function-dispatch-code) (writer).
restore-info-restore-function-source-code (reader).
(setf restore-info-restore-function-source-code) (writer).
restore-objects (function).
store-info (structure).
store-info-p (function).
store-info-reference-phase-code (reader).
(setf store-info-reference-phase-code) (writer).
store-info-storage-phase-code (reader).
(setf store-info-storage-phase-code) (writer).
store-info-type (reader).
(setf store-info-type) (writer).
store-object/phase (function).
store-objects (function).
update-restore-info (function).
update-store-info (function).
with-reference-tables (macro).
cl-binary-store/sbcl-special-hash-tables.lisp:sbcl
cl-binary-store (system).
double-float-= (function).
double-float-hash (function).
string-and-type-= (function).
string-hash (function).
cl-binary-store/type-discrimination.lispcl-binary-store (system).
*good-top-levels* (special variable).
*many-types* (special variable).
*preferred-dispatch-order* (special variable).
another (structure).
another-p (function).
binned-disjoint-types (function).
blarg-td (structure).
blarg-td-p (function).
build-discriminator-tree (function).
class-a (class).
class-b (class).
class-c (class).
copy-another (function).
copy-blarg-td (function).
copy-includes-blarg-td (function).
includes-blarg-td (structure).
includes-blarg-td-p (function).
make-another (function).
make-blarg-td (function).
make-includes-blarg-td (function).
satisfies-something (type).
satisfies-test (function).
strict-subtype-ordering (function).
cl-binary-store/object-info.lispcl-binary-store (system).
object-info (structure).
object-info-slot-names (reader).
(setf object-info-slot-names) (writer).
object-info-type (reader).
(setf object-info-type) (writer).
copy-object-info (function).
make-object-info (function).
object-info-class (reader).
(setf object-info-class) (writer).
object-info-p (function).
object-info-ref-id (reader).
(setf object-info-ref-id) (writer).
object-info-slot-value-filter-func (reader).
(setf object-info-slot-value-filter-func) (writer).
object-info-specialized-constructor (reader).
(setf object-info-specialized-constructor) (writer).
object-info-specialized-deserializer (reader).
(setf object-info-specialized-deserializer) (writer).
object-info-specialized-serializer (reader).
(setf object-info-specialized-serializer) (writer).
object-info-use-initialize-instance (reader).
(setf object-info-use-initialize-instance) (writer).
cl-binary-store/basic-codespace-codes.lispcl-binary-store (system).
+action-code+ (constant).
+array-code+ (constant).
+bignum-code+ (constant).
+complex-code+ (constant).
+complex-double-float-code+ (constant).
+complex-single-float-code+ (constant).
+cons-code+ (constant).
+double-float-code+ (constant).
+double-float-zero-code+ (constant).
+finite-length-list-code+ (constant).
+first-direct-reference-id-code+ (constant).
+first-direct-unsigned-integer-interior-code+ (constant).
+first-one-byte-reference-id-code+ (constant).
+first-small-integer-code+ (constant).
+first-two-byte-reference-id-code+ (constant).
+first-user-code+ (constant).
+fixnum-code+ (constant).
+hash-table-code+ (constant).
+interior-coded-max-integer+ (constant).
+last-direct-reference-id-code+ (constant).
+last-one-byte-reference-id-code+ (constant).
+last-small-integer-code+ (constant).
+last-two-byte-reference-id-code+ (constant).
+last-user-code+ (constant).
+maximum-untagged-unsigned-integer+ (constant).
+minimum-untagged-signed-integer+ (constant).
+new-reference-indicator-code+ (constant).
+nil-code+ (constant).
+object-info-code+ (constant).
+pathname-code+ (constant).
+ratio-code+ (constant).
+sb16-code+ (constant).
+sb32-code+ (constant).
+sb8-code+ (constant).
+simple-base-string-code+ (constant).
+simple-specialized-array-code+ (constant).
+simple-specialized-vector-code+ (constant).
+simple-string-code+ (constant).
+simple-vector-code+ (constant).
+single-float-code+ (constant).
+small-integer-zero-code+ (constant).
+standard/structure-object-code+ (constant).
+symbol-code+ (constant).
+t-code+ (constant).
+tagged-reference-code+ (constant).
+ub16-code+ (constant).
+ub32-code+ (constant).
+ub8-code+ (constant).
+unbound-code+ (constant).
+uninterned-symbol-code+ (constant).
cl-binary-store/reference-coding.lispbasic-codespace-codes.lisp (file).
cl-binary-store (system).
+reference-direct-max-ref-id+ (constant).
+reference-direct-min-ref-id+ (constant).
+reference-one-byte-max-ref-id+ (constant).
+reference-one-byte-min-ref-id+ (constant).
+reference-two-byte-max-ref-id+ (constant).
+reference-two-byte-min-ref-id+ (constant).
decode-reference-direct (function).
decode-reference-one-byte (function).
decode-reference-tagged (function).
decode-reference-two-bytes (function).
encode-reference-direct (function).
encode-reference-one-byte (function).
encode-reference-tagged (function).
encode-reference-two-bytes (function).
cl-binary-store/actions.lispstorage.lisp (file).
features.lisp (file).
basic-codespace-codes.lisp (file).
unsigned-bytes.lisp (file).
cl-binary-store (system).
action (generic function).
action (structure).
action-code (reader).
restore-action& (function).
store-action& (function).
+end-action-code+ (constant).
+magic-number-action-code+ (constant).
+set-reference-action-code+ (constant).
action-p (function).
copy-action (function).
make-action (function).
store-action (generic function).
cl-binary-store/basic-codespace.lispsbcl-special-hash-tables.lisp (file).
type-discrimination.lisp (file).
hash-table.lisp (file).
features.lisp (file).
codespaces.lisp (file).
actions.lisp (file).
object-info.lisp (file).
pathname.lisp (file).
basic-codespace-codes.lisp (file).
cons.lisp (file).
reference-coding.lisp (file).
unsigned-bytes.lisp (file).
referrers-and-fixup.lisp (file).
numbers.lisp (file).
objects.lisp (file).
cl-binary-store (system).
*double-float-refs-table-size* (special variable).
*eq-refs-table-size* (special variable).
*num-eq-refs-table-size* (special variable).
+basic-codespace+ (constant).
cl-binary-store/sap-ref.lispcl-binary-store (system).
sap-ref-16 (macro).
sap-ref-32 (macro).
sap-ref-64 (macro).
sap-ref-8 (macro).
sap-ref-double (macro).
sap-ref-single (macro).
set-sap-ref-16 (macro).
set-sap-ref-32 (macro).
set-sap-ref-64 (macro).
set-sap-ref-8 (macro).
set-sap-ref-double (macro).
set-sap-ref-single (macro).
set-signed-sap-ref-64 (macro).
signed-sap-ref-64 (macro).
vector-sap (macro).
with-pinned-objects (macro).
array-sap (macro).
mask-signed (macro).
negative-to-twos-complement/16 (macro).
negative-to-twos-complement/32 (macro).
negative-to-twos-complement/64 (macro).
negative-to-twos-complement/8 (macro).
set-signed-sap-ref-16 (macro).
set-signed-sap-ref-32 (macro).
set-signed-sap-ref-8 (macro).
signed-sap-ref-16 (macro).
signed-sap-ref-32 (macro).
signed-sap-ref-8 (macro).
cl-binary-store/storage.lispfeatures.lisp (file).
cl-binary-store.lisp (file).
sap-ref.lisp (file).
cl-binary-store (system).
*max-to-read* (special variable).
*max-to-write* (special variable).
ensure-enough-data (function).
out-of-data (condition).
out-of-space (condition).
out-of-space-current-offset (reader method).
out-of-space-wanted-bytes (reader method).
print-object (method).
print-object (method).
read-storage-offset (reader).
(setf read-storage-offset) (writer).
read-storage-sap (reader).
(setf read-storage-sap) (writer).
read-storage-store (reader).
(setf read-storage-store) (writer).
too-much-data (condition).
with-write-storage (macro).
write-storage-offset (reader).
(setf write-storage-offset) (writer).
write-storage-sap (reader).
(setf write-storage-sap) (writer).
write-storage-store (reader).
(setf write-storage-store) (writer).
*load/save-progress-indicator* (special variable).
ask-for-new-amount (function).
check-if-too-much-data (macro).
copy-read-storage (function).
copy-write-storage (function).
ensure-enough-room-to-write (function).
flush-write-storage (function).
make-read-into-storage/stream (function).
make-read-storage (function).
make-write-into-adjustable-ub8-vector (function).
make-write-into-storage/stream (function).
make-write-storage (function).
maybe-shift-data-to-beginning-of-read-storage (function).
read-storage (structure).
read-storage-flusher (reader).
(setf read-storage-flusher) (writer).
read-storage-max (reader).
(setf read-storage-max) (writer).
read-storage-max-to-read (reader).
(setf read-storage-max-to-read) (writer).
read-storage-p (function).
read-storage-size (reader).
(setf read-storage-size) (writer).
read-storage-total-read (reader).
(setf read-storage-total-read) (writer).
read-storage-underlying-stream (reader).
(setf read-storage-underlying-stream) (writer).
refill-read-storage (function).
shift-data-to-beginning (function).
storage-write-byte (function).
too-much-data-bytes (reader method).
too-much-data-max-bytes (reader method).
truly-the (macro).
with-storage/read (macro).
with-storage/write (macro).
with-tracking-data (macro).
write-storage (structure).
write-storage-flusher (reader).
(setf write-storage-flusher) (writer).
write-storage-max (reader).
(setf write-storage-max) (writer).
write-storage-p (function).
write-storage-underlying-stream (reader).
(setf write-storage-underlying-stream) (writer).
cl-binary-store/unsigned-bytes.lispstorage.lisp (file).
features.lisp (file).
cl-binary-store.lisp (file).
basic-codespace-codes.lisp (file).
sap-ref.lisp (file).
cl-binary-store (system).
store-ub16 (function).
store-ub32 (function).
store-ub8/no-tag (function).
store-ub8/tag (function).
maybe-restore-ub8 (function).
restore-ub16 (function).
restore-ub32 (function).
restore-ub8 (function).
cl-binary-store/referrers-and-fixup.lispunsigned-bytes.lisp (file).
features.lisp (file).
basic-codespace-codes.lisp (file).
cl-binary-store (system).
check/store-reference (function).
references (structure).
restore-object-to (macro).
check-reference (function).
copy-fixup (function).
copy-references (function).
ensure-references-vector (function).
fixup (function).
fixup (structure).
fixup-list (reader).
(setf fixup-list) (writer).
fixup-p (function).
fixup-ref-id (reader).
(setf fixup-ref-id) (writer).
grow-references-vector (function).
make-fixup (function).
make-references (function).
maybe-store-reference-instead (macro).
referenced-already (function).
references-p (function).
references-ref-id (reader).
(setf references-ref-id) (writer).
references-vector (reader).
(setf references-vector) (writer).
restore-new-reference-indicator (function).
restore-reference (function).
store-new-reference-indicator (function).
store-reference (function).
update-reference (function).
with-delayed-reference/fixup (macro).
cl-binary-store/numbers.lispunsigned-bytes.lisp (file).
referrers-and-fixup.lisp (file).
features.lisp (file).
sap-ref.lisp (file).
basic-codespace-codes.lisp (file).
cl-binary-store (system).
restore-bignum (function).
restore-complex (function).
restore-complex-double-float (function).
restore-complex-single-float (function).
restore-double-float (function).
restore-double-float-to (macro).
restore-ratio (function).
restore-single-float (function).
store-bignum (function).
store-complex (function).
store-complex-double-float (function).
store-complex-single-float (function).
store-double-float (function).
store-fixnum (function).
store-ratio (function).
store-single-float (function).
store-tagged-unsigned-fixnum (function).
store-tagged-unsigned-integer (function).
bits->num (function).
ensure-integer (function).
ensure-real (function).
num->bits (function).
restore-double-float-zero (function).
restore-fixnum (function).
restore-sb16 (function).
restore-sb32 (function).
restore-sb8 (function).
restore-tagged-fixnum (function).
restore-tagged-unsigned-fixnum (function).
restore-tagged-unsigned-fixnum/interior (function).
store-only-fixnum (function).
store-sb16 (function).
store-sb32 (function).
store-sb8 (function).
store-tagged-unsigned-fixnum/interior (function).
cl-binary-store/reference-count.lispactions.lisp (file).
numbers.lisp (file).
features.lisp (file).
cl-binary-store (system).
action (method).
copy-write-reference-count (function).
make-write-reference-count (function).
store-action (method).
write-reference-count (function).
write-reference-count (structure).
write-reference-count-code (reader).
write-reference-count-p (function).
write-reference-count-reference-count (reader).
cl-binary-store/magic-numbers.lispactions.lisp (file).
numbers.lisp (file).
cl-binary-store (system).
*allow-codespace-switching* (special variable).
*write-version* (special variable).
action (method).
copy-magic-number (function).
magic-number (structure).
magic-number-code (reader).
magic-number-number (reader).
magic-number-p (function).
make-magic-number (function).
store-action (method).
cl-binary-store/end-action.lispactions.lisp (file).
numbers.lisp (file).
cl-binary-store (system).
action (method).
make-end-marker (function).
copy-end-marker (function).
end-marker (structure).
end-marker-code (reader).
end-marker-p (function).
store-action (method).
cl-binary-store/cons.lispreferrers-and-fixup.lisp (file).
numbers.lisp (file).
unsigned-bytes.lisp (file).
features.lisp (file).
symbols.lisp (file).
cl-binary-store (system).
*support-shared-list-structures* (special variable).
length/detect-dotted (function).
restore-cons/indefinite (function).
restore-list/known-length (function).
search-cons (function).
search-cons/finite (function).
search-cons/indefinite (function).
store-cons (function).
store-cons/finite (function).
store-cons/indefinite (function).
cl-binary-store/sbcl-utilities.lisp:sbcl
features.lisp (file).
cl-binary-store (system).
copy-sap (function).
memcpy (function).
cl-binary-store/simple-array.lispreferrers-and-fixup.lisp (file).
numbers.lisp (file).
features.lisp (file).
sap-ref.lisp (file).
storage.lisp (file).
unsigned-bytes.lisp (file).
cl-binary-store (system).
restore-simple-specialized-array (function).
restore-simple-specialized-vector (function).
restore-string (function).
store-simple-specialized-array (function).
store-simple-specialized-vector (function).
store-string (function).
store-string/no-refs (function).
+first-direct-unsigned-integer+ (constant).
+max-direct-encoded-unsigned-integer+ (constant).
chunked/read (function).
chunked/write (function).
encoded-element-type-to-type/packing (function).
make-simple-array-from-encoded-element-type (function).
make-writer/reader (macro).
reader (macro).
restore-simple-base-string (function).
restore-simple-string (function).
sap-ref-fixnum (macro).
set-sap-ref-fixnum (macro).
specialized-array-element-size/bits (function).
store-elts (function).
store-simple-base-string (function).
store-simple-string (function).
with-read-storage (macro).
write-sap-data-to-storage (function).
writer (macro).
cl-binary-store/simple-vector.lispunsigned-bytes.lisp (file).
referrers-and-fixup.lisp (file).
features.lisp (file).
numbers.lisp (file).
cl-binary-store (system).
restore-simple-vector (function).
store-simple-vector (function).
cl-binary-store/symbols.lispunsigned-bytes.lisp (file).
referrers-and-fixup.lisp (file).
features.lisp (file).
numbers.lisp (file).
cl-binary-store (system).
missing-package-during-restore (condition).
print-object (method).
restore-symbol (function).
store-boolean (function).
store-nil (function).
store-symbol (function).
store-t (function).
ask-for-new-package-name (function).
ensure-string (function).
missing-package-package-string (reader method).
missing-package-symbol-string (reader method).
restore-boolean (function).
restore-nil (function).
restore-t (function).
restore-uninterned-symbol (function).
signal-missing-package (function).
cl-binary-store/array.lispunsigned-bytes.lisp (file).
cons.lisp (file).
symbols.lisp (file).
numbers.lisp (file).
referrers-and-fixup.lisp (file).
features.lisp (file).
cl-binary-store (system).
restore-array (function).
store-array (function).
is-type-specifier-p (function).
cl-binary-store/pathname.lispreferrers-and-fixup.lisp (file).
symbols.lisp (file).
numbers.lisp (file).
unsigned-bytes.lisp (file).
features.lisp (file).
cl-binary-store (system).
restore-pathname (function).
store-pathname (function).
cl-binary-store/hash-table.lispreferrers-and-fixup.lisp (file).
symbols.lisp (file).
numbers.lisp (file).
unsigned-bytes.lisp (file).
features.lisp (file).
symbols.lisp (file).
cl-binary-store (system).
restore-hash-table (function).
store-hash-table (function).
cl-binary-store/objects.lispsymbols.lisp (file).
simple-vector.lisp (file).
referrers-and-fixup.lisp (file).
numbers.lisp (file).
unsigned-bytes.lisp (file).
features.lisp (file).
codespaces.lisp (file).
cl-binary-store (system).
missing-slot (condition).
missing-slot-data-slots (reader method).
missing-slot-image-slots (reader method).
missing-slot-type (reader method).
object-type-not-found (condition).
object-type-not-found-object-info (reader method).
print-object (method).
print-object (method).
restore-standard/structure-object (function).
serializable-object-info (generic function).
specialized-object-constructor (generic function).
specialized-serializer/deserializer (generic function).
store-standard/structure-object (function).
*store-class-slots* (special variable).
ask-for-new-class (function).
ask-for-new-slot-name (function).
compute-object-info (function).
function-designator-p (function).
get-object-info (function).
get-slot-names (function).
maybe-store-local-reference-instead (macro).
maybe-store-to-reference-table (function).
missing-slot-name (reader method).
really-find-class (function).
restore-object-info (function).
restore-unbound (function).
signal-object-type-not-found (function).
(setf slot-value*) (function).
store-object-info (function).
store-unbound (function).
use-custom-building-function (function).
validate-slot-names (function).
cl-binary-store/user.lispbasic-codespace.lisp (file).
cons.lisp (file).
storage.lisp (file).
features.lisp (file).
magic-numbers.lisp (file).
reference-count.lisp (file).
end-action.lisp (file).
cl-binary-store (system).
*output-magic-number* (special variable).
*read-version* (special variable).
replace-store-sap-buffer (function).
restore (function).
restore-from-file (function).
restore-from-sap (function).
restore-from-stream (function).
restore-from-vector (function).
store (function).
store-to-extant-vector (function).
store-to-file (function).
store-to-sap (function).
store-to-stream (function).
store-to-vector (function).
Packages are listed by definition order.
cl-binary-storeA package that exports tools used inside cl-binary-store for use by
someone writing their own specialized serialization or deserialization routine.
common-lisp.
sb-mop.
*allow-codespace-switching* (special variable).
*codespaces* (special variable).
*current-codespace* (special variable).
*double-float-refs-table-size* (special variable).
*eq-refs-table-size* (special variable).
*max-to-read* (special variable).
*max-to-write* (special variable).
*num-eq-refs-table-size* (special variable).
*output-end-marker* (special variable).
*output-magic-number* (special variable).
*read-version* (special variable).
*support-shared-list-structures* (special variable).
*track-references* (special variable).
*version-being-read* (special variable).
*write-version* (special variable).
+action-code+ (constant).
+basic-codespace+ (constant).
action (generic function).
action (structure).
action-code (reader).
check/store-reference (function).
copy-sap (function).
define-codespace (macro).
defrestore (macro).
defstore (macro).
delete-codespace (function).
delete-restore (macro).
delete-store (macro).
ensure-enough-data (function).
invalid-input-data (condition).
make-end-marker (function).
maybe-expected-error (condition).
missing-package-during-restore (condition).
missing-slot (condition).
missing-slot-data-slots (generic reader).
missing-slot-image-slots (generic reader).
missing-slot-type (generic reader).
object-info (structure).
object-info-slot-names (reader).
(setf object-info-slot-names) (writer).
object-info-type (reader).
(setf object-info-type) (writer).
object-type-not-found (condition).
object-type-not-found-object-info (generic reader).
out-of-data (condition).
out-of-space (condition).
out-of-space-current-offset (generic reader).
out-of-space-wanted-bytes (generic reader).
read-storage-offset (reader).
(setf read-storage-offset) (writer).
read-storage-sap (reader).
(setf read-storage-sap) (writer).
read-storage-store (reader).
(setf read-storage-store) (writer).
references (structure).
replace-store-sap-buffer (function).
restore (function).
restore-action& (function).
restore-array (function).
restore-bignum (function).
restore-complex (function).
restore-complex-double-float (function).
restore-complex-single-float (function).
restore-double-float (function).
restore-double-float-to (macro).
restore-from-file (function).
restore-from-sap (function).
restore-from-stream (function).
restore-from-vector (function).
restore-object-to (macro).
restore-ratio (function).
restore-simple-specialized-array (function).
restore-simple-specialized-vector (function).
restore-simple-vector (function).
restore-single-float (function).
restore-standard/structure-object (function).
restore-string (function).
restore-symbol (function).
sap-ref-16 (macro).
sap-ref-32 (macro).
sap-ref-64 (macro).
sap-ref-8 (macro).
sap-ref-double (macro).
sap-ref-single (macro).
serializable-object-info (generic function).
set-sap-ref-16 (macro).
set-sap-ref-32 (macro).
set-sap-ref-64 (macro).
set-sap-ref-8 (macro).
set-sap-ref-double (macro).
set-sap-ref-single (macro).
set-signed-sap-ref-64 (macro).
signed-sap-ref-64 (macro).
specialized-object-constructor (generic function).
specialized-serializer/deserializer (generic function).
store (function).
store-action& (function).
store-array (function).
store-bignum (function).
store-boolean (function).
store-complex (function).
store-complex-double-float (function).
store-complex-single-float (function).
store-double-float (function).
store-fixnum (function).
store-nil (function).
store-ratio (function).
store-simple-specialized-array (function).
store-simple-specialized-vector (function).
store-simple-vector (function).
store-single-float (function).
store-standard/structure-object (function).
store-string (function).
store-string/no-refs (function).
store-symbol (function).
store-t (function).
store-tagged-unsigned-fixnum (function).
store-tagged-unsigned-integer (function).
store-to-extant-vector (function).
store-to-file (function).
store-to-sap (function).
store-to-stream (function).
store-to-vector (function).
store-ub16 (function).
store-ub32 (function).
store-ub8/no-tag (function).
store-ub8/tag (function).
too-much-data (condition).
vector-sap (macro).
with-pinned-objects (macro).
with-write-storage (macro).
write-storage-offset (reader).
(setf write-storage-offset) (writer).
write-storage-sap (reader).
(setf write-storage-sap) (writer).
write-storage-store (reader).
(setf write-storage-store) (writer).
*current-codespace/compile-time* (special variable).
*good-top-levels* (special variable).
*load/save-progress-indicator* (special variable).
*many-types* (special variable).
*preferred-dispatch-order* (special variable).
*store-class-slots* (special variable).
+array-code+ (constant).
+bignum-code+ (constant).
+complex-code+ (constant).
+complex-double-float-code+ (constant).
+complex-single-float-code+ (constant).
+cons-code+ (constant).
+double-float-code+ (constant).
+double-float-zero-code+ (constant).
+end-action-code+ (constant).
+finite-length-list-code+ (constant).
+first-direct-reference-id-code+ (constant).
+first-direct-unsigned-integer+ (constant).
+first-direct-unsigned-integer-interior-code+ (constant).
+first-one-byte-reference-id-code+ (constant).
+first-small-integer-code+ (constant).
+first-two-byte-reference-id-code+ (constant).
+first-user-code+ (constant).
+fixnum-code+ (constant).
+hash-table-code+ (constant).
+interior-coded-max-integer+ (constant).
+last-direct-reference-id-code+ (constant).
+last-one-byte-reference-id-code+ (constant).
+last-small-integer-code+ (constant).
+last-two-byte-reference-id-code+ (constant).
+last-user-code+ (constant).
+magic-number-action-code+ (constant).
+max-direct-encoded-unsigned-integer+ (constant).
+maximum-untagged-unsigned-integer+ (constant).
+minimum-untagged-signed-integer+ (constant).
+new-reference-indicator-code+ (constant).
+nil-code+ (constant).
+object-info-code+ (constant).
+pathname-code+ (constant).
+ratio-code+ (constant).
+reference-direct-max-ref-id+ (constant).
+reference-direct-min-ref-id+ (constant).
+reference-one-byte-max-ref-id+ (constant).
+reference-one-byte-min-ref-id+ (constant).
+reference-two-byte-max-ref-id+ (constant).
+reference-two-byte-min-ref-id+ (constant).
+sb16-code+ (constant).
+sb32-code+ (constant).
+sb8-code+ (constant).
+set-reference-action-code+ (constant).
+simple-base-string-code+ (constant).
+simple-specialized-array-code+ (constant).
+simple-specialized-vector-code+ (constant).
+simple-string-code+ (constant).
+simple-vector-code+ (constant).
+single-float-code+ (constant).
+small-integer-zero-code+ (constant).
+standard/structure-object-code+ (constant).
+symbol-code+ (constant).
+t-code+ (constant).
+tagged-reference-code+ (constant).
+ub16-code+ (constant).
+ub32-code+ (constant).
+ub8-code+ (constant).
+unbound-code+ (constant).
+uninterned-symbol-code+ (constant).
action-p (function).
analyze-references-hash-table (function).
another (structure).
another-p (function).
array-sap (macro).
ask-for-new-amount (function).
ask-for-new-class (function).
ask-for-new-package-name (function).
ask-for-new-slot-name (function).
binned-disjoint-types (function).
bits->num (function).
blarg-td (structure).
blarg-td-p (function).
build-discriminator-tree (function).
build-global-state-declarations (function).
build-global-state-let-bindings (function).
build-map-reference-tables (function).
build-reference-tables (function).
build-restore-objects (function).
build-source-code (function).
build-store-objects (function).
check-if-too-much-data (macro).
check-reference (function).
chunked/read (function).
chunked/write (function).
class-a (class).
class-b (class).
class-c (class).
codespace (structure).
codespace-magic-number (reader).
(setf codespace-magic-number) (writer).
codespace-name (reader).
(setf codespace-name) (writer).
codespace-p (function).
codespace-ref-tables (reader).
(setf codespace-ref-tables) (writer).
codespace-restore-global-state-info (reader).
(setf codespace-restore-global-state-info) (writer).
codespace-restore-infos (reader).
(setf codespace-restore-infos) (writer).
codespace-restore-objects (reader).
(setf codespace-restore-objects) (writer).
codespace-restore-objects-source-code (reader).
(setf codespace-restore-objects-source-code) (writer).
codespace-store-global-state-info (reader).
(setf codespace-store-global-state-info) (writer).
codespace-store-infos (reader).
(setf codespace-store-infos) (writer).
codespace-store-objects (reader).
(setf codespace-store-objects) (writer).
codespace-store-objects-source-code (reader).
(setf codespace-store-objects-source-code) (writer).
compile-codespace (function).
compiled-p (function).
compute-object-info (function).
copy-action (function).
copy-another (function).
copy-blarg-td (function).
copy-codespace (function).
copy-end-marker (function).
copy-fixup (function).
copy-global-state (function).
copy-includes-blarg-td (function).
copy-magic-number (function).
copy-object-info (function).
copy-read-storage (function).
copy-ref-table (function).
copy-references (function).
copy-restore-info (function).
copy-store-info (function).
copy-write-reference-count (function).
copy-write-storage (function).
decode-reference-direct (function).
decode-reference-one-byte (function).
decode-reference-tagged (function).
decode-reference-two-bytes (function).
deep-copy-codespace (function).
double-float-= (function).
double-float-hash (function).
encode-reference-direct (function).
encode-reference-one-byte (function).
encode-reference-tagged (function).
encode-reference-two-bytes (function).
encoded-element-type-to-type/packing (function).
end-marker (structure).
end-marker-code (reader).
end-marker-p (function).
ensure-enough-room-to-write (function).
ensure-integer (function).
ensure-real (function).
ensure-references-vector (function).
ensure-string (function).
fixup (function).
fixup (structure).
fixup-list (reader).
(setf fixup-list) (writer).
fixup-p (function).
fixup-ref-id (reader).
(setf fixup-ref-id) (writer).
flush-write-storage (function).
function-designator-p (function).
get-object-info (function).
get-slot-names (function).
global-state (structure).
global-state-construction-code (reader).
(setf global-state-construction-code) (writer).
global-state-dynamic-extent (reader).
(setf global-state-dynamic-extent) (writer).
global-state-name (reader).
(setf global-state-name) (writer).
global-state-p (function).
global-state-type (reader).
(setf global-state-type) (writer).
grow-references-vector (function).
includes-blarg-td (structure).
includes-blarg-td-p (function).
invalid (function).
is-type-specifier-p (function).
length/detect-dotted (function).
magic-number (structure).
magic-number-code (reader).
magic-number-number (reader).
magic-number-p (function).
make-action (function).
make-another (function).
make-blarg-td (function).
make-codespace (function).
make-fixup (function).
make-global-state (function).
make-includes-blarg-td (function).
make-magic-number (function).
make-object-info (function).
make-read-dispatch-table (function).
make-read-into-storage/stream (function).
make-read-storage (function).
make-ref-table (function).
make-references (function).
make-restore-info (function).
make-simple-array-from-encoded-element-type (function).
make-store-info (function).
make-write-into-adjustable-ub8-vector (function).
make-write-into-storage/stream (function).
make-write-reference-count (function).
make-write-storage (function).
make-writer/reader (macro).
mask-signed (macro).
maybe-restore-ub8 (function).
maybe-shift-data-to-beginning-of-read-storage (function).
maybe-store-local-reference-instead (macro).
maybe-store-reference-instead (macro).
maybe-store-to-reference-table (function).
memcpy (function).
missing-package-package-string (generic reader).
missing-package-symbol-string (generic reader).
missing-slot-name (generic reader).
negative-to-twos-complement/16 (macro).
negative-to-twos-complement/32 (macro).
negative-to-twos-complement/64 (macro).
negative-to-twos-complement/8 (macro).
num->bits (function).
object-info-class (reader).
(setf object-info-class) (writer).
object-info-p (function).
object-info-ref-id (reader).
(setf object-info-ref-id) (writer).
object-info-slot-value-filter-func (reader).
(setf object-info-slot-value-filter-func) (writer).
object-info-specialized-constructor (reader).
(setf object-info-specialized-constructor) (writer).
object-info-specialized-deserializer (reader).
(setf object-info-specialized-deserializer) (writer).
object-info-specialized-serializer (reader).
(setf object-info-specialized-serializer) (writer).
object-info-use-initialize-instance (reader).
(setf object-info-use-initialize-instance) (writer).
read-storage (structure).
read-storage-flusher (reader).
(setf read-storage-flusher) (writer).
read-storage-max (reader).
(setf read-storage-max) (writer).
read-storage-max-to-read (reader).
(setf read-storage-max-to-read) (writer).
read-storage-p (function).
read-storage-size (reader).
(setf read-storage-size) (writer).
read-storage-total-read (reader).
(setf read-storage-total-read) (writer).
read-storage-underlying-stream (reader).
(setf read-storage-underlying-stream) (writer).
reader (macro).
really-find-class (function).
ref-table (structure).
ref-table-construction-code (reader).
(setf ref-table-construction-code) (writer).
ref-table-name (reader).
(setf ref-table-name) (writer).
ref-table-p (function).
referenced-already (function).
references-p (function).
references-ref-id (reader).
(setf references-ref-id) (writer).
references-vector (reader).
(setf references-vector) (writer).
refill-read-storage (function).
register-global-state (macro).
register-references (macro).
register-references& (function).
register-restore-state& (function).
register-store-state& (function).
replacing-reference-tables (function).
restore-boolean (function).
restore-cons/indefinite (function).
restore-double-float-zero (function).
restore-fixnum (function).
restore-hash-table (function).
restore-info (structure).
restore-info-p (function).
restore-info-restore-function-dispatch-code (reader).
(setf restore-info-restore-function-dispatch-code) (writer).
restore-info-restore-function-source-code (reader).
(setf restore-info-restore-function-source-code) (writer).
restore-list/known-length (function).
restore-new-reference-indicator (function).
restore-nil (function).
restore-object-info (function).
restore-objects (function).
restore-pathname (function).
restore-reference (function).
restore-sb16 (function).
restore-sb32 (function).
restore-sb8 (function).
restore-simple-base-string (function).
restore-simple-string (function).
restore-t (function).
restore-tagged-fixnum (function).
restore-tagged-unsigned-fixnum (function).
restore-tagged-unsigned-fixnum/interior (function).
restore-ub16 (function).
restore-ub32 (function).
restore-ub8 (function).
restore-unbound (function).
restore-uninterned-symbol (function).
sap-ref-fixnum (macro).
satisfies-something (type).
satisfies-test (function).
search-cons (function).
search-cons/finite (function).
search-cons/indefinite (function).
set-sap-ref-fixnum (macro).
set-signed-sap-ref-16 (macro).
set-signed-sap-ref-32 (macro).
set-signed-sap-ref-8 (macro).
shift-data-to-beginning (function).
signal-missing-package (function).
signal-object-type-not-found (function).
signed-sap-ref-16 (macro).
signed-sap-ref-32 (macro).
signed-sap-ref-8 (macro).
(setf slot-value*) (function).
specialized-array-element-size/bits (function).
storage-write-byte (function).
store-action (generic function).
store-cons (function).
store-cons/finite (function).
store-cons/indefinite (function).
store-elts (function).
store-hash-table (function).
store-info (structure).
store-info-p (function).
store-info-reference-phase-code (reader).
(setf store-info-reference-phase-code) (writer).
store-info-storage-phase-code (reader).
(setf store-info-storage-phase-code) (writer).
store-info-type (reader).
(setf store-info-type) (writer).
store-new-reference-indicator (function).
store-object-info (function).
store-object/phase (function).
store-objects (function).
store-only-fixnum (function).
store-pathname (function).
store-reference (function).
store-sb16 (function).
store-sb32 (function).
store-sb8 (function).
store-simple-base-string (function).
store-simple-string (function).
store-tagged-unsigned-fixnum/interior (function).
store-unbound (function).
strict-subtype-ordering (function).
string-and-type-= (function).
string-hash (function).
too-much-data-bytes (generic reader).
too-much-data-max-bytes (generic reader).
truly-the (macro).
unexpected-data (function).
update-reference (function).
update-restore-info (function).
update-store-info (function).
use-custom-building-function (function).
validate-slot-names (function).
with-delayed-reference/fixup (macro).
with-read-storage (macro).
with-reference-tables (macro).
with-storage/read (macro).
with-storage/write (macro).
with-tracking-data (macro).
write-reference-count (function).
write-reference-count (structure).
write-reference-count-code (reader).
write-reference-count-p (function).
write-reference-count-reference-count (reader).
write-sap-data-to-storage (function).
write-storage (structure).
write-storage-flusher (reader).
(setf write-storage-flusher) (writer).
write-storage-max (reader).
(setf write-storage-max) (writer).
write-storage-p (function).
write-storage-underlying-stream (reader).
(setf write-storage-underlying-stream) (writer).
writer (macro).
cl-binary-store-userA Common Lisp serialization/deserialization library
for Common Lisp objects to a binary format. Optimized for speed and
lightly for flexibility. It is about 10x faster than cl-store and
produces reasonably compact files (plug it into a gzip stream if you
want more). Currently only works on SBCL.
Out of the box we support reading/writing from/to streams,
reading/writing from/to in-memory vectors, reading/writing to raw
memory.
Extending this with your own serializer / deserializer is
straightforward as it is expected that the default standard-object
and structure-object serialization may not meet everyones needs.
All number types are supported, but we provide specialized compact writers for:
ub8, ub16, ub32, ub64, fixnum, single-float, double-float
(complex double-float) (complex single-float)
All array types are supported.
On SBCL we provide fast and compact serialization of vectors and simple-arrays of:
bit (simple-bit-vector in 1D or multi-dimensional)
base-char (simple-base-string in 1D or multi-dimensional)
character (simple-string in 1D or multi-dimensional)
single-float
double-float
fixnum
signed-byte: 8 16 32 64
unsigned-byte: 2 4 7 8 15 16 31 32 62 64
cl-binary-store.
common-lisp.
Definitions are sorted by export status, category, package, and then by lexicographic order.
A request to perform an action. Used for checking codespace versions and for updating reference vector size and for marking the end of data
This is the basic codespace of cl-binary-store.
Set this to NIL if you want to specify the format of file you want to load and not allow it to be set automatically based on the data format of the file.
a map from magic/version-number -> ‘codespace’
a ‘codespace’ bound by store based on
*write-version*. This is also bound during restore operations once we
know the format of the data stream or is set to *read-version*.
A hint for the starting size of the double float tracking reference table
A hint for the starting size of the object tracking hash table used for most objects
The maximum amount of data to restore (2GB) before complaining.
The default maximum of data to write to disk (10GB) before complaining.
A hint for the starting size of the hash table tracking misc number types (complex, ratios, bignums)
If T, once all objects are stored an end marker will be written to the output.
This will trigger the end of restore (for use in cases where there
isn’t an obvious end of file)
If T we will write out a magic number and *write-version* to the output, which will be validated against our existing *codespaces* when we read it back.
The default codespace version to use if no versioning information is in the stream
If this is T, then circular lists of all types and structures that share list parts will be serialized correctly. This is very expensive.
If you let this to NIL, then every object will be stored anew, and there will be no circular reference detection. It’s a huge performance win (you can hit hundreds of MB/sec instead of 10s of MB/sec, but you need to make sure your data is safe to serialize and you don’t care about EQL checks of data.
During restore this is bound to any magic number found previous to this in the file.
Set this to the magic number you wish to write into the file. It may be queried by serialization routines if desired.
Creates and registers a codespace into *codespaces*. Within this environment
there are a three pre-defined symbols:
TRACK-REFERENCES is bound to *track-references*
OBJ within a defstore is the object you should store
CODE within a defrestore is the tag code that has been read.
STORAGE is a ‘read-storage’ if you are in a defrestore, if you are in a
defstore, it is either a ‘write-storage’ or NIL if you are in the reference
counting phase (you can provide a separate function for that phase too).
RESTORE-OBJECT is a (lambda ()) -> object that can be passed to a function during the restore phase
STORE-OBJECT is a (lambda (object)) -> writes stuff out
In define-codespace that has inherited another codespace, delete store capability for a type
Restore a double-float to an object that is not constructed yet.
If you are deserializing an object which contains slots (for
example an array, a list, hash-table, or structure-object or a
standard-object) which may point to other lisp objects which have yet
to be fully reified, then please update your slots with this macro
which will handle circularity fixups for you.
Note that we capture any parameters of place so you may safely use this in loops or with references to variables whose values may be updated later
On sbcl, return a SAP referring to the backing store of vector
Skips the body if storage does not exist, like during the reference scanning phase
Used during the storage phase both during the reference counting step and the serialization step. This function returns T if this object has already been written out, in which case the caller should NOT write OBJECT out to storage. If NIL, then you must write the OBJECT out. If ADD-NEW-REFERENCE is T, in the case where this function returns NIL, we will generate a new reference id for this object so it can be used in the future. The only case where ADD-NEW-REFERENCE should be NIL is if you are explicitly dis-allowing (for performance reasons) circularity, as we optionally do during cons serialization.
For RESTORE operation.
Ensure that we have at least BYTES of data in STORAGE. May signal ‘out-of-data’
unless return-nil-on-eof is t. Do not ask for more than (storage-size storage),
which is guaranteed to be >8192 bytes.
type.
sap.
PLACE may be a string or pathname designating a file to read from, or a stream to
read from (must have element type (unsigned-byte 8)), or a vector.
(restore #(14 39 37 0 2 72 73 15)) -> (values :hi))
(store filename (list (list :hi :bye) :something) :as-separate-objects t)
(restore filename/stream/place) -> (values (list :hi :bye) :something)
(restore-from-vector (store nil :hi :bye)) -> (values :hi :bye)
If ALLOW-CODESPACE-SWITCHING then the file can specify a version different from
READ-VERSION and we will switch to it if it is available.
MAX-TO-READ specifies the maximum amount of data in bytes we should load.
LOAD/SAVE-PROGRESS-INDICATOR, if T, shows you some indications while loading
Can only be used if you are sure that the strings are not references, so that means they must have been stored with store-string/no-refs
Do not call me directly because if you called store-symbol you may have ended up writing a reference to the symbol object instead of the symbol object.
When PLACE is NIL, returns a (vector (unsigned-byte 8) (*)) with fill-pointer
ex: (store nil (list 1.234d56)) #(32 5 11 90 215 48 169 108 33 148 75 5)
When PLACE is a filename/string/pathname writes data to the respective file.
When PLACE is a vector writes data to it and returns num used bytes... if vector is
adjustable, it may be adjusted. Otherwise we error if we run out of space.
If you provide a list of objects in DATA and you specify AS-SEPARATE-OBJECTS
they will come back from RESTORE as multiple values. Otherwise we just store the list.
ex: (restore (store filename (list :hi :bye) :data-is-list-of-separate-objects t)) -> (values :hi :bye).
If OUTPUT-MAGIC-NUMBER we write out WRITE-VERSION at the beginning of the stream and
it will then be validated on restore.
MAX-TO-WRITE is the maximum number of bytes you want to write out before erroring.
SUPPORT-SHARED-LIST-STRUCTURES should be T if you have circular lists or share tails of lists and
want them to come back properly EQL.
TRACK-REFERENCES should be T if you have references between the elements in data.
LOAD/SAVE-PROGRESS-INDICATOR, if T, shows you some indications while loading
Store and tag a fixnum; if inside another tag
Nominally we don’t need to do references here, but if someone has two bignums and takes a ratio of them, we don’t want to store the bignums twice.
Store and tag a number from 0 to max-positive-fixnum.
You can call ‘restore-tagged-unsigned-fixnum’ to restore it (or restore-object). Do
not call this except during the actual storage phase.
Store and tag any unsigned integer. For restoring this, call restore-object as this may be a bignum. Prefer ‘store-tagged-unsigned-fixnum’ if you are sure this isn’t a bignum
This may error with ‘out-of-space’ (which contains out-of-space-current-offset and out-of-space-wanted-bytes). The out-of-space-current-offset is the amount of data that has been written so far. out-of-space-wanted-bytes is not useful unless you happen to be writing very large stuff as it will likely be a small number representing the immediate need. Best to allocate a big chunk and when this finally returns we return the amount of data we wrote to the chunk. Call (replace-store-sap-buffer sap sap-offset) in a handler-bind to do this updating. See test test-sap-write/read for an example.
Returns an (array (unsigned-byte 8) (*)) with the data
Store an (unsigned-byte 16) value UB16 to STORAGE. If TAG is true will emit +UB16-CODE+ to STORAGE first. Set TAG NIL if the deserializer will know from the context that the value is a UB16 to save a byte.
Store an (unsigned-byte 32) value UB32 to STORAGE. If TAG is true will emit +UB32-CODE+ to STORAGE first. Set TAG NIL if the deserializer will know from the context that the value is a UB32 to save a byte.
Store an (unsigned-byte 8) value UB8 to STORAGE. If TAG is nil then we will skip writing a tag byte; use if your deserializer will know that the next byte is a UB8. Do not call except during storage phase
Store an (unsigned-byte 8) value UB8 to STORAGE. If TAG is nil then we will skip writing a tag byte; use if your deserializer will know that the next byte is a UB8. Do not call except during storage phase
sap.
If we hit an +action-code+ during restore,
we will call this which should specialize on command (a ub8).
You can read anything from the file here as arguments to
the action. Return two values, an object/nil and a second
value which is :ignore, :end, or nil if the object is to be
collected for the user. The second value only works if the
object is a top level object (that is one of the objects in
the call to store (store nil obj-a obj-b (make-instance ’end-action))
(eql 2)) storage references restore-object) ¶(eql 0)) storage references restore-object) ¶(eql 1)) storage references restore-object) ¶missing-slot)) ¶missing-slot)) ¶missing-slot)) ¶type.
object-type-not-found)) ¶out-of-space)) ¶out-of-space)) ¶Must return two values. The first value must be a
list of slot-names (symbols) which should be serialized for this
object.
The second value may be NIL or a function which will be called with each slot-name and slot-value and should return a serializable object (like nil) for that slot.
May return a function that will be used to construct an object from
an ‘object-info’ structure and a simple vector of slot-values in the same order as
(object-info-slot-names object-info):
(lambda (object-info slot-values) -> object)
Be careful in the case of circular references: it may be in that case that a slot-value
is a ‘fixup’, in which case you have to provide a function to be called back when the
object is fully reified. See restore-object-to for the logic.
Returns two values, the first value is a
function (or nil) that will be called as a:
(lambda (object storage eq-refs store-object assign-new-reference-id))
and as side effects should write to storage, etc. The second value should be a function
that has a signature (lambda (storage restore-object) -> object)
object-type-not-found) str) ¶missing-slot) stream) ¶missing-package-during-restore) stream) ¶write-storage) stream) ¶read-storage) stream) ¶simple-error.
Things like MISSING-PACKAGE-DURING-RESTORE, MISSING-SLOT
:slot-name
This slot is read-only.
common-lisp.
:type
This slot is read-only.
:data-slots
This slot is read-only.
:image-slots
This slot is read-only.
:object-info
This slot is read-only.
Ran out of data while expecting more while reading /deserializing
| Initarg | Value |
|---|---|
:format-control | out of data |
Ran out of space while writing data
Tried to read / write more than allowed amount of data. If you choose
any of the continue options, the current operation will continue (which might be a make-array
or make-list call which may exceed what you input)
structure-object.
(unsigned-byte 8)
This slot is read-only.
structure-object.
common-lisp.
(find-class (quote structure-object))
simple-vector
#()
common-lisp.
symbol
(quote cl-binary-store:object-info)
(or null function)
boolean
(or null function)
(or null function)
(or null function)
(or null (and fixnum (integer * -1)))
During deserialization this array contains all the references we
have seen so far, and a running count ref-id of assigned ids. Nominally
the array size is hinted at the start of restore, but the code allows it
to grow if needed.
When tracking references, we know all list lengths in advance
[35 63] used for direct reference codes (tag only) ref id [1 29]
[64 127] used for the tag + one byte reference codes (6 bits)
[192 224] is used for small signed integers [-16 16]
[128 191] used for the two byte codes (6 bits)
Note that the next object to be read should be assigned the next consecutive reference id
A reference to an object
nil or a ‘codespace’. This is bound while compiling each codespace.
If T will write out some progress while loading and saving
To change this, setf it and call (rebuild-dispatch).
In benchmarking you will see factors of two speed differences by tweaking this
(test-cl-binary-store-on-data (long-list-of-small-integers) :track-references nil)
If set / let to T, then slots in standard-objects with :class allocation will be stored, otherwise not.
Return a pointer referring to the backing store of an array (on sbcl)
Re-interpret a SIZE bit lisp number as if it were a signed twos complement number
Called during the serialization / storage phase. This is a kludged implicit referencing scheme used when the global *track-references* is disabled. This lets us avoid calculating and writing and reading what each object type is.
Objects may be seen multiple times during serialization,
so where object equality after deserialization is expected (pretty
much every object except numbers) or not determinable (double-floats,
complex, ratios, bignum), we record objects along with reference ids
that we can refer to later in the serialization to point to the
original object. The counting of objects is done explicitly in the
writing phase, so there is nothing to do in the reading phase except
to plunk objects into the right place in the *references* array.
This is hideous. On SBCL we have stored the data fixnum encoded, which means shifted up by one so we shift it back to generate an ’unencoded’ fixnum. We would normally just have blitted things back into the array without telling sbcl what we are doing, but in the presence of potentially malicious data, we have to make sure that our numbers are fixnums, so we shift them back down which makes it impossible for malicious actor to put bogus dat in the array sap. On non sbcl we just check to make sure things are fixnums because we did not blit the data out
Inside serialized specialized arrays, we store fixnums in 64-bit storage spots shifted up one, that is as if all implementations just used a 0 tag bit as the lowest bit.
When we know an object is going to be referred to multiple times,
we place it in the *references* array immediately before we even start
building it because it may not be buildable without restoring other objects
that might refer to it. So we always stick a fixup in the references array
first for any of those newly created objects to hang their requests to be
notified of the final object once it is constructed. BODY must eventually yield
the fully constructed object. Not hygenic,
Used to create a read-storage from user provided stream or sap or stream
Used to create a write-storage from user provided sap, store, or stream.
Provides a function (update bytes &optional eof) which should be called when new bytes are read or written. Used for *load/save-progress-indicator*
Returns an alist with keys being a type and values being sub-types of the values. CL-USER> (binned-disjoint-types ’(fixnum (unsigned-byte 8) standard-object)) -> ((STANDARD-OBJECT) (FIXNUM (UNSIGNED-BYTE 8)))
Wrap body with defined reference tables
Builds the body of a function that reads tag bytes and dispatches them through a big case statement built by make-read-dispatch-table.
Returns T if OBJECT has already been seen and updates its reference count. If OBJECT has not been seen, and ADD-NEW-REFERENCE is T, then adds it to references and returns NIL. If ADD-NEW-REFERENCE is NIL, just returns NIL. This should *ONLY* be called during the reference counting phase, that is when storage is nil.
Used when we cannot do this directly from the sap
name.
Takes a symbol denoting the type of an object and returns an ‘object-info’ allowing for the various user methods to override behaviors.
Result will be between [1 30]. This uses just the tag byte
Result will be between [31 16414]. This uses the tag byte plus another byte
Number ranges from -16 to wherever. This uses the reference-tag byte plus the tagged integer which can be anywhere from 1 byte direct tagged to arbitrarily large.
Result will be between [16415 4210718]. This uses the tag byte plus 2 additional bytes
reference indicies start a 1, so we subtract one here.
Returns a 16 bit value
How many bits per element when reading back in data. We may be reading data that was stored from an array type that does not exist in the reader implementation, so consider this a definition of our serialization scheme.
Ensure that we have room to write BYTES to STORAGE. Returns storage offset.
Return / resize references-vector which can hold ref-id
Resolve a delayed object construction. Returns new-value.
list.
Make sure everything is written out of storage to whatever backing store there is, and assert there is room to write a further number of BYTES returns the current (write-storage-offset write-storage) after flushing. Will signal an error if not enough room available
Return a list of slot names (symbols) skipping :class allocation slots if *store-class-slots* is t.
name.
type.
Returns true if TYPE-SPECIFIER is a valid type specifier.
length of a list, returns nil if the list is dotted
Returns (values new-array array-bytes)
Maybe restore an (unsigned-byte 8) value from storage that has previously been stored by STORE-UB8. If there is no more data available will return NIL.
If all we have is a sap the store is a length 0 vector so this fails gracefully
Called during the deserialization / restore phase. We store the constructed object-info into our reference hash table using its reference id.
max.
size.
name.
Returns T if OBJECT is in REFERENCES and writes out a reference to it to storage.
Otherwise returns NIL if it is not a reference at all. This should
*ONLY* be called during the actual storage phase, not the reference
counting phase.
Read data from storage until we run into an end of data signal, or an +end-action-code+. If you want to call this directly, you should let *current-codespace* to a codespace, as is done in the user facing functions in user.lisp which choose it based on *write-version* and *read-version*.
The reference has already been calculated in the dispatch code for us. If we are actually restoring the next object, it may not be re-ified before someone refers to it, so we have to store a fixup for those other objects to hang their reference onto.
Restore an (integer -65535 0) value from storage that has previously been stored by STORE-SB16.
Restore an (integer -4294967295 0) value from storage that has previously been stored by STORE-UB32.
Despite the name, restore an (integer -255 0) value from storage that has previously been stored by STORE-SB8
Read back a number written by ‘store-tagged-unsigned-fixnum’.
Read back a number written by ‘store-tagged-unsigned-fixnum’.
Use this if you know that this is an unsigned number (so after another tag bit). This opens up the direct coding space for up to +interior-coded-max-integer+.
Restore a (unsigned-byte 16) from STORAGE which has previously been stored by STORE-UB16.
Restore a (unsigned-byte 32) from STORAGE which has previously been stored by STORE-UB32.
Restore an (unsigned-byte 8) value from storage that has previously been stored by STORE-UB8.
You can call this directly since we never store references
This is called during the reference counting phase and only when we have only very simple list circularity (CDR (LAST LIST)) -> LIST or another reference list that we have seen before.
This is only called when *track-references* is t and *support-shared-list-structures* is t.
Move the data in seq to the beginning and update storage-offset and storage-max. Returns the index where new data can be written (read-storage-max storage)
Handle internal ’unbound-slot value
Returns (values bits/elt ub8-encoded-array-type). 64 bit specific. This is used during serialization. Inverse of encoded-element-type-to-type/packing
This is called during the actual storage output phase if we have already computed the list length. This is not called when *support-shared-list-structures* is true.
This is called during the actual storage output phase.
type.
Write an indicator that we should assign a reference id to the next object; that is place it in the restore reference-vector (and increment the ref-id counter).
Store all the objects in stuff to storage. Do not call this directly without let’ing *current-codespace* to a valid entry in *codespaces*. Prefer the functions in user.lisp which do this for you based on *write-version* and *read-version*.
Write a reference id to the output which will be resolved at restore time to an object. The basic-codespace implementation here reserves 6 bits of the codespace for reference ids which makes these pretty cheap.
Store an (integer -255 0) value SB8 to STORAGE. Set TAG to NIL if your deserializer will know this is a SB8 value and the value will be written without a tag byte. It’s a bit odd to have (integer -255 0) but the dispatch cost is negligible.... the tag byte is the sign bit.
Use this paired with restore-tagged-unsigned-fixnum if you are inside another tagged region for storing unsigned numbers. Somewhat more dense as we only need tags 0-3 for tagging unsigned numbers.
These are not displaced strings, etc. Just simple strings.
Used during RESTORE
If all of restored-slot-names are a subset of current-image-slot-names just return restored-slot-names. If not, provide restarts to allow building of a work around for the user. May return a specialized-object-constructor.
max.
missing-package-during-restore)) ¶missing-package-during-restore)) ¶missing-slot)) ¶Call during the serialization phase. You can
write whatever you want to the file from here. Specialize on
the structure-object you made that inherits from ‘action’
end-marker) storage store-code) ¶magic-number) storage store-object) ¶write-reference-count) storage store-object) ¶too-much-data)) ¶too-much-data)) ¶structure-object.
structure-object.
structure-object.
string
"default"
fixnum
0
(make-hash-table :test (quote eql))
(make-hash-table :test (quote equal))
(make-hash-table :test (quote eql))
(make-hash-table :test (quote eql))
(make-hash-table :test (quote eql))
(or null function)
(or null function)
(unsigned-byte 8)
cl-binary-store::+end-action-code+
This slot is read-only.
Something that is instantiated at the start of the store process, regardless of whether track-references is true or not. Like OBJECT-INFO, and LIST-LENGTHS.
A static memory buffer (SAP) with an OFFSET, which is the offset in bytes of the
first valid piece of unread data in SAP. Then MAX which is the end of valid data
within the buffer. Then SIZE is the size in bytes of the SAP (used to inform chunked
copying). FLUSHER is a (lambda (storage)) which when called will try to fill the buffer
with new data. It will return the number of available bytes and may modify offset and
max.
We also have a STORE, which *may* if it exists be a
(simple-array (unsigned-byte 8) (SIZE)) which the static memory
buffer is based on (used only to speed up utf8 encoding).
UNDERLYING-STREAM which will be a stream if we are just a buffer in
front of a stream (unused).
The flusher is responsible for updating total-read and checking against max-to-read
structure-object.
fixnum
0
common-lisp.
fixnum
0
system-area-pointer
fixnum
0
function
(or null (simple-array (unsigned-byte 8) (*)))
(or null stream)
fixnum
0
fixnum
cl-binary-store:*max-to-read*
A ref-table is a hash table which is used solely to track references. It will be nil instantiated unless *track-references* is T. After the reference counting phase, only elements in it that are multiply referenced will be retained and have their value set to T
Information about a defrestore statement
Information about a defstore statement
A static memory buffer (SAP) with an OFFSET, which is the offset in bytes where you should
write new data to. MAX is the size of the SAP buffer. STORE, if it exists, is an
(simple-array (unsigned-byte 8) (MAX)) which is the vector on which the SAP is based.
FLUSHER is a function (lambda (write-storage)) which flushes data out of the buffer and returns the
new OFFSET (and updates OFFSET). UNDERLYING-STREAM, if exists, is the stream which is is used by
the flusher.
structure-object.
fixnum
0
common-lisp.
fixnum
0
system-area-pointer
function
(or null (simple-array (unsigned-byte 8) (*)))
(or null stream)
| Jump to: | (
A B C D E F G I L M N O P R S T U V W |
|---|
| Jump to: | (
A B C D E F G I L M N O P R S T U V W |
|---|
| Jump to: | *
+
B C D F I L M N O P R S T U V W |
|---|
| Jump to: | *
+
B C D F I L M N O P R S T U V W |
|---|
| Jump to: | A B C E F G H I M N O P R S T U W |
|---|
| Jump to: | A B C E F G H I M N O P R S T U W |
|---|