The zcdb Reference Manual

Next: , Previous: , Up: (dir)   [Contents][Index]

The zcdb Reference Manual

This is the zcdb Reference Manual, version 1.0.4, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Aug 15 06:13:31 2022 GMT+0.

Table of Contents

Next: , Previous: , Up: The zcdb Reference Manual   [Contents][Index]

1 Introduction

The ZCDB project is for reading and writing cdb files from Common
Lisp.

It is available under a BSD-like license. See the file LICENSE.txt for
details.

For full documentation, see http://www.xach.com/lisp/zcdb/ or the
doc/index.html file in this archive.

Next: , Previous: , Up: The zcdb Reference Manual   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.

Previous: , Up: Systems   [Contents][Index]

2.1 zcdb

Read and write cdb files, as specified in http://cr.yp.to/cdb.html

Author

Zach Beane <xach@xach.com>

License

MIT

Version

1.0.4

Source

zcdb.asd.

Child Components

Next: , Previous: , Up: The zcdb Reference Manual   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

Previous: , Up: Files   [Contents][Index]

3.1 Lisp

Next: , Previous: , Up: Lisp   [Contents][Index]

3.1.1 zcdb/zcdb.asd

Source

zcdb.asd.

Parent Component

zcdb (system).

ASDF Systems

zcdb.

Next: , Previous: , Up: Lisp   [Contents][Index]

3.1.2 zcdb/package.lisp

Source

zcdb.asd.

Parent Component

zcdb (system).

Packages

zcdb.

Next: , Previous: , Up: Lisp   [Contents][Index]

3.1.3 zcdb/cdb.lisp

Dependency

package.lisp (file).

Source

zcdb.asd.

Parent Component

zcdb (system).

Internals

Next: , Previous: , Up: Lisp   [Contents][Index]

3.1.4 zcdb/reading.lisp

Dependency

cdb.lisp (file).

Source

zcdb.asd.

Parent Component

zcdb (system).

Public Interface
Internals

Previous: , Up: Lisp   [Contents][Index]

3.1.5 zcdb/writing.lisp

Dependency

reading.lisp (file).

Source

zcdb.asd.

Parent Component

zcdb (system).

Public Interface
Internals

Next: , Previous: , Up: The zcdb Reference Manual   [Contents][Index]

4 Packages

Packages are listed by definition order.

Previous: , Up: Packages   [Contents][Index]

4.1 zcdb

Source

package.lisp.

Use List

common-lisp.

Public Interface
Internals

Next: , Previous: , Up: The zcdb Reference Manual   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Public Interface

Next: , Previous: , Up: Public Interface   [Contents][Index]

5.1.1 Macros

Macro: with-output-to-cdb ((cdb file temp-file) &body body)

Evaluate BODY with CDB bound to a CDB-WRITER object. The CDB in progress is written to TEMP-FILE, and then when the CDB is successfully written, TEMP-FILE is renamed to FILE. For atomic operation, FILE and TEMP-FILE must be on the same filesystem.

Package

zcdb.

Source

writing.lisp.

Next: , Previous: , Up: Public Interface   [Contents][Index]

5.1.2 Ordinary functions

Function: add-record (key value cdb-writer)

Add KEY and VALUE to a cdb file. KEY and VALUE should both be (unsigned-byte 8) vectors.

Package

zcdb.

Source

writing.lisp.

Function: lookup (key cdb)
Package

zcdb.

Source

reading.lisp.

Function: map-cdb (function cdb)
Package

zcdb.

Source

reading.lisp.

Previous: , Up: Public Interface   [Contents][Index]

5.1.3 Standalone methods

Method: print-object ((bucket hash-table-bucket) stream)
Source

writing.lisp.

Method: print-object ((record-pointer record-pointer) stream)
Source

writing.lisp.

Previous: , Up: Definitions   [Contents][Index]

5.2 Internals

Next: , Previous: , Up: Internals   [Contents][Index]

5.2.1 Constants

Constant: +initial-hash-value+
Package

zcdb.

Source

cdb.lisp.

Next: , Previous: , Up: Internals   [Contents][Index]

5.2.2 Special variables

Special Variable: *empty-record-pointer*
Package

zcdb.

Source

writing.lisp.

Special Variable: *pointer-padding*
Package

zcdb.

Source

writing.lisp.

Next: , Previous: , Up: Internals   [Contents][Index]

5.2.3 Ordinary functions

Function: bucket-hash-vector (bucket)

Create a hash vector for a bucket. A hash vector has 2x the entries of the bucket, and is initialized to an empty record pointer. The high 24 bits of the hash value of a record pointer, mod the size of the vector, is used as a starting slot, and the vector is walked (wrapping at the end) to find the first free slot for positioning each record pointer entry.

Package

zcdb.

Source

writing.lisp.

Function: call-with-output-to-cdb (cdb-pathname temp-pathname fun)

Call FUN with one argument, a CDB-WRITER instance to which records can be added with ADD-RECORD.

Package

zcdb.

Source

writing.lisp.

Function: cdb-hash (octets)

http://cr.yp.to/cdb/cdb.txt

Package

zcdb.

Source

cdb.lisp.

Function: finish-cdb-writer (cdb-writer)

Write the trailing hash tables and leading table pointers to the cdb file.

Package

zcdb.

Source

writing.lisp.

Function: lookup-record-at (position key stream)
Package

zcdb.

Source

reading.lisp.

Function: make-growable-vector (&key size element-type)
Package

zcdb.

Source

cdb.lisp.

Function: make-octet-vector (size)
Package

zcdb.

Source

reading.lisp.

Function: read-cdb-u32 (stream)
Package

zcdb.

Source

reading.lisp.

Function: stream-lookup (key stream)
Package

zcdb.

Source

reading.lisp.

Function: stream-map-cdb (function stream)
Package

zcdb.

Source

reading.lisp.

Function: table-slot-lookup (key hash table-position initial-slot slot-count stream)
Package

zcdb.

Source

reading.lisp.

Function: write-bucket-hash-table (bucket stream)

Write BUCKET’s hash table vector to STREAM.

Package

zcdb.

Source

writing.lisp.

Function: write-cdb-u32 (u32 stream)

Write an (unsigned-byte 32) value to STREAM in little-endian order.

Package

zcdb.

Source

writing.lisp.

Function: write-hash-tables (cdb-writer)

Write the traililng hash tables to the end of the cdb file. Initializes the position of the buckets in the process.

Package

zcdb.

Source

writing.lisp.

Function: write-pointers (cdb-writer)

Write the leading hash table pointers to the beginning of the cdb file. Must be called after WRITE-HASH-TABLES, or the positions won’t be available.

Package

zcdb.

Source

writing.lisp.

Next: , Previous: , Up: Internals   [Contents][Index]

5.2.4 Generic functions

Generic Reader: buckets (object)
Package

zcdb.

Methods
Reader Method: buckets ((cdb-writer cdb-writer))

automatically generated reader method

Source

writing.lisp.

Target Slot

buckets.

Generic Writer: (setf buckets) (object)
Package

zcdb.

Methods
Writer Method: (setf buckets) ((cdb-writer cdb-writer))

automatically generated writer method

Source

writing.lisp.

Target Slot

buckets.

Generic Reader: end-of-records-position (object)
Package

zcdb.

Methods
Reader Method: end-of-records-position ((cdb-writer cdb-writer))

automatically generated reader method

Source

writing.lisp.

Target Slot

end-of-records-position.

Generic Writer: (setf end-of-records-position) (object)
Package

zcdb.

Methods
Writer Method: (setf end-of-records-position) ((cdb-writer cdb-writer))

automatically generated writer method

Source

writing.lisp.

Target Slot

end-of-records-position.

Generic Reader: entries (object)
Generic Writer: (setf entries) (object)
Package

zcdb.

Methods
Reader Method: entries ((hash-table-bucket hash-table-bucket))
Writer Method: (setf entries) ((hash-table-bucket hash-table-bucket))

A vector of record-pointers.

Source

writing.lisp.

Target Slot

entries.

Generic Function: entry-count (object)
Package

zcdb.

Source

writing.lisp.

Methods
Method: entry-count ((object hash-table-bucket))
Generic Reader: hash-value (object)
Generic Writer: (setf hash-value) (object)
Package

zcdb.

Methods
Reader Method: hash-value ((record-pointer record-pointer))
Writer Method: (setf hash-value) ((record-pointer record-pointer))

The hash value of the record key.

Source

writing.lisp.

Target Slot

hash-value.

Generic Reader: output (object)
Package

zcdb.

Methods
Reader Method: output ((cdb-writer cdb-writer))

automatically generated reader method

Source

writing.lisp.

Target Slot

output.

Generic Writer: (setf output) (object)
Package

zcdb.

Methods
Writer Method: (setf output) ((cdb-writer cdb-writer))

automatically generated writer method

Source

writing.lisp.

Target Slot

output.

Generic Reader: record-position (object)
Generic Writer: (setf record-position) (object)
Package

zcdb.

Methods
Reader Method: record-position ((record-pointer record-pointer))
Writer Method: (setf record-position) ((record-pointer record-pointer))

The file position at which the record is stored.

Source

writing.lisp.

Target Slot

record-position.

Generic Function: slot-count (object)
Package

zcdb.

Source

writing.lisp.

Methods
Method: slot-count ((object hash-table-bucket))
Generic Reader: table-position (object)
Generic Writer: (setf table-position) (object)
Package

zcdb.

Methods
Reader Method: table-position ((hash-table-bucket hash-table-bucket))
Writer Method: (setf table-position) ((hash-table-bucket hash-table-bucket))

The file position at which this table is (eventually) slotted.

Source

writing.lisp.

Target Slot

table-position.

Previous: , Up: Internals   [Contents][Index]

5.2.5 Classes

Class: cdb-writer
Package

zcdb.

Source

writing.lisp.

Direct methods
Direct Default Initargs
InitargValue
:end-of-records-position2048
:buckets(map-into (make-array 256) (lambda nil (make-instance (quote hash-table-bucket))))
Direct slots
Slot: buckets
Initargs

:buckets

Readers

buckets.

Writers

(setf buckets).

Slot: end-of-records-position
Initargs

:end-of-records-position

Readers

end-of-records-position.

Writers

(setf end-of-records-position).

Slot: output
Initargs

:output

Readers

output.

Writers

(setf output).

Class: hash-table-bucket

During construction of the CDB, record pointers are
accumulated into one of 256 hash table buckets, depending on the low 8 bits of the hash value of the key. At the end of record writing, these buckets are used to write out hash table vectors at the end of the file, and write pointers to the hash table vectors at the start of the file.

Package

zcdb.

Source

writing.lisp.

Direct methods
Direct Default Initargs
InitargValue
:table-position0
:entries(make-growable-vector)
Direct slots
Slot: table-position

The file position at which this table is (eventually) slotted.

Initargs

:table-position

Readers

table-position.

Writers

(setf table-position).

Slot: entries

A vector of record-pointers.

Initargs

:entries

Readers

entries.

Writers

(setf entries).

Class: record-pointer

Every key/value record written to a CDB has a
corresponding record pointer, which tracks the key’s hash value and the record’s position in the data file. When all records have been written to the file, these record pointers are organized into hash tables at the end of the cdb file.

Package

zcdb.

Source

writing.lisp.

Direct methods
Direct Default Initargs
InitargValue
:hash-value0
:record-position0
Direct slots
Slot: hash-value

The hash value of the record key.

Initargs

:hash-value

Readers

hash-value.

Writers

(setf hash-value).

Slot: record-position

The file position at which the record is stored.

Initargs

:record-position

Readers

record-position.

Writers

(setf record-position).

Previous: , Up: The zcdb Reference Manual   [Contents][Index]

Appendix A Indexes

Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   (  
A   B   C   E   F   G   H   L   M   O   P   R   S   T   W  
Index Entry  Section
(
(setf buckets): Private generic functions
(setf buckets): Private generic functions
(setf end-of-records-position): Private generic functions
(setf end-of-records-position): Private generic functions
(setf entries): Private generic functions
(setf entries): Private generic functions
(setf hash-value): Private generic functions
(setf hash-value): Private generic functions
(setf output): Private generic functions
(setf output): Private generic functions
(setf record-position): Private generic functions
(setf record-position): Private generic functions
(setf table-position): Private generic functions
(setf table-position): Private generic functions
A
add-record: Public ordinary functions
B
bucket-hash-vector: Private ordinary functions
buckets: Private generic functions
buckets: Private generic functions
C
call-with-output-to-cdb: Private ordinary functions
cdb-hash: Private ordinary functions
E
end-of-records-position: Private generic functions
end-of-records-position: Private generic functions
entries: Private generic functions
entries: Private generic functions
entry-count: Private generic functions
entry-count: Private generic functions
F
finish-cdb-writer: Private ordinary functions
Function, add-record: Public ordinary functions
Function, bucket-hash-vector: Private ordinary functions
Function, call-with-output-to-cdb: Private ordinary functions
Function, cdb-hash: Private ordinary functions
Function, finish-cdb-writer: Private ordinary functions
Function, lookup: Public ordinary functions
Function, lookup-record-at: Private ordinary functions
Function, make-growable-vector: Private ordinary functions
Function, make-octet-vector: Private ordinary functions
Function, map-cdb: Public ordinary functions
Function, read-cdb-u32: Private ordinary functions
Function, stream-lookup: Private ordinary functions
Function, stream-map-cdb: Private ordinary functions
Function, table-slot-lookup: Private ordinary functions
Function, write-bucket-hash-table: Private ordinary functions
Function, write-cdb-u32: Private ordinary functions
Function, write-hash-tables: Private ordinary functions
Function, write-pointers: Private ordinary functions
G
Generic Function, (setf buckets): Private generic functions
Generic Function, (setf end-of-records-position): Private generic functions
Generic Function, (setf entries): Private generic functions
Generic Function, (setf hash-value): Private generic functions
Generic Function, (setf output): Private generic functions
Generic Function, (setf record-position): Private generic functions
Generic Function, (setf table-position): Private generic functions
Generic Function, buckets: Private generic functions
Generic Function, end-of-records-position: Private generic functions
Generic Function, entries: Private generic functions
Generic Function, entry-count: Private generic functions
Generic Function, hash-value: Private generic functions
Generic Function, output: Private generic functions
Generic Function, record-position: Private generic functions
Generic Function, slot-count: Private generic functions
Generic Function, table-position: Private generic functions
H
hash-value: Private generic functions
hash-value: Private generic functions
L
lookup: Public ordinary functions
lookup-record-at: Private ordinary functions
M
Macro, with-output-to-cdb: Public macros
make-growable-vector: Private ordinary functions
make-octet-vector: Private ordinary functions
map-cdb: Public ordinary functions
Method, (setf buckets): Private generic functions
Method, (setf end-of-records-position): Private generic functions
Method, (setf entries): Private generic functions
Method, (setf hash-value): Private generic functions
Method, (setf output): Private generic functions
Method, (setf record-position): Private generic functions
Method, (setf table-position): Private generic functions
Method, buckets: Private generic functions
Method, end-of-records-position: Private generic functions
Method, entries: Private generic functions
Method, entry-count: Private generic functions
Method, hash-value: Private generic functions
Method, output: Private generic functions
Method, print-object: Public standalone methods
Method, print-object: Public standalone methods
Method, record-position: Private generic functions
Method, slot-count: Private generic functions
Method, table-position: Private generic functions
O
output: Private generic functions
output: Private generic functions
P
print-object: Public standalone methods
print-object: Public standalone methods
R
read-cdb-u32: Private ordinary functions
record-position: Private generic functions
record-position: Private generic functions
S
slot-count: Private generic functions
slot-count: Private generic functions
stream-lookup: Private ordinary functions
stream-map-cdb: Private ordinary functions
T
table-position: Private generic functions
table-position: Private generic functions
table-slot-lookup: Private ordinary functions
W
with-output-to-cdb: Public macros
write-bucket-hash-table: Private ordinary functions
write-cdb-u32: Private ordinary functions
write-hash-tables: Private ordinary functions
write-pointers: Private ordinary functions
Jump to:   (  
A   B   C   E   F   G   H   L   M   O   P   R   S   T   W