The zcdb Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The zcdb Reference Manual

This is the zcdb Reference Manual, version 1.0.4, generated automatically by Declt version 2.3 "Robert April" on Wed Mar 14 04:46:06 2018 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

The ZCDB project is for reading and writing cdb files from Common
Lisp.

It is available under a BSD-like license. See the file LICENSE.txt for
details.

For full documentation, see http://www.xach.com/lisp/zcdb/ or the
doc/index.html file in this archive.


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 zcdb

Author

Zach Beane <xach@xach.com>

License

MIT

Description

Read and write cdb files, as specified in http://cr.yp.to/cdb.html

Version

1.0.4

Source

zcdb.asd (file)

Components

Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 zcdb.asd

Location

zcdb.asd

Systems

zcdb (system)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 zcdb/package.lisp

Parent

zcdb (system)

Location

package.lisp

Packages

zcdb


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.3 zcdb/cdb.lisp

Dependency

package.lisp (file)

Parent

zcdb (system)

Location

cdb.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.4 zcdb/reading.lisp

Dependency

cdb.lisp (file)

Parent

zcdb (system)

Location

reading.lisp

Exported Definitions
Internal Definitions

Previous: , Up: Lisp files   [Contents][Index]

3.1.5 zcdb/writing.lisp

Dependency

reading.lisp (file)

Parent

zcdb (system)

Location

writing.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 zcdb

Source

package.lisp (file)

Use List

common-lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Macros

Macro: with-output-to-cdb (CDB FILE TEMP-FILE) &body BODY

Evaluate BODY with CDB bound to a CDB-WRITER object. The CDB in progress is written to TEMP-FILE, and then when the CDB is successfully written, TEMP-FILE is renamed to FILE. For atomic operation, FILE and TEMP-FILE must be on the same filesystem.

Package

zcdb

Source

writing.lisp (file)


Previous: , Up: Exported definitions   [Contents][Index]

5.1.2 Functions

Function: add-record KEY VALUE CDB-WRITER

Add KEY and VALUE to a cdb file. KEY and VALUE should both be (unsigned-byte 8) vectors.

Package

zcdb

Source

writing.lisp (file)

Function: lookup KEY CDB
Package

zcdb

Source

reading.lisp (file)

Function: map-cdb FUNCTION CDB
Package

zcdb

Source

reading.lisp (file)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Constants

Constant: +initial-hash-value+
Package

zcdb

Source

cdb.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Special variables

Special Variable: *empty-record-pointer*
Package

zcdb

Source

writing.lisp (file)

Special Variable: *pointer-padding*
Package

zcdb

Source

writing.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Functions

Function: bucket-hash-vector BUCKET

Create a hash vector for a bucket. A hash vector has 2x the entries of the bucket, and is initialized to an empty record pointer. The high 24 bits of the hash value of a record pointer, mod the size of the vector, is used as a starting slot, and the vector is walked (wrapping at the end) to find the first free slot for positioning each record pointer entry.

Package

zcdb

Source

writing.lisp (file)

Function: call-with-output-to-cdb CDB-PATHNAME TEMP-PATHNAME FUN

Call FUN with one argument, a CDB-WRITER instance to which records can be added with ADD-RECORD.

Package

zcdb

Source

writing.lisp (file)

Function: cdb-hash OCTETS

http://cr.yp.to/cdb/cdb.txt

Package

zcdb

Source

cdb.lisp (file)

Function: finish-cdb-writer CDB-WRITER

Write the trailing hash tables and leading table pointers to the cdb file.

Package

zcdb

Source

writing.lisp (file)

Function: lookup-record-at POSITION KEY STREAM
Package

zcdb

Source

reading.lisp (file)

Function: make-growable-vector &key SIZE ELEMENT-TYPE
Package

zcdb

Source

cdb.lisp (file)

Function: make-octet-vector SIZE
Package

zcdb

Source

reading.lisp (file)

Function: read-cdb-u32 STREAM
Package

zcdb

Source

reading.lisp (file)

Function: stream-lookup KEY STREAM
Package

zcdb

Source

reading.lisp (file)

Function: stream-map-cdb FUNCTION STREAM
Package

zcdb

Source

reading.lisp (file)

Function: table-slot-lookup KEY HASH TABLE-POSITION INITIAL-SLOT SLOT-COUNT STREAM
Package

zcdb

Source

reading.lisp (file)

Function: write-bucket-hash-table BUCKET STREAM

Write BUCKET’s hash table vector to STREAM.

Package

zcdb

Source

writing.lisp (file)

Function: write-cdb-u32 U32 STREAM

Write an (unsigned-byte 32) value to STREAM in little-endian order.

Package

zcdb

Source

writing.lisp (file)

Function: write-hash-tables CDB-WRITER

Write the traililng hash tables to the end of the cdb file. Initializes the position of the buckets in the process.

Package

zcdb

Source

writing.lisp (file)

Function: write-pointers CDB-WRITER

Write the leading hash table pointers to the beginning of the cdb file. Must be called after WRITE-HASH-TABLES, or the positions won’t be available.

Package

zcdb

Source

writing.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.4 Generic functions

Generic Function: buckets OBJECT
Generic Function: (setf buckets) NEW-VALUE OBJECT
Package

zcdb

Methods
Method: buckets (CDB-WRITER cdb-writer)

automatically generated reader method

Source

writing.lisp (file)

Method: (setf buckets) NEW-VALUE (CDB-WRITER cdb-writer)

automatically generated writer method

Source

writing.lisp (file)

Generic Function: end-of-records-position OBJECT
Generic Function: (setf end-of-records-position) NEW-VALUE OBJECT
Package

zcdb

Methods
Method: end-of-records-position (CDB-WRITER cdb-writer)

automatically generated reader method

Source

writing.lisp (file)

Method: (setf end-of-records-position) NEW-VALUE (CDB-WRITER cdb-writer)

automatically generated writer method

Source

writing.lisp (file)

Generic Function: entries OBJECT
Generic Function: (setf entries) NEW-VALUE OBJECT
Package

zcdb

Methods
Method: entries (HASH-TABLE-BUCKET hash-table-bucket)
Method: (setf entries) NEW-VALUE (HASH-TABLE-BUCKET hash-table-bucket)

A vector of record-pointers.

Source

writing.lisp (file)

Generic Function: entry-count OBJECT
Package

zcdb

Source

writing.lisp (file)

Methods
Method: entry-count (OBJECT hash-table-bucket)
Generic Function: hash-value OBJECT
Generic Function: (setf hash-value) NEW-VALUE OBJECT
Package

zcdb

Methods
Method: hash-value (RECORD-POINTER record-pointer)
Method: (setf hash-value) NEW-VALUE (RECORD-POINTER record-pointer)

The hash value of the record key.

Source

writing.lisp (file)

Generic Function: output OBJECT
Generic Function: (setf output) NEW-VALUE OBJECT
Package

zcdb

Methods
Method: output (CDB-WRITER cdb-writer)

automatically generated reader method

Source

writing.lisp (file)

Method: (setf output) NEW-VALUE (CDB-WRITER cdb-writer)

automatically generated writer method

Source

writing.lisp (file)

Generic Function: record-position OBJECT
Generic Function: (setf record-position) NEW-VALUE OBJECT
Package

zcdb

Methods
Method: record-position (RECORD-POINTER record-pointer)
Method: (setf record-position) NEW-VALUE (RECORD-POINTER record-pointer)

The file position at which the record is stored.

Source

writing.lisp (file)

Generic Function: slot-count OBJECT
Package

zcdb

Source

writing.lisp (file)

Methods
Method: slot-count (OBJECT hash-table-bucket)
Generic Function: table-position OBJECT
Generic Function: (setf table-position) NEW-VALUE OBJECT
Package

zcdb

Methods
Method: table-position (HASH-TABLE-BUCKET hash-table-bucket)
Method: (setf table-position) NEW-VALUE (HASH-TABLE-BUCKET hash-table-bucket)

The file position at which this table is (eventually) slotted.

Source

writing.lisp (file)


Previous: , Up: Internal definitions   [Contents][Index]

5.2.5 Classes

Class: cdb-writer ()
Package

zcdb

Source

writing.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
Direct slots
Slot: buckets
Initargs

:buckets

Readers

buckets (generic function)

Writers

(setf buckets) (generic function)

Slot: end-of-records-position
Initargs

:end-of-records-position

Readers

end-of-records-position (generic function)

Writers

(setf end-of-records-position) (generic function)

Slot: output
Initargs

:output

Readers

output (generic function)

Writers

(setf output) (generic function)

Direct Default Initargs
InitargValue
:end-of-records-position2048
:buckets(map-into (make-array 256) (lambda nil (make-instance (quote zcdb::hash-table-bucket))))
Class: hash-table-bucket ()

During construction of the CDB, record pointers are
accumulated into one of 256 hash table buckets, depending on the low 8 bits of the hash value of the key. At the end of record writing, these buckets are used to write out hash table vectors at the end of the file, and write pointers to the hash table vectors at the start of the file.

Package

zcdb

Source

writing.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
Direct slots
Slot: table-position

The file position at which this table is (eventually) slotted.

Initargs

:table-position

Readers

table-position (generic function)

Writers

(setf table-position) (generic function)

Slot: entries

A vector of record-pointers.

Initargs

:entries

Readers

entries (generic function)

Writers

(setf entries) (generic function)

Direct Default Initargs
InitargValue
:table-position0
:entries(zcdb::make-growable-vector)
Class: record-pointer ()

Every key/value record written to a CDB has a
corresponding record pointer, which tracks the key’s hash value and the record’s position in the data file. When all records have been written to the file, these record pointers are organized into hash tables at the end of the cdb file.

Package

zcdb

Source

writing.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
Direct slots
Slot: hash-value

The hash value of the record key.

Initargs

:hash-value

Readers

hash-value (generic function)

Writers

(setf hash-value) (generic function)

Slot: record-position

The file position at which the record is stored.

Initargs

:record-position

Readers

record-position (generic function)

Writers

(setf record-position) (generic function)

Direct Default Initargs
InitargValue
:hash-value0
:record-position0

Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   Z  
Index Entry  Section

F
File, Lisp, zcdb.asd: The zcdb<dot>asd file
File, Lisp, zcdb/cdb.lisp: The zcdb/cdb<dot>lisp file
File, Lisp, zcdb/package.lisp: The zcdb/package<dot>lisp file
File, Lisp, zcdb/reading.lisp: The zcdb/reading<dot>lisp file
File, Lisp, zcdb/writing.lisp: The zcdb/writing<dot>lisp file

L
Lisp File, zcdb.asd: The zcdb<dot>asd file
Lisp File, zcdb/cdb.lisp: The zcdb/cdb<dot>lisp file
Lisp File, zcdb/package.lisp: The zcdb/package<dot>lisp file
Lisp File, zcdb/reading.lisp: The zcdb/reading<dot>lisp file
Lisp File, zcdb/writing.lisp: The zcdb/writing<dot>lisp file

Z
zcdb.asd: The zcdb<dot>asd file
zcdb/cdb.lisp: The zcdb/cdb<dot>lisp file
zcdb/package.lisp: The zcdb/package<dot>lisp file
zcdb/reading.lisp: The zcdb/reading<dot>lisp file
zcdb/writing.lisp: The zcdb/writing<dot>lisp file

Jump to:   F   L   Z  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   (  
A   B   C   E   F   G   H   L   M   O   R   S   T   W  
Index Entry  Section

(
(setf buckets): Internal generic functions
(setf buckets): Internal generic functions
(setf end-of-records-position): Internal generic functions
(setf end-of-records-position): Internal generic functions
(setf entries): Internal generic functions
(setf entries): Internal generic functions
(setf hash-value): Internal generic functions
(setf hash-value): Internal generic functions
(setf output): Internal generic functions
(setf output): Internal generic functions
(setf record-position): Internal generic functions
(setf record-position): Internal generic functions
(setf table-position): Internal generic functions
(setf table-position): Internal generic functions

A
add-record: Exported functions

B
bucket-hash-vector: Internal functions
buckets: Internal generic functions
buckets: Internal generic functions

C
call-with-output-to-cdb: Internal functions
cdb-hash: Internal functions

E
end-of-records-position: Internal generic functions
end-of-records-position: Internal generic functions
entries: Internal generic functions
entries: Internal generic functions
entry-count: Internal generic functions
entry-count: Internal generic functions

F
finish-cdb-writer: Internal functions
Function, add-record: Exported functions
Function, bucket-hash-vector: Internal functions
Function, call-with-output-to-cdb: Internal functions
Function, cdb-hash: Internal functions
Function, finish-cdb-writer: Internal functions
Function, lookup: Exported functions
Function, lookup-record-at: Internal functions
Function, make-growable-vector: Internal functions
Function, make-octet-vector: Internal functions
Function, map-cdb: Exported functions
Function, read-cdb-u32: Internal functions
Function, stream-lookup: Internal functions
Function, stream-map-cdb: Internal functions
Function, table-slot-lookup: Internal functions
Function, write-bucket-hash-table: Internal functions
Function, write-cdb-u32: Internal functions
Function, write-hash-tables: Internal functions
Function, write-pointers: Internal functions

G
Generic Function, (setf buckets): Internal generic functions
Generic Function, (setf end-of-records-position): Internal generic functions
Generic Function, (setf entries): Internal generic functions
Generic Function, (setf hash-value): Internal generic functions
Generic Function, (setf output): Internal generic functions
Generic Function, (setf record-position): Internal generic functions
Generic Function, (setf table-position): Internal generic functions
Generic Function, buckets: Internal generic functions
Generic Function, end-of-records-position: Internal generic functions
Generic Function, entries: Internal generic functions
Generic Function, entry-count: Internal generic functions
Generic Function, hash-value: Internal generic functions
Generic Function, output: Internal generic functions
Generic Function, record-position: Internal generic functions
Generic Function, slot-count: Internal generic functions
Generic Function, table-position: Internal generic functions

H
hash-value: Internal generic functions
hash-value: Internal generic functions

L
lookup: Exported functions
lookup-record-at: Internal functions

M
Macro, with-output-to-cdb: Exported macros
make-growable-vector: Internal functions
make-octet-vector: Internal functions
map-cdb: Exported functions
Method, (setf buckets): Internal generic functions
Method, (setf end-of-records-position): Internal generic functions
Method, (setf entries): Internal generic functions
Method, (setf hash-value): Internal generic functions
Method, (setf output): Internal generic functions
Method, (setf record-position): Internal generic functions
Method, (setf table-position): Internal generic functions
Method, buckets: Internal generic functions
Method, end-of-records-position: Internal generic functions
Method, entries: Internal generic functions
Method, entry-count: Internal generic functions
Method, hash-value: Internal generic functions
Method, output: Internal generic functions
Method, record-position: Internal generic functions
Method, slot-count: Internal generic functions
Method, table-position: Internal generic functions

O
output: Internal generic functions
output: Internal generic functions

R
read-cdb-u32: Internal functions
record-position: Internal generic functions
record-position: Internal generic functions

S
slot-count: Internal generic functions
slot-count: Internal generic functions
stream-lookup: Internal functions
stream-map-cdb: Internal functions

T
table-position: Internal generic functions
table-position: Internal generic functions
table-slot-lookup: Internal functions

W
with-output-to-cdb: Exported macros
write-bucket-hash-table: Internal functions
write-cdb-u32: Internal functions
write-hash-tables: Internal functions
write-pointers: Internal functions

Jump to:   (  
A   B   C   E   F   G   H   L   M   O   R   S   T   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *   +  
B   C   E   H   O   R   S   T  
Index Entry  Section

*
*empty-record-pointer*: Internal special variables
*pointer-padding*: Internal special variables

+
+initial-hash-value+: Internal constants

B
buckets: Internal classes

C
Constant, +initial-hash-value+: Internal constants

E
end-of-records-position: Internal classes
entries: Internal classes

H
hash-value: Internal classes

O
output: Internal classes

R
record-position: Internal classes

S
Slot, buckets: Internal classes
Slot, end-of-records-position: Internal classes
Slot, entries: Internal classes
Slot, hash-value: Internal classes
Slot, output: Internal classes
Slot, record-position: Internal classes
Slot, table-position: Internal classes
Special Variable, *empty-record-pointer*: Internal special variables
Special Variable, *pointer-padding*: Internal special variables

T
table-position: Internal classes

Jump to:   *   +  
B   C   E   H   O   R   S   T  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   C   H   P   R   S   Z  
Index Entry  Section

C
cdb-writer: Internal classes
Class, cdb-writer: Internal classes
Class, hash-table-bucket: Internal classes
Class, record-pointer: Internal classes

H
hash-table-bucket: Internal classes

P
Package, zcdb: The zcdb package

R
record-pointer: Internal classes

S
System, zcdb: The zcdb system

Z
zcdb: The zcdb system
zcdb: The zcdb package

Jump to:   C   H   P   R   S   Z