The lambdalite Reference Manual

Table of Contents

The lambdalite Reference Manual

This is the lambdalite Reference Manual, version 1.0.1, generated automatically by Declt version 3.0 "Montgomery Scott" on Wed Nov 04 14:01:39 2020 GMT+0.

1 Introduction

LambdaLite

A functional, relational database in about 250 lines of Common Lisp.

• "Woah! This is really cool" — marknadal
• "stunningly lovely" — hal68k

SQL. NoSQL. ORMs. Key-value stores. There are a variety of approaches available for dealing with data. LambdaLite might be called functional and relational, for lack of better terms. The "relational" part is straightfoward: data is organized into tables. The "functional" part comes from a break with traditional SQL-style query languages: Lisp function closures are used to express queries over in-memory, in-process Lisp data.

Lisp Plug

People often ask "why Lisp?" It's hard to sum up all the benefits of Lisp in a few words. However, what LambdaLite does would be impossible in most languages. LambdaLite's where clauses are macros that seek out '/'-prefixed keywords and replace them with row attribute references. The resulting expression becomes a function closure that is compiled by most Lisp implementations into native code. Is that cool or what?

Motivation

Consider the following SQL query:

"SELECT * FROM USERS WHERE UPPER(name) = 'BOB'"

A few problems are apparent:

• The UPPER function is part of a separate language with its own syntax and semantics
• The whole query is represented as a string in our application language, so it misses out on highlighting, completion, validation, etc.
• Parameterization of 'BOB' into a variable lacks elegance in most SQL client libraries

Under LambdaLite, we could instead write:

(select :users (where (equal (string-upcase :/name) "BOB")))

Why is this useful?

• It's actual Lisp syntax: everything is a Lisp function (e.g. string-upcase) or macro
• "BOB" can be parameterized natively by any Lisp variable
• The where clause forms a compiled function closure with the full expressiveness of Lisp

Notice that :/name begins with a slash; this is to distinguish it as a row attribute instead of an ordinary Lisp keyword.

Getting Started

LambdaLite is schemaless for flexibility. Rather than defining tables themselves, you define attributes that can be used on any table, using defattributes like so:

(defmacro str-member (&rest strings)
  `(lambda (x) (member x '(,@strings) :test #'string=)))

(defattributes
  :/ticket-id #'integerp
  :/title (lambda (x) (<= 1 (length x) 200))
  :/ticket-type (str-member "defect" "enhancement" "question")
  :/mocl-version (str-member "14.08" "14.05" "14.02" "13.08" "13.06" "n/a")
  :/target-os (str-member "iOS" "Android" "OS X" "n/a")
  :/dev-os (str-member "Mac OS X" "Linux 32-bit" "Linux 64-bit" "n/a")
  :/description (lambda (x) (<= 1 (length x) 64000))
  :/created-date
  :/created-by
  :/modified-date
  :/user-id
  :/display-name (lambda (x) (and x (<= 3 (length x) 20) (ppcre:scan "^[a-zA-Z0-9][a-zA-Z0-9][a-zA-Z0-9]+$" x))))

This will define getter functions that can be used like (:/ticket-id row) as well as validation functions like (valid-title-p title).

Example Session

(require :lambdalite)
(use-package :lambdalite)
(load-db :path "~/db/")
(insert :cars '(:/car-id 1 :/make "Honda" :/color "blue")
              '(:/car-id 2 :/make "Ford" :/color "red"))
  => 2
(select :cars (where (equal :/color "red")))
  => ((:/CAR-ID 2 :/MAKE "Ford" :/COLOR "red"))
(defmacro str-member (&rest strings)
  `(lambda (x) (member x '(,@strings) :test #'string=)))
(defattributes
  :/car-id #'integerp 
  :/make #'stringp 
  :/color (str-member "red" "green" "blue"))
(valid-color-p "asdf")
  => nil
(dolist (row (select :cars)) 
  (format t "Make: ~A, Color: ~A~%" (:/make row) (:/color row)))
  >> Make: Honda, Color: blue
     Make: Ford, Color: red
(mapcar #':/color (select :cars))
  => ("blue" "red")
(insert :cars '(:/car-id 3 :/make "Toyota" :/color "green") 
              '(:/car-id 4 :/make "Audi" :/color "red"))
  => 2
(sort (select :cars) #'string< :key #':/make)
  => ((:/CAR-ID 4 :/MAKE "Audi" :/COLOR "red")
      (:/CAR-ID 2 :/MAKE "Ford" :/COLOR "red")
      (:/CAR-ID 1 :/MAKE "Honda" :/COLOR "blue")
      (:/CAR-ID 3 :/MAKE "Toyota" :/COLOR "green"))

Transactions

LambdaLite provides the with-tx macro to wrap transactions, which are executed serially across threads. For example, the following code is safe under a multi-threaded web server like Hunchentoot:

;; ... create a new ticket
(with-tx 
  (let ((user-id (logged-in-user-id))
        (ticket-id (1+ (length (select :tickets)))))
    (unless user-id 
      (error "Not logged in"))
    (insert :tickets (list :/ticket-id ticket-id :/created-by user-id :/ticket-status "open"))))

Any data commands that are used outside of a with-tx transaction will automatically be treated each individually as separate transactions.

Caveats

"Do things that don't scale" — Paul Graham

LambdaLite is completely unscalable — by design. Don't use it for heavy loads or large data sets. Even medium-sized jobs are a stretch. That's why 'Lite' is in the name.

Compatibility

• LambdaLite works under SBCL and probably other Lisps
• LambdaLite runs on mocl for mobile data storage

Who is using LambdaLite?

LambdaLite powers the mocl support site at https://wukix.com/support/ with delightfully low query latency on a single Amazon EC2 micro instance.

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 lambdalite

Author

Wukix Inc <engineering@wukix.com>

License

MIT

Description

A functional, relational Lisp database

Long Description

A functional, relational database in about 250 lines of Common Lisp

Version

1.0.1

Dependencies

• wu-sugar
• bordeaux-threads

Source

lambdalite.asd (file)

Component

lambdalite.lisp (file)

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

3.1 Lisp

3.1.1 lambdalite.asd

Location

lambdalite.asd

Systems

lambdalite (system)

Packages

lambdalite-asd

3.1.2 lambdalite/lambdalite.lisp

Parent

lambdalite (system)

Location

lambdalite.lisp

Packages

lambdalite

Exported Definitions

• defattributes (macro)
• del (function)
• insert (function)
• keyset (macro)
• list-tables (function)
• load-db (function)
• select (function)
• select1 (function)
• update (function)
• where (macro)
• where-on (macro)
• with-tx (macro)

Internal Definitions

• *db* (special variable)
• *db-path* (special variable)
• *tx* (special variable)
• *tx-lock* (special variable)
• *tx-modified-list* (special variable)
• clone-temporary (function)
• make-filename (function)
• persist (function)
• with-lock (macro)

4 Packages

Packages are listed by definition order.

4.1 lambdalite-asd

Source

lambdalite.asd

Use List

• common-lisp
• asdf/interface

4.2 lambdalite

Source

lambdalite.lisp (file)

Use List

• wu-sugar
• common-lisp

Exported Definitions

• defattributes (macro)
• del (function)
• insert (function)
• keyset (macro)
• list-tables (function)
• load-db (function)
• select (function)
• select1 (function)
• update (function)
• where (macro)
• where-on (macro)
• with-tx (macro)

Internal Definitions

• *db* (special variable)
• *db-path* (special variable)
• *tx* (special variable)
• *tx-lock* (special variable)
• *tx-modified-list* (special variable)
• clone-temporary (function)
• make-filename (function)
• persist (function)
• with-lock (macro)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Macros

Macro: defattributes &body BODY

Defines getter functions and validation functions for row attributes. Example:
(defattributes
:/item-id
:/title (lambda (x) (<= 1 (length x) 200)))
This would create the getter functions :/item-id and :/title, plus the validation function valid-title-p.

Package

lambdalite

Source

lambdalite.lisp (file)

Macro: keyset &rest KEY/VALUE-PAIRS

An ’update function’ for use in UPDATE. Example: (update :things (where foo) (keyset :/bar 1 :/baz 2))

Package

lambdalite

Source

lambdalite.lisp (file)

Macro: where EXPRESSION

Package

lambdalite

Source

lambdalite.lisp (file)

Macro: where-on ROW-BINDING EXPRESSION

Like WHERE, except giving direct access to the row, bound to ROW-BINDING. Unlike WHERE, EXPRESSION runs unmodified. Example: (select :users (where-on u (equal (:/age u) 30)))

Package

lambdalite

Source

lambdalite.lisp (file)

Macro: with-tx &body BODY

Execute a transaction as follows:
1. Every command that writes executes against a temporary table.
2. Back up the real target tables on disk.
3. Swap the target tables with the temporary ones in memory and persist.
4. If 3 does not complete, reverse the original swaps with the temporary tables and restore the backups as primary. 5. Delete backup files and temporary tables.

Package

lambdalite

Source

lambdalite.lisp (file)

5.1.2 Functions

Function: del NAME WHERE-PREDICATE

Usage: (del :foo (where ...))

Package

lambdalite

Source

lambdalite.lisp (file)

Function: insert NAME &rest ROWS

Example: (insert :users (:/user-id 1 :/name "Bob") (:/user-id 2 :/name "Alice"))

Package

lambdalite

Source

lambdalite.lisp (file)

Function: list-tables ()

Returns a list of table names (as keywords).

Package

lambdalite

Source

lambdalite.lisp (file)

Function: load-db &key PATH

Loads data files if they exist at PATH, or initializes an empty setup if not. Call once at startup.

Package

lambdalite

Source

lambdalite.lisp (file)

Function: select NAME &optional WHERE-PREDICATE

Example: (select :users (where (search "Bob" :/name)))

Package

lambdalite

Source

lambdalite.lisp (file)

Function: select1 NAME &optional WHERE-PREDICATE

Like SELECT, but returns the first row found.

Package

lambdalite

Source

lambdalite.lisp (file)

Function: update NAME WHERE-PREDICATE UPDATE-FUNCTION

Example: (update :things (where (evenp :/foo)) (keyset :/bar 1 :/baz 2))

Package

lambdalite

Source

lambdalite.lisp (file)

5.2 Internal definitions

5.2.1 Special variables

Special Variable: *db*

Package

lambdalite

Source

lambdalite.lisp (file)

Special Variable: *db-path*

Package

lambdalite

Source

lambdalite.lisp (file)

Special Variable: *tx*

Package

lambdalite

Source

lambdalite.lisp (file)

Special Variable: *tx-lock*

Package

lambdalite

Source

lambdalite.lisp (file)

Special Variable: *tx-modified-list*

Package

lambdalite

Source

lambdalite.lisp (file)

5.2.2 Macros

Macro: with-lock &body BODY

Package

lambdalite

Source

lambdalite.lisp (file)

5.2.3 Functions

Function: clone-temporary NAME

Package

lambdalite

Source

lambdalite.lisp (file)

Function: make-filename NAME

Package

lambdalite

Source

lambdalite.lisp (file)

Function: persist NAME &optional FILENAME

Package

lambdalite

Source

lambdalite.lisp (file)

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables

A.4 Data types