The esrap-liquid Reference Manual

Table of Contents

The esrap-liquid Reference Manual

This is the esrap-liquid Reference Manual, version 2.3, generated automatically by Declt version 3.0 "Montgomery Scott" on Wed Feb 19 20:27:47 2020 GMT+0.

1 Introduction

ESRAP-LIQUID

MAJOR API CHANGE IN VERSIONS 2.*: we no longer use codewalker, hence rule definitions now compile much faster. The price to pay is that sometimes you need to specify manually, that something is a sub-rule (see V-macrolet in source and section "API Change" in this README).

Packrat parser generator, with possibility to use full Common Lisp when defining parsing rules.

It started as a fork of ESRAP by Nikodemus Siivola (https://github.com/nikodemus/esrap), which is a DSL with (somewhat) more rigid syntax, but (as of now) nicer error reports.

Original idea of a packrat parser generator is described in this article:

• Bryan Ford, 2002, "Packrat Parsing: a Practical Linear Time Algorithm with Backtracking".

  http://pdos.csail.mit.edu/~baford/packrat/thesis/

Operates both on strings and streams -- see section "Streaming" below.

Examples:

; plain characters match to precisely that character in text
ESRAP-LIQUID> (parse '#\a "a")
#\a
1
ESRAP-LIQUID> (parse '#\a "b")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.

; same goes for plain strings
ESRAP-LIQUID> (parse '"foo" "foo")
"foo"
3
ESRAP-LIQUID> (parse '"foo" "bar")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.

; CHARACTER matches any single character
ESRAP-LIQUID> (parse 'character "a")
#\a
1
ESRAP-LIQUID> (parse 'character "b")
#\b
1

; (ANY-STRING <LENGTH>) matches any string of a given length character
ESRAP-LIQUID> (parse '(any-string 3) "foo")
"foo"
3
ESRAP-LIQUID> (parse '(any-string 3) "bar")
"bar"
3
ESRAP-LIQUID> (parse '(any-string 3) "caboom!")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.

; (!! <EXPR>) matches, whenever EXPR fails and consumes one character
ESRAP-LIQUID> (parse '(!! "foo") "bar" :junk-allowed t)
#\b
1
ESRAP-LIQUID> (parse '(!! "foo") "foo")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.

; (|| &rest <EXPRS>) is an ordered choice, matches, whenever one of EXPRS succeeds, and outputs it
ESRAP-LIQUID> (parse '(|| #\a #\b) "a")
#\a
1
ESRAP-LIQUID> (parse '(|| #\a #\b) "b")
#\b
1
ESRAP-LIQUID> (parse '(|| #\a #\b) "c")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.

; (times <EXPR> &key from upto exactly) greedy matches expression multiple times, returns values as a list.
; If :UPTO or :EXACTLY is given, then consumes only that much exprs, even if there are more
; if :FROM is given, fails if consumed less than :FROM expressions
ESRAP-LIQUID> (parse '(times #\a) "")
NIL
0
ESRAP-LIQUID> (parse '(times #\a) "aaa")
(#\a #\a #\a)
3
ESRAP-LIQUID> (parse '(times #\a :exactly 6) "aaaaaa")
(#\a #\a #\a #\a #\a #\a)
6
ESRAP-LIQUID> (parse '(times #\a :exactly 6) "aaa")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.

; (postimes <EXPR>) is an alias for (times <EXPR> :from 1)
ESRAP-LIQUID> (parse '(postimes #\a) "")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.
ESRAP-LIQUID> (parse '(postimes #\a) "aaa")
(#\a #\a #\a)
3

; (? <EXPR>) returns result of parsing of EXPR, when parsing succeeds, and NIL otherwise, does not fail
ESRAP-LIQUID> (parse '(? #\a) "")
NIL
0
ESRAP-LIQUID> (parse '(? #\a) "a")
#\a
1

Other operators, defined by ESRAP-LIQUID, include:

• (character-ranges ranges) -- succeeds, when next character fits into specified ranges
• (& followed-by) -- parses subclause, then rewinds iterator back, like PEEK-CHAR, but with possibly more complex expressions
• (-> followed-by-not-gen) -- same as &, but produces NIL
• (<- preceded-by-not-gen) -- succeeds, if preceeded by something of length 1, produces NIL
• (! not-followed-by) -- an antipode of &, also rewinds iterator
• (pred #' expr) -- succeeds, if #' returns true
• (most-full-parse &rest exprs) -- try to parse all subexpressions and choose the longest one
• (v subexpr &rest args) -- syntactic sugar for DESCEND-WITH-RULE
• (progn-v &rest forms) -- like PROGN, but wraps its forms in DESCEND-WITH-RULE when needed
• (prog1-v &rest forms) -- like PROG1, but wraps its forms in DESCEND-WITH-RULE when needed
• (list-v &rest forms) -- like LIST, but wraps its forms in DESCEND-WITH-RULE when needed

Typical idioms:

; succeed, when all subexpressions succeed, return list of those subexpressions
; Note that now you need to explicitly write (V ...) in order to indicate that
; character means "try to parse this character"
ESRAP-LIQUID> (parse '(list (v #\a) (v #\b) (v #\c)) "abc")
(#\a #\b #\c)
3
; succeed, when all subexpression succeed, return only last subexpression
ESRAP-LIQUID> (parse '(progn (v #\a) (v #\b) (v #\c)) "abc")
#\c
3
; succeed, when all subexpression succeed, return only first subexpression
ESRAP-LIQUID> (parse '(prog1 (v #\a) (v #\b) (v #\c)) "abc")
#\a
3

For more examples, see example-sexp.lisp, example-symbol-table.lisp, example-very-context-sensitive.lisp. For more real-life examples see my YAML parser https://github.com/mabragor/cl-yaclyaml. The parsing part uses ESRAP-LIQUID extensively, in particular, in ways different from traditional ESRAP.

Defining rules

Of course, if you could only write one-liners with PARSE, it won't be interesting at all So, you can define new rules using DEFRULE macro.

ESRAP-LIQUID> (defrule foo+ ()
                (postimes "foo"))
ESRAP-LIQUID> (parse 'foo+ "foofoofoo")
("foo" "foo" "foo")

; simple arguments to rules are also possible
ESRAP-LIQUID> (defrule foo-times (times)
                (times "foo" :exactly times))
ESRAP-LIQUID> (parse '(v foo-times 3) "foofoofoo")
("foo" "foo" "foo")
ESRAP-LIQUID> (parse '(v foo-times 4) "foofoofoo")
#<ESRAP-LIQUID::SIMPLE-ESRAP-ERROR "~a~%">.

Defining esrap-environments

Usually you don't want to mix parsing rules for different projects -- you want a mechanism to scope your rules in some way. For that you can use DEFINE-ESRAP-ENV macro.

(define-esrap-env foobar)

;; Now we can define rules in 'foobar' scope
(define-foobar-rule asdf ()
   ... ; rule definition here)

DEFINE-ESRAP-ENV has a MAINLY-NON-CONTEXT key. If it's T, then DEFINE-$(NAME)-RULE expands into DEF-NOCONTEXT-RULE, rather than DEFRULE. It's still possible to get context-sensitive rules by using DEFINE-C-$(NAME)-RULE. Vice versa, if MAINLY-NON-CONTEXT key is NIL (default), then non-context-sensitive rules can be defined using DEFINE-NC-$(NAME)-RULE.

Capturing-variables : CAP, RECAP, RECAP?

Analogously to capturing groups in regexps, it is possible to capture results of parsing of named rules, to aid destructuring.

Example: instead of clumsy

(define-rule dressed-rule-clumsy ()
  (prog1 (progn (v "foo") (v "bar") (v "baz")
                (v meat))
         (v "rest1") (v "rest2") (v "rest3")))

you may write something like

(define-rule dressed-rule-elegant ()
  (v "foo") (v "bar") (v "baz") (cap 1 meat) (v "rest1") (v "rest2") (v "rest3")
  (recap 1))

I.e. result of parsing of rule with name MEAT is stashed with CAP macro and then accessed using RECAP macro.

Difference between RECAP and RECAP? is that while the former fails parsing if the requested key was not captured, the latter just produces NIL.

See tests for examples of usage. Also see CL-MIZAR parsing.lisp, where this is used a lot.

Streaming

It's possible to parse both strings and streams. Main function for parsing streams is PARSE-STREAM.

Example of usage:

(parse-stream 'some-rule (make-string-input-stream "foobar") :junk-allowed t)

In general, both PARSE and PARSE-STREAMS are just wrappers around more general PARSE-TOKEN-ITER -- which accepts an iterator of tokens. Here, iterators are implemented "pythonic way", i.e. they are classes with NEXT-ITER method (the next method in Python), that throw STOP-ITERATION condition (the StopIteration exception in Python) when there are no more values.

Thus, if you want ESRAP-LIQUID to parse something other than string or stream, just grep for how PARSE-TOKEN-ITER is used.

Change : PARSE-TOKEN-ITER now wraps the iterator into esrap caching iterator, so you don't have to do it manually. Also now ESRAP-ENV makes $(ENVNAME)-PARSE-TOKEN-ITER function accessible.

Gotchas!

-- Since Lisp streams only allow to unread one character, and ESRAP-LIQUID in general does quite a lot of look-ahead, it's not safe to use a stream somewhere else after it was fed to PARSE-STREAM (even when :junk-allowed is set to T). This is because you don't know, what state the stream ends up in.

When you define esrap environment using DEFINE-ESRAP-ENV, say, like

(define-esrap-env foo)

you get both FOO-PARSE and FOO-PARSE-STREAM -- environment versions of main parsing functions.

API Change

In versions 1.* of ESRAP-LIQUID, we used the following implicit conventions, when defining rules:

• character literals (#<something>) were understood as "try to parse this character out of the stream"
• string literals ("something") were understood as "try to parse this string out of the stream"
• free variables were understood as "try to find ESRAP rule with this name and parse it out of the stream"

These conventions are very handy when defining rules. However, the way they were implemented, is not optimal:

• character- and string-literal conventions required a use of implementation-dependent CL-READ-MACRO-TOKENS, to introduce special reader syntax (which excluded use of ESRAP-LIQUID on unsupported implementations)
• free-variable convention required use of a codewalker (HU.DWIM.WALKER was used). While acceptable for small rule sets, when using ESRAP-LIQUID for large (like VHDL) rule sets, compilation really took long time (couple of minutes) and a lot of memory (GIGABYTES!)

So now these conventions are not valid in the whole scope of DEFRULE, but only inside special macrolets. This allowed not to depend on codewalker and on special reader conventions. Now conventions are like this:

• special V macrolet (down-arrow-macrolet) is used to indicate, that here a descent into subrule is meant

  (defrule foo ()
     ...
     (v #\a) ; parse literal char #\a
     (v "asdf") ; parse literal string "asdf"
     (v sub-rule x) ; parse a sub-rule, giving it an argument X (optional)
     ...)
  

• in a lot of places V-macrolet is implicitly assumed (in all macro, defined in ESRAP-LIQUID). thus

  (times #\a) ; will be correctly understood as "parse 0 or more characters 'A'"
  

  but in CL-form PROGN (or LIST) we have to explicitly write V-macrolets

  (progn (v a) (v b) (v c)) ; parse sub-rules a, b and c
  (list (v d) (v e) f) ; parse sub-rules d and e, return variable f
  

Update: it turned out that in practice one most commonly uses V-macrolet inside PROGN, LIST and PROG1 forms. So now ESRAP-LIQUID defines PROGN-V, LIST-V and PROG1-V versions, which work exactly as their non-V analogs, but wrap their sub-forms in DESCEND-WITH-RULE, where needed.

Thus, to make a transition of your ESRAP-LIQUID-using code to 2.* API, you need:

• go over all DEFRULE's and place V-macrolets in some places
• replace C!-vars (related to capturing) with CAP, RECAP, RECAP? macros (see above)

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 esrap-liquid

License

GPL

Description

A Packrat / Parsing Grammar / TDPL parser for Common Lisp.

Version

2.3

Dependencies

• alexandria
• iterate
• cl-ppcre
• cl-interpol

Source

esrap-liquid.asd (file)

Components

• src (module)
• example-sexp.lisp (file)
• example-symbol-table.lisp (file)
• readme (file)

3 Modules

Modules are listed depth-first from the system components tree.

3.1 esrap-liquid/src

Parent

esrap-liquid (system)

Location

src/

Components

• package.lisp (file)
• conditions.lisp (file)
• miscellany.lisp (file)
• iterators.lisp (file)
• rule-storage.lisp (file)
• memoization.lisp (file)
• macro.lisp (file)
• esrap.lisp (file)
• basic-rules.lisp (file)
• esrap-env.lisp (file)
• iter-extensions.lisp (file)

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.

4.1 Lisp

4.1.1 esrap-liquid.asd

Location

esrap-liquid.asd

Systems

esrap-liquid (system)

Packages

esrap-liquid-system

4.1.2 esrap-liquid/src/package.lisp

Parent

src (module)

Location

src/package.lisp

Packages

esrap-liquid

4.1.3 esrap-liquid/src/conditions.lisp

Dependency

package.lisp (file)

Parent

src (module)

Location

src/conditions.lisp

Exported Definitions

• fail-parse (macro)
• fail-parse-format (macro)

Internal Definitions

• *debug* (special variable)
• *tracing-indent* (special variable)
• esrap-error (condition)
• esrap-error-position (method)
• esrap-error-reason (method)
• esrap-error-text (method)
• if-debug (macro)
• if-debug-fun (function)
• internal-esrap-error (condition)
• join (function)
• joinl (function)
• left-recursion (condition)
• left-recursion-nonterminal (method)
• left-recursion-path (method)
• rule-stack (method)
• (setf rule-stack) (method)
• simple-esrap-error (function)
• simple-esrap-error (condition)
• tracing-init (macro)
• tracing-level (macro)

4.1.4 esrap-liquid/src/miscellany.lisp

Dependency

conditions.lisp (file)

Parent

src (module)

Location

src/miscellany.lisp

Exported Definitions

text (function)

4.1.5 esrap-liquid/src/iterators.lisp

Dependency

miscellany.lisp (file)

Parent

src (module)

Location

src/iterators.lisp

Internal Definitions

• buffer-error (function)
• buffer-error (condition)
• buffer-pop (generic function)
• buffer-pop (method)
• buffer-push (generic function)
• buffer-push (method)
• buffer-vector (class)
• buffer-vector-start-length (special variable)
• cache-iterator (class)
• calc-new-buffer-length (function)
• clause-for-in-iter-1 (macro)
• fast-forward (function)
• hard-shrink (generic function)
• hard-shrink (method)
• hard-shrink (method)
• mk-cache-iter (function)
• mk-stream-iter (function)
• mk-string-iter (function)
• next-iter (generic function)
• next-iter (method)
• next-iter (method)
• next-iter (method)
• print-buffer-vector (function)
• print-cache-iterator (function)
• print-iter-state (function)
• rel-rewind (function)
• rewind (function)
• soft-shrink (generic function)
• soft-shrink (method)
• start-of-iter-p (generic function)
• start-of-iter-p (method)
• start-of-iter-p (method)
• stop-iteration (condition)
• stream-iter (class)
• string-iter (class)
• the-iter (special variable)
• the-length (special variable)
• the-position (special variable)
• with-saved-iter-state (macro)

4.1.6 esrap-liquid/src/rule-storage.lisp

Dependency

iterators.lisp (file)

Parent

src (module)

Location

src/rule-storage.lisp

Internal Definitions

• *rule-context-sensitivity* (special variable)
• *rules* (special variable)
• clear-rules (function)
• context-sensitive-rule-p (function)

4.1.7 esrap-liquid/src/memoization.lisp

Dependency

rule-storage.lisp (file)

Parent

src (module)

Location

src/memoization.lisp

Exported Definitions

register-context (macro)

Internal Definitions

• *cache* (special variable)
• *nonterminal-stack* (special variable)
• contexts (special variable)
• ensure-subpos-hash! (function)
• esrap-cache (class)
• failed-parse-p (function)
• (setf get-cached) (function)
• get-cached (generic function)
• get-cached (method)
• (setf get-cached) (function)
• hard-shrink (method)
• hash->assoc (function)
• make-cache (function)
• print-esrap-cache (function)
• soft-shrink (method)
• with-cached-result (macro)

4.1.8 esrap-liquid/src/macro.lisp

Dependency

memoization.lisp (file)

Parent

src (module)

Location

src/macro.lisp

Exported Definitions

• ∅ (macro)
• ! (macro)
• !! (macro)
• & (macro)
• -> (macro)
• <- (macro)
• ? (macro)
• cond-parse (macro)
• defrule (macro)
• descend-with-rule (macro)
• list-v (macro)
• maybe-wrap-in-descent (function)
• most-full-parse (macro)
• postimes (macro)
• pred (macro)
• prog1-v (macro)
• progm (macro)
• progn-v (macro)
• times (macro)

Internal Definitions

• %defrule (macro)
• *cap-stash* (special variable)
• *rule-stack* (special variable)
• def-nocontext-rule (macro)
• make-result (macro)
• make-rule-lambda (function)
• propagate-cap-stash-upwards (function)
• the-position-boundary (macro)
• with-fresh-cap-stash (macro)
• with-sub-cap-stash (macro)
• wrap-with-esrap-macrolets (function)

4.1.9 esrap-liquid/src/esrap.lisp

Dependency

macro.lisp (file)

Parent

src (module)

Location

src/esrap.lisp

Exported Definitions

• character-ranges (macro)
• parse (function)
• parse-stream (function)

Internal Definitions

• %parse-token-iter (function)
• *indentation-hint-table* (special variable)
• hint-slime-indentation (function)
• iter-last-text (function)
• max-failed-position (special variable)
• max-message (special variable)
• max-rule-stack (special variable)
• mk-esrap-iter-from-stream (function)
• mk-esrap-iter-from-string (function)
• parse-token-iter (function)
• positive-mood (special variable)
• with-tmp-rule (macro)

4.1.10 esrap-liquid/src/basic-rules.lisp

Dependency

esrap.lisp (file)

Parent

src (module)

Location

src/basic-rules.lisp

Exported Definitions

any-string (macro)

Internal Definitions

• eof-error-p (function)
• eof-p (function)

4.1.11 esrap-liquid/src/esrap-env.lisp

Dependency

basic-rules.lisp (file)

Parent

src (module)

Location

src/esrap-env.lisp

Exported Definitions

• define-esrap-env (macro)
• in-esrap-env (macro)

Internal Definitions

• install-common-rules (function)
• reintern-to-right-package (function)

4.1.12 esrap-liquid/src/iter-extensions.lisp

Dependency

esrap-env.lisp (file)

Parent

src (module)

Location

src/iter-extensions.lisp

Internal Definitions

mk-tokenizer (function)

4.2 Static

4.2.1 esrap-liquid/example-sexp.lisp

Dependency

src (module)

Parent

esrap-liquid (system)

Location

example-sexp.lisp

4.2.2 esrap-liquid/example-symbol-table.lisp

Dependency

example-sexp.lisp (file)

Parent

esrap-liquid (system)

Location

example-symbol-table.lisp

4.2.3 esrap-liquid/README

Dependency

example-symbol-table.lisp (file)

Parent

esrap-liquid (system)

Location

/home/quickref/quicklisp/dists/quicklisp/software/esrap-liquid-20161031-git/README (not found)

5 Packages

Packages are listed by definition order.

5.1 esrap-liquid-system

Source

esrap-liquid.asd

Use List

• asdf/interface
• common-lisp

5.2 esrap-liquid

Source

package.lisp (file)

Use List

• iterate
• alexandria.1.0.0
• common-lisp

Exported Definitions

• ∅ (macro)
• ! (macro)
• !! (macro)
• & (macro)
• -> (macro)
• <- (macro)
• ? (macro)
• any-string (macro)
• character-ranges (macro)
• cond-parse (macro)
• define-esrap-env (macro)
• defrule (macro)
• descend-with-rule (macro)
• fail-parse (macro)
• fail-parse-format (macro)
• in-esrap-env (macro)
• list-v (macro)
• maybe-wrap-in-descent (function)
• most-full-parse (macro)
• parse (function)
• parse-stream (function)
• postimes (macro)
• pred (macro)
• prog1-v (macro)
• progm (macro)
• progn-v (macro)
• register-context (macro)
• text (function)
• times (macro)

Internal Definitions

• %defrule (macro)
• %parse-token-iter (function)
• *cache* (special variable)
• *cap-stash* (special variable)
• *debug* (special variable)
• *indentation-hint-table* (special variable)
• *nonterminal-stack* (special variable)
• *rule-context-sensitivity* (special variable)
• *rule-stack* (special variable)
• *rules* (special variable)
• *tracing-indent* (special variable)
• buffer-error (function)
• buffer-error (condition)
• buffer-pop (generic function)
• buffer-pop (method)
• buffer-push (generic function)
• buffer-push (method)
• buffer-vector (class)
• buffer-vector-start-length (special variable)
• cache-iterator (class)
• calc-new-buffer-length (function)
• clause-for-in-iter-1 (macro)
• clear-rules (function)
• context-sensitive-rule-p (function)
• contexts (special variable)
• def-nocontext-rule (macro)
• ensure-subpos-hash! (function)
• eof-error-p (function)
• eof-p (function)
• esrap-cache (class)
• esrap-error (condition)
• esrap-error-position (generic function)
• esrap-error-position (method)
• esrap-error-reason (generic function)
• esrap-error-reason (method)
• esrap-error-text (generic function)
• esrap-error-text (method)
• failed-parse-p (function)
• fast-forward (function)
• (setf get-cached) (function)
• get-cached (generic function)
• get-cached (method)
• (setf get-cached) (function)
• hard-shrink (generic function)
• hard-shrink (method)
• hard-shrink (method)
• hard-shrink (method)
• hash->assoc (function)
• hint-slime-indentation (function)
• if-debug (macro)
• if-debug-fun (function)
• install-common-rules (function)
• internal-esrap-error (condition)
• iter-last-text (function)
• join (function)
• joinl (function)
• left-recursion (condition)
• left-recursion-nonterminal (generic function)
• left-recursion-nonterminal (method)
• left-recursion-path (generic function)
• left-recursion-path (method)
• make-cache (function)
• make-result (macro)
• make-rule-lambda (function)
• max-failed-position (special variable)
• max-message (special variable)
• max-rule-stack (special variable)
• mk-cache-iter (function)
• mk-esrap-iter-from-stream (function)
• mk-esrap-iter-from-string (function)
• mk-stream-iter (function)
• mk-string-iter (function)
• mk-tokenizer (function)
• next-iter (generic function)
• next-iter (method)
• next-iter (method)
• next-iter (method)
• parse-token-iter (function)
• positive-mood (special variable)
• print-buffer-vector (function)
• print-cache-iterator (function)
• print-esrap-cache (function)
• print-iter-state (function)
• propagate-cap-stash-upwards (function)
• reintern-to-right-package (function)
• rel-rewind (function)
• rewind (function)
• rule-stack (generic function)
• rule-stack (method)
• (setf rule-stack) (method)
• (setf rule-stack) (generic function)
• simple-esrap-error (function)
• simple-esrap-error (condition)
• soft-shrink (generic function)
• soft-shrink (method)
• soft-shrink (method)
• start-of-iter-p (generic function)
• start-of-iter-p (method)
• start-of-iter-p (method)
• stop-iteration (condition)
• stream-iter (class)
• string-iter (class)
• the-iter (special variable)
• the-length (special variable)
• the-position (special variable)
• the-position-boundary (macro)
• tracing-init (macro)
• tracing-level (macro)
• with-cached-result (macro)
• with-fresh-cap-stash (macro)
• with-saved-iter-state (macro)
• with-sub-cap-stash (macro)
• with-tmp-rule (macro)
• wrap-with-esrap-macrolets (function)

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

6.1 Exported definitions

6.1.1 Macros

Macro: ∅ &rest CLAUSES

Package

esrap-liquid

Source

macro.lisp (file)

Macro: ! EXPR

Succeeds, whenever parsing of EXPR fails. Does not consume, returns NIL, for compatibility with TEXT

Package

esrap-liquid

Source

macro.lisp (file)

Macro: !! EXPR

Succeeds, whenever parsing of EXPR fails. Consumes, assumes than EXPR parses just one character.

Package

esrap-liquid

Source

macro.lisp (file)

Macro: & SUBEXPR

Package

esrap-liquid

Source

macro.lisp (file)

Macro: -> SUBEXPR

Package

esrap-liquid

Source

macro.lisp (file)

Macro: <- SUBEXPR

Package

esrap-liquid

Source

macro.lisp (file)

Macro: ? SUBEXPR

Package

esrap-liquid

Source

macro.lisp (file)

Macro: any-string LENGTH

Package

esrap-liquid

Source

basic-rules.lisp (file)

Macro: character-ranges &rest CHAR-SPECS

Package

esrap-liquid

Source

esrap.lisp (file)

Macro: cond-parse &rest CLAUSES

Package

esrap-liquid

Source

macro.lisp (file)

Macro: define-esrap-env SYMBOL &key MAINLY-NON-CONTEXT

Package

esrap-liquid

Source

esrap-env.lisp (file)

Macro: defrule NAME ARGS &body BODY

Package

esrap-liquid

Source

macro.lisp (file)

Macro: descend-with-rule O!-SYM &rest ARGS

Package

esrap-liquid

Source

macro.lisp (file)

Macro: fail-parse &optional REASON

Package

esrap-liquid

Source

conditions.lisp (file)

Macro: fail-parse-format &optional REASON &rest ARGS

Package

esrap-liquid

Source

conditions.lisp (file)

Macro: in-esrap-env SYMBOL

Package

esrap-liquid

Source

esrap-env.lisp (file)

Macro: list-v &rest ARGS

LIST with automatic descent wrapping.

Package

esrap-liquid

Source

macro.lisp (file)

Macro: most-full-parse &rest CLAUSES

Package

esrap-liquid

Source

macro.lisp (file)

Macro: postimes SUBEXPR

Package

esrap-liquid

Source

macro.lisp (file)

Macro: pred PREDICATE SUBEXPR

Package

esrap-liquid

Source

macro.lisp (file)

Macro: prog1-v &rest FORMS

PROG1 with automatic descent wrapping.

Package

esrap-liquid

Source

macro.lisp (file)

Macro: progm START MEAT END

Prog Middle.

Package

esrap-liquid

Source

macro.lisp (file)

Macro: progn-v &rest FORMS

PROGN with automatic descent wrapping.

Package

esrap-liquid

Source

macro.lisp (file)

Macro: register-context CONTEXT-SYM

Package

esrap-liquid

Source

memoization.lisp (file)

Macro: times SUBEXPR &key FROM UPTO EXACTLY

Package

esrap-liquid

Source

macro.lisp (file)

6.1.2 Functions

Function: maybe-wrap-in-descent THING

Package

esrap-liquid

Source

macro.lisp (file)

Function: parse EXPRESSION TEXT &key START END JUNK-ALLOWED

Parses TEXT using EXPRESSION from START to END. Incomplete parses are allowed only if JUNK-ALLOWED is true.

Package

esrap-liquid

Source

esrap.lisp (file)

Function: parse-stream EXPRESSION STREAM &key JUNK-ALLOWED

Parses STREAM using EXPRESSION. Incomplete parses are allowed if JUNK-ALLOWED is true.

Package

esrap-liquid

Source

esrap.lisp (file)

Function: text &rest ARGUMENTS

Arguments must be strings, or lists whose leaves are strings. Catenates all the strings in arguments into a single string.

Package

esrap-liquid

Source

miscellany.lisp (file)

6.2 Internal definitions

6.2.1 Special variables

Special Variable: *cache*

Package

esrap-liquid

Source

memoization.lisp (file)

Special Variable: *cap-stash*

Assoc list used to capture temporary variables

Package

esrap-liquid

Source

macro.lisp (file)

Special Variable: *debug*

Package

esrap-liquid

Source

conditions.lisp (file)

Special Variable: *indentation-hint-table*

Package

esrap-liquid

Source

esrap.lisp (file)

Special Variable: *nonterminal-stack*

Package

esrap-liquid

Source

memoization.lisp (file)

Special Variable: *rule-context-sensitivity*

Package

esrap-liquid

Source

rule-storage.lisp (file)

Special Variable: *rule-stack*

Package

esrap-liquid

Source

macro.lisp (file)

Special Variable: *rules*

Package

esrap-liquid

Source

rule-storage.lisp (file)

Special Variable: *tracing-indent*

Package

esrap-liquid

Source

conditions.lisp (file)

Special Variable: buffer-vector-start-length

Package

esrap-liquid

Source

iterators.lisp (file)

Special Variable: contexts

Package

esrap-liquid

Source

memoization.lisp (file)

Special Variable: max-failed-position

Package

esrap-liquid

Source

esrap.lisp (file)

Special Variable: max-message

Package

esrap-liquid

Source

esrap.lisp (file)

Special Variable: max-rule-stack

Package

esrap-liquid

Source

esrap.lisp (file)

Special Variable: positive-mood

Package

esrap-liquid

Source

esrap.lisp (file)

Special Variable: the-iter

Package

esrap-liquid

Source

iterators.lisp (file)

Special Variable: the-length

Package

esrap-liquid

Source

iterators.lisp (file)

Special Variable: the-position

Package

esrap-liquid

Source

iterators.lisp (file)

6.2.2 Macros

Macro: %defrule NAME ARGS &body BODY

Package

esrap-liquid

Source

macro.lisp (file)

Macro: clause-for-in-iter-1 &key (FOR VAR) (IN-ITER ITER) GENERATE

Package

esrap-liquid

Source

iterators.lisp (file)

Macro: def-nocontext-rule NAME ARGS &body BODY

Package

esrap-liquid

Source

macro.lisp (file)

Macro: if-debug FORMAT-STR &rest ARGS

Package

esrap-liquid

Source

conditions.lisp (file)

Macro: make-result RESULT &optional LENGTH BEGINNING

Package

esrap-liquid

Source

macro.lisp (file)

Macro: the-position-boundary &body BODY

Package

esrap-liquid

Source

macro.lisp (file)

Macro: tracing-init &body BODY

Package

esrap-liquid

Source

conditions.lisp (file)

Macro: tracing-level &body BODY

Package

esrap-liquid

Source

conditions.lisp (file)

Macro: with-cached-result (SYMBOL &rest ARGS) &body FORMS

Package

esrap-liquid

Source

memoization.lisp (file)

Macro: with-fresh-cap-stash &body BODY

Extra level of indirection in *CAP-STASH* is needed to be able to insert new values in it

Package

esrap-liquid

Source

macro.lisp (file)

Macro: with-saved-iter-state (ITER) &body BODY

Package

esrap-liquid

Source

iterators.lisp (file)

Macro: with-sub-cap-stash &body BODY

Package

esrap-liquid

Source

macro.lisp (file)

Macro: with-tmp-rule (VAR EXPRESSION) &body BODY

Package

esrap-liquid

Source

esrap.lisp (file)

6.2.3 Functions

Function: %parse-token-iter EXPRESSION TOKEN-ITER &key JUNK-ALLOWED

Package

esrap-liquid

Source

esrap.lisp (file)

Function: buffer-error STR &rest ARGS

Package

esrap-liquid

Source

iterators.lisp (file)

Function: calc-new-buffer-length OLD-BUFFER-VECTOR START-POINTER

If we actually only use half of the buffer, shrink it in half.

Package

esrap-liquid

Source

iterators.lisp (file)

Function: clear-rules ()

Package

esrap-liquid

Source

rule-storage.lisp (file)

Function: context-sensitive-rule-p SYMBOL

Package

esrap-liquid

Source

rule-storage.lisp (file)

Function: ensure-subpos-hash! POS-HASH POS

Package

esrap-liquid

Source

memoization.lisp (file)

Function: eof-error-p ()

Package

esrap-liquid

Source

basic-rules.lisp (file)

Function: eof-p ()

Package

esrap-liquid

Source

basic-rules.lisp (file)

Function: failed-parse-p E

Package

esrap-liquid

Source

memoization.lisp (file)

Function: fast-forward CACHE-ITERATOR &optional DELTA

Relative fast-forward

Package

esrap-liquid

Source

iterators.lisp (file)

Function: (setf get-cached) RESULT SYMBOL POSITION ARGS CACHE

Package

esrap-liquid

Source

memoization.lisp (file)

Reader

get-cached (generic function)

Function: hash->assoc HASH

Package

esrap-liquid

Source

memoization.lisp (file)

Function: hint-slime-indentation ()

Package

esrap-liquid

Source

esrap.lisp (file)

Function: if-debug-fun FORMAT-STR &rest ARGS

Package

esrap-liquid

Source

conditions.lisp (file)

Function: install-common-rules HASH-TABLE

Package

esrap-liquid

Source

esrap-env.lisp (file)

Function: iter-last-text POSITION

Package

esrap-liquid

Source

esrap.lisp (file)

Function: join JOINEE &rest LST

Package

esrap-liquid

Source

conditions.lisp (file)

Function: joinl JOINEE LST

Package

esrap-liquid

Source

conditions.lisp (file)

Function: make-cache ()

Package

esrap-liquid

Source

memoization.lisp (file)

Function: make-rule-lambda NAME ARGS BODY

Package

esrap-liquid

Source

macro.lisp (file)

Function: mk-cache-iter SUB-ITER

Package

esrap-liquid

Source

iterators.lisp (file)

Function: mk-esrap-iter-from-stream STREAM

Package

esrap-liquid

Source

esrap.lisp (file)

Function: mk-esrap-iter-from-string STR START END

Package

esrap-liquid

Source

esrap.lisp (file)

Function: mk-stream-iter STREAM

Package

esrap-liquid

Source

iterators.lisp (file)

Function: mk-string-iter STRING &key START

Package

esrap-liquid

Source

iterators.lisp (file)

Function: mk-tokenizer EXPRESSION TOKEN-ITER &key JUNK-ALLOWED

Package

esrap-liquid

Source

iter-extensions.lisp (file)

Function: parse-token-iter EXPRESSION TOKEN-ITER &key JUNK-ALLOWED

Package

esrap-liquid

Source

esrap.lisp (file)

Function: print-buffer-vector VEC

Package

esrap-liquid

Source

iterators.lisp (file)

Function: print-cache-iterator ITER

Package

esrap-liquid

Source

iterators.lisp (file)

Function: print-esrap-cache CACHE

Package

esrap-liquid

Source

memoization.lisp (file)

Function: print-iter-state &optional CACHED-ITER

Package

esrap-liquid

Source

iterators.lisp (file)

Function: propagate-cap-stash-upwards UP-VAR DOWN-VAR BODY

Package

esrap-liquid

Source

macro.lisp (file)

Function: reintern-to-right-package EXPRESSION PACKAGE

Package

esrap-liquid

Source

esrap-env.lisp (file)

Function: rel-rewind CACHE-ITERATOR &optional DELTA

Relative rewind

Package

esrap-liquid

Source

iterators.lisp (file)

Function: rewind CACHE-ITERATOR &optional NEW-POS

Package

esrap-liquid

Source

iterators.lisp (file)

Function: simple-esrap-error TEXT RULE-STACK POSITION REASON

Package

esrap-liquid

Source

conditions.lisp (file)

Function: wrap-with-esrap-macrolets BODY

Package

esrap-liquid

Source

macro.lisp (file)

6.2.4 Generic functions

Generic Function: buffer-pop OBJ

Pop the last element of the buffer, but not before the start pointer

Package

esrap-liquid

Source

iterators.lisp (file)

Methods

Method: buffer-pop (OBJ buffer-vector)

Generic Function: buffer-push ELT OBJ

Place object in the end of the buffer, necessarily increasing its size

Package

esrap-liquid

Source

iterators.lisp (file)

Methods

Method: buffer-push ELT (OBJ buffer-vector)

Generic Function: esrap-error-position CONDITION

Package

esrap-liquid

Methods

Method: esrap-error-position (CONDITION esrap-error)

Source

conditions.lisp (file)

Generic Function: esrap-error-reason CONDITION

Package

esrap-liquid

Methods

Method: esrap-error-reason (CONDITION esrap-error)

Source

conditions.lisp (file)

Generic Function: esrap-error-text CONDITION

Package

esrap-liquid

Methods

Method: esrap-error-text (CONDITION esrap-error)

Source

conditions.lisp (file)

Generic Function: get-cached SYMBOL POSITION ARGS CACHE

Accessor for cached parsing results.

Package

esrap-liquid

Source

memoization.lisp (file)

Writer

(setf get-cached) (function)

Methods

Method: get-cached SYMBOL POSITION ARGS (CACHE esrap-cache)

Generic Function: hard-shrink OBJ NUM-ELTS-DISCARDED

Shrink buffer (and cache) the hard way, actually reallocating the buffer and cleaning up cache

Package

esrap-liquid

Source

iterators.lisp (file)

Methods

Method: hard-shrink (OBJ esrap-cache) NUM-ELTS-DISCARDED

Source

memoization.lisp (file)

Method: hard-shrink (OBJ cache-iterator) NUM-ELTS-DISCARDED

Method: hard-shrink (OBJ buffer-vector) (NUM-ELTS-DISCARDED integer)

Generic Function: left-recursion-nonterminal CONDITION

Package

esrap-liquid

Methods

Method: left-recursion-nonterminal (CONDITION left-recursion)

Source

conditions.lisp (file)

Generic Function: left-recursion-path CONDITION

Package

esrap-liquid

Methods

Method: left-recursion-path (CONDITION left-recursion)

Source

conditions.lisp (file)

Generic Function: next-iter ITER

Main method of iteration protocol

Package

esrap-liquid

Source

iterators.lisp (file)

Methods

Method: next-iter (ITER cache-iterator)

Method: next-iter (ITER stream-iter)

Method: next-iter (ITER string-iter)

Generic Function: rule-stack CONDITION

Generic Function: (setf rule-stack) NEW-VALUE CONDITION

Package

esrap-liquid

Methods

Method: rule-stack (CONDITION simple-esrap-error)

Method: (setf rule-stack) NEW-VALUE (CONDITION simple-esrap-error)

Source

conditions.lisp (file)

Generic Function: soft-shrink OBJ NUM-ELTS-DISCARDED

Shrink buffer (and cache) the soft way, just moving the start pointer

Package

esrap-liquid

Source

iterators.lisp (file)

Methods

Method: soft-shrink (OBJ esrap-cache) NUM-ELTS-DISCARDED

Source

memoization.lisp (file)

Method: soft-shrink (OBJ buffer-vector) (NUM-ELTS-DISCARDED integer)

Generic Function: start-of-iter-p ITER

T if the given iter is at the start. True by default.

Package

esrap-liquid

Source

iterators.lisp (file)

Methods

Method: start-of-iter-p (ITER cache-iterator)

Method: start-of-iter-p ITER

6.2.5 Conditions

Condition: buffer-error ()

Package

esrap-liquid

Source

iterators.lisp (file)

Direct superclasses

error (condition)

Direct slots

Slot: msg

Initargs

:msg

Initform

(quote nil)

Condition: esrap-error ()

Signaled when an Esrap parse fails. Use ESRAP-ERROR-TEXT to obtain the string that was being parsed, and ESRAP-ERROR-POSITION the position at which the error occurred.

Package

esrap-liquid

Source

conditions.lisp (file)

Direct superclasses

parse-error (condition)

Direct subclasses

• simple-esrap-error (condition)
• internal-esrap-error (condition)
• left-recursion (condition)

Direct methods

• esrap-error-reason (method)
• esrap-error-position (method)
• esrap-error-text (method)

Direct slots

Slot: text

Initargs

:text

Initform

(quote nil)

Readers

esrap-error-text (generic function)

Slot: position

Initargs

:position

Initform

(quote nil)

Readers

esrap-error-position (generic function)

Slot: reason

Initargs

:reason

Initform

(quote nil)

Readers

esrap-error-reason (generic function)

Condition: internal-esrap-error ()

Package

esrap-liquid

Source

conditions.lisp (file)

Direct superclasses

esrap-error (condition)

Condition: left-recursion ()

Signaled when left recursion is detected during Esrap parsing. LEFT-RECURSION-NONTERMINAL names the symbol for which left recursion was detected, and LEFT-RECURSION-PATH lists nonterminals of which the left recursion cycle consists.

Package

esrap-liquid

Source

conditions.lisp (file)

Direct superclasses

esrap-error (condition)

Direct methods

• print-object (method)
• left-recursion-path (method)
• left-recursion-nonterminal (method)

Direct slots

Slot: nonterminal

Initargs

:nonterminal

Initform

(quote nil)

Readers

left-recursion-nonterminal (generic function)

Slot: path

Initargs

:path

Initform

(quote nil)

Readers

left-recursion-path (generic function)

Condition: simple-esrap-error ()

Package

esrap-liquid

Source

conditions.lisp (file)

Direct superclasses

• simple-condition (condition)
• esrap-error (condition)

Direct methods

• print-object (method)
• rule-stack (method)
• rule-stack (method)

Direct slots

Slot: rule-stack

Initargs

:rule-stack

Readers

rule-stack (generic function)

Writers

(setf rule-stack) (generic function)

Condition: stop-iteration ()

Package

esrap-liquid

Source

iterators.lisp (file)

Direct superclasses

error (condition)

6.2.6 Classes

Class: buffer-vector ()

Package

esrap-liquid

Source

iterators.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• buffer-pop (method)
• buffer-push (method)
• hard-shrink (method)
• soft-shrink (method)
• initialize-instance (method)

Direct slots

Slot: vector

Slot: start-pointer

Initform

0

Class: cache-iterator ()

Package

esrap-liquid

Source

iterators.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• hard-shrink (method)
• start-of-iter-p (method)
• next-iter (method)
• initialize-instance (method)

Direct slots

Slot: cached-vals

Slot: cached-pos

Initform

0

Slot: sub-iter

Initargs

:sub-iter

Initform

(error "please, specify underlying iterator")

Class: esrap-cache ()

Package

esrap-liquid

Source

memoization.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• hard-shrink (method)
• soft-shrink (method)
• get-cached (method)

Direct slots

Slot: pos-hashtable

Initform

(make-hash-table :test (function equal))

Slot: start-pos

Initform

0

Class: stream-iter ()

Package

esrap-liquid

Source

iterators.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

next-iter (method)

Direct slots

Slot: stream

Initargs

:stream

Initform

(error "steam is required argument")

Class: string-iter ()

Package

esrap-liquid

Source

iterators.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

next-iter (method)

Direct slots

Slot: pos

Initargs

:start

Initform

0

Slot: str

Initargs

:string

Initform

""

Appendix A Indexes

A.1 Concepts

A.2 Functions