The cl-lex Reference Manual

This is the cl-lex Reference Manual, version 1.1.3, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Feb 26 15:23:48 2024 GMT+0.

Table of Contents


1 Introduction


2 Systems

The main system appears first, followed by any subsystem dependency.


2.1 cl-lex

Common Lisp macros for generating lexical analyzers

Author

David J. Rosenbaum <>

License

GPL3

Version

1.1.3

Dependency

cl-ppcre (system).

Source

cl-lex.asd.

Child Components

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


3.1 Lisp


3.1.1 cl-lex/cl-lex.asd

Source

cl-lex.asd.

Parent Component

cl-lex (system).

ASDF Systems

cl-lex.


3.1.2 cl-lex/packages.lisp

Source

cl-lex.asd.

Parent Component

cl-lex (system).

Packages

cl-lex.


3.1.3 cl-lex/lex.lisp

Dependency

packages.lisp (file).

Source

cl-lex.asd.

Parent Component

cl-lex (system).

Public Interface
Internals

with-gensyms (macro).


4 Packages

Packages are listed by definition order.


4.1 cl-lex

Source

packages.lisp.

Use List

common-lisp.

Public Interface
Internals

with-gensyms (macro).


5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


5.1 Public Interface


5.1.1 Macros

Macro: define-string-lexer (name &body patterns)

Defines a function that takes a string and keyword arguments for the start and end of the string and returns a closure that takes no arguments and will return the next token each time it is called. When the input string is exhausted or no more matches are found the closure will return nil. Each pattern must be a regular expression or a list of one regular expression or a list containing a regular expression as the first element and the rest of the elements of the list forming an implicit progn. If a pattern is a regular expression or a list of one element, then text in the string that matches the regular expression is ignored. If a pattern is a list of two elements then when the regular expression matches text the sub-strings that match each register in the regular expression are bound to the symbols that represent the registers. Any text that matches named registers is bound to a variable with the same name as the register name. If the same name is used for more than one register then subsequent have the appropriate index appended to their names. If a register is not named, then text that matches it is bound to $i where i is the index of the register. The entire matching sub-string is bound to $@. If no text matches a register then its variable is bound to nil. All symbols are interned in the current package when the macro is expanded. The start and end variables passed to the function are accessible from inside the implicit progn’s from the patterns. Patterns are applied in the order they are provided and multiple patterns cannot be applied to the same piece of text. Any text that is not matched to a pattern is skipped. The behavior of the regular expressions can be modified by setting the appropriate variables in the cl-ppcre regex library.

Package

cl-lex.

Source

lex.lisp.


5.1.2 Ordinary functions

Function: stream-lexer (read-source-line string-lexer opening-delimiter-p closing-delimiter-p &key stream)

Returns a closure that takes no arguments and will return each token from stream when called. read-source-line is a function that takes an input stream and returns a line of the source. When EOF is encountered, read-source-line should return t. string-lexer is a function that returns a lexer that behaves as if it was returned by a function defined using define-string-lexer. opening-delimiter-p is a function that takes a character and returns true if it is an opening delimiter and false otherwise. closing-delimiter-p is a function that takes a character. When there no more tokens are left, nil is returned. nil is also returned when a newline is encountered and there are no open delimiters. If the closure is called again after EOF has been encountered a condition of type end-of-file is signalled.

Package

cl-lex.

Source

lex.lisp.


5.2 Internals


5.2.1 Macros

Macro: with-gensyms ((&rest symbols) &body body)

Replaces symbols in body with new symbols created by gensym. body is treated as an implicit progn.

Package

cl-lex.

Source

lex.lisp.


Appendix A Indexes


A.1 Concepts


A.3 Variables