The cl-lex Reference Manual

Defines a function that takes a string and keyword arguments for the start and end of the string and returns a closure that takes no arguments and will return the next token each time it is called. When the input string is exhausted or no more matches are found the closure will return nil. Each pattern must be a regular expression or a list of one regular expression or a list containing a regular expression as the first element and the rest of the elements of the list forming an implicit progn. If a pattern is a regular expression or a list of one element, then text in the string that matches the regular expression is ignored. If a pattern is a list of two elements then when the regular expression matches text the sub-strings that match each register in the regular expression are bound to the symbols that represent the registers. Any text that matches named registers is bound to a variable with the same name as the register name. If the same name is used for more than one register then subsequent have the appropriate index appended to their names. If a register is not named, then text that matches it is bound to $i where i is the index of the register. The entire matching sub-string is bound to $@. If no text matches a register then its variable is bound to nil. All symbols are interned in the current package when the macro is expanded. The start and end variables passed to the function are accessible from inside the implicit progn’s from the patterns. Patterns are applied in the order they are provided and multiple patterns cannot be applied to the same piece of text. Any text that is not matched to a pattern is skipped. The behavior of the regular expressions can be modified by setting the appropriate variables in the cl-ppcre regex library.

Package

cl-lex.

Source

lex.lisp.

Returns a closure that takes no arguments and will return each token from stream when called. read-source-line is a function that takes an input stream and returns a line of the source. When EOF is encountered, read-source-line should return t. string-lexer is a function that returns a lexer that behaves as if it was returned by a function defined using define-string-lexer. opening-delimiter-p is a function that takes a character and returns true if it is an opening delimiter and false otherwise. closing-delimiter-p is a function that takes a character. When there no more tokens are left, nil is returned. nil is also returned when a newline is encountered and there are no open delimiters. If the closure is called again after EOF has been encountered a condition of type end-of-file is signalled.

Package

cl-lex.

Source

lex.lisp.

Replaces symbols in body with new symbols created by gensym. body is treated as an implicit progn.

Package

cl-lex.

Source

lex.lisp.