This is the graylex Reference Manual, version 0.0.1, generated automatically by Declt version 4.0 beta 2 "William Riker" on Sun Sep 15 05:23:26 2024 GMT+0.
The main system appears first, followed by any subsystem dependency.
graylexGray lexer input streams
Alexander Kahl <e-user@fsfe.org>
GPLv3+
0.0.1
cl-ppcre (system).
trivial-gray-streams (system).
alexandria (system).
src (module).
Modules are listed depth-first from the system components tree.
graylex/srcgraylex (system).
package.lisp (file).
buffered-input-stream.lisp (file).
lexer-input-stream.lisp (file).
Files are sorted by type and then listed depth-first from the systems components trees.
graylex/graylex.asdgraylex/src/package.lispgraylex/src/buffered-input-stream.lispgraylex/src/lexer-input-stream.lispgraylex/src/buffered-input-stream.lisppackage.lisp (file).
src (module).
buffered-input-buffer (reader method).
(setf buffered-input-buffer) (writer method).
buffered-input-position (reader method).
(setf buffered-input-position) (writer method).
buffered-input-size (reader method).
(setf buffered-input-size) (writer method).
buffered-input-stream (class).
buffered-stream (reader method).
close (method).
fill-buffer (generic function).
flush-buffer (generic function).
initialize-instance (method).
stream-read-char (method).
stream-read-char (method).
stream-read-sequence (method).
stream-unread-char (method).
buffered-eof-p (reader method).
(setf buffered-eof-p) (writer method).
graylex/src/lexer-input-stream.lispbuffered-input-stream.lisp (file).
src (module).
flush-buffer (method).
initialize-instance (method).
lexer-column (reader method).
(setf lexer-column) (writer method).
lexer-double-buffer (reader method).
(setf lexer-double-buffer) (writer method).
lexer-input-stream (class).
lexer-non-stream-position (reader method).
(setf lexer-non-stream-position) (writer method).
lexer-row (reader method).
(setf lexer-row) (writer method).
lexer-rules (reader method).
(setf lexer-rules) (writer method).
lexer-unread-sequence (generic function).
stream-read-char (method).
stream-read-token (generic function).
unmatched-lexing-sequence (condition).
unmatched-sequence (reader method).
unmatched-sequence-column (reader method).
unmatched-sequence-row (reader method).
eof-watchdog-filter (function).
lexing-buffer-eof (condition).
Packages are listed by definition order.
graylexcommon-lisp.
trivial-gray-streams.
buffered-input-buffer (generic reader).
(setf buffered-input-buffer) (generic writer).
buffered-input-position (generic reader).
(setf buffered-input-position) (generic writer).
buffered-input-size (generic reader).
(setf buffered-input-size) (generic writer).
buffered-input-stream (class).
buffered-stream (generic reader).
fill-buffer (generic function).
flush-buffer (generic function).
lexer-column (generic reader).
(setf lexer-column) (generic writer).
lexer-double-buffer (generic reader).
(setf lexer-double-buffer) (generic writer).
lexer-input-stream (class).
lexer-non-stream-position (generic reader).
(setf lexer-non-stream-position) (generic writer).
lexer-row (generic reader).
(setf lexer-row) (generic writer).
lexer-rules (generic reader).
(setf lexer-rules) (generic writer).
lexer-unread-sequence (generic function).
stream-read-token (generic function).
unmatched-lexing-sequence (condition).
unmatched-sequence (generic reader).
unmatched-sequence-column (generic reader).
unmatched-sequence-row (generic reader).
buffered-eof-p (generic reader).
(setf buffered-eof-p) (generic writer).
eof-watchdog-filter (function).
lexing-buffer-eof (condition).
Definitions are sorted by export status, category, package, and then by lexicographic order.
buffered-input-stream)) ¶buffered-input-stream)) ¶Wrapped input buffer
buffered-input-stream)) ¶buffered-input-stream)) ¶Reading position of the wrapped buffer
buffered-input-stream)) ¶buffered-input-stream)) ¶Size of the input buffer
buffered-input-stream)) ¶Wrapped input stream that is to be buffered
Fill the input buffer
buffered-input-stream)) ¶fill-buffer stream => position
Fill the input buffer by reading from the wrapped stream. Also reset the reading position to zero.
Flush the input buffer
lexer-input-stream)) ¶flush-buffer stream => string
Return unread rest of the wrapped main buffer but also append it to the double buffer.
buffered-input-stream)) ¶flush-buffer stream => string
Return unread rest of the wrapped buffer and replenish it.
lexer-input-stream)) ¶lexer-input-stream)) ¶Current column in lexer stream
lexer-input-stream)) ¶lexer-input-stream)) ¶Double buffer
lexer-input-stream)) ¶lexer-input-stream)) ¶Position in unread sequence
lexer-input-stream)) ¶lexer-input-stream)) ¶Current row in lexer stream
row.
lexer-input-stream)) ¶lexer-input-stream)) ¶List of regexp/keyword conses
Unread a sequence by feeding it into the double buffer
lexer-input-stream) seq) ¶lexer-unread-sequence stream seq => position
Prepend sequence SEQ to the internal double buffer and increase the non-stream position.
Read lexical tokens from the input stream
lexer-input-stream) &optional peek) ¶stream-read-token :before stream &optional peek => string
If the internal double buffer is empty, flush the main buffer first in order to replenish it.
lexer-input-stream) &optional peek) ¶stream-read-token :around stream &optional peek => (class image)
Scan the result from calling the next method if PEEK is NIL:
Discard the matched part from the beginning of the double buffer and either just
decrease the non-stream position or record the column and row progress.
lexer-input-stream) &optional peek) ¶stream-read-token stream &optional peek => (class image)
Scan the lexer’s double buffer successively with all its rules. Rules are
expected to be conses of PCRE-compatible regular expressions and class name
keywords. Heads-up: Every rule get prepended with an implicit start
anchor ("^") to match the beginning of the buffer!
If the double buffer is empty, simply return NIL; is no rule matches, signal an
UNMATCHED-LEXING-SEQUENCE with further details and provide the following
restarts:
- flush-buffer: Call the method of the same name and try to scan again
- skip-characters count: Skip COUNT characters of the reported sequence and try
to scan again
Be sure to *not* use any of these unconditionally, you’ll end up with an
infinite loop! Instead, apply UNMATCHED-SEQUENCE to the condition in your
handler to investigate and act accordingly; e.g. a hypothetical lexer rule could
require at least five characters to match but the unmatched sequence has only
three so reasonable handling code could look like this:
> (handler-bind ((unmatched-lexing-sequence #’(lambda (condition)
(if (< (length (unmatched-sequence condition)) 5)
(invoke-restart ’flush-buffer)
(error condition)))))
(function-that-invokes-stream-read-token))
unmatched-lexing-sequence)) ¶unmatched-lexing-sequence)) ¶unmatched-lexing-sequence)) ¶row.
buffered-input-stream) &key abort) ¶lexer-input-stream) &rest initargs) ¶initialize-instance :after stream &rest initargs => position
Create the double buffer after initialization.
buffered-input-stream) &rest initargs) ¶initialize-instance :after stream &rest initargs => position
Create the input buffer after initialization and fill it for the first time. Also, (re)set EOF-REACHED to NIL.
lexer-input-stream)) ¶stream-read-char stream => char or :eof
Also save read characters into the double buffer.
sb-gray.
buffered-input-stream)) ¶stream-read-char stream => char or :eof
Return next character from wrapped input buffer or :EOF if the end of the input stream is reached.
sb-gray.
buffered-input-stream)) ¶stream-read-char :before stream => position
If reading beyond the internal buffer, replenish it.
sb-gray.
buffered-input-stream) seq start end &key) ¶stream-read-sequence stream seq start end &key => position
Apply READ-SEQUENCE to given arguments and the wrapped input stream. All
mechanisms of BUFFERED-INPUT-STREAM apply.
Please see READ-SEQUENCE’s documentation.
trivial-gray-streams.
buffered-input-stream) char) ¶stream-unread-char stream char => error
Unreading chars is not supported for BUFFERED-INPUT-STREAMS.
sb-gray.
Condition signaling that no lexer rule matches.
error.
Copy of the unmatchable sequence
common-lisp.
:sequence
This slot is read-only.
Row part of unmatching position
:row
This slot is read-only.
Column part of unmatching position
:column
This slot is read-only.
Base class for buffered input streams. The default buffer size is 1024 bytes.
fundamental-character-input-stream.
trivial-gray-stream-mixin.
(setf buffered-eof-p).
buffered-eof-p.
(setf buffered-input-buffer).
buffered-input-buffer.
(setf buffered-input-position).
buffered-input-position.
(setf buffered-input-size).
buffered-input-size.
buffered-stream.
close.
fill-buffer.
flush-buffer.
initialize-instance.
stream-read-char.
stream-read-char.
stream-read-sequence.
stream-unread-char.
Wrapped input stream that is to be buffered
common-lisp.
(alexandria:required-argument :stream)
:stream
This slot is read-only.
Size of the input buffer
1024
:buffer-size
Reading position of the wrapped buffer
0
Wrapped input buffer
Whether EOF in stream has been reached
Lexer input streams provide lexical analysis, tracking of
input row and column and a dynamic second buffer for input tokens longer than
the primary BUFFERED-INPUT-STREAM buffer size.
flush-buffer.
initialize-instance.
(setf lexer-column).
lexer-column.
(setf lexer-double-buffer).
lexer-double-buffer.
(setf lexer-non-stream-position).
lexer-non-stream-position.
(setf lexer-row).
lexer-row.
(setf lexer-rules).
lexer-rules.
lexer-unread-sequence.
stream-read-char.
stream-read-token.
stream-read-token.
stream-read-token.
List of regexp/keyword conses
:rules
Current row in lexer stream
1
Current column in lexer stream
0
Position in unread sequence
0
Double buffer
eof-watchdog-filter string stream => lambda position => position
To be used as a CL-PPCRE filter; evaluates to function that
takes one argument POSITION and signals LEXING-BUFFER-EOF if end of STRING to
scan is reached and no more input is to come from STREAM.
buffered-input-stream)) ¶buffered-input-stream)) ¶Whether EOF in stream has been reached
Signal that a regex tries to scan beyond the internal buffer.
error.
| Jump to: | (
B C E F G I L M S U |
|---|
| Jump to: | (
B C E F G I L M S U |
|---|
| Jump to: | B C D E N R S |
|---|
| Jump to: | B C D E N R S |
|---|
| Jump to: | B C F G L M P S U |
|---|
| Jump to: | B C F G L M P S U |
|---|