The cl-read-macro-tokens Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The cl-read-macro-tokens Reference Manual

This is the cl-read-macro-tokens Reference Manual, version 0.3, generated automatically by Declt version 2.3 "Robert April" on Tue Feb 20 08:15:44 2018 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

cl-read-macro-tokens

Why should things, that affect Lisp-reader, be limited to special characters (called macro-characters)? I want arbitrary symbols to be able to alter reader behavior!

This package adds possibility to define 'macro-tokens'.

Macro-tokens

Macro-token is any symbol, which happens to be in CAR-position of a list.

Example:

CL-USER> (ql:quickload 'cl-read-macro-tokens)
CL-USER> (cl-read-macro-tokens:enable-read-macro-tokens)
CL-USER> (setf (gethash 'foo cl-read-macro-tokens:*read-macro-tokens*)
               ;; Very simple custom reader, which ignores token altogether
               (lambda (stream token)
                 (cl-read-macro-tokens:read-list-old stream token)))
CL-USER> '(foo 1 2 3)
(1 2 3)
CL-USER> (cl-read-macro-tokens:disable-read-macro-tokens)
CL-USER> '(foo 1 2 3)
(FOO 1 2 3)
;; *READ-MACRO-TOKENS* is obviously preserved across enables/disables
CL-USER> (cl-read-macro-tokens:enable-read-macro-tokens)
CL-USER> '(foo 1 2 3)
(1 2 3)

The package works by redefining a function, which is associated to #( macrocharacter in the readtable. Both ENABLE-READ-MACRO-TOKENS and DISABLE-READ-MACRO-TOKENS can be given optional READTABLE argument, which defaults to *READTABLE*.

When read-macro-tokens are enabled, one can temporarily disable them, using WITH-NO-READ-MACRO-TOKENS or WITH-NO-READ-MACRO-TOKENS1 read-macro token.

CL-USER> (setf (gethash 'foo cl-read-macro-tokens:*read-macro-tokens*)
               ;; Very simple custom reader, which ignores token altogether
               (lambda (stream token)
                 (cl-read-macro-tokens:read-list-old stream token)))
CL-USER> '(foo 1 (cl-read-macro-tokens:with-no-read-macro-tokens (foo 2) 3))
(1 (progn (foo 2)) 3)
CL-USER> '(foo 1 (cl-read-macro-tokens:with-no-read-macro-tokens1 (foo 2) 3))
(1 (foo 2) 3)

The former one expands into progn-form, while the other into ordinary list-form. Hence, former is used when defining code, and latter when defining data.

Read-macros revisited

What if you had a macro, that at the same time would be able to modify the lisp-reader in some way? I.e., what if you had a macro, that is at the same time read-macro token?

DEFMACRO!! macro lets you define such objects.

CL-USER> (defmacro!! nihilling-vectors (&body body)
           (let ((*readtable* (copy-readtable))
                 (stash-function (get-dispatch-macro-character #\# #\()))
             (set-dispatch-macro-character #\# #\(
                                           (lambda (stream char subchar)
                                             (funcall stash-function stream char subchar)
                                             nil))
             (call-next-method))
           "Vectors inside the body of this macro are read in as NILs"
           `(progn ,@body))
CL-USER> (nihilling-vectors '(#(1 2 3)))
(NIL)

There is an important subtlety with read-macro-tokens. Namely, if since they affect lisp-reader, which knows nothing about macroexpansions, if I naively define a macro, whose expansions contains read-macro-tokens, things will not work as expected.

CL-USER> (defmacro mask-nihilling-vectors (&body body)
           `(nihilling-vectors ,@body))
CL-USER> (mask-nihilling-vectors '(#(1 2 3)))
'(#(1 2 3))

The reader modification behaviour is lost.

However, DEFMACRO!! takes special care to scan its body and inherit all reader-macro features of tokens, found there. So

CL-USER> (defmacro!! nomask-nihilling-vectors (&body body)
           `(nihilling-vectors ,@body))
CL-USER> (nomask-nihilling-vectors '(#(1 2 3)))
'(NIL)

If there are more than one distinct token in the macroexpansion, the new macro inherits from them all. In fact, it uses CLOS under the hood, do to all this stuff.

READ-MACROLET, WITH-MACRO-CHARACTER and WITH-DISPATCH-MACRO-CHARACTER

OK, since we now may modify reader's behaviour in more ways, let's add few more conveniencies.

READ-MACROLET lets you define read-macro-tokens, that are valid only inside body of some other read-macro-token

CL-USER> (defmacro!! my-reader-context (&body body)
           ;; Define the behaviour with respect to the reader
           (read-macrolet ((reader-context #'reader-context-handler)
                           (reader-context-2 #'reader-context-handler-2))
             (call-next-method))
           ;; Define, how read-in macro would be expanded
           `(progn ,@body)

Usually these macro-token contexts serve to temporarily change meaning of macro-characters. For this WITH-MACRO-CHARACTER and WITH-DISPATCH-MACRO-CHARACTER become handy.

;; Somewhere inside READ-MACROLET
(reader-context (lambda (stream token)
                  (with-macro-character (#\[ #'some-clever-function)
                    (with-macro-character (#\# #\. #'some-even-more-clever-function)
                      ;; So, inside READER-CONTEXTs body meaning of '[' and '#.' would be different from
                      ;; the outside meaning.
                      `(progn ,@(read-list-old stream token))))))

See NIHILLING-CHARS-AND-STRINGS in macro-tests.lisp for a full example using all these macro.

SET-MACRO-TOKEN-READER

If you want something more subtle than establishing new reader rules for all subforms of a given form, then DEFMACRO!! won't do. Or, if you want to define something with custom reader properties, which is not a macro from codewalker point of view (e.g., a function)

For this you may use SET-MACRO-TOKEN-READER (example is taken from ESRAP-LIQUID)

(flet ((the-handler (stream token)
  (let ((expression (with-esrap-reader-context
                      (read stream t nil t))))
    `(,cl-read-macro-tokens::the-token ,expression ,@(read-list-old stream token)))))
  (set-macro-token-reader parse #'the-handler))

CL-USER> (parse "a" "a")
(PARSE (descend-with-rule 'string "a") "a")

As you can see, first form of the subforms is read using special reader rules (established by ESRAP-READER-CONTEXT macro), while all the others are read as usual.

The other thing to note is THE-TOKEN symbol. It is a SYMBOL-MACRO, which expands to the name of the token the handler is used for.

Beware! Gotchas!

TODO:

All in all, in order to do ECL and CLISP, it is easier to write a separately packaged lisp-reader, e.g. based on SBCL's one and then use e.g. READER-INTERCEPT system to substitute ECL's and CLISP's C-ish readers with lispy one.


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 cl-read-macro-tokens

Author

Alexander Popolitov <popolit@gmail.com>

License

GPL

Description

Allows reader macros to be attached to tokens, not only chars

Version

0.3

Dependency

defmacro-enhance

Source

cl-read-macro-tokens.asd (file)

Components

Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 cl-read-macro-tokens.asd

Location

/home/quickbuilder/quicklisp/dists/quicklisp/software/cl-read-macro-tokens-20150923-git/cl-read-macro-tokens.asd

Systems

cl-read-macro-tokens (system)

Packages

cl-read-macro-tokens-system


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 cl-read-macro-tokens/sbcl-backend.lisp

Parent

cl-read-macro-tokens (system)

Location

sbcl-backend.lisp

Packages

cl-read-macro-tokens-patch

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.3 cl-read-macro-tokens/package.lisp

Dependency

sbcl-backend.lisp (file)

Parent

cl-read-macro-tokens (system)

Location

package.lisp

Packages

cl-read-macro-tokens


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.4 cl-read-macro-tokens/cl-read-macro-tokens.lisp

Dependency

package.lisp (file)

Parent

cl-read-macro-tokens (system)

Location

cl-read-macro-tokens.lisp

Exported Definitions
Internal Definitions

Previous: , Up: Lisp files   [Contents][Index]

3.1.5 cl-read-macro-tokens/read-macro.lisp

Dependency

cl-read-macro-tokens.lisp (file)

Parent

cl-read-macro-tokens (system)

Location

read-macro.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Next: , Previous: , Up: Packages   [Contents][Index]

4.1 cl-read-macro-tokens-system

Source

/home/quickbuilder/quicklisp/dists/quicklisp/software/cl-read-macro-tokens-20150923-git/cl-read-macro-tokens.asd

Use List

Next: , Previous: , Up: Packages   [Contents][Index]

4.2 cl-read-macro-tokens-patch

Source

sbcl-backend.lisp (file)

Use List

common-lisp

Internal Definitions

Previous: , Up: Packages   [Contents][Index]

4.3 cl-read-macro-tokens

Source

package.lisp (file)

Use List
Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Macros

Macro: define-read-macro NAME &body BODY
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Macro: defmacro!! NAME ARGS READER-INIT &body BODY
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Macro: disable-read-macro-tokens &optional READTABLE
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Macro: enable-read-macro-tokens &optional READTABLE
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Macro: read-macrolet DEFINITIONS &body BODY
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Macro: set-macro-token-reader TOKEN READER
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Macro: undefine-read-macro NAME
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Macro: with-dispatch-macro-character (CHAR SUBCHAR FUN) &body BODY
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Macro: with-macro-character (CHAR FUN) &body BODY
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Macro: with-no-read-macro-tokens &body BODY
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Macro: with-no-read-macro-tokens1 &body BODY
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Macro: with-read-macro-tokens (&optional READTABLE) &body BODY
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Special variables

Special Variable: *read-macro-tokens-classes*
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Special Variable: *read-macro-tokens-instances*
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Special Variable: the-token-instance
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Symbol macros

Symbol Macro: the-token
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Expansion

(slot-value cl-read-macro-tokens::the-token-instance (quote cl-read-macro-tokens::name))


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Functions

Function: %disable-read-macro-tokens &optional READTABLE
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Function: %enable-read-macro-tokens &optional READTABLE
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Function: %with-no-read-macro-tokens STREAM TOKEN
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Function: %with-no-read-macro-tokens1 STREAM TOKEN
Package

cl-read-macro-tokens

Source

cl-read-macro-tokens.lisp (file)

Function: new-style-fun-type-p LST
Package

cl-read-macro-tokens-patch

Source

sbcl-backend.lisp (file)

Function: random-new-class-name ()
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Function: random-string ()
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Function: read-macro-token-p SYMB
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Function: very-new-style-fun-type-p LST
Package

cl-read-macro-tokens-patch

Source

sbcl-backend.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.4 Generic functions

Generic Function: read-handler OBJ STREAM TOKEN

Main method to define behaviour of tautological macro-tokens.

Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Methods
Method: read-handler (OBJ tautological-read-macro-token) STREAM TOKEN

Previous: , Up: Internal definitions   [Contents][Index]

5.2.5 Classes

Class: tautological-read-macro-token ()
Package

cl-read-macro-tokens

Source

read-macro.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

read-handler (method)

Direct slots
Slot: name
Initargs

:name

Initform

(error "name of tautological class should be supplied")


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   C   F   L  
Index Entry  Section

C
cl-read-macro-tokens.asd: The cl-read-macro-tokens<dot>asd file
cl-read-macro-tokens/cl-read-macro-tokens.lisp: The cl-read-macro-tokens/cl-read-macro-tokens<dot>lisp file
cl-read-macro-tokens/package.lisp: The cl-read-macro-tokens/package<dot>lisp file
cl-read-macro-tokens/read-macro.lisp: The cl-read-macro-tokens/read-macro<dot>lisp file
cl-read-macro-tokens/sbcl-backend.lisp: The cl-read-macro-tokens/sbcl-backend<dot>lisp file

F
File, Lisp, cl-read-macro-tokens.asd: The cl-read-macro-tokens<dot>asd file
File, Lisp, cl-read-macro-tokens/cl-read-macro-tokens.lisp: The cl-read-macro-tokens/cl-read-macro-tokens<dot>lisp file
File, Lisp, cl-read-macro-tokens/package.lisp: The cl-read-macro-tokens/package<dot>lisp file
File, Lisp, cl-read-macro-tokens/read-macro.lisp: The cl-read-macro-tokens/read-macro<dot>lisp file
File, Lisp, cl-read-macro-tokens/sbcl-backend.lisp: The cl-read-macro-tokens/sbcl-backend<dot>lisp file

L
Lisp File, cl-read-macro-tokens.asd: The cl-read-macro-tokens<dot>asd file
Lisp File, cl-read-macro-tokens/cl-read-macro-tokens.lisp: The cl-read-macro-tokens/cl-read-macro-tokens<dot>lisp file
Lisp File, cl-read-macro-tokens/package.lisp: The cl-read-macro-tokens/package<dot>lisp file
Lisp File, cl-read-macro-tokens/read-macro.lisp: The cl-read-macro-tokens/read-macro<dot>lisp file
Lisp File, cl-read-macro-tokens/sbcl-backend.lisp: The cl-read-macro-tokens/sbcl-backend<dot>lisp file

Jump to:   C   F   L  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   %  
D   E   F   G   M   N   R   S   U   V   W  
Index Entry  Section

%
%disable-read-macro-tokens: Internal functions
%enable-read-macro-tokens: Internal functions
%with-no-read-macro-tokens: Internal functions
%with-no-read-macro-tokens1: Internal functions

D
define-read-macro: Exported macros
defmacro!!: Exported macros
disable-read-macro-tokens: Exported macros

E
enable-read-macro-tokens: Exported macros

F
Function, %disable-read-macro-tokens: Internal functions
Function, %enable-read-macro-tokens: Internal functions
Function, %with-no-read-macro-tokens: Internal functions
Function, %with-no-read-macro-tokens1: Internal functions
Function, new-style-fun-type-p: Internal functions
Function, random-new-class-name: Internal functions
Function, random-string: Internal functions
Function, read-macro-token-p: Internal functions
Function, very-new-style-fun-type-p: Internal functions

G
Generic Function, read-handler: Internal generic functions

M
Macro, define-read-macro: Exported macros
Macro, defmacro!!: Exported macros
Macro, disable-read-macro-tokens: Exported macros
Macro, enable-read-macro-tokens: Exported macros
Macro, read-macrolet: Exported macros
Macro, set-macro-token-reader: Exported macros
Macro, undefine-read-macro: Exported macros
Macro, with-dispatch-macro-character: Exported macros
Macro, with-macro-character: Exported macros
Macro, with-no-read-macro-tokens: Exported macros
Macro, with-no-read-macro-tokens1: Exported macros
Macro, with-read-macro-tokens: Exported macros
Method, read-handler: Internal generic functions

N
new-style-fun-type-p: Internal functions

R
random-new-class-name: Internal functions
random-string: Internal functions
read-handler: Internal generic functions
read-handler: Internal generic functions
read-macro-token-p: Internal functions
read-macrolet: Exported macros

S
set-macro-token-reader: Exported macros

U
undefine-read-macro: Exported macros

V
very-new-style-fun-type-p: Internal functions

W
with-dispatch-macro-character: Exported macros
with-macro-character: Exported macros
with-no-read-macro-tokens: Exported macros
with-no-read-macro-tokens1: Exported macros
with-read-macro-tokens: Exported macros

Jump to:   %  
D   E   F   G   M   N   R   S   U   V   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *  
N   S   T  
Index Entry  Section

*
*read-macro-tokens-classes*: Internal special variables
*read-macro-tokens-instances*: Internal special variables

N
name: Internal classes

S
Slot, name: Internal classes
Special Variable, *read-macro-tokens-classes*: Internal special variables
Special Variable, *read-macro-tokens-instances*: Internal special variables
Special Variable, the-token-instance: Internal special variables
Symbol Macro, the-token: Internal symbol macros

T
the-token: Internal symbol macros
the-token-instance: Internal special variables

Jump to:   *  
N   S   T  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   C   P   S   T  
Index Entry  Section

C
cl-read-macro-tokens: The cl-read-macro-tokens system
cl-read-macro-tokens: The cl-read-macro-tokens package
cl-read-macro-tokens-patch: The cl-read-macro-tokens-patch package
cl-read-macro-tokens-system: The cl-read-macro-tokens-system package
Class, tautological-read-macro-token: Internal classes

P
Package, cl-read-macro-tokens: The cl-read-macro-tokens package
Package, cl-read-macro-tokens-patch: The cl-read-macro-tokens-patch package
Package, cl-read-macro-tokens-system: The cl-read-macro-tokens-system package

S
System, cl-read-macro-tokens: The cl-read-macro-tokens system

T
tautological-read-macro-token: Internal classes

Jump to:   C   P   S   T