The tagger Reference Manual

Table of Contents

The tagger Reference Manual

This is the tagger Reference Manual, generated automatically by Declt version 3.0 "Montgomery Scott" on Mon Apr 19 17:55:57 2021 GMT+0.

1 Introduction

This directory contains release 1.2 of the Xerox Part-of-Speech tagger. For more information, print the file doc/tagger/tagger.ps.

Until this project is added to the Quicklisp repository installation must be performed manually in several steps (considering that you've got Quicklisp installed already):

1. Download project sources (either by cloning the repository or downloading it in an archive).
2. Unpack them to some directory, remember it.
3. cd to the ~/quicklisp/local-projects directory.
4. Create symbolic links to the .asd files in the directory from step 2.

Now it is possible to download the application either in parts or entirely:

(ql:quickload "tagger")

When the loading is complete, you can run some simple queries:

(tag-analysis:tag-string "I saw the man on the hill with the telescope.")

I saw the man on the hill with the telescope.
ppss/2 vbd/3 at nn in at nn in/2 at nn/2

(The number following the tag is the arity of the ambiguity class assigned by the lexicon. Words without a number are unambiguous.)

Programmatic Tagging

To use the tagger in a program, create a tagging-ts and use the values of calls to the generic function next-token. Note that reinitialize-instance redirects tagging to a new text with minimal initialization overhead.

For example, the following function, my-tag-files, calls my-process-token-and-tag on each token/tag pair generated by tagging each le in the argument files:

(use-package :tdb)
(use-package :tag-analysis)
(defun my-tag-files (files)
  (let ((token-stream (make-instance 'tagging-ts)))
    (dolist (file files)
      (with-open-file (char-stream file)
	(reinitialize-instance token-stream :char-stream char-stream)
	(loop (multiple-value-bind (token tag)
		  (next-token token-stream)
		(unless token (return))
		(my-process-token-and-tag token tag)))))))

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 tagger

Author

Doug Cutting and Jan Pedersen of the Xerox Palo Alto Research Center

License

Use, reproduction, and distribution of this software is permitted, but only for non-commercial research or educational purposes. see the tagger/COPYRIGHT file for more information.

Description

The Xerox Part-of-Speech Tagger Version 1.2

Dependency

closer-mop

Source

tagger.asd (file)

Components

• package.lisp (file)
• cltl1-compat.lisp (file)
• sysdcl (module)
• util (module)
• corpus (module)
• fsa (module)
• analysis (module)

3 Modules

Modules are listed depth-first from the system components tree.

3.1 tagger/sysdcl

Dependency

cltl1-compat.lisp (file)

Parent

tagger (system)

Location

sysdcl/

Component

tdb-sysdcl-sysdcl.lisp (file)

3.2 tagger/util

Dependency

sysdcl (module)

Parent

tagger (system)

Location

util/

Components

• cl-extensions.lisp (file)
• string-resource.lisp (file)
• cons-resource.lisp (file)
• sv-resource.lisp (file)
• io-structs.lisp (file)
• io-byte8.lisp (file)
• io-builtin.lisp (file)
• svb.lisp (file)
• trie.lisp (file)
• ssb.lisp (file)
• variable-storage.lisp (file)
• vector-resource.lisp (file)
• float-vector.lisp (file)
• hmm.lisp (file)
• skip-list.lisp (file)

3.3 tagger/corpus

Dependency

util (module)

Parent

tagger (system)

Location

corpus/

Component

corpus-protocol.lisp (file)

3.4 tagger/fsa

Dependency

corpus (module)

Parent

tagger (system)

Location

fsa/

Components

• fsa-basics.lisp (file)
• standard-states.lisp (file)
• standard-symbols.lisp (file)
• list-sets.lisp (file)
• skip-list-relations.lisp (file)
• fsa-standard.lisp (file)
• fsa-calculus.lisp (file)

3.5 tagger/analysis

Dependency

fsa (module)

Parent

tagger (system)

Location

analysis/

Components

• analysis-protocol.lisp (file)
• lexicon-protocol.lisp (file)
• tag-basics.lisp (file)
• tag-analysis.lisp (file)
• class-guesser.lisp (file)
• fsa-tokenizer.lisp (file)
• vector-lexicon.lisp (file)
• tag-brown.lisp (file)
• tag-english.lisp (file)
• tag-trainer.lisp (file)

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.

4.1 Lisp

4.1.1 tagger.asd

Location

tagger.asd

Systems

tagger (system)

4.1.2 tagger/package.lisp

Parent

tagger (system)

Location

package.lisp

Packages

• standard-states
• skip-list
• list-sets
• float-vector
• tag-trainer
• vector-resource
• hmm
• binary-io
• standard-symbols
• fsa-tokenizer
• cl-extensions
• vector-lexicon
• class-guesser
• tag-basics
• variable-storage
• cons-resource
• ssb
• trie
• sv-resource
• tagger
• fsa-standard
• svb
• tag-analysis
• finite-state-automata
• string-resource
• tag-english
• tdb
• tagger.internal
• tag-brown
• skip-list-relations

4.1.3 tagger/cltl1-compat.lisp

Dependency

package.lisp (file)

Parent

tagger (system)

Location

cltl1-compat.lisp

Exported Definitions

cltl1-eval-when (macro)

Internal Definitions

defun-compile-time (macro)

4.1.4 tagger/sysdcl/tdb-sysdcl-sysdcl.lisp

Parent

sysdcl (module)

Location

sysdcl/tdb-sysdcl-sysdcl.lisp

Exported Definitions

• *highly-optimized* (special variable)
• *normally-optimized* (special variable)
• def-tdb-system (macro)
• tdb-compile (function)
• tdb-dirpath (function)
• tdb-pathname (function)

Internal Definitions

• *tdb-binary-subdirectory* (special variable)
• *tdb-systems* (special variable)

4.1.5 tagger/util/cl-extensions.lisp

Parent

util (module)

Location

util/cl-extensions.lisp

Exported Definitions

• add-process (macro)
• byte16 (type)
• byte28 (type)
• byte32 (type)
• byte7 (type)
• byte8 (type)
• define-list-sorter (macro)
• effective-function (function)
• fast-read-char (macro)
• fast-read-file-char (macro)
• format-name (function)
• get-qua (macro)
• int29 (type)
• kill-process (macro)
• make-lock (macro)
• once-only (macro)
• process-block (macro)
• this-process (macro)
• with-collection (macro)
• with-effective-function (macro)
• with-lock (macro)
• with-locks (macro)
• without-interrupts (macro)

Internal Definitions

• *the-class-t* (special variable)
• file-length (method)
• file-length (method)
• get-qua-internal (function)

4.1.6 tagger/util/string-resource.lisp

Parent

util (module)

Location

util/string-resource.lisp

Exported Definitions

• +max-string-length+ (constant)
• alloc-string (function)
• free-string (function)
• simple-string-copy (function)
• string-copy (function)

Internal Definitions

• *string-fill-pointers* (special variable)
• *string-table* (special variable)
• *string-table-lengths* (special variable)
• string-fill-pointer (macro)
• string-table-length (macro)

4.1.7 tagger/util/cons-resource.lisp

Parent

util (module)

Location

util/cons-resource.lisp

Exported Definitions

• %cons (function)
• %copy-list (function)
• %copy-tree (function)
• %delete (function)
• %list (macro)
• %make-list (function)
• %pop (macro)
• %push (macro)
• %with-collection (macro)
• free-cell (function)
• free-list (function)
• free-tree (function)

Internal Definitions

*cons-cells* (special variable)

4.1.8 tagger/util/sv-resource.lisp

Parent

util (module)

Location

util/sv-resource.lisp

Exported Definitions

• %vector (macro)
• alloc-sv (function)
• free-sv (function)
• sv-copy (function)

Internal Definitions

• *sv-fill-pointers* (special variable)
• *sv-table* (special variable)
• *sv-table-lengths* (special variable)
• +max-sv-length+ (constant)
• sv-fill-pointer (macro)
• sv-table-length (macro)

4.1.9 tagger/util/io-structs.lisp

Parent

util (module)

Location

util/io-structs.lisp

Exported Definitions

• define-io-fns (macro)
• define-io-struct (macro)
• find-io-fns (macro)
• io-copy-fn (function)
• (setf io-copy-fn) (function)
• io-fns (structure)
• io-fns-arg (function)
• io-fns-name (function)
• io-fns-p (function)
• io-free-fn (function)
• (setf io-free-fn) (function)
• io-order-fn (function)
• (setf io-order-fn) (function)
• io-read-fn (function)
• (setf io-read-fn) (function)
• io-size-fn (function)
• (setf io-size-fn) (function)
• io-write-fn (function)
• (setf io-write-fn) (function)
• make-io-fns (function)
• make-ordered-io-fns (function)
• ordered-io-fns (structure)
• ordered-io-fns-p (function)

Internal Definitions

• %print-io-fns (function)
• *fns-being-debugged* (special variable)
• *io-fns* (special variable)
• copy-io-fns (function)
• copy-ordered-io-fns (function)
• io-copy-fn-name (function)
• (setf io-copy-fn-name) (function)
• io-free-fn-name (function)
• (setf io-free-fn-name) (function)
• io-io-copy-fn (function)
• (setf io-io-copy-fn) (function)
• io-io-copy-fn-name (function)
• (setf io-io-copy-fn-name) (function)
• io-io-fns-name (function)
• io-io-free-fn (function)
• (setf io-io-free-fn) (function)
• io-io-free-fn-name (function)
• (setf io-io-free-fn-name) (function)
• io-io-read-fn (function)
• (setf io-io-read-fn) (function)
• io-io-read-fn-name (function)
• (setf io-io-read-fn-name) (function)
• io-io-size-fn (function)
• (setf io-io-size-fn) (function)
• io-io-size-fn-name (function)
• (setf io-io-size-fn-name) (function)
• io-io-write-fn (function)
• (setf io-io-write-fn) (function)
• io-io-write-fn-name (function)
• (setf io-io-write-fn-name) (function)
• io-order-fn-name (function)
• (setf io-order-fn-name) (function)
• io-read-fn-name (function)
• (setf io-read-fn-name) (function)
• io-size-fn-name (function)
• (setf io-size-fn-name) (function)
• io-write-fn-name (function)
• (setf io-write-fn-name) (function)
• make-pis (function)
• pis-accessors (function)
• (setf pis-accessors) (function)
• pis-slot-names (function)
• (setf pis-slot-names) (function)
• pis-slot-types (function)
• (setf pis-slot-types) (function)

4.1.10 tagger/util/io-byte8.lisp

Parent

util (module)

Location

util/io-byte8.lisp

Exported Definitions

• *default-byte8-stream-buffer-size* (special variable)
• byte16-read (function)
• byte16-write (function)
• byte8-read (macro)
• byte8-stream-length (macro)
• byte8-write (macro)
• close-byte8-stream (macro)
• flush-byte8-stream (macro)
• get-byte8-stream-position (macro)
• int29-read (function)
• int29-write (function)
• open-byte8-stream (macro)
• pad-byte8-stream (macro)
• read-byte32 (function)
• set-byte8-stream-position (macro)
• with-open-byte8-stream (macro)
• write-byte32 (function)

4.1.11 tagger/util/io-builtin.lisp

Parent

util (module)

Location

util/io-builtin.lisp

Exported Definitions

• byte16-order (function)
• byte32-read (function)
• byte32-write (function)
• debug-io-fns (function)
• int29-order (function)
• int29-size (function)
• integer-order (function)
• integer-read (function)
• integer-size (function)
• integer-write (function)
• simple-string-order (function)
• simple-string-size (function)
• simple-stringp-order (function)
• single-float-order (function)
• single-float-read (function)
• single-float-size (function)
• single-float-write (function)
• string-order (function)
• string-read (function)
• string-size (function)
• string-write (function)
• stringp-order (function)
• swapped-byte32 (type)
• undebug-io-fns (function)

Internal Definitions

*lower-case* (special variable)

4.1.12 tagger/util/svb.lisp

Parent

util (module)

Location

util/svb.lisp

Exported Definitions

• copy-string-to-svb (function)
• copy-svb (function)
• copy-svb-to-string (function)
• extend-svb (function)
• make-svb (function)
• svb (structure)
• svb-buffer (function)
• (setf svb-buffer) (function)
• svb-pointer (function)
• (setf svb-pointer) (function)
• svb-push-extend (macro)
• svb-size (function)
• (setf svb-size) (function)
• svbref (macro)

Internal Definitions

• %make-svb (function)
• %print-svb (function)
• *default-svb-size* (special variable)
• sv-replace (function)
• svb-p (function)

4.1.13 tagger/util/trie.lisp

Parent

util (module)

Location

util/trie.lisp

Exported Definitions

• define-trie (macro)
• make-simple-vector-trie (function)
• make-string-trie (function)
• map-st (function)
• map-svt (function)
• sort-trie (function)
• st-get (function)
• (setf st-get) (function)
• svt-get (function)
• (setf svt-get) (function)
• trie (structure)
• trie-node-key (function)
• (setf trie-node-key) (function)
• trie-node-value (function)
• (setf trie-node-value) (function)
• walk-string (function)

Internal Definitions

• %print-trie (function)
• copy-trie (function)
• copy-trie-node (function)
• make-trie (function)
• make-trie-node (function)
• sort-trie-node (function)
• trie-inverse-key-fn (function)
• (setf trie-inverse-key-fn) (function)
• trie-key-fn (function)
• (setf trie-key-fn) (function)
• trie-node (structure)
• trie-node-next (function)
• (setf trie-node-next) (function)
• trie-node-p (function)
• trie-node-subnode (function)
• (setf trie-node-subnode) (function)
• trie-p (function)
• trie-root-subnodes (function)
• (setf trie-root-subnodes) (function)
• trie-root-value (function)
• (setf trie-root-value) (function)

4.1.14 tagger/util/ssb.lisp

Parent

util (module)

Location

util/ssb.lisp

Exported Definitions

• copy-ssb (function)
• make-ssb (function)
• reinitialize-ssb (function)
• ssb-buffer (function)
• (setf ssb-buffer) (function)
• ssb-pointer (function)
• (setf ssb-pointer) (function)
• ssb-push-extend (macro)
• ssb-read-line (function)
• ssb-string (function)
• ssb-string= (function)

Internal Definitions

• %make-ssb (function)
• %print-ssb (function)
• *default-ssb-size* (special variable)
• extend-ssb (function)
• ssb-size (function)
• (setf ssb-size) (function)
• sstring-buffer (structure)
• sstring-buffer-p (function)
• sstring-replace (function)

4.1.15 tagger/util/variable-storage.lisp

Parent

util (module)

Location

util/variable-storage.lisp

Exported Definitions

• alloc-item (function)
• clear-variable-storage (function)
• free-item (function)
• make-variable-storage (function)
• print-variable-storage (function)
• with-storage-balance (macro)

Internal Definitions

• *all-buckets* (special variable)
• *allocated-items* (special variable)
• *hash-storage-p* (special variable)
• *storage-count* (special variable)
• +end-buckets+ (constant)
• +largest-bucket+ (constant)
• +significant-bit-mask+ (constant)
• +significant-bit-shift+ (constant)
• +significant-bits+ (constant)
• ash-1 (macro)
• bucket-size (function)
• copy-storage-bucket (function)
• default-cons (function)
• f+ (macro)
• fash (macro)
• fast-integer-length (function)
• fits-bucket-p (macro)
• fixnum-vector (type)
• int (macro)
• make-storage-bucket (function)
• print-storage-bucket (function)
• round-bucket (function)
• sb-buckets (function)
• (setf sb-buckets) (function)
• sb-cons-fn (function)
• (setf sb-cons-fn) (function)
• sb-name (function)
• (setf sb-name) (function)
• sb-out-count (function)
• (setf sb-out-count) (function)
• sb-out-size (function)
• (setf sb-out-size) (function)
• size-bucket (function)
• storage-bucket (structure)
• storage-bucket-p (function)

4.1.16 tagger/util/vector-resource.lisp

Parent

util (module)

Location

util/vector-resource.lisp

Exported Definitions

• adjust-byte32v (function)
• adjust-fixv (function)
• adjust-lbyte32v (function)
• adjust-lfixv (function)
• adjust-lsfv (function)
• adjust-lsv (function)
• adjust-sfv (function)
• adjust-sv (function)
• alloc-byte32v (function)
• alloc-fixv (function)
• alloc-lbyte32v (function)
• alloc-lfixv (function)
• alloc-lsfv (function)
• alloc-lsv (function)
• alloc-sfv (function)
• alloc-sv (function)
• byte32-vector (type)
• byte32v-length (macro)
• (setf byte32v-length) (setf expander)
• byte32vref (macro)
• clear-byte32v-storage (function)
• clear-fixv-storage (function)
• clear-sfv-storage (function)
• clear-sv-storage (function)
• copy-byte32v (function)
• copy-fixv (function)
• copy-sfv (function)
• copy-sv (function)
• fill-byte32v (function)
• fill-fixv (function)
• fill-lbyte32v (function)
• fill-lfixv (function)
• fill-lsfv (function)
• fill-lsv (function)
• fill-sfv (function)
• fill-sv (function)
• fixnum-vector (type)
• fixv-length (macro)
• (setf fixv-length) (setf expander)
• fixvref (macro)
• free-byte32v (function)
• free-fixv (function)
• free-fn (macro)
• (setf free-fn) (setf expander)
• free-sfv (function)
• free-sv (function)
• lert (macro)
• lert* (macro)
• let-byte32v (macro)
• let-fixv (macro)
• let-sfv (macro)
• let-sv (macro)
• make-byte32v (function)
• make-fixv (function)
• make-sfv (function)
• make-sv (function)
• resource-type (macro)
• (setf resource-type) (setf expander)
• sfv-add (function)
• sfv-div (function)
• sfv-length (macro)
• (setf sfv-length) (setf expander)
• sfv-max (function)
• sfvref (macro)
• single-float-vector (type)
• sv-length (macro)
• (setf sv-length) (setf expander)

Internal Definitions

• %alloc-byte32v (function)
• %alloc-fixv (function)
• %alloc-sfv (function)
• %alloc-sv (function)
• *byte32v-storage* (special variable)
• *fixv-storage* (special variable)
• *sfv-storage* (special variable)
• *sv-storage* (special variable)
• define-vector-resource (macro)

4.1.17 tagger/util/float-vector.lisp

Parent

util (module)

Location

util/float-vector.lisp

Exported Definitions

• sfv-read (function)
• sfv-size (function)
• sfv-write (function)

4.1.18 tagger/util/hmm.lisp

Parent

util (module)

Location

util/hmm.lisp

Exported Definitions

• hmm (structure)
• hmm-a (function)
• (setf hmm-a) (function)
• hmm-b (function)
• (setf hmm-b) (function)
• hmm-est (structure)
• hmm-m (function)
• (setf hmm-m) (function)
• hmm-maximal-path (function)
• hmm-n (function)
• (setf hmm-n) (function)
• hmm-pi (function)
• (setf hmm-pi) (function)
• hmm-read (function)
• hmm-read-file (function)
• hmm-train (function)
• hmm-train-multiple (function)
• hmm-write (function)
• hmm-write-file (function)
• make-hmm (function)
• make-hmm-est (function)
• make-sfm (function)
• sfmref (macro)

Internal Definitions

• %make-hmm-est (function)
• +negative-infinity+ (constant)
• compute-a-prime (function)
• compute-alpha (function)
• compute-b-prime (function)
• compute-beta (function)
• compute-c (function)
• compute-log-likelihood (function)
• compute-maxes-at-sym (function)
• compute-pi-prime (function)
• constrain-hmm (function)
• constrain-matrix (function)
• constrain-vector (function)
• copy-hmm (function)
• copy-hmm-est (function)
• copy-sfm (function)
• fill-sfm (function)
• finalize-path (function)
• hmm-add (function)
• hmm-div (function)
• hmm-est-log-a (function)
• (setf hmm-est-log-a) (function)
• hmm-est-log-b (function)
• (setf hmm-est-log-b) (function)
• hmm-est-log-pi (function)
• (setf hmm-est-log-pi) (function)
• hmm-est-m (function)
• (setf hmm-est-m) (function)
• hmm-est-maxes (function)
• (setf hmm-est-maxes) (function)
• hmm-est-n (function)
• (setf hmm-est-n) (function)
• hmm-est-p (function)
• hmm-est-probs (function)
• (setf hmm-est-probs) (function)
• hmm-est-resource (function)
• (setf hmm-est-resource) (function)
• hmm-p (function)
• hmm-train-step (function)
• initialize-path (function)
• log-matrix-transposed (function)
• log-vector (function)
• next-array (function)
• normalize-a-prime (function)
• normalize-alpha (function)
• return-array (function)
• sfm-add (function)
• sfm-div (function)
• sfm-read (function)
• sfm-write (function)

4.1.19 tagger/util/skip-list.lisp

Parent

util (module)

Location

util/skip-list.lisp

Exported Definitions

• clear-skip-list (function)
• do-skip-list (macro)
• free-skip-list (function)
• make-skip-list (function)
• map-skip-list (function)
• skip-list (structure)
• skip-list-copy (function)
• skip-list-difference (function)
• skip-list-empty-p (macro)
• skip-list-get (function)
• (setf skip-list-get) (setf expander)
• skip-list-intersection (function)
• skip-list-length (function)
• (setf skip-list-length) (function)
• skip-list-merge (function)
• skip-list-pop (function)
• skip-list-remove (function)
• skip-list-top (function)
• skip-list-union (function)

Internal Definitions

• %make-skip-list (function)
• %print-skip-list (function)
• *free-nodes* (special variable)
• *free-skip-lists* (special variable)
• +first-level+ (constant)
• +max-level+ (constant)
• advance-finger (function)
• cached-set-finger (function)
• coin-toss (macro)
• copy-skip-list (function)
• do-nodes (macro)
• finger-end (function)
• finger-next (function)
• finger-start (function)
• free-node (function)
• grow-header (function)
• insert-node (function)
• make-node (function)
• node-key (macro)
• node-level (macro)
• node-next (macro)
• node-value (macro)
• pop-node (function)
• random-level (function)
• remove-node (function)
• set-finger (function)
• shift-nodes (function)
• shrink-header (function)
• skip-list-delta (function)
• skip-list-destructive-intersection (function)
• skip-list-finger (function)
• (setf skip-list-finger) (function)
• skip-list-header (function)
• (setf skip-list-header) (function)
• skip-list-level (function)
• (setf skip-list-level) (function)
• skip-list-order-fn (function)
• (setf skip-list-order-fn) (function)
• skip-list-p (function)
• skip-list-put (function)
• top-node (macro)

4.1.20 tagger/corpus/corpus-protocol.lisp

Parent

corpus (module)

Location

corpus/corpus-protocol.lisp

Exported Definitions

• close-corpus (generic function)
• close-corpus (method)
• close-doc (generic function)
• close-doc (method)
• corpus (class)
• corpus-id-limit (generic function)
• corpus-id-limit (method)
• display-doc (generic function)
• display-doc (method)
• do-docs (macro)
• do-titles (macro)
• doc-title (generic function)
• doc-title (method)
• initialize-corpus (generic function)
• initialize-corpus (method)
• map-docs (generic function)
• match-doc-titles (generic function)
• match-doc-titles (method)
• open-corpus (generic function)
• open-corpus (method)
• open-doc (generic function)
• update-corpus (generic function)
• update-corpus (method)

4.1.21 tagger/fsa/fsa-basics.lisp

Parent

fsa (module)

Location

fsa/fsa-basics.lisp

Exported Definitions

• *epsilon* (special variable)
• copy-delta (generic function)
• copy-delta (method)
• copy-delta (method)
• delta-get (function)
• determinize (generic function)
• determinize (method)
• determinize (method)
• dfa (class)
• fsa (class)
• fsa-delta (method)
• (setf fsa-delta) (method)
• fsa-final-states (method)
• (setf fsa-final-states) (method)
• fsa-start-state (method)
• (setf fsa-start-state) (method)
• in-language-p (generic function)
• in-language-p (method)
• in-language-p (method)
• invert (generic function)
• invert (method)
• invert (method)
• make-dfa (generic function)
• make-nfa (generic function)
• make-ordered-set (generic function)
• make-relation (generic function)
• make-state (generic function)
• make-state-map (function)
• make-state-relation (function)
• minimize (generic function)
• minimize (method)
• nfa (class)
• ordered-set (class)
• relation (class)
• relation-empty-p (generic function)
• relation-get (generic function)
• (setf relation-get) (generic function)
• relation-map (generic function)
• set-clear (generic function)
• set-copy (generic function)
• set-empty-p (generic function)
• set-insert (generic function)
• set-intersect (generic function)
• set-length (generic function)
• set-map (generic function)
• set-member-p (generic function)
• set-minus (generic function)
• set-order-fn (generic function)
• set-union (generic function)
• state-order-fn (generic function)
• states-get (function)
• symbol-order-fn (generic function)
• symbol-order-fn (method)
• symbols-get (function)

Internal Definitions

• add-arcs (function)
• add-dests (function)
• copy-states (function)
• epsilon-closure (method)
• epsilon-closure (method)
• extended-delta (method)
• extended-delta (method)
• fsa-print (method)
• make-delta (function)
• make-fsa-delta-load-form (method)
• make-fsa-final-states-load-form (method)
• make-set-set (function)
• make-state-set (function)
• make-symbol-relation (function)
• make-symbol-set (function)
• map-state (function)
• merge-states (generic function)
• merge-states (method)

4.1.22 tagger/fsa/standard-states.lisp

Parent

fsa (module)

Location

fsa/standard-states.lisp

Exported Definitions

• fixnum-states (class)
• last-state (method)
• (setf last-state) (method)
• make-state (method)
• state-order-fn (method)

4.1.23 tagger/fsa/standard-symbols.lisp

Parent

fsa (module)

Location

fsa/standard-symbols.lisp

Exported Definitions

• char-symbols (class)
• fixnum-symbols (class)
• make-transducer-pair (function)
• standard-symbols (class)
• symbol-order-fn (method)
• symbol-order-fn (method)
• symbol-order-fn (method)
• symbol-order-fn (method)
• symbol-order-fn (method)
• symbol-symbols (class)
• tp-lower (function)
• (setf tp-lower) (function)
• tp-upper (function)
• (setf tp-upper) (function)
• transducer-pair-p (function)
• transducer-symbols (class)

Internal Definitions

• copy-transducer-pair (function)
• transducer-pair (structure)

4.1.24 tagger/fsa/list-sets.lisp

Parent

fsa (module)

Location

fsa/list-sets.lisp

Exported Definitions

• list-set (class)
• list-sets (class)
• make-ordered-set (method)
• set-clear (method)
• set-copy (method)
• set-empty-p (method)
• set-insert (method)
• set-intersect (method)
• set-length (method)
• set-map (method)
• set-member-p (method)
• set-minus (method)
• set-order-fn (method)
• set-union (method)

Internal Definitions

• list-head (method)
• (setf list-head) (method)
• list-order (method)
• (setf list-order) (method)

4.1.25 tagger/fsa/skip-list-relations.lisp

Parent

fsa (module)

Location

fsa/skip-list-relations.lisp

Exported Definitions

• make-relation (method)
• relation-empty-p (method)
• relation-get (method)
• (setf relation-get) (method)
• relation-map (method)
• skip-list-relation (class)
• skip-list-relations (class)

Internal Definitions

• relation-skip-list (method)
• (setf relation-skip-list) (method)

4.1.26 tagger/fsa/fsa-standard.lisp

Parent

fsa (module)

Location

fsa/fsa-standard.lisp

Exported Definitions

• dfa-standard (class)
• fsa-standard (class)
• make-dfa (method)
• make-nfa (method)
• nfa-standard (class)

4.1.27 tagger/fsa/fsa-calculus.lisp

Parent

fsa (module)

Location

fsa/fsa-calculus.lisp

Exported Definitions

• fsa-closure (function)
• fsa-concat (function)
• fsa-minus (function)
• fsa-optional (function)
• fsa-plus (function)
• fsa-symbol (function)
• fsa-union (function)
• regexp-to-fsa (function)

Internal Definitions

• copy-to-nfa (function)
• make-pair-order-fn (function)

4.1.28 tagger/analysis/analysis-protocol.lisp

Parent

analysis (module)

Location

analysis/analysis-protocol.lisp

Exported Definitions

• *analysis-directory* (special variable)
• analysis (class)
• char-ts (class)
• clean-text-string (function)
• do-tokens (macro)
• get-text-string (generic function)
• make-ts (generic function)
• next-token (generic function)
• paragraph-start-pos (generic function)
• (setf paragraph-start-pos) (generic function)
• print-paragraph (function)
• sentence-start-pos (generic function)
• (setf sentence-start-pos) (generic function)
• token-end-pos (generic function)
• (setf token-end-pos) (generic function)
• token-filter (class)
• token-start-pos (generic function)
• ts (class)
• ts-char-stream (method)
• (setf ts-char-stream) (method)
• ts-doc-id (method)
• (setf ts-doc-id) (method)
• ts-types (generic function)

Internal Definitions

*line-width* (special variable)

4.1.29 tagger/analysis/lexicon-protocol.lisp

Parent

analysis (module)

Location

analysis/lexicon-protocol.lisp

Exported Definitions

• *the-keyword-package* (special variable)
• *tokenizer-class* (special variable)
• class-order (function)
• lexicon (class)
• lexicon-classes (generic function)
• lexicon-classes (method)
• lexicon-classes (method)
• lexicon-filter (class)
• lexicon-lookup (generic function)
• lexicon-lookup (method)
• lexicon-lookup (method)
• lexicon-open-class (generic function)
• lexicon-open-class (method)
• lexicon-open-class (method)
• (setf lexicon-open-class) (method)
• lexicon-tags (function)
• next-token (method)
• re-read-lexicon (function)
• ts-lexicon (method)
• (setf ts-lexicon) (method)

4.1.30 tagger/analysis/tag-basics.lisp

Parent

analysis (module)

Location

analysis/tag-basics.lisp

Exported Definitions

• basic-tag-ts (class)
• check-tag-state (method)
• class->number (function)
• class-count (function)
• number->class (function)
• number->tag (function)
• tag->number (function)
• tag-count (function)
• tagger-pathname (method)
• (setf tagger-pathname) (method)
• tagger-symbol-biases (method)
• (setf tagger-symbol-biases) (method)
• tagger-transition-biases (method)
• (setf tagger-transition-biases) (method)
• write-tag-state (method)

Internal Definitions

• ts-class-trie (method)
• (setf ts-class-trie) (method)
• ts-class-vector (method)
• (setf ts-class-vector) (method)
• ts-tag-trie (method)
• (setf ts-tag-trie) (method)
• ts-tag-vector (method)
• (setf ts-tag-vector) (method)

4.1.31 tagger/analysis/tag-analysis.lisp

Parent

analysis (module)

Location

analysis/tag-analysis.lisp

Exported Definitions

• get-text-string (method)
• make-ts (method)
• next-token (method)
• next-token (method)
• paragraph-start-pos (method)
• (setf paragraph-start-pos) (method)
• sentence-start-pos (method)
• (setf sentence-start-pos) (method)
• tag-analysis (class)
• tag-file (function)
• tag-string (function)
• tagging-ts (class)
• token-class (method)
• token-end-pos (method)
• token-start-pos (method)
• tokenize-file (function)

Internal Definitions

• *free-bt-head* (special variable)
• alloc-bt (function)
• bt-class (function)
• (setf bt-class) (function)
• bt-end-pos (function)
• (setf bt-end-pos) (function)
• bt-form (function)
• (setf bt-form) (function)
• bt-start-pos (function)
• (setf bt-start-pos) (function)
• bt-tag (function)
• (setf bt-tag) (function)
• buffered-token (structure)
• buffered-token-p (function)
• copy-buffered-token (function)
• count-tags (function)
• dict-ts (method)
• (setf dict-ts) (method)
• free-bt (function)
• make-bt (function)
• print-tagging-ts (class)
• tag-next-sentence (function)
• ts-hmm-est (method)
• (setf ts-hmm-est) (method)

4.1.32 tagger/analysis/class-guesser.lisp

Parent

analysis (module)

Location

analysis/class-guesser.lisp

Exported Definitions

• lexicon-lookup (method)
• next-token (method)
• read-string-trie (function)
• suffix-lexicon (class)
• suffix-pathname (method)
• (setf suffix-pathname) (method)
• train-guesser-on-files (function)
• write-string-trie (function)

Internal Definitions

• *suffix-limit* (special variable)
• closed-class-table (method)
• (setf closed-class-table) (method)
• covering-class (function)
• dict-ts (method)
• (setf dict-ts) (method)
• do-plist (macro)
• mask-tag-table (function)
• sort-fixnum-plist (function)
• suffix-training-ts (class)
• suffix-trie (method)
• (setf suffix-trie) (method)
• suffix-trie (method)
• (setf suffix-trie) (method)
• suffix-ts-count (method)
• (setf suffix-ts-count) (method)
• truncate-trie (function)

4.1.33 tagger/analysis/fsa-tokenizer.lisp

Parent

analysis (module)

Location

analysis/fsa-tokenizer.lisp

Exported Definitions

• def-tokenizer-fsa (macro)
• fsa-tokenizer (class)
• get-text-string (method)
• make-dfa (method)
• make-nfa (method)
• make-tokenizer-fsa (function)
• next-token (method)
• print-token-stream (method)
• token-end-pos (method)
• (setf token-end-pos) (method)
• token-start-pos (method)
• tokenizer-fsa (method)
• tokenizer-fsa (class)
• ts-types (method)

Internal Definitions

• *english-tokenizer-fsa* (special variable)
• *line-length* (special variable)
• +char-code-bits+ (constant)
• adjust-stream-head (function)
• apply-tokenizer (function)
• copy-stream-head (function)
• copy-tokenizer-tables (function)
• delta-ref (macro)
• fsa-backups (method)
• (setf fsa-backups) (method)
• fsa-types (method)
• (setf fsa-types) (method)
• initialize-stream-head (function)
• make-stream-head (function)
• make-tokenizer-tables (function)
• make-tt (method)
• sh-end (function)
• (setf sh-end) (function)
• sh-pos (function)
• (setf sh-pos) (function)
• sh-read-char (function)
• sh-read-pointer (function)
• (setf sh-read-pointer) (function)
• sh-ssb (function)
• (setf sh-ssb) (function)
• sh-stream (function)
• (setf sh-stream) (function)
• stream-head (structure)
• stream-head-p (function)
• tokenizer-dfa (class)
• tokenizer-nfa (class)
• tokenizer-tables (structure)
• tokenizer-tables-p (function)
• tokenizer-tt (method)
• tt-delta (function)
• (setf tt-delta) (function)
• tt-finals (function)
• (setf tt-finals) (function)
• tt-start-state (function)
• (setf tt-start-state) (function)

4.1.34 tagger/analysis/vector-lexicon.lisp

Parent

analysis (module)

Location

analysis/vector-lexicon.lisp

Exported Definitions

• lexicon-classes (method)
• lexicon-lookup (method)
• read-vector-lexicon (function)
• vector-lexicon (class)
• write-vector-lexicon (method)

Internal Definitions

• all-stems-identical-p (function)
• binary-search (function)
• get-class-number (function)
• invert-class-numbers (function)
• make-class-numbers (function)
• make-keyword (macro)
• read-until-newline (function)
• read-until-space (function)
• vl-class-tags (method)
• (setf vl-class-tags) (method)
• vl-classes (method)
• (setf vl-classes) (method)
• vl-length (macro)
• vl-pathname (method)
• (setf vl-pathname) (method)
• vl-stems (method)
• (setf vl-stems) (method)
• vl-surface (method)
• (setf vl-surface) (method)

4.1.35 tagger/analysis/tag-brown.lisp

Parent

analysis (module)

Location

analysis/tag-brown.lisp

Exported Definitions

• brown-tokenizer (class)
• evaluate-brown (function)
• next-token (method)
• next-token (method)

Internal Definitions

• *brown-lexicon* (special variable)
• *brown-open-class* (special variable)
• *brown-tokenizer-fsa* (special variable)
• *brown-transition-biases* (special variable)
• brown-evaluation-ts (class)
• brown-lexicon (class)
• brown-truth-ts (method)
• (setf brown-truth-ts) (method)
• brown-truth-ts (class)
• do-plist (macro)
• fast-alpha-char-p (macro)
• make-tag (function)
• parse-token (function)
• ts-ambiguous (method)
• (setf ts-ambiguous) (method)
• ts-confusion-table (method)
• (setf ts-confusion-table) (method)
• ts-error-trie (method)
• (setf ts-error-trie) (method)
• ts-errors (method)
• (setf ts-errors) (method)
• ts-output-p (method)
• (setf ts-output-p) (method)
• ts-total (method)
• (setf ts-total) (method)

4.1.36 tagger/analysis/tag-english.lisp

Parent

analysis (module)

Location

analysis/tag-english.lisp

Exported Definitions

• *english-tokenizer-fsa* (special variable)
• english-tokenizer (class)

4.1.37 tagger/analysis/tag-trainer.lisp

Parent

analysis (module)

Location

analysis/tag-trainer.lisp

Exported Definitions

• next-token (method)
• train-combination (function)
• train-on-docs (function)
• train-on-files (function)
• write-tag-state (method)

Internal Definitions

• *tag-iterations* (special variable)
• *tag-quantum* (special variable)
• bias-hmm (function)
• compute-epsilon (function)
• convert-symbol-biases (function)
• convert-transition-biases (function)
• dict-ts (method)
• (setf dict-ts) (method)
• get-random-id (function)
• make-training-hmm (function)
• stream-to-tokens (function)
• tokens-to-input-seq (function)
• train-on-docs-internal (function)
• train-on-files-internal (function)
• train-on-streams (function)
• training-ts (class)
• ts-hmm (method)
• (setf ts-hmm) (method)
• valid-id-p (function)

5 Packages

Packages are listed by definition order.

5.1 standard-states

Source

package.lisp (file)

Use List

• finite-state-automata
• common-lisp

Used By List

• fsa-tokenizer
• fsa-standard

Exported Definitions

• fixnum-states (class)
• last-state (generic function)
• last-state (method)
• (setf last-state) (method)
• (setf last-state) (generic function)

5.2 skip-list

Source

package.lisp (file)

Use List

common-lisp

Used By List

• vector-lexicon
• skip-list-relations

Exported Definitions

• clear-skip-list (function)
• do-skip-list (macro)
• free-skip-list (function)
• make-skip-list (function)
• map-skip-list (function)
• skip-list (structure)
• skip-list-copy (function)
• skip-list-difference (function)
• skip-list-empty-p (macro)
• skip-list-get (function)
• (setf skip-list-get) (setf expander)
• skip-list-intersection (function)
• skip-list-length (function)
• (setf skip-list-length) (function)
• skip-list-merge (function)
• skip-list-pop (function)
• skip-list-remove (function)
• skip-list-top (function)
• skip-list-union (function)

Internal Definitions

• %make-skip-list (function)
• %print-skip-list (function)
• *free-nodes* (special variable)
• *free-skip-lists* (special variable)
• +first-level+ (constant)
• +max-level+ (constant)
• advance-finger (function)
• cached-set-finger (function)
• coin-toss (macro)
• copy-skip-list (function)
• do-nodes (macro)
• finger-end (function)
• finger-next (function)
• finger-start (function)
• free-node (function)
• grow-header (function)
• insert-node (function)
• make-node (function)
• node-key (macro)
• node-level (macro)
• node-next (macro)
• node-value (macro)
• pop-node (function)
• random-level (function)
• remove-node (function)
• set-finger (function)
• shift-nodes (function)
• shrink-header (function)
• skip-list-delta (function)
• skip-list-destructive-intersection (function)
• skip-list-finger (function)
• (setf skip-list-finger) (function)
• skip-list-header (function)
• (setf skip-list-header) (function)
• skip-list-level (function)
• (setf skip-list-level) (function)
• skip-list-order-fn (function)
• (setf skip-list-order-fn) (function)
• skip-list-p (function)
• skip-list-put (function)
• top-node (macro)

5.3 list-sets

Source

package.lisp (file)

Use List

• finite-state-automata
• common-lisp

Used By List

fsa-standard

Exported Definitions

• list-set (class)
• list-sets (class)

Internal Definitions

• list-head (generic function)
• list-head (method)
• (setf list-head) (method)
• (setf list-head) (generic function)
• list-order (generic function)
• list-order (method)
• (setf list-order) (method)
• (setf list-order) (generic function)

5.4 float-vector

Source

package.lisp (file)

Use List

• vector-resource
• variable-storage
• binary-io
• cons-resource
• common-lisp

Used By List

• tag-trainer
• hmm

Exported Definitions

• sfv-read (function)
• sfv-size (function)
• sfv-write (function)

5.5 tag-trainer

Source

package.lisp (file)

Use List

• tag-basics
• tdb
• float-vector
• hmm
• sv-resource
• string-resource
• cons-resource
• cl-extensions
• common-lisp

Exported Definitions

• train-combination (function)
• train-on-docs (function)
• train-on-files (function)

Internal Definitions

• *tag-iterations* (special variable)
• *tag-quantum* (special variable)
• bias-hmm (function)
• compute-epsilon (function)
• convert-symbol-biases (function)
• convert-transition-biases (function)
• dict-ts (generic function)
• dict-ts (method)
• (setf dict-ts) (method)
• (setf dict-ts) (generic function)
• get-random-id (function)
• make-training-hmm (function)
• stream-to-tokens (function)
• tokens-to-input-seq (function)
• train-on-docs-internal (function)
• train-on-files-internal (function)
• train-on-streams (function)
• training-ts (class)
• ts-hmm (generic function)
• ts-hmm (method)
• (setf ts-hmm) (method)
• (setf ts-hmm) (generic function)
• valid-id-p (function)

5.6 vector-resource

Source

package.lisp (file)

Use List

• tagger.internal
• variable-storage
• cl-extensions
• common-lisp

Used By List

float-vector

Exported Definitions

• adjust-byte32v (function)
• adjust-fixv (function)
• adjust-lbyte32v (function)
• adjust-lfixv (function)
• adjust-lsfv (function)
• adjust-lsv (function)
• adjust-sfv (function)
• adjust-sv (function)
• alloc-byte32v (function)
• alloc-fixv (function)
• alloc-lbyte32v (function)
• alloc-lfixv (function)
• alloc-lsfv (function)
• alloc-lsv (function)
• alloc-sfv (function)
• alloc-sv (function)
• byte32-vector (type)
• byte32v-length (macro)
• (setf byte32v-length) (setf expander)
• byte32vref (macro)
• clear-byte32v-storage (function)
• clear-fixv-storage (function)
• clear-sfv-storage (function)
• clear-sv-storage (function)
• copy-byte32v (function)
• copy-fixv (function)
• copy-sfv (function)
• copy-sv (function)
• fill-byte32v (function)
• fill-fixv (function)
• fill-lbyte32v (function)
• fill-lfixv (function)
• fill-lsfv (function)
• fill-lsv (function)
• fill-sfv (function)
• fill-sv (function)
• fixnum-vector (type)
• fixv-length (macro)
• (setf fixv-length) (setf expander)
• fixvref (macro)
• free-byte32v (function)
• free-fixv (function)
• free-fn (macro)
• (setf free-fn) (setf expander)
• free-sfv (function)
• free-sv (function)
• lert (macro)
• lert* (macro)
• let-byte32v (macro)
• let-fixv (macro)
• let-sfv (macro)
• let-sv (macro)
• make-byte32v (function)
• make-fixv (function)
• make-sfv (function)
• make-sv (function)
• resource-type (macro)
• (setf resource-type) (setf expander)
• sfv-add (function)
• sfv-div (function)
• sfv-length (macro)
• (setf sfv-length) (setf expander)
• sfv-max (function)
• sfvref (macro)
• single-float-vector (type)
• sv-length (macro)
• (setf sv-length) (setf expander)

Internal Definitions

• %alloc-byte32v (function)
• %alloc-fixv (function)
• %alloc-sfv (function)
• %alloc-sv (function)
• *byte32v-storage* (special variable)
• *fixv-storage* (special variable)
• *sfv-storage* (special variable)
• *sv-storage* (special variable)
• define-vector-resource (macro)

5.7 hmm

Source

package.lisp (file)

Use List

• binary-io
• float-vector
• cons-resource
• cl-extensions
• common-lisp

Used By List

• tag-trainer
• tag-analysis

Exported Definitions

• hmm (structure)
• hmm-a (function)
• (setf hmm-a) (function)
• hmm-b (function)
• (setf hmm-b) (function)
• hmm-est (structure)
• hmm-m (function)
• (setf hmm-m) (function)
• hmm-maximal-path (function)
• hmm-n (function)
• (setf hmm-n) (function)
• hmm-pi (function)
• (setf hmm-pi) (function)
• hmm-read (function)
• hmm-read-file (function)
• hmm-train (function)
• hmm-train-multiple (function)
• hmm-write (function)
• hmm-write-file (function)
• make-hmm (function)
• make-hmm-est (function)
• make-sfm (function)
• sfmref (macro)

Internal Definitions

• %make-hmm-est (function)
• +negative-infinity+ (constant)
• compute-a-prime (function)
• compute-alpha (function)
• compute-b-prime (function)
• compute-beta (function)
• compute-c (function)
• compute-log-likelihood (function)
• compute-maxes-at-sym (function)
• compute-pi-prime (function)
• constrain-hmm (function)
• constrain-matrix (function)
• constrain-vector (function)
• copy-hmm (function)
• copy-hmm-est (function)
• copy-sfm (function)
• fill-sfm (function)
• finalize-path (function)
• hmm-add (function)
• hmm-div (function)
• hmm-est-log-a (function)
• (setf hmm-est-log-a) (function)
• hmm-est-log-b (function)
• (setf hmm-est-log-b) (function)
• hmm-est-log-pi (function)
• (setf hmm-est-log-pi) (function)
• hmm-est-m (function)
• (setf hmm-est-m) (function)
• hmm-est-maxes (function)
• (setf hmm-est-maxes) (function)
• hmm-est-n (function)
• (setf hmm-est-n) (function)
• hmm-est-p (function)
• hmm-est-probs (function)
• (setf hmm-est-probs) (function)
• hmm-est-resource (function)
• (setf hmm-est-resource) (function)
• hmm-p (function)
• hmm-train-step (function)
• initialize-path (function)
• log-matrix-transposed (function)
• log-vector (function)
• next-array (function)
• normalize-a-prime (function)
• normalize-alpha (function)
• return-array (function)
• sfm-add (function)
• sfm-div (function)
• sfm-read (function)
• sfm-write (function)

5.8 binary-io

Source

package.lisp (file)

Use List

• tagger.internal
• cons-resource
• string-resource
• cl-extensions
• common-lisp

Used By List

• vector-lexicon
• hmm
• float-vector

Exported Definitions

• *default-byte8-stream-buffer-size* (special variable)
• byte16-order (function)
• byte16-read (function)
• byte16-write (function)
• byte32-read (function)
• byte32-write (function)
• byte8-read (macro)
• byte8-stream-length (macro)
• byte8-write (macro)
• close-byte8-stream (macro)
• debug-io-fns (function)
• define-io-fns (macro)
• define-io-struct (macro)
• find-io-fns (macro)
• flush-byte8-stream (macro)
• get-byte8-stream-position (macro)
• int29-order (function)
• int29-read (function)
• int29-size (function)
• int29-write (function)
• integer-order (function)
• integer-read (function)
• integer-size (function)
• integer-write (function)
• io-copy-fn (function)
• (setf io-copy-fn) (function)
• io-fns (structure)
• io-fns-arg (function)
• io-fns-name (function)
• io-fns-p (function)
• io-free-fn (function)
• (setf io-free-fn) (function)
• io-order-fn (function)
• (setf io-order-fn) (function)
• io-read-fn (function)
• (setf io-read-fn) (function)
• io-size-fn (function)
• (setf io-size-fn) (function)
• io-write-fn (function)
• (setf io-write-fn) (function)
• make-io-fns (function)
• make-ordered-io-fns (function)
• open-byte8-stream (macro)
• ordered-io-fns (structure)
• ordered-io-fns-p (function)
• pad-byte8-stream (macro)
• read-byte32 (function)
• set-byte8-stream-position (macro)
• simple-string-order (function)
• simple-string-size (function)
• simple-stringp-order (function)
• single-float-order (function)
• single-float-read (function)
• single-float-size (function)
• single-float-write (function)
• string-order (function)
• string-read (function)
• string-size (function)
• string-write (function)
• stringp-order (function)
• swapped-byte32 (type)
• undebug-io-fns (function)
• with-open-byte8-stream (macro)
• write-byte32 (function)

Internal Definitions

• %print-io-fns (function)
• *fns-being-debugged* (special variable)
• *io-fns* (special variable)
• *lower-case* (special variable)
• copy-io-fns (function)
• copy-ordered-io-fns (function)
• copy-parsed-io-struct (function)
• io-copy-fn-name (function)
• (setf io-copy-fn-name) (function)
• io-free-fn-name (function)
• (setf io-free-fn-name) (function)
• io-io-copy-fn (function)
• (setf io-io-copy-fn) (function)
• io-io-copy-fn-name (function)
• (setf io-io-copy-fn-name) (function)
• io-io-fns-name (function)
• io-io-free-fn (function)
• (setf io-io-free-fn) (function)
• io-io-free-fn-name (function)
• (setf io-io-free-fn-name) (function)
• io-io-read-fn (function)
• (setf io-io-read-fn) (function)
• io-io-read-fn-name (function)
• (setf io-io-read-fn-name) (function)
• io-io-size-fn (function)
• (setf io-io-size-fn) (function)
• io-io-size-fn-name (function)
• (setf io-io-size-fn-name) (function)
• io-io-write-fn (function)
• (setf io-io-write-fn) (function)
• io-io-write-fn-name (function)
• (setf io-io-write-fn-name) (function)
• io-order-fn-name (function)
• (setf io-order-fn-name) (function)
• io-read-fn-name (function)
• (setf io-read-fn-name) (function)
• io-size-fn-name (function)
• (setf io-size-fn-name) (function)
• io-write-fn-name (function)
• (setf io-write-fn-name) (function)
• make-pis (function)
• pis-accessors (function)
• (setf pis-accessors) (function)
• pis-slot-names (function)
• (setf pis-slot-names) (function)
• pis-slot-types (function)
• (setf pis-slot-types) (function)

5.9 standard-symbols

Source

package.lisp (file)

Use List

• finite-state-automata
• common-lisp

Used By List

fsa-standard

Exported Definitions

• char-symbols (class)
• fixnum-symbols (class)
• make-transducer-pair (function)
• standard-symbols (class)
• symbol-symbols (class)
• tp-lower (function)
• (setf tp-lower) (function)
• tp-upper (function)
• (setf tp-upper) (function)
• transducer-pair-p (function)
• transducer-symbols (class)

Internal Definitions

• copy-transducer-pair (function)
• transducer-pair (structure)

5.10 fsa-tokenizer

Source

package.lisp (file)

Nickname

fsa-t

Use List

• standard-states
• fsa-standard
• finite-state-automata
• tdb
• string-resource
• ssb
• cl-extensions
• common-lisp

Used By List

• tag-english
• tag-brown
• tag-analysis
• tag-basics

Exported Definitions

• def-tokenizer-fsa (macro)
• fsa-tokenizer (class)
• make-tokenizer-fsa (function)
• print-token-stream (generic function)
• print-token-stream (method)
• tokenizer-fsa (generic function)
• tokenizer-fsa (method)
• tokenizer-fsa (class)

Internal Definitions

• *english-tokenizer-fsa* (special variable)
• *line-length* (special variable)
• +char-code-bits+ (constant)
• adjust-stream-head (function)
• apply-tokenizer (function)
• copy-stream-head (function)
• copy-tokenizer-tables (function)
• delta-ref (macro)
• fsa-backups (generic function)
• fsa-backups (method)
• (setf fsa-backups) (method)
• (setf fsa-backups) (generic function)
• fsa-types (generic function)
• fsa-types (method)
• (setf fsa-types) (method)
• (setf fsa-types) (generic function)
• initialize-stream-head (function)
• make-stream-head (function)
• make-tokenizer-tables (function)
• make-tt (generic function)
• make-tt (method)
• sh-end (function)
• (setf sh-end) (function)
• sh-pos (function)
• (setf sh-pos) (function)
• sh-read-char (function)
• sh-read-pointer (function)
• (setf sh-read-pointer) (function)
• sh-ssb (function)
• (setf sh-ssb) (function)
• sh-stream (function)
• (setf sh-stream) (function)
• stream-head (structure)
• stream-head-p (function)
• tokenizer-dfa (class)
• tokenizer-nfa (class)
• tokenizer-tables (structure)
• tokenizer-tables-p (function)
• tokenizer-tt (generic function)
• tokenizer-tt (method)
• tt-delta (function)
• (setf tt-delta) (function)
• tt-finals (function)
• (setf tt-finals) (function)
• tt-start-state (function)
• (setf tt-start-state) (function)

5.11 cl-extensions

Source

package.lisp (file)

Use List

• tagger.internal
• common-lisp

Used By List

• tag-trainer
• tag-english
• tag-brown
• vector-lexicon
• class-guesser
• tag-analysis
• hmm
• vector-resource
• binary-io
• cons-resource
• tag-basics
• trie
• svb
• fsa-tokenizer
• tdb
• string-resource
• ssb

Exported Definitions

• add-process (macro)
• byte16 (type)
• byte28 (type)
• byte32 (type)
• byte7 (type)
• byte8 (type)
• define-list-sorter (macro)
• effective-function (function)
• fast-read-char (macro)
• fast-read-file-char (macro)
• format-name (function)
• get-qua (macro)
• int29 (type)
• kill-process (macro)
• make-lock (macro)
• once-only (macro)
• process-block (macro)
• this-process (macro)
• with-collection (macro)
• with-effective-function (macro)
• with-lock (macro)
• with-locks (macro)
• without-interrupts (macro)

Internal Definitions

• *the-class-t* (special variable)
• file-length (generic function)
• file-length (method)
• file-length (method)
• get-qua-internal (function)
• string-input-stream-limit (function)

5.12 vector-lexicon

Source

package.lisp (file)

Use List

• tdb
• trie
• binary-io
• skip-list
• ssb
• svb
• sv-resource
• cons-resource
• string-resource
• cl-extensions
• common-lisp

Used By List

• tag-english
• tag-brown

Exported Definitions

• read-vector-lexicon (function)
• vector-lexicon (class)
• write-vector-lexicon (generic function)
• write-vector-lexicon (method)

Internal Definitions

• all-stems-identical-p (function)
• binary-search (function)
• get-class-number (function)
• invert-class-numbers (function)
• make-class-numbers (function)
• make-keyword (macro)
• read-until-newline (function)
• read-until-space (function)
• vl-class-tags (generic function)
• vl-class-tags (method)
• (setf vl-class-tags) (method)
• (setf vl-class-tags) (generic function)
• vl-classes (generic function)
• vl-classes (method)
• (setf vl-classes) (method)
• (setf vl-classes) (generic function)
• vl-length (macro)
• vl-pathname (generic function)
• vl-pathname (method)
• (setf vl-pathname) (method)
• (setf vl-pathname) (generic function)
• vl-stems (generic function)
• vl-stems (method)
• (setf vl-stems) (method)
• (setf vl-stems) (generic function)
• vl-surface (generic function)
• vl-surface (method)
• (setf vl-surface) (method)
• (setf vl-surface) (generic function)

5.13 class-guesser

Source

package.lisp (file)

Use List

• tag-basics
• tdb
• trie
• sv-resource
• string-resource
• cons-resource
• cl-extensions
• common-lisp

Used By List

• tag-english
• tag-brown

Exported Definitions

• read-string-trie (function)
• suffix-lexicon (class)
• suffix-pathname (generic function)
• suffix-pathname (method)
• (setf suffix-pathname) (method)
• (setf suffix-pathname) (generic function)
• train-guesser-on-files (function)
• write-string-trie (function)

Internal Definitions

• *suffix-limit* (special variable)
• closed-class-table (generic function)
• closed-class-table (method)
• (setf closed-class-table) (method)
• (setf closed-class-table) (generic function)
• covering-class (function)
• dict-ts (generic function)
• dict-ts (method)
• (setf dict-ts) (method)
• (setf dict-ts) (generic function)
• do-plist (macro)
• mask-tag-table (function)
• sort-fixnum-plist (function)
• suffix-training-ts (class)
• suffix-trie (generic function)
• suffix-trie (method)
• (setf suffix-trie) (method)
• suffix-trie (method)
• (setf suffix-trie) (method)
• (setf suffix-trie) (generic function)
• suffix-ts-count (generic function)
• suffix-ts-count (method)
• (setf suffix-ts-count) (method)
• (setf suffix-ts-count) (generic function)
• truncate-trie (function)

5.14 tag-basics

Source

package.lisp (file)

Use List

• fsa-tokenizer
• tdb
• trie
• cl-extensions
• common-lisp

Used By List

• tag-trainer
• tag-english
• tag-brown
• class-guesser
• tag-analysis

Exported Definitions

• basic-tag-ts (class)
• check-tag-state (generic function)
• check-tag-state (method)
• class->number (function)
• class-count (function)
• number->class (function)
• number->tag (function)
• tag->number (function)
• tag-count (function)
• tagger-pathname (generic function)
• tagger-pathname (method)
• (setf tagger-pathname) (method)
• (setf tagger-pathname) (generic function)
• tagger-symbol-biases (generic function)
• tagger-symbol-biases (method)
• (setf tagger-symbol-biases) (method)
• (setf tagger-symbol-biases) (generic function)
• tagger-transition-biases (generic function)
• tagger-transition-biases (method)
• (setf tagger-transition-biases) (method)
• (setf tagger-transition-biases) (generic function)
• write-tag-state (generic function)
• write-tag-state (method)
• write-tag-state (method)

Internal Definitions

• ts-class-trie (generic function)
• ts-class-trie (method)
• (setf ts-class-trie) (method)
• (setf ts-class-trie) (generic function)
• ts-class-vector (generic function)
• ts-class-vector (method)
• (setf ts-class-vector) (method)
• (setf ts-class-vector) (generic function)
• ts-tag-trie (generic function)
• ts-tag-trie (method)
• (setf ts-tag-trie) (method)
• (setf ts-tag-trie) (generic function)
• ts-tag-vector (generic function)
• ts-tag-vector (method)
• (setf ts-tag-vector) (method)
• (setf ts-tag-vector) (generic function)

5.15 variable-storage

Source

package.lisp (file)

Use List

• tagger.internal
• cons-resource
• common-lisp

Used By List

• float-vector
• vector-resource

Exported Definitions

• alloc-item (function)
• clear-variable-storage (function)
• free-item (function)
• make-variable-storage (function)
• print-variable-storage (function)
• with-storage-balance (macro)

Internal Definitions

• *all-buckets* (special variable)
• *allocated-items* (special variable)
• *hash-storage-p* (special variable)
• *storage-count* (special variable)
• +end-buckets+ (constant)
• +largest-bucket+ (constant)
• +significant-bit-mask+ (constant)
• +significant-bit-shift+ (constant)
• +significant-bits+ (constant)
• ash-1 (macro)
• bucket-size (function)
• copy-storage-bucket (function)
• default-cons (function)
• f+ (macro)
• fash (macro)
• fast-integer-length (function)
• fits-bucket-p (macro)
• fixnum-vector (type)
• int (macro)
• make-storage-bucket (function)
• print-storage-bucket (function)
• round-bucket (function)
• sb-buckets (function)
• (setf sb-buckets) (function)
• sb-cons-fn (function)
• (setf sb-cons-fn) (function)
• sb-name (function)
• (setf sb-name) (function)
• sb-out-count (function)
• (setf sb-out-count) (function)
• sb-out-size (function)
• (setf sb-out-size) (function)
• size-bucket (function)
• storage-bucket (structure)
• storage-bucket-p (function)