The langutils Reference Manual

This is the langutils Reference Manual, version 1.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon May 15 04:02:04 2023 GMT+0.

Table of Contents


1 Introduction


2 Systems

The main system appears first, followed by any subsystem dependency.


2.1 langutils

Language utilities

Author

Ian Eslick

License

BSD

Version

1.0

Dependencies
  • s-xml-rpc (system).
  • stdutils (system).
Source

langutils.asd.

Child Component

src (module).


3 Modules

Modules are listed depth-first from the system components tree.


3.1 langutils/src

Source

langutils.asd.

Parent Component

langutils (system).

Child Components

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.


4.1 Lisp


4.1.1 langutils/langutils.asd

Source

langutils.asd.

Parent Component

langutils (system).

ASDF Systems

langutils.

Packages

langutils.system.


4.1.2 langutils/src/package.lisp

Source

langutils.asd.

Parent Component

src (module).

Packages

4.1.3 langutils/src/config.lisp

Dependency

package.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Internals

4.1.4 langutils/src/tokens.lisp

Dependency

config.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.5 langutils/src/reference.lisp

Dependency

tokens.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.6 langutils/src/stopwords.lisp

Dependency

reference.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.7 langutils/src/my-meta.lisp

Dependency

stopwords.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.8 langutils/src/tokenize.lisp

Dependency

my-meta.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.9 langutils/src/lexicon.lisp

Dependency

tokenize.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.10 langutils/src/lemma.lisp

Dependency

lexicon.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.11 langutils/src/porter.lisp

Dependency

lemma.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Internals

4.1.12 langutils/src/contextual-rule-parser.lisp

Dependency

porter.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Internals

4.1.13 langutils/src/tagger-data.lisp

Dependency

contextual-rule-parser.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Internals

4.1.14 langutils/src/tagger.lisp

Dependency

tagger-data.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.15 langutils/src/chunker-constants.lisp

Dependency

tagger.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Internals

4.1.16 langutils/src/chunker.lisp

Dependency

chunker-constants.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.17 langutils/src/concept.lisp

Dependency

chunker.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.18 langutils/src/init.lisp

Dependency

concept.lisp (file).

Source

langutils.asd.

Parent Component

src (module).

Public Interface

5 Packages

Packages are listed by definition order.


5.1 langutils.system

Source

langutils.asd.

Use List
  • asdf/interface.
  • common-lisp.

5.2 langutils

Source

package.lisp.

Use List
  • common-lisp.
  • stdutils.
Public Interface
Internals

5.3 langutils-tokenize

Source

package.lisp.

Use List
Public Interface
Internals

5.4 my-meta

Source

package.lisp.

Use List

common-lisp.

Used By List

langutils-tokenize.

Public Interface
Internals

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


6.1 Public Interface


6.1.1 Macros

Macro: with-list-meta ((source-symbol list) &body body)
Package

my-meta.

Source

my-meta.lisp.

Macro: with-stream-meta ((source-symbol stream) &body body)
Package

my-meta.

Source

my-meta.lisp.

Macro: with-string-meta ((source-symbol string-buffer &key start end) &body body)
Package

my-meta.

Source

my-meta.lisp.


6.1.2 Setf expanders

Setf Expander: (setf get-lexicon-entry) (word)
Package

langutils.

Source

lexicon.lisp.

Reader

get-lexicon-entry (function).

Writer

set-lexicon-entry (function).


6.1.3 Ordinary functions

Function: associate-concepts (phrases)

Return the list of phrase/list/token-arrays as pairs with the first element being the original and the second being a canonicalized concept instance

Package

langutils.

Source

concept.lisp.

Function: chunk (text)

Returns a phrase-list for the provided text

Package

langutils.

Source

chunker.lisp.

Function: chunk-tokenized (text)

Returns a phrase-list for the provided tokenized string

Package

langutils.

Source

chunker.lisp.

Function: clean-langutils ()
Package

langutils.

Source

init.lisp.

Function: clean-tagger ()
Package

langutils.

Source

tagger.lisp.

Function: concise-stopword? (id)

Identifies id as a ’concise-stopword’ word.
concise-stopwords are a *very* small list of words. Mainly pronouns and determiners

Package

langutils.

Source

stopwords.lisp.

Function: contains-is? (ids)

Tests list of ids for ’is’ words

Package

langutils.

Source

stopwords.lisp.

Function: disable-meta-syntax ()
Package

my-meta.

Source

my-meta.lisp.

Function: enable-meta-syntax ()
Package

my-meta.

Source

my-meta.lisp.

Function: force-concept (c)
Package

langutils.

Source

concept.lisp.

Function: get-lemma (word &key pos noun porter)

Provides the root word string for the provided word string

Package

langutils.

Source

lemma.lisp.

Function: get-lemma-for-id (id &key pos noun porter)

Returns a lemma id for the provided word id. pos only returns the root for the provided pos type. noun will stem nouns to the singular form by default and porter determines whether the porter algorithm is used for unknown terms. pos type causes the noun argument to be ignored

Package

langutils.

Source

lemma.lisp.

Function: get-lexicon-case-forms (word)
Package

langutils.

Source

lexicon.lisp.

Function: get-lexicon-default-pos (word)
Package

langutils.

Source

lexicon.lisp.

Function: get-lexicon-entry (word)
Package

langutils.

Source

lexicon.lisp.

Setf expander for this function

(setf get-lexicon-entry).

Function: get-token-count ()

Return the current token counter

Package

langutils.

Source

tokens.lisp.

Function: head-verb (phrase &key filter-common)
Package

langutils.

Source

chunker.lisp.

Function: head-verbs (phrases &key filter-common)
Package

langutils.

Source

chunker.lisp.

Function: id-for-token (token &optional trim)

This takes string ’tokens’ and returns a unique id for that character sequence - beware of whitespace, etc.

Package

langutils.

Source

tokens.lisp.

Function: ids-for-tokens (tokens)
Package

langutils.

Source

tokens.lisp.

Function: in-pos-class? (element class)
Package

langutils.

Source

lemma.lisp.

Function: init-langutils ()
Package

langutils.

Source

init.lisp.

Function: init-tagger (&optional lexical-rule-file contextual-rule-file)
Package

langutils.

Source

tagger.lisp.

Function: initial-tag (token)

Return an initial tag for a given token string using the langutils lexicon and the tagger lexical rules (via guess-tag)

Package

langutils.

Source

tagger.lisp.

Reader: lexicon-entry-id (instance)
Writer: (setf lexicon-entry-id) (instance)
Package

langutils.

Source

lexicon.lisp.

Target Slot

id.

Reader: lexicon-entry-roots (instance)
Writer: (setf lexicon-entry-roots) (instance)
Package

langutils.

Source

lexicon.lisp.

Target Slot

roots.

Reader: lexicon-entry-surface-forms (instance)
Writer: (setf lexicon-entry-surface-forms) (instance)
Package

langutils.

Source

lexicon.lisp.

Target Slot

surface-forms.

Function: lexicon-entry-tag (entry)
Package

langutils.

Source

lexicon.lisp.

Reader: lexicon-entry-tags (instance)
Writer: (setf lexicon-entry-tags) (instance)
Package

langutils.

Source

lexicon.lisp.

Target Slot

tags.

Function: make-concept (ta)
Package

langutils.

Source

concept.lisp.

Function: make-phrase (text-array tag-array &optional type)

Take two arrays of test and tags and create a phrase that points at a vdoc created from the two arrays

Package

langutils.

Source

reference.lisp.

Function: make-phrase-from-sentence (tok-string &optional tag-array)
Package

langutils.

Source

reference.lisp.

Function: make-phrase-from-vdoc (doc start len &optional type)
Package

langutils.

Source

reference.lisp.

Function: make-vector-document (text &optional tags)
Package

langutils.

Source

reference.lisp.

Function: morph-case-surface-forms (root &optional pos-class)

All cases of morphological surface forms of the provided root

Package

langutils.

Source

lemma.lisp.

Function: morph-surface-forms (root &optional pos-class)

Takes a word or id and returns all surface form ids or all forms of class ’pos-class’ where pos-class is a symbol of langutils::V,A,N

Package

langutils.

Source

lemma.lisp.

Function: morph-surface-forms-text (root &optional pos-class)
Package

langutils.

Source

lemma.lisp.

Function: phrase->concept (p &key lemmatized)

Create a canonical concept from an arbitrary phrase by removing determiners and lemmatizing verbs.

Package

langutils.

Source

concept.lisp.

Function: phrase-words (phrase &optional index)
Package

langutils.

Source

reference.lisp.

Function: read-and-tag-file (file)
Package

langutils.

Source

tagger.lisp.

Function: read-file-as-tagged-document (file)
Package

langutils.

Source

tagger.lisp.

Function: reset-langutils ()
Package

langutils.

Source

init.lisp.

Function: root-noun (phrase)
Package

langutils.

Source

chunker.lisp.

Function: root-nouns (phrases)
Package

langutils.

Source

chunker.lisp.

Function: stopword? (id)

Identifies id as a ’stopword’

Package

langutils.

Source

stopwords.lisp.

Function: string->concept (s &key lemmatized)
Package

langutils.

Source

concept.lisp.

Function: string->token-array (string)
Package

langutils.

Source

tokens.lisp.

Function: string-concise-stopword? (word)

Check the word if it is a ’concise-stopword’ word.
concise-stopwords are a *very* small list of words. Mainly pronouns and determiners

Package

langutils.

Source

stopwords.lisp.

Function: string-contains-is? (words)

Checks the list for a string containing ’is’

Package

langutils.

Source

stopwords.lisp.

Function: string-stopword? (word)
Package

langutils.

Source

stopwords.lisp.

Function: string-tag (string &optional stream)

Tokenizes and tags the string returning
a standard tagged string using ’/’ as a separator

Package

langutils.

Source

reference.lisp.

Function: string-tag-tokenized (string &optional stream)
Package

langutils.

Source

reference.lisp.

Function: suspicious-string? (string)

Determine if the alpha-num and number balance is reasonable for lingustic processing or if non-alpha-nums are present

Package

langutils.

Source

tokens.lisp.

Function: tag (string)
Package

langutils.

Source

tagger.lisp.

Function: tag-tokenized (string)
Package

langutils.

Source

tagger.lisp.

Function: token-array->concept (tokens &key lemmatized)
Package

langutils.

Source

concept.lisp.

Function: token-for-id (id)

Return a string token for a given token id

Package

langutils.

Source

tokens.lisp.

Function: tokenize-stream (stream &key by-sentence fragment)

Converts a stream into a string and tokenizes, optionally, one sentence
at a time which is nice for large files. Pretty hairy code: a token processor inside a stream scanner. The stream scanner walks the input stream and tokenizes all punctuation (except periods). After a sequences of non-whitespace has been read, the inline tokenizer looks at the end of the string for mis-tokenized words (can ’ t -> ca n’t)

Package

langutils-tokenize.

Source

tokenize.lisp.

Function: tokenize-string (string)

Returns a fresh, linguistically tokenized string

Package

langutils-tokenize.

Source

tokenize.lisp.

Function: tokens-for-ids (ids)

Return a list of string tokens for each id in ids

Package

langutils.

Source

tokens.lisp.

Function: vector-document (input)
Package

langutils.

Source

reference.lisp.

Function: vector-tag (string)

Returns a ’document’ which is a class containing a pair of vectors representing the string in the internal token format. Handles arbitrary data.

Package

langutils.

Source

tagger.lisp.

Function: vector-tag-tokenized (string &key end-tokens)

Returns a document representing the string using the
internal token dictionary; requires the string to be tokenized. Parses the string into tokens (whitespace separators) then populates the two temp arrays above with token id’s and initial tags. Contextual rules are applied and a new vector document is produced which
is a copy of the enclosed data. This is all done at once so good compilers can open-code the array refs and simplify the calling
of the labels functions.

Package

langutils.

Source

tagger.lisp.

Function: words->concept (slist &key lemmatized)
Package

langutils.

Source

concept.lisp.


6.1.4 Generic functions

Generic Function: add-word (p index word tag)
Package

langutils.

Methods
Method: add-word ((p altered-phrase) index word tag)
Source

reference.lisp.

Generic Function: change-word (p index new-token &optional new-pos)
Package

langutils.

Methods
Method: change-word ((p altered-phrase) index new-token &optional new-pos)
Source

reference.lisp.

Method: change-word ((p phrase) index new-token &optional new-pos)
Source

reference.lisp.

Generic Function: concat-concepts (&rest concepts)
Package

langutils.

Methods
Method: concat-concepts (&rest concepts)
Source

concept.lisp.

Generic Function: concept->string (cname)
Package

langutils.

Methods
Method: concept->string ((cname concept))
Source

concept.lisp.

Generic Function: concept->token-array (cname)
Package

langutils.

Methods
Method: concept->token-array ((cname concept))

Concepts are immutable, don’t change them!

Source

concept.lisp.

Generic Function: concept->words (cname)
Package

langutils.

Methods
Method: concept->words ((cname concept))
Source

concept.lisp.

Generic Function: concept-contains (csuper csub)
Package

langutils.

Methods
Method: concept-contains ((csuper concept) (csub concept))
Source

concept.lisp.

Generic Function: conceptually-equal (ph1 ph2)
Package

langutils.

Methods
Method: conceptually-equal ((cn1 concept) (cn2 concept))
Source

concept.lisp.

Method: conceptually-equal ((cn concept) (ph phrase))
Source

concept.lisp.

Method: conceptually-equal ((ph phrase) (cn concept))
Source

concept.lisp.

Method: conceptually-equal ((ph1 phrase) (ph2 phrase))
Source

concept.lisp.

Generic Reader: document-annotations (object)
Package

langutils.

Methods
Reader Method: document-annotations ((vector-document vector-document))

automatically generated reader method

Source

reference.lisp.

Target Slot

annotations.

Generic Writer: (setf document-annotations) (object)
Package

langutils.

Methods
Writer Method: (setf document-annotations) ((vector-document vector-document))

automatically generated writer method

Source

reference.lisp.

Target Slot

annotations.

Generic Reader: document-tags (object)
Package

langutils.

Methods
Reader Method: document-tags ((vector-document vector-document))

automatically generated reader method

Source

reference.lisp.

Target Slot

tags.

Generic Writer: (setf document-tags) (object)
Package

langutils.

Methods
Writer Method: (setf document-tags) ((vector-document vector-document))

automatically generated writer method

Source

reference.lisp.

Target Slot

tags.

Generic Reader: document-text (object)
Package

langutils.

Methods
Reader Method: document-text ((vector-document vector-document))

automatically generated reader method

Source

reference.lisp.

Target Slot

text.

Generic Writer: (setf document-text) (object)
Package

langutils.

Methods
Writer Method: (setf document-text) ((vector-document vector-document))

automatically generated writer method

Source

reference.lisp.

Target Slot

text.

Generic Function: find-phrase (p doc &key match start ignore-start ignore-end lemma concept-terms)
Package

langutils.

Methods
Method: find-phrase ((p phrase) (doc vector-document) &key match start ignore-start ignore-end lemma concept-terms)

Find the specified phrase starting at start, matching text and/or tags according to match. The lemma parameter indicates whether the phrases match under the lemma operator and ignore-start and ignore-end causes the search to not match a region within the document

Source

reference.lisp.

Generic Function: find-phrase-intervals (p doc &key match start lemma concept-terms)
Package

langutils.

Methods
Method: find-phrase-intervals ((p array) (doc vector-document) &key match start lemma concept-terms)

Find all phrase intervals in the vector document

Source

reference.lisp.

Method: find-phrase-intervals ((p phrase) (doc vector-document) &key match start lemma concept-terms)

Find all phrase intervals in the vector document

Source

reference.lisp.

Generic Function: get-adverb-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-adverb-chunks ((doc vector-document) &optional interval)

Return a list of all adverbial phrases

Source

chunker.lisp.

Generic Function: get-annotation (doc key)
Package

langutils.

Methods
Method: get-annotation ((p phrase) key)

First returned value is the association value or null if none. The second is true if the key exists, nil otherwise

Source

reference.lisp.

Method: get-annotation ((doc vector-document) key)

First returned value is the association value or null if none. The second is true if the key exists, nil otherwise

Source

reference.lisp.

Generic Function: get-event-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-event-chunks ((doc vector-document) &optional interval)

Return vx+nx (simple verb arg) phrase objects

Source

chunker.lisp.

Generic Function: get-extended-event-chunks1 (doc &optional interval)
Package

langutils.

Methods
Method: get-extended-event-chunks1 ((doc vector-document) &optional interval)

Return vx+nx+pp... objects

Source

chunker.lisp.

Generic Function: get-extended-event-chunks2 (doc &optional interval)
Package

langutils.

Methods
Method: get-extended-event-chunks2 ((doc vector-document) &optional interval)

Return vx+nx+pp... objects

Source

chunker.lisp.

Generic Function: get-imperative-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-imperative-chunks ((doc vector-document) &optional interval)
Source

chunker.lisp.

Generic Function: get-nx-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-nx-chunks ((doc vector-document) &optional interval)

Return a list of all nx phrases

Source

chunker.lisp.

Generic Function: get-p-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-p-chunks ((doc vector-document) &optional interval)

Return a list of all prepositions as phrases

Source

chunker.lisp.

Generic Function: get-pp-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-pp-chunks ((doc vector-document) &optional interval)

Return a list of all prepositions as phrases

Source

chunker.lisp.

Generic Function: get-tag (doc offset)
Package

langutils.

Methods
Method: get-tag ((phrase altered-phrase) index)
Source

reference.lisp.

Method: get-tag ((p phrase) offset)
Source

reference.lisp.

Method: get-tag ((doc vector-document) offset)
Source

reference.lisp.

Generic Function: get-token-id (doc offset)
Package

langutils.

Methods
Method: get-token-id ((phrase altered-phrase) index)
Source

reference.lisp.

Method: get-token-id ((p phrase) offset)
Source

reference.lisp.

Method: get-token-id ((doc vector-document) offset)
Source

reference.lisp.

Generic Function: get-vx-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-vx-chunks ((doc vector-document) &optional interval)

Return a list of all primitive vx phrases - no arguments

Source

chunker.lisp.

Generic Function: lemmatize (sequence &key strip-det pos noun porter last-only)
Package

langutils.

Methods
Method: lemmatize ((sequence array) &key strip-det pos noun porter last-only)
Source

lemma.lisp.

Method: lemmatize ((sequence list) &key strip-det pos noun porter last-only)

Non-destructive lemmatization of provided sequence

Source

lemma.lisp.

Generic Function: lemmatize-phrase (p &optional offset)
Package

langutils.

Methods
Method: lemmatize-phrase ((p altered-phrase) &optional offset)

Destructive lemmatization of a phrase

Source

reference.lisp.

Method: lemmatize-phrase ((p phrase) &optional offset)

Destructive lemmatization of a phrase

Source

reference.lisp.

Generic Function: length-of (doc)
Package

langutils.

Methods
Method: length-of ((doc vector-document))
Source

reference.lisp.

Generic Function: make-alterable-phrase (p)
Package

langutils.

Methods
Method: make-alterable-phrase ((p phrase))
Source

reference.lisp.

Generic Function: phrase->string (p &key with-tags with-info newline)
Package

langutils.

Methods
Method: phrase->string ((p phrase) &key with-tags with-info newline)
Source

reference.lisp.

Generic Function: phrase->token-array (p)
Package

langutils.

Methods
Method: phrase->token-array ((p phrase))

Used in conceptnet to index into a node data structure NOTE: could be faster with direct, declared array copy

Source

reference.lisp.

Generic Function: phrase-distance (p1 p2)
Package

langutils.

Methods
Method: phrase-distance ((p1 phrase) (p2 phrase))

Distance between the nearest end of two phrases

Source

reference.lisp.

Generic Function: phrase-document (object)
Package

langutils.

Methods
Method: phrase-document ((p altered-phrase))
Source

reference.lisp.

Reader Method: phrase-document ((phrase phrase))

automatically generated reader method

Source

reference.lisp.

Target Slot

document.

Generic Writer: (setf phrase-document) (object)
Package

langutils.

Methods
Writer Method: (setf phrase-document) ((phrase phrase))

automatically generated writer method

Source

reference.lisp.

Target Slot

document.

Generic Function: phrase-end (object)
Package

langutils.

Methods
Method: phrase-end ((p altered-phrase))
Source

reference.lisp.

Reader Method: phrase-end ((phrase phrase))

automatically generated reader method

Source

reference.lisp.

Target Slot

end.

Generic Writer: (setf phrase-end) (object)
Package

langutils.

Methods
Writer Method: (setf phrase-end) ((phrase phrase))

automatically generated writer method

Source

reference.lisp.

Target Slot

end.

Generic Function: phrase-equal (ph1 ph2)
Package

langutils.

Methods
Method: phrase-equal ((ph1 phrase) (ph2 phrase))
Source

reference.lisp.

Generic Function: phrase-lemmas (ph)
Package

langutils.

Methods
Method: phrase-lemmas ((ph phrase))

Returns the lemmatized phrase represented by the underlying phrase

Source

reference.lisp.

Generic Function: phrase-length (p)
Package

langutils.

Methods
Method: phrase-length ((p altered-phrase))
Source

reference.lisp.

Method: phrase-length ((p phrase))
Source

reference.lisp.

Generic Function: phrase-overlap (ph1 ph2)
Package

langutils.

Methods
Method: phrase-overlap ((ph1 phrase) (ph2 phrase))
Source

reference.lisp.

Generic Function: phrase-start (object)
Package

langutils.

Methods
Method: phrase-start ((p altered-phrase))
Source

reference.lisp.

Reader Method: phrase-start ((phrase phrase))

automatically generated reader method

Source

reference.lisp.

Target Slot

start.

Generic Writer: (setf phrase-start) (object)
Package

langutils.

Methods
Writer Method: (setf phrase-start) ((phrase phrase))

automatically generated writer method

Source

reference.lisp.

Target Slot

start.

Generic Reader: phrase-type (object)
Package

langutils.

Methods
Reader Method: phrase-type ((phrase phrase))

automatically generated reader method

Source

reference.lisp.

Target Slot

type.

Generic Writer: (setf phrase-type) (object)
Package

langutils.

Methods
Writer Method: (setf phrase-type) ((phrase phrase))

automatically generated writer method

Source

reference.lisp.

Target Slot

type.

Generic Function: print-phrase (p &key stream with-tags with-info newline)
Package

langutils.

Methods
Method: print-phrase ((p phrase) &key stream with-tags with-info newline)
Source

reference.lisp.

Generic Function: print-phrase-lemmas (ph)
Package

langutils.

Methods
Method: print-phrase-lemmas ((ph phrase))
Source

reference.lisp.

Generic Function: print-vector-document (doc &key stream with-tags with-newline)
Package

langutils.

Methods
Method: print-vector-document ((doc vector-document) &key stream with-tags with-newline)
Source

reference.lisp.

Generic Function: print-window (p wsize &key stream with-tags with-info newline)
Package

langutils.

Methods
Method: print-window ((p phrase) wsize &key stream with-tags with-info newline)
Source

reference.lisp.

Generic Function: read-vector-document (filename)
Package

langutils.

Methods
Method: read-vector-document (filename)
Source

reference.lisp.

Generic Function: read-vector-document-to-string (doc &key with-tags)
Package

langutils.

Methods
Method: read-vector-document-to-string ((doc vector-document) &key with-tags)
Source

reference.lisp.

Generic Function: remove-word (p index)
Package

langutils.

Methods
Method: remove-word ((p altered-phrase) index)
Source

reference.lisp.

Method: remove-word ((p phrase) index)
Source

reference.lisp.

Generic Function: set-annotation (doc key value &key method)
Package

langutils.

Methods
Method: set-annotation ((p phrase) key value &key method)

Add an annotation to object using method :override, :push, :duplicate-key

Source

reference.lisp.

Method: set-annotation ((doc vector-document) key value &key method)

Add an annotation to object using method :override, :push, :duplicate-key

Source

reference.lisp.

Generic Function: suspicious-word? (word)
Package

langutils.

Methods
Method: suspicious-word? ((word fixnum))

Find a suspicious word using it’s token id

Source

tokens.lisp.

Generic Reader: token-vector (object)
Package

langutils.

Methods
Reader Method: token-vector ((concept concept))

Stores the representation of the concept as an array of token ids

Source

concept.lisp.

Target Slot

token-vector.

Generic Function: unset-annotation (doc key)
Package

langutils.

Methods
Method: unset-annotation ((p phrase) key)
Source

reference.lisp.

Method: unset-annotation ((doc vector-document) key)
Source

reference.lisp.

Generic Function: vector-document-string (doc &key with-tags with-newline)
Package

langutils.

Methods
Method: vector-document-string ((doc vector-document) &key with-tags with-newline)
Source

reference.lisp.

Generic Function: vector-document-words (doc)
Package

langutils.

Methods
Method: vector-document-words ((doc vector-document))
Source

reference.lisp.

Generic Function: write-vector-document (doc filename &key with-tags if-exists)
Package

langutils.

Methods
Method: write-vector-document ((doc vector-document) filename &key with-tags if-exists)
Source

reference.lisp.


6.1.5 Standalone methods

Method: print-object ((p phrase) stream)
Source

reference.lisp.

Method: print-object ((cn concept) stream)
Source

concept.lisp.

Method: print-object ((object meta) stream)
Source

my-meta.lisp.


6.1.6 Structures

Structure: lexicon-entry
Package

langutils.

Source

lexicon.lisp.

Direct superclasses

structure-object.

Direct slots
Slot: tags
Readers

lexicon-entry-tags.

Writers

(setf lexicon-entry-tags).

Slot: id
Readers

lexicon-entry-id.

Writers

(setf lexicon-entry-id).

Slot: roots
Readers

lexicon-entry-roots.

Writers

(setf lexicon-entry-roots).

Slot: surface-forms
Readers

lexicon-entry-surface-forms.

Writers

(setf lexicon-entry-surface-forms).

Slot: case-forms
Readers

lexicon-entry-case-forms.

Writers

(setf lexicon-entry-case-forms).


6.1.7 Classes

Class: altered-phrase
Package

langutils.

Source

reference.lisp.

Direct superclasses

phrase.

Direct methods
Direct slots
Slot: custom-document
Initargs

:custom-document

Readers

altered-phrase-custom-document.

Writers

(setf altered-phrase-custom-document).

Class: phrase
Package

langutils.

Source

reference.lisp.

Direct subclasses

altered-phrase.

Direct methods
Direct slots
Slot: type
Package

common-lisp.

Initargs

:type

Readers

phrase-type.

Writers

(setf phrase-type).

Slot: document
Initargs

:document

Readers

phrase-document.

Writers

(setf phrase-document).

Slot: start
Initargs

:start

Readers

phrase-start.

Writers

(setf phrase-start).

Slot: end
Initargs

:end

Readers

phrase-end.

Writers

(setf phrase-end).

Slot: annotations
Initargs

:annotations

Readers

phrase-annotations.

Writers

(setf phrase-annotations).

Class: vector-document
Package

langutils.

Source

reference.lisp.

Direct methods
Direct slots
Slot: text
Type

(array fixnum)

Initargs

:text

Readers

document-text.

Writers

(setf document-text).

Slot: tags
Type

(array symbol)

Initargs

:tags

Readers

document-tags.

Writers

(setf document-tags).

Slot: annotations
Type

list

Initargs

:annotations

Readers

document-annotations.

Writers

(setf document-annotations).


6.2 Internals


6.2.1 Constants

Constant: *max-token-nums*

The maximum number of numbers allowed in a valid token

Package

langutils.

Source

tokens.lisp.

Constant: *max-token-others*

The maximum number of non alpha-numeric characters in a valid token

Package

langutils.

Source

tokens.lisp.

Constant: *whitespace-chars*
Package

langutils.

Source

tokens.lisp.

Constant: adv-pattern
Package

langutils.

Source

chunker-constants.lisp.

Constant: noun-pattern
Package

langutils.

Source

chunker-constants.lisp.

Constant: p-pattern
Package

langutils.

Source

chunker-constants.lisp.

Constant: verb-pattern
Package

langutils.

Source

chunker-constants.lisp.


6.2.2 Special variables

Special Variable: *add-to-map-hook*
Package

langutils.

Source

tokens.lisp.

Special Variable: *auto-init*

Whether to call initialize-langutils when the .fasl is loaded

Package

langutils.

Source

config.lisp.

Special Variable: *common-verbs*
Package

langutils.

Source

chunker.lisp.

Special Variable: *concept-store-scratch-array*

Allows us to lookup concepts from arrays without allocating lots of unnecessary data

Package

langutils.

Source

concept.lisp.

Special Variable: *concept-vhash*
Package

langutils.

Source

concept.lisp.

Special Variable: *concise-stopwords*
Package

langutils.

Source

stopwords.lisp.

Special Variable: *config-paths*
Package

langutils.

Source

config.lisp.

Special Variable: *contextual-rule-args*

The templates for parsing contextual rules and constructing matching templates over word/pos arrays

Package

langutils.

Source

contextual-rule-parser.lisp.

Special Variable: *default-concise-stopwords-file*

Path to a *very* small list of words. Mainly pronouns and determiners

Package

langutils.

Source

config.lisp.

Special Variable: *default-contextual-rule-file*

Path to the brill contextual rule file

Package

langutils.

Source

config.lisp.

Special Variable: *default-lexical-rule-file*

Path to the brill lexical rule file

Package

langutils.

Source

config.lisp.

Special Variable: *default-lexicon-file*

Path to the lexicon file

Package

langutils.

Source

config.lisp.

Special Variable: *default-stems-file*

Path to the word stems file

Package

langutils.

Source

config.lisp.

Special Variable: *default-stopwords-file*

Path to a stopwords file

Package

langutils.

Source

config.lisp.

Special Variable: *default-token-map-file*

Path to the token map file

Package

langutils.

Source

config.lisp.

Special Variable: *external-token-map*
Package

langutils.

Source

tokens.lisp.

Special Variable: *id-for-token-hook*
Package

langutils.

Source

tokens.lisp.

Special Variable: *id-table*
Package

langutils.

Source

tokens.lisp.

Special Variable: *is-token*
Package

langutils.

Source

stopwords.lisp.

Special Variable: *lexicon*
Package

langutils.

Source

lexicon.lisp.

Special Variable: *meta-readtable*
Package

my-meta.

Source

my-meta.lisp.

Special Variable: *pos-class-map*
Package

langutils.

Source

lemma.lisp.

Special Variable: *report-status*

Where to print langutils messages; default to none

Package

langutils.

Source

config.lisp.

Special Variable: *s-token*
Package

langutils.

Source

stopwords.lisp.

Special Variable: *saved-readtable*
Package

my-meta.

Source

my-meta.lisp.

Special Variable: *stopwords*
Package

langutils.

Source

stopwords.lisp.

Special Variable: *suspicious-words*

Memoize known suspicious words that have been tokenized in this hash

Package

langutils.

Source

tokens.lisp.

Special Variable: *tagger-bigrams*

Bigram hash (not implemented yet)

Package

langutils.

Source

tagger.lisp.

Special Variable: *tagger-contextual-rules*

Table to hold the contextual rule closures

Package

langutils.

Source

tagger.lisp.

Special Variable: *tagger-lexical-rules*

Table to hold the lexical rule closures

Package

langutils.

Source

tagger.lisp.

Special Variable: *tagger-wordlist*

Wordlist hash (not implemented yet)

Package

langutils.

Source

tagger.lisp.

Special Variable: *temp-phrase*
Package

langutils.

Source

reference.lisp.

Special Variable: *test*
Package

langutils.

Source

reference.lisp.

Special Variable: *token-counter*
Package

langutils.

Source

tokens.lisp.

Special Variable: *token-counter-hook*
Package

langutils.

Source

tokens.lisp.

Special Variable: *token-dirty-bit*
Package

langutils.

Source

tokens.lisp.

Special Variable: *token-for-id-hook*
Package

langutils.

Source

tokens.lisp.

Special Variable: *token-table*
Package

langutils.

Source

tokens.lisp.

Special Variable: *tokens-load-file*
Package

langutils.

Source

tokens.lisp.

Special Variable: known-abbreviations
Package

langutils-tokenize.

Source

tokenize.lisp.


6.2.3 Macros

Macro: def-contextual-rule-parser (name &body template-list)

Given a list of structures, defines a generator named ’name’ that takes
a Brill contextual rule list (list of strings) and generates an applicable closure. The closure accepts an argument list of (tokens tags offset) and will apply the rule and related side effect to the two arrays at the provided offset. Patterns are to be given in the form:
("SURROUNDTAG" (match (0 oldtag) (-1 tag1) (+1 tag2)) =>
(setf oldtag newtag))

Package

langutils.

Source

contextual-rule-parser.lisp.

Macro: list-match (x &key source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Macro: list-match-type (x v &key source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Macro: stream-match (x &key source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Macro: stream-match-type (x v &key source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Macro: string-match (x &key source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Macro: string-match-type (x v &key source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Macro: with-static-memory-allocation (() &rest body)
Package

langutils.

Source

lexicon.lisp.

Macro: write-log (name msg &rest args)
Package

langutils.

Source

config.lisp.


6.2.4 Ordinary functions

Function: *get-determiners* ()
Package

langutils.

Source

lemma.lisp.

Function: add-basic-entry (word tags &key roots surface)

Add a word and it’s probability ordered tags to the lexicon

Package

langutils.

Source

lexicon.lisp.

Function: add-external-mapping (id-for-token token-for-id add-to-map token-counter)
Package

langutils.

Source

tokens.lisp.

Function: add-root (word pos-root-pair)

Add a root form to word if not exists

Package

langutils.

Source

lexicon.lisp.

Function: add-root-forms (word pos-root-pairs)
Package

langutils.

Source

lexicon.lisp.

Function: add-roots (word root-pairs)

Set the root list (pairs of pos_type/root) for the entry for ’word’

Package

langutils.

Source

lexicon.lisp.

Function: add-surface-form (root surface-form)

Add a surface form to a root word

Package

langutils.

Source

lexicon.lisp.

Function: add-to-map-hook (token id)
Package

langutils.

Source

tokens.lisp.

Function: add-unknown-lexicon-entry (word guessed-tag)
Package

langutils.

Source

lexicon.lisp.

Function: all-vx+nx-phrases (phrases)

Overly hairy function for finding all vx phrases that are followed by nx. Get event chunks is a better way to do this.

Package

langutils.

Source

chunker.lisp.

Function: alpha-lowercase (ch)

Return T if the given character is an alpha character

Package

langutils-tokenize.

Source

tokenize.lisp.

Function: alpha-misc (ch)
Package

langutils-tokenize.

Source

tokenize.lisp.

Function: alpha-uppercase (ch)
Package

langutils-tokenize.

Source

tokenize.lisp.

Function: apply-contextual-rules (elements)
Package

langutils.

Source

tagger.lisp.

Function: apply-rules (datum rule-list)

Apply rules to the values in values presuming that the returned list is also a list of values that can be passed to the next rule

Package

langutils.

Source

tagger-data.lisp.

Function: clean-lexicon ()
Package

langutils.

Source

lexicon.lisp.

Function: clean-stopwords ()
Package

langutils.

Source

stopwords.lisp.

Function: compile-list (l &key source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Function: compileit (x &key meta-parser-type source-symbol)
Package

my-meta.

Source

my-meta.lisp.

Function: consonantp (str i)
Package

langutils.

Source

porter.lisp.

Function: copy-lexicon-entry (instance)
Package

langutils.

Source

lexicon.lisp.

Function: copy-meta (instance)
Package

my-meta.

Source

my-meta.lisp.

Function: cvc (str lim)
Package

langutils.

Source

porter.lisp.

Function: default-tag (token)

Simple default tagging based on capitalization of token string

Package

langutils.

Source

tagger.lisp.

Function: doublec (str i)
Package

langutils.

Source

porter.lisp.

Function: duplicate-from (source start end)
Package

langutils.

Source

tagger.lisp.

Function: ends (str ending)
Package

langutils.

Source

porter.lisp.

Function: ensure-common-verbs ()
Package

langutils.

Source

chunker.lisp.

Function: ensure-concept (tokens)
Package

langutils.

Source

concept.lisp.

Function: ensure-lexicon-entry (word &key roots surface)
Package

langutils.

Source

lexicon.lisp.

Function: ensure-token-counts ()

Reset token count if not already set

Package

langutils.

Source

tokens.lisp.

Function: gen-rule-arg-bindings (pattern)

Generate let bindings for the args referenced in the match pattern

Package

langutils.

Source

contextual-rule-parser.lisp.

Function: gen-rule-arg-decls (pattern)

Generate type declarations for canonical variables from table entry

Package

langutils.

Source

contextual-rule-parser.lisp.

Function: gen-rule-closure (template)

Generate the code for the rule closure as one of the cond forms matching the name of the closure pattern to the rule pattern

Package

langutils.

Source

contextual-rule-parser.lisp.

Function: gen-rule-closure-decl ()

Optimize the compiled closure through type and optimization declarations

Package

langutils.

Source

contextual-rule-parser.lisp.

Function: gen-rule-match (pattern)

Generate the conditional code to match this rule

Package

langutils.

Source

contextual-rule-parser.lisp.

Function: get-bind-entry (var)

Given a canonical variable name, create its let binding and extraction expression from the rule file entry

Package

langutils.

Source

contextual-rule-parser.lisp.

Function: guess-tag (token initial-tag rule-list)

Using rules in rule-table guess the tag of the token ’token’

Package

langutils.

Source

tagger-data.lisp.

Function: handle-config-entry (entry)
Package

langutils.

Source

config.lisp.

Function: id-for-token-hook (token)
Package

langutils.

Source

tokens.lisp.

Function: ids-for-string (string)
Package

langutils.

Source

tokens.lisp.

Function: init-concise-stopwords (&optional path)
Package

langutils.

Source

stopwords.lisp.

Function: init-lexicon (&optional lexicon-file lemma-file)

Populates the lexicon with ’word tag1 tag2’ structured lines from lexicon-file

Package

langutils.

Source

lexicon.lisp.

Function: init-stopwords (&optional path)
Package

langutils.

Source

stopwords.lisp.

Function: init-word-test ()
Package

langutils.

Source

stopwords.lisp.

Function: initialize-tokens ()
Package

langutils.

Source

tokens.lisp.

Reader: lexicon-entry-case-forms (instance)
Writer: (setf lexicon-entry-case-forms) (instance)
Package

langutils.

Source

lexicon.lisp.

Target Slot

case-forms.

Function: lexicon-entry-p (object)
Package

langutils.

Source

lexicon.lisp.

Function: load-contextual-rules (rule-file)
Package

langutils.

Source

tagger-data.lisp.

Function: load-lexical-rules (rule-file &optional bigram-hash word-hash)

Return a list of closure implementing the lexical rules in rule-file to tag words not found in the lexicon

Package

langutils.

Source

tagger-data.lisp.

Function: load-tagger-files (lexical-rules contextual-rules &key bigrams wordlist)
Package

langutils.

Source

tagger.lisp.

Function: m (str lim)
Package

langutils.

Source

porter.lisp.

Function: make-cases (word)
Package

langutils.

Source

lexicon.lisp.

Function: make-contextual-rule (pattern)
Package

langutils.

Source

tagger-data.lisp.

Function: make-lexical-rule (list lh bh wh)

Look through list for rule name

Package

langutils.

Source

tagger-data.lisp.

Function: make-lexicon-entry (&key tags id roots surface-forms case-forms)
Package

langutils.

Source

lexicon.lisp.

Function: make-meta (&key char form)
Package

my-meta.

Source

my-meta.lisp.

Reader: meta-char (instance)
Writer: (setf meta-char) (instance)
Package

my-meta.

Source

my-meta.lisp.

Target Slot

char.

Reader: meta-form (instance)
Writer: (setf meta-form) (instance)
Package

my-meta.

Source

my-meta.lisp.

Target Slot

form.

Function: meta-p (object)
Package

my-meta.

Source

my-meta.lisp.

Function: meta-reader (s c)
Package

my-meta.

Source

my-meta.lisp.

Function: person-token-offset (array)
Package

langutils.

Source

reference.lisp.

Function: print-token-array (tokens start stop &key pos pos-start stream with-tags newline)
Package

langutils.

Source

reference.lisp.

Function: r (str s sfp)
Package

langutils.

Source

porter.lisp.

Function: read-config ()
Package

langutils.

Source

config.lisp.

Function: read-file-to-string (file)
Package

langutils.

Source

tagger.lisp.

Function: relative-pathname (path)
Package

langutils.

Source

config.lisp.

Function: reset-token-counts ()

Reset all the token datastructures to an initialized but empty state

Package

langutils.

Source

tokens.lisp.

Function: return-vector-doc (elements)
Package

langutils.

Source

tagger.lisp.

Function: select-token (token &key strip-det noun pos porter lemma)

Internal per-token function

Package

langutils.

Source

lemma.lisp.

Function: set-lexicon-entry (word entry)
Package

langutils.

Source

lexicon.lisp.

Setf expanders to this function

(setf get-lexicon-entry).

Function: setto (str suffix)
Package

langutils.

Source

porter.lisp.

Function: stem (str)
Package

langutils.

Source

porter.lisp.

Function: step1ab (str)
Package

langutils.

Source

porter.lisp.

Function: step1c (str)
Package

langutils.

Source

porter.lisp.

Function: step2 (str)
Package

langutils.

Source

porter.lisp.

Function: step3 (str)
Package

langutils.

Source

porter.lisp.

Function: step4 (str)
Package

langutils.

Source

porter.lisp.

Function: step5 (str)
Package

langutils.

Source

porter.lisp.

Function: symbol-name-equal (src target &key key test)
Package

my-meta.

Source

my-meta.lisp.

Function: temp-phrase ()
Package

langutils.

Source

reference.lisp.

Function: test-concept-equality ()
Package

langutils.

Source

concept.lisp.

Function: test-phrase (text)

Prints all the phrases found in the text for simple experimenting

Package

langutils.

Source

chunker.lisp.

Function: test-vector-tag-tokenized (string)
Package

langutils.

Source

tagger.lisp.

Function: token-array->words (tokens)
Package

langutils.

Source

reference.lisp.

Function: token-counter-hook ()
Package

langutils.

Source

tokens.lisp.

Function: token-for-id-hook (id)
Package

langutils.

Source

tokens.lisp.

Function: tokenize-file2 (source-file target-file &key if-exists)

Tokenizes a pure text file a sentence at a time

Package

langutils-tokenize.

Source

tokenize.lisp.

Function: vowelinstem (str)
Package

langutils.

Source

porter.lisp.

Function: write-temp (token tag pos)
Package

langutils.

Source

tagger.lisp.


6.2.5 Generic functions

Generic Reader: altered-phrase-custom-document (object)
Package

langutils.

Methods
Reader Method: altered-phrase-custom-document ((altered-phrase altered-phrase))

automatically generated reader method

Source

reference.lisp.

Target Slot

custom-document.

Generic Writer: (setf altered-phrase-custom-document) (object)
Package

langutils.

Methods
Writer Method: (setf altered-phrase-custom-document) ((altered-phrase altered-phrase))

automatically generated writer method

Source

reference.lisp.

Target Slot

custom-document.

Generic Function: clear-concept-cache ()
Package

langutils.

Methods
Method: clear-concept-cache ()
Source

concept.lisp.

Generic Function: copy-phrase (p &optional annotations)
Package

langutils.

Methods
Method: copy-phrase ((p phrase) &optional annotations)
Source

reference.lisp.

Generic Function: document-window-as-string (document start end)
Package

langutils.

Methods
Method: document-window-as-string (document start end)
Source

reference.lisp.

Generic Function: get-basic-chunks (doc &optional interval)
Package

langutils.

Methods
Method: get-basic-chunks ((doc vector-document) &optional interval)

Returns a list of PHRASEs referencing ’doc’ for all supported primitive phrase types

Source

chunker.lisp.

Generic Function: lookup-canonical-concept-instance (ta)
Package

langutils.

Methods
Method: lookup-canonical-concept-instance ((lf list))

List of fixnums to lookup a concept instance

Source

concept.lisp.

Method: lookup-canonical-concept-instance ((ta array))

Take a token array and see if there is already a concept instance

Source

concept.lisp.

Generic Function: make-document-from-phrase (p)
Package

langutils.

Methods
Method: make-document-from-phrase ((p phrase))

Copy referenced phrase data into it’s own document

Source

reference.lisp.

Generic Reader: phrase-annotations (object)
Package

langutils.

Methods
Reader Method: phrase-annotations ((phrase phrase))

automatically generated reader method

Source

reference.lisp.

Target Slot

annotations.

Generic Writer: (setf phrase-annotations) (object)
Package

langutils.

Methods
Writer Method: (setf phrase-annotations) ((phrase phrase))

automatically generated writer method

Source

reference.lisp.

Target Slot

annotations.

Generic Function: register-new-concept-instance (c)
Package

langutils.

Methods
Method: register-new-concept-instance ((c concept))
Source

concept.lisp.

Generic Function: vector-doc-as-ids (doc)
Package

langutils.

Methods
Method: vector-doc-as-ids ((doc vector-document))

Converts the word array to ids with shared structure
for the other elements; keeps the data ’in the family’ so the source or destination documents should be short lived

Source

reference.lisp.

Generic Function: vector-doc-as-words (doc)
Package

langutils.

Methods
Method: vector-doc-as-words ((doc vector-document))
Source

reference.lisp.


6.2.6 Conditions

Condition: end-of-sentence
Package

langutils-tokenize.

Source

tokenize.lisp.

Direct superclasses

condition.


6.2.7 Structures

Structure: meta
Package

my-meta.

Source

my-meta.lisp.

Direct superclasses

structure-object.

Direct methods

print-object.

Direct slots
Slot: char
Package

common-lisp.

Readers

meta-char.

Writers

(setf meta-char).

Slot: form
Readers

meta-form.

Writers

(setf meta-form).


6.2.8 Classes

Class: concept
Package

langutils.

Source

concept.lisp.

Direct methods
Direct slots
Slot: token-vector

Stores the representation of the concept as an array of token ids

Type

(array fixnum)

Initargs

:token-vector

Readers

token-vector.

Writers

This slot is read-only.


6.2.9 Types

Type: alpha ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: alpha-lower ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: alpha-upper ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: alphanum ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: digit ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: non-digit ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: non-digit-or-ws ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: non-punc-or-white ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: non-whitespace ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: punctuation ()
Package

langutils-tokenize.

Source

tokenize.lisp.

Type: whitespace ()
Package

langutils-tokenize.

Source

tokenize.lisp.


Appendix A Indexes


A.1 Concepts


A.2 Functions

Jump to:   (   *  
A   C   D   E   F   G   H   I   L   M   P   R   S   T   U   V   W  
Index Entry  Section

(
(setf altered-phrase-custom-document): Private generic functions
(setf altered-phrase-custom-document): Private generic functions
(setf document-annotations): Public generic functions
(setf document-annotations): Public generic functions
(setf document-tags): Public generic functions
(setf document-tags): Public generic functions
(setf document-text): Public generic functions
(setf document-text): Public generic functions
(setf get-lexicon-entry): Public setf expanders
(setf lexicon-entry-case-forms): Private ordinary functions
(setf lexicon-entry-id): Public ordinary functions
(setf lexicon-entry-roots): Public ordinary functions
(setf lexicon-entry-surface-forms): Public ordinary functions
(setf lexicon-entry-tags): Public ordinary functions
(setf meta-char): Private ordinary functions
(setf meta-form): Private ordinary functions
(setf phrase-annotations): Private generic functions
(setf phrase-annotations): Private generic functions
(setf phrase-document): Public generic functions
(setf phrase-document): Public generic functions
(setf phrase-end): Public generic functions
(setf phrase-end): Public generic functions
(setf phrase-start): Public generic functions
(setf phrase-start): Public generic functions
(setf phrase-type): Public generic functions
(setf phrase-type): Public generic functions

*
*get-determiners*: Private ordinary functions

A
add-basic-entry: Private ordinary functions
add-external-mapping: Private ordinary functions
add-root: Private ordinary functions
add-root-forms: Private ordinary functions
add-roots: Private ordinary functions
add-surface-form: Private ordinary functions
add-to-map-hook: Private ordinary functions
add-unknown-lexicon-entry: Private ordinary functions
add-word: Public generic functions
add-word: Public generic functions
all-vx+nx-phrases: Private ordinary functions
alpha-lowercase: Private ordinary functions
alpha-misc: Private ordinary functions
alpha-uppercase: Private ordinary functions
altered-phrase-custom-document: Private generic functions
altered-phrase-custom-document: Private generic functions
apply-contextual-rules: Private ordinary functions
apply-rules: Private ordinary functions
associate-concepts: Public ordinary functions

C
change-word: Public generic functions
change-word: Public generic functions
change-word: Public generic functions
chunk: Public ordinary functions
chunk-tokenized: Public ordinary functions
clean-langutils: Public ordinary functions
clean-lexicon: Private ordinary functions
clean-stopwords: Private ordinary functions
clean-tagger: Public ordinary functions
clear-concept-cache: Private generic functions
clear-concept-cache: Private generic functions
compile-list: Private ordinary functions
compileit: Private ordinary functions
concat-concepts: Public generic functions
concat-concepts: Public generic functions
concept->string: Public generic functions
concept->string: Public generic functions
concept->token-array: Public generic functions
concept->token-array: Public generic functions
concept->words: Public generic functions
concept->words: Public generic functions
concept-contains: Public generic functions
concept-contains: Public generic functions
conceptually-equal: Public generic functions
conceptually-equal: Public generic functions
conceptually-equal: Public generic functions
conceptually-equal: Public generic functions
conceptually-equal: Public generic functions
concise-stopword?: Public ordinary functions
consonantp: Private ordinary functions
contains-is?: Public ordinary functions
copy-lexicon-entry: Private ordinary functions
copy-meta: Private ordinary functions
copy-phrase: Private generic functions
copy-phrase: Private generic functions
cvc: Private ordinary functions

D
def-contextual-rule-parser: Private macros
default-tag: Private ordinary functions
disable-meta-syntax: Public ordinary functions
document-annotations: Public generic functions
document-annotations: Public generic functions
document-tags: Public generic functions
document-tags: Public generic functions
document-text: Public generic functions
document-text: Public generic functions
document-window-as-string: Private generic functions
document-window-as-string: Private generic functions
doublec: Private ordinary functions
duplicate-from: Private ordinary functions

E
enable-meta-syntax: Public ordinary functions
ends: Private ordinary functions
ensure-common-verbs: Private ordinary functions
ensure-concept: Private ordinary functions
ensure-lexicon-entry: Private ordinary functions
ensure-token-counts: Private ordinary functions

F
find-phrase: Public generic functions
find-phrase: Public generic functions
find-phrase-intervals: Public generic functions
find-phrase-intervals: Public generic functions
find-phrase-intervals: Public generic functions
force-concept: Public ordinary functions
Function, (setf lexicon-entry-case-forms): Private ordinary functions
Function, (setf lexicon-entry-id): Public ordinary functions
Function, (setf lexicon-entry-roots): Public ordinary functions
Function, (setf lexicon-entry-surface-forms): Public ordinary functions
Function, (setf lexicon-entry-tags): Public ordinary functions
Function, (setf meta-char): Private ordinary functions
Function, (setf meta-form): Private ordinary functions
Function, *get-determiners*: Private ordinary functions
Function, add-basic-entry: Private ordinary functions
Function, add-external-mapping: Private ordinary functions
Function, add-root: Private ordinary functions
Function, add-root-forms: Private ordinary functions
Function, add-roots: Private ordinary functions
Function, add-surface-form: Private ordinary functions
Function, add-to-map-hook: Private ordinary functions
Function, add-unknown-lexicon-entry: Private ordinary functions
Function, all-vx+nx-phrases: Private ordinary functions
Function, alpha-lowercase: Private ordinary functions
Function, alpha-misc: Private ordinary functions
Function, alpha-uppercase: Private ordinary functions
Function, apply-contextual-rules: Private ordinary functions
Function, apply-rules: Private ordinary functions
Function, associate-concepts: Public ordinary functions
Function, chunk: Public ordinary functions
Function, chunk-tokenized: Public ordinary functions
Function, clean-langutils: Public ordinary functions
Function, clean-lexicon: Private ordinary functions
Function, clean-stopwords: Private ordinary functions
Function, clean-tagger: Public ordinary functions
Function, compile-list: Private ordinary functions
Function, compileit: Private ordinary functions
Function, concise-stopword?: Public ordinary functions
Function, consonantp: Private ordinary functions
Function, contains-is?: Public ordinary functions
Function, copy-lexicon-entry: Private ordinary functions
Function, copy-meta: Private ordinary functions
Function, cvc: Private ordinary functions
Function, default-tag: Private ordinary functions
Function, disable-meta-syntax: Public ordinary functions
Function, doublec: Private ordinary functions
Function, duplicate-from: Private ordinary functions
Function, enable-meta-syntax: Public ordinary functions
Function, ends: Private ordinary functions
Function, ensure-common-verbs: Private ordinary functions
Function, ensure-concept: Private ordinary functions
Function, ensure-lexicon-entry: Private ordinary functions
Function, ensure-token-counts: Private ordinary functions
Function, force-concept: Public ordinary functions
Function, gen-rule-arg-bindings: Private ordinary functions
Function, gen-rule-arg-decls: Private ordinary functions
Function, gen-rule-closure: Private ordinary functions
Function, gen-rule-closure-decl: Private ordinary functions
Function, gen-rule-match: Private ordinary functions
Function, get-bind-entry: Private ordinary functions
Function, get-lemma: Public ordinary functions
Function, get-lemma-for-id: Public ordinary functions
Function, get-lexicon-case-forms: Public ordinary functions
Function, get-lexicon-default-pos: Public ordinary functions
Function, get-lexicon-entry: Public ordinary functions
Function, get-token-count: Public ordinary functions
Function, guess-tag: Private ordinary functions
Function, handle-config-entry: Private ordinary functions
Function, head-verb: Public ordinary functions
Function, head-verbs: Public ordinary functions
Function, id-for-token: Public ordinary functions
Function, id-for-token-hook: Private ordinary functions
Function, ids-for-string: Private ordinary functions
Function, ids-for-tokens: Public ordinary functions
Function, in-pos-class?: Public ordinary functions
Function, init-concise-stopwords: Private ordinary functions
Function, init-langutils: Public ordinary functions
Function, init-lexicon: Private ordinary functions
Function, init-stopwords: Private ordinary functions
Function, init-tagger: Public ordinary functions
Function, init-word-test: Private ordinary functions
Function, initial-tag: Public ordinary functions
Function, initialize-tokens: Private ordinary functions
Function, lexicon-entry-case-forms: Private ordinary functions
Function, lexicon-entry-id: Public ordinary functions
Function, lexicon-entry-p: Private ordinary functions
Function, lexicon-entry-roots: Public ordinary functions
Function, lexicon-entry-surface-forms: Public ordinary functions
Function, lexicon-entry-tag: Public ordinary functions
Function, lexicon-entry-tags: Public ordinary functions
Function, load-contextual-rules: Private ordinary functions
Function, load-lexical-rules: Private ordinary functions
Function, load-tagger-files: Private ordinary functions
Function, m: Private ordinary functions
Function, make-cases: Private ordinary functions
Function, make-concept: Public ordinary functions
Function, make-contextual-rule: Private ordinary functions
Function, make-lexical-rule: Private ordinary functions
Function, make-lexicon-entry: Private ordinary functions
Function, make-meta: Private ordinary functions
Function, make-phrase: Public ordinary functions
Function, make-phrase-from-sentence: Public ordinary functions
Function, make-phrase-from-vdoc: Public ordinary functions
Function, make-vector-document: Public ordinary functions
Function, meta-char: Private ordinary functions
Function, meta-form: Private ordinary functions
Function, meta-p: Private ordinary functions
Function, meta-reader: Private ordinary functions
Function, morph-case-surface-forms: Public ordinary functions
Function, morph-surface-forms: Public ordinary functions
Function, morph-surface-forms-text: Public ordinary functions
Function, person-token-offset: Private ordinary functions
Function, phrase->concept: Public ordinary functions
Function, phrase-words: Public ordinary functions
Function, print-token-array: Private ordinary functions
Function, r: Private ordinary functions
Function, read-and-tag-file: Public ordinary functions
Function, read-config: Private ordinary functions
Function, read-file-as-tagged-document: Public ordinary functions
Function, read-file-to-string: Private ordinary functions
Function, relative-pathname: Private ordinary functions
Function, reset-langutils: Public ordinary functions
Function, reset-token-counts: Private ordinary functions
Function, return-vector-doc: Private ordinary functions
Function, root-noun: Public ordinary functions
Function, root-nouns: Public ordinary functions
Function, select-token: Private ordinary functions
Function, set-lexicon-entry: Private ordinary functions
Function, setto: Private ordinary functions
Function, stem: Private ordinary functions
Function, step1ab: Private ordinary functions
Function, step1c: Private ordinary functions
Function, step2: Private ordinary functions
Function, step3: Private ordinary functions
Function, step4: Private ordinary functions
Function, step5: Private ordinary functions
Function, stopword?: Public ordinary functions
Function, string->concept: Public ordinary functions
Function, string->token-array: Public ordinary functions
Function, string-concise-stopword?: Public ordinary functions
Function, string-contains-is?: Public ordinary functions
Function, string-stopword?: Public ordinary functions
Function, string-tag: Public ordinary functions
Function, string-tag-tokenized: Public ordinary functions
Function, suspicious-string?: Public ordinary functions
Function, symbol-name-equal: Private ordinary functions
Function, tag: Public ordinary functions
Function, tag-tokenized: Public ordinary functions
Function, temp-phrase: Private ordinary functions
Function, test-concept-equality: Private ordinary functions
Function, test-phrase: Private ordinary functions
Function, test-vector-tag-tokenized: Private ordinary functions
Function, token-array->concept: Public ordinary functions
Function, token-array->words: Private ordinary functions
Function, token-counter-hook: Private ordinary functions
Function, token-for-id: Public ordinary functions
Function, token-for-id-hook: Private ordinary functions
Function, tokenize-file2: Private ordinary functions
Function, tokenize-stream: Public ordinary functions
Function, tokenize-string: Public ordinary functions
Function, tokens-for-ids: Public ordinary functions
Function, vector-document: Public ordinary functions
Function, vector-tag: Public ordinary functions
Function, vector-tag-tokenized: Public ordinary functions
Function, vowelinstem: Private ordinary functions
Function, words->concept: Public ordinary functions
Function, write-temp: Private ordinary functions

G
gen-rule-arg-bindings: Private ordinary functions
gen-rule-arg-decls: Private ordinary functions
gen-rule-closure: Private ordinary functions
gen-rule-closure-decl: Private ordinary functions
gen-rule-match: Private ordinary functions
Generic Function, (setf altered-phrase-custom-document): Private generic functions
Generic Function, (setf document-annotations): Public generic functions
Generic Function, (setf document-tags): Public generic functions
Generic Function, (setf document-text): Public generic functions
Generic Function, (setf phrase-annotations): Private generic functions
Generic Function, (setf phrase-document): Public generic functions
Generic Function, (setf phrase-end): Public generic functions
Generic Function, (setf phrase-start): Public generic functions
Generic Function, (setf phrase-type): Public generic functions
Generic Function, add-word: Public generic functions
Generic Function, altered-phrase-custom-document: Private generic functions
Generic Function, change-word: Public generic functions
Generic Function, clear-concept-cache: Private generic functions
Generic Function, concat-concepts: Public generic functions
Generic Function, concept->string: Public generic functions
Generic Function, concept->token-array: Public generic functions
Generic Function, concept->words: Public generic functions
Generic Function, concept-contains: Public generic functions
Generic Function, conceptually-equal: Public generic functions
Generic Function, copy-phrase: Private generic functions
Generic Function, document-annotations: Public generic functions
Generic Function, document-tags: Public generic functions
Generic Function, document-text: Public generic functions
Generic Function, document-window-as-string: Private generic functions
Generic Function, find-phrase: Public generic functions
Generic Function, find-phrase-intervals: Public generic functions
Generic Function, get-adverb-chunks: Public generic functions
Generic Function, get-annotation: Public generic functions
Generic Function, get-basic-chunks: Private generic functions
Generic Function, get-event-chunks: Public generic functions
Generic Function, get-extended-event-chunks1: Public generic functions
Generic Function, get-extended-event-chunks2: Public generic functions
Generic Function, get-imperative-chunks: Public generic functions
Generic Function, get-nx-chunks: Public generic functions
Generic Function, get-p-chunks: Public generic functions
Generic Function, get-pp-chunks: Public generic functions
Generic Function, get-tag: Public generic functions
Generic Function, get-token-id: Public generic functions
Generic Function, get-vx-chunks: Public generic functions
Generic Function, lemmatize: Public generic functions
Generic Function, lemmatize-phrase: Public generic functions
Generic Function, length-of: Public generic functions
Generic Function, lookup-canonical-concept-instance: Private generic functions
Generic Function, make-alterable-phrase: Public generic functions
Generic Function, make-document-from-phrase: Private generic functions
Generic Function, phrase->string: Public generic functions
Generic Function, phrase->token-array: Public generic functions
Generic Function, phrase-annotations: Private generic functions
Generic Function, phrase-distance: Public generic functions
Generic Function, phrase-document: Public generic functions
Generic Function, phrase-end: Public generic functions
Generic Function, phrase-equal: Public generic functions
Generic Function, phrase-lemmas: Public generic functions
Generic Function, phrase-length: Public generic functions
Generic Function, phrase-overlap: Public generic functions
Generic Function, phrase-start: Public generic functions
Generic Function, phrase-type: Public generic functions
Generic Function, print-phrase: Public generic functions
Generic Function, print-phrase-lemmas: Public generic functions
Generic Function, print-vector-document: Public generic functions
Generic Function, print-window: Public generic functions
Generic Function, read-vector-document: Public generic functions
Generic Function, read-vector-document-to-string: Public generic functions
Generic Function, register-new-concept-instance: Private generic functions
Generic Function, remove-word: Public generic functions
Generic Function, set-annotation: Public generic functions
Generic Function, suspicious-word?: Public generic functions
Generic Function, token-vector: Public generic functions
Generic Function, unset-annotation: Public generic functions
Generic Function, vector-doc-as-ids: Private generic functions
Generic Function, vector-doc-as-words: Private generic functions
Generic Function, vector-document-string: Public generic functions
Generic Function, vector-document-words: Public generic functions
Generic Function, write-vector-document: Public generic functions
get-adverb-chunks: Public generic functions
get-adverb-chunks: Public generic functions
get-annotation: Public generic functions
get-annotation: Public generic functions
get-annotation: Public generic functions
get-basic-chunks: Private generic functions
get-basic-chunks: Private generic functions
get-bind-entry: Private ordinary functions
get-event-chunks: Public generic functions
get-event-chunks: Public generic functions
get-extended-event-chunks1: Public generic functions
get-extended-event-chunks1: Public generic functions
get-extended-event-chunks2: Public generic functions
get-extended-event-chunks2: Public generic functions
get-imperative-chunks: Public generic functions
get-imperative-chunks: Public generic functions
get-lemma: Public ordinary functions
get-lemma-for-id: Public ordinary functions
get-lexicon-case-forms: Public ordinary functions
get-lexicon-default-pos: Public ordinary functions
get-lexicon-entry: Public ordinary functions
get-nx-chunks: Public generic functions
get-nx-chunks: Public generic functions
get-p-chunks: Public generic functions
get-p-chunks: Public generic functions
get-pp-chunks: Public generic functions
get-pp-chunks: Public generic functions
get-tag: Public generic functions
get-tag: Public generic functions
get-tag: Public generic functions
get-tag: Public generic functions
get-token-count: Public ordinary functions
get-token-id: Public generic functions
get-token-id: Public generic functions
get-token-id: Public generic functions
get-token-id: Public generic functions
get-vx-chunks: Public generic functions
get-vx-chunks: Public generic functions
guess-tag: Private ordinary functions

H
handle-config-entry: Private ordinary functions
head-verb: Public ordinary functions
head-verbs: Public ordinary functions

I
id-for-token: Public ordinary functions
id-for-token-hook: Private ordinary functions
ids-for-string: Private ordinary functions
ids-for-tokens: Public ordinary functions
in-pos-class?: Public ordinary functions
init-concise-stopwords: Private ordinary functions
init-langutils: Public ordinary functions
init-lexicon: Private ordinary functions
init-stopwords: Private ordinary functions
init-tagger: Public ordinary functions
init-word-test: Private ordinary functions
initial-tag: Public ordinary functions
initialize-tokens: Private ordinary functions

L
lemmatize: Public generic functions
lemmatize: Public generic functions
lemmatize: Public generic functions
lemmatize-phrase: Public generic functions
lemmatize-phrase: Public generic functions
lemmatize-phrase: Public generic functions
length-of: Public generic functions
length-of: Public generic functions
lexicon-entry-case-forms: Private ordinary functions
lexicon-entry-id: Public ordinary functions
lexicon-entry-p: Private ordinary functions
lexicon-entry-roots: Public ordinary functions
lexicon-entry-surface-forms: Public ordinary functions
lexicon-entry-tag: Public ordinary functions
lexicon-entry-tags: Public ordinary functions
list-match: Private macros
list-match-type: Private macros
load-contextual-rules: Private ordinary functions
load-lexical-rules: Private ordinary functions
load-tagger-files: Private ordinary functions
lookup-canonical-concept-instance: Private generic functions
lookup-canonical-concept-instance: Private generic functions
lookup-canonical-concept-instance: Private generic functions

M
m: Private ordinary functions
Macro, def-contextual-rule-parser: Private macros
Macro, list-match: Private macros
Macro, list-match-type: Private macros
Macro, stream-match: Private macros
Macro, stream-match-type: Private macros
Macro, string-match: Private macros
Macro, string-match-type: Private macros
Macro, with-list-meta: Public macros
Macro, with-static-memory-allocation: Private macros
Macro, with-stream-meta: Public macros
Macro, with-string-meta: Public macros
Macro, write-log: Private macros
make-alterable-phrase: Public generic functions
make-alterable-phrase: Public generic functions
make-cases: Private ordinary functions
make-concept: Public ordinary functions
make-contextual-rule: Private ordinary functions
make-document-from-phrase: Private generic functions
make-document-from-phrase: Private generic functions
make-lexical-rule: Private ordinary functions
make-lexicon-entry: Private ordinary functions
make-meta: Private ordinary functions
make-phrase: Public ordinary functions
make-phrase-from-sentence: Public ordinary functions
make-phrase-from-vdoc: Public ordinary functions
make-vector-document: Public ordinary functions
meta-char: Private ordinary functions
meta-form: Private ordinary functions
meta-p: Private ordinary functions
meta-reader: Private ordinary functions
Method, (setf altered-phrase-custom-document): Private generic functions
Method, (setf document-annotations): Public generic functions
Method, (setf document-tags): Public generic functions
Method, (setf document-text): Public generic functions
Method, (setf phrase-annotations): Private generic functions
Method, (setf phrase-document): Public generic functions
Method, (setf phrase-end): Public generic functions
Method, (setf phrase-start): Public generic functions
Method, (setf phrase-type): Public generic functions
Method, add-word: Public generic functions
Method, altered-phrase-custom-document: Private generic functions
Method, change-word: Public generic functions
Method, change-word: Public generic functions
Method, clear-concept-cache: Private generic functions
Method, concat-concepts: Public generic functions
Method, concept->string: Public generic functions
Method, concept->token-array: Public generic functions
Method, concept->words: Public generic functions
Method, concept-contains: Public generic functions
Method, conceptually-equal: Public generic functions
Method, conceptually-equal: Public generic functions
Method, conceptually-equal: Public generic functions
Method, conceptually-equal: Public generic functions
Method, copy-phrase: Private generic functions
Method, document-annotations: Public generic functions
Method, document-tags: Public generic functions
Method, document-text: Public generic functions
Method, document-window-as-string: Private generic functions
Method, find-phrase: Public generic functions
Method, find-phrase-intervals: Public generic functions
Method, find-phrase-intervals: Public generic functions
Method, get-adverb-chunks: Public generic functions
Method, get-annotation: Public generic functions
Method, get-annotation: Public generic functions
Method, get-basic-chunks: Private generic functions
Method, get-event-chunks: Public generic functions
Method, get-extended-event-chunks1: Public generic functions
Method, get-extended-event-chunks2: Public generic functions
Method, get-imperative-chunks: Public generic functions
Method, get-nx-chunks: Public generic functions
Method, get-p-chunks: Public generic functions
Method, get-pp-chunks: Public generic functions
Method, get-tag: Public generic functions
Method, get-tag: Public generic functions
Method, get-tag: Public generic functions
Method, get-token-id: Public generic functions
Method, get-token-id: Public generic functions
Method, get-token-id: Public generic functions
Method, get-vx-chunks: Public generic functions
Method, lemmatize: Public generic functions
Method, lemmatize: Public generic functions
Method, lemmatize-phrase: Public generic functions
Method, lemmatize-phrase: Public generic functions
Method, length-of: Public generic functions
Method, lookup-canonical-concept-instance: Private generic functions
Method, lookup-canonical-concept-instance: Private generic functions
Method, make-alterable-phrase: Public generic functions
Method, make-document-from-phrase: Private generic functions
Method, phrase->string: Public generic functions
Method, phrase->token-array: Public generic functions
Method, phrase-annotations: Private generic functions
Method, phrase-distance: Public generic functions
Method, phrase-document: Public generic functions
Method, phrase-document: Public generic functions
Method, phrase-end: Public generic functions
Method, phrase-end: Public generic functions
Method, phrase-equal: Public generic functions
Method, phrase-lemmas: Public generic functions
Method, phrase-length: Public generic functions
Method, phrase-length: Public generic functions
Method, phrase-overlap: Public generic functions
Method, phrase-start: Public generic functions
Method, phrase-start: Public generic functions
Method, phrase-type: Public generic functions
Method, print-object: Public standalone methods
Method, print-object: Public standalone methods
Method, print-object: Public standalone methods
Method, print-phrase: Public generic functions
Method, print-phrase-lemmas: Public generic functions
Method, print-vector-document: Public generic functions
Method, print-window: Public generic functions
Method, read-vector-document: Public generic functions
Method, read-vector-document-to-string: Public generic functions
Method, register-new-concept-instance: Private generic functions
Method, remove-word: Public generic functions
Method, remove-word: Public generic functions
Method, set-annotation: Public generic functions
Method, set-annotation: Public generic functions
Method, suspicious-word?: Public generic functions
Method, token-vector: Public generic functions
Method, unset-annotation: Public generic functions
Method, unset-annotation: Public generic functions
Method, vector-doc-as-ids: Private generic functions
Method, vector-doc-as-words: Private generic functions
Method, vector-document-string: Public generic functions
Method, vector-document-words: Public generic functions
Method, write-vector-document: Public generic functions
morph-case-surface-forms: Public ordinary functions
morph-surface-forms: Public ordinary functions
morph-surface-forms-text: Public ordinary functions

P
person-token-offset: Private ordinary functions
phrase->concept: Public ordinary functions
phrase->string: Public generic functions
phrase->string: Public generic functions
phrase->token-array: Public generic functions
phrase->token-array: Public generic functions
phrase-annotations: Private generic functions
phrase-annotations: Private generic functions
phrase-distance: Public generic functions
phrase-distance: Public generic functions
phrase-document: Public generic functions
phrase-document: Public generic functions
phrase-document: Public generic functions
phrase-end: Public generic functions
phrase-end: Public generic functions
phrase-end: Public generic functions
phrase-equal: Public generic functions
phrase-equal: Public generic functions
phrase-lemmas: Public generic functions
phrase-lemmas: Public generic functions
phrase-length: Public generic functions
phrase-length: Public generic functions
phrase-length: Public generic functions
phrase-overlap: Public generic functions
phrase-overlap: Public generic functions
phrase-start: Public generic functions
phrase-start: Public generic functions
phrase-start: Public generic functions
phrase-type: Public generic functions
phrase-type: Public generic functions
phrase-words: Public ordinary functions
print-object: Public standalone methods
print-object: Public standalone methods
print-object: Public standalone methods
print-phrase: Public generic functions
print-phrase: Public generic functions
print-phrase-lemmas: Public generic functions
print-phrase-lemmas: Public generic functions
print-token-array: Private ordinary functions
print-vector-document: Public generic functions
print-vector-document: Public generic functions
print-window: Public generic functions
print-window: Public generic functions

R
r: Private ordinary functions
read-and-tag-file: Public ordinary functions
read-config: Private ordinary functions
read-file-as-tagged-document: Public ordinary functions
read-file-to-string: Private ordinary functions
read-vector-document: Public generic functions
read-vector-document: Public generic functions
read-vector-document-to-string: Public generic functions
read-vector-document-to-string: Public generic functions
register-new-concept-instance: Private generic functions
register-new-concept-instance: Private generic functions
relative-pathname: Private ordinary functions
remove-word: Public generic functions
remove-word: Public generic functions
remove-word: Public generic functions
reset-langutils: Public ordinary functions
reset-token-counts: Private ordinary functions
return-vector-doc: Private ordinary functions
root-noun: Public ordinary functions
root-nouns: Public ordinary functions

S
select-token: Private ordinary functions
set-annotation: Public generic functions
set-annotation: Public generic functions
set-annotation: Public generic functions
set-lexicon-entry: Private ordinary functions
Setf Expander, (setf get-lexicon-entry): Public setf expanders
setto: Private ordinary functions
stem: Private ordinary functions
step1ab: Private ordinary functions
step1c: Private ordinary functions
step2: Private ordinary functions
step3: Private ordinary functions
step4: Private ordinary functions
step5: Private ordinary functions
stopword?: Public ordinary functions
stream-match: Private macros
stream-match-type: Private macros
string->concept: Public ordinary functions
string->token-array: Public ordinary functions
string-concise-stopword?: Public ordinary functions
string-contains-is?: Public ordinary functions
string-match: Private macros
string-match-type: Private macros
string-stopword?: Public ordinary functions
string-tag: Public ordinary functions
string-tag-tokenized: Public ordinary functions
suspicious-string?: Public ordinary functions
suspicious-word?: Public generic functions
suspicious-word?: Public generic functions
symbol-name-equal: Private ordinary functions

T
tag: Public ordinary functions
tag-tokenized: Public ordinary functions
temp-phrase: Private ordinary functions
test-concept-equality: Private ordinary functions
test-phrase: Private ordinary functions
test-vector-tag-tokenized: Private ordinary functions
token-array->concept: Public ordinary functions
token-array->words: Private ordinary functions
token-counter-hook: Private ordinary functions
token-for-id: Public ordinary functions
token-for-id-hook: Private ordinary functions
token-vector: Public generic functions
token-vector: Public generic functions
tokenize-file2: Private ordinary functions
tokenize-stream: Public ordinary functions
tokenize-string: Public ordinary functions
tokens-for-ids: Public ordinary functions

U
unset-annotation: Public generic functions
unset-annotation: Public generic functions
unset-annotation: Public generic functions

V
vector-doc-as-ids: Private generic functions
vector-doc-as-ids: Private generic functions
vector-doc-as-words: Private generic functions
vector-doc-as-words: Private generic functions
vector-document: Public ordinary functions
vector-document-string: Public generic functions
vector-document-string: Public generic functions
vector-document-words: Public generic functions
vector-document-words: Public generic functions
vector-tag: Public ordinary functions
vector-tag-tokenized: Public ordinary functions
vowelinstem: Private ordinary functions

W
with-list-meta: Public macros
with-static-memory-allocation: Private macros
with-stream-meta: Public macros
with-string-meta: Public macros
words->concept: Public ordinary functions
write-log: Private macros
write-temp: Private ordinary functions
write-vector-document: Public generic functions
write-vector-document: Public generic functions


A.3 Variables

Jump to:   *  
A   C   D   E   F   I   K   N   P   R   S   T   V  
Index Entry  Section

*
*add-to-map-hook*: Private special variables
*auto-init*: Private special variables
*common-verbs*: Private special variables
*concept-store-scratch-array*: Private special variables
*concept-vhash*: Private special variables
*concise-stopwords*: Private special variables
*config-paths*: Private special variables
*contextual-rule-args*: Private special variables
*default-concise-stopwords-file*: Private special variables
*default-contextual-rule-file*: Private special variables
*default-lexical-rule-file*: Private special variables
*default-lexicon-file*: Private special variables
*default-stems-file*: Private special variables
*default-stopwords-file*: Private special variables
*default-token-map-file*: Private special variables
*external-token-map*: Private special variables
*id-for-token-hook*: Private special variables
*id-table*: Private special variables
*is-token*: Private special variables
*lexicon*: Private special variables
*max-token-nums*: Private constants
*max-token-others*: Private constants
*meta-readtable*: Private special variables
*pos-class-map*: Private special variables
*report-status*: Private special variables
*s-token*: Private special variables
*saved-readtable*: Private special variables
*stopwords*: Private special variables
*suspicious-words*: Private special variables
*tagger-bigrams*: Private special variables
*tagger-contextual-rules*: Private special variables
*tagger-lexical-rules*: Private special variables
*tagger-wordlist*: Private special variables
*temp-phrase*: Private special variables
*test*: Private special variables
*token-counter*: Private special variables
*token-counter-hook*: Private special variables
*token-dirty-bit*: Private special variables
*token-for-id-hook*: Private special variables
*token-table*: Private special variables
*tokens-load-file*: Private special variables
*whitespace-chars*: Private constants

A
adv-pattern: Private constants
annotations: Public classes
annotations: Public classes

C
case-forms: Public structures
char: Private structures
Constant, *max-token-nums*: Private constants
Constant, *max-token-others*: Private constants
Constant, *whitespace-chars*: Private constants
Constant, adv-pattern: Private constants
Constant, noun-pattern: Private constants
Constant, p-pattern: Private constants
Constant, verb-pattern: Private constants
custom-document: Public classes

D
document: Public classes

E
end: Public classes

F
form: Private structures

I
id: Public structures

K
known-abbreviations: Private special variables

N
noun-pattern: Private constants

P
p-pattern: Private constants

R
roots: Public structures

S
Slot, annotations: Public classes
Slot, annotations: Public classes
Slot, case-forms: Public structures
Slot, char: Private structures
Slot, custom-document: Public classes
Slot, document: Public classes
Slot, end: Public classes
Slot, form: Private structures
Slot, id: Public structures
Slot, roots: Public structures
Slot, start: Public classes
Slot, surface-forms: Public structures
Slot, tags: Public structures
Slot, tags: Public classes
Slot, text: Public classes
Slot, token-vector: Private classes
Slot, type: Public classes
Special Variable, *add-to-map-hook*: Private special variables
Special Variable, *auto-init*: Private special variables
Special Variable, *common-verbs*: Private special variables
Special Variable, *concept-store-scratch-array*: Private special variables
Special Variable, *concept-vhash*: Private special variables
Special Variable, *concise-stopwords*: Private special variables
Special Variable, *config-paths*: Private special variables
Special Variable, *contextual-rule-args*: Private special variables
Special Variable, *default-concise-stopwords-file*: Private special variables
Special Variable, *default-contextual-rule-file*: Private special variables
Special Variable, *default-lexical-rule-file*: Private special variables
Special Variable, *default-lexicon-file*: Private special variables
Special Variable, *default-stems-file*: Private special variables
Special Variable, *default-stopwords-file*: Private special variables
Special Variable, *default-token-map-file*: Private special variables
Special Variable, *external-token-map*: Private special variables
Special Variable, *id-for-token-hook*: Private special variables
Special Variable, *id-table*: Private special variables
Special Variable, *is-token*: Private special variables
Special Variable, *lexicon*: Private special variables
Special Variable, *meta-readtable*: Private special variables
Special Variable, *pos-class-map*: Private special variables
Special Variable, *report-status*: Private special variables
Special Variable, *s-token*: Private special variables
Special Variable, *saved-readtable*: Private special variables
Special Variable, *stopwords*: Private special variables
Special Variable, *suspicious-words*: Private special variables
Special Variable, *tagger-bigrams*: Private special variables
Special Variable, *tagger-contextual-rules*: Private special variables
Special Variable, *tagger-lexical-rules*: Private special variables
Special Variable, *tagger-wordlist*: Private special variables
Special Variable, *temp-phrase*: Private special variables
Special Variable, *test*: Private special variables
Special Variable, *token-counter*: Private special variables
Special Variable, *token-counter-hook*: Private special variables
Special Variable, *token-dirty-bit*: Private special variables
Special Variable, *token-for-id-hook*: Private special variables
Special Variable, *token-table*: Private special variables
Special Variable, *tokens-load-file*: Private special variables
Special Variable, known-abbreviations: Private special variables
start: Public classes
surface-forms: Public structures

T
tags: Public structures
tags: Public classes
text: Public classes
token-vector: Private classes
type: Public classes

V
verb-pattern: Private constants


A.4 Data types

Jump to:   A   C   D   E   F   I   L   M   N   P   R   S   T   V   W  
Index Entry  Section

A
alpha: Private types
alpha-lower: Private types
alpha-upper: Private types
alphanum: Private types
altered-phrase: Public classes

C
chunker-constants.lisp: The langutils/src/chunker-constants․lisp file
chunker.lisp: The langutils/src/chunker․lisp file
Class, altered-phrase: Public classes
Class, concept: Private classes
Class, phrase: Public classes
Class, vector-document: Public classes
concept: Private classes
concept.lisp: The langutils/src/concept․lisp file
Condition, end-of-sentence: Private conditions
config.lisp: The langutils/src/config․lisp file
contextual-rule-parser.lisp: The langutils/src/contextual-rule-parser․lisp file

D
digit: Private types

E
end-of-sentence: Private conditions

F
File, chunker-constants.lisp: The langutils/src/chunker-constants․lisp file
File, chunker.lisp: The langutils/src/chunker․lisp file
File, concept.lisp: The langutils/src/concept․lisp file
File, config.lisp: The langutils/src/config․lisp file
File, contextual-rule-parser.lisp: The langutils/src/contextual-rule-parser․lisp file
File, init.lisp: The langutils/src/init․lisp file
File, langutils.asd: The langutils/langutils․asd file
File, lemma.lisp: The langutils/src/lemma․lisp file
File, lexicon.lisp: The langutils/src/lexicon․lisp file
File, my-meta.lisp: The langutils/src/my-meta․lisp file
File, package.lisp: The langutils/src/package․lisp file
File, porter.lisp: The langutils/src/porter․lisp file
File, reference.lisp: The langutils/src/reference․lisp file
File, stopwords.lisp: The langutils/src/stopwords․lisp file
File, tagger-data.lisp: The langutils/src/tagger-data․lisp file
File, tagger.lisp: The langutils/src/tagger․lisp file
File, tokenize.lisp: The langutils/src/tokenize․lisp file
File, tokens.lisp: The langutils/src/tokens․lisp file

I
init.lisp: The langutils/src/init․lisp file

L
langutils: The langutils system
langutils: The langutils package
langutils-tokenize: The langutils-tokenize package
langutils.asd: The langutils/langutils․asd file
langutils.system: The langutils․system package
lemma.lisp: The langutils/src/lemma․lisp file
lexicon-entry: Public structures
lexicon.lisp: The langutils/src/lexicon․lisp file

M
meta: Private structures
Module, src: The langutils/src module
my-meta: The my-meta package
my-meta.lisp: The langutils/src/my-meta․lisp file

N
non-digit: Private types
non-digit-or-ws: Private types
non-punc-or-white: Private types
non-whitespace: Private types

P
Package, langutils: The langutils package
Package, langutils-tokenize: The langutils-tokenize package
Package, langutils.system: The langutils․system package
Package, my-meta: The my-meta package
package.lisp: The langutils/src/package․lisp file
phrase: Public classes
porter.lisp: The langutils/src/porter․lisp file
punctuation: Private types

R
reference.lisp: The langutils/src/reference․lisp file

S
src: The langutils/src module
stopwords.lisp: The langutils/src/stopwords․lisp file
Structure, lexicon-entry: Public structures
Structure, meta: Private structures
System, langutils: The langutils system

T
tagger-data.lisp: The langutils/src/tagger-data․lisp file
tagger.lisp: The langutils/src/tagger․lisp file
tokenize.lisp: The langutils/src/tokenize․lisp file
tokens.lisp: The langutils/src/tokens․lisp file
Type, alpha: Private types
Type, alpha-lower: Private types
Type, alpha-upper: Private types
Type, alphanum: Private types
Type, digit: Private types
Type, non-digit: Private types
Type, non-digit-or-ws: Private types
Type, non-punc-or-white: Private types
Type, non-whitespace: Private types
Type, punctuation: Private types
Type, whitespace: Private types

V
vector-document: Public classes

W
whitespace: Private types