The lassie Reference Manual

Parent

lassie (system)

Location

mapper.lisp

Internal Definitions

compose-mappers (function)
concatente-mappers (function)
curry-mapper (function)
encode-mapper (function)
make-encoded-term-document-lister (function)
make-encoded-term-document-mapper (function)
make-mapper (function)
map-lines (generic function)
map-lines (method)
map-lines (method)
map-lines (method)
null-mapper (function)
test-compose-mappers (function)

3.1.6 `lassie/assemble.lisp`

Dependency

mapper.lisp (file)

Parent

lassie (system)

Location

assemble.lisp

Exported Definitions

assemble-co-occurrence-matrix (generic function)
assemble-co-occurrence-matrix (method)
assemble-co-occurrence-matrix (method)
assemble-document-vector (generic function)
assemble-document-vector (method)
assemble-document-vector (method)
assemble-term-vector (generic function)
assemble-term-vector (method)
lsa-assembler (class)
ri-term-assembler (class)

Internal Definitions

assemble-occurence-vector (function)
incf-and-maybe-grow (function)
make-lsa-assembler (function)
make-ri-term-assembler (function)
n-documents (method)
n-terms (method)

3.1.7 `lassie/lsa.lisp`

Dependency

assemble.lisp (file)

Parent

lassie (system)

Location

lsa.lisp

Exported Definitions

construct-document-vector (function)
construct-term-vector (function)
document->vector (function)
document-features (function)
document-indexer (method)
document-mapper (method)
document-vector-features (function)
load-lsa (function)
lsa (function)
lsa (class)
normalizer (method)
save-lsa (function)
svd (method)
term->vector (function)
term-features (function)
term-indexer (method)
term-mapper (method)
term-vector-features (function)

Internal Definitions

assembler (method)
construct-lsa-vector (function)
coordinate (function)
extract-lsa-features (function)
extract-svd-features (function)
inner* (function)
single-float-vector (type)

3.1.8 `lassie/lsa-extra.lisp`

Dependency

lsa.lisp (file)

Parent

lassie (system)

Location

lsa-extra.lisp

Exported Definitions

cosine-similarity (function)
most-similar-documents (function)

Internal Definitions

insert-into-sorted-vector (function)

4 Packages

Packages are listed by definition order.

4.1 `lassie`

The core functionality of Lassie.

Source

package.lisp (file)

Use List

lassie.assembler
lassie.indexer
common-lisp

Exported Definitions

construct-document-vector (function)
construct-term-vector (function)
cosine-similarity (function)
document->vector (function)
document-features (function)
document-indexer (generic function)
document-indexer (method)
document-mapper (generic function)
document-mapper (method)
document-vector-features (function)
load-lsa (function)
lsa (function)
lsa (class)
most-similar-documents (function)
normalizer (generic function)
normalizer (method)
save-lsa (function)
svd (generic function)
svd (method)
term->vector (function)
term-features (function)
term-indexer (generic function)
term-indexer (method)
term-mapper (generic function)
term-mapper (method)
term-vector-features (function)

Internal Definitions

assembler (generic function)
assembler (method)
compose-mappers (function)
concatente-mappers (function)
construct-lsa-vector (function)
coordinate (function)
curry-mapper (function)
encode-mapper (function)
extract-lsa-features (function)
extract-svd-features (function)
inner* (function)
insert-into-sorted-vector (function)
make-encoded-term-document-lister (function)
make-encoded-term-document-mapper (function)
make-mapper (function)
map-lines (generic function)
map-lines (method)
map-lines (method)
map-lines (method)
null-mapper (function)
single-float-vector (type)
test-compose-mappers (function)

4.2 `lassie.assembler`

Different assemblers and normalizers that one plug
into Lassie. Assemblers to construct a co-occurence matrix or document vector from a corpus, and normalizers to perform post processing on the assembled data. Normalizers can be printed and read readably.

Source

package.lisp (file)

Use List

common-lisp

Used By List

Exported Definitions

assemble-co-occurrence-matrix (generic function)
assemble-co-occurrence-matrix (method)
assemble-co-occurrence-matrix (method)
assemble-document-vector (generic function)
assemble-document-vector (method)
assemble-document-vector (method)
assemble-term-vector (generic function)
assemble-term-vector (method)
column-power-normalizer (class)
lsa-assembler (class)
normalize-document-vector (generic function)
normalize-document-vector (method)
normalize-document-vector (method)
normalize-document-vector (method)
normalize-document-vector (method)
normalize-document-vector (method)
normalize-document-vector (method)
normalize-matrix (generic function)
normalize-matrix (method)
normalize-matrix (method)
normalize-matrix (method)
normalize-matrix (method)
normalize-matrix (method)
normalize-matrix (method)
normalize-term-vector (generic function)
null-normalizer (class)
pmi-normalizer (class)
ri-term-assembler (class)
row-centering-normalizer (class)
sign-normalizer (class)
tf-idf-normalizer (class)

Internal Definitions

assemble-occurence-vector (function)
class-counts (generic function)
class-counts (method)
column-norm (function)
document-class-fn (generic function)
document-class-fn (method)
hash-table= (function)
idfs (generic function)
idfs (method)
incf-and-maybe-grow (function)
inverse-document-frequency (function)
make-column-power-normalizer (function)
make-lsa-assembler (function)
make-null-normalizer (function)
make-ri-term-assembler (function)
make-tf-idf-normalizer (function)
n-columns (function)
n-documents (generic function)
n-documents (method)
n-documents (method)
n-documents-with-term (function)
n-rows (function)
n-terms (generic function)
n-terms (method)
norm (function)
normalize-column (function)
normalize-vector (function)
power (generic function)
power (method)
row-average (function)
row-averages (generic function)
row-averages (method)
sum-column (function)
sum-matrix (function)
sum-row (function)
sum-vector (function)
term-counts (generic function)
term-counts (method)
term-counts-per-class (generic function)
term-counts-per-class (method)
term-frequency (function)
term-total (generic function)
term-total (method)
term-total-per-class (generic function)
term-total-per-class (method)
test-pmi-normalizer (function)
test-tf-idf-normalizer (function)

4.3 `lassie.indexer`

Indexers provide a - sometimes reversible - mapping
from objects and indices. The word ‘index’ is used here in a very general sense, random indexers, for instance, map to a set of indices. Within Lassie they are used in conjunction with assemblers that know how to change the co-occurence matrix when encountering given an index. They can be printed and read readably.

Source

package.lisp (file)

Use List

common-lisp

Used By List

Exported Definitions

->index (generic function)
->index (method)
->index (method)
->index (method)
<-index (generic function)
<-index (method)
<-index (method)
make-counting-indexer (function)
make-hashing-indexer (function)
make-random-indexer (function)

Internal Definitions

%make-hashing-indexer (function)
%make-random-indexer (function)
alist->hashing-indexer (function)
copy-counting-indexer (function)
copy-hashing-indexer (function)
copy-random-indexer (function)
counting-indexer (structure)
counting-indexer-count (function)
(setf counting-indexer-count) (function)
counting-indexer-p (function)
hashing-indexer (structure)
hashing-indexer->alist (function)
hashing-indexer-index->object (function)
(setf hashing-indexer-index->object) (function)
hashing-indexer-next-index (function)
(setf hashing-indexer-next-index) (function)
hashing-indexer-object->index (function)
(setf hashing-indexer-object->index) (function)
hashing-indexer-p (function)
make-index-vector (function)
random-indexer (structure)
random-indexer-length (function)
(setf random-indexer-length) (function)
random-indexer-n (function)
(setf random-indexer-n) (function)
random-indexer-object->index (function)
(setf random-indexer-object->index) (function)
random-indexer-p (function)
test-hashing-indexer (function)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Functions

Function: construct-document-vector LSA FEATURES

Construct a document vector from FEATURES. Inverse of DOCUMENT-VECTOR-FEATURES.

Package: lassie
Source: lsa.lisp (file)

Function: construct-term-vector LSA FEATURES

Construct a term vector from FEATURES. Inverse of TERM-VECTOR-FEATURES.

Package: lassie
Source: lsa.lisp (file)

Function: cosine-similarity X Y &key N

Package: lassie
Source: lsa-extra.lisp (file)

Function: document->vector LSA DOCUMENT

Turn DOCUMENT into a document vector.

Package: lassie
Source: lsa.lisp (file)

Function: document-features LSA DOCUMENT

Convenience function that returns the features of DOCUMENT after turning into into a vector with LSA.

Package: lassie
Source: lsa.lisp (file)

Function: document-vector-features LSA VECTOR-OR-INDEX

Return the feature vector for the document given by document VECTOR or INDEX.

Package: lassie
Source: lsa.lisp (file)

Function: load-lsa &key FILENAME SVD-FILENAME

Return the lsa loaded from FILENAME and SVD-FILENAME.

Package: lassie
Source: lsa.lisp (file)

Function: lsa &key TERM-MAPPER DOCUMENT-MAPPER TERM-LISTER DOCUMENT-LISTER TERM-INDEXER DOCUMENT-INDEXER ASSEMBLER NORMALIZER SUPERVISOR LEARNING-RATE NORMALIZATION-FACTOR

Perform LSA and return the lsa object that contains the SVD and remembers the mappers, indexers, ASSEMBLER and NORMALIZER for easy querying later by for example DOCUMENT-FEATURES.

This fat function assembles the co-occurrence matrix by iterating over all terms by TERM-LISTER and all documents by DOCUMENT-LISTER (either may be NIL). If DOCUMENT-LISTER is provided then DOCUMENT-MAPPER is employed to iterate over the terms of each document. Similarly TERM-MAPPER complements TERM-LISTER. TERM-INDEXER and DOCUMENT-INDEXER provide a - sometimes invertible - mapping from terms/documents to indices.

After the initial construction the mappers and indexers are stored in the LSA instance because they are needed to assemble term/document vectors later.

Finally the co-occurrence matrix is decomposed into singular vector pairs that define the feature spaces.

SUPERVISOR is a FSVD supervisor on which FSVD:SUPERVISE-SVD is invoked to control iteration (see FSVD:SVD). The lsa instance being constructed is passed as the :LSA argument to allow inspecting, saving, etc.

Package: lassie
Source: lsa.lisp (file)

Function: make-counting-indexer &key (COUNT COUNT)

Package: lassie.indexer
Source: indexer.lisp (file)

Function: make-hashing-indexer &key TEST

Package: lassie.indexer
Source: indexer.lisp (file)

Function: make-random-indexer &key LENGTH N TEST

Package: lassie.indexer
Source: indexer.lisp (file)

Function: most-similar-documents LSA DOCUMENT-FEATURES &key N TEST FILTER MEASURE

Return a vector of index and similarity pairs of the - at most N - documents whose features are most similar to DOCUMENT-FEATURES according to the similarity MEASURE.

Package: lassie
Source: lsa-extra.lisp (file)

Function: save-lsa LSA &key FILENAME SVD-FILENAME

Save LSA to FILENAME and its svd to SVD-FILENAME.

Package: lassie
Source: lsa.lisp (file)

Function: term->vector LSA TERM

Turn TERM into a document vector.

Package: lassie
Source: lsa.lisp (file)

Function: term-features LSA TERM

Convenience function that returns the features of TERM after turning into into a vector with LSA.

Package: lassie
Source: lsa.lisp (file)

Function: term-vector-features LSA VECTOR-OR-INDEX

Return the feature vector for the term given by term VECTOR or INDEX.

Package: lassie
Source: lsa.lisp (file)

5.1.2 Generic functions

Generic Function: ->index INDEXER OBJECT &key ALLOCATE-NEW-INDEX-P

Return an index representing OBJECT.

Package

Source

indexer.lisp (file)

Methods

Method: ->index (INDEXER random-indexer) OBJECT &key ALLOCATE-NEW-INDEX-P

Method: ->index (INDEXER hashing-indexer) OBJECT &key ALLOCATE-NEW-INDEX-P

Method: ->index (INDEXER counting-indexer) OBJECT &key ALLOCATE-NEW-INDEX-P

Generic Function: <-index INDEXER INDEX

Return the object that is encoded to INDEX.

Package

Source

indexer.lisp (file)

Methods

Method: <-index (INDEXER hashing-indexer) INDEX

Method: <-index (INDEXER counting-indexer) OBJECT

Generic Function: assemble-co-occurrence-matrix ASSEMBLER LISTER

Assemble MATRIX and remember how to perform the
same kind of activity on subsequent calls to ASSEMBLE-TERM-VECTOR and ASSEMBLE-DOCUMENT-VECTOR.

Package

Source

assemble.lisp (file)

Methods

Method: assemble-co-occurrence-matrix (ASSEMBLER ri-term-assembler) LISTER

Method: assemble-co-occurrence-matrix (ASSEMBLER lsa-assembler) LISTER

Generic Function: assemble-document-vector ASSEMBLER LISTER

Iterate over terms of LISTER and assemble a
document vector in the same way as the matrix was assembled previously.

Package

Source

assemble.lisp (file)

Methods

Method: assemble-document-vector (ASSEMBLER ri-term-assembler) LISTER

Method: assemble-document-vector (ASSEMBLER lsa-assembler) LISTER

Generic Function: assemble-term-vector ASSEMBLER LISTER

Iterate over documents of LISTER and assemble a
term vector in the same way as the matrix was assembled previously.

Package

Source

assemble.lisp (file)

Methods

Method: assemble-term-vector (ASSEMBLER lsa-assembler) LISTER

Generic Function: document-indexer OBJECT

Package

Methods

Method: document-indexer (LSA lsa)

Document indexer.

Source: lsa.lisp (file)

Generic Function: document-mapper OBJECT

Package

Methods

Method: document-mapper (LSA lsa)

A mapper over all terms that occur in a given document.

Source: lsa.lisp (file)

Generic Function: normalize-document-vector NORMALIZER DOCUMENT-VECTOR DOCUMENT

Returned the normalized DOCUMENT-VECTOR. Possibly desctructive.

Package

Source

Methods

Method: normalize-document-vector (NORMALIZER column-power-normalizer) DOCUMENT-VECTOR DOCUMENT

Method: normalize-document-vector (NORMALIZER row-centering-normalizer) DOCUMENT-VECTOR DOCUMENT

Method: normalize-document-vector (NORMALIZER null-normalizer) DOCUMENT-VECTOR DOCUMENT

Method: normalize-document-vector (NORMALIZER sign-normalizer) DOCUMENT-VECTOR DOCUMENT

Method: normalize-document-vector (NORMALIZER pmi-normalizer) DOCUMENT-VECTOR DOCUMENT

Method: normalize-document-vector (NORMALIZER tf-idf-normalizer) DOCUMENT-VECTOR DOCUMENT

Generic Function: normalize-matrix NORMALIZER MATRIX

Return the normalized MATRIX possibly destructively
and remember how to perform the same kind normalizations on subsequent calls to NORMALIZE-TERM and NORMALIZE-DOCUMENT.

Package

Source

Methods

Method: normalize-matrix (NORMALIZER column-power-normalizer) MATRIX

Method: normalize-matrix (NORMALIZER row-centering-normalizer) MATRIX

Method: normalize-matrix (NORMALIZER null-normalizer) MATRIX

Method: normalize-matrix (NORMALIZER sign-normalizer) MATRIX

Method: normalize-matrix (NORMALIZER pmi-normalizer) MATRIX

Method: normalize-matrix (NORMALIZER tf-idf-normalizer) MATRIX

Generic Function: normalize-term-vector NORMALIZER TERM-VECTOR TERM

Return the normalized TERM-VECTOR. Possibly desctructive.

Package: lassie.assembler
Source: normalizer.lisp (file)

Generic Function: normalizer OBJECT

Package

Methods

Method: normalizer (LSA lsa)

Performs some last minute transformations on the assembled matrix.

Source: lsa.lisp (file)

Generic Function: svd OBJECT

Package

Methods

Method: svd (LSA lsa)

The singular value decomposition.

Source: lsa.lisp (file)

Generic Function: term-indexer OBJECT

Package

Methods

Method: term-indexer (LSA lsa)

Term indexer.

Source: lsa.lisp (file)

Generic Function: term-mapper OBJECT

Package

Methods

Method: term-mapper (LSA lsa)

A mapper over all documents in which a given term occurs.

Source: lsa.lisp (file)

5.1.3 Classes

Class: column-power-normalizer ()

Package

Source

Direct superclasses

standard-object (class)

Direct methods

normalize-document-vector (method)
normalize-matrix (method)
print-object (method)
power (method)

Direct slots

Slot: power

Initargs: :power
Initform: 2
Readers: power (generic function)

Class: lsa ()

This is not much more than a convenience class that
remembers how the SVD was produced to be able to extract features later, or just to know what a given row or column corresponds to.

Package

Source

lsa.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

svd (method)
normalizer (method)
assembler (method)
document-indexer (method)
term-indexer (method)
document-mapper (method)
term-mapper (method)

Direct slots

Slot: term-mapper

A mapper over all documents in which a given term occurs.

Initargs: :term-mapper
Readers: term-mapper (generic function)

Slot: document-mapper

A mapper over all terms that occur in a given document.

Initargs: :document-mapper
Readers: document-mapper (generic function)

Slot: term-indexer

Term indexer.

Initargs: :term-indexer
Readers: term-indexer (generic function)

Slot: document-indexer

Document indexer.

Initargs: :document-indexer
Readers: document-indexer (generic function)

Slot: assembler

Turns co-occurrences into a matrix, term and document vectors.

Initargs: :assembler
Readers: assembler (generic function)

Slot: normalizer

Performs some last minute transformations on the assembled matrix.

Initargs: :normalizer
Readers: normalizer (generic function)

Slot: svd

The singular value decomposition.

Initargs: :svd
Readers: svd (generic function)

Class: lsa-assembler ()

The standard assembler that adds ...

Package

Source

assemble.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

ri-term-assembler (class)

Direct methods

assemble-document-vector (method)
assemble-term-vector (method)
assemble-co-occurrence-matrix (method)
print-object (method)
n-documents (method)
n-terms (method)

Direct slots

Slot: n-terms

Initargs: :n-terms
Readers: n-terms (generic function)

Slot: n-documents

Initargs: :n-documents
Readers: n-documents (generic function)

Class: null-normalizer ()

Package

Source

Direct superclasses

standard-object (class)

Direct methods

normalize-document-vector (method)
normalize-matrix (method)
print-object (method)

Class: pmi-normalizer ()

Package

Source

Direct superclasses

standard-object (class)

Direct methods

normalize-document-vector (method)
normalize-matrix (method)
n-documents (method)
term-total-per-class (method)
term-counts-per-class (method)
class-counts (method)
term-counts (method)
term-total (method)
document-class-fn (method)

Direct slots

Slot: document-class-fn

Initargs: :document-class-fn
Initform: (error "document-class-fn is required.")
Readers: document-class-fn (generic function)

Slot: term-total

Initargs: :term-total
Readers: term-total (generic function)

Slot: term-counts

Initargs: :term-counts
Readers: term-counts (generic function)

Slot: class-counts

Initargs: :class-counts
Readers: class-counts (generic function)

Slot: term-counts-per-class

Initargs: :term-counts-per-class
Readers: term-counts-per-class (generic function)

Slot: term-total-per-class

Initargs: :term-total-per-class
Readers: term-total-per-class (generic function)

Slot: n-documents

Initargs: :n-documents
Readers: n-documents (generic function)

Class: ri-term-assembler ()

Terms are random indexed, documents are not.

Package

Source

assemble.lisp (file)

Direct superclasses

lsa-assembler (class)

Direct methods

assemble-document-vector (method)
assemble-co-occurrence-matrix (method)
print-object (method)

Class: row-centering-normalizer ()

Package

Source

Direct superclasses

standard-object (class)

Direct methods

normalize-document-vector (method)
normalize-matrix (method)
row-averages (method)

Direct slots

Slot: row-averages

Readers: row-averages (generic function)

Class: sign-normalizer ()

Package

Source

Direct superclasses

standard-object (class)

Direct methods

normalize-document-vector (method)
normalize-matrix (method)

Class: tf-idf-normalizer ()

Package

Source

Direct superclasses

standard-object (class)

Direct methods

normalize-document-vector (method)
normalize-matrix (method)
print-object (method)
idfs (method)

Direct slots

Slot: idfs

The inverse document frequencies in the originally assembled matrix.

Initargs: :idfs
Readers: idfs (generic function)

5.2 Internal definitions

5.2.1 Functions

Function: %make-hashing-indexer &key (NEXT-INDEX NEXT-INDEX) (INDEX->OBJECT INDEX->OBJECT) (OBJECT->INDEX OBJECT->INDEX)

Package: lassie.indexer
Source: indexer.lisp (file)

Function: %make-random-indexer &key (LENGTH LENGTH) (N N) (OBJECT->INDEX OBJECT->INDEX)

Package: lassie.indexer
Source: indexer.lisp (file)

Function: alist->hashing-indexer TEST ALIST

Package: lassie.indexer
Source: indexer.lisp (file)

Function: assemble-occurence-vector LISTER SIZE

Return a vector of SIZE whose elements represent the frequency with which their indices were listed by LISTER.

Package: lassie.assembler
Source: assemble.lisp (file)

Function: column-norm MATRIX COLUMN &key POWER

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: compose-mappers &rest MAPPERS

Return a mapper that maps from the same set as the first of MAPPERS maps from and maps to what the last of MAPPERS maps to, composing them in a chain. If MAPPERS is NIL #’FUNCALL, the identity mapper, is returned.

Package: lassie
Source: mapper.lisp (file)

Function: concatente-mappers &rest MAPPERS

Return a mapper that is the concatention of MAPPERS.

Package: lassie
Source: mapper.lisp (file)

Function: construct-lsa-vector LSA FEATURES SV-SIDE

Package: lassie
Source: lsa.lisp (file)

Function: coordinate BASIS VECTOR

Return the length of the projection of VECTOR to BASIS.

Package: lassie
Source: lsa.lisp (file)

Function: copy-counting-indexer INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: copy-hashing-indexer INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: copy-random-indexer INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: counting-indexer-count INSTANCE

Function: (setf counting-indexer-count) VALUE INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: counting-indexer-p OBJECT

Package: lassie.indexer
Source: indexer.lisp (file)

Function: curry-mapper MAPPER &rest CURRIED-ARGS

What makes a mapper is that the first is a function that is somehow applied to arguments. Currying a mapper leaves the function parameter alone and curries the rest of the parameters.

Package: lassie
Source: mapper.lisp (file)

Function: encode-mapper MAPPER ENCODER &key ALLOCATE-NEW-INDEX-P

Translate MAPPER by encoding its sole argument with ENCODER.

Package: lassie
Source: mapper.lisp (file)

Function: extract-lsa-features LSA VECTOR-OR-INDEX SV-OWN SV-OTHER

Package: lassie
Source: lsa.lisp (file)

Function: extract-svd-features SVD VECTOR FN

Package: lassie
Source: lsa.lisp (file)

Function: hash-table= ALIST HASH-TABLE

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: hashing-indexer->alist INDEXER

Package: lassie.indexer
Source: indexer.lisp (file)

Function: hashing-indexer-index->object INSTANCE

Function: (setf hashing-indexer-index->object) VALUE INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: hashing-indexer-next-index INSTANCE

Function: (setf hashing-indexer-next-index) VALUE INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: hashing-indexer-object->index INSTANCE

Function: (setf hashing-indexer-object->index) VALUE INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: hashing-indexer-p OBJECT

Package: lassie.indexer
Source: indexer.lisp (file)

Function: incf-and-maybe-grow MATRIX DELTA &rest INDICES

Package: lassie.assembler
Source: assemble.lisp (file)

Function: inner* VECTOR1 VECTOR2

Package: lassie
Source: lsa.lisp (file)

Function: insert-into-sorted-vector ITEM VEC &key MAX-LENGTH KEY TEST

Insert ITEM into VECTOR while keeping it sorted by TEST. Extend the vector if needed while respecting MAX-LENGTH

Package: lassie
Source: lsa-extra.lisp (file)

Function: inverse-document-frequency MATRIX TERM

Relative importance of TERM across MATRIX.

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: make-column-power-normalizer &key POWER

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: make-encoded-term-document-lister TERM-MAPPER DOCUMENT-MAPPER TERM-ENCODER DOCUMENT-ENCODER TERM-LISTER DOCUMENT-LISTER

Return a lister that maps to encoded terms and documents. If DOCUMENT-LISTER is not NIL get its documents, encode them and list their terms with DOCUMENT-MAPPER. Act similary with TERM-LISTER and TERM-MAPPER.

Package: lassie
Source: mapper.lisp (file)

Function: make-encoded-term-document-mapper DOCUMENT-MAPPER TERM-ENCODER DOCUMENT-ENCODER

Return a mapper that applies to a document and calls its function argument with two parameters: the encoded term and the encoded document.

Package: lassie
Source: mapper.lisp (file)

Function: make-index-vector N LENGTH

Create a random index vector of LENGTH with N 1s and N -1s. It is stored as a sparse vector (only the indices of non-zero elements where the first N are +1 the rest are -1).

Package: lassie.indexer
Source: indexer.lisp (file)

Function: make-lsa-assembler &key N-TERMS N-DOCUMENTS

Package: lassie.assembler
Source: assemble.lisp (file)

Function: make-mapper &rest SEQUENCES

Return a mapper that maps from SEQUENCES to elements of SEQUENCES.

Package: lassie
Source: mapper.lisp (file)

Function: make-null-normalizer ()

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: make-ri-term-assembler ()

Package: lassie.assembler
Source: assemble.lisp (file)

Function: make-tf-idf-normalizer &key IDFS

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: n-columns MATRIX

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: n-documents-with-term MATRIX TERM

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: n-rows MATRIX

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: norm VECTOR &key POWER

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: normalize-column MATRIX COLUMN &key POWER

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: normalize-vector VECTOR &key POWER

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: null-mapper &rest ARGS

Package: lassie
Source: mapper.lisp (file)

Function: random-indexer-length INSTANCE

Function: (setf random-indexer-length) VALUE INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: random-indexer-n INSTANCE

Function: (setf random-indexer-n) VALUE INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: random-indexer-object->index INSTANCE

Function: (setf random-indexer-object->index) VALUE INSTANCE

Package: lassie.indexer
Source: indexer.lisp (file)

Function: random-indexer-p OBJECT

Package: lassie.indexer
Source: indexer.lisp (file)

Function: row-average MATRIX ROW

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: sum-column MATRIX COLUMN

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: sum-matrix MATRIX

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: sum-row MATRIX ROW

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: sum-vector VECTOR

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: term-frequency MATRIX TERM DOCUMENT

A normalized measure of how often TERM appears in DOCUMENT.

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: test-compose-mappers ()

Package: lassie
Source: mapper.lisp (file)

Function: test-hashing-indexer ()

Package: lassie.indexer
Source: indexer.lisp (file)

Function: test-pmi-normalizer ()

Package: lassie.assembler
Source: normalizer.lisp (file)

Function: test-tf-idf-normalizer ()

Package: lassie.assembler
Source: normalizer.lisp (file)

5.2.2 Generic functions

Generic Function: assembler OBJECT

Package

Methods

Method: assembler (LSA lsa)

Turns co-occurrences into a matrix, term and document vectors.

Source: lsa.lisp (file)

Generic Function: class-counts OBJECT

Package

Methods

Method: class-counts (PMI-NORMALIZER pmi-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: document-class-fn OBJECT

Package

Methods

Method: document-class-fn (PMI-NORMALIZER pmi-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: idfs OBJECT

Package

Methods

Method: idfs (TF-IDF-NORMALIZER tf-idf-normalizer)

The inverse document frequencies in the originally assembled matrix.

Source: normalizer.lisp (file)

Generic Function: map-lines FUNCTION OBJECT

Package

Source

mapper.lisp (file)

Methods

Method: map-lines FUNCTION (STREAM stream)

Method: map-lines FUNCTION (PATHNAME pathname)

Method: map-lines FUNCTION (STRING string)

Generic Function: n-documents OBJECT

Package

Methods

Method: n-documents (LSA-ASSEMBLER lsa-assembler)

automatically generated reader method

Source: assemble.lisp (file)

Method: n-documents (PMI-NORMALIZER pmi-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: n-terms OBJECT

Package

Methods

Method: n-terms (LSA-ASSEMBLER lsa-assembler)

automatically generated reader method

Source: assemble.lisp (file)

Generic Function: power OBJECT

Package

Methods

Method: power (COLUMN-POWER-NORMALIZER column-power-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: row-averages OBJECT

Package

Methods

Method: row-averages (ROW-CENTERING-NORMALIZER row-centering-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: term-counts OBJECT

Package

Methods

Method: term-counts (PMI-NORMALIZER pmi-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: term-counts-per-class OBJECT

Package

Methods

Method: term-counts-per-class (PMI-NORMALIZER pmi-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: term-total OBJECT

Package

Methods

Method: term-total (PMI-NORMALIZER pmi-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

Generic Function: term-total-per-class OBJECT

Package

Methods

Method: term-total-per-class (PMI-NORMALIZER pmi-normalizer)

automatically generated reader method

Source: normalizer.lisp (file)

5.2.3 Structures

Structure: counting-indexer ()

Simply assigns a new index to every object.

Package

Source

indexer.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

<-index (method)
->index (method)

Direct slots

Slot: count

Initform: 0
Readers: counting-indexer-count (function)
Writers: (setf counting-indexer-count) (function)

Structure: hashing-indexer ()

Package

Source

indexer.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

print-object (method)
<-index (method)
->index (method)

Direct slots

Slot: next-index

Initform: 0
Readers: hashing-indexer-next-index (function)
Writers: (setf hashing-indexer-next-index) (function)

Slot: index->object

Readers: hashing-indexer-index->object (function)
Writers: (setf hashing-indexer-index->object) (function)

Slot: object->index

Readers: hashing-indexer-object->index (function)
Writers: (setf hashing-indexer-object->index) (function)

Structure: random-indexer ()

Package