The neural-classifier Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The neural-classifier Reference Manual

This is the neural-classifier Reference Manual, version 0.1, generated automatically by Declt version 3.0 "Montgomery Scott" on Tue Dec 22 14:29:58 2020 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

Neural-classifier

Build Status CI

neural-classifier is a neural network library based on the first chapters from this book. It is divided on two systems: neural-classifier which is a general API for neural networks and neural-classifier/mnist which contains helper functions for working with MNIST/EMNIST datasets. For API documentation visit this page.

How to work with MNIST dataset?

Dependencies

magicl and nibbles can be downloaded with quicklisp.

What if the network shows good accuracy but fails to recognize my own digits?

If the accuracy returned by train-epochs is good, but the network fails to recognize digits draws by your own hand, try EMNIST database instead of MNIST. Copy four emnist-digits-* files to your MNIST directory preserving the name of destination files. Images in EMNIST set are transposed (x and y coordinates swapped), so do the same with your own images.


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 neural-classifier

Author

Vasily Postnicov <shamaz.mazum at gmail dot com>

License

2-clause BSD

Description

Classification of samples based on neural network.

Version

0.1

Dependencies
Source

neural-classifier.asd (file)

Components

Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 neural-classifier.asd

Location

/home/quickref/quicklisp/dists/quicklisp/software/neural-classifier-20201220-git/neural-classifier.asd

Systems

neural-classifier (system)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 neural-classifier/package.lisp

Parent

neural-classifier (system)

Location

package.lisp

Packages

neural-classifier


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.3 neural-classifier/magicl-blas.lisp

Dependency

package.lisp (file)

Parent

neural-classifier (system)

Location

magicl-blas.lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.4 neural-classifier/definitions.lisp

Dependency

magicl-blas.lisp (file)

Parent

neural-classifier (system)

Location

definitions.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.5 neural-classifier/utility.lisp

Dependency

definitions.lisp (file)

Parent

neural-classifier (system)

Location

utility.lisp

Exported Definitions

idx-abs-max (function)

Internal Definitions

Previous: , Up: Lisp files   [Contents][Index]

3.1.6 neural-classifier/neural-network.lisp

Dependency

utility.lisp (file)

Parent

neural-classifier (system)

Location

neural-network.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 neural-classifier

Source

package.lisp (file)

Use List

common-lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Special variables

Special Variable: *decay-rate*

Regularization parameter @c(λ/N), where @c(N) is the number of objects in the training set and @c(λ) must be about 1-10. If not sure, start with zero (which is the default).

Package

neural-classifier

Source

definitions.lisp (file)

Special Variable: *learn-rate*

Speed of gradient descent algorithm. Bigger values result in faster learning, but too big is bad.

Package

neural-classifier

Source

definitions.lisp (file)

Special Variable: *minibatch-size*

Number of samples to be used in stochastic gradient descent algorithm.

Package

neural-classifier

Source

definitions.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.2 Functions

Function: calculate ()

Calculate the output from the network @c(neural-network) for the object @c(object). The input transformation function is applied to the @c(object) and the output transformation function is applied to the output column from the network.

Package

neural-classifier

Source

neural-network.lisp (file)

Function: idx-abs-max MATRIX

Returns index of first element with maximal absolute value by calling isamax() function from BLAS. Works only for rows or columns.

Package

neural-classifier

Source

utility.lisp (file)

Function: make-neural-network ()

Create a new neural network.

@c(layout) is a list of positive integers which describes the amount of neurons in each layer (starting from input layer).

@c(activation-funcs) is a list all the elements of which are either @c(:sigmoid), @c(:tanh), @c(:rlu) or @c(:softmax). The length of this list must be equal to the length of @c(layout) minus one. The last element cannot be @c(:rlu). @c(:softmax) can be only the last element.

@c(input-trans) is a function which is applied to an object passed to @c(calculate) to transform it into an input column (that is a matrix with the type @c(magicl:matrix/single-float) and the shape @c(Nx1), where @c(N) is the first number in the @c(layout)). For example, if we are recognizing digits from the MNIST set, this function can take a number of an image in the set and return @c(784x1) matrix.

@c(output-trans) is a function which is applied to the output of @c(calculate) function (that is a matrix with the type @c(magicl:matrix/single-float) and the shape Mx1, where M is the last number in the @c(layout)) to return some object with user-defined meaning (called a label). Again, if we are recognizing digits, this function transforms @c(10x1) matrix to a number from 0 to 9.

@c(input-trans%) is just like @c(input-trans), but is used while training. It can include additional transformations to extend your training set (e.g. it can add some noise to resulting vector, rotate a picture by a small random angle, etc.).

@c(label-trans) is a function which is applied to a label to get a column (that is a matrix with the type @c(magicl:matrix/single-float) and the shape @c(Mx1), where @c(M) is the last number in the @c(layout)) which is optimal output from the network for this object. With digits recognition, this function may take a digit @c(n) and return @c(10x1) matrix of all zeros with exception for @c(n)-th element which would be @c(1f0).

Default value for all transformation functions is @c(identity).

Package

neural-classifier

Source

neural-network.lisp (file)

Function: rate ()

Calculate accuracy of the @c(neural-network) (that is a ratio of correctly guessed samples to all samples) using testing data from the generator @c(generator). Each item returned by @c(generator) must be a cons pair containing an object which is passed to the network and its label. @c(test) is a function used to compare the expected label and the actual one.

Package

neural-classifier

Source

neural-network.lisp (file)

Function: train-epoch ()

Perform a training of @c(neural-network) on every object returned by the generator @c(generator). Each item returned by @c(generator) must be a cons pair containing an object which is passed to the neural network and its label. @c(input-trans%) and @c(label-trans) functions passed to @c(make-neural-network) are applied to @c(car) and @c(cdr) of each cons pair.

Package

neural-classifier

Source

neural-network.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.3 Generic functions

Generic Function: neural-network-input-trans OBJECT
Generic Function: (setf neural-network-input-trans) NEW-VALUE OBJECT
Package

neural-classifier

Methods
Method: neural-network-input-trans (NEURAL-NETWORK neural-network)
Method: (setf neural-network-input-trans) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates an input object to a vector

Source

definitions.lisp (file)

Generic Function: neural-network-input-trans% OBJECT
Generic Function: (setf neural-network-input-trans%) NEW-VALUE OBJECT
Package

neural-classifier

Methods
Method: neural-network-input-trans% (NEURAL-NETWORK neural-network)
Method: (setf neural-network-input-trans%) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates an input object to a vector (used for training)

Source

definitions.lisp (file)

Generic Function: neural-network-label-trans OBJECT
Generic Function: (setf neural-network-label-trans) NEW-VALUE OBJECT
Package

neural-classifier

Methods
Method: neural-network-label-trans (NEURAL-NETWORK neural-network)
Method: (setf neural-network-label-trans) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates a label to a vector

Source

definitions.lisp (file)

Generic Function: neural-network-layout OBJECT
Package

neural-classifier

Methods
Method: neural-network-layout (NEURAL-NETWORK neural-network)

Number of neurons in each layer of the network

Source

definitions.lisp (file)

Generic Function: neural-network-output-trans OBJECT
Generic Function: (setf neural-network-output-trans) NEW-VALUE OBJECT
Package

neural-classifier

Methods
Method: neural-network-output-trans (NEURAL-NETWORK neural-network)
Method: (setf neural-network-output-trans) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates an output vector to a label.

Source

definitions.lisp (file)


Previous: , Up: Exported definitions   [Contents][Index]

5.1.4 Classes

Class: neural-network ()

Class for neural networks

Package

neural-classifier

Source

definitions.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
Direct slots
Slot: layout

Number of neurons in each layer of the network

Type

list

Initargs

:layout

Initform

(error "specify number of neurons in each layer")

Readers

neural-network-layout (generic function)

Slot: activation-funcs

List of activation functions.

Type

list

Initargs

:activation-funcs

Readers

neural-network-activation-funcs (generic function)

Writers

(setf neural-network-activation-funcs) (generic function)

Slot: weights

Weight matrices for each layer

Type

list

Readers

neural-network-weights (generic function)

Writers

(setf neural-network-weights) (generic function)

Slot: biases

Bias vectors for each layer

Type

list

Readers

neural-network-biases (generic function)

Writers

(setf neural-network-biases) (generic function)

Slot: input-trans

Function which translates an input object to a vector

Type

function

Initargs

:input-trans

Initform

(function identity)

Readers

neural-network-input-trans (generic function)

Writers

(setf neural-network-input-trans) (generic function)

Slot: output-trans

Function which translates an output vector to a label.

Type

function

Initargs

:output-trans

Initform

(function identity)

Readers

neural-network-output-trans (generic function)

Writers

(setf neural-network-output-trans) (generic function)

Slot: input-trans%

Function which translates an input object to a vector (used for training)

Type

function

Initargs

:input-trans%

Initform

(function identity)

Readers

neural-network-input-trans% (generic function)

Writers

(setf neural-network-input-trans%) (generic function)

Slot: label-trans

Function which translates a label to a vector

Type

function

Initargs

:label-trans

Initform

(function identity)

Readers

neural-network-label-trans (generic function)

Writers

(setf neural-network-label-trans) (generic function)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Functions

Function: activation-fn SYMBOL
Package

neural-classifier

Source

utility.lisp (file)

Function: activation-fn-derivative SYMBOL
Package

neural-classifier

Source

utility.lisp (file)

Function: calculate-delta ()

Calculate partial derivative of the cost function by z for all layers

Package

neural-classifier

Source

neural-network.lisp (file)

Function: calculate-gradient ()

Calculate gradient of the cost function

Package

neural-classifier

Source

neural-network.lisp (file)

Function: calculate-gradient-minibatch ()

Calculate gradient of the cost function based on multiple input samples

Package

neural-classifier

Source

neural-network.lisp (file)

Function: calculate-z-and-out ()

Calculate argument and value of activation function for all layers

Package

neural-classifier

Source

neural-network.lisp (file)

Function: learn ()
Package

neural-classifier

Source

neural-network.lisp (file)

Function: random-normal &key MEAN SIGMA
Package

neural-classifier

Source

utility.lisp (file)

Function: rlu Z
Package

neural-classifier

Source

utility.lisp (file)

Function: rlu-derivative Z
Package

neural-classifier

Source

utility.lisp (file)

Function: sigma Z
Package

neural-classifier

Source

utility.lisp (file)

Function: sigma-derivative Z
Package

neural-classifier

Source

utility.lisp (file)

Function: softmax V
Package

neural-classifier

Source

utility.lisp (file)

Function: softmax-derivative Z
Package

neural-classifier

Source

utility.lisp (file)

Function: tanh-derivative Z
Package

neural-classifier

Source

utility.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Generic functions

Generic Function: neural-network-activation-funcs OBJECT
Generic Function: (setf neural-network-activation-funcs) NEW-VALUE OBJECT
Package

neural-classifier

Methods
Method: neural-network-activation-funcs (NEURAL-NETWORK neural-network)
Method: (setf neural-network-activation-funcs) NEW-VALUE (NEURAL-NETWORK neural-network)

List of activation functions.

Source

definitions.lisp (file)

Generic Function: neural-network-biases OBJECT
Generic Function: (setf neural-network-biases) NEW-VALUE OBJECT
Package

neural-classifier

Methods
Method: neural-network-biases (NEURAL-NETWORK neural-network)
Method: (setf neural-network-biases) NEW-VALUE (NEURAL-NETWORK neural-network)

Bias vectors for each layer

Source

definitions.lisp (file)

Generic Function: neural-network-weights OBJECT
Generic Function: (setf neural-network-weights) NEW-VALUE OBJECT
Package

neural-classifier

Methods
Method: neural-network-weights (NEURAL-NETWORK neural-network)
Method: (setf neural-network-weights) NEW-VALUE (NEURAL-NETWORK neural-network)

Weight matrices for each layer

Source

definitions.lisp (file)


Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Types

Type: activation-symbol ()
Package

neural-classifier

Source

definitions.lisp (file)

Type: non-negative-fixnum ()
Package

neural-classifier

Source

definitions.lisp (file)

Type: positive-fixnum ()
Package

neural-classifier

Source

definitions.lisp (file)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   N  
Index Entry  Section

F
File, Lisp, neural-classifier.asd: The neural-classifier․asd file
File, Lisp, neural-classifier/definitions.lisp: The neural-classifier/definitions․lisp file
File, Lisp, neural-classifier/magicl-blas.lisp: The neural-classifier/magicl-blas․lisp file
File, Lisp, neural-classifier/neural-network.lisp: The neural-classifier/neural-network․lisp file
File, Lisp, neural-classifier/package.lisp: The neural-classifier/package․lisp file
File, Lisp, neural-classifier/utility.lisp: The neural-classifier/utility․lisp file

L
Lisp File, neural-classifier.asd: The neural-classifier․asd file
Lisp File, neural-classifier/definitions.lisp: The neural-classifier/definitions․lisp file
Lisp File, neural-classifier/magicl-blas.lisp: The neural-classifier/magicl-blas․lisp file
Lisp File, neural-classifier/neural-network.lisp: The neural-classifier/neural-network․lisp file
Lisp File, neural-classifier/package.lisp: The neural-classifier/package․lisp file
Lisp File, neural-classifier/utility.lisp: The neural-classifier/utility․lisp file

N
neural-classifier.asd: The neural-classifier․asd file
neural-classifier/definitions.lisp: The neural-classifier/definitions․lisp file
neural-classifier/magicl-blas.lisp: The neural-classifier/magicl-blas․lisp file
neural-classifier/neural-network.lisp: The neural-classifier/neural-network․lisp file
neural-classifier/package.lisp: The neural-classifier/package․lisp file
neural-classifier/utility.lisp: The neural-classifier/utility․lisp file

Jump to:   F   L   N  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   (  
A   C   F   G   I   L   M   N   R   S   T  
Index Entry  Section

(
(setf neural-network-activation-funcs): Internal generic functions
(setf neural-network-activation-funcs): Internal generic functions
(setf neural-network-biases): Internal generic functions
(setf neural-network-biases): Internal generic functions
(setf neural-network-input-trans%): Exported generic functions
(setf neural-network-input-trans%): Exported generic functions
(setf neural-network-input-trans): Exported generic functions
(setf neural-network-input-trans): Exported generic functions
(setf neural-network-label-trans): Exported generic functions
(setf neural-network-label-trans): Exported generic functions
(setf neural-network-output-trans): Exported generic functions
(setf neural-network-output-trans): Exported generic functions
(setf neural-network-weights): Internal generic functions
(setf neural-network-weights): Internal generic functions

A
activation-fn: Internal functions
activation-fn-derivative: Internal functions

C
calculate: Exported functions
calculate-delta: Internal functions
calculate-gradient: Internal functions
calculate-gradient-minibatch: Internal functions
calculate-z-and-out: Internal functions

F
Function, activation-fn: Internal functions
Function, activation-fn-derivative: Internal functions
Function, calculate: Exported functions
Function, calculate-delta: Internal functions
Function, calculate-gradient: Internal functions
Function, calculate-gradient-minibatch: Internal functions
Function, calculate-z-and-out: Internal functions
Function, idx-abs-max: Exported functions
Function, learn: Internal functions
Function, make-neural-network: Exported functions
Function, random-normal: Internal functions
Function, rate: Exported functions
Function, rlu: Internal functions
Function, rlu-derivative: Internal functions
Function, sigma: Internal functions
Function, sigma-derivative: Internal functions
Function, softmax: Internal functions
Function, softmax-derivative: Internal functions
Function, tanh-derivative: Internal functions
Function, train-epoch: Exported functions

G
Generic Function, (setf neural-network-activation-funcs): Internal generic functions
Generic Function, (setf neural-network-biases): Internal generic functions
Generic Function, (setf neural-network-input-trans%): Exported generic functions
Generic Function, (setf neural-network-input-trans): Exported generic functions
Generic Function, (setf neural-network-label-trans): Exported generic functions
Generic Function, (setf neural-network-output-trans): Exported generic functions
Generic Function, (setf neural-network-weights): Internal generic functions
Generic Function, neural-network-activation-funcs: Internal generic functions
Generic Function, neural-network-biases: Internal generic functions
Generic Function, neural-network-input-trans: Exported generic functions
Generic Function, neural-network-input-trans%: Exported generic functions
Generic Function, neural-network-label-trans: Exported generic functions
Generic Function, neural-network-layout: Exported generic functions
Generic Function, neural-network-output-trans: Exported generic functions
Generic Function, neural-network-weights: Internal generic functions

I
idx-abs-max: Exported functions

L
learn: Internal functions

M
make-neural-network: Exported functions
Method, (setf neural-network-activation-funcs): Internal generic functions
Method, (setf neural-network-biases): Internal generic functions
Method, (setf neural-network-input-trans%): Exported generic functions
Method, (setf neural-network-input-trans): Exported generic functions
Method, (setf neural-network-label-trans): Exported generic functions
Method, (setf neural-network-output-trans): Exported generic functions
Method, (setf neural-network-weights): Internal generic functions
Method, neural-network-activation-funcs: Internal generic functions
Method, neural-network-biases: Internal generic functions
Method, neural-network-input-trans: Exported generic functions
Method, neural-network-input-trans%: Exported generic functions
Method, neural-network-label-trans: Exported generic functions
Method, neural-network-layout: Exported generic functions
Method, neural-network-output-trans: Exported generic functions
Method, neural-network-weights: Internal generic functions

N
neural-network-activation-funcs: Internal generic functions
neural-network-activation-funcs: Internal generic functions
neural-network-biases: Internal generic functions
neural-network-biases: Internal generic functions
neural-network-input-trans: Exported generic functions
neural-network-input-trans: Exported generic functions
neural-network-input-trans%: Exported generic functions
neural-network-input-trans%: Exported generic functions
neural-network-label-trans: Exported generic functions
neural-network-label-trans: Exported generic functions
neural-network-layout: Exported generic functions
neural-network-layout: Exported generic functions
neural-network-output-trans: Exported generic functions
neural-network-output-trans: Exported generic functions
neural-network-weights: Internal generic functions
neural-network-weights: Internal generic functions

R
random-normal: Internal functions
rate: Exported functions
rlu: Internal functions
rlu-derivative: Internal functions

S
sigma: Internal functions
sigma-derivative: Internal functions
softmax: Internal functions
softmax-derivative: Internal functions

T
tanh-derivative: Internal functions
train-epoch: Exported functions

Jump to:   (  
A   C   F   G   I   L   M   N   R   S   T  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *  
A   B   I   L   O   S   W  
Index Entry  Section

*
*decay-rate*: Exported special variables
*learn-rate*: Exported special variables
*minibatch-size*: Exported special variables

A
activation-funcs: Exported classes

B
biases: Exported classes

I
input-trans: Exported classes
input-trans%: Exported classes

L
label-trans: Exported classes
layout: Exported classes

O
output-trans: Exported classes

S
Slot, activation-funcs: Exported classes
Slot, biases: Exported classes
Slot, input-trans: Exported classes
Slot, input-trans%: Exported classes
Slot, label-trans: Exported classes
Slot, layout: Exported classes
Slot, output-trans: Exported classes
Slot, weights: Exported classes
Special Variable, *decay-rate*: Exported special variables
Special Variable, *learn-rate*: Exported special variables
Special Variable, *minibatch-size*: Exported special variables

W
weights: Exported classes

Jump to:   *  
A   B   I   L   O   S   W  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   A   C   N   P   S   T  
Index Entry  Section

A
activation-symbol: Internal types

C
Class, neural-network: Exported classes

N
neural-classifier: The neural-classifier system
neural-classifier: The neural-classifier package
neural-network: Exported classes
non-negative-fixnum: Internal types

P
Package, neural-classifier: The neural-classifier package
positive-fixnum: Internal types

S
System, neural-classifier: The neural-classifier system

T
Type, activation-symbol: Internal types
Type, non-negative-fixnum: Internal types
Type, positive-fixnum: Internal types

Jump to:   A   C   N   P   S   T