The neural-classifier Reference Manual

Table of Contents

The neural-classifier Reference Manual

This is the neural-classifier Reference Manual, version 0.1, generated automatically by Declt version 3.0 "Montgomery Scott" on Mon Apr 19 17:11:02 2021 GMT+0.

1 Introduction

Neural-classifier

neural-classifier is a neural network library based on the first chapters from this book. It is divided on two systems: neural-classifier which is a general API for neural networks and neural-classifier/mnist which contains helper functions for working with MNIST/EMNIST datasets. For API documentation visit this page.

How to work with MNIST dataset?

• Unpack files in mnist/dataset directory.
• Load neural-classifier/mnist system: (ql:quickload :neural-classifier/mnist).
• Eval (neural-classifier-mnist:load-mnist-database) (this will take about 10-15 seconds).
• Create a neural network: (defparameter *nn* neural-classifier-mnist:make-mnist-classifier 35) where 35 is a number of hidden neurons.
• Execute (neural-classifier-mnist:train-epochs *nn* 10) to train the network for 10 epochs. This function will return data about the network's accuracy for each epoch.
• To test your own digits convert them to 784x1 matrix of type magicl:matrix/single-float and pass it to neural-classifier:calculate function.
• Also you can play with some other hyper-parameters, not only the number of hidden neurons. neural-classifier:*learn-rate* is how fast gradient descent algorithm works (i.e. how fast your network learns), neural-classifier:*decay-rate* is related to regularization and should be about 5/N where N is a number of training samples. Zero means no regularization.

How to build custom nets and data?

See GH pages for this project (link above). In general you need to write functions which translate your data and labels into magicl:matrix/single-float matrices. Then you create a net with neural-classifier:make-neural-network function and snakes generator which returns conses in the form (DATA . LABEL). To train a network for one epoch you call (neural-classifier:train-epoch).

Dependencies

• blas and lapack foreign libraries.
• magicl for matrix operations.
• nibbles for loading MNIST data.

magicl and nibbles can be downloaded with quicklisp.

What if the network shows good accuracy but fails to recognize my own digits?

If the accuracy returned by train-epochs is good, but the network fails to recognize digits draws by your own hand, try EMNIST database instead of MNIST. Copy four emnist-digits-* files to your MNIST directory preserving the name of destination files. Images in EMNIST set are transposed (x and y coordinates swapped), so do the same with your own images.

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 neural-classifier

Author

Vasily Postnicov

License

2-clause BSD

Description

Classification of samples based on neural network.

Version

0.1

Dependencies

• magicl/ext-blas
• magicl/ext-lapack
• snakes
• cl-randist

Source

neural-classifier.asd (file)

Components

• package.lisp (file)
• sbcl-hacks.lisp (file)
• magicl-blas.lisp (file)
• definitions.lisp (file)
• utility.lisp (file)
• optimizers.lisp (file)
• neural-network.lisp (file)

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

3.1 Lisp

3.1.1 neural-classifier.asd

Location

/home/quickref/quicklisp/dists/quicklisp/software/neural-classifier-20210411-git/neural-classifier.asd

Systems

neural-classifier (system)

3.1.2 neural-classifier/package.lisp

Parent

neural-classifier (system)

Location

package.lisp

Packages

neural-classifier

3.1.3 neural-classifier/sbcl-hacks.lisp

Dependency

package.lisp (file)

Parent

neural-classifier (system)

Location

sbcl-hacks.lisp

Packages

neural-classifier-sbcl

Internal Definitions

• %expf (function)
• %sqrtf (function)
• %tanhf (function)
• print-condition-and-continue (function)
• symbolicate (function)

3.1.4 neural-classifier/magicl-blas.lisp

Dependency

sbcl-hacks.lisp (file)

Parent

neural-classifier (system)

Location

magicl-blas.lisp

3.1.5 neural-classifier/definitions.lisp

Dependency

magicl-blas.lisp (file)

Parent

neural-classifier (system)

Location

definitions.lisp

Exported Definitions

• *decay-rate* (special variable)
• *learn-rate* (special variable)
• *minibatch-size* (special variable)
• *momentum-coeff* (special variable)
• neural-network (class)
• neural-network-input-trans (method)
• (setf neural-network-input-trans) (method)
• neural-network-input-trans% (method)
• (setf neural-network-input-trans%) (method)
• neural-network-label-trans (method)
• (setf neural-network-label-trans) (method)
• neural-network-layout (method)
• neural-network-output-trans (method)
• (setf neural-network-output-trans) (method)

Internal Definitions

• activation-symbol (type)
• neural-network-activation-funcs (method)
• (setf neural-network-activation-funcs) (method)
• neural-network-biases (method)
• (setf neural-network-biases) (method)
• neural-network-weights (method)
• (setf neural-network-weights) (method)
• non-negative-fixnum (type)
• positive-fixnum (type)

3.1.6 neural-classifier/utility.lisp

Dependency

definitions.lisp (file)

Parent

neural-classifier (system)

Location

utility.lisp

Exported Definitions

idx-abs-max (function)

Internal Definitions

• activation (generic function)
• activation (method)
• activation (method)
• activation (method)
• activation (method)
• activation (method)
• activation-derivative (generic function)
• activation-derivative (method)
• activation-derivative (method)
• activation-derivative (method)
• activation-derivative (method)
• random-normal (function)
• sasum (function)
• sigmoid (function)

3.1.7 neural-classifier/optimizers.lisp

Dependency

utility.lisp (file)

Parent

neural-classifier (system)

Location

optimizers.lisp

Exported Definitions

• adagrad-optimizer (class)
• make-optimizer (function)
• momentum-optimizer (class)
• nesterov-optimizer (class)
• rmsprop-optimizer (class)
• sgd-optimizer (class)

Internal Definitions

• learn (generic function)
• memoizing-optimizer (class)
• momentum-coeff (method)
• (setf momentum-coeff) (method)
• momentum-coeff (method)
• (setf momentum-coeff) (method)
• optimizer (class)
• optimizer-biases (method)
• (setf optimizer-biases) (method)
• optimizer-initial-value (method)
• (setf optimizer-initial-value) (method)
• optimizer-weights (method)
• (setf optimizer-weights) (method)

3.1.8 neural-classifier/neural-network.lisp

Dependency

optimizers.lisp (file)

Parent

neural-classifier (system)

Location

neural-network.lisp

Exported Definitions

• calculate (function)
• make-neural-network (function)
• rate (function)
• train-epoch (function)

Internal Definitions

• calculate-delta (function)
• calculate-gradient (function)
• calculate-gradient-minibatch (function)
• calculate-z-and-out (function)
• learn (method)
• learn (method)
• learn (method)
• learn (method)
• learn (method)

4 Packages

Packages are listed by definition order.

4.1 neural-classifier

Source

package.lisp (file)

Use List

common-lisp

Exported Definitions

• *decay-rate* (special variable)
• *learn-rate* (special variable)
• *minibatch-size* (special variable)
• *momentum-coeff* (special variable)
• adagrad-optimizer (class)
• calculate (function)
• idx-abs-max (function)
• make-neural-network (function)
• make-optimizer (function)
• momentum-optimizer (class)
• nesterov-optimizer (class)
• neural-network (class)
• neural-network-input-trans (generic function)
• neural-network-input-trans (method)
• (setf neural-network-input-trans) (method)
• (setf neural-network-input-trans) (generic function)
• neural-network-input-trans% (generic function)
• neural-network-input-trans% (method)
• (setf neural-network-input-trans%) (method)
• (setf neural-network-input-trans%) (generic function)
• neural-network-label-trans (generic function)
• neural-network-label-trans (method)
• (setf neural-network-label-trans) (method)
• (setf neural-network-label-trans) (generic function)
• neural-network-layout (generic function)
• neural-network-layout (method)
• neural-network-output-trans (generic function)
• neural-network-output-trans (method)
• (setf neural-network-output-trans) (method)
• (setf neural-network-output-trans) (generic function)
• rate (function)
• rmsprop-optimizer (class)
• sgd-optimizer (class)
• train-epoch (function)

Internal Definitions

• activation (generic function)
• activation (method)
• activation (method)
• activation (method)
• activation (method)
• activation (method)
• activation-derivative (generic function)
• activation-derivative (method)
• activation-derivative (method)
• activation-derivative (method)
• activation-derivative (method)
• activation-symbol (type)
• calculate-delta (function)
• calculate-gradient (function)
• calculate-gradient-minibatch (function)
• calculate-z-and-out (function)
• learn (generic function)
• learn (method)
• learn (method)
• learn (method)
• learn (method)
• learn (method)
• memoizing-optimizer (class)
• momentum-coeff (generic function)
• momentum-coeff (method)
• (setf momentum-coeff) (method)
• momentum-coeff (method)
• (setf momentum-coeff) (method)
• (setf momentum-coeff) (generic function)
• neural-network-activation-funcs (generic function)
• neural-network-activation-funcs (method)
• (setf neural-network-activation-funcs) (method)
• (setf neural-network-activation-funcs) (generic function)
• neural-network-biases (generic function)
• neural-network-biases (method)
• (setf neural-network-biases) (method)
• (setf neural-network-biases) (generic function)
• neural-network-weights (generic function)
• neural-network-weights (method)
• (setf neural-network-weights) (method)
• (setf neural-network-weights) (generic function)
• non-negative-fixnum (type)
• optimizer (class)
• optimizer-biases (generic function)
• optimizer-biases (method)
• (setf optimizer-biases) (method)
• (setf optimizer-biases) (generic function)
• optimizer-initial-value (generic function)
• optimizer-initial-value (method)
• (setf optimizer-initial-value) (method)
• (setf optimizer-initial-value) (generic function)
• optimizer-weights (generic function)
• optimizer-weights (method)
• (setf optimizer-weights) (method)
• (setf optimizer-weights) (generic function)
• positive-fixnum (type)
• random-normal (function)
• sasum (function)
• sigmoid (function)

4.2 neural-classifier-sbcl

Source

sbcl-hacks.lisp (file)

Use List

common-lisp

Internal Definitions

• %expf (function)
• %sqrtf (function)
• %tanhf (function)
• print-condition-and-continue (function)
• symbolicate (function)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Special variables

Special Variable: *decay-rate*

Regularization parameter @c(λ/N), where @c(N) is the number of objects in the training set and @c(λ) must be about 1-10. If not sure, start with zero (which is the default).

Package

neural-classifier

Source

definitions.lisp (file)

Special Variable: *learn-rate*

Learning speed for gradient descent algorithms. Bigger values result in faster learning, but too big is bad. Default value is good for SGD, SGD with momentum and NAG optimizers. For Adagrad and RMSprop try 0.001f0.

Package

neural-classifier

Source

definitions.lisp (file)

Special Variable: *minibatch-size*

Number of samples to be used for one update of network parameters.

Package

neural-classifier

Source

definitions.lisp (file)

Special Variable: *momentum-coeff*

Hyperparameter for SGD optimizers which use momentum. Zero means just usual SGD. RMSprop also uses this parameter in accumulation of squared partial derivatives of network parameters. Good values are 0.8-0.9.

Package

neural-classifier

Source

definitions.lisp (file)

5.1.2 Functions

Function: calculate ()

Calculate output from the network @c(neural-network) for the object @c(object). The input transformation function (specified by @c(:input-trans) when creating a network) is applied to the @c(object) and the output transformation function (specified by @c(:output-trans)) is applied to output Nx1 matrix from the network.

Package

neural-classifier

Source

neural-network.lisp (file)

Function: idx-abs-max MATRIX

Returns index of first element with maximal absolute value by calling isamax() function from BLAS. Works only for rows or columns.

Package

neural-classifier

Source

utility.lisp (file)

Function: make-neural-network ()

Create a new neural network.
@begin(list)
@item(@c(layout) is a list of positive integers which describes the amount of neurons in each layer (starting from input layer).) @item(@c(activation-funcs) is a list all the elements of which are either @c(:sigmoid), @c(:tanh), @c(:abs), @c(:relu) or @c(:softmax). The length of this list must be equal to the length of @c(layout) minus one because the input layer does not have an activation function. The last element cannot be @c(:abs) or @c(:relu) and @c(:softmax) can only be the last element.) @item(@c(input-trans) is a function which is applied to an object passed to @c(calculate) to transform it into an input column (that is a matrix with the type @c(magicl:matrix/single-float) and the shape @c(Nx1), where @c(N) is the first number in the @c(layout)). For example, if we are recognizing digits from the MNIST set, this function can take a number of an image in the set and return @c(784x1) matrix.)
@item(@c(output-trans) is a function which is applied to the output of @c(calculate) function (that is a matrix with the type @c(magicl:matrix/single-float) and the shape Mx1, where M is the last number in the @c(layout)) to return some object with user-defined meaning (called a label). Again, if we are recognizing digits, this function transforms @c(10x1) matrix to a number from 0 to 9.)
@item(@c(input-trans%) is just like @c(input-trans), but is used while training. It can include additional transformations to extend your training set (e.g. it can add some noise to input data, rotate an input picture by a small random angle, etc.).) @item(@c(label-trans) is a function which is applied to a label to get a column (that is a matrix with the type @c(magicl:matrix/single-float) and the shape @c(Mx1), where @c(M) is the last number in the @c(layout)) which is the optimal output from the network for this object. With digits recognition, this function may take a digit @c(n) and return @c(10x1) matrix of all zeros with exception for @c(n)-th element which would be @c(1f0).)
@end(list)
Default value for all transformation functions is @c(identity).

Package

neural-classifier

Source

neural-network.lisp (file)

Function: make-optimizer TYPE NETWORK

Make optimizer of type @c(type) for a network @c(network).

Package

neural-classifier

Source

optimizers.lisp (file)

Function: rate ()

Calculate accuracy of the @c(neural-network) (ratio of correctly guessed samples to all samples) using testing data from the generator @c(generator). Each item returned by @c(generator) must be a cons pair in the form @c((data-object . label)), as with @c(train-epoch) function. @c(test) is a function used to compare the expected label with the label returned by the network.

Package

neural-classifier

Source

neural-network.lisp (file)

Function: train-epoch ()

Perform training of @c(neural-network) on every object returned by the generator @c(generator). Each item returned by @c(generator) must be in the form @c((data-object . label)) cons
pair. @c(input-trans%) and @c(label-trans) functions passes to @c(make-neural-network) are applied to @c(car) and @c(cdr) of each pair respectively.

Package

neural-classifier

Source

neural-network.lisp (file)

5.1.3 Generic functions

Generic Function: neural-network-input-trans OBJECT

Generic Function: (setf neural-network-input-trans) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: neural-network-input-trans (NEURAL-NETWORK neural-network)

Method: (setf neural-network-input-trans) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates an input object to a vector

Source

definitions.lisp (file)

Generic Function: neural-network-input-trans% OBJECT

Generic Function: (setf neural-network-input-trans%) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: neural-network-input-trans% (NEURAL-NETWORK neural-network)

Method: (setf neural-network-input-trans%) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates an input object to a vector (used for training)

Source

definitions.lisp (file)

Generic Function: neural-network-label-trans OBJECT

Generic Function: (setf neural-network-label-trans) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: neural-network-label-trans (NEURAL-NETWORK neural-network)

Method: (setf neural-network-label-trans) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates a label to a vector

Source

definitions.lisp (file)

Generic Function: neural-network-layout OBJECT

Package

neural-classifier

Methods

Method: neural-network-layout (NEURAL-NETWORK neural-network)

Number of neurons in each layer of the network

Source

definitions.lisp (file)

Generic Function: neural-network-output-trans OBJECT

Generic Function: (setf neural-network-output-trans) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: neural-network-output-trans (NEURAL-NETWORK neural-network)

Method: (setf neural-network-output-trans) NEW-VALUE (NEURAL-NETWORK neural-network)

Function which translates an output vector to a label.

Source

definitions.lisp (file)

5.1.4 Classes

Class: adagrad-optimizer ()

Adagrad optimizer

Package

neural-classifier

Source

optimizers.lisp (file)

Direct superclasses

memoizing-optimizer (class)

Direct methods

learn (method)

Direct Default Initargs

Initarg	Value
:initial-value	1.0e-8

Class: momentum-optimizer ()

SGD optimizer with momentum

Package

neural-classifier

Source

optimizers.lisp (file)

Direct superclasses

memoizing-optimizer (class)

Direct subclasses

nesterov-optimizer (class)

Direct methods

• learn (method)
• momentum-coeff (method)
• momentum-coeff (method)

Direct slots

Slot: coeff

Type

single-float

Initargs

:coeff

Initform

neural-classifier:*momentum-coeff*

Readers

momentum-coeff (generic function)

Writers

(setf momentum-coeff) (generic function)

Class: nesterov-optimizer ()

Nesterov accelerated SGD, improvement of SGD with momentum

Package

neural-classifier

Source

optimizers.lisp (file)

Direct superclasses

momentum-optimizer (class)

Direct methods

learn (method)

Class: neural-network ()

Class for neural networks

Package

neural-classifier

Source

definitions.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• initialize-instance (method)
• neural-network-label-trans (method)
• neural-network-label-trans (method)
• neural-network-input-trans% (method)
• neural-network-input-trans% (method)
• neural-network-output-trans (method)
• neural-network-output-trans (method)
• neural-network-input-trans (method)
• neural-network-input-trans (method)
• neural-network-biases (method)
• neural-network-biases (method)
• neural-network-weights (method)
• neural-network-weights (method)
• neural-network-activation-funcs (method)
• neural-network-activation-funcs (method)
• neural-network-layout (method)

Direct slots

Slot: layout

Number of neurons in each layer of the network

Type

list

Initargs

:layout

Initform

(error "specify number of neurons in each layer")

Readers

neural-network-layout (generic function)

Slot: activation-funcs

List of activation functions.

Type

list

Initargs

:activation-funcs

Readers

neural-network-activation-funcs (generic function)

Writers

(setf neural-network-activation-funcs) (generic function)

Slot: weights

Weight matrices for each layer

Type

list

Readers

neural-network-weights (generic function)

Writers

(setf neural-network-weights) (generic function)

Slot: biases

Bias vectors for each layer

Type

list

Readers

neural-network-biases (generic function)

Writers

(setf neural-network-biases) (generic function)

Slot: input-trans

Function which translates an input object to a vector

Type

function

Initargs

:input-trans

Initform

(function identity)

Readers

neural-network-input-trans (generic function)

Writers

(setf neural-network-input-trans) (generic function)

Slot: output-trans

Function which translates an output vector to a label.

Type

function

Initargs

:output-trans

Initform

(function identity)

Readers

neural-network-output-trans (generic function)

Writers

(setf neural-network-output-trans) (generic function)

Slot: input-trans%

Function which translates an input object to a vector (used for training)

Type

function

Initargs

:input-trans%

Initform

(function identity)

Readers

neural-network-input-trans% (generic function)

Writers

(setf neural-network-input-trans%) (generic function)

Slot: label-trans

Function which translates a label to a vector

Type

function

Initargs

:label-trans

Initform

(function identity)

Readers

neural-network-label-trans (generic function)

Writers

(setf neural-network-label-trans) (generic function)

Class: rmsprop-optimizer ()

RMSprop optimizer

Package

neural-classifier

Source

optimizers.lisp (file)

Direct superclasses

memoizing-optimizer (class)

Direct methods

• learn (method)
• momentum-coeff (method)
• momentum-coeff (method)

Direct slots

Slot: coeff

Type

single-float

Initargs

:coeff

Initform

neural-classifier:*momentum-coeff*

Readers

momentum-coeff (generic function)

Writers

(setf momentum-coeff) (generic function)

Direct Default Initargs

Initarg	Value
:initial-value	1.0e-8

Class: sgd-optimizer ()

The simplest SGD optimizer

Package

neural-classifier

Source

optimizers.lisp (file)

Direct superclasses

optimizer (class)

Direct methods

learn (method)

5.2 Internal definitions

5.2.1 Functions

Function: %expf ARG0

Package

neural-classifier-sbcl

Source

sbcl-hacks.lisp (file)

Function: %sqrtf ARG0

Package

neural-classifier-sbcl

Source

sbcl-hacks.lisp (file)

Function: %tanhf ARG0

Package

neural-classifier-sbcl

Source

sbcl-hacks.lisp (file)

Function: calculate-delta ()

Calculate partial derivative of the cost function by z for all layers

Package

neural-classifier

Source

neural-network.lisp (file)

Function: calculate-gradient ()

Calculate gradient of the cost function

Package

neural-classifier

Source

neural-network.lisp (file)

Function: calculate-gradient-minibatch ()

Calculate gradient of the cost function based on multiple input samples

Package

neural-classifier

Source

neural-network.lisp (file)

Function: calculate-z-and-out ()

Calculate argument and value of activation function for all layers

Package

neural-classifier

Source

neural-network.lisp (file)

Function: print-condition-and-continue C

Package

neural-classifier-sbcl

Source

sbcl-hacks.lisp (file)

Function: random-normal &key MEAN SIGMA

Package

neural-classifier

Source

utility.lisp (file)

Function: sasum MATRIX

Package

neural-classifier

Source

utility.lisp (file)

Function: sigmoid Z

Package

neural-classifier

Source

utility.lisp (file)

Function: symbolicate &rest ARGS

Package

neural-classifier-sbcl

Source

sbcl-hacks.lisp (file)

5.2.2 Generic functions

Generic Function: activation VECTOR TYPE

Apply activation function TYPE to a VECTOR

Package

neural-classifier

Source

utility.lisp (file)

Methods

Method: activation VECTOR (TYPE (eql softmax))

Method: activation VECTOR (TYPE (eql relu))

Method: activation VECTOR (TYPE (eql abs))

Method: activation VECTOR (TYPE (eql tanh))

Method: activation VECTOR (TYPE (eql sigmoid))

Generic Function: activation-derivative VECTOR TYPE

Apply derivative of activation function TYPE to a VECTOR

Package

neural-classifier

Source

utility.lisp (file)

Methods

Method: activation-derivative VECTOR (TYPE (eql relu))

Method: activation-derivative VECTOR (TYPE (eql abs))

Method: activation-derivative VECTOR (TYPE (eql tanh))

Method: activation-derivative VECTOR (TYPE (eql sigmoid))

Generic Function: learn OPTIMIZER NEURAL-NETWORK SAMPLES

Update network parameters using SAMPLES for training.

Package

neural-classifier

Source

optimizers.lisp (file)

Methods

Method: learn (OPTIMIZER rmsprop-optimizer) NEURAL-NETWORK SAMPLES

Source

neural-network.lisp (file)

Method: learn (OPTIMIZER adagrad-optimizer) NEURAL-NETWORK SAMPLES

Source

neural-network.lisp (file)

Method: learn (OPTIMIZER nesterov-optimizer) NEURAL-NETWORK SAMPLES

Source

neural-network.lisp (file)

Method: learn (OPTIMIZER momentum-optimizer) NEURAL-NETWORK SAMPLES

Source

neural-network.lisp (file)

Method: learn (OPTIMIZER sgd-optimizer) NEURAL-NETWORK SAMPLES

Source

neural-network.lisp (file)

Generic Function: momentum-coeff OBJECT

Generic Function: (setf momentum-coeff) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: momentum-coeff (RMSPROP-OPTIMIZER rmsprop-optimizer)

automatically generated reader method

Source

optimizers.lisp (file)

Method: (setf momentum-coeff) NEW-VALUE (RMSPROP-OPTIMIZER rmsprop-optimizer)

automatically generated writer method

Source

optimizers.lisp (file)

Method: momentum-coeff (MOMENTUM-OPTIMIZER momentum-optimizer)

automatically generated reader method

Source

optimizers.lisp (file)

Method: (setf momentum-coeff) NEW-VALUE (MOMENTUM-OPTIMIZER momentum-optimizer)

automatically generated writer method

Source

optimizers.lisp (file)

Generic Function: neural-network-activation-funcs OBJECT

Generic Function: (setf neural-network-activation-funcs) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: neural-network-activation-funcs (NEURAL-NETWORK neural-network)

Method: (setf neural-network-activation-funcs) NEW-VALUE (NEURAL-NETWORK neural-network)

List of activation functions.

Source

definitions.lisp (file)

Generic Function: neural-network-biases OBJECT

Generic Function: (setf neural-network-biases) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: neural-network-biases (NEURAL-NETWORK neural-network)

Method: (setf neural-network-biases) NEW-VALUE (NEURAL-NETWORK neural-network)

Bias vectors for each layer

Source

definitions.lisp (file)

Generic Function: neural-network-weights OBJECT

Generic Function: (setf neural-network-weights) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: neural-network-weights (NEURAL-NETWORK neural-network)

Method: (setf neural-network-weights) NEW-VALUE (NEURAL-NETWORK neural-network)

Weight matrices for each layer

Source

definitions.lisp (file)

Generic Function: optimizer-biases OBJECT

Generic Function: (setf optimizer-biases) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: optimizer-biases (MEMOIZING-OPTIMIZER memoizing-optimizer)

automatically generated reader method

Source

optimizers.lisp (file)

Method: (setf optimizer-biases) NEW-VALUE (MEMOIZING-OPTIMIZER memoizing-optimizer)

automatically generated writer method

Source

optimizers.lisp (file)

Generic Function: optimizer-initial-value OBJECT

Generic Function: (setf optimizer-initial-value) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: optimizer-initial-value (MEMOIZING-OPTIMIZER memoizing-optimizer)

automatically generated reader method

Source

optimizers.lisp (file)

Method: (setf optimizer-initial-value) NEW-VALUE (MEMOIZING-OPTIMIZER memoizing-optimizer)

automatically generated writer method

Source

optimizers.lisp (file)

Generic Function: optimizer-weights OBJECT

Generic Function: (setf optimizer-weights) NEW-VALUE OBJECT

Package

neural-classifier

Methods

Method: optimizer-weights (MEMOIZING-OPTIMIZER memoizing-optimizer)

automatically generated reader method

Source

optimizers.lisp (file)

Method: (setf optimizer-weights) NEW-VALUE (MEMOIZING-OPTIMIZER memoizing-optimizer)

automatically generated writer method

Source

optimizers.lisp (file)

5.2.3 Classes

Class: memoizing-optimizer ()

Optimizer which memoizes some old state related to weights and biases. Not to be instantiated.

Package

neural-classifier

Source

optimizers.lisp (file)

Direct superclasses

optimizer (class)

Direct subclasses

• momentum-optimizer (class)
• adagrad-optimizer (class)
• rmsprop-optimizer (class)

Direct methods

• initialize-instance (method)
• optimizer-initial-value (method)
• optimizer-initial-value (method)
• optimizer-biases (method)
• optimizer-biases (method)
• optimizer-weights (method)
• optimizer-weights (method)

Direct slots

Slot: weights

Type

list

Readers

optimizer-weights (generic function)

Writers

(setf optimizer-weights) (generic function)

Slot: biases

Type

list

Readers

optimizer-biases (generic function)

Writers

(setf optimizer-biases) (generic function)

Slot: initial-value

Type

single-float

Initargs

:initial-value

Initform

0.0

Readers

optimizer-initial-value (generic function)

Writers

(setf optimizer-initial-value) (generic function)

Class: optimizer ()

Generic optimizer class. Not to be instantiated

Package

neural-classifier

Source

optimizers.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

• sgd-optimizer (class)
• memoizing-optimizer (class)

5.2.4 Types

Type: activation-symbol ()

Package

neural-classifier

Source

definitions.lisp (file)

Type: non-negative-fixnum ()

Package

neural-classifier

Source

definitions.lisp (file)

Type: positive-fixnum ()

Package

neural-classifier

Source

definitions.lisp (file)

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables

A.4 Data types