The simple-neural-network Reference Manual

This is the simple-neural-network Reference Manual, version 3.1, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Feb 26 17:53:29 2024 GMT+0.

Table of Contents


1 Introduction


2 Systems

The main system appears first, followed by any subsystem dependency.


2.1 simple-neural-network

Simple neural network

Author

Guillaume Le Vaillant

License

GPL-3

Version

3.1

Dependencies
  • cl-store (system).
  • lparallel (system).
Source

simple-neural-network.asd.

Child Component

simple-neural-network.lisp (file).


3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


3.1 Lisp


3.1.1 simple-neural-network/simple-neural-network.asd

Source

simple-neural-network.asd.

Parent Component

simple-neural-network (system).

ASDF Systems

simple-neural-network.


3.1.2 simple-neural-network/simple-neural-network.lisp

Source

simple-neural-network.asd.

Parent Component

simple-neural-network (system).

Packages

simple-neural-network.

Public Interface
Internals

4 Packages

Packages are listed by definition order.


4.1 simple-neural-network

Source

simple-neural-network.lisp.

Nickname

snn

Use List

common-lisp.

Public Interface
Internals

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


5.1 Public Interface


5.1.1 Ordinary functions

Function: accuracy (neural-network inputs targets &key test)

Return the rate of good guesses computed by the NEURAL-NETWORK when testing it with some INPUTS and TARGETS. TEST must be a function taking an output and a target returning T if the output is considered to be close enough to the target, and NIL otherwise. SAME-CATEGORY-P is used by default.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: copy (neural-network)

Return a copy of the NEURAL-NETWORK.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: create-neural-network (input-size output-size &rest hidden-layers-sizes)

Create a neural network having INPUT-SIZE inputs, OUTPUT-SIZE outputs, and optionally some intermediary layers whose sizes are specified by HIDDEN-LAYERS-SIZES. The neural network is initialized with random weights and biases.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: find-learning-rate (neural-network inputs targets &key batch-size momentum-coefficient epochs iterations minimum maximum)

Return the best learing rate found in ITERATIONS steps of dichotomic search (between MINIMUM and MAXIMUM). In each step, the NEURAL-NETWORK is trained EPOCHS times using some INPUTS, TARGETS, BATCH-SIZE and MOMENTUM-COEFFICIENT.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: find-normalization (inputs)

Return four values. The first is a normalization function taking an input and returning a normalized input. Applying this normalization function to the inputs gives a data set in which each variable has mean 0 and standard deviation 1. The second is a denormalization function that can compute the original input from the normalized one. The third is the code of the normalization function. The fourth is the code of the denormalization function.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: index-of-max-value (values)

Return the index of the greatest value in VALUES.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: mean-absolute-error (neural-network inputs targets)

Return the mean absolute error on the outputs computed by the NEURAL-NETWORK when testing it with some INPUTS and TARGETS.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Reader: neural-network-biases (instance)
Writer: (setf neural-network-biases) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

biases.

Reader: neural-network-layers (instance)
Writer: (setf neural-network-layers) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

layers.

Reader: neural-network-weights (instance)
Writer: (setf neural-network-weights) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

weights.

Function: predict (neural-network input &optional output)

Return the output computed by the NEURAL-NETWORK for a given INPUT. If OUTPUT is not NIL, the output is written in it, otherwise a new vector is allocated.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: restore (place)

Restore the neural network stored in PLACE, which must be a stream or a pathname-designator.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: same-category-p (output target)

Return T if calls to INDEX-OF-MAX-VALUE on OUTPUT and TARGET return the same value, and NIL otherwise.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: store (neural-network place)

Store the NEURAL-NETWORK to PLACE, which must be a stream or a pathname-designator.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: train (neural-network inputs targets learning-rate &key batch-size momentum-coefficient)

Train the NEURAL-NETWORK with the given LEARNING-RATE and MOMENTUM-COEFFICIENT using some INPUTS and TARGETS. The weights are updated every BATCH-SIZE inputs.

Package

simple-neural-network.

Source

simple-neural-network.lisp.


5.2 Internals


5.2.1 Macros

Macro: %dotimes ((var count &optional result) &body body)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Macro: %mapc (function list &rest more-lists)
Package

simple-neural-network.

Source

simple-neural-network.lisp.


5.2.2 Ordinary functions

Function: activation (x)

Activation function for the neurons.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: activation-prime (y)

Derivative of the activation function.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: add-gradients (input weight-gradients bias-gradients delta)

Add the gradients computed for an input to sum of the gradients for previous inputs.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: add-weight-gradient (input gradients delta index)

Add the gradients computed for an input for the weights of the neuron at INDEX in a layer to the sum of the gradients for previous inputs.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: average-gradient (gradient batch-size)

Compute the average gradients for a layer.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: average-gradients (neural-network batch-size)

Compute the average gradients for the whole NEURAL-NETWORK.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: backpropagate (neural-network)

Propagate the error of the output layer of the NEURAL-NETWORK back to the first layer and compute the gradients.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: clear-momentum (momentum)

Reset momentum to 0.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: clear-momentums (neural-network)

Reset all the momentums to 0.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: compute-delta (previous-delta output weights delta)

Compute the error of the OUTPUT layer based on the error of the next layer.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: compute-output-delta (neural-network target)

Compute the error between the output layer of the NEURAL-NETWORK and the TARGET.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: compute-single-delta (previous-delta output weights delta index)

Compute the delta for the neuron at INDEX in the OUTPUT layer.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: compute-value (input output weights biases index)

Compute the values of the neuron at INDEX in the OUTPUT layer.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: compute-values (input output weights biases)

Compute the values of the neurons in the OUTPUT layer.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: copy-neural-network (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: denormalize (input means standard-deviations)

Return the original input computed from its normalized variant.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: get-output (neural-network)

Return the output layer of the NEURAL-NETWORK.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: make-double-float-array (size)

Make a new array of SIZE double floats.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: make-neural-network (&key layers weights biases deltas weight-gradients weight-momentums bias-gradients bias-momentums)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: make-random-weights (input-size output-size)

Generate a matrix (OUTPUT-SIZE * INPUT-SIZE) of random weights.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: means (inputs)

Return the means of the variables of the INPUTS.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Reader: neural-network-bias-gradients (instance)
Writer: (setf neural-network-bias-gradients) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

bias-gradients.

Reader: neural-network-bias-momentums (instance)
Writer: (setf neural-network-bias-momentums) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

bias-momentums.

Reader: neural-network-deltas (instance)
Writer: (setf neural-network-deltas) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

deltas.

Function: neural-network-p (object)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Reader: neural-network-weight-gradients (instance)
Writer: (setf neural-network-weight-gradients) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

weight-gradients.

Reader: neural-network-weight-momentums (instance)
Writer: (setf neural-network-weight-momentums) (instance)
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Target Slot

weight-momentums.

Function: normalize (input means standard-deviations)

Return a normalized variant of the INPUT.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: propagate (neural-network)

Propagate the values of the input layer of the NEURAL-NETWORK to the output layer.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: set-input (neural-network input)

Set the input layer of the NEURAL-NETWORK to INPUT.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: standard-deviations (inputs means)

Return the standard deviations of the variables of the INPUTS.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: update-weights (weights gradients momentums gradient-coefficient momentum-coefficient)

Update the WEIGHTS and MOMENTUMS of a layer and clear the GRADIENTS.

Package

simple-neural-network.

Source

simple-neural-network.lisp.

Function: update-weights-and-biases (neural-network learning-rate momentum-coefficient)

Update all the weights and biases of the NEURAL-NETWORK.

Package

simple-neural-network.

Source

simple-neural-network.lisp.


5.2.3 Structures

Structure: neural-network
Package

simple-neural-network.

Source

simple-neural-network.lisp.

Direct superclasses

structure-object.

Direct slots
Slot: layers
Readers

neural-network-layers.

Writers

(setf neural-network-layers).

Slot: weights
Readers

neural-network-weights.

Writers

(setf neural-network-weights).

Slot: biases
Readers

neural-network-biases.

Writers

(setf neural-network-biases).

Slot: deltas
Readers

neural-network-deltas.

Writers

(setf neural-network-deltas).

Slot: weight-gradients
Readers

neural-network-weight-gradients.

Writers

(setf neural-network-weight-gradients).

Slot: weight-momentums
Readers

neural-network-weight-momentums.

Writers

(setf neural-network-weight-momentums).

Slot: bias-gradients
Readers

neural-network-bias-gradients.

Writers

(setf neural-network-bias-gradients).

Slot: bias-momentums
Readers

neural-network-bias-momentums.

Writers

(setf neural-network-bias-momentums).


5.2.4 Types

Type: double-float-array ()
Package

simple-neural-network.

Source

simple-neural-network.lisp.


Appendix A Indexes


A.1 Concepts


A.2 Functions

Jump to:   %   (  
A   B   C   D   F   G   I   M   N   P   R   S   T   U  
Index Entry  Section

%
%dotimes: Private macros
%mapc: Private macros

(
(setf neural-network-bias-gradients): Private ordinary functions
(setf neural-network-bias-momentums): Private ordinary functions
(setf neural-network-biases): Public ordinary functions
(setf neural-network-deltas): Private ordinary functions
(setf neural-network-layers): Public ordinary functions
(setf neural-network-weight-gradients): Private ordinary functions
(setf neural-network-weight-momentums): Private ordinary functions
(setf neural-network-weights): Public ordinary functions

A
accuracy: Public ordinary functions
activation: Private ordinary functions
activation-prime: Private ordinary functions
add-gradients: Private ordinary functions
add-weight-gradient: Private ordinary functions
average-gradient: Private ordinary functions
average-gradients: Private ordinary functions

B
backpropagate: Private ordinary functions

C
clear-momentum: Private ordinary functions
clear-momentums: Private ordinary functions
compute-delta: Private ordinary functions
compute-output-delta: Private ordinary functions
compute-single-delta: Private ordinary functions
compute-value: Private ordinary functions
compute-values: Private ordinary functions
copy: Public ordinary functions
copy-neural-network: Private ordinary functions
create-neural-network: Public ordinary functions

D
denormalize: Private ordinary functions

F
find-learning-rate: Public ordinary functions
find-normalization: Public ordinary functions
Function, (setf neural-network-bias-gradients): Private ordinary functions
Function, (setf neural-network-bias-momentums): Private ordinary functions
Function, (setf neural-network-biases): Public ordinary functions
Function, (setf neural-network-deltas): Private ordinary functions
Function, (setf neural-network-layers): Public ordinary functions
Function, (setf neural-network-weight-gradients): Private ordinary functions
Function, (setf neural-network-weight-momentums): Private ordinary functions
Function, (setf neural-network-weights): Public ordinary functions
Function, accuracy: Public ordinary functions
Function, activation: Private ordinary functions
Function, activation-prime: Private ordinary functions
Function, add-gradients: Private ordinary functions
Function, add-weight-gradient: Private ordinary functions
Function, average-gradient: Private ordinary functions
Function, average-gradients: Private ordinary functions
Function, backpropagate: Private ordinary functions
Function, clear-momentum: Private ordinary functions
Function, clear-momentums: Private ordinary functions
Function, compute-delta: Private ordinary functions
Function, compute-output-delta: Private ordinary functions
Function, compute-single-delta: Private ordinary functions
Function, compute-value: Private ordinary functions
Function, compute-values: Private ordinary functions
Function, copy: Public ordinary functions
Function, copy-neural-network: Private ordinary functions
Function, create-neural-network: Public ordinary functions
Function, denormalize: Private ordinary functions
Function, find-learning-rate: Public ordinary functions
Function, find-normalization: Public ordinary functions
Function, get-output: Private ordinary functions
Function, index-of-max-value: Public ordinary functions
Function, make-double-float-array: Private ordinary functions
Function, make-neural-network: Private ordinary functions
Function, make-random-weights: Private ordinary functions
Function, mean-absolute-error: Public ordinary functions
Function, means: Private ordinary functions
Function, neural-network-bias-gradients: Private ordinary functions
Function, neural-network-bias-momentums: Private ordinary functions
Function, neural-network-biases: Public ordinary functions
Function, neural-network-deltas: Private ordinary functions
Function, neural-network-layers: Public ordinary functions
Function, neural-network-p: Private ordinary functions
Function, neural-network-weight-gradients: Private ordinary functions
Function, neural-network-weight-momentums: Private ordinary functions
Function, neural-network-weights: Public ordinary functions
Function, normalize: Private ordinary functions
Function, predict: Public ordinary functions
Function, propagate: Private ordinary functions
Function, restore: Public ordinary functions
Function, same-category-p: Public ordinary functions
Function, set-input: Private ordinary functions
Function, standard-deviations: Private ordinary functions
Function, store: Public ordinary functions
Function, train: Public ordinary functions
Function, update-weights: Private ordinary functions
Function, update-weights-and-biases: Private ordinary functions

G
get-output: Private ordinary functions

I
index-of-max-value: Public ordinary functions

M
Macro, %dotimes: Private macros
Macro, %mapc: Private macros
make-double-float-array: Private ordinary functions
make-neural-network: Private ordinary functions
make-random-weights: Private ordinary functions
mean-absolute-error: Public ordinary functions
means: Private ordinary functions

N
neural-network-bias-gradients: Private ordinary functions
neural-network-bias-momentums: Private ordinary functions
neural-network-biases: Public ordinary functions
neural-network-deltas: Private ordinary functions
neural-network-layers: Public ordinary functions
neural-network-p: Private ordinary functions
neural-network-weight-gradients: Private ordinary functions
neural-network-weight-momentums: Private ordinary functions
neural-network-weights: Public ordinary functions
normalize: Private ordinary functions

P
predict: Public ordinary functions
propagate: Private ordinary functions

R
restore: Public ordinary functions

S
same-category-p: Public ordinary functions
set-input: Private ordinary functions
standard-deviations: Private ordinary functions
store: Public ordinary functions

T
train: Public ordinary functions

U
update-weights: Private ordinary functions
update-weights-and-biases: Private ordinary functions