The cl-random-forest Reference Manual

Table of Contents

The cl-random-forest Reference Manual

This is the cl-random-forest Reference Manual, version 0.1, generated automatically by Declt version 3.0 "Montgomery Scott" on Tue Dec 22 12:43:07 2020 GMT+0.

1 Introduction

* cl-random-forest

[[http://quickdocs.org/cl-random-forest/][http://quickdocs.org/badge/cl-random-forest.svg]]
[[https://github.com/masatoi/cl-random-forest/actions?query=workflow%3ACI][https://github.com/masatoi/cl-random-forest/workflows/CI/badge.svg]]


Cl-random-forest is a implementation of Random Forest for multiclass classification and univariate regression written in Common Lisp. It also includes a implementation of Global Refinement of Random Forest (Ren, Cao, Wei and Sun. "Global Refinement of Random Forest" CVPR2015). This refinement makes faster and more accurate than standard Random Forest.

** Features and Limitations

- Faster and more accurate than other major implementations such as scikit-learn (Python/Cython) or ranger (R/C++)

|         | scikit-learn     | ranger           | cl-random-forest |
| MNIST   | 96.95%, 41.72sec | 97.17%, 69.34sec | *98.29%*, *12.68sec* |
| letter  | 96.38%, 2.569sec | 96.42%, *1.828sec* | *97.32%*, 3.497sec |
| covtype | 94.89%, 263.7sec | 83.95%, 139.0sec | *96.01%*, *103.9sec* |
| usps    | 93,47%, 3.583sec | 93.57%, 11.70sec | *94.96%*, *0.686sec* |

- Supporting parallelization of training and prediction (tested on SBCL, CCL)

- It also includes Global Pruning algorithm of Random Forest which can make the model extremely compact

- Currently, multivariate regression is not implemented

** Installation

In quicklisp’s local-projects directory,

#+BEGIN_SRC
git clone https://github.com/masatoi/cl-online-learning.git
git clone https://github.com/masatoi/cl-random-forest.git
#+END_SRC

In Lisp,

#+BEGIN_SRC lisp
(ql:quickload :cl-random-forest)
#+END_SRC

When using Roswell,

#+BEGIN_SRC 
ros install masatoi/cl-online-learning masatoi/cl-random-forest
#+END_SRC

** Usage
*** Classification
**** Prepare training dataset
A dataset consists of a target vector and a input data matrix.
For classification, the target vector should be a fixnum simple-vector and the data matrix should be a 2-dimensional double-float array whose row corresponds one datum.
Note that the target is a integer starting from 0.
For example, the following dataset is valid for 4-class classification with 2-dimensional input.

#+BEGIN_SRC lisp
(defparameter *target*
  (make-array 11 :element-type 'fixnum
                 :initial-contents '(0 0 1 1 2 2 2 3 3 3 3)))

(defparameter *datamatrix*
  (make-array '(11 2)
              :element-type 'double-float
              :initial-contents '((-1.0d0 -2.0d0)
                                  (-2.0d0 -1.5d0)
                                  (1.0d0 -2.0d0)
                                  (3.0d0 -1.5d0)
                                  (-2.0d0 2.0d0)
                                  (-3.0d0 1.0d0)
                                  (-2.0d0 1.0d0)
                                  (3.0d0 2.0d0)
                                  (2.0d0 2.0d0)
                                  (1.0d0 2.0d0)
                                  (1.0d0 1.0d0))))
#+END_SRC

[[./docs/img/clrf-example-simple.png]]

**** Make Decision Tree

To construct a decision tree, MAKE-DTREE function is available. This function receives the number of classes, the data matrix and the target vector and then returns a decision tree object. This function also receives optionally the max depth of the tree and the minimum number of samples in the region the tree divides and the number of trials of splits.

#+BEGIN_SRC lisp
(defparameter *n-class* 4)

(defparameter *dtree*
  (make-dtree *n-class* *datamatrix* *target*
              :max-depth 5 :min-region-samples 1 :n-trial 10))
#+END_SRC

Next, make a prediction from the constructed decision tree with PREDICT-DTREE function. For example, to predict the first datum in the data matrix with this decision tree, do as follows.

#+BEGIN_SRC lisp
(predict-dtree *dtree* *datamatrix* 0)
;; => 0 (correct class id)
#+END_SRC

To make predictions for the entire dataset and calculate the accuracy, use TEST-DTREE function.

#+BEGIN_SRC lisp
(test-dtree *dtree* *datamatrix* *target*)
;; Accuracy: 100.0%, Correct: 11, Total: 11
#+END_SRC

**** Make Random Forest

To construct a random forest, MAKE-FOREST function is available. In addition to the MAKE-DTREE function arguments, this function receives optionally the number of decision trees and the bagging ratio that is used for sampling from training data to construct each sub decision trees.

#+BEGIN_SRC lisp
(defparameter *forest*
  (make-forest *n-class* *datamatrix* *target*
               :n-tree 10 :bagging-ratio 1.0
               :max-depth 5 :min-region-samples 1 :n-trial 10))
#+END_SRC

Prediction and test of random forest are done in the almost same way as decision trees. PREDICT-FOREST function and TEST-FOREST function are available for each purpose.

#+BEGIN_SRC lisp
(predict-forest *forest* *datamatrix* 0)
;; => 0 (correct class id)

(test-forest *forest* *datamatrix* *target*)
;; Accuracy: 100.0%, Correct: 11, Total: 11
#+END_SRC

**** Global Refinement of Random Forest

Cl-random-forest has a way to improve pre-trained random forest using global information between each decision trees.
For this purpose, we make an another dataset from original dataset and pre-trained random forest. 
When an original datum input into the random forest, the datum enters into a region which corresponds one leaf node for each decision trees.
The datum of the new dataset represents which position of leaf node the original datum entered for each decision tree.
Then we train a linear classifier (AROW) using this new dataset and the original target.

#+BEGIN_SRC lisp
;; Make refine learner
(defparameter *forest-learner* (make-refine-learner *forest*))

;; Make refine dataset
(defparameter *forest-refine-dataset* (make-refine-dataset *forest* *datamatrix*))

;; Train refine learner
(train-refine-learner *forest-learner* *forest-refine-dataset* *target*)

;; Test refine learner
(test-refine-learner  *forest-learner* *forest-refine-dataset* *target*)
#+END_SRC

This TRAIN-REFINE-LEARNER function can be used to learn the dataset collectively, but it may be necessary to call this function several times until learning converges. TRAIN-REFINE-LEARNER-PROCESS function is used for training until converged.

#+BEGIN_SRC lisp
(train-refine-learner-process *forest-learner* *forest-refine-dataset* *target*
                              *forest-refine-dev-dataset* *dev-target*)
#+END_SRC

**** Global Pruning of Random Forest

***** Pruning
Global pruning is a method for compactization of the model size of the random forest using information of the global-refinement learner. A leaf node in a decision tree is no longer necessary when its corresponding element of the weight vector of the global-refinement learner has a small value norm.

To prune a forest destructively, after training the global-refinement learner, run PRUNING! function.

#+BEGIN_SRC lisp
;; Prune *forest*
(pruning! *forest* *forest-learner* 0.1)
#+END_SRC

The third argument is pruning rate. In this case, 10% leaf nodes are deleted.

***** Re-learning

After pruning, it is required to re-learn the global-refinement learner.

#+BEGIN_SRC lisp
;; Re-learning of refine-learner
(setf *forest-refine-dataset* (make-refine-dataset *forest* *datamatrix*))
(setf *forest-learner* (make-refine-learner *forest*))
(train-refine-learner *forest-learner* *forest-refine-dataset* *target*)
(test-refine-learner  *forest-learner* *forest-refine-dataset* *target*)
#+END_SRC

The following figure shows the accuracy for test dataset and the number of leaf nodes when repeating pruning and re-learning on the MNIST dataset. We can see that the performance hardly changes even if the number of leaf nodes decreases to about 1/10.

[[./docs/img/clrf-mnist-pruning.png]]

**** Parallelization
The following several functions can be parallelized with [[https://lparallel.org/][lparallel]].

- MAKE-FOREST
- MAKE-REGRESSION-FOREST
- MAKE-REFINE-DATASET
- TRAIN-REFINE-LEARNER

To enable/disable parallelization, set lparallel's kernel object. For example, to enable parallelization with 4 threads,

#+BEGIN_SRC lisp
;; Enable parallelization
(setf lparallel:*kernel* (lparallel:make-kernel 4))

;; Disable parallelization
(setf lparallel:*kernel* nil)
#+END_SRC

*** Regression
**** Prepare training dataset
In case of classification, the target is a vector of integer values, whereas in regression is a vector of continuous values.

#+BEGIN_SRC lisp
(defparameter *n* 100)

(defparameter *datamatrix*
  (let ((arr (make-array (list *n* 1) :element-type 'double-float)))
    (loop for i from 0 below *n* do
      (setf (aref arr i 0) (random-uniform (- pi) pi)))
    arr))

(defparameter *target*
  (let ((arr (make-array *n* :element-type 'double-float)))
    (loop for i from 0 below *n* do
      (setf (aref arr i) (+ (sin (aref *datamatrix* i 0))
                            (random-normal :sd 0.1d0))))
    arr))

(defparameter *test*
  (let ((arr (make-array (list *n* 1) :element-type 'double-float)))
    (loop for i from 0 below *n*
          for x from (- pi) to pi by (/ (* 2 pi) *n*)
          do (setf (aref arr i 0) x))
    arr))

(defparameter *test-target*
  (let ((arr (make-array *n* :element-type 'double-float)))
    (loop for i from 0 below *n* do
      (setf (aref arr i) (sin (aref *test* i 0))))
    arr))
#+END_SRC

**** Make Regression Tree

#+BEGIN_SRC lisp
;; Make regression tree
(defparameter *rtree*
  (make-rtree *datamatrix* *target* :max-depth 5 :min-region-samples 5 :n-trial 10))

;; Testing
(test-rtree *rtree* *test* *test-target*)
; RMSE: 0.09220732459820888d0

;; Make a prediction for first data point of test dataset
(predict-rtree *rtree* *test* 0)
; => -0.08374452528780077d0
#+END_SRC

**** Make Random Forest for Regression

#+BEGIN_SRC lisp
;; Make regression tree forest
(defparameter *rforest*
  (make-regression-forest *datamatrix* *target*
                          :n-tree 100 :bagging-ratio 0.6
                          :max-depth 5 :min-region-samples 5 :n-trial 10))

;; Testing
(test-regression-forest *rforest* *test* *test-target*)
; RMSE: 0.05006872795207973d0

;; Make a prediction for first data point of test dataset
(predict-regression-forest *rforest* *test* 0)
; => -0.16540771296145781d0
#+END_SRC

[[./docs/img/clrf-regression.png]]

** Author
Satoshi Imai (satoshi.imai@gmail.com)

** Licence
This software is released under the MIT License, see LICENSE.txt.

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 cl-random-forest

Author

Satoshi Imai

License

MIT Licence

Description

Random Forest and Global Refinement for Common Lisp

Long Description

* cl-random-forest

[[http://quickdocs.org/cl-random-forest/][http://quickdocs.org/badge/cl-random-forest.svg]]
[[https://github.com/masatoi/cl-random-forest/actions?query=workflow%3ACI][https://github.com/masatoi/cl-random-forest/workflows/CI/badge.svg]]

Cl-random-forest is a implementation of Random Forest for multiclass classification and univariate regression written in Common Lisp. It also includes a implementation of Global Refinement of Random Forest (Ren, Cao, Wei and Sun. "Global Refinement of Random Forest" CVPR2015). This refinement makes faster and more accurate than standard Random Forest.

** Features and Limitations

- Faster and more accurate than other major implementations such as scikit-learn (Python/Cython) or ranger (R/C++)

| | scikit-learn | ranger | cl-random-forest |
| MNIST | 96.95%, 41.72sec | 97.17%, 69.34sec | *98.29%*, *12.68sec* |
| letter | 96.38%, 2.569sec | 96.42%, *1.828sec* | *97.32%*, 3.497sec |
| covtype | 94.89%, 263.7sec | 83.95%, 139.0sec | *96.01%*, *103.9sec* |
| usps | 93,47%, 3.583sec | 93.57%, 11.70sec | *94.96%*, *0.686sec* |

- Supporting parallelization of training and prediction (tested on SBCL, CCL)

- It also includes Global Pruning algorithm of Random Forest which can make the model extremely compact

- Currently, multivariate regression is not implemented

** Installation

In quicklisp’s local-projects directory,

#+BEGIN_SRC
git clone https://github.com/masatoi/cl-online-learning.git
git clone https://github.com/masatoi/cl-random-forest.git
#+END_SRC

In Lisp,

#+BEGIN_SRC lisp
(ql:quickload :cl-random-forest)
#+END_SRC

When using Roswell,

#+BEGIN_SRC
ros install masatoi/cl-online-learning masatoi/cl-random-forest
#+END_SRC

** Usage
*** Classification
**** Prepare training dataset
A dataset consists of a target vector and a input data matrix.
For classification, the target vector should be a fixnum simple-vector and the data matrix should be a 2-dimensional double-float array whose row corresponds one datum.
Note that the target is a integer starting from 0.
For example, the following dataset is valid for 4-class classification with 2-dimensional input.

#+BEGIN_SRC lisp
(defparameter *target*
(make-array 11 :element-type ’fixnum
:initial-contents ’(0 0 1 1 2 2 2 3 3 3 3)))

(defparameter *datamatrix*
(make-array ’(11 2)
:element-type ’double-float
:initial-contents ’((-1.0d0 -2.0d0)
(-2.0d0 -1.5d0)
(1.0d0 -2.0d0)
(3.0d0 -1.5d0)
(-2.0d0 2.0d0)
(-3.0d0 1.0d0)
(-2.0d0 1.0d0)
(3.0d0 2.0d0)
(2.0d0 2.0d0)
(1.0d0 2.0d0)
(1.0d0 1.0d0))))
#+END_SRC

[[./docs/img/clrf-example-simple.png]]

**** Make Decision Tree

To construct a decision tree, MAKE-DTREE function is available. This function receives the number of classes, the data matrix and the target vector and then returns a decision tree object. This function also receives optionally the max depth of the tree and the minimum number of samples in the region the tree divides and the number of trials of splits.

#+BEGIN_SRC lisp
(defparameter *n-class* 4)

(defparameter *dtree*
(make-dtree *n-class* *datamatrix* *target*
:max-depth 5 :min-region-samples 1 :n-trial 10))
#+END_SRC

Next, make a prediction from the constructed decision tree with PREDICT-DTREE function. For example, to predict the first datum in the data matrix with this decision tree, do as follows.

#+BEGIN_SRC lisp
(predict-dtree *dtree* *datamatrix* 0)
;; => 0 (correct class id)
#+END_SRC

To make predictions for the entire dataset and calculate the accuracy, use TEST-DTREE function.

#+BEGIN_SRC lisp
(test-dtree *dtree* *datamatrix* *target*)
;; Accuracy: 100.0%, Correct: 11, Total: 11
#+END_SRC

**** Make Random Forest

To construct a random forest, MAKE-FOREST function is available. In addition to the MAKE-DTREE function arguments, this function receives optionally the number of decision trees and the bagging ratio that is used for sampling from training data to construct each sub decision trees.

#+BEGIN_SRC lisp
(defparameter *forest*
(make-forest *n-class* *datamatrix* *target*
:n-tree 10 :bagging-ratio 1.0
:max-depth 5 :min-region-samples 1 :n-trial 10))
#+END_SRC

Prediction and test of random forest are done in the almost same way as decision trees. PREDICT-FOREST function and TEST-FOREST function are available for each purpose.

#+BEGIN_SRC lisp
(predict-forest *forest* *datamatrix* 0)
;; => 0 (correct class id)

(test-forest *forest* *datamatrix* *target*)
;; Accuracy: 100.0%, Correct: 11, Total: 11
#+END_SRC

**** Global Refinement of Random Forest

Cl-random-forest has a way to improve pre-trained random forest using global information between each decision trees.
For this purpose, we make an another dataset from original dataset and pre-trained random forest.
When an original datum input into the random forest, the datum enters into a region which corresponds one leaf node for each decision trees.
The datum of the new dataset represents which position of leaf node the original datum entered for each decision tree.
Then we train a linear classifier (AROW) using this new dataset and the original target.

#+BEGIN_SRC lisp
;; Make refine learner
(defparameter *forest-learner* (make-refine-learner *forest*))

;; Make refine dataset
(defparameter *forest-refine-dataset* (make-refine-dataset *forest* *datamatrix*))

;; Train refine learner
(train-refine-learner *forest-learner* *forest-refine-dataset* *target*)

;; Test refine learner
(test-refine-learner *forest-learner* *forest-refine-dataset* *target*)
#+END_SRC

This TRAIN-REFINE-LEARNER function can be used to learn the dataset collectively, but it may be necessary to call this function several times until learning converges. TRAIN-REFINE-LEARNER-PROCESS function is used for training until converged.

#+BEGIN_SRC lisp
(train-refine-learner-process *forest-learner* *forest-refine-dataset* *target*
*forest-refine-dev-dataset* *dev-target*)
#+END_SRC

**** Global Pruning of Random Forest

***** Pruning
Global pruning is a method for compactization of the model size of the random forest using information of the global-refinement learner. A leaf node in a decision tree is no longer necessary when its corresponding element of the weight vector of the global-refinement learner has a small value norm.

To prune a forest destructively, after training the global-refinement learner, run PRUNING! function.

#+BEGIN_SRC lisp
;; Prune *forest*
(pruning! *forest* *forest-learner* 0.1)
#+END_SRC

The third argument is pruning rate. In this case, 10% leaf nodes are deleted.

***** Re-learning

After pruning, it is required to re-learn the global-refinement learner.

#+BEGIN_SRC lisp
;; Re-learning of refine-learner
(setf *forest-refine-dataset* (make-refine-dataset *forest* *datamatrix*))
(setf *forest-learner* (make-refine-learner *forest*))
(train-refine-learner *forest-learner* *forest-refine-dataset* *target*)
(test-refine-learner *forest-learner* *forest-refine-dataset* *target*)
#+END_SRC

The following figure shows the accuracy for test dataset and the number of leaf nodes when repeating pruning and re-learning on the MNIST dataset. We can see that the performance hardly changes even if the number of leaf nodes decreases to about 1/10.

[[./docs/img/clrf-mnist-pruning.png]]

**** Parallelization
The following several functions can be parallelized with [[https://lparallel.org/][lparallel]].

- MAKE-FOREST
- MAKE-REGRESSION-FOREST
- MAKE-REFINE-DATASET
- TRAIN-REFINE-LEARNER

To enable/disable parallelization, set lparallel’s kernel object. For example, to enable parallelization with 4 threads,

#+BEGIN_SRC lisp
;; Enable parallelization
(setf lparallel:*kernel* (lparallel:make-kernel 4))

;; Disable parallelization
(setf lparallel:*kernel* nil)
#+END_SRC

*** Regression
**** Prepare training dataset
In case of classification, the target is a vector of integer values, whereas in regression is a vector of continuous values.

#+BEGIN_SRC lisp
(defparameter *n* 100)

(defparameter *datamatrix*
(let ((arr (make-array (list *n* 1) :element-type ’double-float)))
(loop for i from 0 below *n* do
(setf (aref arr i 0) (random-uniform (- pi) pi)))
arr))

(defparameter *target*
(let ((arr (make-array *n* :element-type ’double-float)))
(loop for i from 0 below *n* do
(setf (aref arr i) (+ (sin (aref *datamatrix* i 0))
(random-normal :sd 0.1d0))))
arr))

(defparameter *test*
(let ((arr (make-array (list *n* 1) :element-type ’double-float)))
(loop for i from 0 below *n*
for x from (- pi) to pi by (/ (* 2 pi) *n*)
do (setf (aref arr i 0) x))
arr))

(defparameter *test-target*
(let ((arr (make-array *n* :element-type ’double-float)))
(loop for i from 0 below *n* do
(setf (aref arr i) (sin (aref *test* i 0))))
arr))
#+END_SRC

**** Make Regression Tree

#+BEGIN_SRC lisp
;; Make regression tree
(defparameter *rtree*
(make-rtree *datamatrix* *target* :max-depth 5 :min-region-samples 5 :n-trial 10))

;; Testing
(test-rtree *rtree* *test* *test-target*)
; RMSE: 0.09220732459820888d0

;; Make a prediction for first data point of test dataset
(predict-rtree *rtree* *test* 0)
; => -0.08374452528780077d0
#+END_SRC

**** Make Random Forest for Regression

#+BEGIN_SRC lisp
;; Make regression tree forest
(defparameter *rforest*
(make-regression-forest *datamatrix* *target*
:n-tree 100 :bagging-ratio 0.6
:max-depth 5 :min-region-samples 5 :n-trial 10))

;; Testing
(test-regression-forest *rforest* *test* *test-target*)
; RMSE: 0.05006872795207973d0

;; Make a prediction for first data point of test dataset
(predict-regression-forest *rforest* *test* 0)
; => -0.16540771296145781d0
#+END_SRC

[[./docs/img/clrf-regression.png]]

** Author
Satoshi Imai (satoshi.imai@gmail.com)

** Licence
This software is released under the MIT License, see LICENSE.txt.

Version

0.1

Dependencies

• cl-libsvm-format
• cl-online-learning
• alexandria
• lparallel

Source

cl-random-forest.asd (file)

Component

src (module)

3 Modules

Modules are listed depth-first from the system components tree.

3.1 cl-random-forest/src

Parent

cl-random-forest (system)

Location

src/

Components

• utils.lisp (file)
• random-forest.lisp (file)
• reconstruction.lisp (file)
• feature-importance.lisp (file)

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.

4.1 Lisp

4.1.1 cl-random-forest.asd

Location

cl-random-forest.asd

Systems

cl-random-forest (system)

Packages

cl-random-forest-asd

4.1.2 cl-random-forest/src/utils.lisp

Parent

src (module)

Location

src/utils.lisp

Packages

cl-random-forest.utils

Exported Definitions

• clol-dataset->datamatrix/target (function)
• clol-dataset->datamatrix/target-regression (function)
• dotimes/pdotimes (macro)
• mapc/pmapc (macro)
• mapcar/pmapcar (macro)
• push-ntimes (macro)
• read-data (function)
• read-data-regression (function)
• write-to-r-format-from-clol-dataset (function)

Internal Definitions

• do-index-value-list (macro)
• format-datum (function)

4.1.3 cl-random-forest/src/random-forest.lisp

Dependency

utils.lisp (file)

Parent

src (module)

Location

src/random-forest.lisp

Packages

cl-random-forest

Exported Definitions

• cross-validation-forest-with-refine-learner (function)
• make-dtree (function)
• make-forest (function)
• make-refine-dataset (function)
• make-refine-learner (function)
• make-refine-vector (function)
• make-regression-forest (function)
• make-rtree (function)
• predict-dtree (function)
• predict-forest (function)
• predict-refine-learner (function)
• predict-regression-forest (function)
• predict-rtree (function)
• pruning! (function)
• test-dtree (function)
• test-forest (function)
• test-refine-learner (function)
• test-regression-forest (function)
• test-rtree (function)
• train-refine-learner (function)
• train-refine-learner-process (macro)

Internal Definitions

• %make-dtree (function)
• %make-forest (function)
• %make-node (function)
• %print-dtree (function)
• %print-forest (function)
• %print-node (function)
• argmax (function)
• balanced-bootstrap-sample-indices (function)
• bootstrap-sample-indices (function)
• calc-accuracy (function)
• children-l2-norm (function)
• class-distribution (function)
• class-distribution-forest (function)
• clean-dtree! (function)
• collect-leaf-parent (function)
• collect-leaf-parent-sorted (function)
• copy-dtree (function)
• copy-forest (function)
• copy-node (function)
• copy-tmp->best! (function)
• delete-children! (function)
• do-leaf (function)
• dtree (structure)
• dtree-best-arr1 (function)
• (setf dtree-best-arr1) (function)
• dtree-best-arr2 (function)
• (setf dtree-best-arr2) (function)
• dtree-best-index1 (function)
• (setf dtree-best-index1) (function)
• dtree-best-index2 (function)
• (setf dtree-best-index2) (function)
• dtree-class-count-array (function)
• (setf dtree-class-count-array) (function)
• dtree-datamatrix (function)
• (setf dtree-datamatrix) (function)
• dtree-datum-dim (function)
• (setf dtree-datum-dim) (function)
• dtree-gain-test (function)
• (setf dtree-gain-test) (function)
• dtree-id (function)
• (setf dtree-id) (function)
• dtree-max-depth (function)
• (setf dtree-max-depth) (function)
• dtree-max-leaf-index (function)
• (setf dtree-max-leaf-index) (function)
• dtree-min-region-samples (function)
• (setf dtree-min-region-samples) (function)
• dtree-n-class (function)
• (setf dtree-n-class) (function)
• dtree-n-trial (function)
• (setf dtree-n-trial) (function)
• dtree-p (function)
• dtree-remove-sample-indices? (function)
• (setf dtree-remove-sample-indices?) (function)
• dtree-root (function)
• (setf dtree-root) (function)
• dtree-save-parent-node? (function)
• (setf dtree-save-parent-node?) (function)
• dtree-target (function)
• (setf dtree-target) (function)
• dtree-tmp-arr1 (function)
• (setf dtree-tmp-arr1) (function)
• dtree-tmp-arr2 (function)
• (setf dtree-tmp-arr2) (function)
• dtree-tmp-index1 (function)
• (setf dtree-tmp-index1) (function)
• dtree-tmp-index2 (function)
• (setf dtree-tmp-index2) (function)
• entropy (function)
• find-leaf (function)
• forest (structure)
• forest-bagging-ratio (function)
• (setf forest-bagging-ratio) (function)
• forest-class-count-array (function)
• (setf forest-class-count-array) (function)
• forest-datamatrix (function)
• (setf forest-datamatrix) (function)
• forest-datum-dim (function)
• (setf forest-datum-dim) (function)
• forest-dtree-list (function)
• (setf forest-dtree-list) (function)
• forest-gain-test (function)
• (setf forest-gain-test) (function)
• forest-index-offset (function)
• (setf forest-index-offset) (function)
• forest-max-depth (function)
• (setf forest-max-depth) (function)
• forest-min-region-samples (function)
• (setf forest-min-region-samples) (function)
• forest-n-class (function)
• (setf forest-n-class) (function)
• forest-n-leaf (function)
• (setf forest-n-leaf) (function)
• forest-n-tree (function)
• (setf forest-n-tree) (function)
• forest-n-trial (function)
• (setf forest-n-trial) (function)
• forest-p (function)
• forest-target (function)
• (setf forest-target) (function)
• gini (function)
• leaf? (function)
• make-l2-norm (function)
• make-l2-norm-binary (function)
• make-l2-norm-multiclass (function)
• make-node (function)
• make-partial-arr (function)
• make-random-test (function)
• make-regression-refine-dataset (function)
• make-regression-refine-learner (function)
• make-regression-refine-vector (function)
• make-root-node (function)
• maximize-activation/count (function)
• node (structure)
• node-class-distribution (function)
• node-depth (function)
• (setf node-depth) (function)
• node-dtree (function)
• (setf node-dtree) (function)
• node-information-gain (function)
• (setf node-information-gain) (function)
• node-leaf-index (function)
• (setf node-leaf-index) (function)
• node-left-node (function)
• (setf node-left-node) (function)
• node-n-sample (function)
• (setf node-n-sample) (function)
• node-p (function)
• node-parent-node (function)
• (setf node-parent-node) (function)
• node-regression-mean (function)
• node-right-node (function)
• (setf node-right-node) (function)
• node-sample-indices (function)
• (setf node-sample-indices) (function)
• node-test-attribute (function)
• (setf node-test-attribute) (function)
• node-test-threshold (function)
• (setf node-test-threshold) (function)
• predict-regression-refine-learner (function)
• random-uniform (function)
• region-min/max (function)
• rtree? (function)
• set-activation-matrix! (function)
• set-best-children! (function)
• set-leaf-index! (function)
• set-leaf-index-forest! (function)
• split-node! (function)
• split-sample-indices (function)
• square (function)
• stop-split? (function)
• test-refine-learner-binary (function)
• test-refine-learner-multiclass (function)
• test-regression-refine-learner (function)
• train-refine-learner-binary (function)
• train-refine-learner-multiclass (function)
• train-refine-learner-process-inner (function)
• train-regression-refine-learner (function)
• traverse (function)
• variance (function)

4.1.4 cl-random-forest/src/reconstruction.lisp

Dependency

random-forest.lisp (file)

Parent

src (module)

Location

src/reconstruction.lisp

Exported Definitions

• decode-datum (function)
• encode-datum (function)
• make-leaf-node-vector (function)
• reconstruction-forest (function)

Internal Definitions

• reconstruction-backward (function)
• reconstruction-dtree (function)

4.1.5 cl-random-forest/src/feature-importance.lisp

Dependency

random-forest.lisp (file)

Parent

src (module)

Location

src/feature-importance.lisp

Exported Definitions

forest-feature-importance (function)

Internal Definitions

• dtree-feature-importance (function)
• dtree-feature-importance-impurity (function)
• dtree-oob-sample-indices (function)
• find-leaf-randomized (function)
• forest-feature-importance-impurity (function)
• make-oob-sample-indices (function)
• normalize-arr! (function)
• predict-dtree-randomized (function)
• predict-rtree-randomized (function)
• rtree-feature-importance (function)
• test-dtree-oob (function)
• test-dtree-oob-randomized (function)
• test-rtree-oob (function)
• test-rtree-oob-randomized (function)

5 Packages

Packages are listed by definition order.

5.1 cl-random-forest-asd

Source

cl-random-forest.asd

Use List

• asdf/interface
• common-lisp

5.2 cl-random-forest.utils

Source

utils.lisp (file)

Nickname

clrf.utils

Use List

common-lisp

Used By List

cl-random-forest

Exported Definitions

• clol-dataset->datamatrix/target (function)
• clol-dataset->datamatrix/target-regression (function)
• dotimes/pdotimes (macro)
• mapc/pmapc (macro)
• mapcar/pmapcar (macro)
• push-ntimes (macro)
• read-data (function)
• read-data-regression (function)
• write-to-r-format-from-clol-dataset (function)

Internal Definitions

• do-index-value-list (macro)
• format-datum (function)

5.3 cl-random-forest

Source

random-forest.lisp (file)

Nickname

clrf

Use List

• cl-random-forest.utils
• common-lisp

Exported Definitions

• cross-validation-forest-with-refine-learner (function)
• decode-datum (function)
• encode-datum (function)
• forest-feature-importance (function)
• make-dtree (function)
• make-forest (function)
• make-leaf-node-vector (function)
• make-refine-dataset (function)
• make-refine-learner (function)
• make-refine-vector (function)
• make-regression-forest (function)
• make-rtree (function)
• predict-dtree (function)
• predict-forest (function)
• predict-refine-learner (function)
• predict-regression-forest (function)
• predict-rtree (function)
• pruning! (function)
• reconstruction-forest (function)
• test-dtree (function)
• test-forest (function)
• test-refine-learner (function)
• test-regression-forest (function)
• test-rtree (function)
• train-refine-learner (function)
• train-refine-learner-process (macro)

Internal Definitions

• %make-dtree (function)
• %make-forest (function)
• %make-node (function)
• %print-dtree (function)
• %print-forest (function)
• %print-node (function)
• argmax (function)
• balanced-bootstrap-sample-indices (function)
• bootstrap-sample-indices (function)
• calc-accuracy (function)
• children-l2-norm (function)
• class-distribution (function)
• class-distribution-forest (function)
• clean-dtree! (function)
• collect-leaf-parent (function)
• collect-leaf-parent-sorted (function)
• copy-dtree (function)
• copy-forest (function)
• copy-node (function)
• copy-tmp->best! (function)
• delete-children! (function)
• do-leaf (function)
• dtree (structure)
• dtree-best-arr1 (function)
• (setf dtree-best-arr1) (function)
• dtree-best-arr2 (function)
• (setf dtree-best-arr2) (function)
• dtree-best-index1 (function)
• (setf dtree-best-index1) (function)
• dtree-best-index2 (function)
• (setf dtree-best-index2) (function)
• dtree-class-count-array (function)
• (setf dtree-class-count-array) (function)
• dtree-datamatrix (function)
• (setf dtree-datamatrix) (function)
• dtree-datum-dim (function)
• (setf dtree-datum-dim) (function)
• dtree-feature-importance (function)
• dtree-feature-importance-impurity (function)
• dtree-gain-test (function)
• (setf dtree-gain-test) (function)
• dtree-id (function)
• (setf dtree-id) (function)
• dtree-max-depth (function)
• (setf dtree-max-depth) (function)
• dtree-max-leaf-index (function)
• (setf dtree-max-leaf-index) (function)
• dtree-min-region-samples (function)
• (setf dtree-min-region-samples) (function)
• dtree-n-class (function)
• (setf dtree-n-class) (function)
• dtree-n-trial (function)
• (setf dtree-n-trial) (function)
• dtree-oob-sample-indices (function)
• dtree-p (function)
• dtree-remove-sample-indices? (function)
• (setf dtree-remove-sample-indices?) (function)
• dtree-root (function)
• (setf dtree-root) (function)
• dtree-save-parent-node? (function)
• (setf dtree-save-parent-node?) (function)
• dtree-target (function)
• (setf dtree-target) (function)
• dtree-tmp-arr1 (function)
• (setf dtree-tmp-arr1) (function)
• dtree-tmp-arr2 (function)
• (setf dtree-tmp-arr2) (function)
• dtree-tmp-index1 (function)
• (setf dtree-tmp-index1) (function)
• dtree-tmp-index2 (function)
• (setf dtree-tmp-index2) (function)
• entropy (function)
• find-leaf (function)
• find-leaf-randomized (function)
• forest (structure)
• forest-bagging-ratio (function)
• (setf forest-bagging-ratio) (function)
• forest-class-count-array (function)
• (setf forest-class-count-array) (function)
• forest-datamatrix (function)
• (setf forest-datamatrix) (function)
• forest-datum-dim (function)
• (setf forest-datum-dim) (function)
• forest-dtree-list (function)
• (setf forest-dtree-list) (function)
• forest-feature-importance-impurity (function)
• forest-gain-test (function)
• (setf forest-gain-test) (function)
• forest-index-offset (function)
• (setf forest-index-offset) (function)
• forest-max-depth (function)
• (setf forest-max-depth) (function)
• forest-min-region-samples (function)
• (setf forest-min-region-samples) (function)
• forest-n-class (function)
• (setf forest-n-class) (function)
• forest-n-leaf (function)
• (setf forest-n-leaf) (function)
• forest-n-tree (function)
• (setf forest-n-tree) (function)
• forest-n-trial (function)
• (setf forest-n-trial) (function)
• forest-p (function)
• forest-target (function)
• (setf forest-target) (function)
• gini (function)
• leaf? (function)
• make-l2-norm (function)
• make-l2-norm-binary (function)
• make-l2-norm-multiclass (function)
• make-node (function)
• make-oob-sample-indices (function)
• make-partial-arr (function)
• make-random-test (function)
• make-regression-refine-dataset (function)
• make-regression-refine-learner (function)
• make-regression-refine-vector (function)
• make-root-node (function)
• maximize-activation/count (function)
• node (structure)
• node-class-distribution (function)
• node-depth (function)
• (setf node-depth) (function)
• node-dtree (function)
• (setf node-dtree) (function)
• node-information-gain (function)
• (setf node-information-gain) (function)
• node-leaf-index (function)
• (setf node-leaf-index) (function)
• node-left-node (function)
• (setf node-left-node) (function)
• node-n-sample (function)
• (setf node-n-sample) (function)
• node-p (function)
• node-parent-node (function)
• (setf node-parent-node) (function)
• node-regression-mean (function)
• node-right-node (function)
• (setf node-right-node) (function)
• node-sample-indices (function)
• (setf node-sample-indices) (function)
• node-test-attribute (function)
• (setf node-test-attribute) (function)
• node-test-threshold (function)
• (setf node-test-threshold) (function)
• normalize-arr! (function)
• predict-dtree-randomized (function)
• predict-regression-refine-learner (function)
• predict-rtree-randomized (function)
• random-uniform (function)
• reconstruction-backward (function)
• reconstruction-dtree (function)
• region-min/max (function)
• rtree-feature-importance (function)
• rtree? (function)
• set-activation-matrix! (function)
• set-best-children! (function)
• set-leaf-index! (function)
• set-leaf-index-forest! (function)
• split-node! (function)
• split-sample-indices (function)
• square (function)
• stop-split? (function)
• test-dtree-oob (function)
• test-dtree-oob-randomized (function)
• test-refine-learner-binary (function)
• test-refine-learner-multiclass (function)
• test-regression-refine-learner (function)
• test-rtree-oob (function)
• test-rtree-oob-randomized (function)
• train-refine-learner-binary (function)
• train-refine-learner-multiclass (function)
• train-refine-learner-process-inner (function)
• train-regression-refine-learner (function)
• traverse (function)
• variance (function)

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

6.1 Exported definitions

6.1.1 Macros

Macro: dotimes/pdotimes (VAR N) &body BODY

Package

cl-random-forest.utils

Source

utils.lisp (file)

Macro: mapc/pmapc FN &rest LSTS

Package

cl-random-forest.utils

Source

utils.lisp (file)

Macro: mapcar/pmapcar FN &rest LSTS

Package

cl-random-forest.utils

Source

utils.lisp (file)

Macro: push-ntimes N LST &body BODY

Package

cl-random-forest.utils

Source

utils.lisp (file)

Macro: train-refine-learner-process REFINE-LEARNER TRAIN-DATASET TRAIN-TARGET TEST-DATASET TEST-TARGET &key MAX-EPOCH

Package

cl-random-forest

Source

random-forest.lisp (file)

6.1.2 Functions

Function: clol-dataset->datamatrix/target DATASET

Package

cl-random-forest.utils

Source

utils.lisp (file)

Function: clol-dataset->datamatrix/target-regression DATASET

Package

cl-random-forest.utils

Source

utils.lisp (file)

Function: cross-validation-forest-with-refine-learner N-FOLD N-CLASS DATAMATRIX TARGET &key N-TREE BAGGING-RATIO MAX-DEPTH MIN-REGION-SAMPLES N-TRIAL GAIN-TEST REMOVE-SAMPLE-INDICES? GAMMA

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: decode-datum FOREST LEAF-INDEX-VECTOR &optional LEAF-NODE-VECTOR

Package

cl-random-forest

Source

reconstruction.lisp (file)

Function: encode-datum FOREST DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

reconstruction.lisp (file)

Function: forest-feature-importance FOREST DATAMATRIX TARGET

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: make-dtree N-CLASS DATAMATRIX TARGET &key MAX-DEPTH MIN-REGION-SAMPLES N-TRIAL GAIN-TEST REMOVE-SAMPLE-INDICES? SAVE-PARENT-NODE? SAMPLE-INDICES

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-forest N-CLASS DATAMATRIX TARGET &key N-TREE BAGGING-RATIO MAX-DEPTH MIN-REGION-SAMPLES N-TRIAL GAIN-TEST REMOVE-SAMPLE-INDICES? SAVE-PARENT-NODE? BALANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-leaf-node-vector FOREST

Package

cl-random-forest

Source

reconstruction.lisp (file)

Function: make-refine-dataset FOREST DATAMATRIX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-refine-learner FOREST &optional GAMMA

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-refine-vector FOREST DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-regression-forest DATAMATRIX TARGET &key N-TREE BAGGING-RATIO MAX-DEPTH MIN-REGION-SAMPLES N-TRIAL GAIN-TEST REMOVE-SAMPLE-INDICES? SAVE-PARENT-NODE?

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-rtree DATAMATRIX TARGET &key MAX-DEPTH MIN-REGION-SAMPLES N-TRIAL GAIN-TEST REMOVE-SAMPLE-INDICES? SAVE-PARENT-NODE? SAMPLE-INDICES

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: predict-dtree DTREE DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: predict-forest FOREST DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: predict-refine-learner FOREST REFINE-LEARNER DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: predict-regression-forest FOREST DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: predict-rtree RTREE DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: pruning! FOREST LEARNER &optional PRUNING-RATE MIN-DEPTH

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: read-data DATA-PATH DATA-DIMENSION

Package

cl-random-forest.utils

Source

utils.lisp (file)

Function: read-data-regression DATA-PATH DATA-DIMENSION

Package

cl-random-forest.utils

Source

utils.lisp (file)

Function: reconstruction-forest FOREST DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

reconstruction.lisp (file)

Function: test-dtree DTREE DATAMATRIX TARGET &key QUIET-P

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-forest FOREST DATAMATRIX TARGET &key QUIET-P

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-refine-learner REFINE-LEARNER REFINE-DATASET TARGET &key QUIET-P MINI-BATCH-SIZE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-regression-forest FOREST DATAMATRIX TARGET &key QUIET-P

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-rtree RTREE DATAMATRIX TARGET &key QUIET-P

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: train-refine-learner REFINE-LEARNER REFINE-DATASET TARGET

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: write-to-r-format-from-clol-dataset DATASET FILE

Package

cl-random-forest.utils

Source

utils.lisp (file)

6.2 Internal definitions

6.2.1 Macros

Macro: do-index-value-list (INDEX VALUE LIST) &body BODY

Package

cl-random-forest.utils

Source

utils.lisp (file)

6.2.2 Functions

Function: %make-dtree &key (N-CLASS N-CLASS) (CLASS-COUNT-ARRAY CLASS-COUNT-ARRAY) (DATUM-DIM DATUM-DIM) (DATAMATRIX DATAMATRIX) (TARGET TARGET) (ROOT ROOT) (MAX-DEPTH MAX-DEPTH) (MIN-REGION-SAMPLES MIN-REGION-SAMPLES) (N-TRIAL N-TRIAL) (GAIN-TEST GAIN-TEST) (REMOVE-SAMPLE-INDICES? REMOVE-SAMPLE-INDICES?) (SAVE-PARENT-NODE? SAVE-PARENT-NODE?) (TMP-ARR1 TMP-ARR1) (TMP-INDEX1 TMP-INDEX1) (TMP-ARR2 TMP-ARR2) (TMP-INDEX2 TMP-INDEX2) (BEST-ARR1 BEST-ARR1) (BEST-INDEX1 BEST-INDEX1) (BEST-ARR2 BEST-ARR2) (BEST-INDEX2 BEST-INDEX2) (MAX-LEAF-INDEX MAX-LEAF-INDEX) (ID ID)

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: %make-forest &key (N-TREE N-TREE) (BAGGING-RATIO BAGGING-RATIO) (DATAMATRIX DATAMATRIX) (TARGET TARGET) (DTREE-LIST DTREE-LIST) (N-CLASS N-CLASS) (CLASS-COUNT-ARRAY CLASS-COUNT-ARRAY) (DATUM-DIM DATUM-DIM) (MAX-DEPTH MAX-DEPTH) (MIN-REGION-SAMPLES MIN-REGION-SAMPLES) (N-TRIAL N-TRIAL) (GAIN-TEST GAIN-TEST) (N-LEAF N-LEAF) (INDEX-OFFSET INDEX-OFFSET)

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: %make-node &key (SAMPLE-INDICES SAMPLE-INDICES) (N-SAMPLE N-SAMPLE) (DEPTH DEPTH) (TEST-ATTRIBUTE TEST-ATTRIBUTE) (TEST-THRESHOLD TEST-THRESHOLD) (INFORMATION-GAIN INFORMATION-GAIN) (PARENT-NODE PARENT-NODE) (LEFT-NODE LEFT-NODE) (RIGHT-NODE RIGHT-NODE) (DTREE DTREE) (LEAF-INDEX LEAF-INDEX)

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: %print-dtree OBJ STREAM

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: %print-forest OBJ STREAM

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: %print-node OBJ STREAM

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: argmax ARR

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: balanced-bootstrap-sample-indices N N-CLASS TARGET

Bagging classifiers induced over balanced bootstrap samples. Also known as Balanced Random Forest.

This implementation counts the number of examples in each class, which might be inefficient.

> In this work we provide such an explanation, and we conclude that
> in almost all imbalanced scenarios, practitioners should bag classifiers induced over
> balanced bootstrap samples.

Wallace, Byron C., et al. “Class imbalance, redux.”
2011 IEEE 11th International Conference on Data Mining (ICDM), 2011.

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: bootstrap-sample-indices N DATAMATRIX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: calc-accuracy N-CORRECT LEN &key QUIET-P

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: children-l2-norm NODE L2-NORM-ARR FOREST

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: class-distribution SAMPLE-INDICES TERMINATE-INDEX DTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: class-distribution-forest FOREST DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: clean-dtree! DTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: collect-leaf-parent FOREST

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: collect-leaf-parent-sorted FOREST LEARNER

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: copy-dtree INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: copy-forest INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: copy-node INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: copy-tmp->best! DTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: delete-children! NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: do-leaf FN NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-best-arr1 INSTANCE

Function: (setf dtree-best-arr1) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-best-arr2 INSTANCE

Function: (setf dtree-best-arr2) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-best-index1 INSTANCE

Function: (setf dtree-best-index1) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-best-index2 INSTANCE

Function: (setf dtree-best-index2) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-class-count-array INSTANCE

Function: (setf dtree-class-count-array) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-datamatrix INSTANCE

Function: (setf dtree-datamatrix) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-datum-dim INSTANCE

Function: (setf dtree-datum-dim) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-feature-importance DTREE DATAMATRIX TARGET

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: dtree-feature-importance-impurity DTREE

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: dtree-gain-test INSTANCE

Function: (setf dtree-gain-test) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-id INSTANCE

Function: (setf dtree-id) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-max-depth INSTANCE

Function: (setf dtree-max-depth) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-max-leaf-index INSTANCE

Function: (setf dtree-max-leaf-index) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-min-region-samples INSTANCE

Function: (setf dtree-min-region-samples) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-n-class INSTANCE

Function: (setf dtree-n-class) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-n-trial INSTANCE

Function: (setf dtree-n-trial) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-oob-sample-indices DTREE

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: dtree-p OBJECT

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-remove-sample-indices? INSTANCE

Function: (setf dtree-remove-sample-indices?) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-root INSTANCE

Function: (setf dtree-root) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-save-parent-node? INSTANCE

Function: (setf dtree-save-parent-node?) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-target INSTANCE

Function: (setf dtree-target) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-tmp-arr1 INSTANCE

Function: (setf dtree-tmp-arr1) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-tmp-arr2 INSTANCE

Function: (setf dtree-tmp-arr2) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-tmp-index1 INSTANCE

Function: (setf dtree-tmp-index1) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: dtree-tmp-index2 INSTANCE

Function: (setf dtree-tmp-index2) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: entropy SAMPLE-INDICES TERMINATE-INDEX DTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: find-leaf NODE DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: find-leaf-randomized NODE DATAMATRIX DATUM-INDEX RANDOMIZED-ATTRIBUTE OOB-SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: forest-bagging-ratio INSTANCE

Function: (setf forest-bagging-ratio) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-class-count-array INSTANCE

Function: (setf forest-class-count-array) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-datamatrix INSTANCE

Function: (setf forest-datamatrix) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-datum-dim INSTANCE

Function: (setf forest-datum-dim) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-dtree-list INSTANCE

Function: (setf forest-dtree-list) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-feature-importance-impurity FOREST

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: forest-gain-test INSTANCE

Function: (setf forest-gain-test) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-index-offset INSTANCE

Function: (setf forest-index-offset) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-max-depth INSTANCE

Function: (setf forest-max-depth) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-min-region-samples INSTANCE

Function: (setf forest-min-region-samples) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-n-class INSTANCE

Function: (setf forest-n-class) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-n-leaf INSTANCE

Function: (setf forest-n-leaf) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-n-tree INSTANCE

Function: (setf forest-n-tree) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-n-trial INSTANCE

Function: (setf forest-n-trial) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-p OBJECT

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: forest-target INSTANCE

Function: (setf forest-target) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: format-datum DATUM STREAM

Package

cl-random-forest.utils

Source

utils.lisp (file)

Function: gini SAMPLE-INDICES TERMINATE-INDEX DTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: leaf? NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-l2-norm LEARNER

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-l2-norm-binary LEARNER

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-l2-norm-multiclass LEARNER

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-node SAMPLE-INDICES PARENT-NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-oob-sample-indices TOTAL-SIZE SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: make-partial-arr ARR LEN

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-random-test NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-regression-refine-dataset FOREST DATAMATRIX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-regression-refine-learner FOREST &optional GAMMA

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-regression-refine-vector FOREST DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: make-root-node DTREE &key SAMPLE-INDICES

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: maximize-activation/count ACTIVATION-MATRIX END-OF-MINI-BATCH TARGET CYCLE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-class-distribution NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-depth INSTANCE

Function: (setf node-depth) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-dtree INSTANCE

Function: (setf node-dtree) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-information-gain INSTANCE

Function: (setf node-information-gain) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-leaf-index INSTANCE

Function: (setf node-leaf-index) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-left-node INSTANCE

Function: (setf node-left-node) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-n-sample INSTANCE

Function: (setf node-n-sample) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-p OBJECT

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-parent-node INSTANCE

Function: (setf node-parent-node) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-regression-mean NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-right-node INSTANCE

Function: (setf node-right-node) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-sample-indices INSTANCE

Function: (setf node-sample-indices) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-test-attribute INSTANCE

Function: (setf node-test-attribute) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: node-test-threshold INSTANCE

Function: (setf node-test-threshold) VALUE INSTANCE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: normalize-arr! ARR

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: predict-dtree-randomized DTREE DATAMATRIX DATUM-INDEX RANDOMIZED-ATTRIBUTE OOB-SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: predict-regression-refine-learner FOREST REFINE-LEARNER DATAMATRIX DATUM-INDEX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: predict-rtree-randomized RTREE DATAMATRIX DATUM-INDEX RANDOMIZED-ATTRIBUTE OOB-SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: random-uniform START END

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: reconstruction-backward NODE INPUT-RANGE-ARRAY

Package

cl-random-forest

Source

reconstruction.lisp (file)

Function: reconstruction-dtree LEAF-NODE INPUT-RANGE-ARRAY

Package

cl-random-forest

Source

reconstruction.lisp (file)

Function: region-min/max SAMPLE-INDICES DATAMATRIX ATTRIBUTE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: rtree-feature-importance RTREE DATAMATRIX TARGET

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: rtree? DTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: set-activation-matrix! ACTIVATION-MATRIX REFINE-LEARNER N-CLASS SV-VEC REFINE-DATASET CYCLE END-OF-MINI-BATCH

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: set-best-children! NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: set-leaf-index! DTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: set-leaf-index-forest! FOREST

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: split-node! NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: split-sample-indices SAMPLE-INDICES TRUE-ARRAY FALSE-ARRAY ATTRIBUTE THRESHOLD DATAMATRIX

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: square X

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: stop-split? NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-dtree-oob DTREE DATAMATRIX TARGET &key QUIET-P OOB-SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: test-dtree-oob-randomized DTREE DATAMATRIX TARGET RANDOMIZED-ATTRIBUTE &key QUIET-P OOB-SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: test-refine-learner-binary REFINE-LEARNER REFINE-DATASET TARGET &key QUIET-P

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-refine-learner-multiclass REFINE-LEARNER REFINE-DATASET TARGET &key QUIET-P MINI-BATCH-SIZE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-regression-refine-learner REFINE-LEARNER REFINE-DATASET TARGET &key QUIET-P

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: test-rtree-oob RTREE DATAMATRIX TARGET &key QUIET-P OOB-SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: test-rtree-oob-randomized RTREE DATAMATRIX TARGET RANDOMIZED-ATTRIBUTE &key QUIET-P OOB-SAMPLE-INDICES

Package

cl-random-forest

Source

feature-importance.lisp (file)

Function: train-refine-learner-binary REFINE-LEARNER REFINE-DATASET TARGET

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: train-refine-learner-multiclass REFINE-LEARNER REFINE-DATASET TARGET

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: train-refine-learner-process-inner REFINE-LEARNER TRAIN-DATASET TRAIN-TARGET TEST-DATASET TEST-TARGET &key MAX-EPOCH

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: train-regression-refine-learner REFINE-LEARNER REFINE-DATASET TARGET

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: traverse FN NODE

Package

cl-random-forest

Source

random-forest.lisp (file)

Function: variance SAMPLE-INDICES TERMINATE-INDEX RTREE

Package

cl-random-forest

Source

random-forest.lisp (file)

6.2.3 Structures

Structure: dtree ()

Package

cl-random-forest

Source

random-forest.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

print-object (method)

Direct slots

Slot: n-class

Readers

dtree-n-class (function)

Writers

(setf dtree-n-class) (function)

Slot: class-count-array

Readers

dtree-class-count-array (function)

Writers

(setf dtree-class-count-array) (function)

Slot: datum-dim

Readers

dtree-datum-dim (function)

Writers

(setf dtree-datum-dim) (function)

Slot: datamatrix

Readers

dtree-datamatrix (function)

Writers

(setf dtree-datamatrix) (function)

Slot: target

Readers

dtree-target (function)

Writers

(setf dtree-target) (function)

Slot: root

Readers

dtree-root (function)

Writers

(setf dtree-root) (function)

Slot: max-depth

Readers

dtree-max-depth (function)

Writers

(setf dtree-max-depth) (function)

Slot: min-region-samples

Readers

dtree-min-region-samples (function)

Writers

(setf dtree-min-region-samples) (function)

Slot: n-trial

Readers

dtree-n-trial (function)

Writers

(setf dtree-n-trial) (function)

Slot: gain-test

Readers

dtree-gain-test (function)

Writers

(setf dtree-gain-test) (function)

Slot: remove-sample-indices?

Readers

dtree-remove-sample-indices? (function)

Writers

(setf dtree-remove-sample-indices?) (function)

Slot: save-parent-node?

Readers

dtree-save-parent-node? (function)

Writers

(setf dtree-save-parent-node?) (function)

Slot: tmp-arr1

Readers

dtree-tmp-arr1 (function)

Writers

(setf dtree-tmp-arr1) (function)

Slot: tmp-index1

Readers

dtree-tmp-index1 (function)

Writers

(setf dtree-tmp-index1) (function)

Slot: tmp-arr2

Readers

dtree-tmp-arr2 (function)

Writers

(setf dtree-tmp-arr2) (function)

Slot: tmp-index2

Readers

dtree-tmp-index2 (function)

Writers

(setf dtree-tmp-index2) (function)

Slot: best-arr1

Readers

dtree-best-arr1 (function)

Writers

(setf dtree-best-arr1) (function)

Slot: best-index1

Readers

dtree-best-index1 (function)

Writers

(setf dtree-best-index1) (function)

Slot: best-arr2

Readers

dtree-best-arr2 (function)

Writers

(setf dtree-best-arr2) (function)

Slot: best-index2

Readers

dtree-best-index2 (function)

Writers

(setf dtree-best-index2) (function)

Slot: max-leaf-index

Readers

dtree-max-leaf-index (function)

Writers

(setf dtree-max-leaf-index) (function)

Slot: id

Readers

dtree-id (function)

Writers

(setf dtree-id) (function)

Structure: forest ()

Package

cl-random-forest

Source

random-forest.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

print-object (method)

Direct slots

Slot: n-tree

Readers

forest-n-tree (function)

Writers

(setf forest-n-tree) (function)

Slot: bagging-ratio

Readers

forest-bagging-ratio (function)

Writers

(setf forest-bagging-ratio) (function)

Slot: datamatrix

Readers

forest-datamatrix (function)

Writers

(setf forest-datamatrix) (function)

Slot: target

Readers

forest-target (function)

Writers

(setf forest-target) (function)

Slot: dtree-list

Readers

forest-dtree-list (function)

Writers

(setf forest-dtree-list) (function)

Slot: n-class

Readers

forest-n-class (function)

Writers

(setf forest-n-class) (function)

Slot: class-count-array

Readers

forest-class-count-array (function)

Writers

(setf forest-class-count-array) (function)

Slot: datum-dim

Readers

forest-datum-dim (function)

Writers

(setf forest-datum-dim) (function)

Slot: max-depth

Readers

forest-max-depth (function)

Writers

(setf forest-max-depth) (function)

Slot: min-region-samples

Readers

forest-min-region-samples (function)

Writers

(setf forest-min-region-samples) (function)

Slot: n-trial

Readers

forest-n-trial (function)

Writers

(setf forest-n-trial) (function)

Slot: gain-test

Readers

forest-gain-test (function)

Writers

(setf forest-gain-test) (function)

Slot: n-leaf

Readers

forest-n-leaf (function)

Writers

(setf forest-n-leaf) (function)

Slot: index-offset

Readers

forest-index-offset (function)

Writers

(setf forest-index-offset) (function)

Structure: node ()

Package

cl-random-forest

Source

random-forest.lisp (file)

Direct superclasses

structure-object (structure)

Direct methods

print-object (method)

Direct slots

Slot: sample-indices

Readers

node-sample-indices (function)

Writers

(setf node-sample-indices) (function)

Slot: n-sample

Readers

node-n-sample (function)

Writers

(setf node-n-sample) (function)

Slot: depth

Readers

node-depth (function)

Writers

(setf node-depth) (function)

Slot: test-attribute

Readers

node-test-attribute (function)

Writers

(setf node-test-attribute) (function)

Slot: test-threshold

Readers

node-test-threshold (function)

Writers

(setf node-test-threshold) (function)

Slot: information-gain

Readers

node-information-gain (function)

Writers

(setf node-information-gain) (function)

Slot: parent-node

Readers

node-parent-node (function)

Writers

(setf node-parent-node) (function)

Slot: left-node

Readers

node-left-node (function)

Writers

(setf node-left-node) (function)

Slot: right-node

Readers

node-right-node (function)

Writers

(setf node-right-node) (function)

Slot: dtree

Readers

node-dtree (function)

Writers

(setf node-dtree) (function)

Slot: leaf-index

Readers

node-leaf-index (function)

Writers

(setf node-leaf-index) (function)

Appendix A Indexes

A.1 Concepts

A.2 Functions