The teddy Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The teddy Reference Manual

This is the teddy Reference Manual, generated automatically by Declt version 3.0 "Montgomery Scott" on Wed Nov 04 15:24:31 2020 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

I want you to meet Teddy. Teddy wanna be like pandas. Pandas are cool. Teddy want be cool too!

Reasoning

This library provides some Common Lisp facitilies to work with data frames.

Common Lisp already has numcl to operate on arrays, and now we need a more abstract tool to work with data like data sheets.

Teddy make it possible to define a dataframe full of data, to slice it in different ways, to join data frames and see some statistics about the data.

This is a proof of the concept and API will be changed. Check the ChangeLog.md to learn about new abilities and refactoring details.

How to create a data-frame

Here is how we can create a simple data-frame:

POFTHEDAY> (teddy/data-frame:make-data-frame
            '("Idx" "Integers" "Uniform floats" "Gaussian")
            :rows
            (loop repeat 10
                  for idx upfrom 0
                  collect (list idx
                                (random 100)
                                (random 1.0)
                                (statistics:random-normal
                                 :mean 5.0
                                 :sd 0.2))))
+-----+----------+----------------+----------+
| Idx | Integers | Uniform floats | Gaussian |
+-----+----------+----------------+----------+
|   0 |       41 |           0.27 |   4.89d0 |
|   1 |       98 |           0.08 |   4.93d0 |
|   2 |        8 |           0.45 |   5.15d0 |
|   3 |       56 |           0.63 |   4.87d0 |
|   4 |       79 |           0.42 |   4.72d0 |
|   5 |       19 |           0.04 |   4.73d0 |
|   6 |        1 |           0.34 |   4.93d0 |
|   7 |       79 |           0.60 |   5.25d0 |
|   8 |       42 |           0.08 |   5.10d0 |
|   9 |        7 |           0.86 |   5.31d0 |
+-----+----------+----------------+----------+

Data manipulation

Now we can slice it by columns, rows or both:

POFTHEDAY> (teddy/data-frame:head *d* 2)
+-----+----------+----------------+----------+
| Idx | Integers | Uniform floats | Gaussian |
+-----+----------+----------------+----------+
|   0 |       41 |           0.27 |   4.89d0 |
|   1 |       98 |           0.08 |   4.93d0 |
+-----+----------+----------------+----------+
POFTHEDAY> (teddy/data-frame:tail *d* 2)
+-----+----------+----------------+----------+
| Idx | Integers | Uniform floats | Gaussian |
+-----+----------+----------------+----------+
|   8 |       42 |           0.08 |   5.10d0 |
|   9 |        7 |           0.86 |   5.31d0 |
+-----+----------+----------------+----------+
POFTHEDAY> (teddy/data-frame:slice
            *d*
            :columns '("idx" "gaussian"))
+-----+----------+
| Idx | Gaussian |
+-----+----------+
|   0 |   4.89d0 |
|   1 |   4.93d0 |
|   2 |   5.15d0 |
|   3 |   4.87d0 |
|   4 |   4.72d0 |
|   5 |   4.73d0 |
|   6 |   4.93d0 |
|   7 |   5.25d0 |
|   8 |   5.10d0 |
|   9 |   5.31d0 |
+-----+----------+
POFTHEDAY> (teddy/data-frame:slice *d*
            :columns '("idx" "gaussian")
            :from 4
            :to 6)
+-----+----------+
| Idx | Gaussian |
+-----+----------+
|   4 |   4.72d0 |
|   5 |   4.73d0 |
+-----+----------+

Analyzing data

Also, we might want to see some descriptive statistical data about our data frame. This is pretty easy with Teddy:

POFTHEDAY> (teddy/stats:stats *d*)
+----------------+--------+--------+--------+--------+--------+-------+-------+---------+
| Column         | Min    | p25    | p50    | p75    | Max    | Mean  | SD    | Sum     |
+----------------+--------+--------+--------+--------+--------+-------+-------+---------+
| Idx            |      0 |      2 |   4.50 |      7 |      9 |  4.50 |  3.03 |      45 |
| Integers       |      1 |      8 |  41.50 |     79 |     98 | 43.00 | 34.40 |     430 |
| Uniform floats |   0.04 |   0.08 |   0.38 |   0.60 |   0.86 |  0.38 |  0.27 |    3.75 |
| Gaussian       | 4.72d0 | 4.87d0 | 4.93d0 | 5.15d0 | 5.31d0 |  4.99 |  0.20 | 49.88d0 |
+----------------+--------+--------+--------+--------+--------+-------+-------+---------+

Probably, we can make some extandable protocol to calculate other properties.

Iteration over rows

Data frame stores data as columns. Each column is a vector of particular type. If you want to process a row, you can create an iterator and use it to go through rows like that:

POFTHEDAY> (loop with iterator = (teddy/data-frame:make-iterator *d*)
                 for row = (funcall iterator)
                 while row
                 do (format t "Row: ~S~%"
                            row))
Row: (0 41 0.26806116 4.887522971759381d0)
Row: (1 98 0.081421256 4.928584134866222d0)
Row: (2 8 0.45165908 5.147222819038834d0)
Row: (3 56 0.62647486 4.874349648519968d0)
Row: (4 79 0.41671002 4.7239718274963485d0)
Row: (5 19 0.04152584 4.727268395019779d0)
Row: (6 1 0.3369373 4.93339303609316d0)
Row: (7 79 0.59791017 5.2466443304900965d0)
Row: (8 42 0.076958776 5.103448455243024d0)
Row: (9 7 0.85732913 5.310498824093041d0)

Plotting data

Plotting facilities as rudimentary. All functions related to plotting are in the teddy/plot package. Right now GNUPlot is used via eazy-gnuplot library.

Here is how we can plot our data from all columns:


POFTHEDAY> (teddy/plot:plot *d*
            "docs/media/0099/simple-plot.png")

If we want to plot only gaussian, then it will be wrong, because we need histogram type of plot. This it "to be done":

POFTHEDAY> (teddy/plot:plot
            (teddy/data-frame:slice *d*
                                    :columns '("Idx" "Gaussian"))
            "docs/media/0099/gaussian.png")

Another type of plots Teddy is able to render right now is a "timeseries":


POFTHEDAY> (defparameter *moscow-population*
             (teddy/data-frame:make-data-frame
              '("Date" "Population")
              :rows '(("1350-01-01" 30000)
                      ("1840-01-01" 349000)
                      ("1907-01-01" 1345700)
                      ("1967-01-01" 6422000)
                      ("1994-01-01" 9066000)
                      ("2010-01-01" 11500000)
                      ("2020-01-01" 12680000))))
*MOSCOW-POPULATION*
POFTHEDAY> (teddy/plot:plot-timeseries
            *moscow-population* "docs/media/0099/moscow2.png"
            :title "Moscow population")
"docs/media/0099/moscow.png"

Join the effort to make Teddy really useful for data analysis!

Right now, Teddy installable only from Ultralisp, because it is the best place to host unstable fast changing Common Lisp libraries.

Credentials


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Next: , Previous: , Up: Systems   [Contents][Index]

2.1 teddy

Author

Alexander Artemenko <svetlyak.40wt@gmail.com>

License

UNLICENSE

Description

A data framework for Common Lisp, wanna be like Pandas for Python.

Defsystem Dependency

asdf-finalizers

Dependencies
Source

teddy.asd (file)


Next: , Previous: , Up: Systems   [Contents][Index]

2.2 teddy/index

Author

Alexander Artemenko <svetlyak.40wt@gmail.com>

License

UNLICENSE

Dependencies
Source

teddy.asd (file)

Component

lisp.lisp (file)


Next: , Previous: , Up: Systems   [Contents][Index]

2.3 teddy/stats

Author

Alexander Artemenko <svetlyak.40wt@gmail.com>

License

UNLICENSE

Dependencies
Source

teddy.asd (file)

Component

lisp.lisp (file)


Next: , Previous: , Up: Systems   [Contents][Index]

2.4 teddy/print

Author

Alexander Artemenko <svetlyak.40wt@gmail.com>

License

UNLICENSE

Dependencies
Source

teddy.asd (file)

Component

lisp.lisp (file)


Next: , Previous: , Up: Systems   [Contents][Index]

2.5 teddy/plot

Author

Alexander Artemenko <svetlyak.40wt@gmail.com>

License

UNLICENSE

Dependencies
Source

teddy.asd (file)

Component

lisp.lisp (file)


Next: , Previous: , Up: Systems   [Contents][Index]

2.6 teddy/data-frame

Author

Alexander Artemenko <svetlyak.40wt@gmail.com>

License

UNLICENSE

Dependencies
Source

teddy.asd (file)

Component

lisp.lisp (file)


Previous: , Up: Systems   [Contents][Index]

2.7 teddy/utils

Author

Alexander Artemenko <svetlyak.40wt@gmail.com>

License

UNLICENSE

Dependencies
Source

teddy.asd (file)

Component

lisp.lisp (file)


Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 teddy.asd

Location

/home/quickref/quicklisp/dists/quicklisp/software/teddy-20200715-git/teddy.asd

Systems

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 teddy/index/lisp.lisp

Parent

teddy/index (system)

Location

index.lisp

Packages

teddy/index

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.3 teddy/stats/lisp.lisp

Parent

teddy/stats (system)

Location

stats.lisp

Packages

teddy/stats

Exported Definitions

stats (function)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.4 teddy/print/lisp.lisp

Parent

teddy/print (system)

Location

print.lisp

Packages

teddy/data-frame

Exported Definitions

set-num-digits-after-point (function)

Internal Definitions

make-value-formatter (function)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.5 teddy/plot/lisp.lisp

Parent

teddy/plot (system)

Location

plot.lisp

Packages

teddy/plot

Exported Definitions
Internal Definitions

filename->terminal (function)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.6 teddy/data-frame/lisp.lisp

Parent

teddy/data-frame (system)

Location

data-frame.lisp

Internal Definitions

Previous: , Up: Lisp files   [Contents][Index]

3.1.7 teddy/utils/lisp.lisp

Parent

teddy/utils (system)

Location

utils.lisp

Packages

teddy/utils

Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Next: , Previous: , Up: Packages   [Contents][Index]

4.1 teddy/index

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Packages   [Contents][Index]

4.2 teddy/stats

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions

stats (function)


Next: , Previous: , Up: Packages   [Contents][Index]

4.3 teddy/data-frame

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions

set-num-digits-after-point (function)

Internal Definitions

Next: , Previous: , Up: Packages   [Contents][Index]

4.4 teddy/plot

Source

lisp.lisp (file)

Use List

common-lisp

Exported Definitions
Internal Definitions

filename->terminal (function)


Previous: , Up: Packages   [Contents][Index]

4.5 teddy/utils

Source

lisp.lisp (file)

Use List

common-lisp

Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Functions

Function: make-index DATA-FRAME COLUMN
Package

teddy/index

Source

lisp.lisp (file)

Function: plot DATA-FRAME FILENAME &key TITLE DEBUG
Package

teddy/plot

Source

lisp.lisp (file)

Function: plot-timeseries DATAFRAME FILENAME &key X Y TITLE

Plots a timeseries where axis x has a timestamp values.
If column for axis y is not given, then second column will be used.

Package

teddy/plot

Source

lisp.lisp (file)

Function: set-num-digits-after-point NUM
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: stats DATA-FRAME

Returns a new dataframe where each column holds different stats on corresponding columns from original data-frame.

Package

teddy/stats

Source

lisp.lisp (file)


Previous: , Up: Exported definitions   [Contents][Index]

5.1.2 Generic functions

Generic Function: scalar INDEX VALUE COLUMN-NAME
Package

teddy/index

Methods
Method: scalar (INDEX index) VALUE COLUMN-NAME

Returns a cell content corresponding to the value and column-name.

Source

lisp.lisp (file)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Functions

Function: column-idx DATA-FRAME NAME

Returns a number of the column

Package

teddy/data-frame

Source

lisp.lisp (file)

Function: ensure-vector OBJ &key ELEMENT-TYPE
Package

teddy/utils

Source

lisp.lisp (file)

Function: filename->terminal FILENAME

Transforms string or pathname into a list suitable for passing as :terminal value to eazy-gnuplot:gp-setup.

Package

teddy/plot

Source

lisp.lisp (file)

Function: get-column DATA-FRAME NAME &key AS

Returns column by name

Package

teddy/data-frame

Source

lisp.lisp (file)

Function: head DATA-FRAME &optional ROWS
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: infer-type VALUE
Package

teddy/utils

Source

lisp.lisp (file)

Function: make-data-frame COLUMN-NAMES &key ROWS COLUMNS TYPES
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: make-iterator DATA-FRAME

Returns a function from zero arguments which will return a next row on each call. When all rows will be returned, iterator will return nil.

Rows are returned as lists.

Package

teddy/data-frame

Source

lisp.lisp (file)

Function: make-value-formatter NUM-DIGITS-AFTER-POINT
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: num-columns DATA-FRAME
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: num-rows DATA-FRAME
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: row-index INDEX VALUE

Internal function to getting the row index by the value.

Package

teddy/index

Source

lisp.lisp (file)

Function: slice DATA-FRAME &key COLUMNS FROM TO
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: tail DATA-FRAME &optional ROWS
Package

teddy/data-frame

Source

lisp.lisp (file)

Function: vector-to-list VECTOR
Package

teddy/utils

Source

lisp.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Generic functions

Generic Function: get-column-names OBJECT
Package

teddy/data-frame

Methods
Method: get-column-names (DATA-FRAME data-frame)

Column names

Source

lisp.lisp (file)

Generic Function: get-columns OBJECT
Package

teddy/data-frame

Methods
Method: get-columns (DATA-FRAME data-frame)

A vector of column vectors

Source

lisp.lisp (file)

Generic Function: get-types OBJECT
Package

teddy/data-frame

Methods
Method: get-types (DATA-FRAME data-frame)

A list of columns

Source

lisp.lisp (file)

Generic Function: index-column OBJECT
Package

teddy/index

Methods
Method: index-column (INDEX index)

automatically generated reader method

Source

lisp.lisp (file)

Generic Function: index-data-frame OBJECT
Package

teddy/index

Methods
Method: index-data-frame (INDEX index)

automatically generated reader method

Source

lisp.lisp (file)

Generic Function: index-positions OBJECT
Package

teddy/index

Methods
Method: index-positions (INDEX index)

automatically generated reader method

Source

lisp.lisp (file)

Generic Function: scalar OBJ ROW-IDX COLUMN-NAME

Returns a cell content corresponding to the row-idx and column-name.

Package

teddy/data-frame

Source

lisp.lisp (file)

Methods
Method: scalar (DATA-FRAME data-frame) ROW-IDX COLUMN-NAME

Returns a cell content corresponding to the row-idx and column-name.


Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Classes

Class: data-frame ()
Package

teddy/data-frame

Source

lisp.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
Direct slots
Slot: names

Column names

Type

(simple-array string (*))

Initargs

:names

Readers

get-column-names (generic function)

Slot: types

A list of columns

Type

simple-vector

Initargs

:types

Readers

get-types (generic function)

Slot: columns

A vector of column vectors

Type

(simple-array simple-vector (*))

Initargs

:columns

Readers

get-columns (generic function)

Class: index ()
Package

teddy/index

Source

lisp.lisp (file)

Direct superclasses

standard-object (class)

Direct methods
Direct slots
Slot: data-frame
Type

teddy/data-frame::data-frame

Initargs

:data-frame

Readers

index-data-frame (generic function)

Slot: column
Initargs

:column

Readers

index-column (generic function)

Slot: positions
Type

hash-table

Initform

(make-hash-table :test (quote equal))

Readers

index-positions (generic function)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   T  
Index Entry  Section

F
File, Lisp, teddy.asd: The teddy․asd file
File, Lisp, teddy/data-frame/lisp.lisp: The teddy/data-frame/lisp․lisp file
File, Lisp, teddy/index/lisp.lisp: The teddy/index/lisp․lisp file
File, Lisp, teddy/plot/lisp.lisp: The teddy/plot/lisp․lisp file
File, Lisp, teddy/print/lisp.lisp: The teddy/print/lisp․lisp file
File, Lisp, teddy/stats/lisp.lisp: The teddy/stats/lisp․lisp file
File, Lisp, teddy/utils/lisp.lisp: The teddy/utils/lisp․lisp file

L
Lisp File, teddy.asd: The teddy․asd file
Lisp File, teddy/data-frame/lisp.lisp: The teddy/data-frame/lisp․lisp file
Lisp File, teddy/index/lisp.lisp: The teddy/index/lisp․lisp file
Lisp File, teddy/plot/lisp.lisp: The teddy/plot/lisp․lisp file
Lisp File, teddy/print/lisp.lisp: The teddy/print/lisp․lisp file
Lisp File, teddy/stats/lisp.lisp: The teddy/stats/lisp․lisp file
Lisp File, teddy/utils/lisp.lisp: The teddy/utils/lisp․lisp file

T
teddy.asd: The teddy․asd file
teddy/data-frame/lisp.lisp: The teddy/data-frame/lisp․lisp file
teddy/index/lisp.lisp: The teddy/index/lisp․lisp file
teddy/plot/lisp.lisp: The teddy/plot/lisp․lisp file
teddy/print/lisp.lisp: The teddy/print/lisp․lisp file
teddy/stats/lisp.lisp: The teddy/stats/lisp․lisp file
teddy/utils/lisp.lisp: The teddy/utils/lisp․lisp file

Jump to:   F   L   T  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   C   E   F   G   H   I   M   N   P   R   S   T   V  
Index Entry  Section

C
column-idx: Internal functions

E
ensure-vector: Internal functions

F
filename->terminal: Internal functions
Function, column-idx: Internal functions
Function, ensure-vector: Internal functions
Function, filename->terminal: Internal functions
Function, get-column: Internal functions
Function, head: Internal functions
Function, infer-type: Internal functions
Function, make-data-frame: Internal functions
Function, make-index: Exported functions
Function, make-iterator: Internal functions
Function, make-value-formatter: Internal functions
Function, num-columns: Internal functions
Function, num-rows: Internal functions
Function, plot: Exported functions
Function, plot-timeseries: Exported functions
Function, row-index: Internal functions
Function, set-num-digits-after-point: Exported functions
Function, slice: Internal functions
Function, stats: Exported functions
Function, tail: Internal functions
Function, vector-to-list: Internal functions

G
Generic Function, get-column-names: Internal generic functions
Generic Function, get-columns: Internal generic functions
Generic Function, get-types: Internal generic functions
Generic Function, index-column: Internal generic functions
Generic Function, index-data-frame: Internal generic functions
Generic Function, index-positions: Internal generic functions
Generic Function, scalar: Exported generic functions
Generic Function, scalar: Internal generic functions
get-column: Internal functions
get-column-names: Internal generic functions
get-column-names: Internal generic functions
get-columns: Internal generic functions
get-columns: Internal generic functions
get-types: Internal generic functions
get-types: Internal generic functions

H
head: Internal functions

I
index-column: Internal generic functions
index-column: Internal generic functions
index-data-frame: Internal generic functions
index-data-frame: Internal generic functions
index-positions: Internal generic functions
index-positions: Internal generic functions
infer-type: Internal functions

M
make-data-frame: Internal functions
make-index: Exported functions
make-iterator: Internal functions
make-value-formatter: Internal functions
Method, get-column-names: Internal generic functions
Method, get-columns: Internal generic functions
Method, get-types: Internal generic functions
Method, index-column: Internal generic functions
Method, index-data-frame: Internal generic functions
Method, index-positions: Internal generic functions
Method, scalar: Exported generic functions
Method, scalar: Internal generic functions

N
num-columns: Internal functions
num-rows: Internal functions

P
plot: Exported functions
plot-timeseries: Exported functions

R
row-index: Internal functions

S
scalar: Exported generic functions
scalar: Exported generic functions
scalar: Internal generic functions
scalar: Internal generic functions
set-num-digits-after-point: Exported functions
slice: Internal functions
stats: Exported functions

T
tail: Internal functions

V
vector-to-list: Internal functions

Jump to:   C   E   F   G   H   I   M   N   P   R   S   T   V  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   C   D   N   P   S   T  
Index Entry  Section

C
column: Internal classes
columns: Internal classes

D
data-frame: Internal classes

N
names: Internal classes

P
positions: Internal classes

S
Slot, column: Internal classes
Slot, columns: Internal classes
Slot, data-frame: Internal classes
Slot, names: Internal classes
Slot, positions: Internal classes
Slot, types: Internal classes

T
types: Internal classes

Jump to:   C   D   N   P   S   T  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   C   D   I   P   S   T  
Index Entry  Section

C
Class, data-frame: Internal classes
Class, index: Internal classes

D
data-frame: Internal classes

I
index: Internal classes

P
Package, teddy/data-frame: The teddy/data-frame package
Package, teddy/index: The teddy/index package
Package, teddy/plot: The teddy/plot package
Package, teddy/stats: The teddy/stats package
Package, teddy/utils: The teddy/utils package

S
System, teddy: The teddy system
System, teddy/data-frame: The teddy/data-frame system
System, teddy/index: The teddy/index system
System, teddy/plot: The teddy/plot system
System, teddy/print: The teddy/print system
System, teddy/stats: The teddy/stats system
System, teddy/utils: The teddy/utils system

T
teddy: The teddy system
teddy/data-frame: The teddy/data-frame system
teddy/data-frame: The teddy/data-frame package
teddy/index: The teddy/index system
teddy/index: The teddy/index package
teddy/plot: The teddy/plot system
teddy/plot: The teddy/plot package
teddy/print: The teddy/print system
teddy/stats: The teddy/stats system
teddy/stats: The teddy/stats package
teddy/utils: The teddy/utils system
teddy/utils: The teddy/utils package

Jump to:   C   D   I   P   S   T