This is the dfio Reference Manual, version 1.0.2, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Feb 26 16:17:27 2024 GMT+0.
The main system appears first, followed by any subsystem dependency.
dfioCommon Lisp library for reading and writing data-frames
Data frame I/O
Steve Nunez <steve@symbolics.tech>
(GIT https://github.com/Lisp-Stat/dfio.git)
MS-PL
DFIO is used to read and write data frames from string, streams or disk. When building a library like this, one question that needs to be answered is "where do you draw the line on formats?". For example, should DFIO read/write JSON? The answer to that question is ’no’, and the reason is that there are many JSON formats – they are applicaton specific. Application specific format should be read in an application specific package, and this is the case, for example, with Vega. Formats like CSV, HD5 or Feather are application agnostic and should be included in DFIO.
1.0.2
alexandria (system).
anaphora (system).
data-frame (system).
dexador (system).
fare-csv (system).
let-plus (system).
pkgdcl.lisp (file).
decimal.lisp (file).
string-table.lisp (file).
data-column.lisp (file).
utils.lisp (file).
write.lisp (file).
delimited-text.lisp (file).
data.lisp (file).
rdata.lisp (file).
Files are sorted by type and then listed depth-first from the systems components trees.
dfio/dfio.asddfio/pkgdcl.lispdfio/decimal.lispdfio/string-table.lispdfio/data-column.lispdfio/utils.lispdfio/write.lispdfio/delimited-text.lispdfio/data.lispdfio/rdata.lispdfio/decimal.lisppkgdcl.lisp (file).
dfio (system).
parse-rational (function).
parse-rational-error (condition).
parse-real (function).
+exponent-chars+ (special variable).
gobble-positive-integer (function).
gobble-sign (function).
dfio/string-table.lispdecimal.lisp (file).
dfio (system).
print-object (method).
string-table (function).
string-table (structure).
string-table-add (function).
string-table-count (function).
string-table-duplicate (condition).
string-table-intern (function).
string-table-lookup (function).
string-table-not-found (condition).
string-table-strings (function).
copy-string-table (function).
string-table-get (function).
(setf string-table-get) (function).
string-table-p (function).
string-table-table (reader).
(setf string-table-table) (writer).
dfio/data-column.lispstring-table.lisp (file).
dfio (system).
data-column (function).
data-column (class).
data-column-add (function).
data-column-counts (function).
data-column-vector (function).
non-negative-integer (type).
dfio/utils.lispdata-column.lisp (file).
dfio (system).
string-to-keyword (function).
string-to-symbol (function).
symbol-name-to-pathname (function).
%in-stream (function).
%out-stream (function).
*default-external-format* (special variable).
str-strm-file (type).
with-csv-output-stream (macro).
with-input-stream (macro).
dfio/write.lisputils.lisp (file).
dfio (system).
save (function).
write-df (function).
write-properties (function).
dfio/delimited-text.lispwrite.lisp (file).
dfio (system).
csv-to-data-columns (function).
dfio/data.lispdelimited-text.lisp (file).
dfio (system).
data (function).
dfio/rdata.lispdata.lisp (file).
dfio (system).
*r-default-datasets* (special variable).
ability.cov (special variable).
airmiles (special variable).
airpassengers (special variable).
airquality (special variable).
anscombe (special variable).
attenu (special variable).
attitude (special variable).
austres (special variable).
base-url (special variable).
bjsales (special variable).
bod (special variable).
cars (special variable).
chickweight (special variable).
chickwts (special variable).
co2-1 (special variable).
co2-2 (special variable).
crimtab (special variable).
discoveries (special variable).
dnase (special variable).
esoph (special variable).
euro (special variable).
eustockmarkets (special variable).
faithful (special variable).
formaldehyde (special variable).
freeny (special variable).
haireyecolor (special variable).
harman23.cor (special variable).
harman74.cor (special variable).
index (special variable).
indometh (special variable).
infert (special variable).
insectsprays (special variable).
iris (special variable).
iris3 (special variable).
islands (special variable).
johnsonjohnson (special variable).
lakehuron (special variable).
lh (special variable).
lifecyclesavings (special variable).
loblolly (special variable).
longley (special variable).
lynx (special variable).
morley (special variable).
mpg (special variable).
mtcars (special variable).
nhtemp (special variable).
nile (special variable).
nottem (special variable).
npk (special variable).
nycflights13-airlines (special variable).
nycflights13-airports (special variable).
nycflights13-flights (special variable).
nycflights13-planes (special variable).
nycflights13-weather (special variable).
occupationalstatus (special variable).
orange (special variable).
orchardsprays (special variable).
plantgrowth (special variable).
precip (special variable).
presidents (special variable).
pressure (special variable).
puromycin (special variable).
quakes (special variable).
randu (special variable).
rivers (special variable).
rock (special variable).
seatbelts (special variable).
stackloss (special variable).
sunspot.month (special variable).
sunspot.year (special variable).
sunspots (special variable).
swiss (special variable).
theoph (special variable).
titanic (special variable).
toothgrowth (special variable).
treering (special variable).
trees (special variable).
ucbadmissions (special variable).
ukdriverdeaths (special variable).
ukgas (special variable).
usaccdeaths (special variable).
usarrests (special variable).
usjudgeratings (special variable).
uspersonalexpenditure (special variable).
uspop (special variable).
vadeaths (special variable).
volcano (special variable).
warpbreaks (special variable).
women (special variable).
worldphones (special variable).
wwwusage (special variable).
*r-default-dataframes* (special variable).
student-sleep (special variable).
Packages are listed by definition order.
dfio.string-tablealexandria.
anaphora.
common-lisp.
let-plus.
string-table (function).
string-table (structure).
string-table (slot).
string-table-add (function).
string-table-count (function).
string-table-duplicate (condition).
string-table-intern (function).
string-table-lookup (function).
string-table-not-found (condition).
string-table-strings (function).
copy-string-table (function).
string-table-get (function).
(setf string-table-get) (function).
string-table-p (function).
string-table-table (reader).
(setf string-table-table) (writer).
dfioalexandria.
anaphora.
common-lisp.
data-frame.
dfio.data-column.
let-plus.
data (function).
read-csv (function).
save (function).
string-to-keyword (function).
string-to-symbol (function).
symbol-name-to-pathname (function).
write-csv (function).
write-df (function).
write-properties (function).
%in-stream (function).
%out-stream (function).
*default-external-format* (special variable).
csv-to-data-columns (function).
str-strm-file (type).
with-csv-output-stream (macro).
with-input-stream (macro).
dfio.decimalanaphora.
common-lisp.
let-plus.
parse-rational (function).
parse-rational-error (condition).
parse-real (function).
+exponent-chars+ (special variable).
gobble-positive-integer (function).
gobble-sign (function).
dfio.data-columnanaphora.
common-lisp.
dfio.decimal.
dfio.string-table.
let-plus.
dfio.
data-column (function).
data-column (class).
data-column-add (function).
data-column-counts (function).
data-column-vector (function).
non-negative-integer (type).
rdatacommon-lisp.
*r-default-datasets* (special variable).
ability.cov (special variable).
airmiles (special variable).
airpassengers (special variable).
airquality (special variable).
anscombe (special variable).
attenu (special variable).
attitude (special variable).
austres (special variable).
base-url (special variable).
bjsales (special variable).
bod (special variable).
cars (special variable).
chickweight (special variable).
chickwts (special variable).
co2-1 (special variable).
co2-2 (special variable).
crimtab (special variable).
discoveries (special variable).
dnase (special variable).
esoph (special variable).
euro (special variable).
eustockmarkets (special variable).
faithful (special variable).
formaldehyde (special variable).
freeny (special variable).
haireyecolor (special variable).
harman23.cor (special variable).
harman74.cor (special variable).
index (special variable).
indometh (special variable).
infert (special variable).
insectsprays (special variable).
iris (special variable).
iris3 (special variable).
islands (special variable).
johnsonjohnson (special variable).
lakehuron (special variable).
lh (special variable).
lifecyclesavings (special variable).
loblolly (special variable).
longley (special variable).
lynx (special variable).
morley (special variable).
mpg (special variable).
mtcars (special variable).
nhtemp (special variable).
nile (special variable).
nottem (special variable).
npk (special variable).
nycflights13-airlines (special variable).
nycflights13-airports (special variable).
nycflights13-flights (special variable).
nycflights13-planes (special variable).
nycflights13-weather (special variable).
occupationalstatus (special variable).
orange (special variable).
orchardsprays (special variable).
plantgrowth (special variable).
precip (special variable).
presidents (special variable).
pressure (special variable).
puromycin (special variable).
quakes (special variable).
randu (special variable).
rivers (special variable).
rock (special variable).
seatbelts (special variable).
stackloss (special variable).
sunspot.month (special variable).
sunspot.year (special variable).
sunspots (special variable).
swiss (special variable).
theoph (special variable).
titanic (special variable).
toothgrowth (special variable).
treering (special variable).
trees (special variable).
ucbadmissions (special variable).
ukdriverdeaths (special variable).
ukgas (special variable).
usaccdeaths (special variable).
usarrests (special variable).
usjudgeratings (special variable).
uspersonalexpenditure (special variable).
uspop (special variable).
vadeaths (special variable).
volcano (special variable).
warpbreaks (special variable).
women (special variable).
worldphones (special variable).
wwwusage (special variable).
*r-default-dataframes* (special variable).
student-sleep (special variable).
Definitions are sorted by export status, category, package, and then by lexicographic order.
All data sets included by default in R
Ability and Intelligence Tests
Passenger Miles on Commercial US Airlines, 1937-1960
Monthly Airline Passenger Numbers 1949-1960
New York Air Quality Measurements
Anscombe’s Quartet of ’Identical’ Simple Linear Regressions
The Joyner-Boore Attenuation Data
The Chatterjee-Price Attitude Data
Quarterly Time Series of the Number of Australian Residents
Base URL for datasets included in R
Sales Data with Leading Indicator
Biochemical Oxygen Demand
Speed and Stopping Distances of Cars
Weight versus age of chicks on different diets
Chicken Weights by Feed Type
Carbon Dioxide Uptake in Grass Plants
Mauna Loa Atmospheric CO2 Concentration
Student’s 3000 Criminals Data
Yearly Numbers of Important Discoveries
Elisa assay of DNase
Smoking, Alcohol and (O)esophageal Cancer
Conversion Rates of Euro Currencies
Daily Closing Prices of Major European Stock Indices, 1991-1998
Old Faithful Geyser Data
Determination of Formaldehyde
Freeny’s Revenue Data
Hair and Eye Color of Statistics Students
Harman Example 2.3
Harman Example 7.4
Pharmacokinetics of Indomethacin
Infertility after Spontaneous and Induced Abortion
Effectiveness of Insect Sprays
Edgar Anderson’s Iris Data
Edgar Anderson’s Iris Data
Areas of the World’s Major Landmasses
Quarterly Earnings per Johnson & Johnson Share
Level of Lake Huron 1875-1972
Luteinizing Hormone in Blood Samples
Intercountry Life-Cycle Savings Data
Growth of Loblolly pine trees
Longley’s Economic Regression Data
Annual Canadian Lynx trappings 1821-1934
Michelson Speed of Light Data
Fuel economy data from 1999 to 2008 for 38 popular models of cars
Motor Trend Car Road Tests
Average Yearly Temperatures in New Haven
Flow of the River Nile
Average Monthly Temperatures at Nottingham, 1920-1939
Classical N, P, K Factorial Experiment
Airline name lookup table by carrier code
Airport metadata
On-time data for all flights that departed NYC (i.e. JFK, LGA or EWR) in 2013
Metadata for all airplane tail numbers found in the FAA aircraft registry
Hourly meterological data for LGA, JFK and EWR in 2013
Occupational Status of Fathers and their Sons
Growth of Orange Trees
Potency of Orchard Sprays
Results from an Experiment on Plant Growth
Annual Precipitation in US Cities
Quarterly Approval Ratings of US Presidents
Vapor Pressure of Mercury as a Function of Temperature
Reaction Velocity of an Enzymatic Reaction
Locations of Earthquakes off Fiji
Random Numbers from Congruential Generator RANDU
Lengths of Major North American Rivers
Measurements on Petroleum Rock Samples
Road Casualties in Great Britain 1969-84
Brownlee’s Stack Loss Plant Data
Monthly Sunspot Data, from 1749 to Present
Yearly Sunspot Data, 1700-1988
Monthly Sunspot Numbers, 1749-1983
Swiss Fertility and Socioeconomic Indicators (1888) Data
Pharmacokinetics of Theophylline
Survival of passengers on the Titanic
The Effect of Vitamin C on Tooth Growth in Guinea Pigs
Yearly Treering Data, -6000-1979
Diameter, Height and Volume for Black Cherry Trees
Student Admissions at UC Berkeley
Road Casualties in Great Britain 1969-84
UK Quarterly Gas Consumption
Accidental Deaths in the US 1973-1978
Violent Crime Rates by US State
Lawyers’ Ratings of State Judges in the US Superior Court
Personal Expenditure Data
Populations Recorded by the US Census
Death Rates in Virginia (1940)
Topographic Information on Auckland’s Maunga Whau Volcano
The Number of Breaks in Yarn during Weaving
Average Heights and Weights for American Women
The World’s Telephones
Internet Usage per Minute
Load a data frame from a CSV or LISP data source located on the local filesystem named by D. Intended for example data sets for Lisp-Stat system. Parameters may be either a KEYWORD or STRING. JSON files require application specific loaders, so not handled here. Use (read-vega ...) for example.
Description
Each package using lisp-stat should define its own logical host, and a directory called DATA. Once done, you can load the example data sets like so:
LS-USER> (data my-example :system :glimpse)
If the system is named GLIMPSE. To load a data set from R, assuming you have configured a logical host named RDATA:
LS-USER> (data :antigua :system :rdata :directory :daag :type :csv)
Return the counts.
Return the collected elements as a vector.
Parse a decimal rational in (subseq string start end) of the form [sign][whole][.[fraction]][exponent] where
sign ::= + | - | empty
whole ::= digit*
fraction ::= digit*
exponent ::= exponent-char[sign]digit+
with the restriction that WHOLE and FRACTION cannot be empty at the same time. EXPONENT-CHAR is a string and contains the valid exponent chars.
Whitespace is NOT trimmed, and leads to an error. In case of a parsing failure, PARSE-RATIONAL-ERROR is used.
Return (values NUMBER DECIMAL-DOT? EXPONENT-CHAR). NUMBER is a RATIONAL, DECIMAL-DOT? is T when a decimal dot is present, otherwise NIL, EXPONENT-CHAR contains the exponent character, NIL if not present.
Numbers of the form .112 and 112. are valid syntax, representing 0.112 and 112.0, respectively.
Examples:
(parse-rational "7") => (values 7 NIL NIL)
(parse-rational "7.") => (values 7 T NIL)
(parse-rational "0.7") => (values 7/10 T NIL)
(parse-rational ".7") => (values 7/10 T NIL)
(parse-rational "7.e2") => (values 700 T #e)
(parse-rational ".7d1") => (values 7 T #d)
Wrapper for PARSE-RATIONAL, converting non-integers to floats. The float type is determined by the -float arguments for each exponent character. Integers are not converted to floats. Return a single value, type of (or integer float).
See PARSE-RATIONAL for accepted formats, errors, etc.
Read a CSV file, stream, string or URL into a DATA-FRAME, which is returned.
When SKIP-FIRST-ROW?, the first row is read separately and COLUMN-KEYS-OR-FUNCTION is used to form column keys.
When COLUMN-KEYS-OR-FUNCTION is a sequence, it is used for column keys, regardless of the value of SKIP-FIRST-ROW?.
PACKAGE indicates the package to intern column names into.
MAP-ALIST maps values during the import. This is useful if you want special mappings for missing, though the mechanism is general. Returns two values, the data-frame and the source
dfio.
Save DF in the file named by PATHSPEC. By default, a suffix of .lisp is added.
dfio.
Add STRING mapped to VALUE to STRING-TABLE, raising STRING-TABLE-DUPLICATE if STRING is already in the table. Return VALUE.
Number of distinct strings in the table.
If STRING is already in STRING-TABLE, return its value, otherwise add it and return NEW-VALUE. When used with the default argument for NEW-VALUE, EQUAL strings are always mapped to values that are EQ.
Return the value corresponding to STRING in STRING-TABLE, or raise the STRING-TABLE-NOT-FOUND error.
List of strings in STRING-TABLE.
Map string to a keyword.
The current implementation replaces #. and #space with a #-, and upcases all other characters.
dfio.
Map STRING to a symbol in PACKAGE, replacing #., #_ and #space with a #-, and upcasing all other characters. Exports symbol.
dfio.
Map the symbol-name of S to something that can be part of a logical-pathname
dfio.
Write DF to STRING-OR-STREAM in CSV format. STRING-OR-STREAM can be a STREAM, a STRING or a file PATHSPEC.
Keywords:
string-or-stream: stream to write to. Default: nil, returning a string
add-first-row: add column names as the first row
separator: separator to use when reading or writing CSV files. A character. By default, a comma: #,
quote: quote character to use when reading or writing CSV files. A character. By default, a double-quote: #"
eol: line ending to use when writing CSV files. A string. By default, +CRLF+ as specified by creativyst.
Notes:
The :newline keyword requires a sequence, so use :newline ’(#newline)
dfio.
Write DATA-SYMBOL to STREAM in a format suitable for reading back in with the Lisp reader
dfio.
Write the variable PROPERTY strings to stream so they can be read back in when LOADed from a lisp file. By convention, the name of the function that sets the property is the same as the property. Example (write-property mtcars :label)
dfio.
Error used by parse-rational and parse-real.
String is already in the table.
error.
String not found in table.
error.
A table of distinct strings, optionally mapping each one to a value.
structure-object.
hash-table
(make-hash-table :test (function equalp))
list
symbol
:default-float-format
dfio.data-column::non-negative-integer
0
dfio.data-column::non-negative-integer
0
integer
0
integer
0
dfio.data-column::non-negative-integer
0
dfio.string-table:string-table
:map-table
dfio.data-column::non-negative-integer
0
dfio.string-table:string-table
(dfio.string-table:string-table)
External format used for opening files
dfio.
Data frames corresponding to the default R datasets
Default exponent characters.
Student’s Sleep Data
creates a stream from the given thing, trying to DWIM
dfio.
Read a CSV file (or stream, or string), accumulate the values in DATA-COLUMNs, return a list of these. Rows are checked to have the same number of elements.
When SKIP-FIRST-ROW?, the first row is read separately and returned as the second value (list of strings), otherwise it is considered data like all other rows.
dfio.
If (SUBSEQ STRING START END) starts with a nonnegative integer (ie a sequence of digits 0-9), return the integer and position at which it ends as two values.
Otherwise, return NIL and 0.
START < END has to hold, END cannot be NIL. Consequences are undefined when START >= END.
Return (values SIGNUM INDEX), where SIGNUM is -1 or 1 depending on whether (CHAR STRING START) was a sign, and INDEX is the index of the subsequent character (START or START+1).
Synonym for GETHASH, used internally.
Synonym for (SETF GETHASH), used internally, checks that STRING is a string.
| Jump to: | %
(
C D F G M P R S W |
|---|
| Jump to: | %
(
C D F G M P R S W |
|---|
| Jump to: | *
+
A B C D E F H I J L M N O P Q R S T U V W |
|---|
| Jump to: | *
+
A B C D E F H I J L M N O P Q R S T U V W |
|---|
| Jump to: | C D F N P R S T U W |
|---|
| Jump to: | C D F N P R S T U W |
|---|