The cl-csv Reference Manual

Table of Contents

The cl-csv Reference Manual

This is the cl-csv Reference Manual, version 1.0.6, generated automatically by Declt version 3.0 "Montgomery Scott" on Fri Jun 26 10:08:05 2020 GMT+0.

1 Introduction

cl-csv

This library aims to simplify working with csvs to the bare minimum of tedium

• reads/writes csvs from/to strings, streams and files
• support streaming reads (allowing processing very large csvs, through read-csv's row-fn paramter)
• supports custom data formating
• settable quote, separator and quote-escapes
• supports multiline quoted data
• A test suite
• Detailed state about the process on error (line number, column number, char index), current collection state

Rationale

I had many scattered, not well tested, not easily runable pieces of csv code. I was unhappy with this situation then decided to refactor all of this into a single location. I wrote tests for it and had a library so I thought I might release it. This project started as extensions and bugfixes on arnesi's CSV.

I then looked around and saw there are other csv libs out there that probably mostly accomplished what I had set out to do. However, I already had code that was tested and had an easier license (BSD) so, I figured why not just release it anyway.

Other Available CSV libs

• http://members.optusnet.com.au/apicard/csv-parser.lisp
• http://www.cliki.net/fare-csv

Escaping and quotes

There are two modes for escaping currently

• :quote - by default cl-csv treats "" as an escape for a single double-quote
• :following - read the character after the escape sequence verbatim, commonly the *quote-escape* will be set to #\\ when the escape mode is following.

Signals and Restarts

• *enable-signals* will cause a csv-data-read or csv-row-read to be signaled for each piece of data and row read. There is a filter restart available which will cause the filter value to be used instead. Enabling signals is ~2xs as slow as not, so by default they are not enabled.
• in-csv iterate clause and read-csv support continue and filter restarts for errors occuring during read-csv-row

Functional Filters

• row-fn (lambda (row) ) - does something with a row instead of collecting it
• map-fn (lambda (row) ) - can modify a row on the way to collecting it
• data-map-fn (lambda (datum &key csv-reader &allow-other-keys) ) can modify a single datum on the way to collecting it, used to implement empty string to null conversions

Library Integration

• data-table - functions for building data-tables from csv's, must (asdf:load-system :cl-csv-data-table)
• clsql must (asdf:load-system :cl-csv-clsql)
  • import-from-csv
  • serial-import-from-csv
• iterate - provides an in-csv driver clause for iterating over a CSV

Examples

;; read a file into a list of lists
(cl-csv:read-csv #P"file.csv")
=> (("1" "2" "3") ("4" "5" "6"))

;; read a file that's tab delimited
(cl-csv:read-csv #P"file.tab" :separator #\Tab)

;; read a file and return a list of objects created from each row
(cl-csv:read-csv #P"file.csv"
                 :map-fn #'(lambda (row)
                             (make-instance 'object
                                            :foo (nth 0 row)
                                            :baz (nth 2 row))))
;; read csv from a string (streams also supported)
(cl-csv:read-csv "1,2,3
4,5,6")
=> (("1" "2" "3") ("4" "5" "6"))

;; loop over a CSV for effect
(let ((sum 0))
  (cl-csv:do-csv (row #P"file.csv")
    (incf sum (parse-integer (nth 0 row))))
  sum)
  
  
;; loop over a CSV using iterate
(iter (for (foo bar baz) in-csv #P"file.csv")
  (collect (make-instance 'object :foo foo :baz baz)))

Changelog

• v 1.0.2 - standardized on csv-reader for a name instead of using table in some places and reader in others Updated iterate clauses

• v 1.0.1 - New parser, same interface, faster and cleaner, and more flexible added data-map-fn, use this to handle null conversion

• v 1.0 - Long term stable version

Authors

• Acceleration.net
  • Russ Tyndall
  • Nathan Bird
  • Ryan Davis

;; Copyright (c) 2011 Russ Tyndall , Acceleration.net http://www.acceleration.net
;; Copyright (c) 2002-2006, Edward Marco Baringer
;; All rights reserved.
;;
;; Redistribution and use in source and binary forms, with or without
;; modification, are permitted provided that the following conditions are
;; met:
;;
;;  - Redistributions of source code must retain the above copyright
;;    notice, this list of conditions and the following disclaimer.
;;
;;  - Redistributions in binary form must reproduce the above copyright
;;    notice, this list of conditions and the following disclaimer in the
;;    documentation and/or other materials provided with the distribution.
;;
;;  - Neither the name of Edward Marco Baringer, nor BESE, nor the names
;;    of its contributors may be used to endorse or promote products
;;    derived from this software without specific prior written permission.
;;
;; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
;; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
;; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
;; A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT
;; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
;; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
;; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
;; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
;; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
;; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
;; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 cl-csv

Author

Russ Tyndall (russ@acceleration.net), Acceleration.net

License

BSD

Description

Facilities for reading and writing CSV format files

Version

1.0.6

Dependencies

• iterate
• alexandria
• cl-interpol

Source

cl-csv.asd (file)

Components

• packages.lisp (file)
• vars.lisp (file)
• read-until.lisp (file)
• csv.lisp (file)
• parser.lisp (file)

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

3.1 Lisp

3.1.1 cl-csv.asd

Location

cl-csv.asd

Systems

cl-csv (system)

Packages

cl-csv.system

3.1.2 cl-csv/packages.lisp

Parent

cl-csv (system)

Location

packages.lisp

Packages

cl-csv

3.1.3 cl-csv/vars.lisp

Dependency

packages.lisp (file)

Parent

cl-csv (system)

Location

vars.lisp

Exported Definitions

• *enable-signals* (special variable)
• *quote* (special variable)
• *quote-escape* (special variable)
• *separator* (special variable)

Internal Definitions

• *always-quote* (special variable)
• *buffer-size* (special variable)
• *eof-char* (special variable)
• *escape-mode* (special variable)
• *quoted-empty-string-is-nil* (special variable)
• *read-newline* (special variable)
• *trim-outer-whitespace* (special variable)
• *unquoted-empty-string-is-nil* (special variable)
• *write-newline* (special variable)

3.1.4 cl-csv/read-until.lisp

Dependency

vars.lisp (file)

Parent

cl-csv (system)

Location

read-until.lisp

Internal Definitions

read-into-buffer-until (function)

3.1.5 cl-csv/csv.lisp

Dependency

read-until.lisp (file)

Parent

cl-csv (system)

Location

csv.lisp

Exported Definitions

• *default-external-format* (special variable)
• csv-data-read (function)
• csv-data-read (condition)
• csv-parse-error (function)
• csv-parse-error (condition)
• csv-row-read (function)
• csv-row-read (condition)
• data (method)
• (setf data) (method)
• do-csv (macro)
• format-csv-value (generic function)
• format-csv-value (method)
• read-csv (function)
• read-csv-row (function)
• read-csv-sample (function)
• row (method)
• (setf row) (method)
• write-csv (function)
• write-csv-row (function)
• write-csv-value (generic function)
• write-csv-value (method)

Internal Definitions

• %char-in (function)
• %escape-seq? (function)
• %in-stream (function)
• %out-stream (function)
• chars-in (function)
• clause-for-in-csv-1 (macro)
• clause-sampling-2 (macro)
• csv-reader (method)
• (setf csv-reader) (method)
• csv-reader (method)
• (setf csv-reader) (method)
• format-args (method)
• (setf format-args) (method)
• format-control (method)
• (setf format-control) (method)
• restartable-read-row (function)
• white-space? (function)
• with-csv-input-stream (macro)
• with-csv-output-stream (macro)

3.1.6 cl-csv/parser.lisp

Dependency

csv.lisp (file)

Parent

cl-csv (system)

Location

parser.lisp

Internal Definitions

• %escape-is-double-quote (function)
• %next-char (function)
• %trim-datum (function)
• after-quoted? (method)
• (setf after-quoted?) (method)
• buffer (method)
• (setf buffer) (method)
• character-idx (method)
• (setf character-idx) (method)
• character-line-idx (method)
• (setf character-line-idx) (method)
• check-and-distpatch (function)
• check-table-entry (method)
• collect-datum (function)
• collect-row-data (function)
• csv-reader (class)
• data-map-fn (method)
• (setf data-map-fn) (method)
• delimiter (method)
• (setf delimiter) (method)
• didx (method)
• (setf didx) (method)
• dispatch (method)
• (setf dispatch) (method)
• dlen (method)
• (setf dlen) (method)
• dlen-1 (method)
• (setf dlen-1) (method)
• drop-delimiter-chars (function)
• entries (method)
• (setf entries) (method)
• last-item (function)
• (setf last-item) (function)
• line-data (method)
• (setf line-data) (method)
• line-idx (method)
• (setf line-idx) (method)
• make-default-csv-reader (function)
• make-table-entry (function)
• map-empty-string-to-nil (function)
• map-fn (method)
• (setf map-fn) (method)
• parse-stream (method)
• (setf parse-stream) (method)
• read-csv-row-with-reader (function)
• read-csv-with-reader (function)
• read-dispatch-table (class)
• read-dispatch-table-entry (class)
• read-with-dispatch-table (function)
• reading-character (function)
• reading-escaped (method)
• reading-following-escaped (method)
• reading-newline (method)
• reading-quoted (method)
• reading-quoted-or-escaped (method)
• reading-quoted? (method)
• (setf reading-quoted?) (method)
• reading-separator (method)
• reset-table-entry (function)
• row-fn (method)
• (setf row-fn) (method)
• rows (method)
• (setf rows) (method)
• skip-row? (method)
• (setf skip-row?) (method)

4 Packages

Packages are listed by definition order.

4.1 cl-csv.system

Source

cl-csv.asd

Use List

• asdf/interface
• common-lisp

4.2 cl-csv

Source

packages.lisp (file)

Use List

• iterate
• common-lisp-user
• common-lisp

Exported Definitions

• *default-external-format* (special variable)
• *enable-signals* (special variable)
• *quote* (special variable)
• *quote-escape* (special variable)
• *separator* (special variable)
• csv-data-read (function)
• csv-data-read (condition)
• csv-parse-error (function)
• csv-parse-error (condition)
• csv-row-read (function)
• csv-row-read (condition)
• data (generic function)
• data (method)
• (setf data) (method)
• (setf data) (generic function)
• do-csv (macro)
• format-csv-value (generic function)
• format-csv-value (method)
• read-csv (function)
• read-csv-row (function)
• read-csv-sample (function)
• row (generic function)
• row (method)
• (setf row) (method)
• (setf row) (generic function)
• write-csv (function)
• write-csv-row (function)
• write-csv-value (generic function)
• write-csv-value (method)

Internal Definitions

• %char-in (function)
• %escape-is-double-quote (function)
• %escape-seq? (function)
• %in-stream (function)
• %next-char (function)
• %out-stream (function)
• %trim-datum (function)
• *always-quote* (special variable)
• *buffer-size* (special variable)
• *eof-char* (special variable)
• *escape-mode* (special variable)
• *quoted-empty-string-is-nil* (special variable)
• *read-newline* (special variable)
• *trim-outer-whitespace* (special variable)
• *unquoted-empty-string-is-nil* (special variable)
• *write-newline* (special variable)
• after-quoted? (generic function)
• after-quoted? (method)
• (setf after-quoted?) (method)
• (setf after-quoted?) (generic function)
• buffer (generic function)
• buffer (method)
• (setf buffer) (method)
• (setf buffer) (generic function)
• character-idx (generic function)
• character-idx (method)
• (setf character-idx) (method)
• (setf character-idx) (generic function)
• character-line-idx (generic function)
• character-line-idx (method)
• (setf character-line-idx) (method)
• (setf character-line-idx) (generic function)
• chars-in (function)
• check-and-distpatch (function)
• check-table-entry (generic function)
• check-table-entry (method)
• clause-for-in-csv-1 (macro)
• clause-sampling-2 (macro)
• collect-datum (function)
• collect-row-data (function)
• csv-reader (generic function)
• csv-reader (method)
• (setf csv-reader) (method)
• csv-reader (method)
• (setf csv-reader) (method)
• (setf csv-reader) (generic function)
• csv-reader (class)
• data-map-fn (generic function)
• data-map-fn (method)
• (setf data-map-fn) (method)
• (setf data-map-fn) (generic function)
• delimiter (generic function)
• delimiter (method)
• (setf delimiter) (method)
• (setf delimiter) (generic function)
• didx (generic function)
• didx (method)
• (setf didx) (method)
• (setf didx) (generic function)
• dispatch (generic function)
• dispatch (method)
• (setf dispatch) (method)
• (setf dispatch) (generic function)
• dlen (generic function)
• dlen (method)
• (setf dlen) (method)
• (setf dlen) (generic function)
• dlen-1 (generic function)
• dlen-1 (method)
• (setf dlen-1) (method)
• (setf dlen-1) (generic function)
• drop-delimiter-chars (function)
• entries (generic function)
• entries (method)
• (setf entries) (method)
• (setf entries) (generic function)
• format-args (generic function)
• format-args (method)
• (setf format-args) (method)
• (setf format-args) (generic function)
• format-control (generic function)
• format-control (method)
• (setf format-control) (method)
• (setf format-control) (generic function)
• last-item (function)
• (setf last-item) (function)
• line-data (generic function)
• line-data (method)
• (setf line-data) (method)
• (setf line-data) (generic function)
• line-idx (generic function)
• line-idx (method)
• (setf line-idx) (method)
• (setf line-idx) (generic function)
• make-default-csv-reader (function)
• make-table-entry (function)
• map-empty-string-to-nil (function)
• map-fn (generic function)
• map-fn (method)
• (setf map-fn) (method)
• (setf map-fn) (generic function)
• parse-stream (generic function)
• parse-stream (method)
• (setf parse-stream) (method)
• (setf parse-stream) (generic function)
• read-csv-row-with-reader (function)
• read-csv-with-reader (function)
• read-dispatch-table (class)
• read-dispatch-table-entry (class)
• read-into-buffer-until (function)
• read-with-dispatch-table (function)
• reading-character (function)
• reading-escaped (generic function)
• reading-escaped (method)
• reading-following-escaped (generic function)
• reading-following-escaped (method)
• reading-newline (generic function)
• reading-newline (method)
• reading-quoted (generic function)
• reading-quoted (method)
• reading-quoted-or-escaped (generic function)
• reading-quoted-or-escaped (method)
• reading-quoted? (generic function)
• reading-quoted? (method)
• (setf reading-quoted?) (method)
• (setf reading-quoted?) (generic function)
• reading-separator (generic function)
• reading-separator (method)
• reset-table-entry (function)
• restartable-read-row (function)
• row-fn (generic function)
• row-fn (method)
• (setf row-fn) (method)
• (setf row-fn) (generic function)
• rows (generic function)
• rows (method)
• (setf rows) (method)
• (setf rows) (generic function)
• skip-row? (generic function)
• skip-row? (method)
• (setf skip-row?) (method)
• (setf skip-row?) (generic function)
• white-space? (function)
• with-csv-input-stream (macro)
• with-csv-output-stream (macro)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Special variables

Special Variable: *default-external-format*

the external format used for opening files

Package

cl-csv

Source

csv.lisp (file)

Special Variable: *enable-signals*

Should the reading and writing process enable filtering signals

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *quote*

Default quote character

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *quote-escape*

Default setting for escaping quotes - by default this is a vector of #(*quote* *quote*)

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *separator*

Default separator character

Package

cl-csv

Source

vars.lisp (file)

5.1.2 Macros

Macro: do-csv (ROW-VAR STREAM-OR-PATHNAME &rest READ-CSV-KEYS) &body BODY

row-var: a variable that is passed into _body_

stream-or-pathname: a stream or a pathname to read the CSV data from

read-csv-keys: keys and values passed to the _read-csv_ function

body: body of the macro

Package

cl-csv

Source

csv.lisp (file)

5.1.3 Functions

Function: csv-data-read DATA &key CSV-READER

Package

cl-csv

Source

csv.lisp (file)

Function: csv-parse-error MSG &rest ARGS

Package

cl-csv

Source

csv.lisp (file)

Function: csv-row-read ROW &key CSV-READER

Package

cl-csv

Source

csv.lisp (file)

Function: read-csv STREAM-OR-STRING &rest ALL-KEYS &key CSV-READER ROW-FN MAP-FN DATA-MAP-FN SAMPLE SKIP-FIRST-P (SEPARATOR *SEPARATOR*) (QUOTE *QUOTE*) (ESCAPE *QUOTE-ESCAPE*) (UNQUOTED-EMPTY-STRING-IS-NIL *UNQUOTED-EMPTY-STRING-IS-NIL*) (QUOTED-EMPTY-STRING-IS-NIL *QUOTED-EMPTY-STRING-IS-NIL*) (TRIM-OUTER-WHITESPACE *TRIM-OUTER-WHITESPACE*) (NEWLINE *READ-NEWLINE*) (ESCAPE-MODE *ESCAPE-MODE*)

Package

cl-csv

Source

csv.lisp (file)

Function: read-csv-row STREAM-OR-STRING &key CSV-READER (SEPARATOR *SEPARATOR*) (QUOTE *QUOTE*) (ESCAPE *QUOTE-ESCAPE*) (UNQUOTED-EMPTY-STRING-IS-NIL *UNQUOTED-EMPTY-STRING-IS-NIL*) (QUOTED-EMPTY-STRING-IS-NIL *QUOTED-EMPTY-STRING-IS-NIL*) (TRIM-OUTER-WHITESPACE *TRIM-OUTER-WHITESPACE*) (NEWLINE *READ-NEWLINE*) (ESCAPE-MODE *ESCAPE-MODE*)

Package

cl-csv

Source

csv.lisp (file)

Function: read-csv-sample STREAM-OR-STRING SAMPLE-SIZE &key ROW-FN MAP-FN SKIP-FIRST-P (SEPARATOR *SEPARATOR*) (QUOTE *QUOTE*) (ESCAPE *QUOTE-ESCAPE*) (UNQUOTED-EMPTY-STRING-IS-NIL *UNQUOTED-EMPTY-STRING-IS-NIL*) (QUOTED-EMPTY-STRING-IS-NIL *QUOTED-EMPTY-STRING-IS-NIL*) (TRIM-OUTER-WHITESPACE *TRIM-OUTER-WHITESPACE*) (NEWLINE *READ-NEWLINE*)

Package

cl-csv

Source

csv.lisp (file)

Function: write-csv ROWS-OF-ITEMS &key STREAM (SEPARATOR *SEPARATOR*) (QUOTE *QUOTE*) (ESCAPE *QUOTE-ESCAPE*) (NEWLINE *WRITE-NEWLINE*) (ALWAYS-QUOTE *ALWAYS-QUOTE*)

Writes a csv to the given stream.

rows-of-items: iterable

Keywords:
stream: stream to write to. Default: nil.
nil - writes the rows to a string and returns it an open stream
a pathname (overwrites if the file exists)
quote: quoting character. Defaults to *quote* escape: escaping character. Defaults to *quote-escape* newline: newline character. Defaults to *write-newline* always-quote: Defaults to *always-quote*

Package

cl-csv

Source

csv.lisp (file)

Function: write-csv-row ITEMS &key STREAM (SEPARATOR *SEPARATOR*) (QUOTE *QUOTE*) (ESCAPE *QUOTE-ESCAPE*) (NEWLINE *WRITE-NEWLINE*) (ALWAYS-QUOTE *ALWAYS-QUOTE*)

Writes a list items to stream

rows-of-items: iterable

Keywords:

stream: stream to write to. Default: nil.

quote: quoting character. Defaults to *quote*

escape: escaping character. Defaults to *quote-escape*

newline: newline character. Defaults to *write-newline*

always-quote: Defaults to *always-quote*

Package

cl-csv

Source

csv.lisp (file)

5.1.4 Generic functions

Generic Function: data CONDITION

Generic Function: (setf data) NEW-VALUE CONDITION

Package

cl-csv

Methods

Method: data (CONDITION csv-data-read)

Method: (setf data) NEW-VALUE (CONDITION csv-data-read)

Source

csv.lisp (file)

Generic Function: format-csv-value VAL

Print values in ways that are most cross compatible with the csv format

Package

cl-csv

Source

csv.lisp (file)

Methods

Method: format-csv-value VAL

Generic Function: row CONDITION

Generic Function: (setf row) NEW-VALUE CONDITION

Package

cl-csv

Methods

Method: row (CONDITION csv-row-read)

Method: (setf row) NEW-VALUE (CONDITION csv-row-read)

Source

csv.lisp (file)

Generic Function: write-csv-value VAL CSV-STREAM &key FORMATTER QUOTE SEPARATOR ESCAPE ALWAYS-QUOTE NEWLINE

Writes val to csv-stream in a formatted fashion.

Keywords

formatter: used to format val. Defaults to format-csv-value.

quote: quoting character. Defaults to *quote*

escape: escaping character. Defaults to *quote-escape*

newline: newline character. Defaults to *write-newline*

always-quote: Defaults to *always-quote*

Package

cl-csv

Source

csv.lisp (file)

Methods

Method: write-csv-value VAL CSV-STREAM &key FORMATTER QUOTE SEPARATOR ESCAPE ALWAYS-QUOTE NEWLINE &aux ESCAPE FORMATTED-VALUE SHOULD-QUOTE

5.1.5 Conditions

Condition: csv-data-read ()

Package

cl-csv

Source

csv.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• csv-reader (method)
• csv-reader (method)
• data (method)
• data (method)

Direct slots

Slot: data

Initargs

:data

Initform

(quote nil)

Readers

data (generic function)

Writers

(setf data) (generic function)

Slot: csv-reader

Initargs

:csv-reader

Initform

(quote nil)

Readers

csv-reader (generic function)

Writers

(setf csv-reader) (generic function)

Condition: csv-parse-error ()

Package

cl-csv

Source

csv.lisp (file)

Direct superclasses

error (condition)

Direct methods

• format-args (method)
• format-args (method)
• format-control (method)
• format-control (method)

Direct slots

Slot: format-control

Initargs

:format-control

Initform

(quote nil)

Readers

format-control (generic function)

Writers

(setf format-control) (generic function)

Slot: format-args

Initargs

:format-args

Initform

(quote nil)

Readers

format-args (generic function)

Writers

(setf format-args) (generic function)

Condition: csv-row-read ()

Package

cl-csv

Source

csv.lisp (file)

Direct superclasses

condition (condition)

Direct methods

• csv-reader (method)
• csv-reader (method)
• row (method)
• row (method)

Direct slots

Slot: row

Initargs

:row

Initform

(quote nil)

Readers

row (generic function)

Writers

(setf row) (generic function)

Slot: csv-reader

Initargs

:csv-reader

Initform

(quote nil)

Readers

csv-reader (generic function)

Writers

(setf csv-reader) (generic function)

5.2 Internal definitions

5.2.1 Special variables

Special Variable: *always-quote*

Default setting for always quoting

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *buffer-size*

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *eof-char*

The char we use for eof

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *escape-mode*

Controls how escapes are handled.
:quote - replace the entire *quote-escape* sequence with the quote character whenever we find it. Commonly used with "" quote escapes

:following - replace the escape character and the following character with just the following character.
EG: (*quote-escape* #\ )
\ ->
r -> r
’ -> ’

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *quoted-empty-string-is-nil*

Should empty string values, be nil or "".
Unquoted values are always trimmed of surrounding whitespace. Quoted values are never be trimmed

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *read-newline*

Default newline string for reading.
We trim extra whitespace by default *trim-outer-whitespace*

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *trim-outer-whitespace*

Should white space between delimiters and data or quotes be removed

These underscores (if they were spaces) are the locations in question ’a’,_b_,_’ c ’_,_d

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *unquoted-empty-string-is-nil*

Should unquoted empty string values, be nil or "".

Package

cl-csv

Source

vars.lisp (file)

Special Variable: *write-newline*

When writing what should the newline convention be

Package

cl-csv

Source

vars.lisp (file)

5.2.2 Macros

Macro: clause-for-in-csv-1 &key (FOR VAR) (IN-CSV INPUT) (SKIPPING-HEADER SKIP-FIRST-P) (SEPARATOR SEPARATOR) (QUOTE QUOTE) (ESCAPED-QUOTE ESCAPED-QUOTE)

in-csv driver for iterate

Package

cl-csv

Source

csv.lisp (file)

Macro: clause-sampling-2 &key (SAMPLING EXPR) (INTO VAR) (SIZE SIZE)

resevoir sample the input

Package

cl-csv

Source

csv.lisp (file)

Macro: with-csv-input-stream (NAME INP) &body BODY

Package

cl-csv

Source

csv.lisp (file)

Macro: with-csv-output-stream (NAME INP) &body BODY

Package

cl-csv

Source

csv.lisp (file)

5.2.3 Functions

Function: %char-in C TO-CHECK

Package

cl-csv

Source

csv.lisp (file)

Function: %escape-is-double-quote &aux X

Package

cl-csv

Source

parser.lisp (file)

Function: %escape-seq? S I ESCAPE LLEN ELEN

Package

cl-csv

Source

csv.lisp (file)

Function: %in-stream STREAM-OR-STRING

Package

cl-csv

Source

csv.lisp (file)

Function: %next-char READER

Package

cl-csv

Source

parser.lisp (file)

Function: %out-stream STREAM-OR-STRING

creates a stream from the given thing, trying to DWIM

Package

cl-csv

Source

csv.lisp (file)

Function: %trim-datum CSV-READER &aux B

Package

cl-csv

Source

parser.lisp (file)

Function: chars-in CHARS-TO-CHECK VALUE-TO-LOOK-THROUGH

returns true if any of the chars-to-check is found in the value-to-look-through

Package

cl-csv

Source

csv.lisp (file)

Function: check-and-distpatch TABLE C

Check all the entries in a read-dispatch-table to find a match
if it matches, call the function with the table character and entry

Package

cl-csv

Source

parser.lisp (file)

Function: collect-datum CSV-READER &aux DATA-MAP-FN

Package

cl-csv

Source

parser.lisp (file)

Function: collect-row-data CSV-READER &aux MAP-FN ROW-FN

Package

cl-csv

Source

parser.lisp (file)

Function: drop-delimiter-chars TABLE ENTRY

This backs up the buffer till the delimiter is not in it
we call this without having adding the character we just got that dispatched

Package

cl-csv

Source

parser.lisp (file)

Function: last-item BUFF &key N

Function: (setf last-item) NEW BUFF

Package

cl-csv

Source

parser.lisp (file)

Function: make-default-csv-reader ()

Creates the default csv dispatch table
This can usually be fully changed simply by tweaking the special variables
defined in vars. You will need to reinstantiate this object when you change those variables (which is what happens by default)

Package

cl-csv

Source

parser.lisp (file)

Function: make-table-entry DELIMITER DISPATCH &key CLASS

Creates a table entry ensuring everything has the correct types and values

Package

cl-csv

Source

parser.lisp (file)

Function: map-empty-string-to-nil DATA &key CSV-READER &allow-other-keys

Package

cl-csv

Source

parser.lisp (file)

Function: read-csv-row-with-reader STREAM-OR-STRING &key CSV-READER MAP-FN DATA-MAP-FN &allow-other-keys

Read a row of csv from the input

Package

cl-csv

Source

parser.lisp (file)

Function: read-csv-with-reader STREAM-OR-STRING &key CSV-READER ROW-FN MAP-FN DATA-MAP-FN SKIP-FIRST-P &allow-other-keys

Read a whole csv from the input

Package

cl-csv

Source

parser.lisp (file)

Function: read-into-buffer-until ()

This reads into a buffer until either the buffer is full or the
we have read the newline character(s).

If we read the newline characters they will be the last character(s) in the buffer

Package

cl-csv

Source

read-until.lisp (file)

Function: read-with-dispatch-table TABLE STREAM &aux READ-CNT

A generic function for processing all the characters of a stream until a match arises and collecting that data as it goes

Package

cl-csv

Source

parser.lisp (file)

Function: reading-character CSV-READER C &key TABLE-ENTRY

We read a random character that was not otherwise dispatched on

Package

cl-csv

Source

parser.lisp (file)

Function: reset-table-entry TE

resets the entry state when it doesnt match

Package

cl-csv

Source

parser.lisp (file)

Function: restartable-read-row STREAM CSV-READER

Package

cl-csv

Source

csv.lisp (file)

Function: white-space? C

Package

cl-csv

Source

csv.lisp (file)

5.2.4 Generic functions

Generic Function: after-quoted? OBJECT

Generic Function: (setf after-quoted?) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: after-quoted? (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf after-quoted?) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: buffer OBJECT

Generic Function: (setf buffer) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: buffer (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf buffer) NEW-VALUE (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: character-idx OBJECT

Generic Function: (setf character-idx) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: character-idx (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf character-idx) NEW-VALUE (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: character-line-idx OBJECT

Generic Function: (setf character-line-idx) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: character-line-idx (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf character-line-idx) NEW-VALUE (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: check-table-entry TABLE ENTRY C

Package

cl-csv

Methods

Method: check-table-entry TABLE ENTRY C

Given the next character in a stream check if the table entry matches reset if it matches fully or doesnt match

Source

parser.lisp (file)

Generic Function: csv-reader CONDITION

Generic Function: (setf csv-reader) NEW-VALUE CONDITION

Package

cl-csv

Methods

Method: csv-reader (CONDITION csv-row-read)

Method: (setf csv-reader) NEW-VALUE (CONDITION csv-row-read)

Source

csv.lisp (file)

Method: csv-reader (CONDITION csv-data-read)

Method: (setf csv-reader) NEW-VALUE (CONDITION csv-data-read)

Source

csv.lisp (file)

Generic Function: data-map-fn OBJECT

Generic Function: (setf data-map-fn) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: data-map-fn (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf data-map-fn) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: delimiter OBJECT

Generic Function: (setf delimiter) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: delimiter (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf delimiter) NEW-VALUE (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: didx OBJECT

Generic Function: (setf didx) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: didx (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf didx) NEW-VALUE (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: dispatch OBJECT

Generic Function: (setf dispatch) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: dispatch (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf dispatch) NEW-VALUE (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: dlen OBJECT

Generic Function: (setf dlen) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: dlen (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf dlen) NEW-VALUE (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: dlen-1 OBJECT

Generic Function: (setf dlen-1) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: dlen-1 (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf dlen-1) NEW-VALUE (READ-DISPATCH-TABLE-ENTRY read-dispatch-table-entry)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: entries OBJECT

Generic Function: (setf entries) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: entries (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf entries) NEW-VALUE (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: format-args CONDITION

Generic Function: (setf format-args) NEW-VALUE CONDITION

Package

cl-csv

Methods

Method: format-args (CONDITION csv-parse-error)

Method: (setf format-args) NEW-VALUE (CONDITION csv-parse-error)

Source

csv.lisp (file)

Generic Function: format-control CONDITION

Generic Function: (setf format-control) NEW-VALUE CONDITION

Package

cl-csv

Methods

Method: format-control (CONDITION csv-parse-error)

Method: (setf format-control) NEW-VALUE (CONDITION csv-parse-error)

Source

csv.lisp (file)

Generic Function: line-data OBJECT

Generic Function: (setf line-data) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: line-data (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf line-data) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: line-idx OBJECT

Generic Function: (setf line-idx) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: line-idx (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf line-idx) NEW-VALUE (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: map-fn OBJECT

Generic Function: (setf map-fn) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: map-fn (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf map-fn) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: parse-stream OBJECT

Generic Function: (setf parse-stream) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: parse-stream (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf parse-stream) NEW-VALUE (READ-DISPATCH-TABLE read-dispatch-table)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: reading-escaped CSV-READER C &key TABLE-ENTRY

Package

cl-csv

Methods

Method: reading-escaped CSV-READER C &key TABLE-ENTRY

We read an escape sequence and need to handle storing the escaped character

Source

parser.lisp (file)

Generic Function: reading-following-escaped CSV-READER C &key TABLE-ENTRY

Package

cl-csv

Methods

Method: reading-following-escaped CSV-READER C &key TABLE-ENTRY

We read an escape sequence and need to handle storing the escaped character

Source

parser.lisp (file)

Generic Function: reading-newline CSV-READER C &key TABLE-ENTRY

Package

cl-csv

Methods

Method: reading-newline CSV-READER C &key TABLE-ENTRY

We got the newline character which will be handled differently based on if we are in quoted data or not

Source

parser.lisp (file)

Generic Function: reading-quoted CSV-READER C &key TABLE-ENTRY

Package

cl-csv

Methods

Method: reading-quoted CSV-READER C &key TABLE-ENTRY

Method to handle reading a quote
NB: this interacts wierdly with escape-mode :quote

Source

parser.lisp (file)

Generic Function: reading-quoted-or-escaped CSV-READER C &key TABLE-ENTRY

Package

cl-csv

Methods

Method: reading-quoted-or-escaped CSV-READER C &key TABLE-ENTRY

Method to handle reading a quote or a pair of quotes

Source

parser.lisp (file)

Generic Function: reading-quoted? OBJECT

Generic Function: (setf reading-quoted?) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: reading-quoted? (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf reading-quoted?) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: reading-separator CSV-READER C &key TABLE-ENTRY

Package

cl-csv

Methods

Method: reading-separator CSV-READER C &key TABLE-ENTRY

We got the data separator character which will be handled differently based on if we are in quoted data or not

Source

parser.lisp (file)

Generic Function: row-fn OBJECT

Generic Function: (setf row-fn) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: row-fn (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf row-fn) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: rows OBJECT

Generic Function: (setf rows) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: rows (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf rows) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

Generic Function: skip-row? OBJECT

Generic Function: (setf skip-row?) NEW-VALUE OBJECT

Package

cl-csv

Methods

Method: skip-row? (CSV-READER csv-reader)

automatically generated reader method

Source

parser.lisp (file)

Method: (setf skip-row?) NEW-VALUE (CSV-READER csv-reader)

automatically generated writer method

Source

parser.lisp (file)

5.2.5 Classes

Class: csv-reader ()

the state of the csv reader, which is also is a read table

Package

cl-csv

Source

parser.lisp (file)

Direct superclasses

read-dispatch-table (class)

Direct methods

• skip-row? (method)
• skip-row? (method)
• data-map-fn (method)
• data-map-fn (method)
• map-fn (method)
• map-fn (method)
• row-fn (method)
• row-fn (method)
• after-quoted? (method)
• after-quoted? (method)
• reading-quoted? (method)
• reading-quoted? (method)
• line-data (method)
• line-data (method)
• rows (method)
• rows (method)

Direct slots

Slot: rows

Initargs

cl-csv::rows

Initform

(make-array 10 :element-type (quote list) :initial-element nil :adjustable t :fill-pointer 0)

Readers

rows (generic function)

Writers

(setf rows) (generic function)

Slot: line-data

Initform

(make-array 10 :element-type (quote string) :initial-element "" :adjustable t :fill-pointer 0)

Readers

line-data (generic function)

Writers

(setf line-data) (generic function)

Slot: reading-quoted?

Readers

reading-quoted? (generic function)

Writers

(setf reading-quoted?) (generic function)

Slot: after-quoted?

Readers

after-quoted? (generic function)

Writers

(setf after-quoted?) (generic function)

Slot: row-fn

Initargs

:row-fn

Readers

row-fn (generic function)

Writers

(setf row-fn) (generic function)

Slot: map-fn

Initargs

:map-fn

Readers

map-fn (generic function)

Writers

(setf map-fn) (generic function)

Slot: data-map-fn

Initargs

:data-map-fn

Initform

(quote cl-csv::map-empty-string-to-nil)

Readers

data-map-fn (generic function)

Writers

(setf data-map-fn) (generic function)

Slot: skip-row?

Initargs

:skip-row?

Readers

skip-row? (generic function)

Writers

(setf skip-row?) (generic function)

Class: read-dispatch-table ()

A stream parser that collects characters
and when a certain delimiter is matched will call a certain function. These delimiter / function pairs are read-dispatch table entries It contains all the state for the parse process
See: csv-reader

Package

cl-csv

Source

parser.lisp (file)

Direct superclasses

standard-object (class)

Direct subclasses

csv-reader (class)

Direct methods

• print-object (method)
• character-idx (method)
• character-idx (method)
• character-line-idx (method)
• character-line-idx (method)
• line-idx (method)
• line-idx (method)
• entries (method)
• entries (method)
• buffer (method)
• buffer (method)
• parse-stream (method)
• parse-stream (method)

Direct slots

Slot: parse-stream

Initargs

:parse-stream

Readers

parse-stream (generic function)

Writers

(setf parse-stream) (generic function)

Slot: buffer

Initargs

:buffer

Initform

(make-array cl-csv::*buffer-size* :element-type (quote character) :initial-element #\nul :adjustable t :fill-pointer 0)

Readers

buffer (generic function)

Writers

(setf buffer) (generic function)

Slot: entries

Initargs

:entries

Readers

entries (generic function)

Writers

(setf entries) (generic function)

Slot: line-idx

Initargs

:line-idx

Initform

0

Readers

line-idx (generic function)

Writers

(setf line-idx) (generic function)

Slot: character-line-idx

Initargs

:character-line-idx

Initform

0

Readers

character-line-idx (generic function)

Writers

(setf character-line-idx) (generic function)

Slot: character-idx

Initargs

:character-idx

Initform

0

Readers

character-idx (generic function)

Writers

(setf character-idx) (generic function)

Class: read-dispatch-table-entry ()

When a certain delimiter is matched it will call a certain function T matches anything
create these with make-table-entry

Package

cl-csv

Source

parser.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• dispatch (method)
• dispatch (method)
• dlen-1 (method)
• dlen-1 (method)
• dlen (method)
• dlen (method)
• didx (method)
• didx (method)
• delimiter (method)
• delimiter (method)

Direct slots

Slot: delimiter

Type

(vector (or boolean character))

Initargs

:delimiter

Readers

delimiter (generic function)

Writers

(setf delimiter) (generic function)

Slot: didx

Type

fixnum

Initargs

:didx

Initform

-1

Readers

didx (generic function)

Writers

(setf didx) (generic function)

Slot: dlen

Type

fixnum

Initargs

:dlen

Initform

0

Readers

dlen (generic function)

Writers

(setf dlen) (generic function)

Slot: dlen-1

Type

fixnum

Initargs

:dlen-1

Initform

-1

Readers

dlen-1 (generic function)

Writers

(setf dlen-1) (generic function)

Slot: dispatch

Type

function

Initargs

:dispatch

Readers

dispatch (generic function)

Writers

(setf dispatch) (generic function)

Appendix A Indexes

A.1 Concepts

A.2 Functions