This is the transducers Reference Manual, version 1.0.1, generated automatically by Declt version 4.0 beta 2 "William Riker" on Sun Dec 15 05:28:25 2024 GMT+0.
The main system appears first, followed by any subsystem dependency.
transducersErgonomic, efficient data processing.
Colin Woodbury <colin@fosskers.ca>
LGPL-3.0-only
1.0.1
sycamore (system).
transducers (module).
Modules are listed depth-first from the system components tree.
transducers/transducerstransducers (system).
transducers.lisp (file).
reducers.lisp (file).
sources.lisp (file).
entry.lisp (file).
conditions.lisp (file).
utils.lisp (file).
Files are sorted by type and then listed depth-first from the systems components trees.
transducers/transducers.asdtransducers/transducers/transducers.lisptransducers/transducers/reducers.lisptransducers/transducers/sources.lisptransducers/transducers/entry.lisptransducers/transducers/conditions.lisptransducers/transducers/utils.lisptransducers/transducers/transducers.lisptransducers (module).
branch (function).
concatenate (function).
dedup (function).
drop (function).
drop-while (function).
enumerate (function).
filter (function).
filter-map (function).
flatten (function).
from-csv (function).
group-by (function).
inject (function).
intersperse (function).
into-csv (function).
log (function).
make-reduced (function).
map (function).
once (function).
pass (function).
reduced-p (function).
reduced-val (reader).
(setf reduced-val) (writer).
scan (function).
segment (function).
split (function).
step (function).
take (function).
take-while (function).
uncons (function).
unique (function).
window (function).
copy-reduced (function).
par (function).
recsv (function).
reduced (structure).
split-csv-line (function).
table-vals->csv (function).
tri (function).
transducers/transducers/reducers.lisptransducers (module).
all (macro).
allp (function).
any (macro).
anyp (function).
average (function).
cons (function).
count (function).
find (function).
first (function).
fold (function).
for-each (function).
hash-table (function).
last (function).
max (function).
min (function).
snoc (function).
string (function).
vector (function).
transducers/transducers/sources.lisptransducers (module).
*done* (special variable).
generator (structure).
generator-func (reader).
make-generator (function).
make-plist (function).
plist-list (reader).
unfold (function).
transducers/transducers/entry.lisptransducers (module).
transduce (generic function).
file-reduce (function).
file-transduce (function).
generator-reduce (function).
generator-transduce (function).
hash-table-reduce (function).
hash-table-transduce (function).
list-reduce (function).
list-transduce (function).
plist-reduce (function).
plist-transduce (function).
stream-reduce (function).
stream-transduce (function).
string-transduce (function).
vector-reduce (function).
vector-transduce (function).
transducers/transducers/conditions.lisptransducers (module).
empty-transduction (condition).
imbalanced-plist (condition).
empty-argument (condition).
empty-argument-fn (reader method).
empty-transduction-msg (reader method).
imbalanced-plist-key (reader method).
non-positive-integer (condition).
npi-fn (reader method).
npi-n (reader method).
prompt-new-value (function).
transducers/transducers/utils.lisptransducers (module).
ensure-function (function).
ensure-reduced (function).
ensure-unreduced (function).
preserving-reduced (function).
zipmap (function).
Packages are listed by definition order.
transducersErgonomic, efficient data processing.
common-lisp.
all (macro).
allp (function).
any (macro).
anyp (function).
average (function).
branch (function).
comp (function).
concatenate (function).
cons (function).
const (function).
count (function).
cycle (generic function).
dedup (function).
drop (function).
drop-while (function).
empty-transduction (condition).
enumerate (function).
filter (function).
filter-map (function).
find (function).
first (function).
flatten (function).
fold (function).
for-each (function).
from-csv (function).
group-by (function).
hash-table (function).
imbalanced-plist (condition).
inject (function).
intersperse (function).
into-csv (function).
ints (function).
last (function).
log (function).
make-reduced (function).
map (function).
max (function).
min (function).
once (function).
pass (function).
plist (function).
plist (structure).
random (function).
reduced-p (function).
reduced-val (reader).
(setf reduced-val) (writer).
repeat (function).
scan (function).
segment (function).
shuffle (function).
snoc (function).
split (function).
step (function).
string (function).
take (function).
take-while (function).
transduce (generic function).
uncons (function).
unique (function).
vector (function).
window (function).
*done* (special variable).
copy-reduced (function).
empty-argument (condition).
empty-argument-fn (generic reader).
empty-transduction-msg (generic reader).
ensure-function (function).
ensure-reduced (function).
ensure-unreduced (function).
file-reduce (function).
file-transduce (function).
generator (structure).
generator-func (reader).
generator-reduce (function).
generator-transduce (function).
hash-table-reduce (function).
hash-table-transduce (function).
imbalanced-plist-key (generic reader).
list-reduce (function).
list-transduce (function).
make-generator (function).
make-plist (function).
non-positive-integer (condition).
npi-fn (generic reader).
npi-n (generic reader).
par (function).
plist-list (reader).
plist-reduce (function).
plist-transduce (function).
preserving-reduced (function).
prompt-new-value (function).
recsv (function).
reduced (structure).
split-csv-line (function).
stream-reduce (function).
stream-transduce (function).
string-transduce (function).
table-vals->csv (function).
tri (function).
unfold (function).
vector-reduce (function).
vector-transduce (function).
zipmap (function).
Definitions are sorted by export status, category, package, and then by lexicographic order.
Deprecated: Use ‘allp’.
Deprecated: Use ‘anyp’.
Reducer: Yield non-NIL if all elements of the transduction satisfy PRED. Short-circuits with NIL if any element fails the test.
Reducer: Yield non-NIL if any element in the transduction satisfies PRED. Short-circuits the transduction as soon as the condition is met.
Reducer: Calculate the average value of all numeric elements in a transduction.
Transducer: If a PRED yields non-NIL on a value, proceed with transducer chain
TA. Otherwise, follow chain TB. This produces a kind of diamond pattern of data
flow within the transduction:
/4a-5a-6a\
1-2-3 7-8-9
\4b-5b—/
Assuming that TA here is some composition of three transducer steps, and TB is a
composition of two. Naturally, if you have other steps beyond the fork (Step 7
above), you should make sure that they can handle the return values of both
sides!
(transduce (comp (map #’1+)
(branch #’evenp
(map (comp #’write-to-string #’1+))
(map (const "Odd!")))
(map #’length))
#’cons (range 1 6))
=> (1 4 1 4 1)
Function composition.
(funcall (comp #’1+ #’length) "foo") == (1+ (length "foo"))
Transducer: Concatenate all the sublists in the transduction.
Reducer: Collect all results as a list.
Return a function that ignores its argument and returns ITEM instead.
Reducer: Count the number of elements that made it through the transduction.
Transducer: Remove adjacent duplicates from the transduction.
Transducer: Drop the first N elements of the transduction.
Transducer: Drop elements from the front of the transduction that satisfy PRED.
Transducer: Index every value passed through the transduction into a cons pair. Starts at 0.
Transducer: Only keep elements from the transduction that satisfy PRED.
Transducer: Apply a function F to the elements of the transduction, but only
keep results that are non-nil.
(transduce (filter-map #’cl:first) #’cons ’(() (2 3) () (5 6) () (8 9))) => (2 5 8)
Reducer: Find the first element in the transduction that satisfies a given PRED. Yields ‘nil’ if no such element were found.
Reducer: Yield the first value of the transduction. As soon as this first value
is yielded, the entire transduction stops.
# Conditions
- ‘empty-transduction’: when no values made it through the transduction.
Transducer: Entirely flatten all lists in the transduction, regardless of nesting.
Reducer: The fundamental reducer. ‘fold’ creates an ad-hoc reducer based on
a given 2-argument function. An optional SEED value can also be given as the
initial accumulator value, which also becomes the return value in case there
were no input left in the transduction.
Functions like ‘+’ and ‘*’ are automatically valid reducers, because they yield
sane values even when given 0 or 1 arguments. Other functions like ‘max’ cannot
be used as-is as reducers since they can’t be called without arguments. For
functions like this, ‘fold’ is appropriate.
# Conditions
- ‘empty-transduction’: if no SEED is given and the transduction is empty.
Reducer: Run through every item in a transduction for their side effects. Throws away all results and yields t.
Transducer: Interpret the data stream as CSV data.
The first item found is assumed to be the header list, and it will be used to
construct useable hashtables for all subsequent items.
Note: This function makes no attempt to convert types from the
original parsed strings. If you want numbers, you will need to
further parse them yourself.
This function is expected to be passed "bare" to ‘transduce’, so there is no need for the caller to manually pass a REDUCER.
Transducer: Group the input stream into sublists via some function F. The cutoff
criterion is whether the return value of F changes between two consecutive elements of the
transduction.
(transduce (group-by #’evenp) #’cons ’(2 4 6 7 9 1 2 4 6 3))
=> ((2 4 6) (7 9 1) (2 4 6) (3))
Reducer: Collect a stream of key-value cons pairs into a hash table.
Transducer: For each value in the transduction that actually affects the final result (tested with ‘EQ’), inject an extra transduction step into the chain immediately after this point. Accumulates, such that each new injection appears before the previous one.
Transducer: Insert an ELEM between each value of the transduction.
Transducer: Given a sequence of HEADERS, rerender each item in the data stream
into a CSV string. It’s assumed that each item in the transduction is a hash
table whose keys are strings that match the values found in HEADERS.
# Conditions
- ‘empty-argument’: when an empty HEADERS sequence is given.
Source: Yield all integers, beginning with START and advancing by an optional STEP value which can be positive or negative. If you only want a specific range within the transduction, then use ‘take-while’ within your transducer chain.
Reducer: Yield the last value of the transduction.
# Conditions
- ‘empty-transduction’: when no values made it through the transduction.
Transducer: Call some LOGGER function for each step of the transduction. The LOGGER must accept the running results and the current element as input. The original results of the transduction are passed through as-is.
Transducer: Apply a function F to all elements of the transduction.
Deprecated: Use ‘fold’ and pass it ‘cl:max’ instead.
Deprecated: Use ‘fold’ and pass it ‘cl:min’ instead.
Transducer: Inject some ITEM into the front of the transduction.
Transducer: Just pass along each value of the transduction. Same in intent with applying ‘map’ to ‘identity’, but this should be slightly more efficient. It is at least shorter to type.
Source: Yield key-value pairs from a Property List, usually known as a ’plist’.
Source: Yield an endless stream of random numbers.
val.
Source: Endlessly yield a given ITEM.
Transducer: Build up successsive values from the results of previous
applications of a given function F.
(transduce (scan #’+ 0) #’cons ’(1 2 3 4))
=> (0 1 3 6 10)
Transducer: Partition the input into lists of N items. If the input stops, flush
any accumulated state, which may be shorter than N.
# Conditions
- ‘non-positive-integer’: when a non-positive integer N is given.
Source: Endlessly yield random elements from a given vector. Recall also that
strings are vectors too, so:
(transduce (take 5) #’string (shuffle "Númenor"))
=> "mNNrú"
Reducer: Collect all results as a list, but results are reversed.
In theory, slightly more performant than ‘cons’ since it performs no final
reversal.
Transducer: Split off a new transducer chain, feeding it each input as well. It reduces on its own given RA reducer. The final result is a cons-cell where the first value is the result of the original transduction, and the second is that of the branch.
Transducer: Only yield every Nth element of the transduction. The first element
of the transduction is always included.
# Conditions
- ‘non-positive-integer’: when a non-positive integer N is given.
# Examples
(transduce (step 2) #’cons ’(1 2 3 4 5 6 7 8 9))
=> (1 3 5 7 9)
Reducer: Collect a stream of characters into to a single string.
Transducer: Keep only the first N elements of the transduction.
Transducer: Keep only elements which satisfy a given PRED, and stop the transduction as soon as any element fails the test.
Transducer: Split up a transduction of cons cells.
Transducer: Only allow values to pass through the transduction once each. Stateful; this uses a hash table internally so could get quite heavy if you’re not careful.
Reducer: Collect a stream of values into a vector.
Transducer: Yield N-length windows of overlapping values. This is different from
‘segment’ which yields non-overlapping windows. If there were fewer items in the
input than N, then this yields nothing.
# Conditions
- ‘non-positive-integer’: when a non-positive integer N is given.
Source: Yield the values of a given SEQ endlessly.
The entry-point for processing some data source via transductions.
This requires three things:
- A transducer function, or a composed chain of them
- A reducing function
- A source
Note: ‘comp’ can be used to chain transducers together.
When ran, ‘transduce’ will pull values from the source, transform them via the
transducers, and reduce into some single value (likely some collection but not
necessarily). ‘transduce’ will only pull as many values from the source as are
actually needed, and does so one at a time. This ensures that large
sources (like files) don’t consume too much memory.
# Examples
Assuming that you’ve required this library with a local nickname of ‘t’, here’s
how we can filter an infinite source and reduce into a single sum:
(t:transduce (t:comp (t:filter #’oddp)
(t:take 1000)
(t:map (lambda (n) (* n n))))
#’+ (t:ints 1))
;; => 1333333000 (31 bits, #x4F790C08)
Note that due to how transducer and reducer functions are composed internally,
the order provided to ‘comp’ gets applied from top to bottom. In the above
example, this means that ‘filter’ is applied first, and ‘map’ last.
There are a variety of functions to instead reduce into a collection:
(t:transduce (t:map #’1+) #’t:vector ’(1 2 3))
;; => #(2 3 4)
Many standard collections can be easily "sourced", including those that aren’t
normally so conveniently traversed like Hash Tables, Property Lists, and lines
of a file.
;; Read key-value pairs from a plist and recollect into a Hash Table.
(t:transduce #’t:pass #’t:hash-table (t:plist ‘(:a 1 :b 2 :c 3)))
# Custom Sources
Since ‘transduce’ is generic, you can use ‘defmethod’ to define your own custom sources. See ‘sources.lisp’ and ‘entry.lisp’ for examples of how to do this.
plist)) ¶Yields key-value pairs as cons cells.
# Conditions
- ‘imbalanced-pist’: if the number of keys and values do not match.
stream)) ¶pathname)) ¶hash-table)) ¶Yields key-value pairs as cons cells.
vector)) ¶list)) ¶Transducing over an alist works automatically via this method, and the pairs are streamed as-is as cons cells.
string)) ¶A transduction was empty when it was expected not to be.
error.
:msg
This slot is read-only.
A given ‘plist’ source had an uneven number of keys.
error.
:key
This slot is read-only.
A value to signal the end of an unfolding process.
Ensure that X is reduced.
Ensure that X is unreduced.
Transduce over the lines of the file named by a FILENAME.
Transduce over a potentially endless stream of values from a generator GEN.
Transduce over the contents of a given Hash Table.
Transducer: Traverse two transducer paths at the same time, combining the
results of each path with a given function F before moving on. This is similar
to the ‘zip’ concept from other languages.
Given the following transducer chain:
/4a-5a-6a\
1-2-3 7-8-9
\4b-5b—/
The function F would be applied right before Step 7. In this case, the function
would have to expect the output types of Step 6a and 5b as its arguments.
Note 1: If either branch yields a ’reduced’ value, then the entire chain short-circuits and that is the value applied to the reducer one final time.
Note 2: This function has potentially non-intuitive behaviour with regards to functions like ‘filter’ that don’t always contribute to the final result. The function F will only be applied (and thus pass values on) if both branches produced a new value. If either branch ’died’ for a particular value, then so too will the other branch. If this is undesirable, see the higher-order transducer ‘tri’ for an alternative.
A helper function that wraps a reduced value twice since reducing functions (like list-reduce) unwraps them. tconcatenate is a good example: it re-uses its reducer on its input using list-reduce. If that reduction finishes early and returns a reduced value, list-reduce would ’unreduce’ that value and try to continue the transducing process.
Reconvert some ITEMS into a comma-separated string.
Split a LINE of CSV data in a sane way.
This removes any extra whitespace that might be hanging around between elements.
Transduce over the lines of a given STREAM. Note: Closing the stream is the responsiblity of the caller!
Given some HEADERS to compare to, convert a hash TABLE to a rendered CSV string of its values.
The Trident.
Form a hashmap with the KEYS mapped to the corresponding VALS.
Borrowed from Clojure, thanks guys.
empty-argument)) ¶fn.
empty-transduction)) ¶msg.
imbalanced-plist)) ¶key.
non-positive-integer)) ¶fn.
non-positive-integer)) ¶n.
A non-empty sequence was expected, but that didn’t stop the user.
error.
:fn
This slot is read-only.
A non-positive integer was passed to a function that expected one.
error.
A wrapper around a function that can potentially yield endless values.
structure-object.
(function nil *)
This slot is read-only.
A wrapper that signals that reduction has completed.
structure-object.
| Jump to: | (
A B C D E F G H I L M N O P R S T U V W Z |
|---|
| Jump to: | (
A B C D E F G H I L M N O P R S T U V W Z |
|---|
| Jump to: | *
F K L M N S V |
|---|
| Jump to: | *
F K L M N S V |
|---|
| Jump to: | C E F G I M N P R S T U |
|---|
| Jump to: | C E F G I M N P R S T U |
|---|