Next: Introduction, Previous: (dir), Up: (dir) [Contents][Index]
This is the qbase64 Reference Manual, version 0.3.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Aug 15 05:39:23 2022 GMT+0.
Next: Systems, Previous: The qbase64 Reference Manual, Up: The qbase64 Reference Manual [Contents][Index]
qbase64 provides a fast and flexible base64 encoder and decoder for Common Lisp. It provides three interfaces for both encoding and decoding:
ENCODE-BYTES and DECODE-STRING are the easiest to use. They
allow one to encode a byte vector and decode a base64 string in one
go.
ENCODE-STREAM and DECODE-STREAM are gray stream classes that
allow one to write and read bytes from underlying character streams.
ENCODER and DECODER provide the low level interface. The other
interfaces are built on top of these.
Given that a couple of Lisp libraries already exist for working with base64 - cl-base64 and s-base64 - why might you want to pick qbase64? There are two reasons:
Stream-based APIs - neither of the alternatives provide a (gray) stream implementation where you can write bytes to a binary stream that is automatically encoded to an underlying character stream and vice-versa.
Performance - qbase64 was written with the objective of being fast while keeping memory consumption independent of the input size. See the Performance section for benchmarks and other details.
Install using quicklisp:
(ql:quickload :qbase64)
The examples below use ENCODE-BYTES and ENCODE-STREAM.
;;; ENCODE-BYTES
(qbase64:encode-bytes #(1 2 3 4 5 6 7 8))
=> "AQIDBAUGBwg="
;;; ENCODE-STREAM
(with-output-to-string (s)
(with-open-stream (out (make-instance 'qbase64:encode-stream
:underlying-stream s))
(write-sequence #(1 2 3 4) out)
(write-sequence #(5 6 7 8) out)))
=> "AQIDBAUGBwg="
The examples below use DECODE-STRING and DECODE-STREAM.
;;; DECODE-STRING
(qbase64:decode-string "AQIDBAUGBwg=")
=> #(1 2 3 4 5 6 7 8)
;;; DECODE-STREAM
(with-input-from-string (s "AQIDBAUGBwg=")
(with-open-stream (in (make-instance 'qbase64:decode-stream
:underlying-stream s))
(let ((bytes (make-array 4)))
(loop
for position = (read-sequence bytes in)
do (print (subseq bytes 0 position))
while (= position (length bytes))))))
; #(1 2 3 4)
; #(5 6 7 8)
; #()
Normally you wouldn't need to use ENCODER and DECODER directly,
but if you do (say you want more control over memory management), you
can refer to the examples below.
In these examples, fixed length sequences are used for both input and output, and any input buffered by the encoder/decoder is first cleared before further input is fed to it. This allows very tight control over how much memory gets used.
Refer to the doc strings for ENCODER, ENCODE, DECODER and
DECODE for more details.
Note that running the following examples requires FLEXI-STREAMS.
;;; ENCODER
(flexi-streams:with-input-from-sequence (in #(1 2 3 4 5 6 7 8))
(let* ((encoder (qbase64:make-encoder))
(bytes (make-array 4))
(string (make-string 5))
(read-bytes t)
(buffered nil)
(eof nil))
(loop
while (or buffered (not eof))
for end1 = (when read-bytes (read-sequence bytes in))
if (and read-bytes (< end1 (length bytes))) do (setf eof t)
do
(multiple-value-bind (end2 pending)
(if read-bytes
(qbase64:encode encoder bytes string :end1 end1 :finish eof)
(qbase64:encode encoder #() string :finish eof))
(write-string string nil :end end2)
(setf buffered pending
read-bytes (or (not pending) (zerop end2)))))))
; AQIDBAUGBwg=
;;; DECODER
(with-input-from-string (in "AQIDBAUGBwg=")
(let* ((decoder (qbase64:make-decoder))
(string (make-string 4))
(bytes (make-array 5))
(read-string t)
(buffered nil)
(eof nil))
(loop
while (or buffered (not eof))
for end1 = (when read-string (read-sequence string in))
if (and read-string (< end1 (length string))) do (setf eof t)
do
(multiple-value-bind (end2 pending)
(if read-string
(qbase64:decode decoder string bytes :end1 end1)
(qbase64:decode decoder "" bytes))
(print (subseq bytes 0 end2))
(setf buffered pending
read-string (or (not pending) (zerop end2)))))))
; #(1 2 3)
; #(4 5 6)
; #(7 8)
; #()
The library relies on (UNSIGNED-BYTE 8) and fixnum arithmetic to
achieve good performance. Consequently,
When providing bytes for encoding, ensure that each byte is of type
(UNSIGNED-BYTE 8). Although the ARRAY-ELEMENT-TYPE of the byte
array can be T, the elements themselves must conform to this
restriction.
Max length of the byte vector that is used as encoding input or
decoding output should never exceed +MAX-BYTES-LENGTH+.
Max length of the string that is used as encoding output or decoding
input should never exceed +MAX-STRING-LENGTH+.
See this page for CPU and memory benchmarks vs other CL libraries.
Encoding and decoding should be very fast under these conditions:
The byte vector is a SIMPLE-ARRAY of element type (UNSIGNED-BYTE 8).
The string is a SIMPLE-STRING. Theoretically SIMPLE-BASE-STRING
could be even faster.
That said, these are just the optimal conditions. You can safely use
any STRING or VECTOR with qbase64 if needed.
Two base64 encoding schemes are supported: original (the default) and URI.
URI encoding scheme is useful when base64 strings are used as GET or POST values in an HTTP request.
The scheme can be set by using the :SCHEME keyword.
(qbase64:encode-bytes #(251 252 253 254 255) :scheme :original)
=> "+/z9/v8="
(qbase64:encode-bytes #(251 252 253 254 255) :scheme :uri)
=> "-_z9_v8="
The encoded base64 stream can broken into multiple lines using the
:LINEBREAK keyword. By default it is 0, which means that no
newlines are output. Setting it to a positive integer indicates the
column number at which lines should be broken.
(princ (qbase64:encode-bytes #(1 2 3 4 5 6 7 8) :linebreak 4))
; AQID
; BAUG
; Bwg=
During decoding, all whitespace (including newlines) is ignored.
At the moment, API reference is available in the form of doc strings for all the exported symbols.
To report a bug in the library, create a Github issue.
Next: Files, Previous: Introduction, Up: The qbase64 Reference Manual [Contents][Index]
The main system appears first, followed by any subsystem dependency.
Fast and flexible base64 encoder and decoder
Chaitanya Gupta <mail@chaitanyagupta.com>
BSD-3-Clause
0.3.0
Next: Packages, Previous: Systems, Up: The qbase64 Reference Manual [Contents][Index]
Files are sorted by type and then listed depth-first from the systems components trees.
Next: qbase64/package.lisp, Previous: Lisp, Up: Lisp [Contents][Index]
qbase64 (system).
Next: qbase64/utils.lisp, Previous: qbase64/qbase64.asd, Up: Lisp [Contents][Index]
qbase64 (system).
Next: qbase64/stream-utils.lisp, Previous: qbase64/package.lisp, Up: Lisp [Contents][Index]
package.lisp (file).
qbase64 (system).
Next: qbase64/qbase64.lisp, Previous: qbase64/utils.lisp, Up: Lisp [Contents][Index]
utils.lisp (file).
qbase64 (system).
Previous: qbase64/stream-utils.lisp, Up: Lisp [Contents][Index]
stream-utils.lisp (file).
qbase64 (system).
Next: Definitions, Previous: Files, Up: The qbase64 Reference Manual [Contents][Index]
Packages are listed by definition order.
Next: Indexes, Previous: Packages, Up: The qbase64 Reference Manual [Contents][Index]
Definitions are sorted by export status, category, package, and then by lexicographic order.
Next: Internals, Previous: Definitions, Up: Definitions [Contents][Index]
Next: Ordinary functions, Previous: Public Interface, Up: Public Interface [Contents][Index]
Max length of the byte array that is used as encoding input or decoding output
Max length of the string that is used as encoding output or decoding input
Next: Standalone methods, Previous: Constants, Up: Public Interface [Contents][Index]
Decodes the given STRING and writes the resultant bytes to BYTES.
DECODER: The decoder
STRING: The string to decode.
BYTES: This is where the resultant bytes are written into. Should be
a single-dimentional array of (UNSIGNED-BYTE 8) elements.
START1, END1: Bounds for STRING
START2, END2: Bounds for BYTES
Whitespace in string is ignored. It is not necessary that the entire
STRING is decoded in one go. For example,
* There may not be enough space left in BYTES,
* or the length of the string (minus whitespace chars) may not be a
multiple of 4 (base64 decoding works on groups of four characters at
at time).
In these cases, DECODE will decode as much of the string as it can and
write the resultant bytes into BYTES. The remaining string is copied
to an internal buffer by the decoder and used the next time DECODE is
called. Also, the second return value (called PENDINGP, see below) is
set to true.
DECODE can be given an empty STRING in which case the buffered string
is decoded as much as possible.
Returns POSITION, PENDINGP.
POSITION: First index of BYTES that wasn’t updated
PENDINGP: True if not all of the STRING was decoded
Decodes base64 chars in STRING and returns an array
of (UNSIGNED-BYTE 8) elements.
STRING: The string to decode.
SCHEME: The base64 encoding scheme to use. Must be :ORIGINAL (default) or :URI.
Encodes given BYTES and writes the resultant chars to STRING.
ENCODER: The encoder
BYTES: Should be a single-dimentional array of (UNSIGNED-BYTE 8)
elements.
STRING: The encoded characters are written into this string.
START1, END1: Bounds for BYTES
START2, END2: Bounds for STRING
FINISH: Padding characters are output if required, and no new bytes
can be accepted until all the pending bytes are written out.
It is not necessary that all of BYTES are encoded in one go. For
example,
* There may not be enough space left in STRING
* FINISH is not true and the cumulative length of all the bytes given
till now is not a multiple of 3 (base64 encoding works on groups of
three bytes).
In these cases, as much as possible BYTES are encoded and the
resultant chars written into STRING, the remaining bytes are copied to
an internal buffer by the encoder and used the next time ENCODE is
called. Also, the second value returned (called PENDINGP, see below)
is set to true.
If FINISH is true but cumulative length of all the BYTES is not a
multiple of 3, padding characters are written into STRING.
ENCODE can be given an empty BYTES array in which case the internal
buffer is encoded as much as possible.
Returns POSITION, PENDINGP.
POSITION: First index of STRING that wasn’t updated
PENDINGP: True if not all BYTES were encoded
Encode BYTES to base64 and return the string.
BYTES: Should be a single-dimentional array of (UNSIGNED-BYTE 8)
elements.
SCHEME: The base64 encoding scheme to use. Must
be :ORIGINAL (default) or :URI.
LINEBREAK: If 0 (the default), no linebreaks are written. Otherwise its value must be the max number of characters per line.
Creates a DECODER.
SCHEME: The base64 encoding scheme to use. Can be :ORIGINAL or :URI
Creates an ENCODER.
SCHEME: The base64 encoding scheme to use. Can be :ORIGINAL or :URI
Next: Structures, Previous: Ordinary functions, Up: Public Interface [Contents][Index]
sb-gray.
sb-gray.
sb-gray.
trivial-gray-streams.
sb-gray.
trivial-gray-streams.
Next: Classes, Previous: Standalone methods, Up: Public Interface [Contents][Index]
Use a DECODER to decode base64 characters to bytes. Use MAKE-DECODER to create a decoder, then decode base64 chars using DECODE.
Use an ENCODER to encode bytes to string. Create an encoder using MAKE-ENCODER, then start encoding bytes using ENCODE.
structure-object.
qbase64::scheme
:original
(simple-array (unsigned-byte 8))
qbase64::+empty-bytes+
qbase64::positive-fixnum
0
Previous: Structures, Up: Public Interface [Contents][Index]
A binary input stream that converts base64 chars from an
underlying stream to bytes.
Create a DECODE-STREAM using MAKE-INSTANCE. The following
initialization keywords are provided:
UNDERLYING-STREAM: The underlying character input stream from which
base64 chars are read. Must be given.
SCHEME: The base64 encoding scheme to use. Must
be :ORIGINAL (default) or :URI.
Note that DECODE-STREAM does not close the underlying stream when CLOSE is invoked.
:underlying-stream
common-lisp.
qbase64::+empty-string+
(qbase64::make-byte-vector 3)
0
(qbase64::make-byte-vector 1)
A binary output stream that converts bytes to base64 characters
and writes them to an underlyihng character output stream.
Create an ENCODE-STREAM using MAKE-INSTANCE. The following
initialization keywords are provided:
UNDERLYING-STREAM: The underlying character output stream to which
base64 characters are written. Must be given.
SCHEME: The base64 encoding scheme to use. Must
be :ORIGINAL (default) or :URI.
LINEBREAK: If 0 (the default), no linebreaks are written. Otherwise
its value must be the max number of characters per line.
Note that ENCODE-STREAM does not close the underlying stream when CLOSE is invoked.
:underlying-stream
common-lisp.
qbase64::+empty-string+
(qbase64::make-byte-vector 1)
0
:linebreak
0
Previous: Public Interface, Up: Definitions [Contents][Index]
Next: Ordinary functions, Previous: Constants, Up: Internals [Contents][Index]
Next: Generic functions, Previous: Macros, Up: Internals [Contents][Index]
Parses BODY into (values remaining-forms declarations doc-string). Documentation strings are recognized only if DOCUMENTATION is true. Syntax errors in body are signalled and WHOLE is used in the signal arguments when given.
Returns T for a whitespace character.
Next: Classes, Previous: Ordinary functions, Up: Internals [Contents][Index]
automatically generated reader method
automatically generated writer method
Next: Types, Previous: Generic functions, Up: Internals [Contents][Index]
t
Previous: Definitions, Up: The qbase64 Reference Manual [Contents][Index]
| Jump to: | %
(
B C D E F G I L M O P R S W |
|---|
| Jump to: | %
(
B C D E F G I L M O P R S W |
|---|
Next: Data types, Previous: Functions, Up: Indexes [Contents][Index]
| Jump to: | +
B C D E F L O P S U |
|---|
| Jump to: | +
B C D E F L O P S U |
|---|
| Jump to: | C D E F P Q S T U |
|---|
| Jump to: | C D E F P Q S T U |
|---|