The inquisitor Reference Manual

Table of Contents

The inquisitor Reference Manual

This is the inquisitor Reference Manual, version 0.5, generated automatically by Declt version 3.0 "Montgomery Scott" on Mon Dec 02 10:33:05 2019 GMT+0.

1 Introduction

Inquisitor

Encoding/end-of-line detection and external-format abstraction for Common Lisp.

The Library is a sphere whose exact center is any one of its hexagons and whose circumference is inaccessible. -- "The Library of Babel" by Jorge Luis Borges

Features

• encoding/end-of-line name abstraction
• encoding/end-of-line detection
• external-format abstraction
  • make external-format for each implementations
  • make external-format from byte-array, stream and pathname (with auto-detection)
  • [TODO] abstract external-format of babel and flexi-stream
• supports many implementations
  • GNU CLISP
  • Embeddable Common Lisp
  • Steel Bank Common Lisp
  • Clozure Common Lisp
  • Armed Bear Common Lisp
  • Allegro Common Lisp
  • LispWorks

Installation

Get and install via quicklisp:

CL-USER> (ql:quickload :inquisitor)

Usage

Encoding detection

To detect encoding from stream, use (inq:detect-encoding stream scheme). This returns implementation independent encoding name. About scheme, see Encoding scheme.

for example:

CL-USER> (with-open-file (in #P"t/data/unicode/utf-8.txt"
                          :direction :input
                          :element-type '(unsigned-byte 8))
           (inq:detect-encoding in :jp))
:UTF-8

You can see the list of available encodings:

CL-USER> inq:+available-encodings+
(:UTF-8 :UCS-2LE :UCS-2BE :UTF-16 :ISO-2022-JP :EUC-JP :CP932 :BIG5 :ISO-2022-TW
 :GB2312 :GB18030 :ISO-2022-CN :EUC-KR :JOHAB :ISO-2022-KR :ISO-8859-6 :CP1256
 :ISO-8859-7 :CP1253 :ISO-8859-8 :CP1255 :ISO-8859-9 :CP1254 :ISO-8859-5
 :KOI8-R :KOI8-U :CP866 :CP1251 :ISO-8859-2 :CP1250 :ISO-8859-13 :CP1257)

Encoding scheme

Encoding scheme is a hint to detect encoding.

It's mostly impossible to detect encoding universally, because there are two encoding such that use same byte sequences to represent other characters. So, limitting target encodings has benefit to encoding detection.

Here, in inquisitor, languages are used to limit the encodings. Where languages are, roughly speaking, writing systems used in anywhere arround the world. Fixing language is equivalent to fixing possible characters. Becaus of which, encoding detection be slightly eazy.

Supported scheme (languages) is as follows:

• jp: japanese
• tw: taiwanese
• cn: chinese
• kr: korean
• ru: russian (latin-5)
• ar: arabic (latin-6)
• tr: turkish (latin-9)
• gr: greek (latin-7)
• hw: hebrew (latin-8)
• pl: polish (latin-2)
• bl: baltic (latin-7)

End-of-line type detection

If you want to know end-of-line (line break) type, use (inq:detect-end-of-line stream). This returns implementation independent end-of-line name.

CL-USER> (with-open-file (in "t/data/ascii/ascii-crlf.txt"
                             :direction :input
                             :element-type '(unsigned-byte 8))
           (inquisitor:detect-end-of-line in))

:CRLF

Implementation dependent/independent names

If you want to know implementation dependent name of encodings or eol type, use (inq:independent-name dependent-name). Returned value can be used as external-format, or its part.

CL-USER> (inq:independent-name :cp932)
#<ENCODING "CP932" :UNIX>  ; on CLISP
:WINDOWS-CP932  ; on ECL
:SHIFT_JIS  ; on SBCL
:WINDOWS-31J  ; on CCL
:|X-MS932_0213|  ; on ABCL

If you want to know implementation independent name of encodings or eol type, use (inq:dependent-name independent-name).

Eol

If you want to know eol is available on your implementation, use (inq:eol-available-p).

CL-USER> (inq:eol-available-p)
NIL  ; on SBCL

Make external-format

To make external-format from impl independent names, use (inq:make-external-format enc eol).

In SBCL and CCL, same code returns different value.

On SBCL:

CL-USER> (let* ((file #P"t/data/ja/sjis.txt")
                (enc (inq:detect-encoding file :jp))
                (eol (inq:detect-end-of-line file)))
           (inq:make-external-format enc eol))
:SHIFT_JIS

On CCL:

CL-USER> (let* ((file #P"t/data/ja/sjis.txt")
                (enc (inq:detect-encoding file :jp))
                (eol (inq:detect-end-of-line file)))
           (inq:make-external-format enc eol))
#<EXTERNAL-FORMAT :WINDOWS-31J/:UNIX #x302001C574CD>

External-format detection

Inquisitor provides external-format detection method. It detects encoding and eol style, then make external-format from these. It can use with vector, byte stream and pathname.

Let's see examples with CCL.

From vector

CL-USER> (inq:detect-external-format
          (encode-string-to-octets "公的な捜索係、調査官がいる。
わたしは彼らが任務を遂行しているところを見た。")
          :jp)
#<EXTERNAL-FORMAT :UTF-8/:UNIX #x30200046719D>

From stream

CL-USER> (with-open-file (in "t/data/unicode/utf-8.txt"
                             :direction :input
                             :element-type '(unsigned-byte 8))
           (inq:detect-external-format in :jp))
#<EXTERNAL-FORMAT :UTF-8/:UNIX #x30200046719D>

From pathname

CL-USER> (inq:detect-external-format #P"t/data/unicode/utf-8.txt" :jp)
#<EXTERNAL-FORMAT :UTF-8/:UNIX #x30200046719D>

Author

• Shiro Kawai - original code of encoding detection for Gauche
• Masayuki Onjo - porting from Gauche to Common Lisp
• zqwell - porting multilingual encoding detection from libguess
• Shinichi Tanaka (shinichi.tanaka45@gmail.com) - adding end-of-line detection and wrapping external-format

Copyright

Copyright (c) 2000-2007 Shiro Kawai (shiro@acm.org)
Copyright (c) 2007 Masayuki Onjo (onjo@lispuser.net)
Copyright (c) 2011 zqwell (zqwell@gmail.com)
Copyright (c) 2015 Shinichi Tanaka (s

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 inquisitor

Author

Shinichi Tanaka

License

MIT

Description

Encoding/end-of-line detection and of external-format abstraction for Common Lisp

Version

0.5

Dependencies

• alexandria
• anaphora

Source

inquisitor.asd (file)

Component

src (module)

3 Modules

Modules are listed depth-first from the system components tree.

3.1 inquisitor/src

Parent

inquisitor (system)

Location

src/

Components

• encoding (module)
• eol.lisp (file)
• names.lisp (file)
• external-format.lisp (file)
• util.lisp (file)
• inquisitor.lisp (file)

3.2 inquisitor/src/encoding

Parent

src (module)

Location

src/encoding/

Components

• dfa.lisp (file)
• table.lisp (file)
• guess.lisp (file)

4 Files

Files are sorted by type and then listed depth-first from the systems components trees.

4.1 Lisp

4.1.1 inquisitor.asd

Location

inquisitor.asd

Systems

inquisitor (system)

Packages

inquisitor-asd

4.1.2 inquisitor/src/encoding/dfa.lisp

Parent

encoding (module)

Location

src/encoding/dfa.lisp

Packages

inquisitor.encoding.dfa

Exported Definitions

• define-dfa (macro)
• dfa-init (macro)
• dfa-name (macro)
• dfa-none (function)
• dfa-process (macro)
• dfa-top (function)
• generate-order (macro)

Internal Definitions

• <arc> (class)
• <dfa> (class)
• <state> (class)
• arcs (macro)
• arcs-of (method)
• (setf arcs-of) (method)
• dfa-alive (macro)
• dfa-alone (function)
• dfa-next (macro)
• from-state-of (method)
• (setf from-state-of) (method)
• index-of (method)
• (setf index-of) (method)
• index-of (method)
• (setf index-of) (method)
• name-of (method)
• (setf name-of) (method)
• name-of (method)
• (setf name-of) (method)
• ranges-of (method)
• (setf ranges-of) (method)
• resolve-states (function)
• score (macro)
• score-of (method)
• (setf score-of) (method)
• state (macro)
• states (macro)
• states-of (method)
• (setf states-of) (method)
• to-state-of (method)
• (setf to-state-of) (method)

4.1.3 inquisitor/src/encoding/table.lisp

Dependency

dfa.lisp (file)

Parent

encoding (module)

Location

src/encoding/table.lisp

Packages

inquisitor.encoding.table

Internal Definitions

• +big5-ar+ (constant)
• +big5-st+ (constant)
• +cp1250-ar+ (constant)
• +cp1250-st+ (constant)
• +cp1251-ar+ (constant)
• +cp1251-st+ (constant)
• +cp1253-ar+ (constant)
• +cp1253-st+ (constant)
• +cp1254-ar+ (constant)
• +cp1254-st+ (constant)
• +cp1255-ar+ (constant)
• +cp1255-st+ (constant)
• +cp1256-ar+ (constant)
• +cp1256-st+ (constant)
• +cp1257-ar+ (constant)
• +cp1257-st+ (constant)
• +cp866-ar+ (constant)
• +cp866-st+ (constant)
• +cp932-ar+ (constant)
• +cp932-st+ (constant)
• +euc-jp-ar+ (constant)
• +euc-jp-st+ (constant)
• +euc-kr-ar+ (constant)
• +euc-kr-st+ (constant)
• +gb18030-ar+ (constant)
• +gb18030-st+ (constant)
• +gb2312-ar+ (constant)
• +gb2312-st+ (constant)
• +iso-2022-jp-ar+ (constant)
• +iso-2022-jp-st+ (constant)
• +iso-8859-13-ar+ (constant)
• +iso-8859-13-st+ (constant)
• +iso-8859-2-ar+ (constant)
• +iso-8859-2-st+ (constant)
• +iso-8859-5-ar+ (constant)
• +iso-8859-5-st+ (constant)
• +iso-8859-6-ar+ (constant)
• +iso-8859-6-st+ (constant)
• +iso-8859-7-ar+ (constant)
• +iso-8859-7-st+ (constant)
• +iso-8859-8-ar+ (constant)
• +iso-8859-8-st+ (constant)
• +iso-8859-9-ar+ (constant)
• +iso-8859-9-st+ (constant)
• +johab-ar+ (constant)
• +johab-st+ (constant)
• +koi8-r-ar+ (constant)
• +koi8-r-st+ (constant)
• +koi8-u-ar+ (constant)
• +koi8-u-st+ (constant)
• +ucs-2be-ar+ (constant)
• +ucs-2be-st+ (constant)
• +ucs-2le-ar+ (constant)
• +ucs-2le-st+ (constant)
• +utf-16-ar+ (constant)
• +utf-16-st+ (constant)
• +utf-8-ar+ (constant)
• +utf-8-st+ (constant)

4.1.4 inquisitor/src/encoding/guess.lisp

Dependencies

• dfa.lisp (file)
• table.lisp (file)

Parent

encoding (module)

Location

src/encoding/guess.lisp

Packages

inquisitor.encoding.guess

Exported Definitions

• ces-guess-from-vector (function)
• list-available-scheme (function)

Internal Definitions

• +schemes+ (special variable)
• check-byte-order-mark (macro)
• do-guess-loop (macro)
• guess (macro)
• guess-ar (function)
• guess-bl (function)
• guess-cn (function)
• guess-gr (function)
• guess-hw (function)
• guess-jp (function)
• guess-kr (function)
• guess-pl (function)
• guess-ru (function)
• guess-tr (function)
• guess-tw (function)

4.1.5 inquisitor/src/eol.lisp

Parent

src (module)

Location

src/eol.lisp

Packages

inquisitor.eol

Exported Definitions

• eol-available-p (function)
• eol-guess-from-vector (function)

4.1.6 inquisitor/src/names.lisp

Parent

src (module)

Location

src/names.lisp

Packages

inquisitor.names

Exported Definitions

• +available-encodings+ (special variable)
• +available-eols+ (special variable)
• dependent-name (function)
• independent-name (function)
• unicode-p (function)

Internal Definitions

+name-mapping+ (special variable)

4.1.7 inquisitor/src/external-format.lisp

Dependency

names.lisp (file)

Parent

src (module)

Location

src/external-format.lisp

Packages

inquisitor.external-format

Exported Definitions

• %make-external-format (generic function)
• %make-external-format (method)
• make-external-format (function)

4.1.8 inquisitor/src/util.lisp

Parent

src (module)

Location

src/util.lisp

Packages

inquisitor.util

Exported Definitions

• byte-array-p (function)
• byte-input-stream-p (function)
• with-byte-array (macro)

4.1.9 inquisitor/src/inquisitor.lisp

Dependencies

• encoding (module)
• eol.lisp (file)
• names.lisp (file)
• external-format.lisp (file)
• util.lisp (file)

Parent

src (module)

Location

src/inquisitor.lisp

Packages

inquisitor

Exported Definitions

• detect-encoding (generic function)
• detect-encoding (method)
• detect-encoding (method)
• detect-encoding (method)
• detect-end-of-line (generic function)
• detect-end-of-line (method)
• detect-end-of-line (method)
• detect-end-of-line (method)
• detect-external-format (generic function)
• detect-external-format (method)
• detect-external-format (method)
• detect-external-format (method)

Internal Definitions

*default-buffer-size* (special variable)

5 Packages

Packages are listed by definition order.

5.1 inquisitor-asd

Source

inquisitor.asd

Use List

• asdf/interface
• common-lisp

5.2 inquisitor.encoding.dfa

Source

dfa.lisp (file)

Use List

common-lisp

Exported Definitions

• define-dfa (macro)
• dfa-init (macro)
• dfa-name (macro)
• dfa-none (function)
• dfa-process (macro)
• dfa-top (function)
• generate-order (macro)

Internal Definitions

• <arc> (class)
• <dfa> (class)
• <state> (class)
• arcs (macro)
• arcs-of (generic function)
• arcs-of (method)
• (setf arcs-of) (method)
• (setf arcs-of) (generic function)
• dfa-alive (macro)
• dfa-alone (function)
• dfa-next (macro)
• from-state-of (generic function)
• from-state-of (method)
• (setf from-state-of) (method)
• (setf from-state-of) (generic function)
• index-of (generic function)
• index-of (method)
• (setf index-of) (method)
• index-of (method)
• (setf index-of) (method)
• (setf index-of) (generic function)
• name-of (generic function)
• name-of (method)
• (setf name-of) (method)
• name-of (method)
• (setf name-of) (method)
• (setf name-of) (generic function)
• ranges-of (generic function)
• ranges-of (method)
• (setf ranges-of) (method)
• (setf ranges-of) (generic function)
• resolve-states (function)
• score (macro)
• score-of (generic function)
• score-of (method)
• (setf score-of) (method)
• (setf score-of) (generic function)
• state (macro)
• states (macro)
• states-of (generic function)
• states-of (method)
• (setf states-of) (method)
• (setf states-of) (generic function)
• to-state-of (generic function)
• to-state-of (method)
• (setf to-state-of) (method)
• (setf to-state-of) (generic function)

5.3 inquisitor.encoding.table

Source

table.lisp (file)

Use List

common-lisp

Used By List

inquisitor.encoding.guess

Internal Definitions

• +big5-ar+ (constant)
• +big5-st+ (constant)
• +cp1250-ar+ (constant)
• +cp1250-st+ (constant)
• +cp1251-ar+ (constant)
• +cp1251-st+ (constant)
• +cp1253-ar+ (constant)
• +cp1253-st+ (constant)
• +cp1254-ar+ (constant)
• +cp1254-st+ (constant)
• +cp1255-ar+ (constant)
• +cp1255-st+ (constant)
• +cp1256-ar+ (constant)
• +cp1256-st+ (constant)
• +cp1257-ar+ (constant)
• +cp1257-st+ (constant)
• +cp866-ar+ (constant)
• +cp866-st+ (constant)
• +cp932-ar+ (constant)
• +cp932-st+ (constant)
• +euc-jp-ar+ (constant)
• +euc-jp-st+ (constant)
• +euc-kr-ar+ (constant)
• +euc-kr-st+ (constant)
• +gb18030-ar+ (constant)
• +gb18030-st+ (constant)
• +gb2312-ar+ (constant)
• +gb2312-st+ (constant)
• +iso-2022-jp-ar+ (constant)
• +iso-2022-jp-st+ (constant)
• +iso-8859-13-ar+ (constant)
• +iso-8859-13-st+ (constant)
• +iso-8859-2-ar+ (constant)
• +iso-8859-2-st+ (constant)
• +iso-8859-5-ar+ (constant)
• +iso-8859-5-st+ (constant)
• +iso-8859-6-ar+ (constant)
• +iso-8859-6-st+ (constant)
• +iso-8859-7-ar+ (constant)
• +iso-8859-7-st+ (constant)
• +iso-8859-8-ar+ (constant)
• +iso-8859-8-st+ (constant)
• +iso-8859-9-ar+ (constant)
• +iso-8859-9-st+ (constant)
• +johab-ar+ (constant)
• +johab-st+ (constant)
• +koi8-r-ar+ (constant)
• +koi8-r-st+ (constant)
• +koi8-u-ar+ (constant)
• +koi8-u-st+ (constant)
• +ucs-2be-ar+ (constant)
• +ucs-2be-st+ (constant)
• +ucs-2le-ar+ (constant)
• +ucs-2le-st+ (constant)
• +utf-16-ar+ (constant)
• +utf-16-st+ (constant)
• +utf-8-ar+ (constant)
• +utf-8-st+ (constant)

5.4 inquisitor.encoding.guess

Source

guess.lisp (file)

Use List

• inquisitor.encoding.table
• common-lisp

Exported Definitions

• ces-guess-from-vector (function)
• list-available-scheme (function)

Internal Definitions

• +schemes+ (special variable)
• check-byte-order-mark (macro)
• do-guess-loop (macro)
• guess (macro)
• guess-ar (function)
• guess-bl (function)
• guess-cn (function)
• guess-gr (function)
• guess-hw (function)
• guess-jp (function)
• guess-kr (function)
• guess-pl (function)
• guess-ru (function)
• guess-tr (function)
• guess-tw (function)

5.5 inquisitor.eol

Source

eol.lisp (file)

Use List

common-lisp

Exported Definitions

• eol-available-p (function)
• eol-guess-from-vector (function)

5.6 inquisitor.names

Source

names.lisp (file)

Nickname

inq.names

Use List

common-lisp

Exported Definitions

• +available-encodings+ (special variable)
• +available-eols+ (special variable)
• dependent-name (function)
• independent-name (function)
• unicode-p (function)

Internal Definitions

+name-mapping+ (special variable)

5.7 inquisitor.external-format

Source

external-format.lisp (file)

Use List

common-lisp

Exported Definitions

• %make-external-format (generic function)
• %make-external-format (method)
• make-external-format (function)

5.8 inquisitor.util

Source

util.lisp (file)

Use List

common-lisp

Exported Definitions

• byte-array-p (function)
• byte-input-stream-p (function)
• with-byte-array (macro)

5.9 inquisitor

Source

inquisitor.lisp (file)

Nickname

inq

Use List

common-lisp

Exported Definitions

• detect-encoding (generic function)
• detect-encoding (method)
• detect-encoding (method)
• detect-encoding (method)
• detect-end-of-line (generic function)
• detect-end-of-line (method)
• detect-end-of-line (method)
• detect-end-of-line (method)
• detect-external-format (generic function)
• detect-external-format (method)
• detect-external-format (method)
• detect-external-format (method)

Internal Definitions

*default-buffer-size* (special variable)

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

6.1 Exported definitions

6.1.1 Special variables

Special Variable: +available-encodings+

Package

inquisitor.names

Source

names.lisp (file)

Special Variable: +available-eols+

Package

inquisitor.names

Source

names.lisp (file)

6.1.2 Macros

Macro: define-dfa NAME &body STATES

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: dfa-init DFA-ST DFA-AR DFA-NAME

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: dfa-name DFA

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: dfa-process ORDER CH

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: generate-order &rest ENCODINGS

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: with-byte-array (VAR DIM) &body BODY

Package

inquisitor.util

Source

util.lisp (file)

6.1.3 Functions

Function: byte-array-p VEC

Package

inquisitor.util

Source

util.lisp (file)

Function: byte-input-stream-p STREAM

Package

inquisitor.util

Source

util.lisp (file)

Function: ces-guess-from-vector VECTOR SCHEME &optional ORDER STATE

Guess character encoding scheme under the language ‘scheme‘. ‘order‘ is a _dfa state_: _dfa state_ is a list consist of a) nodes of dfa, b) arrows of dfa c) score and d) name. ‘state‘ is a state except dfa state. ‘state‘ may be used or not be used by each guess-* function of scheme, although you must be pass to this function.
Specifying ‘order‘ and ‘state‘ means, we can restart guessing from the point of ‘order‘ and ‘state‘.

‘ces-guess-from-vector‘ returns ‘encoding‘ and ‘order‘: ‘encoding‘ is an _implementation independent_ name, ‘order‘ is a state at this function stops guessing process.

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: dependent-name INDEPENDENT-NAME &optional TYPE

Package

inquisitor.names

Source

names.lisp (file)

Function: dfa-none ORDER

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Function: dfa-top ORDER

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Function: eol-available-p ()

Package

inquisitor.eol

Source

eol.lisp (file)

Function: eol-guess-from-vector BUFFER

Package

inquisitor.eol

Source

eol.lisp (file)

Function: independent-name DEPENDENT-NAME &optional TYPE

Package

inquisitor.names

Source

names.lisp (file)

Function: list-available-scheme ()

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: make-external-format ENC EOL &rest ARGS &key TYPE &allow-other-keys

Package

inquisitor.external-format

Source

external-format.lisp (file)

Function: unicode-p INDEPENDENT-NAME

Package

inquisitor.names

Source

names.lisp (file)

6.1.4 Generic functions

Generic Function: %make-external-format TYPE ENC EOL &rest ARGS &key &allow-other-keys

Package

inquisitor.external-format

Source

external-format.lisp (file)

Methods

Method: %make-external-format (TYPE (eql impl)) ENC EOL &rest ARGS &key &allow-other-keys

Generic Function: detect-encoding INPUT SYMBOL

Package

inquisitor

Source

inquisitor.lisp (file)

Methods

Method: detect-encoding (PATH pathname) (SCHEME symbol)

Detect character encoding scheme under the ‘scheme‘ from ‘pathname‘.

Method: detect-encoding (STREAM stream) (SCHEME symbol)

Detect character encoding scheme under the ‘scheme‘ from ‘stream‘. Note that this method modifies ‘stream‘’s file position.

Method: detect-encoding (BUFFER vector) (SCHEME symbol)

Detect character encoding scheme under the ‘scheme‘ from ‘buffer‘.

Generic Function: detect-end-of-line INPUT

Package

inquisitor

Source

inquisitor.lisp (file)

Methods

Method: detect-end-of-line (PATH pathname)

Detect end-of-line style from ‘pathname‘.

Method: detect-end-of-line (STREAM stream)

Detect end-of-line style from ‘stream‘. Note that this method modifies ‘stream‘’s file position.

Method: detect-end-of-line (BUFFER vector)

Detect end-of-line style from ‘buffer‘.

Generic Function: detect-external-format INPUT SYMBOL

Package

inquisitor

Source

inquisitor.lisp (file)

Methods

Method: detect-external-format (PATH pathname) (SCHEME symbol)

Detect external-format from ‘pathname‘.

Method: detect-external-format (STREAM stream) (SCHEME symbol)

Detect external-format under the ‘scheme‘ from ‘buffer‘. Note that this method method modifies ‘stream‘’s file position.

Method: detect-external-format (BUFFER vector) (SCHEME symbol)

Detect external-format under the ‘scheme‘ from ‘buffer‘.

6.2 Internal definitions

6.2.1 Constants

Constant: +big5-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +big5-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1250-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1250-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1251-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1251-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1253-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1253-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1254-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1254-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1255-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1255-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1256-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1256-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1257-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp1257-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp866-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp866-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp932-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +cp932-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +euc-jp-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +euc-jp-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +euc-kr-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +euc-kr-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +gb18030-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +gb18030-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +gb2312-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +gb2312-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-2022-jp-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-2022-jp-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-13-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-13-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-2-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-2-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-5-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-5-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-6-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-6-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-7-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-7-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-8-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-8-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-9-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +iso-8859-9-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +johab-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +johab-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +koi8-r-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +koi8-r-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +koi8-u-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +koi8-u-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +ucs-2be-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +ucs-2be-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +ucs-2le-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +ucs-2le-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +utf-16-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +utf-16-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +utf-8-ar+

Package

inquisitor.encoding.table

Source

table.lisp (file)

Constant: +utf-8-st+

Package

inquisitor.encoding.table

Source

table.lisp (file)

6.2.2 Special variables

Special Variable: *default-buffer-size*

Specifies default buffer size is consed and used by ‘dedect-encoding‘, ‘detect-end-of-line‘ and ‘detect-external-format‘.

Package

inquisitor

Source

inquisitor.lisp (file)

Special Variable: +name-mapping+

Package

inquisitor.names

Source

names.lisp (file)

Special Variable: +schemes+

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

6.2.3 Macros

Macro: arcs DFA

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: check-byte-order-mark BUFFER BIG-ENDIAN-VALUE LITTLE-ENDIAN-VALUE ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Macro: dfa-alive DFA

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: dfa-next DFA CH

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: do-guess-loop (&rest LOOKAHEAD-VARS) BUFFER &body BODY

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Macro: guess (BUFFER ORDER STATE (&rest VARS) (&rest ENCS)) &body SPECIALIZED-CHECK

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Macro: score DFA

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: state DFA

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Macro: states DFA

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

6.2.4 Functions

Function: dfa-alone DFA ORDER

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Function: guess-ar BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-bl BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-cn BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-gr BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-hw BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-jp BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-kr BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-pl BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-ru BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-tr BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: guess-tw BUFFER &optional ORDER STATE

Package

inquisitor.encoding.guess

Source

guess.lisp (file)

Function: resolve-states STATE-DEFS

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

6.2.5 Generic functions

Generic Function: arcs-of OBJECT

Generic Function: (setf arcs-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: arcs-of (<STATE> <state>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf arcs-of) NEW-VALUE (<STATE> <state>)

automatically generated writer method

Source

dfa.lisp (file)

Generic Function: from-state-of OBJECT

Generic Function: (setf from-state-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: from-state-of (<ARC> <arc>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf from-state-of) NEW-VALUE (<ARC> <arc>)

automatically generated writer method

Source

dfa.lisp (file)

Generic Function: index-of OBJECT

Generic Function: (setf index-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: index-of (<ARC> <arc>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf index-of) NEW-VALUE (<ARC> <arc>)

automatically generated writer method

Source

dfa.lisp (file)

Method: index-of (<STATE> <state>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf index-of) NEW-VALUE (<STATE> <state>)

automatically generated writer method

Source

dfa.lisp (file)

Generic Function: name-of OBJECT

Generic Function: (setf name-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: name-of (<STATE> <state>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf name-of) NEW-VALUE (<STATE> <state>)

automatically generated writer method

Source

dfa.lisp (file)

Method: name-of (<DFA> <dfa>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf name-of) NEW-VALUE (<DFA> <dfa>)

automatically generated writer method

Source

dfa.lisp (file)

Generic Function: ranges-of OBJECT

Generic Function: (setf ranges-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: ranges-of (<ARC> <arc>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf ranges-of) NEW-VALUE (<ARC> <arc>)

automatically generated writer method

Source

dfa.lisp (file)

Generic Function: score-of OBJECT

Generic Function: (setf score-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: score-of (<ARC> <arc>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf score-of) NEW-VALUE (<ARC> <arc>)

automatically generated writer method

Source

dfa.lisp (file)

Generic Function: states-of OBJECT

Generic Function: (setf states-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: states-of (<DFA> <dfa>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf states-of) NEW-VALUE (<DFA> <dfa>)

automatically generated writer method

Source

dfa.lisp (file)

Generic Function: to-state-of OBJECT

Generic Function: (setf to-state-of) NEW-VALUE OBJECT

Package

inquisitor.encoding.dfa

Methods

Method: to-state-of (<ARC> <arc>)

automatically generated reader method

Source

dfa.lisp (file)

Method: (setf to-state-of) NEW-VALUE (<ARC> <arc>)

automatically generated writer method

Source

dfa.lisp (file)

6.2.6 Classes

Class: <arc> ()

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• score-of (method)
• score-of (method)
• index-of (method)
• index-of (method)
• ranges-of (method)
• ranges-of (method)
• to-state-of (method)
• to-state-of (method)
• from-state-of (method)
• from-state-of (method)

Direct slots

Slot: from-state

Initargs

:from-state

Readers

from-state-of (generic function)

Writers

(setf from-state-of) (generic function)

Slot: to-state

Initargs

:to-state

Readers

to-state-of (generic function)

Writers

(setf to-state-of) (generic function)

Slot: ranges

Initargs

:ranges

Readers

ranges-of (generic function)

Writers

(setf ranges-of) (generic function)

Slot: index

Initargs

:index

Readers

index-of (generic function)

Writers

(setf index-of) (generic function)

Slot: score

Initargs

:score

Readers

score-of (generic function)

Writers

(setf score-of) (generic function)

Class: <dfa> ()

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• states-of (method)
• states-of (method)
• name-of (method)
• name-of (method)

Direct slots

Slot: name

Initargs

:name

Readers

name-of (generic function)

Writers

(setf name-of) (generic function)

Slot: states

Initargs

:states

Readers

states-of (generic function)

Writers

(setf states-of) (generic function)

Class: <state> ()

Package

inquisitor.encoding.dfa

Source

dfa.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• arcs-of (method)
• arcs-of (method)
• index-of (method)
• index-of (method)
• name-of (method)
• name-of (method)

Direct slots

Slot: name

Initargs

:name

Readers

name-of (generic function)

Writers

(setf name-of) (generic function)

Slot: index

Initargs

:index

Readers

index-of (generic function)

Writers

(setf index-of) (generic function)

Slot: arcs

Initargs

:arcs

Readers

arcs-of (generic function)

Writers

(setf arcs-of) (generic function)

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables