Next: Introduction, Previous: (dir), Up: (dir) [Contents][Index]
This is the uax-14 Reference Manual, version 1.0.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Thu Sep 15 06:25:27 2022 GMT+0.
Next: Systems, Previous: The uax-14 Reference Manual, Up: The uax-14 Reference Manual [Contents][Index]
## About UAX-14
This is an implementation of the "Unicode Standards Annex #14"(http://www.unicode.org/reports/tr14/)'s line breaking algorithm. It provides a fast and convenient way to determine line breaking opportunities in text.
Note that this algorithm does not support break opportunities that require morphological analysis. In order to handle such cases, please consult a system that provides this kind of capability, such as a hyphenation algorithm.
Also note that this system is completely unaware of layouting decisions. Any kind of layouting decisions, such as which breaks to pick, how to space between words, how to handle bidirectionality, and what to do in emergency situations when there are no breaks on an overfull line are left up to the user.
The system passes all tests offered by the Unicode standard.
## How To
The system will compile binary database files on first load. Should anything go wrong during this process, a note is produced on load. If you would like to prevent this automated loading, push ``uax-14-no-load`` to ``*features*`` before loading. You can then manually load the database files when convenient through ``load-databases``.
Once loaded, you can produce a list of line breaks for a string with ``list-breaks`` or break a string at every opportunity with ``break-string``. Typically however you will want to scan for the next break as you move along the string during layouting. To do so, create a breaker with ``make-breaker``, and call ``next-break`` whenever the next line break opportunity is required.
In pseudo-code, that could look something like this. We assume the local nickname ``uax-14`` for ``org.shirakumo.alloy.uax-14`` here.
::common lisp
(loop with breaker = (uax-14:make-breaker string)
with start = 0 and last = 0
do (multiple-value-bind (pos mandatory) (uax-14:next-break breaker)
(cond (mandatory
(insert-break pos)
(setf start pos))
((beyond-extents-p start pos)
(if (< last start) ; Force a break if we are overfull.
(loop while (beyond-extents-p start pos)
do (let ((next (find-last-fitting-cluster start)))
(insert-break next)
(setf start next))
finally (setf pos start))
(insert-break last))))
(setf last pos)))
::
## External Files
The following files are from their corresponding external sources, last accessed on 2019.09.03:
- ``LineBreak.txt`` https://www.unicode.org/Public/UCD/latest/ucd/LineBreak.txt
- ``LineBreakTest.txt`` https://www.unicode.org/Public/UCD/latest/ucd/auxiliary/LineBreakTest.txt
At the time, Unicode 12.1 was considered the latest version.
## Acknowledgements
The code in this project is largely based on the "linebreak"(https://github.com/foliojs/linebreak) project by Devon Govett et al.
Next: Files, Previous: Introduction, Up: The uax-14 Reference Manual [Contents][Index]
The main system appears first, followed by any subsystem dependency.
Implementation of the Unicode Standards Annex #14’s line breaking algorithm
Nicolas Hafner <shinmera@tymoon.eu>
Nicolas Hafner <shinmera@tymoon.eu>
(GIT https://github.com/Shinmera/uax-14.git)
zlib
1.0.0
documentation-utils (system).
Next: Packages, Previous: Systems, Up: The uax-14 Reference Manual [Contents][Index]
Files are sorted by type and then listed depth-first from the systems components trees.
Next: uax-14/package.lisp, Previous: Lisp, Up: Lisp [Contents][Index]
uax-14 (system).
Next: uax-14/database.lisp, Previous: uax-14/uax-14.asd, Up: Lisp [Contents][Index]
uax-14 (system).
Next: uax-14/uax-14.lisp, Previous: uax-14/package.lisp, Up: Lisp [Contents][Index]
package.lisp (file).
uax-14 (system).
Next: uax-14/documentation.lisp, Previous: uax-14/database.lisp, Up: Lisp [Contents][Index]
database.lisp (file).
uax-14 (system).
Previous: uax-14/uax-14.lisp, Up: Lisp [Contents][Index]
uax-14.lisp (file).
uax-14 (system).
Next: Definitions, Previous: Files, Up: The uax-14 Reference Manual [Contents][Index]
Packages are listed by definition order.
common-lisp.
Next: Indexes, Previous: Packages, Up: The uax-14 Reference Manual [Contents][Index]
Definitions are sorted by export status, category, package, and then by lexicographic order.
Next: Internals, Previous: Definitions, Up: Definitions [Contents][Index]
Next: Ordinary functions, Previous: Public Interface, Up: Public Interface [Contents][Index]
Variable containing the absolute path of the line break database file.
See LOAD-DATABASES
See COMPILE-DATABASES
Variable containing the absolute path of the pair table file.
See LOAD-DATABASES
See COMPILE-DATABASES
Next: Standalone methods, Previous: Special variables, Up: Public Interface [Contents][Index]
Returns a list of all the pieces of the string, broken.
If MANDATORY-ONLY is T, the string is only split at mandatory line break opportunities, otherwise it is split at every opportunity.
See MAKE-BREAKER
See NEXT-BREAK
Compiles the database files from their sources.
This will load an optional part of the system and compile the database
files to an efficient byte representation. If the compilation is
successful, LOAD-DATABASES is called automatically.
See *LINE-BREAK-DATABASE-FILE*
See *PAIR-TABLE-FILE*
See LOAD-DATABASES
Returns a list of all line break opportunities in the string.
The list has the following form:
LIST ::= ENTRY+
ENTRY ::= (position mandatory)
This is equivalent to constructing a breaker and collecting the values
of NEXT-BREAK in a loop.
See MAKE-BREAKER
See NEXT-BREAK
Loads the databases from their files into memory.
If one of the files is missing, a warning of type NO-DATABASE-FILES is
signalled. If the loading succeeds, T is returned.
See *LINE-BREAK-DATABASE-FILE*
See *PAIR-TABLE-FILE*
See NO-DATABASE-FILES
Returns a breaker that can find line break opportunities in the given string.
If the optional breaker argument is supplied, the supplied breaker is
modified and reset to work with the new string instead. This allows
you to re-use a breaker.
Note that while you may pass a non-simple string, modifying this
string without resetting any breaker using it will result in undefined
behaviour.
See BREAKER
Returns the next line breaking opportunity of the breaker, if any.
Returns two values:
POSITION — The character index in the string at which the break
is located, or NIL if no further breaks are possible.
MANDATORY — Whether the break must be made at this location.
Note that there is always in the very least one break opportunity,
namely at the end of the string. However, after consuming this break
opportunity, NEXT-BREAK will return NIL.
Note that you may have to insert additional line breaks as required by
the layout constraints.
See BREAKER
Next: Conditions, Previous: Ordinary functions, Up: Public Interface [Contents][Index]
Next: Structures, Previous: Standalone methods, Up: Public Interface [Contents][Index]
Warning signalled when LOAD-DATABASES is called and the files are not present.
Two restarts must be active when this condition is signalled:
COMPILE — Call COMPILE-DATABASES
ABORT — Abort loading the databases, leaving them at their
previous state.
See LOAD-DATABASES
warning.
Previous: Conditions, Up: Public Interface [Contents][Index]
Contains line breaking state.
An instance of this is only useful for passing to MAKE-BREAKER and
NEXT-BREAK. It contains internal state that manages the line breaking
algorithm.
See MAKE-BREAKER
See NEXT-BREAK
structure-object.
common-lisp.
string
org.shirakumo.alloy.uax-14::idx
0
org.shirakumo.alloy.uax-14::idx
0
(unsigned-byte 8)
0
(unsigned-byte 8)
0
boolean
boolean
org.shirakumo.alloy.uax-14::idx
0
Previous: Public Interface, Up: Definitions [Contents][Index]
Next: Compiler macros, Previous: Special variables, Up: Internals [Contents][Index]
Next: Ordinary functions, Previous: Macros, Up: Internals [Contents][Index]
Next: Types, Previous: Compiler macros, Up: Internals [Contents][Index]
lb8a.
pos.
Previous: Definitions, Up: The uax-14 Reference Manual [Contents][Index]
| Jump to: | %
(
B C D F H L M N P T |
|---|
| Jump to: | %
(
B C D F H L M N P T |
|---|
Next: Data types, Previous: Functions, Up: Indexes [Contents][Index]
| Jump to: | *
C L N P S |
|---|
| Jump to: | *
C L N P S |
|---|
| Jump to: | B C D F I N O P S T U |
|---|
| Jump to: | B C D F I N O P S T U |
|---|