Next: Introduction, Previous: (dir), Up: (dir) [Contents][Index]
This is the cl-conllu Reference Manual, version 0.7, generated automatically by Declt version 3.0 "Montgomery Scott" on Tue Apr 28 10:51:19 2020 GMT+0.
| • Introduction | What cl-conllu is all about | |
| • Systems | The systems documentation | |
| • Files | The files documentation | |
| • Packages | The packages documentation | |
| • Definitions | The symbols documentation | |
| • Indexes | Concepts, functions, variables and data types |
#+title: Library for working with CoNLL-U files with CL
The =cl-conllu= is a Common Lisp library to work with [[http://universaldependencies.org/format.html][CoNLL-U]],
licensed under the [[http://www.apache.org/licenses/LICENSE-2.0][Apache license]].
It is developed and tested with SBCL but should probably run with any
other implementation.
* Install
The =cl-conllu= library is now available from quicklisp distribution,
if you are not planning to change the code, just use:
#+BEGIN_SRC lisp
(ql:quickload :cl-conllu)
#+END_SRC
If you don't have quicklisp installed already, follow [[https://www.quicklisp.org/beta/#installation][these steps]].
If you plan on contributing, clone this project to your
=local-projects= quicklisp directory (usually at
=~/quicklisp/local-projects/=) and use the same command as above to
load the code.
* Documentation
See the https://github.com/own-pt/cl-conllu/wiki
* How to cite
http://arademaker.github.io/bibliography/tilic-stil-2017.html
#+BEGIN_EXAMPLE
@inproceedings{tilic-stil-2017,
author = {Muniz, Henrique and Chalub, Fabricio and Rademaker, Alexandre},
title = {CL-CONLLU: dependências universais em Common Lisp},
booktitle = {V Workshop de Iniciação Científica em Tecnologia da
Informação e da Linguagem Humana (TILic)},
year = {2017},
address = {Uberlândia, MG, Brazil},
note = {https://sites.google.com/view/tilic2017/}
}
#+END_EXAMPLE
Next: Files, Previous: Introduction, Up: Top [Contents][Index]
The main system appears first, followed by any subsystem dependency.
| • The cl-conllu system |
Alexandre Rademaker <alexrad@br.ibm.com>
Apache 2.0
Common Lisp library for dealing with CoNLL-U files
This library provides a set of functions to work with CoNLL-U files. See https://universaldependencies.org/format.html for details about the CoNLL-U format adopted by the Universal Dependencies community. The library has functions for read/write files, apply rules for sentences transformation in batch mode, tree visualization, compare and evaluation trees etc. Documentation available in https://github.com/own-pt/cl-conllu/wiki.
0.7
cl-conllu.asd (file)
Files are sorted by type and then listed depth-first from the systems components trees.
| • Lisp files |
Next: The cl-conllu/packages․lisp file, Previous: Lisp files, Up: Lisp files [Contents][Index]
cl-conllu.asd
cl-conllu (system)
Next: The cl-conllu/data․lisp file, Previous: The cl-conllu․asd file, Up: Lisp files [Contents][Index]
cl-conllu (system)
packages.lisp
Next: The cl-conllu/read-write․lisp file, Previous: The cl-conllu/packages․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
data.lisp
Next: The cl-conllu/evaluate․lisp file, Previous: The cl-conllu/data․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
read-write.lisp
Next: The cl-conllu/confusion-matrix․lisp file, Previous: The cl-conllu/read-write․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
evaluate.lisp
Next: The cl-conllu/html․lisp file, Previous: The cl-conllu/evaluate․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
confusion-matrix.lisp
Next: The cl-conllu/query․lisp file, Previous: The cl-conllu/confusion-matrix․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
html.lisp
Next: The cl-conllu/utils․lisp file, Previous: The cl-conllu/html․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
query.lisp
Next: The cl-conllu/rdf․lisp file, Previous: The cl-conllu/query․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
utils.lisp
Next: The cl-conllu/rdf-wilbur․lisp file, Previous: The cl-conllu/utils․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
rdf.lisp
Next: The cl-conllu/command-line․lisp file, Previous: The cl-conllu/rdf․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
rdf-wilbur.lisp
convert-to-rdf (function)
Next: The cl-conllu/rules․lisp file, Previous: The cl-conllu/rdf-wilbur․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
command-line.lisp
Next: The cl-conllu/editor․lisp file, Previous: The cl-conllu/command-line․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
rules.lisp
Next: The cl-conllu/conllu-prolog․lisp file, Previous: The cl-conllu/rules․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
editor.lisp
conlluedit (function)
Next: The cl-conllu/niceline․lisp file, Previous: The cl-conllu/editor․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
conllu-prolog.lisp
convert-filename (function)
Next: The cl-conllu/tag-converter․lisp file, Previous: The cl-conllu/conllu-prolog․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
niceline.lisp
Next: The cl-conllu/draw․lisp file, Previous: The cl-conllu/niceline․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
tag-converter.lisp
write-token-tag-suffix (function)
Previous: The cl-conllu/tag-converter․lisp file, Up: Lisp files [Contents][Index]
cl-conllu (system)
draw.lisp
Next: Definitions, Previous: Files, Up: Top [Contents][Index]
Packages are listed by definition order.
Next: The cl-conllu package, Previous: Packages, Up: Packages [Contents][Index]
Functions for producing html formatting of objects in the library.
packages.lisp (file)
Next: The conllu․rules package, Previous: The conllu․html package, Up: Packages [Contents][Index]
packages.lisp (file)
Next: The conllu․converters․tags package, Previous: The cl-conllu package, Up: Packages [Contents][Index]
packages.lisp (file)
Next: The conllu․rdf package, Previous: The conllu․rules package, Up: Packages [Contents][Index]
packages.lisp (file)
write-token-tag-suffix (function)
Next: The conllu․evaluate package, Previous: The conllu․converters․tags package, Up: Packages [Contents][Index]
packages.lisp (file)
Next: The conllu․editor package, Previous: The conllu․rdf package, Up: Packages [Contents][Index]
Functions for evaluating datasets and parser outputs in the CoNLL-U format.
packages.lisp (file)
Next: The conllu․converters․niceline package, Previous: The conllu․evaluate package, Up: Packages [Contents][Index]
ruled based transformations
packages.lisp (file)
conlluedit (function)
Next: The conllu․draw package, Previous: The conllu․editor package, Up: Packages [Contents][Index]
packages.lisp (file)
Next: The conllu․user package, Previous: The conllu․converters․niceline package, Up: Packages [Contents][Index]
packages.lisp (file)
Next: The conllu․prolog package, Previous: The conllu․draw package, Up: Packages [Contents][Index]
packages.lisp (file)
Previous: The conllu․user package, Up: Packages [Contents][Index]
packages.lisp (file)
convert-filename (function)
Definitions are sorted by export status, category, package, and then by lexicographic order.
| • Exported definitions | ||
| • Internal definitions |
Next: Internal definitions, Previous: Definitions, Up: Definitions [Contents][Index]
| • Exported special variables | ||
| • Exported functions | ||
| • Exported generic functions | ||
| • Exported classes |
Next: Exported functions, Previous: Exported definitions, Up: Exported definitions [Contents][Index]
HTML for styling the confusion matrix.
html.lisp (file)
Next: Exported generic functions, Previous: Exported special variables, Up: Exported definitions [Contents][Index]
Receives a sentence and reenumerate IDs and HEAD values of each token so that their order (as in sentence-tokens) is respected.
rules.lisp (file)
rules.lisp (file)
Attachment score by sentence (macro-average).
The attachment score is the percentage of words that have correct
arcs to their heads. The unlabeled attachment score (UAS) considers
only who is the head of the token, while the labeled attachment
score (LAS) considers both the head and the arc label (dependency
label / syntactic class).
In order to choose between labeled or unlabeled,
set the key argument LABELED.
References:
- Dependency Parsing. Kubler, Mcdonald and Nivre (pp.79-80)
evaluate.lisp (file)
Attachment score by word (micro-average).
The attachment score is the percentage of words that have correct
arcs to their heads. The unlabeled attachment score (UAS) considers
only who is the head of the token, while the labeled attachment
score (LAS) considers both the head and the arc label (dependency
label / syntactic class).
In order to choose between labeled or unlabeled,
set the key argument LABELED.
References:
- Dependency Parsing. Kubler, Mcdonald and Nivre (pp.79-80)
evaluate.lisp (file)
Returns the number of tokens that are contained in the cell defined
by LABEL1 LABEL2 in the confusion matrix CM.
If DEFAULT-IF-UNDEFINED, returns 0. Otherwise, raises an error in case there is no such cell.
confusion-matrix.lisp (file)
Returns the list of (SENT-ID . TOKEN-ID) of tokens in the cell
LABEL1 LABEL2.
If DEFAULT-IF-UNDEFINED, returns the empty list. Otherwise, raises an error in case there is no such cell.
confusion-matrix.lisp (file)
Returns a list of ’(LABEL1 LABEL2) for each cell in the confusion matrix CM.
confusion-matrix.lisp (file)
confusion-matrix.lisp (file)
Returns the list of all labels in the confusion matrix CM.
confusion-matrix.lisp (file)
Returns a new CONFUSION-MATRIX with new empty cells for each pair (LABEL1 LABEL2) of labels in (confusion-matrix-labels CM) that are undefined in CM.
confusion-matrix.lisp (file)
Returns the list of labels occuring in the rows of the confusion matix CM.
confusion-matrix.lisp (file)
Updates an existing confusion matrix by a list of sentences LIST-SENT1 and LIST-SENT2.
confusion-matrix.lisp (file)
editor.lisp (file)
conllu-prolog.lisp (file)
Converts the collection of sentences (as generated by READ-CONLLU) in CONLL, using the function TEXT-FN to extract the text of each sentence and ID-FN to extract the id of each sentence (we need this as there is no standardized way of knowing this.) Also the generated Turtle file contains a lot of duplication so when you import it into your triple-store, make sure you remove all duplicate triples afterwards.
Converts a list of sentences (e.g. as generated by READ-CONLLU)
in SENTENCES, using the function TEXT-FN to extract the text of each
sentence and ID-FN to extract the id of each sentence (we need this
as there is no standardized way of knowing this.)
Currently only ntriples is supported as RDF-FORMAT.
rdf-wilbur.lisp (file)
utils.lisp (file)
Returns the list of sentences of LIST-SENT1 that are an exact
match to the corresponding sentence of LIST-SENT2 (same position in
list).
LIST-SENT1 and LIST-SENT2 must have the same size with corresponding sentences in order.
evaluate.lisp (file)
Returns the percentage of sentences of LIST-SENT1 that are an exact
match to LIST-SENT2.
LIST-SENT1 and LIST-SENT2 must have the same size with corresponding
sentences in order.
The typical use case is comparing the result of a tagger (or
parser) against a test set, where an exact match is a completely
correct tagging (or parse) for the sentence.
References:
- Dependency Parsing. Kubler, Mcdonald and Nivre (p.79)
evaluate.lisp (file)
Return a function that returns one CoNLL-U sentence per call.
read-write.lisp (file)
utils.lisp (file)
Creates a new confusion matrix from the lists of sentences LIST-SENT1 and LIST-SENT2.
confusion-matrix.lisp (file)
read-write.lisp (file)
Verifies if a sentence tree is projective. Intuitively, this means
that, keeping word order, there’s no two dependency arcs that cross.
More formally, let i -> j mean that j’s head is node i. Let ’->*’
be the transitive closure of ’->’.
A tree if projective when, for each node i, j: if i -> j, then for
each node k between i and j (i < k < j or j < k < i), i ->* k.
References:
- Nivre, Joakim; Inductive Dependency Parsing, 2006
- https://en.wikipedia.org/wiki/Discontinuity_(linguistics)
utils.lisp (file)
evaluate.lisp (file)
evaluate.lisp (file)
evaluate.lisp (file)
Restricted to words which are classified as of syntactical class
(dependency type to head) DEPREL, returns the precision:
the number of true positives divided by the number of words
predicted positive (that is, predicted as of class DEPREL).
We assume that LIST-SENT1 is the classified (predicted) result
and LIST-SENT2 is the list of golden (correct) sentences.
ERROR-TYPE defines what is considered an error (a false negative).
Some usual values are:
- ’(deprel) :: for the deprel tagging task only
- ’(head) :: for considering errors for each syntactic class
- ’(deprel head) :: for considering correct only when both deprel
and head are correct.
evaluate.lisp (file)
query.lisp (file)
query.lisp (file)
read-write.lisp (file)
read-write.lisp (file)
read-write.lisp (file)
tag-converter.lisp (file)
Writes as sentence object input from STREAM as
FORM.SEPARATOR.TAGVALUE (without dots), followed by a whitespace
character.
Example:
;; Consider the file example.txt, with contents:
;; Pudim_NOUN é_VERB bom_ADJ ._PUNCT
;; E_CONJ torta_NOUN também_ADV ._PUNCT
(with-open-file (s "./example.txt")
(write-conllu-to-stream (read-sentence-tag-suffix s ’upostag "_")))
1 Pudim _ NOUN _ _ _ _ _ _
2 é _ VERB _ _ _ _ _ _
3 bom _ ADJ _ _ _ _ _ _
4 . _ PUNCT _ _ _ _ _ _
1 E _ CONJ _ _ _ _ _ _
2 torta _ NOUN _ _ _ _ _ _
3 também _ ADV _ _ _ _ _ _
4 . _ PUNCT _ _ _ _ _ _
tag-converter.lisp (file)
read-write.lisp (file)
Restricted to words which are originally of syntactic class
(dependency type to head) DEPREL, returns the recall:
the number of true positives divided by the number of words
originally positive (that is, originally of class DEPREL).
We assume that LIST-SENT1 is the classified result
and LIST-SENT2 is the list of golden (correct) sentences.
ERROR-TYPE defines what is considered an error (a false negative).
Some usual values are:
- ’(deprel) :: for the deprel tagging task only
- ’(head) :: for considering errors for each syntactic class
- ’(deprel head) :: for considering correct only when both deprel
and head are correct.
evaluate.lisp (file)
Receives SENTENCE, a sentence object, and returns a string
reconstructed from its tokens and mtokens.
If IGNORE-MTOKENS, then tokens’ forms are used. Else, tokens with
id contained in a mtoken are not used, with mtoken’s form being
used instead.
It is possible to special format some tokens. In order to do so,
both SPECIAL-FORMAT-TEST and SPECIAL-FORMAT-FUNCTION should be
passed. Then for each object (token or mtoken) for which
SPECIAL-FORMAT-TEST returns a non-nil result, its form is modified
by SPECIAL-FORMAT-FUNCTION in the final string.
Example:
(sentence-tokens *sentence*)
=> (#<TOKEN The/DET #1-det-3> #<TOKEN US/PROPN #2-compound-3>
#<TOKEN troops/NOUN #3-nsubj-4> #<TOKEN fired/VERB #4-root-0>
#<TOKEN into/ADP #5-case-8> #<TOKEN the/DET #6-det-8>
#<TOKEN hostile/ADJ #7-amod-8> #<TOKEN crowd/NOUN #8-obl-4>
#<TOKEN ,/PUNCT #9-punct-4> #<TOKEN killing/VERB #10-advcl-4>
#<TOKEN 4/NUM #11-obj-10> #<TOKEN ./PUNCT #12-punct-4>)
(sentence->text
sentence)
=> "The US troops fired into the hostile crowd, killing 4."
(sentence->text
sentence
:special-format-test #’(lambda (token)
(eq (token-upostag token)
"VERB"))
:special-format-function (lambda (string)
(format nil "*~a*" (string-upcase string))))
=> "The US troops *FIRED* into the hostile crowd, *KILLING* 4."
Based on the idea from [1], it produces a tree view of the
sentence, still need to improve the priorities of children.
Code at https://github.com/sivareddyg/UDepLambda in file src/deplambda/parser/TreeTransformer.java method ’binarizeTree’
[1] Siva Reddy, O. Tackstrom, M. Collins, T. Kwiatkowski, D. Das, M. Steedman, and M. Lapataw, Transforming Dependency Structures to Logical Forms for Semantic Parsing, Transactions of the Association for Computational Linguistics, pp. 127–140, Apr. 2016.
Tests if, for each slot, sent-1 has the same values as sent-2. For tokens and multiword tokens, it uses token-equal and mtoken-equal, respectively.
draw.lisp (file)
draw.lisp (file)
read-write.lisp (file)
read-write.lisp (file)
Writes sentence as CoNLL-U file in STREAM as FORM.SEPARATOR.TAGVALUE (without
dots), followed by a whitespace character.
If TAG is NIL, then writes only FORMs, followed by a whitespace character.
Example:
;; supposing sentence already defined
(write-sentence-tag-suffix-to-stream (sentence :tag ’xpostag :separator "_"))
Pierre_NNP Vinken_NNP ,_, 61_CD years_NNS old_JJ ,_, will_MD join_VB the_DT board_NN as_IN
a_DT nonexecutive_JJ director_NN Nov._NNP 29_CD ._.
=> NIL
tag-converter.lisp (file)
See documentation for write-sentence-tag-suffix-to-stream
tag-converter.lisp (file)
See documentation for write-sentence-tag-suffix-to-stream
tag-converter.lisp (file)
Next: Exported classes, Previous: Exported functions, Up: Exported definitions [Contents][Index]
Identifier of the corpus or experiment.
confusion-matrix.lisp (file)
Outputs a HTML string for the object.
html.lisp (file)
Previous: Exported generic functions, Up: Exported definitions [Contents][Index]
confusion-matrix.lisp (file)
standard-object (class)
Identifier of the corpus or experiment.
:corpus-id
confusion-matrix-corpus-id (generic function)
(setf confusion-matrix-corpus-id) (generic function)
Function used to label each token.
:key-fn
(function cl-conllu:token-upostag)
confusion-matrix-key-fn (generic function)
(setf confusion-matrix-key-fn) (generic function)
Function which compares two labels. Typically a form of equality.
:test-fn
(function equal)
confusion-matrix-test-fn (generic function)
(setf confusion-matrix-test-fn) (generic function)
Function which sorts labels. By default,
converts labels to string and uses lexicographical order
:sort-fn
(function (lambda (conllu.evaluate::x conllu.evaluate::y) (string<= (format nil "~a" conllu.evaluate::x) (format nil "~a" conllu.evaluate::y))))
confusion-matrix-sort-fn (generic function)
(setf confusion-matrix-sort-fn) (generic function)
Parameter which contains the contents of the confusion matrix.
confusion-matrix-rows (generic function)
(setf confusion-matrix-rows) (generic function)
data.lisp (file)
abstract-token (class)
:start
mtoken-start (generic function)
(setf mtoken-start) (generic function)
:end
mtoken-end (generic function)
(setf mtoken-end) (generic function)
:form
mtoken-form (generic function)
(setf mtoken-form) (generic function)
:misc
"_"
mtoken-misc (generic function)
(setf mtoken-misc) (generic function)
data.lisp (file)
standard-object (class)
:start
0
sentence-start (generic function)
(setf sentence-start) (generic function)
:meta
sentence-meta (generic function)
(setf sentence-meta) (generic function)
:tokens
sentence-tokens (generic function)
(setf sentence-tokens) (generic function)
:mtokens
sentence-mtokens (generic function)
(setf sentence-mtokens) (generic function)
:etokens
sentence-etokens (generic function)
(setf sentence-etokens) (generic function)
data.lisp (file)
abstract-token (class)
:id
token-id (generic function)
(setf token-id) (generic function)
:form
token-form (generic function)
(setf token-form) (generic function)
:lemma
token-lemma (generic function)
(setf token-lemma) (generic function)
:upostag
"_"
token-upostag (generic function)
(setf token-upostag) (generic function)
:xpostag
"_"
token-xpostag (generic function)
(setf token-xpostag) (generic function)
:feats
"_"
token-feats (generic function)
(setf token-feats) (generic function)
:head
"_"
token-head (generic function)
(setf token-head) (generic function)
:deprel
"_"
token-deprel (generic function)
(setf token-deprel) (generic function)
:deps
"_"
token-deps (generic function)
(setf token-deps) (generic function)
:misc
"_"
token-misc (generic function)
(setf token-misc) (generic function)
Previous: Exported definitions, Up: Definitions [Contents][Index]
| • Internal special variables | ||
| • Internal macros | ||
| • Internal functions | ||
| • Internal generic functions | ||
| • Internal conditions | ||
| • Internal classes |
Next: Internal macros, Previous: Internal definitions, Up: Internal definitions [Contents][Index]
conllu-prolog.lisp (file)
conllu-prolog.lisp (file)
List of the 37 universal syntactic relations in UD.
evaluate.lisp (file)
query.lisp (file)
evaluate.lisp (file)
niceline.lisp (file)
Next: Internal functions, Previous: Internal special variables, Up: Internal definitions [Contents][Index]
editor.lisp (file)
Next: Internal generic functions, Previous: Internal macros, Up: Internal definitions [Contents][Index]
editor.lisp (file)
editor.lisp (file)
niceline.lisp (file)
editor.lisp (file)
rules.lisp (file)
command-line.lisp (file)
draw.lisp (file)
query.lisp (file)
editor.lisp (file)
rules.lisp (file)
rules.lisp (file)
rules.lisp (file)
editor.lisp (file)
rules.lisp (file)
rules.lisp (file)
editor.lisp (file)
rules.lisp (file)
rules.lisp (file)
editor.lisp (file)
utils.lisp (file)
query.lisp (file)
conllu-prolog.lisp (file)
conllu-prolog.lisp (file)
editor.lisp (file)
read-write.lisp (file)
rdf.lisp (file)
Returns a new CONFUSION-MATRIX with the same cells values as CM.
confusion-matrix.lisp (file)
Updates an existing confusion matrix by a pair of matching sentences SENT1 and SENT2. That is, SENT1 and SENT2 should be alternative analyses of the same natural language sentence.
confusion-matrix.lisp (file)
confusion-matrix.lisp (file)
niceline.lisp (file)
niceline.lisp (file)
Input: string
Output: list of feature nodes
Returns a list of nodes to be used as objects in triples with the predicate "conll:feats".
Examples:
(let ((wilbur:*nodes* (make-instance ’wilbur:dictionary))
(wilbur:*db* (make-instance ’wilbur:db)))
(wilbur:add-namespace "olia-sys" "http://purl.org/olia/system.owl#")
(wilbur:add-namespace "conll" "http://br.ibm.com/conll/LEMMA")
(convert-features-to-rdf "Mood=Ind|Tense=Past|VerbForm=Fin" (node "sentence1")))
=>
(!"http://br.ibm.com/conll/LEMMA#verbFormFin"
!"http://br.ibm.com/conll/LEMMA#tensePast"
!"http://br.ibm.com/conll/LEMMA#moodInd")
rdf-wilbur.lisp (file)
Input: list of pairs (name value), node
Output: List of triples
Example:
(let ((wilbur:*db* (make-instance ’wilbur:db))
(metadata ’(("sent_id" . "test")
("text" . "The US troops fired into the hostile crowd, killing 4.")))
(sentence-node (node "sentence")))
(convert-sentence-metadata metadata sentence-node))
=>
((#<WILBUR:TRIPLE !"sentence" !conll:metadata/sent_id #"test" {10048AA2E3}>)
(#<WILBUR:TRIPLE !"sentence" !conll:metadata/text #"The US troops fired into the hostile crowd, killing 4." {10048AA753}>))
rdf-wilbur.lisp (file)
rdf-wilbur.lisp (file)
rdf.lisp (file)
rdf-wilbur.lisp (file)
Creates the cell for row LABEL1 and column LABEL2 in confusion matrix CM.
confusion-matrix.lisp (file)
editor.lisp (file)
editor.lisp (file)
editor.lisp (file)
command-line.lisp (file)
conllu-prolog.lisp (file)
rules.lisp (file)
rdf.lisp (file)
rdf.lisp (file)
query.lisp (file)
Compares if two sentences are an exact match.
The typical use case is comparing the result of a tagger (or
parser) against a test sentence.
Returns SENT1 if SENT1 and SENT2 agree for all tokens with respect to
the COMPARED-FIELDS. Otherwise returns NIL.
If they do not have the same number of tokens or if the tokens do not agree on each IDENTITY-FIELD, then the sentences are not ’the same’ and thus an error is returned.
evaluate.lisp (file)
editor.lisp (file)
Predicate for verifying whether the cell for row LABEL1 and column LABEL2 already exist in the confusion matrix CM.
confusion-matrix.lisp (file)
utils.lisp (file)
draw.lisp (file)
editor.lisp (file)
draw.lisp (file)
draw.lisp (file)
utils.lisp (file)
Inserts TOKEN as an occurence in the cell LABEL1 LABEL2 of the confusion matrix CM.
confusion-matrix.lisp (file)
Inserts TOKEN in a SENTENCE object. It will not be inserted exactly as given: its ID will be the same (place where it’ll be inserted) but its head should point to id value prior to the insertion. Therefore, it will be modified. Its TOKEN-SENTENCE slot will be modified as well in order to point to SENTENCE. It changes the SENTENCE object passed.
rules.lisp (file)
rules.lisp (file)
rules.lisp (file)
conllu-prolog.lisp (file)
editor.lisp (file)
read-write.lisp (file)
read-write.lisp (file)
read-write.lisp (file)
read-write.lisp (file)
rdf.lisp (file)
rdf.lisp (file)
rdf.lisp (file)
rdf.lisp (file)
conllu-prolog.lisp (file)
rdf.lisp (file)
rdf.lisp (file)
rdf.lisp (file)
rdf.lisp (file)
draw.lisp (file)
draw.lisp (file)
rdf.lisp (file)
Receives a function and a list of lists and returns the appended result of the aplication of the function to each list.
utils.lisp (file)
editor.lisp (file)
rules.lisp (file)
rules.lisp (file)
rules.lisp (file)
editor.lisp (file)
editor.lisp (file)
The original file contains a set of sentences. The modified file contains some sentences from original modified, this function replaces in original the sentences presented in modified file, matching them using the sentence ids. If the modified file contains sentence not in original, the flag ’add-new’ , if true, says that these sentence must be added in the end of the original file.
command-line.lisp (file)
rules.lisp (file)
Tests if, for each slot, mtoken-1 has the same values as mtoken-2.
editor.lisp (file)
niceline.lisp (file)
editor.lisp (file)
editor.lisp (file)
query.lisp (file)
utils.lisp (file)
conllu-prolog.lisp (file)
conllu-prolog.lisp (file)
conllu-prolog.lisp (file)
niceline.lisp (file)
niceline.lisp (file)
niceline.lisp (file)
rules.lisp (file)
rules.lisp (file)
editor.lisp (file)
editor.lisp (file)
editor.lisp (file)
editor.lisp (file)
Remove the token with the given ID if it is not part of a multi-word token and it does not contain childs. It returns two values, the sentence (changed or not) and a boolean (nil if the sentence was not changed and true if changed. If the removed token is the root of the sentence, a new root must be provided.
editor.lisp (file)
editor.lisp (file)
editor.lisp (file)
editor.lisp (file)
rules.lisp (file)
rules.lisp (file)
rules.lisp (file)
rules.lisp (file)
query.lisp (file)
command-line.lisp (file)
Score by sentence (macro-average).
This is the mean of the percentage of words in each sentence
that are correct with respect to the fields in FIELDS.
We assume that LIST-SENT1 is the classified result
and LIST-SENT2 is the list of golden (correct) sentences.
If IGNORE-PUNCT, tokens which have ’PUNCT’ as upostag
in the golden sentences are ignored.
evaluate.lisp (file)
Returns a list of differences in SENT1 and SENT2.
They must have the same size.
If IGNORE-PUNCT, tokens which have ’PUNCT’ as upostag in sent2 are ignored.
evaluate.lisp (file)
editor.lisp (file)
editor.lisp (file)
evaluate.lisp (file)
editor.lisp (file)
Assume token’s MISC is a list of key-value pairs, return value corresponding to key MISC-KEY.
conllu-prolog.lisp (file)
query.lisp (file)
rdf.lisp (file)
draw.lisp (file)
conllu-prolog.lisp (file)
rules.lisp (file)
rules.lisp (file)
Score by word (micro-average).
This is the total mean of words in all sentences that
are correct with respect to the fields in FIELDS.
We assume that LIST-SENT1 is the classified result and LIST-SENT2 is the list of golden (correct) sentences.
If IGNORE-PUNCT, tokens which have ’PUNCT’ as upostag in the golden sentences are ignored.
evaluate.lisp (file)
Auxiliary function for format-html for confusion-matrix.
html.lisp (file)
conllu-prolog.lisp (file)
Auxiliary function for format-html for the confusion-matrix CM.
html.lisp (file)
Returns a sentence whose ’sent_id’ from the list of sentences in ’filename’.
command-line.lisp (file)
read-write.lisp (file)
tag-converter.lisp (file)
Next: Internal conditions, Previous: Internal functions, Up: Internal definitions [Contents][Index]
Function used to label each token.
confusion-matrix.lisp (file)
Parameter which contains the contents of the confusion matrix.
confusion-matrix.lisp (file)
Function which sorts labels. By default,
converts labels to string and uses lexicographical order
confusion-matrix.lisp (file)
Function which compares two labels. Typically a form of equality.
confusion-matrix.lisp (file)
editor.lisp (file)
niceline.lisp (file)
write a token to a line in 10 tab-separated columns.
read-write.lisp (file)
Next: Internal classes, Previous: Internal generic functions, Up: Internal definitions [Contents][Index]
niceline.lisp (file)
error (condition)
malformed-line (method)
:text
malformed-line (generic function)
editor.lisp (file)
error (condition)
index (method)
:index
index (generic function)
Previous: Internal conditions, Up: Internal definitions [Contents][Index]
data.lisp (file)
standard-object (class)
token-sentence (generic function)
(setf token-sentence) (generic function)
:lineno
token-lineno (generic function)
(setf token-lineno) (generic function)
data.lisp (file)
abstract-token (class)
:prev
etoken-prev (generic function)
(setf etoken-prev) (generic function)
:index
etoken-index (generic function)
(setf etoken-index) (generic function)
:form
etoken-form (generic function)
(setf etoken-form) (generic function)
:lemma
etoken-lemma (generic function)
(setf etoken-lemma) (generic function)
:upostag
"_"
etoken-upostag (generic function)
(setf etoken-upostag) (generic function)
:xpostag
"_"
etoken-xpostag (generic function)
(setf etoken-xpostag) (generic function)
:feats
"_"
etoken-feats (generic function)
(setf etoken-feats) (generic function)
:deps
"_"
etoken-deps (generic function)
(setf etoken-deps) (generic function)
:misc
"_"
etoken-misc (generic function)
(setf etoken-misc) (generic function)
Previous: Definitions, Up: Top [Contents][Index]
| • Concept index | ||
| • Function index | ||
| • Variable index | ||
| • Data type index |
Next: Function index, Previous: Indexes, Up: Indexes [Contents][Index]
| Jump to: | C F L |
|---|
| Jump to: | C F L |
|---|
Next: Variable index, Previous: Concept index, Up: Indexes [Contents][Index]
| Jump to: | (
A B C D E F G I J L M N O P Q R S T U V W |
|---|
| Jump to: | (
A B C D E F G I J L M N O P Q R S T U V W |
|---|
Next: Data type index, Previous: Function index, Up: Indexes [Contents][Index]
| Jump to: | *
C D E F H I K L M N P R S T U X |
|---|
| Jump to: | *
C D E F H I K L M N P R S T U X |
|---|
Previous: Variable index, Up: Indexes [Contents][Index]
| Jump to: | A C E M P S T |
|---|
| Jump to: | A C E M P S T |
|---|