This is the software-evolution-library Reference Manual, version 0.0.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Mon Feb 26 17:51:25 2024 GMT+0.
The main system appears first, followed by any subsystem dependency.
software-evolution-librarysoftware-evolution-library/software-evolution-librarysoftware-evolution-library/utility/gitsoftware-evolution-libraryprogrammatic modification and evaluation of extant software
Eric Schulte and GrammaTech
GPL V3
A common interface abstracts over multiple
types of software objects including abstract syntax trees parsed from
source code, LLVM IR, compiled assembler, and linked ELF binaries.
Mutation and evaluation methods are implemented on top of this
interface supporting Search Based Software Engineering (SBSE)
techniques.
0.0.0
asdf-package-system (system).
software-evolution-library/software-evolution-library (system).
software-evolution-library/software-evolution-libraryEric Schulte and GrammaTech
GPL V3
gt/full (system).
software-evolution-library/utility/git (system).
atomics (system).
Files are sorted by type and then listed depth-first from the systems components trees.
software-evolution-library/software-evolution-library.asdsoftware-evolution-library/software-evolution-library/file-type.lispsoftware-evolution-library/utility/git/file-type.lispsoftware-evolution-library/software-evolution-library.asdsoftware-evolution-library (system).
software-evolution-library/software-evolution-library/file-type.lispsoftware-evolution-library/software-evolution-library (system).
*cross-chance* (special variable).
*crossover-stats* (special variable).
*elitism* (special variable).
*fitness-evals* (special variable).
*fitness-predicate* (special variable).
*fitness-scalar-fn* (special variable).
*generations* (special variable).
*max-population-size* (special variable).
*mcmc-fodder* (special variable).
*mut-rate* (special variable).
*mutation-improvements* (special variable).
*mutation-stats* (special variable).
*population* (special variable).
*running* (special variable).
*start-time* (special variable).
*target-fitness-p* (special variable).
*tie-breaker-predicate* (special variable).
*tournament-eviction-size* (special variable).
*tournament-selector* (special variable).
*tournament-size* (special variable).
*tournament-tie-breaker* (special variable).
*worst-fitness* (special variable).
*worst-fitness-p* (special variable).
+software-evolution-library-branch+ (special variable).
+software-evolution-library-dir+ (special variable).
+software-evolution-library-major-version+ (constant).
+software-evolution-library-version+ (special variable).
analyze-mutation (function).
apply-all-mutations (generic function).
apply-mutation (generic function).
apply-mutations (generic function).
apply-path (generic function).
apply-picked-mutations (generic function).
at-targets (generic function).
build-op (generic function).
classify (function).
compose-mutations (macro).
copy (method).
create-super (generic function).
crossed (function).
crossover (generic function).
default-random-winner (function).
default-select-best (function).
define-mutation (macro).
define-software (macro).
elapsed-time (function).
evaluate (generic function).
evict (function).
evolve (macro).
fitness (reader method).
(setf fitness) (writer method).
fitness-better-p (function).
fitness-equal-p (function).
fitness-extra-data (generic function).
(setf fitness-extra-data) (generic function).
fitness-scalar (function).
from-file (generic function).
from-string (generic function).
generational-evolve (function).
genome (generic function).
genome-string (generic function).
get-targets (reader method).
ignore-phenome-errors (macro).
incorporate (function).
interpreted-phenome (function).
interpreted-phenome (class).
log-message (generic function).
log-new-individuals (generic function).
mcmc (macro).
mcmc-step (generic function).
mutant (function).
mutate (generic function).
mutate (condition).
mutation (class).
mutation-key (generic function).
new-individual (function).
no-mutation-targets (condition).
object (reader method).
(setf object) (writer method).
oid-object (class).
one-point-crossover (generic function).
phenome (generic function).
phenome (condition).
phenome-p (generic function).
pick (generic function).
pick-bad (generic function).
pick-bad-bad (generic function).
pick-bad-good (generic function).
pick-bad-only (generic function).
pick-good (generic function).
pick-mutation-type (generic function).
picker (generic function).
print-object (method).
sequence-mutations (macro).
simple-evaluate (function).
simple-reproduce (function).
simple-select (function).
size (method).
software (class).
summarize-mutation-stats (function).
targeter (generic function).
targets (generic function).
text (reader method).
text (reader method).
text (reader method).
to-file (generic function).
tournament (function).
two-point-crossover (generic function).
worst-numeric-fitness (function).
worst-numeric-fitness-p (function).
*max-saved-mutation-improvements* (special variable).
-search (function).
add-elite-individuals (function).
apply-config (generic function).
call-period-fn (function).
continue-evolutionary-loop-p (function).
create-and-populate-super (function).
deinitialize-evolutionary-loop (function).
expression (generic function).
generate-oid (function).
initialize-evolutionary-loop (function).
loc (reader method).
new-individuals (function).
obj (reader method).
obj (reader method).
obj (reader method).
oid (reader method).
operation (reader method).
operation (reader method).
remove-elite-individuals (function).
validate-evolution-parameters (function).
software-evolution-library/utility/git/file-type.lispsoftware-evolution-library/utility/git (system).
current-git-branch (generic function).
current-git-commit (generic function).
current-git-status (generic function).
git (class).
git-dir (reader method).
(setf git-dir) (writer method).
git-error (condition).
git-url-p (function).
initialize-instance (method).
make-git (function).
run (generic function).
ssh-key (reader method).
(setf ssh-key) (writer method).
work-tree (reader method).
(setf work-tree) (writer method).
define-direct-git-command (macro).
description (reader method).
git-from-directory (function).
Packages are listed by definition order.
software-evolution-library/utility/gitsel/utility/git
gt/full.
current-git-branch (generic function).
current-git-commit (generic function).
current-git-status (generic function).
git (class).
git-dir (generic reader).
(setf git-dir) (generic writer).
git-error (condition).
git-url-p (function).
make-git (function).
run (generic function).
ssh-key (generic reader).
(setf ssh-key) (generic writer).
work-tree (generic reader).
(setf work-tree) (generic writer).
define-direct-git-command (macro).
description (generic reader).
git-from-directory (function).
software-evolution-library/software-evolution-librarysel
software-evolution-library
gt/full.
software-evolution-library/utility/git.
*cross-chance* (special variable).
*crossover-stats* (special variable).
*elitism* (special variable).
*fitness-evals* (special variable).
*fitness-predicate* (special variable).
*fitness-scalar-fn* (special variable).
*generations* (special variable).
*max-population-size* (special variable).
*mcmc-fodder* (special variable).
*mut-rate* (special variable).
*mutation-improvements* (special variable).
*mutation-stats* (special variable).
*population* (special variable).
*running* (special variable).
*start-time* (special variable).
*target-fitness-p* (special variable).
*tie-breaker-predicate* (special variable).
*tournament-eviction-size* (special variable).
*tournament-selector* (special variable).
*tournament-size* (special variable).
*tournament-tie-breaker* (special variable).
*worst-fitness* (special variable).
*worst-fitness-p* (special variable).
+software-evolution-library-branch+ (special variable).
+software-evolution-library-dir+ (special variable).
+software-evolution-library-major-version+ (constant).
+software-evolution-library-version+ (special variable).
analyze-mutation (function).
apply-all-mutations (generic function).
apply-mutation (generic function).
apply-mutations (generic function).
apply-path (generic function).
apply-picked-mutations (generic function).
at-targets (generic function).
build-op (generic function).
classify (function).
compose-mutations (macro).
create-super (generic function).
crossed (function).
crossover (generic function).
default-random-winner (function).
default-select-best (function).
define-mutation (macro).
define-software (macro).
elapsed-time (function).
evaluate (generic function).
evict (function).
evolve (macro).
fitness (generic reader).
(setf fitness) (generic writer).
fitness-better-p (function).
fitness-equal-p (function).
fitness-extra-data (generic function).
(setf fitness-extra-data) (generic function).
fitness-scalar (function).
from-file (generic function).
from-string (generic function).
generational-evolve (function).
genome (generic function).
genome-string (generic function).
get-targets (generic reader).
ignore-phenome-errors (macro).
incorporate (function).
interpreted-phenome (function).
interpreted-phenome (class).
log-message (generic function).
log-new-individuals (generic function).
mcmc (macro).
mcmc-step (generic function).
mutant (function).
mutate (generic function).
mutate (condition).
mutation (class).
mutation-key (generic function).
new-individual (function).
no-mutation-targets (condition).
object (generic reader).
(setf object) (generic writer).
oid-object (class).
one-point-crossover (generic function).
phenome (generic function).
phenome (condition).
phenome-p (generic function).
pick (generic function).
pick-bad (generic function).
pick-bad-bad (generic function).
pick-bad-good (generic function).
pick-bad-only (generic function).
pick-good (generic function).
pick-mutation-type (generic function).
picker (generic function).
sequence-mutations (macro).
simple-evaluate (function).
simple-reproduce (function).
simple-select (function).
software (class).
summarize-mutation-stats (function).
targeter (generic function).
targets (generic function).
text (generic reader).
to-file (generic function).
tournament (function).
two-point-crossover (generic function).
worst-numeric-fitness (function).
worst-numeric-fitness-p (function).
*max-saved-mutation-improvements* (special variable).
-search (function).
add-elite-individuals (function).
apply-config (generic function).
call-period-fn (function).
continue-evolutionary-loop-p (function).
create-and-populate-super (function).
deinitialize-evolutionary-loop (function).
expression (generic function).
generate-oid (function).
initialize-evolutionary-loop (function).
loc (generic reader).
new-individuals (function).
obj (generic reader).
oid (generic reader).
operation (generic reader).
remove-elite-individuals (function).
validate-evolution-parameters (function).
Definitions are sorted by export status, category, package, and then by lexicographic order.
Current major version of the SOFTWARE-EVOLUTION-LIBRARY (without git hash).
Fraction of new individuals generated using crossover rather than mutation.
Variable to hold crossover statistics.
Number of individuals to automatically promote to next population.
Range: 0..(- (length *population*) 1)
When evolving super-mutants, or calling generational-evolve,
*ELITISM* specifies the number of individuals which are automatically
promoted prior to typical generational replacement or eviction
process. The selected individuals will be the ones with the
best fitness.
When using super-mutants, the *ELITISM* value will reduce the number of new
individuals created in each generation by the value of *ELITISM* (since this
number will automatically be promoted).
Track the total number of fitness evaluations.
Function to compare two fitness values to select which is preferred.
Function to convert fitness to a numeric value
Holds the running generation count.
Maximum allowable population size.
Holds the genome elements which may be used by ‘mcmc-step’. Should be initialized to a list of the unique possible genome elements.
Chance to mutate a new individual.
* If <1, new individuals will be mutated once with change *MUT-RATE*.
* If =1, then every new individual will be mutated exactly once.
* If >1, then new individuals will be mutated from 1 to *MUT-RATE* times.
List of recent mutation improvements cons’d with *fitness-evals*.
Variable to hold mutation statistics.
Holds the variant programs to be evolved.
This variable may be read to inspect a running search process, or
written to as part of a running search process.
True when a search process is running, set to nil to stop evolution.
Holds the start time of evolutionary processes.
Predicate indicating whether an individual has reached the target fitness.
Function to compare two tie breaker values to select which is preferred.
Number of individuals to participate in eviction tournaments.
Function used to select winners of a tournament. Returns a list of winners.
Number of individuals to participate in tournament selection.
Function used to break ties in a tournament. Returns a single winner.
Default worst fitness TODO.
Predicate indicating whether an individual has the worst possible fitness.
Current branch of the SOFTWARE-EVOLUTION-LIBRARY.
Path to directory holding SOFTWARE-EVOLUTION-LIBRARY.
Current version of the SOFTWARE-EVOLUTION-LIBRARY.
Define a new mutation named CLASS-NAME composing MUTATIONS. MUTATIONS is a list of the names of mutation classes.
Like ‘defclass’ but inherits TARGETER slot-options from MUTATION. Also, ensures MUTATION is a member of superclasses
Define a new ‘software’ class NAME including a deep ‘copy’ method.
Arguments NAME DIRECT-SUPERCLASSES and OPTIONS are passed through to
‘defclass’ unmodified. Additional optional :COPIER property on each
slot in DIRECT-SLOTS may be one of the following:
* :NONE this slot is not copied and will be nil in the new object
* :DIRECT this slot is copied by direct reference to the slot value
skipping the accessor
* otherwise if the value of :COPIER is nil (default) then the slot is
copied through its accessor, otherwise the value is
assumed to be a function (e.g., ‘copy-tree’) which is used
to copy the slot.
Evolves ‘*population*’ using ‘new-individual’ and TEST.
* SUPER-MUTANT-COUNT evaluate this number of mutants at once in a
combined genome.
Other keyword arguments are used as defined in the ‘-search’ function.
Handle errors in ‘phenome’ execution by returning nil as the first value from the ‘phenome’ method.
MCMC search from ORIGINAL using ‘mcmc-step’ and TEST.
If keyword argument ACCEPT-FN is given it is used to determine when a
newly found candidate replaces the current candidate. If ACCEPT-FN is
not supplied MCMC defaults to using Metropolis Hastings.
Other keyword arguments are used as defined in the ‘-search’ function.
Define a new mutation named CLASS-NAME sequencing MUT-A and MUT-B.
MUT-A and MUT-B are instances of mutations. Instead of collecting
targets for A and then targets for B and then applying A and B as done
by ‘compose-mutations’, ‘sequence-mutations’ first targets and applies A and then targets and applied B.
Default function to collect statistics from an applied mutation.
This function will calculate the improvements to the fitness of SOFTWARE as the result of crossover and mutation using ‘evaluate’ and TEST. Each crossover and mutation will be paired with one of the following tags; :dead, :same, :worse, or :better. Calculated stats will be added to the *crossover-stats* and *mutation-stats* variables for analysis.
Classify the fitness of NEW as :BETTER, :WORSE, :SAME, or :DEAD when compared to OLD. NEW and OLD must have fitness populated.
Generate a new individual from *POPULATION* using crossover.
Choose a random winner from GROUP.
Return the members of GROUP with most PREDICATE fitness. Default selection function for ‘tournament’.
Check if FITNESS-A is strictly better than FITNESS-B.
Return true if FITNESS-A and FITNESS-B are equal
Evolves ‘*population*’ using REPRODUCE EVALUATE-POP and SELECT.
Required arguments are as follows:
REPRODUCE ———– create new individuals from the current population
EVALUATE-POP ——– evaluate-pop the entire population
SELECT ————– select best individuals from the population
Keyword arguments are as follows:
MAX-GENERATIONS —– stop after this many generations
MAX-EVALS ———– stop after this many fitness evaluations
MAX-TIME ———— stop after this many seconds
PERIOD ————– interval of generations evaluations to run PERIOD-FN
PERIOD-FN ———– function to run every PERIOD generations
EVERY-PRE-FN ——– function to run on each new individual before evaluation
EVERY-POST-FN ——- function to run on each new individual after evaluation
ANALYZE-MUTATION-FN - function to call to analyze mutation results
TEST —————- fitness test function for mutation statistics
FILTER ————– remove individuals for which FILTER returns false
Note that REPRODUCE should handle any errors that occur from failed mutations. This differs from how evolve works.
Return nil if URL does not look like a URL to a git valid remote.
Incorporate SOFTWARE into POPULATION, keeping POPULATION size constant.
Create a phenotype of the interpreted software OBJ.
Make a git object in LOCAL-DIRECTORY.
If REMOTE is specified clone from REMOTE into LOCAL-DIRECTORY. If
LOCAL-DIRECTORY exists it should be part of or under a git repository.
If LOCAL-DIRECTORY does not exist it will be created and a fresh git
repository will be initialized therein. A username and password may
be specified as part of the REMOTE URL. E.g. as
’https://USERNAME:PASSWORD@github.com/path/to/repo.git’.
Generate a new mutant from a *POPULATION*.
Generate a new individual from *POPULATION*.
Evaluate NEW-CHILDREN using TEST assigning each a fitness.
Reproduce using every individual in POPULATION.
Return a list of the resulting children and as optional extra value a
list of the mutations applied to produce those children.
DOCFIXME
* RESULTS DOCFIXME
Select an individual from *POPULATION* with a tournament.
Apply MUTATION to every target in SOFTWARE.
Returns the resulting software objects. Returns a list of the applied
mutations as an optional second value.
Apply MUTATION to SOFTWARE, return the resulting software object.
Mutation application may destructively modify the software object, or it may return a
new instance with the mutation applied, and leave the original untouched. Any client
which calls apply-mutation should ensure that the result returned by apply-mutation is
captured, and should not make assumptions about the state of the original.
Example: (let ((mutated-software (apply-mutation (copy software) mutation)))
...
Define an :around method on this function to record mutations.
Apply MUTATION to the first N targets in SOFTWARE.
Returns the resulting software objects. Returns a list of the applied
mutations as an optional second value.
Apply the execution trace PATH behind KEY in SOFTWARE.
Apply MUTATION to N randomly selected targets in SOFTWARE.
Returns the resulting software objects. Returns a list of the applied
mutations as an optional second value.
Return a copy of MUTATION with ‘targets’ set to TARGETS.
Build operation on SOFTWARE from a MUTATION.
Create a super-mutant populated with VARIANT . REST-VARIANTS
Crossover two software objects.
Define an :around method on this function to record crossovers.
Return the current git branch by directly reading git data on disk.
Return the current git commit by directly reading git data on disk.
Return the git status of DIRECTORY as a list of lists of (status file). Return nil if there are no modified, untracked, or deleted files.
Evaluate the software returning a numerical fitness.
Hold extra data returned by the fitness function.
Pass extra data (optionally) returned by the fitness function to the software object.
Initialize SOFTWARE with contents of FILE.
symbol) file) ¶Initialize SOFTWARE with contents of STRING.
symbol) string) ¶The software genotype or “code”, exposed as a simplified data structure. For example an AST genome, e.g., of a ‘cil’ or ‘lisp’ software object my have a tree structure while the genome of an ‘asm’ or ‘llvm’ software object will be a vector.
Return a string of the ‘genome’ of SOFTWARE.
Generate a log message for OBJECT.
Change software in a way amenable to MCMC.
Specifically every step should be reversible, and the resulting walk
should be ergodic.
Mutate the software. May throw a ‘mutate’ error.
Key used to organize mutations in *mutation-stats*.
Crossover at a single point.
Phenotype of the software.
This method will link, compile or serialize the software object as
necessary returning an executable version of the software suitable for
testing and evaluation. Returns multiple values holding in order; (1)
the binary path to which the executable was compiled, (2) the errno,
or a numeric indication of success, of the compilation process, (3)
STDERR of the compilation process, or a string holding error output
relevant to phenome generation, (4) STDOUT of the compilation process,
or a string holding non-error output relevant to phenome
generation, (5) the source file name used during compilation.
interpreted-phenome) &key bin) ¶Return non-nil if SOFTWARE has a phenotype.
Pick an element of GENOME based on KEY of each element.
KEY is passed to ‘proportional-pick’ to return an index. Optional
argument FUNC processes the index to return a result.
Pick a ’bad’ index into a software object. Used to target mutation.
Pick two ’bad’ indexes into a software object. Used to target mutation.
Pick a ’bad’ and a ’good’ indexes into a software object. Used to target mutation.
Pick a single ’bad’ index into a software object. Used to target mutation.
Pick a ’good’ index into a software object. Used to target mutation.
Select a type of mutation to apply to SOFTWARE.
Reader for the PICKER slot of mutation objects
Run the git command COMMAND in the git repository GIT.
Reader for the TARGETER slot of mutation objects
Return all possible targets of MUTATION.
Write SOFTWARE to FILE.
Crossover between two points.
software)) ¶Return the size of the ‘genome’ of SOFTWARE.
fset.
error.
(quote nil)
:description
This slot is read-only.
Mutation errors are thrown when a mutation fails. These may often be safely ignored. A common restart is ‘ignore-failed-mutation’.
error.
This is a particularly benign form of mutation error. A common restart is ‘ignore-failed-mutation’.
DOCFIXME
error.
An object to represent a git repository.
Mixin for an interpreted phenome.
The base class of all software mutations.
The software object to be mutated.
(or software-evolution-library/software-evolution-library:software null)
:object
A calculated target set.
:targets
This slot is read-only.
Attaches a unique oid (object identifier) to each instance.
oid.
Base class for all software objects.
apply-all-mutations.
apply-mutation.
apply-mutations.
apply-picked-mutations.
copy.
crossover.
evaluate.
evaluate.
(setf fitness).
fitness.
(setf fitness-extra-data).
fitness-extra-data.
mutation-key.
mutation-key.
phenome-p.
pick.
pick-bad.
pick-bad-bad.
pick-bad-good.
pick-bad-only.
pick-good.
size.
to-file.
to-file.
:fitness
Maximum number of mutation improvements to hold on to.
Define a git command which reads directly from the file system. These commands don’t actually have to invoke git commands directly.
Perform a search loop with early termination.
Required arguments are as follows:
EVALUATE-FN ——— Test function used to ‘evaluate’ every VARIANT.
REPRODUCE ———– Function to call on every iteration to get the variants
and mutation info as values from the current population.
ACTIONS ————- Actions to be performed in the core search loop after
the new individuals have been bound.
Keyword arguments are as follows:
MAX-EVALS ———– Maximum number of evaluations to perform.
MAX-TIME ———— Maximum time to run.
PERIOD ————– Period (in evals) at which to call PERIOD-FN.
PERIOD-FN ———– Function to call every Period evals.
EVERY-PRE-FN ——– Function to call before every evaluation.
EVERY-POST-FN ——- Function to call after every evaluation.
FILTER ————– Function to filter variants from BODY.
ANALYZE-MUTATION-FN - Function to call to analyze mutation results
The following global variables are implicitly updated by this function
and should be dynamically bound to perform multiple different
simultaneous searches, ‘*running*’, ‘*start-time*’, ‘*fitness-evals*’.
The global variable ‘*target-fitness-p*’ implicitly defines a stopping
criteria for this search.
Add ELITE-INDIVIDUALS to the population.
NEW-POPULATION can be used if *POPULATION* doesn’t contain the current value.
Call the period-fn if needed. This is based on the updated CURRENT-EVALS and the EVALS-DELTA that was used to update it.
Return T if all of the common conditions are met for continuing the
evolutionary loop:
- *running* is t.
- Maximum number of evaluations has not been reached.
- Maximum time has not been reached.
- Maximum generations has not been reached.
Create and populate a super-mutant with supplied list of variants.
Deinitialize common special variables used by the evolutionary loop.
Create a fresh, unique oid (object id) in range [1 ...]
Return first parent directory of DIRECTORY that is a git repository. Return a second value which is the base git repository, the GIT_WORK_TREE. Raise an error if no such parent exists.
Initialize common special variables used by the evolutionary loop and assert their validity.
Generate COUNT new individuals from *POPULATION*.
Remove *ELITISM* individuals from population and return them.
Validate special variables that are used during the evolutionary loop.
Parse CONFIG-FILE and use to configure SOFTWARE.
Return WHAT in SOFTWARE as a lisp expression.
oid-object)) ¶automatically generated reader method
oid.
no-mutation-targets)) ¶| Jump to: | (
-
A B C D E F G I L M N O P R S T V W |
|---|
| Jump to: | (
-
A B C D E F G I L M N O P R S T V W |
|---|
| Jump to: | *
+
C D F G L O S T W |
|---|
| Jump to: | *
+
C D F G L O S T W |
|---|
| Jump to: | C F G I M N O P S |
|---|
| Jump to: | C F G I M N O P S |
|---|