This is the nfiles Reference Manual, version 1.0.0, generated automatically by Declt version 4.0 beta 2 "William Riker" on Thu Sep 15 05:43:35 2022 GMT+0.
Next: Files, Previous: The nfiles Reference Manual, Up: The nfiles Reference Manual [Contents][Index]
The main system appears first, followed by any subsystem dependency.
Manage file persistence and loading.
Atlas Engineer LLC
BSD 3-Clause
1.0.0
Next: Packages, Previous: Systems, Up: The nfiles Reference Manual [Contents][Index]
Files are sorted by type and then listed depth-first from the systems components trees.
Next: nfiles/pathname-helpers.lisp, Previous: Lisp, Up: Lisp [Contents][Index]
nfiles (system).
Next: nfiles/package.lisp, Previous: nfiles/nfiles.asd, Up: Lisp [Contents][Index]
nfiles (system).
Next: nfiles/conditions.lisp, Previous: nfiles/pathname-helpers.lisp, Up: Lisp [Contents][Index]
nfiles (system).
Next: nfiles/gpg.lisp, Previous: nfiles/package.lisp, Up: Lisp [Contents][Index]
nfiles (system).
read-error (condition).
Next: nfiles/nfiles.lisp, Previous: nfiles/conditions.lisp, Up: Lisp [Contents][Index]
nfiles (system).
Previous: nfiles/gpg.lisp, Up: Lisp [Contents][Index]
nfiles (system).
Next: Definitions, Previous: Files, Up: The nfiles Reference Manual [Contents][Index]
Packages are listed by definition order.
Some ‘cl:pathname’ helpers.
common-lisp.
Next: nfiles/gpg, Previous: nfiles/pathname, Up: Packages [Contents][Index]
The main data structures are ‘nfiles:file’ and ‘nfiles:profile’.
Call ‘nfiles:expand’ to return the final ‘nfiles:file’ path.
Call ‘nfiles:content’ (setf-able) to get the ‘nfiles:file’ content.
A basic session:
(defvar *config-file* (make-instance ’nfiles:config-file :base-path #p"my-app/init.lisp"))
(nfiles:expand *config-file*)
; => #P"/home/johndoe/.config/my-app/init.lisp"
(setf (nfiles:content *config-file*) "Hello file!") ; The file is written to disk.
(nfiles:content *config-file*)
; => "Hello file!"
The following convenience macro ensures the file is updated when done with the
body:
(nfiles:with-file-content (content *config-file*)
(format t "Length: ~a~%" (length content))
(setf content (serapeum:string-replace "file" content "config")))
The ‘nfiles:with-paths’ helper allows for let-style bindings of the expanded paths:
(let ((file1 (make-instance ’nfiles:file))
(file2 (make-instance ’nfiles:file :base-path #p"alt")))
(nfiles:with-paths ((path1 file1)
(path2 file2))
(list path1 path2)))
Specialize ‘nfiles:resolve’ to configure how a file path is expanded depending
on the file type and the ‘nfiles:profile’.
The content serialization and deserialization can be specialized via the
‘nfiles:serialize’ and ‘nfiles:deserialize’ methods.
The file reading and writing can be specialized via the ‘nfiles:read-file’ and
‘nfiles:write-file’ methods. These specializations are in charge for calling
the (de)serialization methods.
A ‘nfiles:remote-file’ works the same but needs some specialization to handle
the remote fetching and checksum validation:
(defmethod nfiles:fetch ((profile nfiles:profile) (file remote-counter-file) &key)
(dex:get (nfiles:url file)))
;; Optional:
(defmethod nfiles:check ((profile nfiles:profile) (file remote-counter-file) content &key)
(let ((path (nfiles:expand file)))
(ironclad:byte-array-to-hex-string
(ironclad:digest-file :sha3 path))))
(let ((file (make-instance ’nfiles:remote-file
;; The URL to download from if the file is not found on disk.
:url (quri:uri "https://example.org")
;; Without base-path, the file won’t be saved to disk.
:base-path #p"/tmp/index.html"
:checksum "794df316afac91572b899b52b54f53f04ef71f275a01c44b776013573445868c95317fc4a173a973e90addec7792ff8b637bdd80b1a6c60b03814a6544652a90")))
;; On access, file is automatically downloaded if needed and the checksum is verified:
(nfiles:content file)
;; ...
)
A word of caution: sometimes you may need the handle both the raw form and the
deserialized form. Then what should ‘nfiles:content’ return?
In this use case, your class has 2 semantic values: that of carrying the raw
form and that of being an ‘nfiels:file’ which carries the deserialized form.
To keep things simple, use a dedicated slot to store the raw form.
For instance ‘nfiles:remote-file’ has a ‘nfiles:url-content’ slot. If your file
is stored remotely, a ‘nfiles:remote-file’ gives access to both forms.
Also avoid calling ‘nfiles:content’ (or its setf-function) in the
‘nfiles:read-file’ - ‘nfiles:deserialize’ / ‘nfiles:write-file’ -
‘nfiles:serialize’ pipeline as it would make it horribly confusing to the
user (and probably break the logic).
A thin wrapper around the GPG command line tool. Do not expect too much from it.
common-lisp.
Next: Indexes, Previous: Packages, Up: The nfiles Reference Manual [Contents][Index]
Definitions are sorted by export status, category, package, and then by lexicographic order.
Next: Internals, Previous: Definitions, Up: Definitions [Contents][Index]
Next: Macros, Previous: Public Interface, Up: Public Interface [Contents][Index]
Next: Ordinary functions, Previous: Special variables, Up: Public Interface [Contents][Index]
Bind CONTENT to FILE’s content in BODY.
In case there’s no content, bind CONTENT to DEFAULT.
The new value of CONTENT is saved to FILE on exit.
Trivial wrapper around ‘call-with-gpg-file’.
Bind let-style BINDINGS to ‘file’ path expansions, then run BODY if all these paths or non-nil after ‘nil-pathname-p’.
Next: Generic functions, Previous: Macros, Up: Public Interface [Contents][Index]
Return all instantiated ‘file’ objects belonging to PACKAGES. If no package is specified, return all ‘file’ objects.
Rename PATH to a file of the same name with a unique suffix appended. The file is guaranteed to not conflict with any existing file.
Return the basename, that is:
- if it’s a directory, the name of the directory,
- if it’s a file, the name of the file including its type (extension),
- nil if it’s a nil-pathname (#p"").
Return the content of FILE.
When FORCE-READ is non-nil, the cache is skipped and the file is re-read.
When FORCE-UPDATE is non-nil, the file is re-downloaded if it’s a ‘remote-file’.
When SKIP-UPDATE is non-nil, the file is not re-downloaded if it’s present locally.
The read is asynchronous. By default, ‘content’ waits for the read to finish
before returning the result. But if ‘wait-p’ is nil, it returns directly
with (VALUES NIL THREAD) if the reading THREAD is not done yet.
Set FILE content to VALUE and persist change to disk.
While the content of the FILE object is updated instantly, the file is persisted
in the background.
This is a user-facing function, you should probably not try to call it from one
of the specialized methods as this would set the logic upside down.
Return a ‘bt:thread’ object. Call ‘bt:join-thread’ on it to know when it’s done writing the file.
Like ‘uiop:directory-pathname-p’ but also return T if ‘pathname-name’ is "." and check for existence.
Return PATH with type set to TYPE, if it’s not already the case. Case is ignored.
Return the pathname corresponding to FILE and its ‘profile’.
It’s a convenience wrapper around ‘resolve’ (to avoid specifying the ‘profile’).
Return PATH group name.
Set PATH group to NEW-GROUP (a string).
Return PATH owner name.
Set PATH owner to NEW-USER (a string).
Return list of private ‘gpg-key’s.
Concatenate PATHS.
Return non-nil if PATHNAME is ‘uiop:*nil-pathname*’ or nil.
Return the parent directory of PATH.
Like ‘pathname-type’ but return NIL instead of "" or :UNSPECIFIC.
Return a list of permissions as per ‘+permissions+’.
Set the PERMISSIONS or PATH as per ‘+permissions+’.
Next: Standalone methods, Previous: Ordinary functions, Up: Public Interface [Contents][Index]
Check CONTENT when FILE’s ‘checksum’ is non-empty.
This is meant to return a string which is then automatically compared to ‘checksum’.
If non-empty, the ‘check’ method is called on download against the resulting data.
If it does not match the ‘checksum’, raise an error.
This probably only makes sense for immutable data, thus ‘update-interval’ ought to be 0.
Transform STREAM into a useful form ready to be manipulated on the Lisp side. The STREAM is provided by ‘read-file’.
See ‘serialize’ for the reverse action.
Download the FILE ‘url’ and return the result as a string.
What to do on deserialization error.
The offending file may be backed up with the ‘backup’ function.
Or it may simply be ‘delete’d.
‘ask’ leaves the condition unhandled, so unless you handle it, it will prompt
the debugger with the other options.
What to do when the file download failed.
What to do when the downloaded content does not match ‘checksum’.
Load FILE by calling ‘deserialize’ on a stream opened on the file. The specialized method is in charge for calling ‘deserialize’. See ‘write-file’ for the reverse action.
Don’t load anything when using the ‘virtual-profile’.
Don’t load anything for virtual files.
Decrypt FILE with GPG and return resulting stream. See ‘*gpg-default-recipient*’.
Open FILE and call ‘deserialize’ on its content.
Try to download the file from its ‘url’ if it does not exist.
If file is already on disk and younger than ‘update-interval’, call next ‘read-file’ method instead of fetching online.
Return the final expanded path for ‘file’ depending on its ‘profile’.
This method is meant to be specialized against the user-defined ‘profile’s and ‘file’s.
See ‘expand’ for a convenience wrapper.
Ensure the extension is .lisp or .lisp.gpg.
Transform ‘file’ content in a form meant to be persisted to a file.
For the actual writing to disk, see ‘write-file’.
See ‘deserialize’ for the reverse action.
Re-download the file after this amount of seconds has
elapsed since last update.
If 0, disable automatic re-download.
URL from where to download the file.
If empty, then this behaves just like a regular ‘file’.
url.
The content of the downloaded file.
This is useful if you want to maintain both the raw content and the deserialized
content.
Persist FILE to disk.
DESTINATION is set by default to a staged pathname (using
‘uiop:with-staging-pathname’) which is renamed to the final name (the result of
‘expand’ on FILE) if everything went well.
This guarantees that on error the original file is left untouched.
The specialized method is in charge for calling ‘serialize’.
See ‘read-file’ for the reverse action.
Don’t write anything when using the ‘read-only-profile’.
Don’t write anything for ‘read-only-file’.
Crypt to FILE with GPG. See ‘*gpg-default-recipient*’.
Next: Conditions, Previous: Generic functions, Up: Public Interface [Contents][Index]
Next: Classes, Previous: Standalone methods, Up: Public Interface [Contents][Index]
error.
:path
subprocess-error.
:message
Previous: Conditions, Up: Public Interface [Contents][Index]
Like regular ‘file’ but set directory to ‘uiop:xdg-cache-home’
file.
Like regular ‘file’ but set directory to ‘uiop:xdg-config-home’.
file.
Like regular ‘file’ but set directory to ‘uiop:xdg-data-home’
file.
The main object to manipulate and subclass.
The ‘profile’ slot can be used to drive the specializations of multiple ‘file’
methods. See ‘resolve’, ‘serialize’, etc.
The ‘name’ slot can be used to refer to ‘file’ objects in a human-readable fashion.
The pathname used by the ‘resolve’ method to yield the final path.
pathname
#p"."
This slot is read-only.
The ‘profile’ of the ‘file’.
File expansion is performed against it.
The profile is only set at instantiation time.
nfiles:profile
nfiles:*default-profile*
:profile
This slot is read-only.
Name used to identify the object in a human-readable manner.
string
""
:name
name.
Function of one argument, the condition that may be raised in the reader thread.
trivial-types:function-designator
(function identity)
:read-handler
Function of one argument, the condition that may be raised in the writer thread.
trivial-types:function-designator
(function identity)
:write-handler
Whether to reload or overwrite the file if it was modified since it was last loaded.
(member nfiles:ask nfiles:reload nfiles:overwrite)
(quote nfiles:ask)
:on-external-modification
What to do on deserialization error.
The offending file may be backed up with the ‘backup’ function.
Or it may simply be ‘delete’d.
‘ask’ leaves the condition unhandled, so unless you handle it, it will prompt
the debugger with the other options.
(member nfiles:ask nfiles:backup delete)
(quote nfiles:ask)
:on-deserialization-error
What to do on file read error.
See ‘on-deserialization-error’ for the meaning of the different actions.
(member nfiles:ask nfiles:backup delete)
(quote nfiles:ask)
:on-read-error
If the resolved path has the GPG type (extension), the file is automatically encrypted and decrypted using the specified recipient key.
See ‘nfiles/gpg:*gpg-default-recipient*’.
file.
:key-length
:algorithm
:key-id
Stored as in Unix format.
:creation-date
Stored as in Unix format.
:expiry-date
:fingerprint
:keygrip
If the resolved path has the GPG type (extension), the file is automatically encrypted and decrypted using the specified recipient key.
The ’.lisp’ extension is automatically added if missing.
Like regular ‘file’ but assume a ‘.lisp’ extension, even if not provided.
file.
This is the default profile.
Subclass this profile to make your own, possibly carrying more data.
‘file’ path expansion is specialized against its ‘profile’ slot through the ‘resolve’ method.
File that’s not written to on change.
Note that the file’s ‘content’ can still be modified in-memory.
file.
In this profile, files by default don’t get written to.
File which can be found on a remote system or online.
If the local file is not found, then an attempt is made at downloading the file using ‘fetch’.
file.
URL from where to download the file.
If empty, then this behaves just like a regular ‘file’.
quri.uri:uri
(quri.uri:uri "")
:url
url.
The content of the downloaded file.
This is useful if you want to maintain both the raw content and the deserialized
content.
string
""
:url-content
Last time the file was downloaded.
unsigned-byte
(get-universal-time)
:last-update
This slot is read-only.
Re-download the file after this amount of seconds has
elapsed since last update.
If 0, disable automatic re-download.
unsigned-byte
0
:update-interval
If non-empty, the ‘check’ method is called on download against the resulting data.
If it does not match the ‘checksum’, raise an error.
This probably only makes sense for immutable data, thus ‘update-interval’ ought to be 0.
string
""
:checksum
What to do when the downloaded content does not match ‘checksum’.
(member nfiles:ask nfiles:ignore-checksum nfiles:discard)
(quote nfiles:ask)
:on-invalid-checksum
What to do when the file download failed.
(member nfiles:ask)
(quote nfiles:ask)
:on-fetch-error
Like regular ‘file’ but set directory to ‘uiop:xdg-runtime-dir’
file.
File that’s not read nor written to. It’s meant to handle data
in-memory.
Note that if multiple ‘virtual-file’s expand to the same path, they share the
same content.
To disable content-sharing for a specific ‘file’, their ‘resolve’ method should return ‘uiop:*nil-pathname*’.
In this profile, files are neither read nor written to by default.
Previous: Public Interface, Up: Definitions [Contents][Index]
Next: Special variables, Previous: Internals, Up: Internals [Contents][Index]
Internal ‘*cache*’ associating expanded paths with a dedicated ‘cache-entry’.
Implementation detail.
This is used by ‘remote-file’ to write content to disk when the ‘cache-entry’ is
initialized, without looping indefinitely.
Implementation detail.
This is used by ‘remote-file’ to write content to disk when the ‘cache-entry’ is
initialized, without looping indefinitely.
Set of all ‘file’ objects.
It’s a weak hash table so that garbage-collected files are automatically
removed.
Time in seconds to wait for other write requests.
Next: Ordinary functions, Previous: Special variables, Up: Internals [Contents][Index]
Next: Generic functions, Previous: Macros, Up: Internals [Contents][Index]
Call RESTART if valid.
Files that expand to ‘uiop:*nil-pathname*’ have their own cache entry.
Like ‘call-with-open-file’ but use ‘gpg’ to read and write to file.
OPTIONS are as for ‘open”s ‘:direction’.
Other options are not supported. File is overwritten if it exists, while
nothing is done if file is missing.
Decrement the semaphore counter down to 0.
Return the number of decrements, or NIL if there was none.
Return the key of FILE’s recipient if any, ‘*gpg-default-recipient*’ otherwise.
As second value the email.
As third value the name.
Write STREAM to GPG-FILE using RECIPIENT key. If RECIPIENT is not provided, use default key.
Return the list of sections as a list of strings.
PROMPT takes no trailing colon nor trailing space.
Like ‘uiop:run-program’ but raise a ‘nfiles:process-error’ on error.
Return non-nil if pathname is special like "." or "..". Note that this returns nil on #p"." and #p"..".
Small convenience function to check whether the given URL is empty.
Next: Conditions, Previous: Ordinary functions, Up: Internals [Contents][Index]
automatically generated reader method
automatically generated writer method
The date at which the cache entry was last updated.
Last time the file was downloaded.
This is the ‘file’ object that was used to instantiate the entry’s ‘cached-value’.
automatically generated reader method
automatically generated writer method
automatically generated reader method
automatically generated writer method
Next: Classes, Previous: Generic functions, Up: Internals [Contents][Index]
This condition is used internally to cancel a file read.
error.
Previous: Conditions, Up: Internals [Contents][Index]
This is the ‘file’ object that was used to instantiate the entry’s ‘cached-value’.
nfiles:file
(error "source file must be given.")
:source-file
The date at which the cache entry was last updated.
unsigned-byte
(get-universal-time)
:last-update
:cached-value
(or null bordeaux-threads:thread)
:worker
(or null bordeaux-threads:semaphore)
:worker-notifier
Previous: Definitions, Up: The nfiles Reference Manual [Contents][Index]
| Jump to: | (
A B C D E F G I J K L M N O P R S U V W |
|---|
| Jump to: | (
A B C D E F G I J K L M N O P R S U V W |
|---|
Next: Data types, Previous: Functions, Up: Indexes [Contents][Index]
| Jump to: | *
+
A B C E F K L M N O P R S U V W |
|---|
| Jump to: | *
+
A B C E F K L M N O P R S U V W |
|---|
| Jump to: | C D E F G I L N P R S V |
|---|
| Jump to: | C D E F G I L N P R S V |
|---|