The mmap Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The mmap Reference Manual

This is the mmap Reference Manual, version 1.0.0, generated automatically by Declt version 2.4 patchlevel 1 "Will Decker" on Mon Jul 29 16:15:57 2019 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

About MMAP

This is a utility library providing access to the mmap family of functions in a portable way. It should work on Posix and Windows systems. mmap allows you to directly map a file into the address space of your process without having to manually read it into memory sequentially. Typically this is much more efficient for files that are larger than a few Kb.

Supported operations

The library offers access to the following functions:

It also provides a convenience macro called with-mmap to perform safe, local mappings of files.

(mmap:with-mmap (addr fd size #p"/etc/lsb-release")
  (with-output-to-string (out)
    (loop for i from 0 below size
          for char = (code-char (cffi:mem-aref addr :char i))
          do (write-char char out))))

Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 mmap

Maintainer

Nicolas Hafner <shinmera@tymoon.eu>

Author

Nicolas Hafner <shinmera@tymoon.eu>

Home Page

https://shinmera.github.io/mmap/

Source Control

(:git "https://github.com/shinmera/mmap.git")

Bug Tracker

https://github.com/Shinmera/mmap/issues

License

zlib

Description

Portable mmap (file memory mapping) utility library.

Version

1.0.0

Defsystem Dependency

trivial-features

Dependencies
Source

mmap.asd (file)

Components

Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 mmap.asd

Location

mmap.asd

Systems

mmap (system)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 mmap/package.lisp

Parent

mmap (system)

Location

package.lisp

Packages

mmap


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.3 mmap/generic.lisp

Dependency

package.lisp (file)

Parent

mmap (system)

Location

generic.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.4 mmap/posix.lisp

If Feature

unix

Dependency

generic.lisp (file)

Parent

mmap (system)

Location

posix.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.5 mmap/windows.lisp

If Feature

windows

Dependency

posix.lisp (file)

Parent

mmap (system)

Location

windows.lisp


Previous: , Up: Lisp files   [Contents][Index]

3.1.6 mmap/documentation.lisp

Dependency

windows.lisp (file)

Parent

mmap (system)

Location

documentation.lisp


Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 mmap

Source

package.lisp (file)

Nickname

org.shirakumo.fraf.trial.mmap

Use List

common-lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Macros

Macro: with-mmap (ADDR FD SIZE PATH &rest ARGS) &body BODY

Map the file or number of bytes to a memory region within the body.

This is a convenience macro that calls MMAP with the given arguments, binds the results to the variables ADDR, FD, and SIZE, and automatically ensures that MUNMAP is called with the correct values when the body is exited.

It is safe to change the ADDR, FD, and SIZE bindings, though probably not very good style to do so. It is NOT safe to save the ADDR and SIZE values somewhere and use them outside of the dynamic scope of the body. Attempting to do so is very likely going to burn your process to the ground.

See MMAP
See MUNMAP

Package

mmap

Source

generic.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.2 Compiler macros

Compiler Macro: mmap PATH &key OPEN PROTECTION MMAP SIZE OFFSET
Package

mmap

Source

posix.lisp (file)

Compiler Macro: mprotect ADDR SIZE PROTECTION
Package

mmap

Source

posix.lisp (file)

Compiler Macro: msync ADDR FD SIZE &key FLAGS
Package

mmap

Source

posix.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.3 Functions

Function: mmap PATH &key OPEN PROTECTION MMAP SIZE OFFSET

Map the given path or number of bytes into the address space.

PATH can be either a pathname designator, or NIL. If it is NIL, an anonymous
file is mapped and the MMAP flag list must include the flag :ANONYMOUS.
If it is a path, then the contents of the given file on the file system are
mapped into the address space. The file contents can then be read, written,
or executed depending on the given flags as if normal memory was accessed.
If the file is NIL or its size cannot be automatically determined, you must
pass a valid SIZE. You may optionally pass an OFFSET (in bytes) into the
file from which the mapping begins.

If the map attempt fails, an error of type MMAP-ERROR is signalled.
If the call succeeds, three values are returned:

PTR — A CFFI:FOREIGN-POINTER that points to the start of the place in
memory where the file contents have been mapped. The contents
should be placed in increasing address order, unless the flag
:GROWS-DOWN is active.
FD — An opaque file descriptor. You should not touch this.
SIZE — The size of the region of memory that has been mapped in bytes.

All three values need to be passed on to MUNMAP completely unchanged. Any
change could cause severe issues.

The three options OPEN, PROTECTION, and MMAP are lists of flags. Not all of
those flags are portable, some are only allowed on Linux, some only on non-
Windows systems. To indicate support, the flags are marked as EVERY, POSIX
(non-Windows), LINUX, or WINDOWS.

OPEN
:READ — [EVERY] Opens the file for read access.
:WRITE — [EVERY] Opens the file for write access.
:CREATE — [EVERY] Creates the file if it does not exist yet.
:ENSURE-CREATE — [EVERY] Creates the file if it does not exist yet and
errors if it does.
:TRUNCATE — [EVERY] Truncates the file and replaces it if it exists.
:DIRECT — [EVERY] Causes system buffers to be bypassed.
:FILE-SYNC — [EVERY] Causes writes to the file to be flushed asap.
:DATA-SYNC — [POSIX] Similar to FILE-SYNC, but uses data integrity
semantics rather than file integrity semantics. :DONT-CLAIM-TTY— [POSIX] If the file is a tty and the process does not
already have a controlling tty, this file will
not become the process’ controlling tty.
:NON-BLOCK — [POSIX] Attempt to open the file in non-blocking mode,
causing operations on the fd to return asap.
:NO-FOLLOW — [LINUX] Errors if the file is a symlink.
:ASYNC — [LINUX] Enable signal driven IO.
:DIRECTORY — [LINUX] Errors if the file is not a directory.
:LARGE-FILE — [LINUX] Allows opening files with size not representable
by a 32 bit unsigned integer.

PROTECTION
:READ — [EVERY] Allows reading from the memory region. The OPEN
flag :READ is required for this protection mode.
This flag is required on windows.
:WRITE — [EVERY] Allows writing to the memory region.
:EXEC — [EVERY] Allows executing code in the memory region.
:NONE — [POSIX] Prevents accessing the memory region.

MMAP
:PRIVATE — [EVERY] The underlying file is not changed if the memory
area is written to. Copy-on-write is employed to
ensure separation.
:SHARED — [EVERY] The underlying file is changed if the memory
area is written to and the change will be
visible to other processes. In this case the
OPEN flag :WRITE must be specified.
:ANONYMOUS — [LINUX/WINDOWS] The path should be a number of bytes to
map to memory. The memory region is then mapped
against an "anonymous" file.
:NO-RESERVE — [LINUX] Don’t reserve swap for this mapping. If memory
runs out, a segfault will be generated instead.
:LOCKED — [LINUX] Locks the region to RAM, preventing it from
being swapped out.
:GROWS-DOWN — [LINUX] Causes the memory region to be mapped with a
decreasing address, like in a stack.
:POPULATE — [LINUX] Pre-populate the memory region with the file
contents, which can help performance.
:NON-BLOCK — [LINUX] Only useful with :POPULATE – do not perform a read-ahead.

The default values for the flags are:
:OPEN (:READ) :PROTECTION (:READ) :MMAP (:PRIVATE)

Note that if you are intending to use MPROTECT to change the protection of
the mapped file at a later date, you need to call MMAP with the maximal
combination of protection flags first. If this is not the protection that
you want to start out with, call MPROTECT with the correct combination
immediately after. For instance, if you would like to start with (:READ) and
later want to change it to (:READ :WRITE), call MMAP with (:READ :WRITE),
and immediately after call MPROTECT with (:READ).

See MUNMAP
See WITH-MMAP
See MMAP-ERROR
See http://pubs.opengroup.org/onlinepubs/7908799/xsh/mmap.html
See http://pubs.opengroup.org/onlinepubs/009604499/functions/stat.html
See http://man7.org/linux/man-pages/man2/mmap.2.html
See http://man7.org/linux/man-pages/man2/stat.2.html
See https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-createfilew See https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-getfilesize See https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-createfilemappinga See https://msdn.microsoft.com/en-us/library/windows/desktop/aa366761(v=vs.85).aspx

Package

mmap

Source

posix.lisp (file)

Function: mprotect ADDR SIZE PROTECTION

Changes the access protection of the mapped memory region.

The values passed to this function must be the ones retrieved from a call
to MMAP.

The following protection flags are supported:

:READ — [EVERY] Allows reading from the memory region. The OPEN flag :READ is required for this protection mode. This flag is required on windows.
:WRITE — [EVERY] Allows writing to the memory region.
:EXEC — [EVERY] Allows executing code in the memory region.
:NONE — [POSIX] Prevents accessing the memory region.

This function returns nothing useful.

This function may signal an MMAP-ERROR in case the operating system notices
a problem.

See MMAP
See MMAP-ERROR
See http://pubs.opengroup.org/onlinepubs/9699919799/functions/mprotect.html
See http://man7.org/linux/man-pages/man2/mprotect.2.html
See https://msdn.microsoft.com/en-us/library/windows/desktop/aa366898(v=vs.85).aspx

Package

mmap

Source

posix.lisp (file)

Function: msync ADDR FD SIZE &key FLAGS

Causes writes to the mapped file area to be written to disk.

The values passed to this function must be the ones retrieved from a call
to MMAP.

The following flags are supported:

:SYNC — [EVERY] Writing is synchronous. A call to this function
will not return until the data is flushed to disk.
:ASYNC — [EVERY] Writing is asynchronous and a call will return immediately.
:INVALIDATE — [POSIX] Asks to invalidate other mappings of the same
file, ensuring the view is synchronised.

This function returns nothing useful.

This function may signal an MMAP-ERROR in case the operating system notices
a problem.

See MMAP
See MMAP-ERROR
See http://pubs.opengroup.org/onlinepubs/000095399/functions/msync.html
See http://man7.org/linux/man-pages/man2/msync.2.html
See https://msdn.microsoft.com/en-us/library/windows/desktop/aa366563(v=vs.85).aspx
See https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-flushfilebuffers

Package

mmap

Source

posix.lisp (file)

Function: munmap ADDR FD SIZE

Unmaps the memory region, freeing the address space and its file.

The values passed to this function must be the ones retrieved from a call
to MMAP. Calling MUNMAP with the same values more than once will lead to undefined consequences and may very well corrupt your system to crash. The
same goes for calling MUNMAP with values not directly returned by MMAP,
calling it with changed values returned by MMAP, or attempting to
dereference the PTR after a call to MUNMAP.

This function returns nothing useful.

This function may signal an MMAP-ERROR in case the operating system notices
a problem.

See MMAP
See MMAP-ERROR
See WITH-MMAP
See http://pubs.opengroup.org/onlinepubs/9699919799/functions/mprotect.html
See http://man7.org/linux/man-pages/man2/mprotect.2.html
See https://msdn.microsoft.com/en-us/library/windows/desktop/aa366882(v=vs.85).aspx See https://msdn.microsoft.com/en-us/library/windows/desktop/ms724211(v=vs.85).aspx

Package

mmap

Source

posix.lisp (file)


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.4 Generic functions

Generic Function: code CONDITION

The OS-specific error code returned for the mmap failure.

See MMAP-ERROR

Package

mmap

Methods
Method: code (CONDITION mmap-error)
Source

generic.lisp (file)

Generic Function: message CONDITION

The (hopefully) user-readable error message for the mmap failure.

See MMAP-ERROR

Package

mmap

Methods
Method: message (CONDITION mmap-error)
Source

generic.lisp (file)


Previous: , Up: Exported definitions   [Contents][Index]

5.1.5 Conditions

Condition: mmap-error ()

Error signalled if the mmap attempt fails for some reason.

Possible reasons include, but are not limited to:
- File not found
- File access denied
- Out of memory
- Out of address space
- Mapping not allowed
- Invalid combination of flags

See MMAP
See CODE
See MESSAGE

Package

mmap

Source

generic.lisp (file)

Direct superclasses

simple-error (condition)

Direct methods
Direct slots
Slot: code
Initargs

:code

Readers

code (generic function)

Slot: message
Initargs

:message

Readers

message (generic function)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Symbol macros

Symbol Macro: errno
Package

mmap

Source

posix.lisp (file)

Expansion

(mmap::%var-accessor-errno)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Macros

Macro: with-translated-posix-failure &body BODY
Package

mmap

Source

posix.lisp (file)


Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Functions

Function: %mmap PATH SIZE OFFSET OPEN PROTECTION MMAP
Package

mmap

Source

posix.lisp (file)

Function: %var-accessor-errno ()
Function: (setf %var-accessor-errno) VALUE
Package

mmap

Source

posix.lisp (file)

Function: cfold ENV FORM &rest VARS
Package

mmap

Source

generic.lisp (file)

Function: error-mmap CODE MESSAGE
Package

mmap

Source

generic.lisp (file)

Function: fopen-flag FLAG
Package

mmap

Source

posix.lisp (file)

Function: mmap-flag FLAG
Package

mmap

Source

posix.lisp (file)

Function: msync-flag FLAG
Package

mmap

Source

posix.lisp (file)

Function: protection-flag FLAG
Package

mmap

Source

posix.lisp (file)

Function: strerror ERRNUM
Package

mmap

Source

posix.lisp (file)

Function: translate-path PATH
Package

mmap

Source

generic.lisp (file)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   M  
Index Entry  Section

F
File, Lisp, mmap.asd: The mmap<dot>asd file
File, Lisp, mmap/documentation.lisp: The mmap/documentation<dot>lisp file
File, Lisp, mmap/generic.lisp: The mmap/generic<dot>lisp file
File, Lisp, mmap/package.lisp: The mmap/package<dot>lisp file
File, Lisp, mmap/posix.lisp: The mmap/posix<dot>lisp file
File, Lisp, mmap/windows.lisp: The mmap/windows<dot>lisp file

L
Lisp File, mmap.asd: The mmap<dot>asd file
Lisp File, mmap/documentation.lisp: The mmap/documentation<dot>lisp file
Lisp File, mmap/generic.lisp: The mmap/generic<dot>lisp file
Lisp File, mmap/package.lisp: The mmap/package<dot>lisp file
Lisp File, mmap/posix.lisp: The mmap/posix<dot>lisp file
Lisp File, mmap/windows.lisp: The mmap/windows<dot>lisp file

M
mmap.asd: The mmap<dot>asd file
mmap/documentation.lisp: The mmap/documentation<dot>lisp file
mmap/generic.lisp: The mmap/generic<dot>lisp file
mmap/package.lisp: The mmap/package<dot>lisp file
mmap/posix.lisp: The mmap/posix<dot>lisp file
mmap/windows.lisp: The mmap/windows<dot>lisp file

Jump to:   F   L   M  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   %   (  
C   E   F   G   M   P   S   T   W  
Index Entry  Section

%
%mmap: Internal functions
%var-accessor-errno: Internal functions

(
(setf %var-accessor-errno): Internal functions

C
cfold: Internal functions
code: Exported generic functions
code: Exported generic functions
Compiler Macro, mmap: Exported compiler macros
Compiler Macro, mprotect: Exported compiler macros
Compiler Macro, msync: Exported compiler macros

E
error-mmap: Internal functions

F
fopen-flag: Internal functions
Function, %mmap: Internal functions
Function, %var-accessor-errno: Internal functions
Function, (setf %var-accessor-errno): Internal functions
Function, cfold: Internal functions
Function, error-mmap: Internal functions
Function, fopen-flag: Internal functions
Function, mmap: Exported functions
Function, mmap-flag: Internal functions
Function, mprotect: Exported functions
Function, msync: Exported functions
Function, msync-flag: Internal functions
Function, munmap: Exported functions
Function, protection-flag: Internal functions
Function, strerror: Internal functions
Function, translate-path: Internal functions

G
Generic Function, code: Exported generic functions
Generic Function, message: Exported generic functions

M
Macro, with-mmap: Exported macros
Macro, with-translated-posix-failure: Internal macros
message: Exported generic functions
message: Exported generic functions
Method, code: Exported generic functions
Method, message: Exported generic functions
mmap: Exported compiler macros
mmap: Exported functions
mmap-flag: Internal functions
mprotect: Exported compiler macros
mprotect: Exported functions
msync: Exported compiler macros
msync: Exported functions
msync-flag: Internal functions
munmap: Exported functions

P
protection-flag: Internal functions

S
strerror: Internal functions

T
translate-path: Internal functions

W
with-mmap: Exported macros
with-translated-posix-failure: Internal macros

Jump to:   %   (  
C   E   F   G   M   P   S   T   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   C   E   M   S  
Index Entry  Section

C
code: Exported conditions

E
errno: Internal symbol macros

M
message: Exported conditions

S
Slot, code: Exported conditions
Slot, message: Exported conditions
Symbol Macro, errno: Internal symbol macros

Jump to:   C   E   M   S  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   C   M   P   S  
Index Entry  Section

C
Condition, mmap-error: Exported conditions

M
mmap: The mmap system
mmap: The mmap package
mmap-error: Exported conditions

P
Package, mmap: The mmap package

S
System, mmap: The mmap system

Jump to:   C   M   P   S