The sha3 Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The sha3 Reference Manual

This is the sha3 Reference Manual, version 1.1.1, generated automatically by Declt version 2.3 "Robert April" on Wed Mar 14 04:34:08 2018 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

Build Status

This library is an implementation of the Secure Hash Algorithm 3 (SHA-3), also known as Keccak. The implementation is constrained to messages with an integral number of octets, i.e. sub-byte length messages are not supported.

NOTE that prior to release 1.0.2 this package had a bug in the generation of message digests where multiple calls to sha3-update with partial buffers could lead to input data being ignored and therefore erroneous message digests being generated. Uses with only one call to sha3-update and the high-level routines were not affected by this bug.

NOTE that prior to release 1.1.0 this package computed digests based on the Keccak submission to the SHA-3 contest and did not yet take into account the added suffix that the FIPS 202 SHA-3 final standard adds to messages prior to calculating the digest, since this was not part of the Keccak submission. Starting with 1.1.0 the functions in the sha3 package do by default calculate disgests that match the FIPS 202 standard, and will calculate the old pre-standard digests only if the new optional keyword argument :raw-keccak-p is passed with a true value.

The code should be portable across nearly all ANSI compliant CL implementations with specialized versions tuned for implementations that offer unboxed 64bit arithmetic, unboxed 32bit arithmetic and for implementations with efficient fixnum arithmetic (requiring fixnums that can represent (unsigned-byte 16)). Especially the 64 and 32bit implementations have been mostly optimized for SBCL and CMU CL. For those implementations, digests with a 1024 bit-rate (and 288 bit digest output) can be generated in between 30 (64bit SBCL) to around 100 (32bit CMU CL) cycles/byte on an i7-640M; whereas optimized C/assembler implementations reach around 12 to 50 cycles/byte on 64/32 bit Intel hardware. The reason for the discrepancy probably lies in missing peephole and dependency optimizations in the SBCL/CMU CL compiler backend.

The mid-level interfaces to the digest routines are the functions

For convenience the following high-level functions produce digests in one step from 1d simple-arrays and streams with element-type (unsigned-byte 8), as well as files:

Note that in order to generate a message digest of a string it will have to be converted to a simple-array with element-type (unsigned-byte 8) in the proper output-encoding. This will have to rely on implementation-specific functions and is not part of the SHA3 library.

The file keccak-reference.lisp contains a slow simple reference implementation, and testdriver code, which allows testing of the tuned implementations against this reference and against test data available from the Keccak Site at: http://keccak.noekeon.org/KeccakKAT-3.zip

The testcases from the Keccak test data can be run with the following form:

(keccak:test-keccak-msgkat 
 "/Path/To/MsgKatDirectory"
 (lambda (total-bits bit-rate output-bits message)
   (declare (ignore total-bits bit-rate))
   (sha3:sha3-digest-vector message :output-bit-length output-bits :raw-keccak-p t)))

The adapted SHA-3 testcases from the Keccak Code Package test vectors available under https://github.com/gvanas/KeccakCodePackage/tree/master/TestVectors can be run with the following form:

(keccak:test-sha3-msgkat 
 "/Path/To/MsgKatDirectory"
 (lambda (total-bits bit-rate output-bits message)
   (declare (ignore total-bits bit-rate))
   (sha3:sha3-digest-vector message :output-bit-length output-bits)))

This SHA-3 implementation is licensed under the MIT-style license contained in the file COPYING and the header of each source file. Many thanks go to the Keccak Team (Guido Bertoni, Joan Daemen, Michaƫl Peeters and Gilles Van Assche, cf. http://keccak.team) for their algorithm and excellent documentation and reference implementations.

Please direct any feedback to pmai@pmsf.de. A git repository of this library is available under git://github.com/pmai/sha3.git


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 sha3

Maintainer

Pierre R. Mai <pmai@pmsf.de>

Author

Pierre R. Mai <pmai@pmsf.de>

License

MIT/X11

Description

Secure Hash Algorithm 3 (Keccak) Implementation

Version

1.1.1

Dependency

sb-rotate-byte

Source

sha3.asd (file)

Components

Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 sha3.asd

Location

sha3.asd

Systems

sha3 (system)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 sha3/pkgdef.lisp

Parent

sha3 (system)

Location

pkgdef.lisp

Packages

sha3


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.3 sha3/common.lisp

Dependency

pkgdef.lisp (file)

Parent

sha3 (system)

Location

common.lisp

Internal Definitions

Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.4 sha3/keccak-64bit.lisp

Dependencies
Parent

sha3 (system)

Location

keccak-64bit.lisp

Internal Definitions

Previous: , Up: Lisp files   [Contents][Index]

3.1.5 sha3/sha3.lisp

Dependencies
Parent

sha3 (system)

Location

sha3.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 sha3

Source

pkgdef.lisp (file)

Use List

common-lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Next: , Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Functions

Function: sha3-copy STATE

Return an independent copy of the SHA-3 state ‘state’.

Package

sha3

Source

sha3.lisp (file)

Function: sha3-digest-file PATHNAME &key OUTPUT-BIT-LENGTH RAW-KECCAK-P

Calculate an SHA-3 message-digest of the file specified by ‘pathname’. The bit length of the message digest produced is controlled by ‘output-bit-length’, which can take on the values 224, 256, 288, 384 and 512, which is the default value. Using the optional ‘raw-keccak-p’ keyword argument the SHA-3 mandated 01 suffix that is appended to the actual message prior to padding can be elided to yield message digests that match the original Keccak submission instead of the actual SHA-3 standard. Use this option only for compatibility with historical implementations.

Package

sha3

Source

sha3.lisp (file)

Function: sha3-digest-stream STREAM &key OUTPUT-BIT-LENGTH RAW-KECCAK-P

Calculate an SHA-3 message-digest of data read from ‘stream’, which should be a stream with element type (unsigned-byte 8). The bit length of the message digest produced is controlled by ‘output-bit-length’, which can take on the values 224, 256, 288, 384 and 512, which is the default value. Using the optional ‘raw-keccak-p’ keyword argument the SHA-3 mandated 01 suffix that is appended to the actual message prior to padding can be elided to yield message digests that match the original Keccak submission instead of the actual SHA-3 standard. Use this option only for compatibility with historical implementations.

Package

sha3

Source

sha3.lisp (file)

Function: sha3-digest-vector VECTOR &key START END OUTPUT-BIT-LENGTH RAW-KECCAK-P

Calculate an SHA-3 message-digest of data in ‘vector’, which should be a 1d simple-array with element type (unsigned-byte 8), bounded by ‘start’ and ‘end’. The bit length of the message digest produced is controlled by ‘output-bit-length’, which can take on the values 224, 256, 288, 384 and 512, which is the default value. Using the optional ‘raw-keccak-p’ keyword argument the SHA-3 mandated 01 suffix that is appended to the actual message prior to padding can be elided to yield message digests that match the original Keccak submission instead of the actual SHA-3 standard. Use this option only for compatibility with historical implementations.

Package

sha3

Source

sha3.lisp (file)

Function: sha3-final STATE &key OUTPUT-BIT-LENGTH RAW-KECCAK-P

If the given SHA-3 state ‘state’ has not already been finalized, finalize it by processing any remaining input in its buffer, with the specified suffix of 01 and suitable padding as specified by the SHA-3 standard (the specified SHA-3 suffix can be elided with the optional keyword argument ‘raw-keccak-p’ to generate digests as the initial Keccak submission would have generated). Returns the message digest as a simple-array of (unsigned-byte 8). The length of the returned digest is determined either by the output bit length or bit rate specified on state creation, or for the special case of default parameters being used, by the optional keyword argument ‘output-bit-length’. If the state has previously been finalized, the function will return the digest again.

Package

sha3

Source

sha3.lisp (file)

Function: sha3-init &key OUTPUT-BIT-LENGTH BIT-RATE

Create and return a new SHA-3 state. If ‘output-bit-length’ is specified then the state will run at the bit rate specified for the given output bit length. If ‘output-bit-length’ is unspecified, ‘bit-rate’ can be specified to select a suitable bit rate. If both are left unspecified then a default bit rate of 1024 bits is selected, which is suitable for arbitrary output bit lengths of up to 288 bits.

Package

sha3

Source

sha3.lisp (file)

Function: sha3-state-p OBJECT
Package

sha3

Source

sha3.lisp (file)

Function: sha3-update STATE VECTOR &key START END

Update the given SHA-3 state ‘state’ from ‘vector’, which must be a simple-array with element-type (unsigned-byte 8), bounded by ‘start’ and ‘end’, which must be numeric bounding-indices.

Package

sha3

Source

sha3.lisp (file)


Previous: , Up: Exported definitions   [Contents][Index]

5.1.2 Structures

Structure: sha3-state ()
Package

sha3

Source

sha3.lisp (file)

Direct superclasses

structure-object (structure)

Direct slots
Slot: state
Type

sha3::keccak-1600-state

Initform

(sha3::make-keccak-1600-state)

Readers

sha3-state-state (function)

Writers

(setf sha3-state-state) (function)

Slot: bit-rate
Type

(integer 0 1600)

Initform

1024

Readers

sha3-state-bit-rate (function)

Writers

(setf sha3-state-bit-rate) (function)

Slot: buffer
Type

(simple-array (unsigned-byte 8) (200))

Initform

(make-array 200 :element-type (quote (unsigned-byte 8)))

Readers

sha3-state-buffer (function)

Writers

(setf sha3-state-buffer) (function)

Slot: buffer-index
Type

(integer 0 199)

Initform

0

Readers

sha3-state-buffer-index (function)

Writers

(setf sha3-state-buffer-index) (function)

Slot: finalized-p
Readers

sha3-state-finalized-p (function)

Writers

(setf sha3-state-finalized-p) (function)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Constants

Constant: +buffer-size+

Size of internal buffer to use for ‘sha3-digest-stream’ and ‘sha3-digest-file’ operations.

Package

sha3

Source

sha3.lisp (file)

Constant: +keccak-1600-lane-byte-width+

Lane width in bytes for Keccak-1600.

Package

sha3

Source

common.lisp (file)

Constant: +keccak-1600-lane-width+

Lane width for Keccak-1600.

Package

sha3

Source

common.lisp (file)

Constant: +keccak-state-columns+

Width of Keccak state in the x axis

Package

sha3

Source

common.lisp (file)

Constant: +keccak-state-lanes+

Total number of lanes in Keccak state

Package

sha3

Source

common.lisp (file)

Constant: +keccak-state-rows+

Width of Keccak state in the y axis

Package

sha3

Source

common.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Special variables

Special Variable: *keccak-f-rotate-offsets*

Keccak Rotate Offsets

Package

sha3

Source

common.lisp (file)

Special Variable: *keccak-f-round-constants*

Keccak Round Constants

Package

sha3

Source

common.lisp (file)

Special Variable: *optimize-declaration*

Global optimize declaration used for performance critical functions. This can be changed prior to compiling the package for debugging/testing purposes.

Package

sha3

Source

common.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Macros

Macro: dotimes-unrolled (VAR LIMIT) &body BODY

Unroll the loop body at compile-time.

Package

sha3

Source

common.lisp (file)

Macro: get-rotate-offset X Y
Package

sha3

Source

common.lisp (file)

Macro: with-state-accessors (&rest STATES) &body BODY

Bind the contents of the state(s) array(s) to local variables, and save the content on normal form exit.

Package

sha3

Source

keccak-64bit.lisp (file)

Macro: with-temp-rows (&rest ROWS) &body BODY

Bind local variables for each temporary row.

Package

sha3

Source

keccak-64bit.lisp (file)

Macro: with-temp-state (&rest TEMPS) &body BODY

Bind local variables for each temporary state.

Package

sha3

Source

keccak-64bit.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.4 Functions

Function: copy-sha3-state INSTANCE
Package

sha3

Source

sha3.lisp (file)

Function: keccak-f ()
Package

sha3

Source

keccak-64bit.lisp (file)

Function: keccak-f-rot ()
Package

sha3

Source

keccak-64bit.lisp (file)

Function: keccak-f-round-constant ()
Package

sha3

Source

keccak-64bit.lisp (file)

Function: keccak-state-extract-output STATE OUTPUT-BITS
Package

sha3

Source

keccak-64bit.lisp (file)

Function: keccak-state-merge-input ()
Package

sha3

Source

keccak-64bit.lisp (file)

Function: make-keccak-1600-state ()
Package

sha3

Source

keccak-64bit.lisp (file)

Function: make-sha3-state BIT-RATE
Package

sha3

Source

sha3.lisp (file)

Function: pad-message-to-width MESSAGE BIT-WIDTH ADD-FIPS-202-SUFFIX-P

Destructively pad the given message to the given bit-width according to the Keccak 10*1 padding rules, optionally appending the FIPS 202/SHA-3 mandated 01 suffix first, and return the padded message.

Package

sha3

Source

common.lisp (file)

Function: sha3-state-bit-rate INSTANCE
Package

sha3

Source

sha3.lisp (file)

Function: sha3-state-buffer INSTANCE
Package

sha3

Source

sha3.lisp (file)

Function: sha3-state-buffer-index INSTANCE
Function: (setf sha3-state-buffer-index) VALUE INSTANCE
Package

sha3

Source

sha3.lisp (file)

Function: sha3-state-finalized-p INSTANCE
Function: (setf sha3-state-finalized-p) VALUE INSTANCE
Package

sha3

Source

sha3.lisp (file)

Function: sha3-state-state INSTANCE
Package

sha3

Source

sha3.lisp (file)

Function: trivial-macroexpand-all FORM ENV

Trivial and very restricted code-walker used in partial evaluation macros. Only supports atoms and function forms, no special forms.

Package

sha3

Source

common.lisp (file)


Previous: , Up: Internal definitions   [Contents][Index]

5.2.5 Types

Type: buffer-index ()
Package

sha3

Source

sha3.lisp (file)

Type: keccak-1600-lane ()

Type of a keccak lane for Keccak-1600.

Package

sha3

Source

common.lisp (file)

Type: keccak-1600-state ()

Type of a keccak working state object for Keccak-1600.

Package

sha3

Source

keccak-64bit.lisp (file)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   S  
Index Entry  Section

F
File, Lisp, sha3.asd: The sha3<dot>asd file
File, Lisp, sha3/common.lisp: The sha3/common<dot>lisp file
File, Lisp, sha3/keccak-64bit.lisp: The sha3/keccak-64bit<dot>lisp file
File, Lisp, sha3/pkgdef.lisp: The sha3/pkgdef<dot>lisp file
File, Lisp, sha3/sha3.lisp: The sha3/sha3<dot>lisp file

L
Lisp File, sha3.asd: The sha3<dot>asd file
Lisp File, sha3/common.lisp: The sha3/common<dot>lisp file
Lisp File, sha3/keccak-64bit.lisp: The sha3/keccak-64bit<dot>lisp file
Lisp File, sha3/pkgdef.lisp: The sha3/pkgdef<dot>lisp file
Lisp File, sha3/sha3.lisp: The sha3/sha3<dot>lisp file

S
sha3.asd: The sha3<dot>asd file
sha3/common.lisp: The sha3/common<dot>lisp file
sha3/keccak-64bit.lisp: The sha3/keccak-64bit<dot>lisp file
sha3/pkgdef.lisp: The sha3/pkgdef<dot>lisp file
sha3/sha3.lisp: The sha3/sha3<dot>lisp file

Jump to:   F   L   S  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   (  
C   D   F   G   K   M   P   S   T   W  
Index Entry  Section

(
(setf sha3-state-buffer-index): Internal functions
(setf sha3-state-finalized-p): Internal functions

C
copy-sha3-state: Internal functions

D
dotimes-unrolled: Internal macros

F
Function, (setf sha3-state-buffer-index): Internal functions
Function, (setf sha3-state-finalized-p): Internal functions
Function, copy-sha3-state: Internal functions
Function, keccak-f: Internal functions
Function, keccak-f-rot: Internal functions
Function, keccak-f-round-constant: Internal functions
Function, keccak-state-extract-output: Internal functions
Function, keccak-state-merge-input: Internal functions
Function, make-keccak-1600-state: Internal functions
Function, make-sha3-state: Internal functions
Function, pad-message-to-width: Internal functions
Function, sha3-copy: Exported functions
Function, sha3-digest-file: Exported functions
Function, sha3-digest-stream: Exported functions
Function, sha3-digest-vector: Exported functions
Function, sha3-final: Exported functions
Function, sha3-init: Exported functions
Function, sha3-state-bit-rate: Internal functions
Function, sha3-state-buffer: Internal functions
Function, sha3-state-buffer-index: Internal functions
Function, sha3-state-finalized-p: Internal functions
Function, sha3-state-p: Exported functions
Function, sha3-state-state: Internal functions
Function, sha3-update: Exported functions
Function, trivial-macroexpand-all: Internal functions

G
get-rotate-offset: Internal macros

K
keccak-f: Internal functions
keccak-f-rot: Internal functions
keccak-f-round-constant: Internal functions
keccak-state-extract-output: Internal functions
keccak-state-merge-input: Internal functions

M
Macro, dotimes-unrolled: Internal macros
Macro, get-rotate-offset: Internal macros
Macro, with-state-accessors: Internal macros
Macro, with-temp-rows: Internal macros
Macro, with-temp-state: Internal macros
make-keccak-1600-state: Internal functions
make-sha3-state: Internal functions

P
pad-message-to-width: Internal functions

S
sha3-copy: Exported functions
sha3-digest-file: Exported functions
sha3-digest-stream: Exported functions
sha3-digest-vector: Exported functions
sha3-final: Exported functions
sha3-init: Exported functions
sha3-state-bit-rate: Internal functions
sha3-state-buffer: Internal functions
sha3-state-buffer-index: Internal functions
sha3-state-finalized-p: Internal functions
sha3-state-p: Exported functions
sha3-state-state: Internal functions
sha3-update: Exported functions

T
trivial-macroexpand-all: Internal functions

W
with-state-accessors: Internal macros
with-temp-rows: Internal macros
with-temp-state: Internal macros

Jump to:   (  
C   D   F   G   K   M   P   S   T   W  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *   +  
B   C   F   S  
Index Entry  Section

*
*keccak-f-rotate-offsets*: Internal special variables
*keccak-f-round-constants*: Internal special variables
*optimize-declaration*: Internal special variables

+
+buffer-size+: Internal constants
+keccak-1600-lane-byte-width+: Internal constants
+keccak-1600-lane-width+: Internal constants
+keccak-state-columns+: Internal constants
+keccak-state-lanes+: Internal constants
+keccak-state-rows+: Internal constants

B
bit-rate: Exported structures
buffer: Exported structures
buffer-index: Exported structures

C
Constant, +buffer-size+: Internal constants
Constant, +keccak-1600-lane-byte-width+: Internal constants
Constant, +keccak-1600-lane-width+: Internal constants
Constant, +keccak-state-columns+: Internal constants
Constant, +keccak-state-lanes+: Internal constants
Constant, +keccak-state-rows+: Internal constants

F
finalized-p: Exported structures

S
Slot, bit-rate: Exported structures
Slot, buffer: Exported structures
Slot, buffer-index: Exported structures
Slot, finalized-p: Exported structures
Slot, state: Exported structures
Special Variable, *keccak-f-rotate-offsets*: Internal special variables
Special Variable, *keccak-f-round-constants*: Internal special variables
Special Variable, *optimize-declaration*: Internal special variables
state: Exported structures

Jump to:   *   +  
B   C   F   S  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   B   K   P   S   T  
Index Entry  Section

B
buffer-index: Internal types

K
keccak-1600-lane: Internal types
keccak-1600-state: Internal types

P
Package, sha3: The sha3 package

S
sha3: The sha3 system
sha3: The sha3 package
sha3-state: Exported structures
Structure, sha3-state: Exported structures
System, sha3: The sha3 system

T
Type, buffer-index: Internal types
Type, keccak-1600-lane: Internal types
Type, keccak-1600-state: Internal types

Jump to:   B   K   P   S   T