The shorty Reference Manual

Table of Contents

Next: , Previous: , Up: (dir)   [Contents][Index]

The shorty Reference Manual

This is the shorty Reference Manual, version 1.0.0, generated automatically by Declt version 2.3 "Robert April" on Wed Mar 14 04:34:25 2018 GMT+0.


Next: , Previous: , Up: Top   [Contents][Index]

1 Introduction

shorty

A toy URL datastore.

shorty

Overview

This is a toy project which can be used to store any amount of data in the URL of a URL-shortening service. It exploits the fact that most URL shortening services do not check that a URL points to a real website, and as such, any data can be URL-shortened.

How it works:

Install

(ql:quickload :shorty)

Usage

To store a file in a URL:

(shorty:shorten-file "/path/to/in-file")

To write a file to disk given a URL:

(shorty:unshorten-file "http://short-url" :file "/path/to/out-file")

Known Limitations

There are 2 main issues that will likely never be fixed, as this is just meant to be a proof of concept toy.

License

Copyright © 2017-2018 Michael Fiano.

Licensed under the MIT License.


Next: , Previous: , Up: Top   [Contents][Index]

2 Systems

The main system appears first, followed by any subsystem dependency.


Previous: , Up: Systems   [Contents][Index]

2.1 shorty

Maintainer

Michael Fiano <mail@michaelfiano.com>

Author

Michael Fiano <mail@michaelfiano.com>

Home Page

https://www.michaelfiano.com/projects/shorty

Source Control

(:git "git@github.com:mfiano/shorty.git")

Bug Tracker

https://github.com/mfiano/shorty/issues

License

MIT

Description

Shorten URLs using third-party services.

Long Description

# shorty

A toy URL datastore.

![shorty](https://files.lispcoder.net/images/projects/shorty/shorty.gif)

## Overview

This is a toy project which can be used to store any amount of data in the URL of a URL-shortening service. It exploits the fact that most URL shortening services do not check that a URL points to a real website, and as such, any data can be URL-shortened.

How it works:

- Takes a file pathname as input.

- Compresses the file with the DEFLATE compression scheme.

- Encodes the compressed file using the Base64 encoding scheme.

- Splits the data into chunks of typically a few kilobytes long, depending on
the URL shortening service used.

- Creates a short URL for each chunk.

- Concatenates the path components of all short URLs into 1 long string,
separated by a forward slash ‘/‘.

- Again, creates a short URL using the concatenated path components as input.

- The result is a single short URL that has an arbitrary amount of data stored
in the URL.

## Install

“‘lisp
(ql:quickload :shorty)
“‘

## Usage

To store a file in a URL:

“‘lisp
(shorty:shorten-file "/path/to/in-file")
“‘

To write a file to disk given a URL:

“‘lisp
(shorty:unshorten-file "http://short-url" :file "/path/to/out-file")
“‘

## Known Limitations

There are 2 main issues that will likely never be fixed, as this is just meant to be a proof of concept toy.

- Local limit: Shortening and unshortening are done completely in memory, so you need enough heap space.

- Remote limit: During the concatenation of URL path components to create the single short URL at the end of the shortening process, the resulting string must also fit into a chunk size supported by the specified URL-shortening service. The path component length depends on the service. Do not expect to store more than a few megabytes of data at best.

## License

Copyright © 2017-2018 [Michael Fiano](mailto:mail@michaelfiano.com).

Licensed under the MIT License.

Version

1.0.0

Dependencies
Source

shorty.asd (file)

Components

Next: , Previous: , Up: Top   [Contents][Index]

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.


Previous: , Up: Files   [Contents][Index]

3.1 Lisp


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.1 shorty.asd

Location

/home/quickbuilder/quicklisp/dists/quicklisp/software/shorty-20180228-git/shorty.asd

Systems

shorty (system)


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.2 shorty/package.lisp

Parent

shorty (system)

Location

package.lisp

Packages

shorty


Next: , Previous: , Up: Lisp files   [Contents][Index]

3.1.3 shorty/services.lisp

Dependency

package.lisp (file)

Parent

shorty (system)

Location

services.lisp

Internal Definitions

Previous: , Up: Lisp files   [Contents][Index]

3.1.4 shorty/shorty.lisp

Dependency

services.lisp (file)

Parent

shorty (system)

Location

shorty.lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

4 Packages

Packages are listed by definition order.


Previous: , Up: Packages   [Contents][Index]

4.1 shorty

Source

package.lisp (file)

Use List

common-lisp

Exported Definitions
Internal Definitions

Next: , Previous: , Up: Top   [Contents][Index]

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


Next: , Previous: , Up: Definitions   [Contents][Index]

5.1 Exported definitions


Previous: , Up: Exported definitions   [Contents][Index]

5.1.1 Functions

Function: shorten-file FILE &key SERVICE

Shorten the file corresponding to the pathname FILE, producing a single short URL which encodes all of its data.

Package

shorty

Source

shorty.lisp (file)

Function: unshorten-file URL &key FILE

Unshorten the file stored in URL, storing the result to disk in the pathname corresponding to FILE.

Package

shorty

Source

shorty.lisp (file)


Previous: , Up: Definitions   [Contents][Index]

5.2 Internal definitions


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.1 Special variables

Special Variable: *input-buffer*
Package

shorty

Source

shorty.lisp (file)


Next: , Previous: , Up: Internal definitions   [Contents][Index]

5.2.2 Functions

Function: build-query SERVICE DATA

Build a query URL string for SERVICE and the encoded DATA.

Package

shorty

Source

shorty.lisp (file)

Function: compress-data DATA

Compress DATA using the DEFLATE compression scheme.

Package

shorty

Source

shorty.lisp (file)

Function: decode-data DATA

Decode DATA encoded in the Base64 encoding scheme.

Package

shorty

Source

shorty.lisp (file)

Function: encode-data DATA

Encode DATA using the Base64 encoding scheme.

Package

shorty

Source

shorty.lisp (file)

Function: finalize-shorten SERVICE CHUNKS

Shortens all shortened URL CHUNKS into a single short URL for the specified SERVICE.

Package

shorty

Source

shorty.lisp (file)

Function: generate-chunk-size SERVICE

Generate a random chunk size within 25% of the maximum chunk size allowed for SERVICE.

Package

shorty

Source

shorty.lisp (file)

Function: get-path-component URL

Get the path component of URL.

Package

shorty

Source

shorty.lisp (file)

Function: get-redirect URL

Get the 301 redirection target from the header of URL.

Package

shorty

Source

shorty.lisp (file)

Function: get-redirect-path URL

Get the path component of the 301 redirection target of URL.

Package

shorty

Source

shorty.lisp (file)

Function: read-chunk SERVICE DATA

Read a chunk of DATA as a string, with a size up to the maximum size supported by SERVICE.

Package

shorty

Source

shorty.lisp (file)

Function: rejoin-chunks URL

Rejoin all chunks encoded in the short URL URL, returning a DEFLATE-compressed, Base64-encoded string.

Package

shorty

Source

shorty.lisp (file)

Function: resolve-chunk-urls URL

Resolve all the chunks contained in the short URL URL, returning a list of more short URL’s.

Package

shorty

Source

shorty.lisp (file)

Function: shorten-chunks SERVICE DATA

Split DATA into chunks, collecting a list of shortened URL’s of each chunk, for the specified SERVICE.

Package

shorty

Source

shorty.lisp (file)

Function: shorten-data SERVICE DATA

Query the specified SERVICE for a short URL that redirects to a long URL storing the encoded DATA.

Package

shorty

Source

shorty.lisp (file)

Function: uncompress-data DATA

Uncompress DATA encoded in the DEFLATE compression scheme.

Package

shorty

Source

shorty.lisp (file)

Function: unshorten-chunks URL

Unshorten the data stored in URL, returning an octet vector.

Package

shorty

Source

shorty.lisp (file)


Previous: , Up: Internal definitions   [Contents][Index]

5.2.3 Generic functions

Generic Function: generate-host TYPE
Package

shorty

Source

shorty.lisp (file)

Methods
Method: generate-host (TYPE (eql google))

Generate a URL prefix using Google’s domain name. This is a workaround for bugs present in both drakma and dexador HTTP client libraries.

Method: generate-host (TYPE (eql random))

Generate a URL prefix with a random 4-character domain name under the .com TLD, and a random 4-character sub-domain, which will prefix the encoded data in a URL shortening service query, in order to mimic an real website link.

Generic Function: service-properties SERVICE
Package

shorty

Methods
Method: service-properties (SERVICE (eql tinyurl))
Source

services.lisp (file)

Method: service-properties (SERVICE (eql isgd))
Source

services.lisp (file)


Previous: , Up: Top   [Contents][Index]

Appendix A Indexes


Next: , Previous: , Up: Indexes   [Contents][Index]

A.1 Concepts

Jump to:   F   L   S  
Index Entry  Section

F
File, Lisp, shorty.asd: The shorty<dot>asd file
File, Lisp, shorty/package.lisp: The shorty/package<dot>lisp file
File, Lisp, shorty/services.lisp: The shorty/services<dot>lisp file
File, Lisp, shorty/shorty.lisp: The shorty/shorty<dot>lisp file

L
Lisp File, shorty.asd: The shorty<dot>asd file
Lisp File, shorty/package.lisp: The shorty/package<dot>lisp file
Lisp File, shorty/services.lisp: The shorty/services<dot>lisp file
Lisp File, shorty/shorty.lisp: The shorty/shorty<dot>lisp file

S
shorty.asd: The shorty<dot>asd file
shorty/package.lisp: The shorty/package<dot>lisp file
shorty/services.lisp: The shorty/services<dot>lisp file
shorty/shorty.lisp: The shorty/shorty<dot>lisp file

Jump to:   F   L   S  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.2 Functions

Jump to:   B   C   D   E   F   G   M   R   S   U  
Index Entry  Section

B
build-query: Internal functions

C
compress-data: Internal functions

D
decode-data: Internal functions

E
encode-data: Internal functions

F
finalize-shorten: Internal functions
Function, build-query: Internal functions
Function, compress-data: Internal functions
Function, decode-data: Internal functions
Function, encode-data: Internal functions
Function, finalize-shorten: Internal functions
Function, generate-chunk-size: Internal functions
Function, get-path-component: Internal functions
Function, get-redirect: Internal functions
Function, get-redirect-path: Internal functions
Function, read-chunk: Internal functions
Function, rejoin-chunks: Internal functions
Function, resolve-chunk-urls: Internal functions
Function, shorten-chunks: Internal functions
Function, shorten-data: Internal functions
Function, shorten-file: Exported functions
Function, uncompress-data: Internal functions
Function, unshorten-chunks: Internal functions
Function, unshorten-file: Exported functions

G
generate-chunk-size: Internal functions
generate-host: Internal generic functions
generate-host: Internal generic functions
generate-host: Internal generic functions
Generic Function, generate-host: Internal generic functions
Generic Function, service-properties: Internal generic functions
get-path-component: Internal functions
get-redirect: Internal functions
get-redirect-path: Internal functions

M
Method, generate-host: Internal generic functions
Method, generate-host: Internal generic functions
Method, service-properties: Internal generic functions
Method, service-properties: Internal generic functions

R
read-chunk: Internal functions
rejoin-chunks: Internal functions
resolve-chunk-urls: Internal functions

S
service-properties: Internal generic functions
service-properties: Internal generic functions
service-properties: Internal generic functions
shorten-chunks: Internal functions
shorten-data: Internal functions
shorten-file: Exported functions

U
uncompress-data: Internal functions
unshorten-chunks: Internal functions
unshorten-file: Exported functions

Jump to:   B   C   D   E   F   G   M   R   S   U  

Next: , Previous: , Up: Indexes   [Contents][Index]

A.3 Variables

Jump to:   *  
S  
Index Entry  Section

*
*input-buffer*: Internal special variables

S
Special Variable, *input-buffer*: Internal special variables

Jump to:   *  
S  

Previous: , Up: Indexes   [Contents][Index]

A.4 Data types

Jump to:   P   S  
Index Entry  Section

P
Package, shorty: The shorty package

S
shorty: The shorty system
shorty: The shorty package
System, shorty: The shorty system

Jump to:   P   S