The cl-mechanize Reference Manual

Table of Contents

The cl-mechanize Reference Manual

This is the cl-mechanize Reference Manual, version 0.0, generated automatically by Declt version 3.0 "Montgomery Scott" on Tue Dec 22 12:31:57 2020 GMT+0.

1 Introduction

About

CL-MECHANIZE is a WWW::Mechanize work-alike for Common Lisp implemented as a thin wrapper on top of Drakma.

This package is still in the early stages of development. Please read the source before using; do not expect this to do anything useful.

Getting

git clone git://github.com/joachifm/cl-mechanize.git cl-mechanize

Use QuickLisp to easily install the dependencies.

Usage

$ lisp

;; Add system definition to the ASDF search path
(pushnew (merge-pathnames "relative/path/to/cl-mechanize"
                          (user-homedir-pathname))
         asdf:*central-registry*)

;; Load system
(asdf:operate 'asdf:load-op :cl-mechanize)
(in-package :cl-mechanize-user)

;; Create browser object
(defvar *browser* (make-instance 'browser)

;; Do a google search
(fetch "http://www.google.com" *browser*)

(let* ((page (browser-page *browser*))
       (search-form (car (page-forms page))))
  (setf (form-inputs search-form)
        '(("q" . "google")))
  (submit search-form *browser*)

  (let ((results (browser-page *browser*)))
    (format t "~A~%" (ppcre:all-matches-as-strings "<title>[a-z].*</title>"
                                                 (page-content results)))
    (dolist (link (page-links results))
      (format t "~A~%" (link-text link)))

    ;; Traverse the DOM
    (stp:do-recursively (n (page-dom results))
        ...)))

2 Systems

The main system appears first, followed by any subsystem dependency.

2.1 cl-mechanize

Author

Joachim Fasting <joachim.fasting@gmail.com>

License

BSD

Description

A WWW::Mechanize work-alike

Version

0.0

Dependencies

• puri
• drakma
• closure-html
• cxml-stp
• cl-ppcre

Source

cl-mechanize.asd (file)

Components

• packages.lisp (file)
• cl-mechanize.lisp (file)

3 Files

Files are sorted by type and then listed depth-first from the systems components trees.

3.1 Lisp

3.1.1 cl-mechanize.asd

Location

cl-mechanize.asd

Systems

cl-mechanize (system)

Packages

cl-mechanize-asd

3.1.2 cl-mechanize/packages.lisp

Parent

cl-mechanize (system)

Location

packages.lisp

Packages

• cl-mechanize-user
• cl-mechanize

3.1.3 cl-mechanize/cl-mechanize.lisp

Dependency

packages.lisp (file)

Parent

cl-mechanize (system)

Location

cl-mechanize.lisp

Exported Definitions

• *accept-cookies-p* (special variable)
• *cookie-jar-dir* (special variable)
• *cookie-lifetime-policy* (special variable)
• *user-agent* (special variable)
• back (function)
• browser (class)
• browser-cookie-jar (method)
• (setf browser-cookie-jar) (method)
• browser-history (method)
• (setf browser-history) (method)
• browser-page (method)
• (setf browser-page) (method)
• browser-user-agent (method)
• (setf browser-user-agent) (method)
• fetch (function)
• follow (function)
• form (class)
• form-action (method)
• (setf form-action) (method)
• form-inputs (method)
• (setf form-inputs) (method)
• form-method (method)
• (setf form-method) (method)
• form-name (method)
• (setf form-name) (method)
• link (class)
• link-attrs (method)
• (setf link-attrs) (method)
• link-tag (method)
• (setf link-tag) (method)
• link-text (method)
• (setf link-text) (method)
• link-uri (method)
• (setf link-uri) (method)
• link-url (method)
• (setf link-url) (method)
• page (class)
• page-content (method)
• (setf page-content) (method)
• page-dom (method)
• (setf page-dom) (method)
• page-forms (method)
• (setf page-forms) (method)
• page-links (method)
• (setf page-links) (method)
• page-uri (method)
• (setf page-uri) (method)
• reload (function)
• submit (function)

Internal Definitions

• browser-status (method)
• (setf browser-status) (method)

4 Packages

Packages are listed by definition order.

4.1 cl-mechanize-asd

Source

cl-mechanize.asd

Use List

• asdf/interface
• common-lisp

4.2 cl-mechanize-user

Source

packages.lisp (file)

Nicknames

• net.browser-user
• mechanize-user
• browser-user

Use List

• cl-mechanize
• common-lisp

4.3 cl-mechanize

Source

packages.lisp (file)

Nicknames

• net.browser
• mechanize
• browser

Use List

common-lisp

Used By List

cl-mechanize-user

Exported Definitions

• *accept-cookies-p* (special variable)
• *cookie-jar-dir* (special variable)
• *cookie-lifetime-policy* (special variable)
• *user-agent* (special variable)
• back (function)
• browser (class)
• browser-cookie-jar (generic function)
• browser-cookie-jar (method)
• (setf browser-cookie-jar) (method)
• (setf browser-cookie-jar) (generic function)
• browser-history (generic function)
• browser-history (method)
• (setf browser-history) (method)
• (setf browser-history) (generic function)
• browser-page (generic function)
• browser-page (method)
• (setf browser-page) (method)
• (setf browser-page) (generic function)
• browser-user-agent (generic function)
• browser-user-agent (method)
• (setf browser-user-agent) (method)
• (setf browser-user-agent) (generic function)
• fetch (function)
• follow (function)
• form (class)
• form-action (generic function)
• form-action (method)
• (setf form-action) (method)
• (setf form-action) (generic function)
• form-inputs (generic function)
• form-inputs (method)
• (setf form-inputs) (method)
• (setf form-inputs) (generic function)
• form-method (generic function)
• form-method (method)
• (setf form-method) (method)
• (setf form-method) (generic function)
• form-name (generic function)
• form-name (method)
• (setf form-name) (method)
• (setf form-name) (generic function)
• link (class)
• link-attrs (generic function)
• link-attrs (method)
• (setf link-attrs) (method)
• (setf link-attrs) (generic function)
• link-tag (generic function)
• link-tag (method)
• (setf link-tag) (method)
• (setf link-tag) (generic function)
• link-text (generic function)
• link-text (method)
• (setf link-text) (method)
• (setf link-text) (generic function)
• link-uri (generic function)
• link-uri (method)
• (setf link-uri) (method)
• (setf link-uri) (generic function)
• link-url (generic function)
• link-url (method)
• (setf link-url) (method)
• (setf link-url) (generic function)
• page (class)
• page-content (generic function)
• page-content (method)
• (setf page-content) (method)
• (setf page-content) (generic function)
• page-dom (generic function)
• page-dom (method)
• (setf page-dom) (method)
• (setf page-dom) (generic function)
• page-forms (generic function)
• page-forms (method)
• (setf page-forms) (method)
• (setf page-forms) (generic function)
• page-links (generic function)
• page-links (method)
• (setf page-links) (method)
• (setf page-links) (generic function)
• page-uri (generic function)
• page-uri (method)
• (setf page-uri) (method)
• (setf page-uri) (generic function)
• reload (function)
• submit (function)

Internal Definitions

• browser-status (generic function)
• browser-status (method)
• (setf browser-status) (method)
• (setf browser-status) (generic function)

5 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.

5.1 Exported definitions

5.1.1 Special variables

Special Variable: *accept-cookies-p*

Accept cookies from visited sites?

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Special Variable: *cookie-jar-dir*

Folder where cookies are stored.
Set to NIL to disable storing/loading cookies to/from disk.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Special Variable: *cookie-lifetime-policy*

How long to keep cookies.

Setting to :EXPIRY keeps cookies until they expire

Setting to :SESSION does not store any cookies.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Special Variable: *user-agent*

Default User-Agent string.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

5.1.2 Functions

Function: back BROWSER

Go back in history.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Function: fetch URI BROWSER &key METHOD PARAMETERS

Send a request and fetch the response.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Function: follow LINK BROWSER

Follow a link on the current page.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Function: reload BROWSER

Repeat the current request.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Function: submit FORM BROWSER

Submit a form on the current page.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

5.1.3 Generic functions

Generic Function: browser-cookie-jar OBJECT

Generic Function: (setf browser-cookie-jar) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: browser-cookie-jar (BROWSER browser)

Method: (setf browser-cookie-jar) NEW-VALUE (BROWSER browser)

The current cookie jar object. Updated by FETCH.

Source

cl-mechanize.lisp (file)

Generic Function: browser-history OBJECT

Generic Function: (setf browser-history) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: browser-history (BROWSER browser)

Method: (setf browser-history) NEW-VALUE (BROWSER browser)

A list of visited pages, in chronological order.

Source

cl-mechanize.lisp (file)

Generic Function: browser-page OBJECT

Generic Function: (setf browser-page) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: browser-page (BROWSER browser)

Method: (setf browser-page) NEW-VALUE (BROWSER browser)

The current page object. Updated by FETCH.

Source

cl-mechanize.lisp (file)

Generic Function: browser-user-agent OBJECT

Generic Function: (setf browser-user-agent) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: browser-user-agent (BROWSER browser)

Method: (setf browser-user-agent) NEW-VALUE (BROWSER browser)

User-Agent string used by GET.

Source

cl-mechanize.lisp (file)

Generic Function: form-action OBJECT

Generic Function: (setf form-action) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: form-action (FORM form)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf form-action) NEW-VALUE (FORM form)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: form-inputs OBJECT

Generic Function: (setf form-inputs) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: form-inputs (FORM form)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf form-inputs) NEW-VALUE (FORM form)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: form-method OBJECT

Generic Function: (setf form-method) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: form-method (FORM form)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf form-method) NEW-VALUE (FORM form)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: form-name OBJECT

Generic Function: (setf form-name) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: form-name (FORM form)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf form-name) NEW-VALUE (FORM form)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: link-attrs OBJECT

Generic Function: (setf link-attrs) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: link-attrs (LINK link)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf link-attrs) NEW-VALUE (LINK link)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: link-tag OBJECT

Generic Function: (setf link-tag) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: link-tag (LINK link)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf link-tag) NEW-VALUE (LINK link)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: link-text OBJECT

Generic Function: (setf link-text) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: link-text (LINK link)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf link-text) NEW-VALUE (LINK link)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: link-uri OBJECT

Generic Function: (setf link-uri) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: link-uri (LINK link)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf link-uri) NEW-VALUE (LINK link)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: link-url OBJECT

Generic Function: (setf link-url) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: link-url (LINK link)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf link-url) NEW-VALUE (LINK link)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: page-content OBJECT

Generic Function: (setf page-content) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: page-content (PAGE page)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf page-content) NEW-VALUE (PAGE page)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: page-dom OBJECT

Generic Function: (setf page-dom) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: page-dom (PAGE page)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf page-dom) NEW-VALUE (PAGE page)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: page-forms OBJECT

Generic Function: (setf page-forms) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: page-forms (PAGE page)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf page-forms) NEW-VALUE (PAGE page)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: page-links OBJECT

Generic Function: (setf page-links) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: page-links (PAGE page)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf page-links) NEW-VALUE (PAGE page)

automatically generated writer method

Source

cl-mechanize.lisp (file)

Generic Function: page-uri OBJECT

Generic Function: (setf page-uri) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: page-uri (PAGE page)

automatically generated reader method

Source

cl-mechanize.lisp (file)

Method: (setf page-uri) NEW-VALUE (PAGE page)

automatically generated writer method

Source

cl-mechanize.lisp (file)

5.1.4 Classes

Class: browser ()

Encapsulates the browser state.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• browser-status (method)
• browser-status (method)
• browser-page (method)
• browser-page (method)
• browser-history (method)
• browser-history (method)
• browser-cookie-jar (method)
• browser-cookie-jar (method)
• browser-user-agent (method)
• browser-user-agent (method)

Direct slots

Slot: user-agent

User-Agent string used by GET.

Initargs

:user-agent

Initform

cl-mechanize:*user-agent*

Readers

browser-user-agent (generic function)

Writers

(setf browser-user-agent) (generic function)

Slot: cookie-jar

The current cookie jar object. Updated by FETCH.

Initform

(make-instance (quote drakma:cookie-jar))

Readers

browser-cookie-jar (generic function)

Writers

(setf browser-cookie-jar) (generic function)

Slot: history

A list of visited pages, in chronological order.

Readers

browser-history (generic function)

Writers

(setf browser-history) (generic function)

Slot: page

The current page object. Updated by FETCH.

Readers

browser-page (generic function)

Writers

(setf browser-page) (generic function)

Slot: status

The HTTP status code of the last request.

Readers

browser-status (generic function)

Writers

(setf browser-status) (generic function)

Class: form ()

Represents a form element.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• form-inputs (method)
• form-inputs (method)
• form-method (method)
• form-method (method)
• form-action (method)
• form-action (method)
• form-name (method)
• form-name (method)

Direct slots

Slot: name

Initargs

:name

Readers

form-name (generic function)

Writers

(setf form-name) (generic function)

Slot: action

Initargs

:action

Readers

form-action (generic function)

Writers

(setf form-action) (generic function)

Slot: method

Initargs

:method

Initform

:get

Readers

form-method (generic function)

Writers

(setf form-method) (generic function)

Slot: inputs

Initargs

:inputs

Readers

form-inputs (generic function)

Writers

(setf form-inputs) (generic function)

Class: link ()

Represents a link element.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• link-attrs (method)
• link-attrs (method)
• link-tag (method)
• link-tag (method)
• link-url (method)
• link-url (method)
• link-uri (method)
• link-uri (method)
• link-text (method)
• link-text (method)

Direct slots

Slot: text

Initargs

:text

Readers

link-text (generic function)

Writers

(setf link-text) (generic function)

Slot: uri

Initargs

:uri

Readers

link-uri (generic function)

Writers

(setf link-uri) (generic function)

Slot: url

Initargs

:url

Readers

link-url (generic function)

Writers

(setf link-url) (generic function)

Slot: tag

Initargs

:tag

Initform

:a

Readers

link-tag (generic function)

Writers

(setf link-tag) (generic function)

Slot: attrs

Initargs

:attrs

Readers

link-attrs (generic function)

Writers

(setf link-attrs) (generic function)

Class: page ()

Contains the result of fetching a page.

Package

cl-mechanize

Source

cl-mechanize.lisp (file)

Direct superclasses

standard-object (class)

Direct methods

• page-content (method)
• page-content (method)
• page-dom (method)
• page-dom (method)
• page-forms (method)
• page-forms (method)
• page-links (method)
• page-links (method)
• page-uri (method)
• page-uri (method)

Direct slots

Slot: uri

Initargs

:uri

Readers

page-uri (generic function)

Writers

(setf page-uri) (generic function)

Slot: links

Initargs

:links

Readers

page-links (generic function)

Writers

(setf page-links) (generic function)

Slot: forms

Initargs

:forms

Readers

page-forms (generic function)

Writers

(setf page-forms) (generic function)

Slot: dom

Initargs

:dom

Readers

page-dom (generic function)

Writers

(setf page-dom) (generic function)

Slot: content

Initargs

:content

Readers

page-content (generic function)

Writers

(setf page-content) (generic function)

5.2 Internal definitions

5.2.1 Generic functions

Generic Function: browser-status OBJECT

Generic Function: (setf browser-status) NEW-VALUE OBJECT

Package

cl-mechanize

Methods

Method: browser-status (BROWSER browser)

Method: (setf browser-status) NEW-VALUE (BROWSER browser)

The HTTP status code of the last request.

Source

cl-mechanize.lisp (file)

Appendix A Indexes

A.1 Concepts

A.2 Functions

A.3 Variables

A.4 Data types