The pgloader Reference Manual

This is the pgloader Reference Manual, generated automatically by Declt version 4.0 beta 2 "William Riker" on Wed May 15 06:31:05 2024 GMT+0.

Table of Contents


1 Introduction


2 Systems

The main system appears first, followed by any subsystem dependency.


2.1 pgloader

Load data into PostgreSQL

Author

Dimitri Fontaine <>

License

The PostgreSQL Licence

Dependencies
  • uiop (system).
  • cl-log (system).
  • postmodern (system).
  • cl-postgres (system).
  • simple-date (system).
  • qmynd (system).
  • split-sequence (system).
  • cl-csv (system).
  • cl-fad (system).
  • lparallel (system).
  • esrap (system).
  • alexandria (system).
  • drakma (system).
  • flexi-streams (system).
  • usocket (system).
  • local-time (system).
  • command-line-arguments (system).
  • db3 (system).
  • ixf (system).
  • py-configparser (system).
  • sqlite (system).
  • cl-base64 (system).
  • trivial-backtrace (system).
  • cl-markdown (system).
  • metabang-bind (system).
  • mssql (system).
  • uuid (system).
  • quri (system).
  • cl-ppcre (system).
  • cl-mustache (system).
  • yason (system).
  • closer-mop (system).
  • zs3 (system).
Source

pgloader.asd.

Child Component

src (module).


3 Modules

Modules are listed depth-first from the system components tree.


3.1 pgloader/src

Source

pgloader.asd.

Parent Component

pgloader (system).

Child Components

3.2 pgloader/src/monkey

Source

pgloader.asd.

Parent Component

src (module).

Child Components

3.3 pgloader/src/utils

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Child Components

3.4 pgloader/src/pgsql

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Child Components

3.5 pgloader/src/sources

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Child Components

3.6 pgloader/src/sources/common

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.7 pgloader/src/sources/csv

Dependency

common (module).

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.8 pgloader/src/sources/fixed

Dependency

common (module).

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.9 pgloader/src/sources/db3

Dependencies
Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.10 pgloader/src/sources/ixf

Dependency

common (module).

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.11 pgloader/src/sources/sqlite

Dependency

common (module).

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.12 pgloader/src/sources/mssql

Dependency

common (module).

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.13 pgloader/src/sources/mysql

Dependency

common (module).

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.14 pgloader/src/sources/pgsql

Dependency

common (module).

Source

pgloader.asd.

Parent Component

sources (module).

Child Components

3.15 pgloader/src/pg-copy

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Child Components

3.16 pgloader/src/load

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Child Components

3.18 pgloader/src/regress

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Child Component

regress.lisp (file).


4 Files

Files are sorted by type and then listed depth-first from the systems components trees.


4.1 Lisp


4.1.1 pgloader/pgloader.asd

Source

pgloader.asd.

Parent Component

pgloader (system).

ASDF Systems

pgloader.


4.1.2 pgloader/src/params.lisp

Source

pgloader.asd.

Parent Component

src (module).

Packages

pgloader.params.

Public Interface
Internals

4.1.4 pgloader/src/monkey/bind.lisp

Source

pgloader.asd.

Parent Component

monkey (module).


4.1.5 pgloader/src/monkey/mssql.lisp

Source

pgloader.asd.

Parent Component

monkey (module).


4.1.6 pgloader/src/utils/charsets.lisp

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.7 pgloader/src/utils/logs.lisp

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.8 pgloader/src/utils/utils.lisp

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface

4.1.9 pgloader/src/utils/state.lisp

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.10 pgloader/src/utils/transforms.lisp

Source

pgloader.asd.

Parent Component

utils (module).

Packages

pgloader.user-symbols.

Public Interface
Internals

4.1.11 pgloader/src/utils/read-sql-files.lisp

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.12 pgloader/src/utils/queries.lisp

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.13 pgloader/src/utils/quoting.lisp

Dependency

utils.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface

4.1.14 pgloader/src/utils/catalog.lisp

Dependency

quoting.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.15 pgloader/src/utils/alter-table.lisp

Dependency

catalog.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.16 pgloader/src/utils/citus.lisp

Dependency

catalog.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.17 pgloader/src/utils/reject.lisp

Dependency

state.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Internals

%process-bad-row (function).


4.1.18 pgloader/src/utils/pretty-print-state.lisp

Dependency

state.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.19 pgloader/src/utils/report.lisp

Dependencies
Source

pgloader.asd.

Parent Component

utils (module).

Public Interface

report-full-summary (function).

Internals

4.1.20 pgloader/src/utils/monitor.lisp

Dependencies
Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.21 pgloader/src/utils/threads.lisp

Dependency

monitor.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface

make-kernel (function).


4.1.22 pgloader/src/utils/archive.lisp

Dependency

monitor.lisp (file).

Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.23 pgloader/src/utils/connection.lisp

Dependencies
Source

pgloader.asd.

Parent Component

utils (module).

Public Interface
Internals

4.1.24 pgloader/src/pgsql/connection.lisp

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface
Internals

4.1.25 pgloader/src/pgsql/pgsql-ddl.lisp

Dependency

connection.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface
Internals

4.1.26 pgloader/src/pgsql/pgsql-ddl-citus.lisp

Dependency

pgsql-ddl.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface

4.1.27 pgloader/src/pgsql/pgsql-schema.lisp

Dependency

pgsql-ddl-citus.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface
Internals

4.1.28 pgloader/src/pgsql/merge-catalogs.lisp

Dependency

pgsql-schema.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface

merge-catalogs (function).

Internals

4.1.29 pgloader/src/pgsql/pgsql-trigger.lisp

Dependency

merge-catalogs.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Internals

4.1.30 pgloader/src/pgsql/pgsql-index-filter.lisp

Dependency

pgsql-trigger.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface

4.1.31 pgloader/src/pgsql/pgsql-finalize-catalogs.lisp

Dependency

pgsql-index-filter.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface
Internals

4.1.32 pgloader/src/pgsql/pgsql-create-schema.lisp

Dependencies
Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface
Internals

4.1.33 pgloader/src/sources/common/api.lisp

Source

pgloader.asd.

Parent Component

common (module).

Public Interface
Internals

4.1.34 pgloader/src/sources/common/methods.lisp

Dependency

api.lisp (file).

Source

pgloader.asd.

Parent Component

common (module).

Public Interface

4.1.35 pgloader/src/sources/common/md-methods.lisp

Dependency

methods.lisp (file).

Source

pgloader.asd.

Parent Component

common (module).

Public Interface

4.1.36 pgloader/src/sources/common/matviews.lisp

Dependency

md-methods.lisp (file).

Source

pgloader.asd.

Parent Component

common (module).

Public Interface

4.1.37 pgloader/src/sources/common/casting-rules.lisp

Dependency

matviews.lisp (file).

Source

pgloader.asd.

Parent Component

common (module).

Public Interface
Internals

4.1.38 pgloader/src/sources/common/files-and-pathnames.lisp

Dependency

casting-rules.lisp (file).

Source

pgloader.asd.

Parent Component

common (module).

Public Interface
Internals

filter-directory (function).


4.1.39 pgloader/src/sources/common/project-fields.lisp

Dependency

files-and-pathnames.lisp (file).

Source

pgloader.asd.

Parent Component

common (module).

Public Interface

4.1.40 pgloader/src/sources/csv/csv-guess.lisp

Source

pgloader.asd.

Parent Component

csv (module).

Public Interface

guess-csv-params (function).

Internals

4.1.41 pgloader/src/sources/csv/csv.lisp

Source

pgloader.asd.

Parent Component

csv (module).

Public Interface
Internals

4.1.42 pgloader/src/sources/fixed/fixed-guess.lisp

Source

pgloader.asd.

Parent Component

fixed (module).

Internals

4.1.43 pgloader/src/sources/fixed/fixed.lisp

Dependency

fixed-guess.lisp (file).

Source

pgloader.asd.

Parent Component

fixed (module).

Public Interface
Internals

parse-row (function).


4.1.44 pgloader/src/sources/copy.lisp

Dependencies
Source

pgloader.asd.

Parent Component

sources (module).

Public Interface
Internals

4.1.45 pgloader/src/sources/db3/db3-cast-rules.lisp

Source

pgloader.asd.

Parent Component

db3 (module).

Public Interface
Internals

4.1.46 pgloader/src/sources/db3/db3-connection.lisp

Dependency

db3-cast-rules.lisp (file).

Source

pgloader.asd.

Parent Component

db3 (module).

Public Interface
Internals

4.1.47 pgloader/src/sources/db3/db3-schema.lisp

Dependency

db3-connection.lisp (file).

Source

pgloader.asd.

Parent Component

db3 (module).

Public Interface

4.1.48 pgloader/src/sources/db3/db3.lisp

Dependency

db3-schema.lisp (file).

Source

pgloader.asd.

Parent Component

db3 (module).

Public Interface

4.1.49 pgloader/src/sources/ixf/ixf-cast-rules.lisp

Source

pgloader.asd.

Parent Component

ixf (module).

Public Interface

cast (method).

Internals

4.1.50 pgloader/src/sources/ixf/ixf-connection.lisp

Dependency

ixf-cast-rules.lisp (file).

Source

pgloader.asd.

Parent Component

ixf (module).

Public Interface

4.1.51 pgloader/src/sources/ixf/ixf-schema.lisp

Dependency

ixf-connection.lisp (file).

Source

pgloader.asd.

Parent Component

ixf (module).

Public Interface
Internals

4.1.52 pgloader/src/sources/ixf/ixf.lisp

Dependency

ixf-schema.lisp (file).

Source

pgloader.asd.

Parent Component

ixf (module).

Public Interface

4.1.53 pgloader/src/sources/sqlite/sqlite-cast-rules.lisp

Source

pgloader.asd.

Parent Component

sqlite (module).

Public Interface
Internals

4.1.54 pgloader/src/sources/sqlite/sqlite-connection.lisp

Dependency

sqlite-cast-rules.lisp (file).

Source

pgloader.asd.

Parent Component

sqlite (module).

Public Interface
Internals

4.1.55 pgloader/src/sources/sqlite/sqlite-schema.lisp

Dependency

sqlite-connection.lisp (file).

Source

pgloader.asd.

Parent Component

sqlite (module).

Public Interface
Internals

4.1.56 pgloader/src/sources/sqlite/sqlite.lisp

Dependency

sqlite-schema.lisp (file).

Source

pgloader.asd.

Parent Component

sqlite (module).

Public Interface
Internals

parse-value (function).


4.1.57 pgloader/src/sources/mssql/mssql-cast-rules.lisp

Source

pgloader.asd.

Parent Component

mssql (module).

Public Interface
Internals

4.1.58 pgloader/src/sources/mssql/mssql-connection.lisp

Dependency

mssql-cast-rules.lisp (file).

Source

pgloader.asd.

Parent Component

mssql (module).

Public Interface
Internals

4.1.59 pgloader/src/sources/mssql/mssql-schema.lisp

Dependency

mssql-connection.lisp (file).

Source

pgloader.asd.

Parent Component

mssql (module).

Public Interface
Internals

*table-type* (special variable).


4.1.60 pgloader/src/sources/mssql/mssql.lisp

Dependency

mssql-schema.lisp (file).

Source

pgloader.asd.

Parent Component

mssql (module).

Public Interface

4.1.61 pgloader/src/sources/mssql/mssql-index-filters.lisp

Dependency

mssql.lisp (file).

Source

pgloader.asd.

Parent Component

mssql (module).

Public Interface

translate-index-filter (method).

Internals

parse-index-filter-clause (function).


4.1.62 pgloader/src/sources/mysql/mysql-cast-rules.lisp

Source

pgloader.asd.

Parent Component

mysql (module).

Public Interface
Internals

4.1.63 pgloader/src/sources/mysql/mysql-connection.lisp

Dependency

mysql-cast-rules.lisp (file).

Source

pgloader.asd.

Parent Component

mysql (module).

Public Interface
Internals

4.1.64 pgloader/src/sources/mysql/mysql-schema.lisp

Dependency

mysql-connection.lisp (file).

Source

pgloader.asd.

Parent Component

mysql (module).

Public Interface
Internals

4.1.65 pgloader/src/sources/mysql/mysql.lisp

Dependency

mysql-schema.lisp (file).

Source

pgloader.asd.

Parent Component

mysql (module).

Public Interface
Internals

4.1.66 pgloader/src/sources/pgsql/pgsql-cast-rules.lisp

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface
Internals

pgsql-column-ctype (method).


4.1.67 pgloader/src/sources/pgsql/pgsql.lisp

Dependency

pgsql-cast-rules.lisp (file).

Source

pgloader.asd.

Parent Component

pgsql (module).

Public Interface

4.1.68 pgloader/src/pg-copy/copy-batch.lisp

Source

pgloader.asd.

Parent Component

pg-copy (module).

Internals

4.1.69 pgloader/src/pg-copy/copy-format.lisp

Dependency

copy-batch.lisp (file).

Source

pgloader.asd.

Parent Component

pg-copy (module).

Public Interface

format-vector-row (function).

Internals

4.1.70 pgloader/src/pg-copy/copy-db-write.lisp

Dependency

copy-format.lisp (file).

Source

pgloader.asd.

Parent Component

pg-copy (module).

Public Interface

copy-init-error (condition).

Internals

4.1.71 pgloader/src/pg-copy/copy-rows-in-stream.lisp

Dependency

copy-db-write.lisp (file).

Source

pgloader.asd.

Parent Component

pg-copy (module).

Internals

4.1.72 pgloader/src/pg-copy/copy-rows-in-batch.lisp

Dependency

copy-rows-in-stream.lisp (file).

Source

pgloader.asd.

Parent Component

pg-copy (module).

Internals

4.1.73 pgloader/src/pg-copy/copy-rows-in-batch-through-s3.lisp

Dependency

copy-rows-in-batch.lisp (file).

Source

pgloader.asd.

Parent Component

pg-copy (module).

Internals

4.1.74 pgloader/src/pg-copy/copy-retry-batch.lisp

Dependency

copy-rows-in-batch-through-s3.lisp (file).

Source

pgloader.asd.

Parent Component

pg-copy (module).

Internals

4.1.75 pgloader/src/pg-copy/copy-from-queue.lisp

Dependency

copy-retry-batch.lisp (file).

Source

pgloader.asd.

Parent Component

pg-copy (module).

Public Interface

copy-rows-from-queue (function).

Internals

prepare-copy-parameters (function).


4.1.76 pgloader/src/load/api.lisp

Source

pgloader.asd.

Parent Component

load (module).

Public Interface

4.1.77 pgloader/src/load/copy-data.lisp

Dependency

api.lisp (file).

Source

pgloader.asd.

Parent Component

load (module).

Public Interface

4.1.78 pgloader/src/load/load-file.lisp

Dependency

copy-data.lisp (file).

Source

pgloader.asd.

Parent Component

load (module).

Public Interface

copy-database (method).


4.1.79 pgloader/src/load/migrate-database.lisp

Dependency

load-file.lisp (file).

Source

pgloader.asd.

Parent Component

load (module).

Public Interface
Internals

4.1.80 pgloader/src/parsers/parse-ini.lisp

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface
Internals

4.1.81 pgloader/src/parsers/template.lisp

Dependency

parse-ini.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

initialize-context (function).

Internals

4.1.82 pgloader/src/parsers/command-utils.lisp

Dependency

template.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

4.1.83 pgloader/src/parsers/command-keywords.lisp

Dependency

command-utils.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

def-keyword-rule (macro).


4.1.84 pgloader/src/parsers/command-regexp.lisp

Dependency

command-keywords.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

process-quoted-regex (function).


4.1.85 pgloader/src/parsers/parse-pgpass.lisp

Dependency

command-regexp.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

4.1.86 pgloader/src/parsers/command-db-uri.lisp

Dependency

parse-pgpass.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

4.1.87 pgloader/src/parsers/command-source.lisp

Dependency

command-db-uri.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

filename-character-p (function).


4.1.88 pgloader/src/parsers/command-options.lisp

Dependency

command-source.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

4.1.89 pgloader/src/parsers/command-sql-block.lisp

Dependency

command-options.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

execute-sql-code-block (function).

Internals

sql-code-block (function).


4.1.90 pgloader/src/parsers/command-sexp.lisp

Dependency

command-sql-block.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

4.1.91 pgloader/src/parsers/command-csv.lisp

Dependency

command-sexp.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-csv (function).

Internals

4.1.92 pgloader/src/parsers/command-ixf.lisp

Dependency

command-csv.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-ixf (function).


4.1.93 pgloader/src/parsers/command-fixed.lisp

Dependency

command-ixf.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-fixed (function).


4.1.94 pgloader/src/parsers/command-copy.lisp

Dependency

command-fixed.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-copy (function).


4.1.95 pgloader/src/parsers/command-dbf.lisp

Dependency

command-copy.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-dbf (function).

Internals

lisp-code-for-dbf-dry-run (function).


4.1.96 pgloader/src/parsers/command-cast-rules.lisp

Dependency

command-dbf.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

4.1.97 pgloader/src/parsers/command-materialize-views.lisp

Dependency

command-cast-rules.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).


4.1.98 pgloader/src/parsers/command-alter-table.lisp

Dependency

command-materialize-views.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).


4.1.99 pgloader/src/parsers/command-distribute.lisp

Dependency

command-alter-table.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Internals

create-table-from-dsn-table-name (function).


4.1.100 pgloader/src/parsers/command-mysql.lisp

Dependency

command-distribute.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-mysql (function).

Internals

lisp-code-for-mysql-dry-run (function).


4.1.101 pgloader/src/parsers/command-including-like.lisp

Dependency

command-mysql.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).


4.1.102 pgloader/src/parsers/command-mssql.lisp

Dependency

command-including-like.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-mssql (function).

Internals

lisp-code-for-mssql-dry-run (function).


4.1.103 pgloader/src/parsers/command-sqlite.lisp

Dependency

command-mssql.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-sqlite (function).

Internals

lisp-code-for-sqlite-dry-run (function).


4.1.104 pgloader/src/parsers/command-pgsql.lisp

Dependency

command-sqlite.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

lisp-code-for-loading-from-pgsql (function).

Internals

lisp-code-for-pgsql-dry-run (function).


4.1.105 pgloader/src/parsers/command-archive.lisp

Dependency

command-pgsql.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).


4.1.106 pgloader/src/parsers/command-parser.lisp

Dependency

command-archive.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface
Internals

4.1.107 pgloader/src/parsers/parse-sqlite-type-name.lisp

Dependency

command-parser.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface

parse-sqlite-type-name (function).


4.1.108 pgloader/src/parsers/date-format.lisp

Dependency

parse-sqlite-type-name.lisp (file).

Source

pgloader.asd.

Parent Component

parsers (module).

Public Interface
Internals

4.1.109 pgloader/src/api.lisp

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Public Interface
Internals

4.1.110 pgloader/src/regress/regress.lisp

Source

pgloader.asd.

Parent Component

regress (module).

Internals

4.1.111 pgloader/src/main.lisp

Dependencies
Source

pgloader.asd.

Parent Component

src (module).

Internals

5 Packages

Packages are listed by definition order.


5.1 pgloader.syslog

Source

package.lisp.

Use List

5.2 pgloader.user-symbols

Source

transforms.lisp.


5.3 pgloader.source.fixed

Source

package.lisp.

Use List
Public Interface
Internals

5.4 pgloader.connection

Source

package.lisp.

Use List
Used By List
Public Interface
Internals

5.6 pgloader.source.mssql

Source

package.lisp.

Use List
Used By List

pgloader.source.mssql.index-filter.

Public Interface
Internals

5.7 pgloader.source.csv

Source

package.lisp.

Use List
Public Interface
Internals

5.8 pgloader.archive

Source

package.lisp.

Use List
Used By List

pgloader.connection.

Public Interface
Internals

5.9 pgloader.pgsql

Source

package.lisp.

Use List
Used By List
Public Interface
Internals

5.10 pgloader.logs

Source

package.lisp.

Use List
Public Interface
Internals

5.11 pgloader.queries

Source

package.lisp.

Use List
Used By List

pgloader.utils.

Public Interface
Internals

5.12 pgloader.parse-date

Source

package.lisp.

Use List
  • common-lisp.
  • esrap.
Public Interface
Internals

5.13 pgloader.transforms

Source

package.lisp.

Use List

common-lisp.

Used By List

pgloader.pgsql.

Public Interface
Internals

5.14 pgloader.source.mysql

Source

package.lisp.

Use List
Public Interface
Internals

5.15 pgloader.pgcopy

Source

package.lisp.

Use List
Used By List
Public Interface
Internals

5.16 pgloader.source.pgsql

Source

package.lisp.

Use List
Public Interface
Internals

pgsql-column-ctype (generic function).


5.17 pgloader.source.mssql.index-filter

Source

package.lisp.

Use List
Internals

parse-index-filter-clause (function).


5.18 pgloader.sql

Source

package.lisp.

Use List

common-lisp.

Used By List

pgloader.parser.

Public Interface

read-queries (function).

Internals

5.19 pgloader

Source

package.lisp.

Use List
Public Interface
Internals

5.20 pgloader.source.copy

Source

package.lisp.

Use List
Public Interface
Internals

5.21 pgloader.catalog

Source

package.lisp.

Use List
Used By List
Public Interface
Internals

5.22 pgloader.source.sqlite

Source

package.lisp.

Use List
Public Interface
Internals

5.23 pgloader.state

Source

package.lisp.

Use List
Used By List
Public Interface
Internals

5.24 pgloader.parser

Source

package.lisp.

Use List
Used By List

pgloader.

Public Interface
Internals

5.25 pgloader.source.ixf

Source

package.lisp.

Use List
Public Interface
Internals

5.26 pgloader.quoting

Source

package.lisp.

Use List
Used By List
Public Interface

5.27 pgloader.sources

Source

package.lisp.

Use List
Used By List
Public Interface
Internals

5.28 pgloader.ini

Source

package.lisp.

Use List
Public Interface
Internals

5.29 pgloader.citus

Source

package.lisp.

Use List
Used By List

pgloader.utils.

Public Interface
Internals

5.30 pgloader.load

Source

package.lisp.

Use List
Public Interface
Internals

5.31 pgloader.monitor

Source

package.lisp.

Use List
Used By List
Public Interface
Internals

5.32 pgloader.source.db3

Source

package.lisp.

Use List
Public Interface
Internals

5.33 pgloader.params

Source

params.lisp.

Use List

common-lisp.

Used By List
Public Interface
Internals

6 Definitions

Definitions are sorted by export status, category, package, and then by lexicographic order.


6.1 Public Interface


6.1.1 Special variables

Special Variable: *cast-rules*

Specific casting rules added in the command.

Package

pgloader.sources.

Source

casting-rules.lisp.

Special Variable: *client-min-messages*
Package

pgloader.params.

Source

params.lisp.

Special Variable: *context*

Alist of (names . values) intialized from the environment at run-time, and from a –context command line argument, then used in the commands when they are using the Mustache templating feature.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *copy-batch-rows*

How many rows to batch per COPY transaction.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *copy-batch-size*

Maximum memory size allowed for a single batch.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *db3-default-cast-rules*

Data Type Casting rules to migrate from DB3 to PostgreSQL

Package

pgloader.source.db3.

Source

db3-cast-rules.lisp.

Special Variable: *decoding-as*

Special per-table encoding/decoding overloading rules for MySQL.

Package

pgloader.source.mysql.

Source

mysql.lisp.

Special Variable: *default-cast-rules*

Default casting rules.

Package

pgloader.sources.

Source

casting-rules.lisp.

Special Variable: *default-tmpdir*

Place where to fetch and expand archives on-disk.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *dry-run*

Set to non-nil to only run checks about the load setup.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *fd-path-root*

Where to load files from, when loading from an archive or expanding regexps.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *identifier-case*

Dealing with source databases casing rules.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *log-filename*

Main pgloader log file

Package

pgloader.params.

Source

params.lisp.

Special Variable: *log-messengers*

Currently active log-messengers

Package

pgloader.logs.

Source

logs.lisp.

Special Variable: *log-min-messages*
Package

pgloader.params.

Source

params.lisp.

Special Variable: *monitoring-kernel*

Internal lparallel kernel to manage the separate monitor thread.

Package

pgloader.monitor.

Source

monitor.lisp.

Special Variable: *monitoring-queue*

Internal lparallel queue where to send and receive messages from.

Package

pgloader.monitor.

Source

monitor.lisp.

Special Variable: *mssql-default-cast-rules*

Data Type Casting to migrate from MSSQL to PostgreSQL

Package

pgloader.source.mssql.

Source

mssql-cast-rules.lisp.

Special Variable: *mssql-settings*

An alist of GUC names and values.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *mysql-default-cast-rules*

Data Type Casting rules to migrate from MySQL to PostgreSQL

Package

pgloader.source.mysql.

Source

mysql-cast-rules.lisp.

Special Variable: *mysql-settings*

An alist of GUC names and values.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *on-error-stop*

Set to non-nil to for pgloader to refrain from handling errors, quitting instead.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *pg-settings*

An alist of GUC names and values.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *pgsql-default-cast-rules*

Data Type Casting to migrate from PostgtreSQL to PostgreSQL

Package

pgloader.source.pgsql.

Source

pgsql-cast-rules.lisp.

Special Variable: *pgsql-reserved-keywords*

We need to always quote PostgreSQL reserved keywords

Package

pgloader.params.

Source

params.lisp.

Special Variable: *prefetch-rows*

How many rows do read in advance in the reader queue.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *preserve-index-names*

Dealing with source databases index naming.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *report-stream*

Stream where to format the output stream.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *root-dir*

Top directory where to store all data logs and reject files.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *rows-per-range*

How many rows to read in each reader’s thread, per SQL query.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *self-upgrade-immutable-systems*

Used for –self-upgrade.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *sqlite-default-cast-rules*

Data Type Casting to migrate from SQLite to PostgreSQL

Package

pgloader.source.sqlite.

Source

sqlite-cast-rules.lisp.

Special Variable: *summary-pathname*

Pathname where to output the summary.

Package

pgloader.params.

Source

params.lisp.

Special Variable: *version-string*

pgloader version strings, following Emacs versionning model.

Package

pgloader.params.

Source

params.lisp.

Special Variable: +os-code-error+
Package

pgloader.params.

Source

params.lisp.

Special Variable: +os-code-error-bad-source+
Package

pgloader.params.

Source

params.lisp.

Special Variable: +os-code-error-regress+
Package

pgloader.params.

Source

params.lisp.

Special Variable: +os-code-error-usage+
Package

pgloader.params.

Source

params.lisp.

Special Variable: +os-code-success+
Package

pgloader.params.

Source

params.lisp.


6.1.2 Macros

Macro: log-message (category description &rest arguments)

Protect against evaluating ARGUMENTS in cases where we don’t log at the given CATEGORY log-level.

Package

pgloader.monitor.

Source

monitor.lisp.

Macro: push-to-end (item place)
Package

pgloader.catalog.

Source

catalog.lisp.

Macro: timing (&body forms)

return both how much real time was spend in body and its result

Package

pgloader.monitor.

Source

monitor.lisp.

Macro: with-connection ((var connection &rest args &key &allow-other-keys) &body forms)

Connect to DB-CONNECTION and handle any condition when doing so, and when connected execute FORMS in a protected way so that we always disconnect at the end.

Package

pgloader.connection.

Source

connection.lisp.

Macro: with-disabled-triggers ((table-name &key disable-triggers) &body forms)

Run FORMS with PostgreSQL triggers disabled for TABLE-NAME if DISABLE-TRIGGERS is T A PostgreSQL connection must be opened already where this macro is used.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Macro: with-monitor ((&key start-logger) &body body)

Start and stop the monitor around BODY code. The monitor is responsible for processing logs into a central logfile

Package

pgloader.monitor.

Source

monitor.lisp.

Macro: with-pgsql-connection ((pgconn) &body forms)

Run FROMS within a PostgreSQL connection to DBNAME. To get the connection spec from the DBNAME, use ‘get-connection-spec’.

Package

pgloader.pgsql.

Source

connection.lisp.

Macro: with-pgsql-transaction ((&key pgconn database) &body forms)

Run FORMS within a PostgreSQL transaction to DBNAME, reusing DATABASE if given.

Package

pgloader.pgsql.

Source

connection.lisp.

Macro: with-schema ((var table-name) &body body)

When table-name is a CONS, SET search_path TO its CAR and return its CDR, otherwise just return the TABLE-NAME. A PostgreSQL connection must be established when calling this function.

Package

pgloader.catalog.

Source

catalog.lisp.

Macro: with-stats-collection ((table-name &key section dbname use-result-as-read use-result-as-rows) &body forms)

Measure time spent in running BODY into STATE, accounting the seconds to given DBNAME and TABLE-NAME

Package

pgloader.monitor.

Source

monitor.lisp.


6.1.3 Ordinary functions

Function: add-to-search-path (catalog &key label section log-level client-min-messages)

Add catalog schemas in the database search_path.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: apply-casting-rules (table-name column-name dtype ctype default nullable extra &key rules)

Apply the given RULES to the MySQL SOURCE type definition

Package

pgloader.sources.

Source

casting-rules.lisp.

Function: apply-identifier-case (identifier)

Return given IDENTIFIER with CASE handled to be PostgreSQL compatible.

Package

pgloader.quoting.

Source

quoting.lisp.

Function: archivep (archive-file)

Return non-nil when the ARCHIVE-FILE is something we know how to expand.

Package

pgloader.archive.

Source

archive.lisp.

Function: build-identifier (sep &rest parts)

Concatenante PARTS into a PostgreSQL identifier, with SEP in between
parts. That’s useful for creating an index name from a table’s oid and name.

Package

pgloader.quoting.

Source

quoting.lisp.

Function: camelcase-to-colname (string)

Transform input STRING into a suitable column name. lahmanID lahman_id
playerID player_id
birthYear birth_year

Package

pgloader.quoting.

Source

quoting.lisp.

Reader: catalog-distribution-rules (instance)
Writer: (setf catalog-distribution-rules) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

distribution-rules.

Reader: catalog-name (instance)
Writer: (setf catalog-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: catalog-schema-list (instance)
Writer: (setf catalog-schema-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

schema-list.

Reader: catalog-types-without-btree (instance)
Writer: (setf catalog-types-without-btree) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

types-without-btree.

Function: citus-backfill-table-p (table)

Returns non-nil when given TABLE should be backfilled with the distribution key.

Package

pgloader.catalog.

Source

citus.lisp.

Function: citus-distribute-schema (catalog distribution-rules)

Distribute a CATALOG with given user provided DISTRIBUTION-RULES. Return the list of rules applied.

Package

pgloader.citus.

Source

citus.lisp.

Reader: citus-distributed-rule-from (instance)
Writer: (setf citus-distributed-rule-from) (instance)
Package

pgloader.catalog.

Source

citus.lisp.

Target Slot

from.

Function: citus-distributed-rule-p (object)
Package

pgloader.citus.

Source

citus.lisp.

Reader: citus-distributed-rule-table (instance)
Writer: (setf citus-distributed-rule-table) (instance)
Package

pgloader.catalog.

Source

citus.lisp.

Target Slot

table.

Reader: citus-distributed-rule-using (instance)
Writer: (setf citus-distributed-rule-using) (instance)
Package

pgloader.catalog.

Source

citus.lisp.

Target Slot

using.

Function: citus-format-sql-select (source-table target-table)

Return the SQL statement to use to fetch data from the COPY context, including backfilling the distribution key in related tables.

Package

pgloader.catalog.

Source

citus.lisp.

Function: citus-reference-rule-p (object)
Package

pgloader.citus.

Source

citus.lisp.

Reader: citus-reference-rule-table (instance)
Writer: (setf citus-reference-rule-table) (instance)
Package

pgloader.citus.

Source

citus.lisp.

Target Slot

table.

Reader: column-comment (instance)
Writer: (setf column-comment) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

comment.

Reader: column-default (instance)
Writer: (setf column-default) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

default.

Reader: column-extra (instance)
Writer: (setf column-extra) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

extra.

Reader: column-name (instance)
Writer: (setf column-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: column-nullable (instance)
Writer: (setf column-nullable) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

nullable.

Reader: column-transform (instance)
Writer: (setf column-transform) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

transform.

Reader: column-transform-default (instance)
Writer: (setf column-transform-default) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

transform-default.

Reader: column-type-mod (instance)
Writer: (setf column-type-mod) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

type-mod.

Reader: column-type-name (instance)
Writer: (setf column-type-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

type-name.

Function: comment-on-tables-and-columns (catalog &key label section)

Install comments on tables and columns from CATALOG.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: convert-ini-into-commands (filename)

Read the INI file at FILENAME and convert each section of it to a command in the new pgloader format.

Package

pgloader.ini.

Source

parse-ini.lisp.

Function: convert-ini-into-files (filename target-directory &key with-data-inline include-sql-file)

Reads the INI file at FILENAME and creates files names <section>.load for each section in the INI file, in TARGET-DIRECTORY.

When WITH-DATA-INLINE is true, read the CSV file listed as the section’s filename and insert its content in the command itself, as inline data.

When INCLUDE-SQL-FILE is :if-exists, try to find a sibling file to the data file, with the same name and with the "sql" type, and use its content in a BEFORE LOAD DO clause.

When INCLUDE-SQL-FILE is t, not finding the SQL file is an error.

Package

pgloader.ini.

Source

parse-ini.lisp.

Function: copy-rows-from-queue (copy queue &key disable-triggers on-error-stop columns)

Fetch rows from the QUEUE, prepare them in batches and send them down to PostgreSQL, and when that’s done update stats.

Package

pgloader.pgcopy.

Source

copy-from-queue.lisp.

Function: create-distributed-table (distribute-rules)
Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-extensions (catalog &key if-not-exists include-drop client-min-messages)

Create all extensions from the given database CATALOG.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-indexes-again (target catalog &key max-parallel-create-index section drop-indexes)

Create the indexes that we dropped previously.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-indexes-in-kernel (pgconn table kernel channel &key label)

Create indexes for given table in dbname, using given lparallel KERNEL and CHANNEL so that the index build happen in concurrently with the data copying.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-pgsql-fkeys (catalog &key section label log-level)

Actually create the Foreign Key References that where declared in the MySQL database

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-schemas (catalog &key include-drop client-min-messages)

Create all schemas from the given database CATALOG.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-sqltypes (catalog &key if-not-exists include-drop client-min-messages)

Create the needed data types for given CATALOG.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-state ()
Package

pgloader.state.

Source

state.lisp.

Function: create-table (maybe-qualified-name)

Create a table instance from the db-uri component, either a string or a cons of two strings: (schema . table).

Package

pgloader.catalog.

Source

catalog.lisp.

Function: create-tables (catalog &key if-not-exists include-drop client-min-messages)

Create all tables from the given database CATALOG.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-triggers (catalog &key label section client-min-messages)

Create the catalog objects that come after the data has been loaded.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: create-views (catalog &key if-not-exists include-drop client-min-messages)

Create all tables from the given database CATALOG.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: distribute (list-of-ranges count)

Split a list of ranges into COUNT sublists.

Package

pgloader.utils.

Source

utils.lisp.

Function: drop-indexes (table-or-catalog &key cascade log-level)

Drop indexes in PGSQL-INDEX-LIST. A PostgreSQL connection must already be active when calling that function.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: drop-pgsql-fkeys (catalog &key cascade log-level)

Drop all Foreign Key Definitions given, to prepare for a clean run.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: elapsed-time-since (start &optional end)

Return how many seconds ticked between START and now

Package

pgloader.monitor.

Source

monitor.lisp.

Function: ensure-quoted (value &optional quote-char)
Package

pgloader.quoting.

Source

quoting.lisp.

Function: ensure-unquoted (identifier &optional quote-char)
Package

pgloader.quoting.

Source

quoting.lisp.

Function: execute-sql-code-block (pgconn section commands label)

Exceute given SQL commands.

Package

pgloader.parser.

Source

command-sql-block.lisp.

Function: expand-archive (archive-file &key tmpdir)

Expand given ARCHIVE-FILE in TMPDIR/(pathname-name ARCHIVE-FILE). Return the pathname where we did expand the archive file.

Package

pgloader.archive.

Source

archive.lisp.

Function: expand-user-homedir-pathname (namestring)

Expand NAMESTRING replacing leading ~ with (user-homedir-pathname)

Package

pgloader.utils.

Source

utils.lisp.

Reader: extension-name (instance)
Writer: (setf extension-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: extension-schema (instance)
Writer: (setf extension-schema) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

schema.

Function: fetch-pgsql-catalog (dbname &key table source-catalog including excluding variant pgversion)

Fetch PostgreSQL catalogs for the target database. A PostgreSQL connection must be opened.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: finalize-catalogs (catalog variant)

Finalize the target PostgreSQL catalogs, dumbing down datatypes when the target actually is Redshift rather than core PostgreSQL.

Package

pgloader.pgsql.

Source

pgsql-finalize-catalogs.lisp.

Reader: fkey-columns (instance)
Writer: (setf fkey-columns) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

columns.

Reader: fkey-condef (instance)
Writer: (setf fkey-condef) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

condef.

Reader: fkey-deferrable (instance)
Writer: (setf fkey-deferrable) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

deferrable.

Reader: fkey-delete-rule (instance)
Writer: (setf fkey-delete-rule) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

delete-rule.

Reader: fkey-foreign-columns (instance)
Writer: (setf fkey-foreign-columns) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

foreign-columns.

Reader: fkey-foreign-table (instance)
Writer: (setf fkey-foreign-table) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

foreign-table.

Reader: fkey-initially-deferred (instance)
Writer: (setf fkey-initially-deferred) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

initially-deferred.

Reader: fkey-match-rule (instance)
Writer: (setf fkey-match-rule) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

match-rule.

Reader: fkey-name (instance)
Writer: (setf fkey-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: fkey-oid (instance)
Writer: (setf fkey-oid) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

oid.

Reader: fkey-table (instance)
Writer: (setf fkey-table) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

table.

Reader: fkey-update-rule (instance)
Writer: (setf fkey-update-rule) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

update-rule.

Function: flush-summary (&key reset)
Package

pgloader.monitor.

Source

monitor.lisp.

Function: format-vector-row (nb-cols row)
Package

pgloader.pgsql.

Source

copy-format.lisp.

Function: get-matching-filenames (directory regex)

Apply given REGEXP to the DIRECTORY contents and return the list of matching files.

Package

pgloader.archive.

Source

archive.lisp.

Function: get-pathname (dbname table-name &key fd-path-root)

Return a pathname where to read or write the file data

Package

pgloader.sources.

Source

files-and-pathnames.lisp.

Function: get-state-section (state section)
Package

pgloader.state.

Source

state.lisp.

Function: getenv-default (name &optional default)

Return the value of the NAME variable as found in the environment, or DEFAULT if that variable isn’t set

Package

pgloader.params.

Source

params.lisp.

Function: guess-csv-params (filename-or-stream nb-cols &key sample-size)

Try a bunch of field separators with LINES and return the first one that returns COLS number of columns

Package

pgloader.source.csv.

Source

csv-guess.lisp.

Function: http-fetch-file (url &key tmpdir)

Download a file from URL into TMPDIR.

Package

pgloader.archive.

Source

archive.lisp.

Reader: index-columns (instance)
Writer: (setf index-columns) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

columns.

Reader: index-condef (instance)
Writer: (setf index-condef) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

condef.

Reader: index-conname (instance)
Writer: (setf index-conname) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

conname.

Reader: index-filter (instance)
Writer: (setf index-filter) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

filter.

Reader: index-fk-deps (instance)
Writer: (setf index-fk-deps) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

fk-deps.

Reader: index-name (instance)
Writer: (setf index-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: index-oid (instance)
Writer: (setf index-oid) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

oid.

Reader: index-primary (instance)
Writer: (setf index-primary) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

primary.

Reader: index-schema (instance)
Writer: (setf index-schema) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

schema.

Reader: index-sql (instance)
Writer: (setf index-sql) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

sql.

Reader: index-table (instance)
Writer: (setf index-table) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

table.

Reader: index-type (instance)
Writer: (setf index-type) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

type.

Reader: index-unique (instance)
Writer: (setf index-unique) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

unique.

Function: init-params-from-environment ()

Some of our parameters get their default value from the env. Do that at runtime when using a compiled binary.

Package

pgloader.params.

Source

params.lisp.

Function: initialize-context (filename)

Initialize a context from the environment variables and from the given context-filename (might be nil). CONTEXT-FILENAME is an INI file.

Package

pgloader.parser.

Source

template.lisp.

Function: intern-symbol (symbol-name &optional overrides)

Return a symbol in either PGLOADER.TRANSFORMS if it exists there
already (it’s a user provided function) or a PGLOADER.USER-SYMBOLS package.

OVERRIDES is an alist of symbol . value, allowing called to force certain values: the classic example is how to parse the "nil" symbol-name. Given OVERRIDES as ’((nil . nil)) the returned symbol will be cl:nil rather than pgloader.user-symbols::nil.

Package

pgloader.transforms.

Source

transforms.lisp.

Function: lisp-code-for-loading-from-copy (copy-conn pg-db-conn &key encoding fields target-table-name columns gucs before after options)
Package

pgloader.parser.

Source

command-copy.lisp.

Function: lisp-code-for-loading-from-csv (csv-conn pg-db-conn &key encoding fields target-table-name columns gucs before after options &allow-other-keys)
Package

pgloader.parser.

Source

command-csv.lisp.

Function: lisp-code-for-loading-from-dbf (dbf-db-conn pg-db-conn &key target-table-name encoding gucs casts options before after-schema after &allow-other-keys)
Package

pgloader.parser.

Source

command-dbf.lisp.

Function: lisp-code-for-loading-from-fixed (fixed-conn pg-db-conn &key encoding fields target-table-name columns gucs before after options &allow-other-keys)
Package

pgloader.parser.

Source

command-fixed.lisp.

Function: lisp-code-for-loading-from-ixf (ixf-db-conn pg-db-conn &key target-table-name gucs options before after-schema after &allow-other-keys)
Package

pgloader.parser.

Source

command-ixf.lisp.

Function: lisp-code-for-loading-from-mssql (ms-db-conn pg-db-conn &key gucs mssql-gucs casts before after after-schema options distribute views alter-schema alter-table including excluding &allow-other-keys)
Package

pgloader.parser.

Source

command-mssql.lisp.

Function: lisp-code-for-loading-from-mysql (my-db-conn pg-db-conn &key gucs mysql-gucs casts options views before after after-schema alter-table alter-schema distribute including excluding decoding &allow-other-keys)
Package

pgloader.parser.

Source

command-mysql.lisp.

Function: lisp-code-for-loading-from-pgsql (pg-src-db-conn pg-dst-db-conn &key gucs casts options before after after-schema alter-table alter-schema including excluding views distribute &allow-other-keys)
Package

pgloader.parser.

Source

command-pgsql.lisp.

Function: lisp-code-for-loading-from-sqlite (sqlite-db-conn pg-db-conn &key gucs casts options before after-schema after alter-table alter-schema including excluding &allow-other-keys)
Package

pgloader.parser.

Source

command-sqlite.lisp.

Function: list-all-columns (catalog &key table-type including excluding)

Get the list of PostgreSQL column names per table.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: list-all-fkeys (catalog &key including excluding)

Get the list of PostgreSQL index definitions per table.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: list-all-indexes (catalog &key including excluding pgversion)

Get the list of PostgreSQL index definitions per table.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: list-all-sqltypes (catalog &key including excluding)

Set the catalog’s schema extension list and sqltype list

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: list-encodings-and-aliases ()

Return an alist of encoding names supported by the current implementation, associated with a list of encoding name aliases for each of them.

Package

pgloader.utils.

Source

charsets.lisp.

Function: list-missing-fk-deps (catalog)

Add in the CATALOG the foreign keys we don’t have to deal with directly but that the primary keys we are going to DROP then CREATE again depend on: we need to take care of those first.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: list-reserved-keywords (pgconn)

Connect to PostgreSQL DBNAME and fetch reserved keywords.

Package

pgloader.pgsql.

Source

connection.lisp.

Function: list-schemas ()

Return the list of PostgreSQL schemas in the already established PostgreSQL connection.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: list-table-oids (catalog &key variant)

Return an hash table mapping TABLE-NAME to its OID for all table in the TABLE-NAMES list. A PostgreSQL connection must be established already.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: list-typenames-without-btree-support ()

Fetch PostgresQL data types without btree support, so that it’s possible to later CREATE INDEX ... ON ... USING gist(...), or even something else than gist.

Package

pgloader.pgsql.

Source

connection.lisp.

Function: load-data (&key from into encoding fields target-table-name options gucs casts before after start-logger flush-summary)

Load data from SOURCE into TARGET.

Package

pgloader.

Source

api.lisp.

Function: make-catalog (&key name schema-list types-without-btree distribution-rules)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-citus-distributed-rule (&key table using from)
Package

pgloader.catalog.

Source

citus.lisp.

Function: make-citus-reference-rule (&key table)
Package

pgloader.catalog.

Source

citus.lisp.

Function: make-column (&key table name type-name type-mod nullable default comment transform extra transform-default)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-external-format (name)

Return an object suitable as an external format in the current implementation.

Package

pgloader.utils.

Source

charsets.lisp.

Function: make-fkey (&key name oid table columns pkey foreign-table foreign-columns condef update-rule delete-rule match-rule deferrable initially-deferred)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-including-expr-from-catalog (catalog)

Return an expression suitable to be used as an :including parameter.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: make-including-expr-from-view-names (view-names)

Turn MATERIALIZING VIEWs list of view names into an INCLUDING parameter.

Package

pgloader.pgsql.

Source

pgsql-schema.lisp.

Function: make-index (&key name oid schema table type primary unique columns sql conname condef filter fk-deps)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-kernel (worker-count &key bindings)

Wrapper around lparallel:make-kernel that sets our usual bindings.

Package

pgloader.utils.

Source

threads.lisp.

Function: make-match-rule (&key rule schema action args)
Package

pgloader.catalog.

Source

alter-table.lisp.

Function: make-matview (&key source-name name schema definition)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-pgstate (&key tables tabnames read rows errs secs rs ws bytes)
Package

pgloader.state.

Source

state.lisp.

Function: make-procedure (&key schema name returns language body)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-regex-match-rule (&key target)
Package

pgloader.catalog.

Source

alter-table.lisp.

Function: make-schema (&key source-name name catalog in-search-path table-list view-list matview-list extension-list sqltype-list)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-sqltype (&key name schema type source-def extra extension)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-state (&key preload data postload bytes secs)
Package

pgloader.state.

Source

state.lisp.

Function: make-string-match-rule (&key target)
Package

pgloader.catalog.

Source

alter-table.lisp.

Function: make-table (&key source-name name schema oid comment storage-parameter-list tablespace row-count-estimate field-list column-list index-list fkey-list trigger-list citus-rule)
Package

pgloader.catalog.

Source

catalog.lisp.

Function: make-trigger (&key name table action procedure)
Package

pgloader.catalog.

Source

catalog.lisp.

Reader: match-rule-action (instance)
Writer: (setf match-rule-action) (instance)
Package

pgloader.catalog.

Source

alter-table.lisp.

Target Slot

action.

Reader: match-rule-args (instance)
Writer: (setf match-rule-args) (instance)
Package

pgloader.catalog.

Source

alter-table.lisp.

Target Slot

args.

Reader: match-rule-rule (instance)
Writer: (setf match-rule-rule) (instance)
Package

pgloader.catalog.

Source

alter-table.lisp.

Target Slot

rule.

Reader: match-rule-schema (instance)
Writer: (setf match-rule-schema) (instance)
Package

pgloader.catalog.

Source

alter-table.lisp.

Target Slot

schema.

Reader: matview-definition (instance)
Writer: (setf matview-definition) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

definition.

Reader: matview-name (instance)
Writer: (setf matview-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: matview-schema (instance)
Writer: (setf matview-schema) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

schema.

Reader: matview-source-name (instance)
Writer: (setf matview-source-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

source-name.

Function: maybe-drop-indexes (catalog &key drop-indexes)

Drop the indexes for TABLE-NAME on TARGET PostgreSQL connection, and returns a list of indexes to create again. A PostgreSQL connection must already be active when calling that function.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: merge-catalogs (source-catalog target-catalog)

In order for the data loading to be as fast as possible, we DROP the constraints and indexes on the target table. Once the data is loaded we want to install the same constraints as found pre-existing in the TARGET-CATALOG rather than the one we casted from the SOURCE-CATALOG.

Also, we want to recheck the cast situation and the selected transformation functions of each column.

Package

pgloader.pgsql.

Source

merge-catalogs.lisp.

Function: new-label (section label &optional dbname)

Send an event to create a new LABEL for registering a shared state under SECTION.

Package

pgloader.monitor.

Source

monitor.lisp.

Function: parse-cli-casts (casts)

Parse additional CAST rules when we get them from the CLI.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-cli-encoding (encoding)

Parse the –encoding option

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-cli-fields (type fields)

Parse the –fields option.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-cli-gucs (gucs)

Parse PostgreSQL GUCs as per the SET clause when we get them from the CLI.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-cli-options (type options)

Parse options as per the WITH clause when we get them from the CLI.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-cli-type (type)

Parse the –type option

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-column-typemod (data-type column-type)

Given int(7), returns the number 7.

Beware that some data-type are using a typmod looking definition for things that are not typmods at all: enum.

Package

pgloader.transforms.

Source

transforms.lisp.

Function: parse-commands (commands-template &key start end junk-allowed)

Parse a command and return a LAMBDA form that takes no parameter.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-commands-from-file (maybe-relative-filename)

The command could be using from :inline, in which case we want to parse as much as possible then use the command against an already opened stream where we moved at the beginning of the data.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-date-format (format-string)

Parse a given format string and return a format specification for parse-date-string

Package

pgloader.parse-date.

Source

date-format.lisp.

Function: parse-date-string (date-string &optional format)

Apply this function when input date in like ’20041002152952’

Package

pgloader.parse-date.

Source

date-format.lisp.

Function: parse-ini-file (filename)

Parse an old-style INI file into a list of PARAMS structures

Package

pgloader.ini.

Source

parse-ini.lisp.

Function: parse-source-string (source-string)
Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-source-string-for-type (type source-string)

use the parse rules as per xxx-source rules

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-sql-file (filename)

Parse FILENAME for SQL statements

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-sqlite-type-name (type-name)
Package

pgloader.parser.

Source

parse-sqlite-type-name.lisp.

Function: parse-target-pg-db-uri (command-file)

Partially parse COMMAND-FILE and return its target connection string.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: parse-target-string (target-string)
Package

pgloader.parser.

Source

command-parser.lisp.

Function: pgsql-connect-and-execute-with-timing (pgconn section label sql)

Run pgsql-execute-with-timing within a newly establised connection.

Package

pgloader.pgsql.

Source

connection.lisp.

Function: pgsql-execute (sql &key log-level client-min-messages on-error-stop)

Execute given SQL list of statements in current transaction.

When ON-ERROR-STOP is non-nil (the default), we stop at the first sql statement that fails. That’s because this facility is meant for DDL. With ON_ERROR_STOP nil, log the problem and continue thanks to PostgreSQL savepoints.

Package

pgloader.pgsql.

Source

connection.lisp.

Function: pgsql-execute-with-timing (section label sql-list &key log-level on-error-stop client-min-messages)

Execute given SQL and resgister its timing into STATE.

Package

pgloader.pgsql.

Source

connection.lisp.

Function: pgstate-decf (pgstate name &key read rows errs secs rs ws bytes)
Package

pgloader.state.

Source

state.lisp.

Reader: pgstate-errs (instance)
Writer: (setf pgstate-errs) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

errs.

Function: pgstate-get-label (pgstate name)
Package

pgloader.state.

Source

state.lisp.

Function: pgstate-incf (pgstate name &key read rows errs secs rs ws bytes)
Package

pgloader.state.

Source

state.lisp.

Function: pgstate-new-label (pgstate label)

Instanciate a new pgtable structure to hold our stats, and return it.

Package

pgloader.state.

Source

state.lisp.

Reader: pgstate-read (instance)
Writer: (setf pgstate-read) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

read.

Reader: pgstate-rows (instance)
Writer: (setf pgstate-rows) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

rows.

Reader: pgstate-secs (instance)
Writer: (setf pgstate-secs) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

secs.

Function: pgstate-setf (pgstate name &key read rows errs secs rs ws bytes)
Package

pgloader.state.

Source

state.lisp.

Reader: pgstate-tables (instance)
Writer: (setf pgstate-tables) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

tables.

Reader: pgstate-tabnames (instance)
Writer: (setf pgstate-tabnames) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

tabnames.

Reader: pgtable-bytes (instance)
Writer: (setf pgtable-bytes) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

bytes.

Reader: pgtable-reject-data (instance)
Writer: (setf pgtable-reject-data) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

reject-data.

Reader: pgtable-reject-logs (instance)
Writer: (setf pgtable-reject-logs) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

reject-logs.

Reader: pgtable-rows (instance)
Writer: (setf pgtable-rows) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

rows.

Reader: pgtable-secs (instance)
Writer: (setf pgtable-secs) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

secs.

Reader: pgtable-start (instance)
Writer: (setf pgtable-start) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

start.

Reader: pgtable-stop (instance)
Writer: (setf pgtable-stop) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

stop.

Function: pretty-print-bytes (bytes &key unit)

Return a string to reprensent bytes in human readable format, with units

Package

pgloader.utils.

Source

utils.lisp.

Reader: procedure-body (instance)
Writer: (setf procedure-body) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

body.

Reader: procedure-language (instance)
Writer: (setf procedure-language) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

language.

Reader: procedure-name (instance)
Writer: (setf procedure-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: procedure-returns (instance)
Writer: (setf procedure-returns) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

returns.

Reader: procedure-schema (instance)
Writer: (setf procedure-schema) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

schema.

Function: process-bad-row (table condition data)

Send an event to log the bad row DATA in the reject and log files for given TABLE-NAME (a label in section :data), for reason found in CONDITION.

Package

pgloader.monitor.

Source

monitor.lisp.

Function: process-relative-pathnames (filename command)

Walk the COMMAND to replace relative pathname with absolute ones, merging them within the directory where we found the command FILENAME.

Package

pgloader.parser.

Source

command-parser.lisp.

Function: project-fields (&key fields columns compile)

The simplest projection happens when both FIELDS and COLS are nil: in this case the projection is an identity, we simply return what we got.

Other forms of projections consist of forming columns with the result of applying a transformation function. In that case a cols entry is a list of ’(colname type expression), the expression being the (already compiled) function to use here.

Package

pgloader.sources.

Source

project-fields.lisp.

Function: quoted-p (s &optional quote-char)

Return true if s is a double-quoted string

Package

pgloader.quoting.

Source

quoting.lisp.

Function: read-ini-file (filename)
Package

pgloader.ini.

Source

parse-ini.lisp.

Function: read-queries (filename)

read SQL queries in given file and split them, returns a list

Package

pgloader.sql.

Source

read-sql-files.lisp.

Function: reformat-then-process (&key fields columns target)

Return a lambda form to apply to each row we read.

The lambda closes over the READ paramater, which is a counter of how many lines we did read in the file.

Package

pgloader.sources.

Source

project-fields.lisp.

Reader: regex-match-rule-target (instance)
Writer: (setf regex-match-rule-target) (instance)
Package

pgloader.catalog.

Source

alter-table.lisp.

Target Slot

target.

Function: report-full-summary (state legend total-secs)

Report the full story when given three different sections of reporting.

Package

pgloader.state.

Source

report.lisp.

Function: reset-sequences (target catalog &key section)

Reset all sequences created during this MySQL migration.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: run-commands (source &key start-logger flush-summary summary log-filename log-min-messages client-min-messages)

SOURCE can be a function, which is run, a list, which is compiled as CL code then run, a pathname containing one or more commands that are parsed then run, or a commands string that is then parsed and each command run.

Package

pgloader.

Source

api.lisp.

Function: sanitize-user-gucs (gucs)

Forbid certain actions such as setting a client_encoding different from utf8.

Package

pgloader.pgsql.

Source

connection.lisp.

Reader: schema-catalog (instance)
Writer: (setf schema-catalog) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

catalog.

Reader: schema-extension-list (instance)
Writer: (setf schema-extension-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

extension-list.

Reader: schema-in-search-path (instance)
Writer: (setf schema-in-search-path) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

in-search-path.

Reader: schema-name (instance)
Writer: (setf schema-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: schema-source-name (instance)
Writer: (setf schema-source-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

source-name.

Reader: schema-sqltype-list (instance)
Writer: (setf schema-sqltype-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

sqltype-list.

Reader: schema-table-list (instance)
Writer: (setf schema-table-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

table-list.

Reader: schema-view-list (instance)
Writer: (setf schema-view-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

view-list.

Function: send-event (event)

Add a new event to be processed by the monitor.

Package

pgloader.monitor.

Source

monitor.lisp.

Function: set-table-oids (catalog &key variant)

MySQL allows using the same index name against separate tables, which PostgreSQL forbids. To get unicity in index names without running out of characters (we are allowed only 63), we use the table OID instead.

This function grabs the table OIDs in the PostgreSQL database and update the definitions with them.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: show-encodings ()

List known encodings names and aliases from charsets::*lisp-encodings*.

Package

pgloader.utils.

Source

charsets.lisp.

Function: split-range (min max &optional count)

Split the range from MIN to MAX into sub-ranges of COUNT elements.

Package

pgloader.utils.

Source

utils.lisp.

Function: sql (url &rest args)

Abstract the hash-table based implementation of our SQL file system.

Package

pgloader.queries.

Source

queries.lisp.

Function: sql-url-for-variant (base filename &optional variant)

Build a SQL URL for given VARIANT

Package

pgloader.queries.

Source

queries.lisp.

Reader: sqltype-extension (instance)
Writer: (setf sqltype-extension) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

extension.

Reader: sqltype-extra (instance)
Writer: (setf sqltype-extra) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

extra.

Reader: sqltype-name (instance)
Writer: (setf sqltype-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: sqltype-schema (instance)
Writer: (setf sqltype-schema) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

schema.

Reader: sqltype-source-def (instance)
Writer: (setf sqltype-source-def) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

source-def.

Reader: sqltype-type (instance)
Writer: (setf sqltype-type) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

type.

Function: start-logger (&key log-filename log-min-messages client-min-messages)

Start the pgloader log manager and messenger.

Package

pgloader.logs.

Source

logs.lisp.

Function: start-monitor (&key start-logger queue log-filename log-min-messages client-min-messages)

Start the monitor and its logger.

Package

pgloader.monitor.

Source

monitor.lisp.

Reader: state-data (instance)
Writer: (setf state-data) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

data.

Reader: state-postload (instance)
Writer: (setf state-postload) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

postload.

Reader: state-preload (instance)
Writer: (setf state-preload) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

preload.

Reader: state-secs (instance)
Writer: (setf state-secs) (instance)
Package

pgloader.state.

Source

state.lisp.

Target Slot

secs.

Function: stop-logger ()

Stop the pgloader manager and messengers.

Package

pgloader.logs.

Source

logs.lisp.

Function: stop-monitor (&key kernel channel stop-logger)

Stop the current monitor task.

Package

pgloader.monitor.

Source

monitor.lisp.

Reader: string-match-rule-target (instance)
Writer: (setf string-match-rule-target) (instance)
Package

pgloader.catalog.

Source

alter-table.lisp.

Target Slot

target.

Reader: table-citus-rule (instance)
Writer: (setf table-citus-rule) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

citus-rule.

Reader: table-column-list (instance)
Writer: (setf table-column-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

column-list.

Reader: table-comment (instance)
Writer: (setf table-comment) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

comment.

Reader: table-field-list (instance)
Writer: (setf table-field-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

field-list.

Reader: table-fkey-list (instance)
Writer: (setf table-fkey-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

fkey-list.

Reader: table-index-list (instance)
Writer: (setf table-index-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

index-list.

Reader: table-name (instance)
Writer: (setf table-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Reader: table-oid (instance)
Writer: (setf table-oid) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

oid.

Reader: table-row-count-estimate (instance)
Writer: (setf table-row-count-estimate) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

row-count-estimate.

Reader: table-schema (instance)
Writer: (setf table-schema) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

schema.

Reader: table-source-name (instance)
Writer: (setf table-source-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

source-name.

Reader: table-storage-parameter-list (instance)
Writer: (setf table-storage-parameter-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

storage-parameter-list.

Reader: table-tablespace (instance)
Writer: (setf table-tablespace) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

tablespace.

Reader: table-trigger-list (instance)
Writer: (setf table-trigger-list) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

trigger-list.

Reader: trigger-action (instance)
Writer: (setf trigger-action) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

action.

Reader: trigger-name (instance)
Writer: (setf trigger-name) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

name.

Function: trigger-p (object)
Package

pgloader.catalog.

Source

catalog.lisp.

Reader: trigger-procedure (instance)
Writer: (setf trigger-procedure) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

procedure.

Reader: trigger-table (instance)
Writer: (setf trigger-table) (instance)
Package

pgloader.catalog.

Source

catalog.lisp.

Target Slot

table.

Function: truncate-tables (catalog-or-table)

Truncate given TABLE-NAME in database DBNAME. A PostgreSQL connection must already be active when calling that function.

Package

pgloader.pgsql.

Source

pgsql-create-schema.lisp.

Function: typemod-expr-matches-p (rule-typemod-expr typemod)

Check if an expression such as (< 10) matches given typemod.

Package

pgloader.transforms.

Source

transforms.lisp.

Function: typemod-expr-to-function (expr)

Transform given EXPR into a callable function object.

Package

pgloader.transforms.

Source

transforms.lisp.

Function: unquote (string &optional quote escape)

Given ’0’, returns 0.

Package

pgloader.utils.

Source

utils.lisp.

Function: update-stats (section label &key read rows errs secs rs ws bytes start stop)

Send an event to update stats for given SECTION and LABEL.

Package

pgloader.monitor.

Source

monitor.lisp.

Function: write-command-to-string (config section &key with-data-inline end-command)

Return the new syntax for the command found in SECTION.

When WITH-DATA-INLINE is true, instead of using the SECTION’s filename option, use the constant INLINE in the command.

Package

pgloader.ini.

Source

parse-ini.lisp.


6.1.4 Generic functions

Generic Function: add-column (object column &key)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-column ((index index) column &key)

Add COLUMN name to INDEX and return the INDEX.

Method: add-column ((table table) column &key)

Add COLUMN to TABLE and return the TABLE.

Generic Function: add-extension (object extension-name &key)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-extension ((schema schema) extension-name &key)

Add EXTENSION-NAME to SCHEMA and return the new extension instance.

Generic Function: add-field (table field &key)
Package

pgloader.catalog.

Methods
Method: add-field ((table table) field &key)

Add COLUMN to TABLE and return the TABLE.

Source

catalog.lisp.

Generic Function: add-fkey (object fkey &key)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-fkey ((table table) fkey &key)

Add FKEY to TABLE and return the TABLE.

Generic Function: add-index (object index &key)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-index ((table table) index &key)

Add INDEX to TABLE and return the TABLE.

Generic Function: add-schema (object schema-name &key in-search-path)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-schema ((catalog catalog) schema-name &key in-search-path)

Add SCHEMA-NAME to CATALOG and return the new schema instance.

Generic Function: add-sqltype (object column &key)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-sqltype ((schema schema) sqltype &key)

Add SQLTYPE instance to SCHEMA and return SQLTYPE.

Generic Function: add-table (object table-name &key comment oid)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-table ((schema schema) table-name &key comment oid)

Add TABLE-NAME to SCHEMA and return the new table instance.

Generic Function: add-view (object view-name &key comment)
Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: add-view ((schema schema) view-name &key comment)

Add TABLE-NAME to SCHEMA and return the new table instance.

Generic Function: adjust-data-types (catalog variant)

Adjust PostgreSQL data types depending on the variant we target.

Package

pgloader.pgsql.

Source

pgsql-finalize-catalogs.lisp.

Methods
Method: adjust-data-types ((catalog catalog) (variant (eql :redshift)))
Method: adjust-data-types ((catalog catalog) (variant (eql :pgdg)))
Generic Function: alter-schema (object alter-schema-rule-list)
Package

pgloader.catalog.

Source

alter-table.lisp.

Methods
Method: alter-schema ((schema schema) alter-schema-rule-list)

Apply ALTER-SCHEMA-RULE-LIST to SCHEMA.

Method: alter-schema ((catalog catalog) alter-schema-rule-list)

Apply ALTER-SCHEMA-RULE-LIST to all schema of CATALOG.

Generic Function: alter-table (object alter-table-rule-list)
Package

pgloader.catalog.

Source

alter-table.lisp.

Methods
Method: alter-table ((table table) alter-table-rule-list)

Apply ALTER-TABLE-RULE-LIST to TABLE.

Method: alter-table ((schema schema) alter-table-rule-list)

Apply ALTER-TABLE-RULE-LIST to all tables and views of SCHEMA.

Method: alter-table ((catalog catalog) alter-table-rule-list)

Apply ALTER-TABLE-RULE-LIST to all schema of CATALOG.

Generic Function: cast (object &key table &allow-other-keys)

Cast a FIELD definition from a source database into a PostgreSQL COLUMN definition.

Package

pgloader.catalog.

Source

catalog.lisp.

Methods
Method: cast ((field column) &key &allow-other-keys)

Return the PostgreSQL type definition from the given PostgreSQL column definition

Source

pgsql-cast-rules.lisp.

Method: cast ((col mysql-column) &key table)

Return the PostgreSQL type definition from given MySQL column definition.

Source

mysql-cast-rules.lisp.

Method: cast ((field mssql-column) &key &allow-other-keys)

Return the PostgreSQL type definition from given MS SQL column definition.

Source

mssql-cast-rules.lisp.

Method: cast ((col coldef) &key &allow-other-keys)

Return the PostgreSQL type definition from given SQLite column definition.

Source

sqlite-cast-rules.lisp.

Method: cast ((col ixf-column) &key &allow-other-keys)

Return the PostgreSQL type definition from given IXF column definition.

Source

ixf-cast-rules.lisp.

Method: cast ((field db3-coldef) &key table)

Return the PostgreSQL type definition given the DB3 one.

Source

db3-cast-rules.lisp.

Method: cast ((catalog catalog) &key)

Cast all fields of all tables in all schemas in CATALOG into columns.

Method: cast ((schema schema) &key)

Cast all fields of all tables in SCHEMA into columns.

Method: cast ((table table) &key)

Cast all fields in table into columns.

Generic Function: check-connection (connection)

Check that we can actually connect.

Package

pgloader.connection.

Source

connection.lisp.

Methods
Method: check-connection ((c db-connection))

Check that it is possible to connect to db-connection C.

Method: check-connection ((fd fd-connection))

Check that it is possible to connect to db-connection C.

Generic Function: cleanup (db-copy catalog &key materialize-views)

Clean-up after prepare-pgsql-database failure.

Package

pgloader.sources.

Source

api.lisp.

Methods
Method: cleanup ((pgsql copy-pgsql) (catalog catalog) &key materialize-views)

When there is a PostgreSQL error at prepare-pgsql-database step, we might need to clean-up any view created in the source PostgreSQL connection for the migration purpose.

Source

pgsql.lisp.

Method: cleanup ((mysql copy-mysql) (catalog catalog) &key materialize-views)

When there is a PostgreSQL error at prepare-pgsql-database step, we might need to clean-up any view created in the MySQL connection for the migration purpose.

Source

mysql.lisp.

Method: cleanup ((mssql copy-mssql) (catalog catalog) &key materialize-views)

When there is a PostgreSQL error at prepare-pgsql-database step, we might need to clean-up any view created in the MS SQL connection for the migration purpose.

Source

mssql.lisp.

Method: cleanup ((copy db-copy) (catalog catalog) &key materialize-views)

In case anything wrong happens at ‘prepare-pgsql-database’ step, this function will be called to clean-up the mess left behind, if any.

Source

methods.lisp.

Generic Function: clone-connection (connection)

Instanciate a new connection object with similar properties.

Package

pgloader.connection.

Source

connection.lisp.

Methods
Method: clone-connection ((c mysql-connection))
Source

mysql-connection.lisp.

Method: clone-connection ((c mssql-connection))
Source

mssql-connection.lisp.

Method: clone-connection ((slconn sqlite-connection))
Source

sqlite-connection.lisp.

Method: clone-connection ((c ixf-connection))
Source

ixf-connection.lisp.

Method: clone-connection ((c dbf-connection))
Source

db3-connection.lisp.

Method: clone-connection ((c pgsql-connection))
Source

connection.lisp.

Method: clone-connection ((c db-connection))
Method: clone-connection ((fd fd-connection))
Generic Function: clone-copy-for (md-copy path-spec)

Create a new instance for copying PATH-SPEC data.

Package

pgloader.sources.

Source

api.lisp.

Methods
Method: clone-copy-for ((copy copy-copy) path-spec)

Create a copy of FIXED for loading data from PATH-SPEC.

Source

copy.lisp.

Method: clone-copy-for ((fixed copy-fixed) path-spec)

Create a copy of FIXED for loading data from PATH-SPEC.

Source

fixed.lisp.

Method: clone-copy-for ((csv copy-csv) path-spec)

Create a copy of CSV for loading data from PATH-SPEC.

Source

csv.lisp.

Method: clone-copy-for ((copy md-copy) path-spec)

Create a copy of CSV for loading data from PATH-SPEC.

Source

md-methods.lisp.

Generic Function: close-connection (connection)

Close a connection to the data source.

Package

pgloader.connection.

Source

connection.lisp.

Methods
Method: close-connection ((myconn mysql-connection))
Source

mysql-connection.lisp.

Method: close-connection ((msconn mssql-connection))
Source

mssql-connection.lisp.

Method: close-connection ((slconn sqlite-connection))
Source

sqlite-connection.lisp.

Method: close-connection ((ixfconn ixf-connection))
Source

ixf-connection.lisp.

Method: close-connection ((dbfconn dbf-connection))
Source

db3-connection.lisp.

Method: close-connection ((md md-connection))

Reset.

Source

files-and-pathnames.lisp.

Method: close-connection ((pgconn pgsql-connection))

Close a PostgreSQL connection.

Source

connection.lisp.

Generic Reader: columns (object)
Package

pgloader.sources.

Methods
Reader Method: columns ((copy copy))

automatically generated reader method

Source

api.lisp.

Target Slot

columns.

Generic Writer: (setf columns) (object)
Package