# CL-postgres reference manual

The CL-postgres module implements a rather low-level interface for communicating with a PostgreSQL database server. It is part of the Postmodern library, but can be used separately.

## Contents

1. Connecting
2. Querying
3. Reading values
4. Row readers
5. Bulk Copying
6. Conditions
7. Symbol-index

## Connecting

class database-connection

Objects of this type represent database connections.

function open-database (database user password host &optional (port 5432) (use-ssl :no))
→ database-connection

Create and open a connection for the specified server, database, and user. `use-ssl` may be `:no`, `:yes`, or `:try`, where `:try` means 'if the server supports it'. When it is anything but `:no`, you must have the CL+SSL package loaded to initiate the connection.

On SBCL and Clozure CL, the value `:unix` may be passed for `host`, in order to connect using a Unix domain socket instead of a TCP socket.

function close-database (database-connection)

Close a database connection. It is advisable to call this on connections when you are done with them. Otherwise the open socket will stick around until it is garbage collected, and no one will tell the database server that we are done with it.

function reopen-database (database-connection)

Re-establish a database connection for a previously closed connection object. (Calling this on a connection that is still open is harmless.)

function database-open-p (database-connection)
→ boolean

Test whether a database connection is still open.

method connection-meta (database-connection)
→ hash-table

This method provides access to a hash table that is associated with the current database connection. When the connection is closed and re-opened this hash table is reset. The most obvious use for this is for storing information about the prepared statements that have been parsed for this connection.

method connection-parameters (database-connection)
→ hash-table

This method returns a mapping (string to string) containing all the configuration parameters for the connection.

variable *unix-socket-dir*

On SBCL, when using the `:unix` keyword as host argument when creating a connection, this variable determines the directory in which CL-Postgres will look for the socket file.

variable *ssl-certificate-file*
variable *ssl-key-file*

When using SSL (see `open-database`), these can be used to provide client key and certificate files. They can be either `NIL`, for no file, or a pathname.

function wait-for-notification (database-connection)

This function blocks until a notification is received on the connection. The PostgreSQL `LISTEN` command must be used to enable listening for notifications.

## Querying

function exec-query (database-connection query &optional (row-reader '`ignore-row-reader`))
→ result

Sends the given query to the given connection, and interprets the results (if there are any) with the given row-reader. If the database returns information about the amount of rows affected, this is returned as a second value.

function prepare-query (database-connection name query)

Parse and plan the given query, and store it under the given name. Note that prepared statements are per-connection, so they can only be executed through the same connection that prepared them.

function exec-prepared (database-connection name parameters &optional (row-reader '`ignore-row-reader`))
→ result

Execute the prepared statement by the given name. Parameters should be given as a list. Each value in this list should be of a type that `to-sql-string` has been specialised on. (Byte arrays will be passed in their binary form, without being put through `to-sql-string`.) The result of the executing the statement, if any, is interpreted by the given row reader, and returned. Again, the number or affected rows is optionally returned as a second value.

function unprepare-query (database-connection name)

Close the prepared statement by the given name. This will free resources and allow the name to be associated with a new prepared query.

method to-sql-string (value)
→ (values string needs-escaping)

Convert a Lisp value to its textual unescaped SQL representation. Returns a second value indicating whether this value should be escaped if it is to be put directly into a query.

You can define `to-sql-string` methods for your own datatypes if you want to be able to pass them to `exec-prepared`. When a non-`NIL` second value is returned, this may be `T` to indicate that the first value should simply be escaped as a string, or a second string providing a type prefix for the value. (This is used by S-SQL.)

variable *silently-truncate-rationals*

When a rational number is passed into a query (as per `to-sql-string`), but it can not be expressed within 38 decimal digits (for example `1/3`), it will be truncated, and lose some precision. Set this variable to `nil` to suppress that behaviour and raise an error instead.

variable *query-log*

When debugging, it can be helpful to inspect the queries that are being sent to the database. Set this variable to an output stream value (`*standard-output*`, for example) to have CL-postgres log every query it makes.

variable *query-callback*

When profiling or debugging, the `*query-log*` may not give enough information, or reparsing its output may not be feasible. This variable may be set to a designator of function taking two arguments. This function will be then called after every query, and receive query string and internal time units (as in `(CL:GET-INTERNAL-REAL-TIME)`) spent in query as its arguments.

Default value of this variable is `'LOG-QUERY`, which takes care of `*QUERY-LOG*` processing. If you provide custom query callback and wish to keep `*QUERY-LOG*` functionality, you will have to call `LOG-QUERY` from your callback function

function log-query (query internal-time)

This function is default value of `*QUERY-CALLBACK*` and logs queries to `*QUERY-LOG*` if it is not `NIL`.

## Reading values

CL-postgres knows how to convert commonly used PostgreSQL data types to Lisp values. This table shows the mapping:

PostgreSQL | Lisp
--- | ---
smallint | integer
integer | integer
bigint | integer
numeric | ratio
real | float
double precision | double-float
boolean | boolean
varchar | string
text | string
bytea | (vector (unsigned-byte 8))

The mapping from PostgreSQL types (identified by OID numbers) to the functions that interpret them is kept in so-called SQL readtables. All types for which no reader is defined will be returned as string values containing their PostgreSQL representation.

variable *sql-readtable*

This variable is used to choose the current readtable. For simple use, you will not have to touch this, but it is possible that code within a Lisp image requires different readers in different situations, in which case you can create separate read tables.

function copy-sql-readtable (table)
→ readtable

Copies a given readtable.

function default-sql-readtable ()
→ readtable

Returns the default readtable, containing only the readers defined by CL-postgres itself.

function set-sql-reader (oid function &key table binary-p)

Define a new reader for a given type. `table` defaults to `*sql-readtable*`. The reader function should take a single argument, a string, and transform that into some kind of equivalent Lisp value. When `binary-p` is true, the reader function is supposed to directly read the binary representation of the value. In most cases this is not recommended, but if you want to use it: provide a function that takes a binary input stream and an integer (the size of the value, in bytes), and reads the value from that stream. Note that reading less or more bytes than the given size will horribly break your connection.

function set-sql-datetime-readers (&key date timestamp timestamp-with-timezone time interval table)

Since there is no widely recognised standard way of representing dates and times in Common Lisp, and reading these from string representation is clunky and slow, this function provides a way to easily plug in binary readers for the `date`, `time`, `timestamp`, and `interval` types. It should be given functions with the following signatures:

`:date (days)`
  Where `days` is the amount of days since January 1st, 2000.
`:timestamp (useconds)`
  Timestamps have a microsecond resolution. Again, the zero point is the start of the year 2000, UTC.
`:timestamp-with-timezone`
  Like `:timestamp`, but for values of the 'timestamp with time zone' type (which PostgreSQL internally stores exactly the same as regular timestamps).
`:time (useconds)`
  Refers to a time of day, counting from midnight.
`:interval (months days useconds)`
  An interval is represented as several separate components. The reason that days and microseconds are separated is that you might want to take leap seconds into account.

## Row readers

Row readers are a way to read and group the results of queries. Roughly, they are functions that perform the iteration over the rows and cells in the result, and do something with the returned values.

macro row-reader ((fields) &body body)
→ function

Creates a row-reader, using the given name for the variable. Inside the body this variable refers to a vector of field descriptions. On top of that, two local functions are bound, `next-row` and `next-field`. The first will start reading the next row in the result, and returns a boolean indicating whether there is another row. The second will read and return one field, and should be passed the corresponding field description from the fields argument as a parameter.

A row reader should take care to iterate over all the rows in a result, and within each row iterate over all the fields. This means it should contain an outer loop that calls `next-row`, and every time `next-row` returns `T` it should iterate over the fields vector and call `next-field` for every field.

The definition of `list-row-reader` should give you an idea what a row reader looks like:

    (row-reader (fields)
      (loop :while (next-row)
            :collect (loop :for field :across fields
                           :collect (next-field field))))

Obviously, row readers should not do things with the database connection like, say, close it or start a new query, since it still reading out the results from the current query.

macro def-row-reader (name (fields) &body body)

The `defun`-like variant of `row-reader`: creates a row reader and gives it a top-level function name.

method field-name (field)
→ string

This can be used to get information about the fields read by a row reader. Given a field description, it returns the name the database associated with this column.

method field-type (field)
→ oid

This extracts the PostgreSQL OID associated with this column. You can, if you really want to, query the pg_types table to find out more about the types denoted by OIDs.

function list-row-reader (socket fields)
→ list

A row reader that builds a list of lists from the query results.

function alist-row-reader (socket fields)
→ alist

A row reader that returns a list of alists, which associate column names with values.

function ignore-row-reader (socket fields)

A row reader that completely ignores the result of a query.

## Bulk Copying

When loading large amounts of data into PostgreSQL, it can be done significantly faster using the bulk copying feature. The drawback to this approach is that you don't find out about data integrity errors until the entire batch is completed but sometimes the speed is worth it

function open-db-writer (db table &optional columns)

Opens a table stream into which rows can be written one at a time using `db-write-row`. `db` is either a connection object or a list of arguments that could be passed to `open-database`. `table` is the name of an existing table into which this writer will write rows. If you don't have data for all columns, use `columns` to indicate those that you do.

function close-db-writer (writer &key abort)

Closes a bulk writer opened by `open-db-writer`. Will close the associated database connection when it was created for this copier, or `abort` is true.

function db-write-row (writer row-data)

Writes `row-data` into the table and columns referenced by the writer. `row-data` is a list of Lisp objects, one for each column included when opening the writer. Arrays (the elements of which must all be the same type) will be serialized into their PostgreSQL representation before being written into the DB.

## Conditions

Opening or querying a database may raise errors. CL-postgres will wrap the errors that the server returns in a lisp condition, and raise conditions of the same type when it detects some problem itself. Socket errors are let through as they are.

condition database-error

The type of database-related conditions. For errors that you may want to catch by type, the `cl-postgres-error` package defines a bucket of subtypes used for specific errors. See the `cl-postgres/package.lisp` file for a list.

method database-error-message (database-error)
→ string

A short message associated with this error.

method database-error-detail (database-error)
→ string

A longer description of the problem, or `NIL` if none is available.

method database-error-code (database-error)
→ string

The error code PostgreSQL associated with this error, if any. See the PostgreSQL manual for their meaning.

method database-error-query (database-error)
→ string

The query that led to this error, or `NIL` if no query was involved.

method database-error-cause (database-error)
→ condition

The condition that caused this error, or `NIL` when it was not caused by another condition.

function database-error-constraint-name (database-error)
→ string

For integrity-violation errors, returns the name of the constraint that was violated (or `nil` if no constraint was found.)

condition database-connection-error

Subtype of `database-error`. An error of this type (or one of its subclasses) is signaled when a query is attempted with a connection object that is no longer connected, or a database connection becomes invalid during a query. Always provides a `:reconnect` restart, which will cause the library to make an attempt to restore the connection and re-try the query.

The following shows an example use of this feature, a way to ensure that the first connection error causes a reconnect attempt, while others pass through as normal. A variation on this theme could continue trying to reconnect, with successively longer pauses.

    (defun call-with-single-reconnect (fun)
      (let ((reconnected nil))
        (handler-bind
            ((database-connection-error
              (lambda (err)
                (when (not reconnected)
                  (setf reconnected t)
                  (invoke-restart :reconnect)))))
          (funcall fun))))

condition postgresql-notification

The condition that is signalled when a notification message is received from the PostgreSQL server. This is a `WARNING` condition which is caught by the `WAIT-FOR-NOTIFICATION` function that implements synchronous waiting for notifications.

method postgresql-notification-channel (postgresql-notification)
→ string

The channel string of this notification.

method postgresql-notification-payload (postgresql-notification)
→ string

The payload of this notification.

method postgresql-notification-pid (postgresql-notification)
→ integer

The process ID of the process that sent the notification.

## Symbol-index

- alist-row-reader
- close-database
- close-db-writer
- connection-meta
- connection-parameters
- copy-sql-readtable
- database-connection
- database-connection-error
- database-error
- database-error-cause
- database-error-code
- database-error-constraint-name
- database-error-detail
- database-error-message
- database-error-query
- database-open-p
- def-row-reader
- default-sql-readtable
- exec-prepared
- exec-query
- field-name
- field-type
- ignore-row-reader
- list-row-reader
- log-query
- next-field
- next-row
- open-database
- open-db-writer
- postgresql-notification
- postgresql-notification-channel
- postgresql-notification-payload
- postgresql-notification-pid
- prepare-query
- *query-callback*
- *query-log*
- reopen-database
- row-reader
- set-sql-datetime-readers
- set-sql-reader
- write-db-row
- *silently-truncate-rationals*
- *sql-readtable*
- *ssl-certificate-file*
- *ssl-key-file*
- to-sql-string
- *unix-socket-dir*
- wait-for-notification