# Introduction

## What is RefDB?

- RefDB is a reference and note database. Use it to manage your collection of offprints of scientific publications, or basically any other collection of printed or electronic documents along with your personal notes.

- RefDB is also a bibliography tool for markup languages. The bibliography tool allows to create printable or online documents with formatted citations and bibliographies from DocBook SGML/XML, TEI XML, and LaTeX sources.

- RefDB can also serve as a data source for other applications. You can either use the SRU interface, or use the Perl and PHP libraries to integrate RefDB functionality directly into your own projects.

## Who should use RefDB?

RefDB was designed with the needs of a scientist in mind. At least one third of a scientist's work hours are spent reading and evaluating other scientist's publications. In no time you have a pile of offprints or PDF files that you would like to archive in a way that keeps the information accessible. RefDB helps you to keep track of those papers by creating a database with the essential bibliographic information, keywords for easy retrieval, your personal comments, the reprint status, a hint where you can find the offprint (or a link to the electronic version), and additional information for your personal categorization.

Extended notes help you organize your references according to topics and allow you to associate additional information with references, keywords, authors, or periodicals. This goes far beyond the one-note per reference approach of commercial tools, although this is of course still supported. Extended notes are associated to the user that created them, but they are visible to all users unless you restrict the display accordingly.

Scientists rarely work alone on a subject. More commonly they are members of a group with similar research interests. It would be a waste of time and effort if each member of the group hunts offprints individually, resulting in duplicate copies, and if each member keeps his personal reference database, making it hard to consolidate an information pool of the group. Instead of selling a complete database engine to each scientist and "allowing" the users to peek at each other's databases, RefDB uses the inverse approach: designed as a client/server application from the ground up, it allows a group to maintain a common reference database, but still allows each individual to view only those references that she added or explicitly selected. At the same time, each group member can find out in an instant which offprints are available in the whole group. However, there is no problem if you want to use RefDB as an individual user.

Eventually you'll want to publish your own data. You'll have to cite dozens of papers that you've already read. Your RefDB database will make it a breeze to retrieve those papers by author, by keyword, or by any other field or combination of fields. If you write your document using one of the supported SGML or XML DTDs or as a LaTeX document, RefDB can create the bibliography based on the citations in your document. It will also help you to format the citations and the bibliography according to the instructions of your publisher. Your paper got rejected and you want to submit it to a different journal? Fear not, RefDB can easily re-transform your document and format it for a different publisher.

Of course it is also feasible to manage your personal library of contemporary literature. Or your favourite cookie recipes. RefDB wasn't precisely designed for these purposes, but it's up to you to make the best use of the package.

## General features

- RefDB uses a three-tier architecture (see ), consisting of clients on the workstation side, and an application server as well as the SQL database server on the server side.

  RefDB three-tier architecture

  Optionally you may use an internal database engine instead of the external SQL server, resulting in a simpler two-tier setup.

- The data storage proper is done by a SQL database engine. Currently MySQL and PostgreSQL are supported as external database engines. Additionally, an internal SQL engine based on SQLite is available, which allows data storage in a single, operating-system and architecture-independent file without any administration overhead.

- By default, incoming data are converted to UTF-8 and stored as such (see ). Outgoing data can be converted to other character encodings if needed. However, RefDB can be set up in a very flexible manner to accommodate other character encoding requirements as long as they are supported by your operating system.

  RefDB character encoding support

- The server can run as a daemon in a non-privileged account if security concerns require this. Besides, users can start it as a standalone application on demand. As just about everything is configurable, a thoughtful setup will allow to run several copies of the server on the same box in parallel.

## Interfaces

- The RefDB command-line clients are no fat applications, but rather a collection of small, portable tools implemented in ANSI C to perform all necessary client-side tasks on any platform with a decent C compiler.

- The RefDB command-line clients can be run in an interactive mode or in a batch mode. The latter is useful for scripting purpose or for use as backends of graphical user interfaces.

- A PHP interface allows to perform most RefDB operations through a web interface.

- RefDB also provides clients which allow to access RefDB databases through a SRU (Search and Retrieve via URL) interface. A simple standalone client for personal use and a CGI program for shared use through a web server are available.

- Separately available packages integrate RefDB functionality into Emacs and vim, turning these editors into integrated authoring environments for DocBook and TEI documents with easy access to your reference data.

- The adepts of the Perl programming language might be delighted to know that the RefDBClient module allows Perl programs to directly communicate with a refdbd server without using the C clients. This allows the rapid development of custom programs that access RefDB databases.

## Reference management features

- Reference data are imported from text files or from stdin and exported to text files or to stdout. The data model is fairly close to the RIS specification that most Windows-based reference managers understand. RefDB natively understands both the tagged RIS format used by other reference management software and an XML format according to the risx.dtd similar to RIS.

- Input filters can be used to convert references from various sources to the RIS format. Currently RefDB ships with Medline, BibTeX, MARC, and DocBook filters. You are free to use or write any other input filter that you may need. These filters must either create an output file or write the results to stdout for further plumbing. This way, input filters can be written in almost any programming language and it should be easy to extend the list of reference information formats that RefDB can import.

  In addition, SGML- or XML-based data formats can be easily transformed to risx using DSSSL or XSLT stylesheets.

- Extended notes are supplied as XML files according to the xnote.dtd.

- The query language is fairly simple yet powerful. You can search in all fields in the database. You can use the Boolean operators AND, OR, NOT to combine search expressions. You can use brackets () to group search expressions. You can use either literal matches or regular expressions in all alphanumeric fields (i.e. most except e.g. the publication year). This gives you enormous flexibility in your search strategies. The readline library reads the user input in all interactive clients. You can recall any previous search strings with a few keystrokes and re-run them or modify them as needed.

- The query results can be displayed in a variety of formats. The standard backends create screen, HTML, XHTML, BiBTeX, DocBook (SGML and XML), TEI (XML), MODS (XML), RIS, and risx formats. All output can either be viewed on stdout or with a pager, or the output can be redirected into a file or into a pipe for further manipulation. RefDB provides a simple API to implement custom backends if you need other output formats.

  In addition, Perl programmers can use the risx output and the RefDBClient module to easily create any output format they desire.

- RefDB supports all character encodings available on your platform. While the available encodings in the database may be limited by the database engine, RefDB can convert incoming data as well as exported data with only few limitations.

- RefDB handles the L1 field of the RIS input files (denoting a local PDF or Postscript copy) in a very flexible way. You can specify a path to the file on your harddrive or on the web. The local path can be split into a variable and a static part. The variable part can be specified on the command line e.g. if you access your data remotely via a NFS-mounted share.

- RefDB knows the concept of personal reference lists. This feature is useful if a database is shared among several users in a workgroup. In this case, all users benefit from the larger stock of references available in the database. RefDB keeps track of the user who added a reference to the database. You can use a switch in the getref command to restrict your search on those references that are associated with your username. On the other hand, if you find out that even your colleagues have one or two interesting papers, you can use the pickref command to add these references to your personal reference list. The personal part of the reference information (the reprint status, the availability, and the notes) are saved for each user individually. In addition to this default list, each user can create an unlimited number of additional lists.

  In a similar fashion, the display of extended notes can be limited to the notes of the current user, or all available extended notes are used.

## Bibliography tool features

Bibliography support is currently implemented for DocBook SGML and XML documents, TEI and TEILite XML documents, and for LaTeX documents.

### SGML and XML

- RefDB scans your input document for citations and generates a bibliography which contains the cited references. You can include this bibliography into your document and process it with RefDB's tools to obtain formatted citations and bibliographies.

- RefDB creates bibliographies for DocBook (SGML, DTD-based XML, and schema-based XML), TEI (DTD-based XML and schema-based XML), and bibtex.

- The implementation of the SGML/XML citations and bibliographies is "non-destructive", i.e. switching from one bibliographic output format to a different one is limited to re-creating the bibliography and transforming your document again.

  Alternatively you may use a short notation for your citations that makes do with a minimum of markup. The short notation needs an additional (but fully reversible) preprocessing step before bibliographies and formatted output can be created.

- RefDB uses cooked bibliography output to provide full control over the formatting. This somewhat blurs the distinction between structure and formatting but it is the most efficient way to handle the task of formatting bibliographies and citations. The TEI bibliography output is even worse as it heavily abuses the all-purpose seg element as a wrapper where the elements intended for a particular purpose do not allow a cooked use.

- Alternatively you can request raw bibliography output. In this case you can either rely on the default bibliography formatting provided by the stock DocBook or TEI stylesheet, or you resort to other bibliography formatting tools.

### LaTeX

- The BibTeX output integrates seamlessly with the LaTeX/BibTeX tools, so there is only one additional command to run on your way from the LaTeX source to the finished document.

- RefDB creates a BibTeX bibliography file based on the information in a `.aux` file. The latter is created by `latex` from the LaTeX source document. This intermediate bibliography file serves as the input file for `bibtex`. So instead of keeping all your references in a flat text file, RefDB will create a short bibliography file with the references that your particular document requires.

- RefDB performs only a very limited amount of formatting for those items which are not well supported in BibTeX (e.g. with BibTeX you need two separate bibliography files if you switch from a format that uses abbreviated journal names to a format that requires the full names). All other formatting is left to the LaTeX/BibTeX system.

## Credits

Throughout the creation of this software the author borrowed code and solutions to particular problems from other software packages in a shameless way. But this is one of the purposes of free software, after all. The following is a hopefully complete list of all programs and libraries that in one way or another helped to get RefDB on its way.

- btparse

- expat

- getopt

- libdbi

- libdbi-drivers

- libslack

- MySQL

- ncftp

- PostgreSQL

- readline

- SQLite

- wget