<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook V4.4//EN"
 "http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd"
[
     <!ENTITY % local SYSTEM "local.ent">
     %local;
     <!ENTITY % entities  SYSTEM "entities.ent">
     %entities;
     <!ENTITY % idcommon  SYSTEM "common/common.ent">
     %idcommon;
]>
<refentry id="zebrasrv">
 <refentryinfo>
  <productname>zebra</productname>
  <productnumber>&version;</productnumber>
  <orgname>Index Data</orgname>
 </refentryinfo>

 <refmeta>
  <refentrytitle>zebrasrv</refentrytitle>
  <manvolnum>8</manvolnum>
  <refmiscinfo class="manual">Commands</refmiscinfo>
 </refmeta>

 <refnamediv>
  <refname>zebrasrv</refname>
  <refpurpose>Zebra Server</refpurpose>
 </refnamediv>

 <refsynopsisdiv>
  &zebrasrv-synopsis;
 </refsynopsisdiv>
 <refsect1><title>DESCRIPTION</title>
  <para>Zebra is a high-performance, general-purpose structured text indexing
   and retrieval engine. It reads structured records in a variety of input
   formats (e.g. email, &acro.xml;, &acro.marc;) and allows access to them through exact
   boolean search expressions and relevance-ranked free-text queries.
  </para>
  <para>
   <command>zebrasrv</command> is the &acro.z3950; and &acro.sru; frontend
   server for the <command>Zebra</command> search engine and indexer.
  </para>
  <para>
   On Unix you can run the <command>zebrasrv</command>
   server from the command line - and put it
   in the background. It may also operate under the inet daemon.
   On WIN32 you can run the server as a console application or
   as a WIN32 Service.
  </para>
 </refsect1>
 <refsect1>
  <title>OPTIONS</title>

  <para>
   The options for <command>zebrasrv</command> are the same
   as those for &yaz;' <command>yaz-ztest</command>.
   Option <literal>-c</literal> specifies a Zebra configuration
   file - if omitted <filename>zebra.cfg</filename> is read.
  </para>

  &zebrasrv-options;
 </refsect1>

 <refsect1 id="protocol-support">
  <title>&acro.z3950; Protocol Support and Behavior</title>

  <refsect2 id="zebrasrv-initialization">
   <title>&acro.z3950; Initialization</title>

   <para>
    During initialization, the server will negotiate to version 3 of the
    &acro.z3950; protocol, and the option bits for Search, Present, Scan,
    NamedResultSets, and concurrentOperations will be set, if requested by
    the client. The maximum PDU size is negotiated down to a maximum of
    1 MB by default.
   </para>

  </refsect2>

  <refsect2 id="zebrasrv-search">
   <title>&acro.z3950; Search</title>

   <para>
    The supported query type are 1 and 101. All operators are currently
    supported with the restriction that only proximity units of type "word"
    are supported for the proximity operator.
    Queries can be arbitrarily complex.
    Named result sets are supported, and result sets can be used as operands
    without limitations.
    Searches may span multiple databases.
   </para>

   <para>
    The server has full support for piggy-backed retrieval (see
    also the following section).
   </para>

  </refsect2>

  <refsect2 id="zebrasrv-present">
   <title>&acro.z3950; Present</title>
   <para>
    The present facility is supported in a standard fashion. The requested
    record syntax is matched against the ones supported by the profile of
    each record retrieved. If no record syntax is given, &acro.sutrs; is the
    default. The requested element set name, again, is matched against any
    provided by the relevant record profiles.
   </para>
  </refsect2>
  <refsect2 id="zebrasrv-scan">
   <title>&acro.z3950; Scan</title>
   <para>
    The attribute combinations provided with the termListAndStartPoint are
    processed in the same way as operands in a query (see above).
    Currently, only the term and the globalOccurrences are returned with
    the termInfo structure.
   </para>
  </refsect2>
  <refsect2 id="zebrasrv-sort">
   <title>&acro.z3950; Sort</title>

   <para>
    &acro.z3950; specifies three different types of sort criteria.
    Of these Zebra supports the attribute specification type in which
    case the use attribute specifies the "Sort register".
    Sort registers are created for those fields that are of type "sort" in
    the default.idx file.
    The corresponding character mapping file in default.idx specifies the
    ordinal of each character used in the actual sort.
   </para>

   <para>
    &acro.z3950; allows the client to specify sorting on one or more input
    result sets and one output result set.
    Zebra supports sorting on one result set only which may or may not
    be the same as the output result set.
   </para>
  </refsect2>
  <refsect2 id="zebrasrv-close">
   <title>&acro.z3950; Close</title>
   <para>
    If a Close PDU is received, the server will respond with a Close PDU
    with reason=FINISHED, no matter which protocol version was negotiated
    during initialization. If the protocol version is 3 or more, the
    server will generate a Close PDU under certain circumstances,
    including a session timeout (60 minutes by default), and certain kinds of
    protocol errors. Once a Close PDU has been sent, the protocol
    association is considered broken, and the transport connection will be
    closed immediately upon receipt of further data, or following a short
    timeout.
   </para>
  </refsect2>

  <refsect2 id="zebrasrv-explain">
   <title>&acro.z3950; Explain</title>
   <para>
    Zebra maintains a "classic"
    <ulink url="&url.z39.50.explain;">&acro.z3950; Explain</ulink> database
    on the side.
    This database is called <literal>IR-Explain-1</literal> and can be
    searched using the attribute set <literal>exp-1</literal>.
   </para>
   <para>
    The records in the explain database are of type
    <literal>grs.sgml</literal>.
    The root element for the Explain grs.sgml records is
    <literal>explain</literal>, thus
    <filename>explain.abs</filename> is used for indexing.
   </para>
   <note>
    <para>
     Zebra <emphasis>must</emphasis> be able to locate
     <filename>explain.abs</filename> in order to index the Explain
     records properly. Zebra will work without it but the information
     will not be searchable.
    </para>
   </note>
  </refsect2>
 </refsect1>
 <refsect1 id="zebrasrv-sru">
  <title>The &acro.sru; Server</title>
  <para>
   In addition to &acro.z3950;, Zebra supports the more recent and
   web-friendly IR protocol <ulink url="&url.sru;">&acro.sru;</ulink>.
   &acro.sru; can be carried over &acro.soap; or a &acro.rest;-like protocol
   that uses HTTP &acro.get; or &acro.post; to request search responses.  The request
   itself is made of parameters such as
   <literal>query</literal>,
   <literal>startRecord</literal>,
   <literal>maximumRecords</literal>
   and
   <literal>recordSchema</literal>;
   the response is an &acro.xml; document containing hit-count, result-set
   records, diagnostics, etc.  &acro.sru; can be thought of as a re-casting
   of &acro.z3950; semantics in web-friendly terms; or as a standardisation
   of the ad-hoc query parameters used by search engines such as Google
   and AltaVista; or as a superset of A9's OpenSearch (which it
   predates).
  </para>
  <para>
   Zebra supports &acro.z3950;, &acro.sru; &acro.get;, SRU &acro.post;, SRU &acro.soap; (&acro.srw;)
   - on the same port, recognising what protocol is used by each incoming
   requests and handling them accordingly.  This is a achieved through
   the use of Deep Magic; civilians are warned not to stand too close.
  </para>
  <refsect2 id="zebrasrv-sru-run">
   <title>Running zebrasrv as an &acro.sru; Server</title>
   <para>
    Because Zebra supports all protocols on one port, it would
    seem to follow that the &acro.sru; server is run in the same way as
    the &acro.z3950; server, as described above.  This is true, but only in
    an uninterestingly vacuous way: a Zebra server run in this manner
    will indeed recognise and accept &acro.sru; requests; but since it
    doesn't know how to handle the &acro.cql; queries that these protocols
    use, all it can do is send failure responses.
   </para>
   <note>
    <para>
     It is possible to cheat, by having &acro.sru; search Zebra with
     a &acro.pqf; query instead of &acro.cql;, using the
     <literal>x-pquery</literal>
     parameter instead of
     <literal>query</literal>.
     This is a
     <emphasis role="strong">non-standard extension</emphasis>
     of &acro.cql;, and a
     <emphasis role="strong">very naughty</emphasis>
     thing to do, but it does give you a way to see Zebra serving &acro.sru;
     ``right out of the box''.  If you start your favourite Zebra
     server in the usual way, on port 9999, then you can send your web
     browser to:
    </para>
    <screen>
     http://localhost:9999/Default?version=1.1
     &amp;operation=searchRetrieve
     &amp;x-pquery=mineral
     &amp;startRecord=1
     &amp;maximumRecords=1
    </screen>
    <para>
     This will display the &acro.xml;-formatted &acro.sru; response that includes the
     first record in the result-set found by the query
     <literal>mineral</literal>.  (For clarity, the &acro.sru; URL is shown
     here broken across lines, but the lines should be joined together
     to make single-line URL for the browser to submit.)
    </para>
   </note>
   <para>
    In order to turn on Zebra's support for &acro.cql; queries, it's necessary
    to have the &yaz; generic front-end (which Zebra uses) translate them
    into the &acro.z3950; Type-1 query format that is used internally.  And
    to do this, the generic front-end's own configuration file must be
    used.  See <xref linkend="gfs-config"/>;
    the salient point for &acro.sru; support is that
    <command>zebrasrv</command>
    must be started with the
    <literal>-f&nbsp;frontendConfigFile</literal>
    option rather than the
    <literal>-c&nbsp;zebraConfigFile</literal>
    option,
    and that the front-end configuration file must include both a
    reference to the Zebra configuration file and the &acro.cql;-to-&acro.pqf;
    translator configuration file.
   </para>
   <para>
    A minimal front-end configuration file that does this would read as
    follows:
   </para>
   <screen>
    <![CDATA[
    <yazgfs>
    <server>
    <config>zebra.cfg</config>
    <cql2rpn>../../tab/pqf.properties</cql2rpn>
   </server>
   </yazgfs>
    ]]></screen>
   <para>
    The
    <literal>&lt;config&gt;</literal>
    element contains the name of the Zebra configuration file that was
    previously specified by the
    <literal>-c</literal>
    command-line argument, and the
    <literal>&lt;cql2rpn&gt;</literal>
    element contains the name of the &acro.cql; properties file specifying how
    various &acro.cql; indexes, relations, etc. are translated into Type-1
    queries.
   </para>
   <para>
    A zebra server running with such a configuration can then be
    queried using proper, conformant &acro.sru; URLs with &acro.cql; queries:
   </para>
   <screen>
    http://localhost:9999/Default?version=1.1
    &amp;operation=searchRetrieve
    &amp;query=title=utah and description=epicent*
    &amp;startRecord=1
    &amp;maximumRecords=1
   </screen>
  </refsect2>
 </refsect1>
 <refsect1 id="zebrasrv-sru-support">
  <title>&acro.sru; Protocol Support and Behavior</title>
  <para>
   Zebra running as an &acro.sru; server supports SRU version 1.1, including
   &acro.cql; version 1.1.  In particular, it provides support for the
   following elements of the protocol.
  </para>

  <refsect2 id="zebrasrvr-search-and-retrieval">
   <title>&acro.sru; Search and Retrieval</title>
   <para>
    Zebra supports the searchRetrieve operation.
   </para>
   <para>
    One of the great strengths of &acro.sru; is that it mandates a standard
    query language, &acro.cql;, and that all conforming implementations can
    therefore be trusted to correctly interpret the same queries.  It
    is with some shame, then, that we admit that Zebra also supports
    an additional query language, our own Prefix Query Format
    (<ulink url="&url.yaz.pqf;">&acro.pqf;</ulink>).
    A &acro.pqf; query is submitted by using the extension parameter
    <literal>x-pquery</literal>,
    in which case the
    <literal>query</literal>
    parameter must be omitted, which makes the request not valid &acro.sru;.
    Please feel free to use this facility within your own
    applications; but be aware that it is not only non-standard &acro.sru;
    but not even syntactically valid, since it omits the mandatory
    <literal>query</literal> parameter.
   </para>
  </refsect2>

  <refsect2 id="zebrasrv-sru-scan">
   <title>&acro.sru; Scan</title>
   <para>
    Zebra supports scan operation.
    Scanning using &acro.cql; syntax is the default, where the
    standard <literal>scanClause</literal> parameter is used.
   </para>
   <para>
    In addition, a
    mutant form of &acro.sru; scan is supported, using
    the non-standard <literal>x-pScanClause</literal> parameter in
    place of the standard <literal>scanClause</literal> to scan on a
    &acro.pqf; query clause.
   </para>
  </refsect2>

  <refsect2 id="zebrasrv-sru-explain">
   <title>&acro.sru; Explain</title>
   <para>
    Zebra supports explain.
   </para>
   <para>
    The ZeeRex record explaining a database may be requested either
    with a fully fledged &acro.sru; request (with
    <literal>operation</literal>=<literal>explain</literal>
    and version-number specified)
    or with a simple HTTP &acro.get; at the server's basename.
    The ZeeRex record returned in response is the one embedded
    in the &yaz; Frontend Server configuration file that is described in the
    <xref linkend="gfs-config"/>.
   </para>
   <para>
    Unfortunately, the data found in the
    &acro.cql;-to-&acro.pqf; text file must be added by hand-craft into the explain
    section of the &yaz; Frontend Server configuration file to be able
    to provide a suitable explain record.
    Too bad, but this is all extreme
    new alpha stuff, and a lot of work has yet to be done ..
   </para>
   <para>
    There is no linkage whatsoever between the &acro.z3950; explain model
    and the &acro.sru; explain response (well, at least not implemented
    in Zebra, that is ..).  Zebra does not provide a means using
    &acro.z3950; to obtain the ZeeRex record.
   </para>
  </refsect2>

  <refsect2 id="zebrasrv-non-sru-ops">
   <title>Other &acro.sru; operations</title>
   <para>
    In the &acro.z3950; protocol, Initialization, Present, Sort and Close
    are separate operations.  In &acro.sru;, however, these operations do not
    exist.
   </para>
   <itemizedlist>
    <listitem>
     <para>
      &acro.sru; has no explicit initialization handshake phase, but
      commences immediately with searching, scanning and explain
      operations.
     </para>
    </listitem>
    <listitem>
     <para>
      Neither does &acro.sru; have a close operation, since the protocol is
      stateless and each request is self-contained.  (It is true that
      multiple &acro.sru; request/response pairs may be implemented as
      multiple HTTP request/response pairs over a single persistent
      TCP/IP connection; but the closure of that connection is not a
      protocol-level operation.)
     </para>
    </listitem>
    <listitem>
     <para>
      Retrieval in &acro.sru; is part of the
      <literal>searchRetrieve</literal> operation, in which a search
      is submitted and the response includes a subset of the records
      in the result set.  There is no direct analogue of &acro.z3950;'s
      Present operation which requests records from an established
      result set.  In &acro.sru;, this is achieved by sending a subsequent
      <literal>searchRetrieve</literal> request with the query
      <literal>cql.resultSetId=</literal><emphasis>id</emphasis> where
      <emphasis>id</emphasis> is the identifier of the previously
      generated result-set.
     </para>
    </listitem>
    <listitem>
     <para>
      Sorting in &acro.cql; is done within the
      <literal>searchRetrieve</literal> operation - in v1.1, by an
      explicit <literal>sort</literal> parameter, but the forthcoming
      v1.2 or v2.0 will most likely use an extension of the query
      language, <ulink url="&url.cql.sorting;">&acro.cql; sorting</ulink>.
     </para>
    </listitem>
   </itemizedlist>
   <para>
    It can be seen, then, that while Zebra operating as an &acro.sru; server
    does not provide the same set of operations as when operating as a
    &acro.z3950; server, it does provide equivalent functionality.
   </para>
  </refsect2>
 </refsect1>

 <refsect1 id="zebrasrv-sru-examples">
  <title>&acro.sru; Examples</title>
  <para>
   Surf into <literal>http://localhost:9999</literal>
   to get an explain response, or use
   <screen><![CDATA[
    http://localhost:9999/?version=1.1&operation=explain
    ]]></screen>
  </para>
  <para>
   See number of hits for a query
   <screen><![CDATA[
    http://localhost:9999/?version=1.1&operation=searchRetrieve
    &query=text=(plant%20and%20soil)
    ]]></screen>
  </para>
  <para>
   Fetch record 5-7 in Dublin Core format
   <screen><![CDATA[
    http://localhost:9999/?version=1.1&operation=searchRetrieve
    &query=text=(plant%20and%20soil)
    &startRecord=5&maximumRecords=2&recordSchema=dc
    ]]></screen>
  </para>
  <para>
   Even search using &acro.pqf; queries using the <emphasis>extended naughty
    parameter</emphasis> <literal>x-pquery</literal>
   <screen><![CDATA[
    http://localhost:9999/?version=1.1&operation=searchRetrieve
    &x-pquery=@attr%201=text%20@and%20plant%20soil
    ]]></screen>
  </para>
  <para>
   Or scan indexes using the <emphasis>extended extremely naughty
    parameter</emphasis> <literal>x-pScanClause</literal>
   <screen><![CDATA[
    http://localhost:9999/?version=1.1&operation=scan
    &x-pScanClause=@attr%201=text%20something
    ]]></screen>
   <emphasis>Don't do this in production code!</emphasis>
   But it's a great fast debugging aid.
  </para>

 </refsect1>

 <refsect1 id="gfs-config"><title>&yaz; server virtual hosts</title>
  &zebrasrv-virtual;
 </refsect1>

 <refsect1><title>SEE ALSO</title>
  <para>
   <citerefentry>
    <refentrytitle>zebraidx</refentrytitle>
    <manvolnum>1</manvolnum>
   </citerefentry>
  </para>
 </refsect1>
</refentry>

 <!-- Keep this comment at the end of the file
 Local variables:
 mode: sgml
 sgml-omittag:t
 sgml-shorttag:t
 sgml-minimize-attributes:nil
 sgml-always-quote-attributes:t
 sgml-indent-step:1
 sgml-indent-data:t
 sgml-parent-document: "idzebra.xml"
 sgml-local-catalogs: nil
 sgml-namecase-general:t
 End:
 -->
