Synchronous Updates in RIPE Database
====================================

This file contains description of the goals, design and implementation
overview. For other topics, go to:

* INSTALL (for installation instructions)
* doc/README.conf (for the format of syncupdates.conf)

---

1. Goals

* To enable communication of the update process and the user through TCP
  connections.
* To be compatible with current mail update queues and filters.
* To be scalable and controllable, thus resistant to misuse of resources.
* Provide all the features that mail updates provide, except for mail-from
  authentication.
* Provide a generic interface, allowing arbitrary clients.


2. Design

2.1 User Interaction

  Protocol The protocol is http1.1, specified in rfc2068 . This allows a
  well defined input format, as well as generic client support.
 

2.2 Input

  In the input, the following should be provided:

  * Data containing objects that are submitted. Identical to message body
    in email updates. A maximum size is defined (2MB), submissions with zero
    length or submissions exceeding the maximum data size are discarded.
  * Optional commands to specify the action, identical to commands provided
    in the Subject: field of mail updates. Can be "HELP or "NEW". The default
    is the same as the default action of dbupdate.

2.3 Output

  Upon receiving the input, the server processes the input, gives the
  appropriate output and disconnects. Following is the output algorithm:

OUTPUT
------
strlen(data) > MAXSIZE:
  error("Data exceeds maximum allowed size")
data==NULL:
  help==NULL || help==no:
    error("No input")
  help!=NULL && help!=no:
    print_help_message()
    strlen(data) < MAXSIZE:
  help!=NULL && help!=no:
    print_help_message()
  new!=NULL && new!=no:
    out=process_message(data,NEW)
    print_result(out)
  new==NULL || new==no:
    out=process_message(data,NULL)
    print_result(out)

FORMAT OF OUTPUT
----------------
type(output)==error:
  print errno." ".errstr
default:
  print "200 ".msgstr

2.3 Server Design

  Server design is split into two parts, backend and frontend. Frontend
  accepts client connections, and filters and groups them in the same way
  mail2queue does. Then it forwards the submissions to the related backend
  port. Backend listens on N ports, where N equals to number of queues in
  mail updates. This simulates different queues in mail updates. Ports are
  checked sequentially, and only one job at a time is accepted.

2.3.1 Backend (Scheduling)

listen on N ports
repeat
  foreach port n:
  accept one connection and pass to dbupdate.
  pass the result from dbupdate over the connection.

2.3.2 Frontend (Filtering and queueing)

accept connections on port P
repeat
  foreach new connection p,
    read input
    determine queue number n to be sent the connection to - 0 means
    discard/refuse.
      connect backend socket n and send data
    wait for reply
    output reply
 
3. Implementation

  Due to the nature of the protocol, a web server is needed. Typically a
  bare-bones apache server will suffice for full functionality. Future
  enhancements like ssl support may be provided via apache. The standard
  tools used for web security and performance can be applied for the insured
  quality of the service.

  Server frontend is to be implemented as a CGI application, while backend
  will be a dmon listening on N predefined ports, which accepts connections
  only from the CGI application.

  Both the backend and the frontend may have more than one instance. This
  should allow scalability of the implementation.

  There is no need for a client implementation as reference, since an
  application as simple as wget may be used for test purposes.
 
  As some of the functionality of dbupdate (using pgp signed messages,
  submitting key-cert objects, forcing new action and requesting help)
  depends on the mail header contents of the submission, these functionalities
  will not be available at the first stage. Key-cert object submissions and
  use of pgp signed messages can promptly be used after fixing dbupdate.
  However, it's not at the moment possible to communicate NEW and HOWTO
  commands with dbupdate except using Subject: field of the email, therefore
  another way should be implemented in dbupdate to be able to use these
  facilities in synchronous updates.

3.1 Monitoring Script - sup_watch

  The monitoring script, sup_watch monitors backend processes on the server.
  Alerts are sent via email if one of the backends stop running.

$Id: README,v 1.6 2002/10/18 10:39:15 can Exp $

