
RIPE database release 2.00


NOTES for second time users:

- Please use the new sample config file to build your config file since part
  of the syntax changed considerably!
  
- A reindex is necessary (before getting updates in) due to changes in
  the index formats.

- Check out the RELEASE-NOTES file for new features available

- A document on how to use the database is now available in doc/howto.
  The most recent version of the help documentation is always available 
  from:

  <http://www.ripe.net/docs/ripe-157.html>
  <ftp://ftp.ripe.net/ripe/docs/ripe-157.ps>
  <ftp://ftp.ripe.net/ripe/docs/ripe-157.txt>
  
  Help on using <auto-dbm@ripe.net> is available by sending
  an e-mail to <auto-dbm@ripe.net> with HELP in the Subject
  line.
  
  Help on using whois is available by doing
  
  'whois -h whois.ripe.net HELP'

- whoisd needs to be started as 'whoisd -D' for normal daemon mode



GETTING STARTED

Before you READ this part, please make sure you have read INSTALL and have
installed the software properly, since all examples in this README assume
that the software is installed.

If you just run a secondary copy of the RIPE database, please read the
file SECONDARIES in this directory. It will explain what to do get get a
secondary copy of the database running.

The file 'doc/howto' contains valuable information on how to use the
database, tips & tricks section and pointers to documentation on the RIPE
database.
 
Make sure you walk through the sample configuration file
'etc/ripedb.config', and change everything you wish or need to change.
All options are documented in the sample configuration file that is
provided in the distribution.

If you just run a RIPE like database, there is not much to change, except
if you wish to change some language. The definition of objects in the
database is also in that file, feel free to have a look to see if you
understand.

!!!! MAKE SURE YOU CHANGE ALL PATHNAMES IN THE SAMPLE CONFIG FILE 
     AND THAT THE PATHS EXIST !!!

Description of (part of) the available programs:


'newdb':

newdb creation an empty database file. You should normally not need to
use this program since there is no need to generate the plain text
database files yourself anymore, they will be auto-generated when they
don't exist yet.


'showdb':

If you wish to check the index you can use the little script 'showdb' in
the bin directory. Just call it as:

bin/showdb <databasefile>

It will output for you all keys, followed by the number of references to this
key, and the list of all file offsets for this key in the main database
file. This program was merely written for debugging.


'whoisd':

As soon as you have started the indexing you can fire of the whoisd. This
whoisd will work while the indexer is still working, it will simply find
all that has been indexed so far. The whoisd can better not be run from
inetd, since it takes too much time to read the config every time it is
started. It detaches itself from the terminal and acts as a daemon
itself. It will wait for requests on the standard whois port 43 (or other
port when -p is used), and fork of children to handle requests.

(Make sure you run this as root, since it will try and listen to port 43,
a privileged port. Also make sure nothing is currently running that port,
and inetd is not managing that port. whoisd children will run with the
UID as specified in the config file). whoisd uses the REUSEADDR socket
option so it could happen that you have multiple whoisds listening to the
same port!!

'bin/whoisd -D'

It will read the config, and start listening for requests. Whenever you
change the configuration file, make sure to kill and restart the whoisd.

The whoisd currently understands all whois arguments (Start the whois
program without any arguments).

In addition it knows about the following arguments:

-g              used by the syncdb program
-k              keep connection open
                mostly used by tools like prtracroute
-V              identifies the query program to the server
-U              used by the networkupdate program
-V              debug mode

If the whoisd is called with command line arguments, it will not behave
as a daemon, but will simply act as a whois client, ie looking up the
command line arguments on the local databases, and will exit. It will
also then understand all of the above options. whoisd can be used from
inetd when called without any arguments (untested).

'whois':

Most whois clients cannot yet handle the previously mentioned flags, and
therefore you should quote these arguments, so they are passed directly
to the whois server:

whois -h whoishost.some.where "-r 192.87.45"
whois -h whoishost.some.where "-s RIPE terpstra"

A perl RIPE whois client that supports all these options is supplied in
this package. After installation it should be in 'bin/whois'. A version
in C can be retrieved from:

ftp://ftp.ripe.net/tools/ripe-whois-tools-VERSION.tar.gz


'dbupdate':

I urge you to first test this extensively on a test database. If you wish
to have whoisd (use 'whoisd -D -V' for very verbose output) work on that
database as well, make sure you change the source for this database, and
add it to the config file. You need to add this test database to your
config anyway because that is where the update program will look for the
files it should update.

CAREFUL: do NOT update databases across NFS mounted filesystems. The
         internal locking of the database does not properly work across
         NFS filesystems. Therefore, it is not recommended to use the
         dbupdate program for updating your databases directly. You can
         use the 'networkupdate' program instead (you need to add the
         acceptable hosts to your config file)

Then try:

bin/dbupdate -v <the_temporary_file

This will try and put the objects you have in your temporary file in the
database. You can have a look afterwards in the database file to see if they
are actually in there, and use showdb to check whether the indexes were
properly made. If you have your whoisd working on this test database as well,
it should give immediate response for the new objects.

Now, edit your temporary file so that one object will get an extra field
"delete: unused objects".

bin/dbupdate -v <the_temporary_file

again. It should now tell you it deleted the object you added the delete flag
to, and if you had more objects in your temp file, it should complain that
updating these objects would result in a NOOP.
Now have a look at the test.db file once more. You should find that the
object you deleted starts with "*XX:" and all the others are still in there.
A showdb on that file will also tell you that the keys for the deleted
object are gone.

This is the basic way the update mechanism works.

Another option for dbupdate is the -M flag (this is the most common mode
for 'dbupdate'!), which causes dbupdate to think that the_temporary_file
is a mail message with all the necessary headers, and in stead of just
updating the objects and generating the output on standard output, it
will send an acknowledgement mail back to the sender of the mail fed into
it (Make sure that you have proper acknowledgement messages defined in
you config file, else people that get an E-mail by accident might get
confused, see also TESTMODE option in the config file). This is easily
tested by sending a mail to oneself with database objects in it, then
once you received this message save it to some temporary file and then
call:

bin/dbupdate -v -M <temporary_file

You should then get a ack mail sent back to yourself. You can then easily
integrate this with your mailer. For instance in the sendmail aliases file
you could do something like:

my-dbm:		"|/wherever/dbase/bin/dbupdate -v -M"

and that should process the incoming mail, and send back ack mails.  There is
a few things to watch out for in this scenario:

- make sure that dbupdate can WRITE to the various databases you wish to
  manage. If called from sendmail, the default user will be deamon, so
  you may have to set dbupdate with an appropriate set uid or gid bit to
  be able to write to the database. If you have a special user
  (recommended) just for the database software, you can use this users
  .forward file (on UNIX systems) to pipe mail message into dbupdate.
  This avoids the use of suid of guid bits.

- perl has some build in checks for suid perl scripts. These are all
  taken care of, so dbupdate should not complain about not being able to
  do things because it is running suid or sgid. Have a look in the code
  for dbupdate to see what silly things one has to do to get around these
  checks ...

Now however, you have a clobbered database. This is no problem, although you
may want to clean that database every once in a while (say once a week or so).
To do so try:

bin/indexdb -c data/test.db

And have another look at test.db if it is finished. You should no longer have
all the deleted objects in there.

Once you understand all that has happened above, you got the basic hang of
the update procedure. Now you can play around a bit with it to see what it
can and cannot do, and what it will and will not accept.


SYNTAX CHECKING

The syntax checking consists of 3 parts:

- object definition check
- values syntax check
- update check

The object definition check will check whether all the attributes for this
object are recognized, whether all mandatory attributes are present and that
stuff. If the object passes this stage, it will go into the syntax check.
The syntax checking is done by src/syntax.pl. If you wish to make your own
additions, have a look in that file to see how it works. The general way is
that you have to add an if statement with your attribute in it:

if ($key eq "xx") {
}

where "xx" is your attribute. You have 2 variables and the object itself to
play with at that point:

$key = the short form attribute
$value = the value of that attribute
$object{$key} = the value inside the object itself (if you modify this, you
        really modify the object - careful with this)

You should then do your syntax checking and return nothing is the
syntax is fine, or:

return $O_WARNING, "a descriptive warning message here";

in case you find this a warning, or:

return $O_ERROR, "a descriptive error message here";

in case you have spotted a real error. Warnings are usually used in case you
have changed something in that attribute value.

There are some basic routines you can call for checking in src/misc.pl
see the examples for "ac" (administrative contact) for instance. I did
not say it was easy, we may change this to be more friendly at a later
stage.

If you need some help here send mail to <ripe-dbm@ripe.net>
---------
