**********************************************************************


NOTE:

This is a *BETA* version of the RIPE database. Please only use this
version of the software for test purposes.


********************************************************************** 


GETTING STARTED

Before you READ this part, please make sure you have read INSTALL and have
installed the software properly, since all examples in this README assume
that the software is installed.

For those of you who seem to recognize the text in this document,
please read on because there are some significant changes in the
software. 

If you just run a secondary copy of the RIPE database, please read the
file SECONDARIES in this directory. It will explain what to do get get a
secondary copy of the database running.

Make sure you walk through the configuration file conf, and change everything
you wish or need to change. For more info see the manpage for dbconfig(5).

If you just run a RIPE like database, there is not much to change, except
if you wish to change some language. The definition of objects in the
database is also in that file, feel free to have a look to see if you
understand. More documentation on the config file is underway (I hope ;-)

!!!! MAKE SURE YOU CHANGE ALL PATHNAMES IN THE SAMPLE CONFIG FILE !!!!

Then, make sure you have RIPE like databases as mentioned as sources in the
directories specified in the config file. These should be ordinary text
files, with objects separated by empty lines. 

For all these files you can now do:

	bin/netdbm <pathname>

where pathname is the pathname of the text file for the database. See
the manual page for netdbm for options to do classless indexes and
verbose modes.

If you wish to check the index you can use the little script "showdbm" in the
bin directory. Just call it as:

bin/showdbm <databasefile>

It will output for you all keys, followed by the number of references to this
key, and the list of all file offsets for this key in the main database
file. This program was merely written for debugging.

As soon as you have started the indexing (in the background I assume,
although you should be careful with that, since it can generate
output), you can fire of the whoisd. This whoisd will work while the
indexer(s) is(are) still working, it will simply find all that has
been indexed so far. The whoisd MUST not be run from inetd, since it
takes too much time to read the config every time it is started and it
detaches itself from the terminal and acts as a daemon itself.
It will wait for requests on the standard whois port 43, and fork
of children to handle requests. If all is well, it will not generate
any output, except for the logfiles, so:

(Make sure you run this as root, since it will try and listen to port 43, a
privileged port. Also make sure nothing is currently running that port, and
inetd is not managing that port). whoisd uses the REUSEADDR socket
option so it could happen that you have multiple whoisds listening to
the same port!!

	bin/whoisd

It will read the config, and start listening for requests. Whenever you
change the configuration file, make sure to kill and restart the whoisd.

The whoisd currently understands the following arguments (not command line
arguments, but arguments passed by a whois client)

-a              search all databases
-F              fast raw output (implies -Fr)
-L              find all Less specific matches
-m              find first level More specific matches
-M              find all More specific matches
-r              turn off recursive lookups
-s source       search databases with source "source"
-S              tell server to leave out "syntactic sugar"
-t type         requests template for object of type "type"
-T type         only look for objects of type "type"

Most whois clients cannot yet handle these flags, and therefore you should
quote these arguments, so they are passed directly to the whois server:

whois -h whoishost.some.where "-r 192.87.45"
whois -h whoishost.some.where "-s RIPE terpstra"

A Perl RIPE whois client that supports all these options is supplied in
this package. After installation it should be in bin/whois. A version
in C can be retrieved as ftp://ftp.ripe.net/tools/ripe-whois.tar.Z

If the whoisd is called with command line arguments, it will not behave as a
daemon, but will simply act as a client, ie looking up the command line
arguments on the local databases, and will exit. It will also then
understand all of the above options.

Whoisd has a debug mode (-d). See the whoisd manual page for more info.

You should now have a running database server.

USING THE UPDATING TOOLS

I urge you to first test this extensively on a test database. If you wish to
have whoisd work on that database as well, make sure you change the source
for this database, and add it to the config file. You need to add this test
database to your config anyway because that is where the update program will
look for the files it should update.

CAREFUL: do NOT update databases across NFS mounted filesystems. The internal
locking of the database does not properly work across NFS filesystems.

Now try:

bin/newdb data/test.db

or something similar. This should create a file data/test.db which has a
generation date, and a copyright notice in it.

Now make a file somehere with some database objects in it, with the correct
source that points via the config file at the new test database.

Then try:

bin/dbupdate -v the_temporary_file

This will try and put the objects you have in your temporary file in the
database. You can have a look afterwards in the database file to see if they
are actually in there, and use showdbm to check whether the indexes were
properly made. If you have your whoisd working on this test database as well,
it should give immediate response for the new objects.

Now, edit your temporary file so that one object will get an extra field
"delete:" in long format, or "*ud:" in short format. Then run

bin/dbupdate -v the_temporary_file

again. It should now tell you it deleted the object you added the delete flag
to, and if you had more objects in your temp file, it should complain that
updating these objects would result in a NOOP.
Now have a look at the test.db file once more. You should find that the
object you deleted starts with "*XX:" and all the others are still in there.
A showdbm on that file will also tell you that the keys for the deleted
object are gone.

This is the basic way the update mechanism works.

Another option for dbupdate is the -M flag, which causes dbupdate to think
that the_temporary_file is a mail message with all the necessary headers, and
in stead of just updating the objects and generating the output on standard
output, it will send an acknowledgement mail back to the sender of the mail
fed into it. This is easily tested by sending a mail to oneself with database
objects in it, then once you received this message save it to some temporary
file and then call:

bin/dbupdate -v -M temporary_file

You should then get a ack mail sent back to yourself. You can then easily
integrate this with your mailer. For instance in the sendmail aliases file
you could do something like:

my-dbm:		"|/wherever/dbase-beta/bin/dbupdate -v -M"

and that should process the incoming mail, and send back ack mails.  There is
a few things to watch out for in this scenario:

- make sure that dbupdate can WRITE to the various database you wish to
manage. If called from sendmail, the default user will be deamon, so you may
have to set dbupdate with an appropriate set uid or gid bit to be able to
write to the database. If you have a special user just for the
database software, you can use this users .forward file (on UNIX
systems) to pipe mail message into dbupdate. This avoids the use of
suid of guid bits.

- perl has some build in checks for suid perl scripts. These are all taken
care of, so dbupdate should not complain about not being able to do things
because it is running suid or sgid. Have a look in the code for dbupdate to
see what silly things one has to do to get around these checks ...

Now however, you have a clobbered database. This is no problem, although you
may want to clean that database every once in a while (say once a week or so).
To do so try:

bin/cleandb data/test.db

And have another look at test.db if it is finished. You should no longer have
all the deleted objects in there. You will see that the new database is no
longer sorted, but if you do wish a sorted version, try:

bin/sortdb data/test.db

That will output a sorted version of your database on standard output.

Once you understand all that has happened above, you got the basic hang of
the update procedure. Now you can play around a bit with it to see what it
can and cannot do, and what it will and will not accept.

SYNTAX CHECKING

The syntax checking consists of 3 parts:

- object definition check
- values syntax check
- update check

The object definition check will check whether all the attributes for this
object are recognized, whether all mandatory attributes are present and that
stuff. If the object passes this stage, it will go into the syntax check.
The syntax checking is done by src/syntax.pl. If you wish to make your own
additions, have a look in that file to see how it works. The general way is
that you have to add an if statement with your attribute in it:

if ($key eq "xx") {
}

where "xx" is your attribute. You have 2 variables and the object itself to
play with at that point:

$key = the short form attribute
$value = the value of that attribute
$object{$key} = the value inside the object itself (if you modify this, you
        really modify the object - careful with this)

You should then do your syntax checking and return nothing is the
syntax is fine, or:

return $O_WARNING, "a descriptive warning message here";

in case you find this a warning, or:

return $O_ERROR, "a descriptive error message here";

in case you have spotted a real error. Warnings are usually used in case you
have changed something in that attribute value.
It takes some Perl knowledge to do the actual syntax checking. I will write
more about this in a later stage. There are some basic routines you can call
for checking in src/misc.pl see the examples for "ac" (administrative
contac)t for instance. I did not say it was easy, we may change this to be
more friendly at a later stage. 

And yes, I agree that this is a horrible piece of code ...

If you need some help here, send me some mail and I will try to help you
out.

Marten Terpstra
RIPE Network Coordination Centre
