$Id: mailinglist,v 2.4 1992/01/31 12:03:04 berg Rel $

			How to set up mailing lists
			---------------------------

		Written by Stephen R. van den Berg.
					berg@messua.informatik.rwth-aachen.de
					berg@physik.tu-muenchen.de

This document mainly describes a sendmail environment, much of it applies
to non-sendmail mail agents as well.


Contents:
---------	1. Intro
		2. Bouncing mail
		3. The disadvantages
		4. How to circumvent these disadvantages
		5. Why use procmail to filter the mailinglist mail?
		6. How do I use procmail to filter the mailinglist mail?
		7. Now, what does the above all do?
		8. The result of this exercise

1. Intro
   -----

The simplest and most direct way to do it is by insert a line in
the /usr/lib/aliases file looking like:

mylist: fred,john, wilma, barney@bedrock, pebbles

Now all the mail arriving at your machine for "mylist" (either local or
mylist@your.domain) will be automatically forwarded to all the mentioned
addresses (fred, john, etc.).

The address mylist@your.domain is intended for submissions to the list that
are supposed to be forwarded to all the subscribers.  For the administrative
tasks like removals from the list, new subscribtions to the list, or address
changes of subscribers one should create a second entry in the /usr/lib/aliases
file:

mylist-request: your_login_name@your.domain


2. Bouncing mail
   -------------

In order to deal with bouncing mail gracefully, an extra precaution should
be taken.  If for example mail to wilma bounces (user non-existent, mail
filesystem full, etc.), it will bounce back to the original sender.
Now, the only person that should be concerned with distribution failures
should be the mylist-request holder.  Therefore you should be using a
sendmail special alias like:

owner-mylist: mylist-request@your.domain

This way local mail will bounce back to mylist-request@your.domain.


3. The disadvantages
   -----------------

If you are using the above methods, some obvious disadvantages come to mind
however:

a. The subscriber list cannot exceed 1000 bytes (on most sendmails).
b. The subscriber list cannot be changed on-the-fly (/usr/lib/aliases needs
   to be edited, and newaliases has to be run).
c. People cannot be prevented from submitting messages like "Please remove
   me from this mailinglist" to mylist (and thereby annoying all subscribers).
d. People cannot be guarded from themselves in case they insert
   "Return-Receipt-To:" fields in their headers (if they are particularly
   unlucky, they will receive an acknowledge mail from *every* subscriber's
   sendmail).
e. People including "Errors-To:" or "Sender:" fields can cause the bounce
   messages to bypass owner-mylist anyway.
f. There is no way of limiting the number of submitters, i.e. every person
   who knows the name of the mailing list and who can send mail to your.domain
   is able to submit messages to the list.  This means, for example, that you
   cannot limit a mailing list to local users (i.e. only local users can
   submit).
g. You are unable to insert a "Reply-To: mylist@your.domain" in case you
   would want to (this makes replying to the list easier).


4. How to circumvent these disadvantages
  -------------------------------------

a. Can be circumvented by using nested aliases like:
	mylist: mylist1, mylist2
	mylist1: fred,john
	mylist2: wilma,barney@bedrock,pebbles
   This can however, become extremely messy to maintain.

b. This can partly be avoided if you use aliases like:
	mylist: :input:/path/to/the/memberfile
   The memberfile should contain:
	fred,john,wilma,barney@bedrock,pebbles
   You cannot avoid using newaliases however, and *will* get extremely messy
   if you have to start using nested aliases.

c. Can only be taken care of by using a mailfilter like procmail.

d. Can only be taken care of by using a mailfilter like procmail.

e. Can only be taken care of by using a mailfilter like procmail.

f. Can only be taken care of by using a mailfilter like procmail.

h. Can only be taken care of by using a mailfilter like procmail.


5. Why use procmail to filter the mailinglist mail?
   ------------------------------------------------

Instead of using a mailfilter you could also take care of most of the problems
three till seven by editing the sendmail.cf file.  I strongly would recommend
against this approach however, since this will be too much of a customizing
operation and surely will not be a trivial task (in all cases).	 As a general
rule: don't mess with a sendmail.cf file once it is working :-).

Now, you could, instead of procmail, simply use immediate VNIX commands
like grep, sed, awk to do the mail filtering.  Again, there are some obvious
disadvantages with this approach:

A. In case any system resources go out (no more file descriptors, no more
   swap space, process table full, file system full (for temporary files))
   your awk or shell script will fail generously (i.e. several bad things
   could happen: mail munged, truncated, lost, hanging awk or sh programs,
   etc., you get the picture).

B. All mail headers (including From: and Reply-To:) could very well be
   multi-line headers; it will be very difficult to make it understandable
   to awk that somehow the header line could continue on the next line
   (in case you want to remove a header, or do some complicated substitution).

C. Another hairy problem will be determining the end of the header, of course
   that is solvable, but you have to make some extra precautions in your
   awk script to ensure that any substitutions/changes will not occur in
   the body of the message.

Procmail does not *directly* allow you to change any headers, but that
feature is not really necessary since you can tell procmail to send ONLY the
header through some filter of your choice.

To comment on the previously mentioned three disadvantages:

A. procmail takes care of that.	 Should the filter have problems anyway,
   procmail will graciously notice that the filter was in some kind of
   trouble, and will try something else with the original unmunged mail
   (you can specify what it should do of course, obvious choices: try
   the same filter again, drop the mail in a file and send you a notice,
   forward the mail to you instead (unfiltered), etc.)

B. procmail will concatenate any headers that were continued according to
   the RCF 822 recommendations, i.e. your filters will see one line per header.

C. procmail can be told to send the header, the body or both through the
   filter, hence your filter need not watch out to avoid doing any
   substitutions in the body, and the filter can therefore be a lot simpler.

Procmail has some additional advantages too:

 -- It will probably all go a bit faster, since only the header of the mail
    is being piped through the filter.	Also, procmail reads in the mail in
    16KB chunks, not line by line as sed does.

 -- You could use procmail to filter out any messages to the normal mailing
    list that should have gone to the mylist-request and remail them to
    mylist-request.

Well, anyway, as you see, procmail does not give you everything you would want,
but this was intentional in accordance to the true VNIX spirit (modularity).
What procmail does provide is a *very* reliable hook (you might say it
provides an anchor :-) for any mail processing you might do.  For the more
complex things you still have to use shell scripts or call other programs
from within procmail, but then again, that saves you from learning any
particular syntax procmail would have had to do the same.

As it happens, the accompanying formail program is able to cater to most
(if not all) of your needs regarding mail munging.

If, on the other hand, you want to do more complex things like moderated
mailing lists with several moderators, etc., if would suggest you take
a look at the more complex/specialised mail-server packages like:
listserv available on cs.bu.edu, author: tasos@cs.bu.edu
Of course, most of what these packages can do, can be done with procmail as
well; it is just that you might be forced to write some additional shell
scripts/programs to accomplish the same.


6. How do I use procmail to filter the mailinglist mail?
   -----------------------------------------------------

First you have to create these two entries in your /usr/lib/aliases file of
machine "your.domain" (the mylist: line should be typed in as one long line):

mylist: "|IFS=' ';exec /usr/local/bin/procmail /some/path/listrc subscribers=/some/path/memberlist list=mylist@your.domain listreq=mylist-request@your.domain"
mylist-request: your_login_name@your.domain
owner-mylist: mylist-request

Create a file named /some/path/memberlist which contains the names of the
subscribers separated by whitespace (blanks, tabs or newlines) like:

 fred john  wilma  barney@bedrock  pebbles

The /some/path/listrc file you should look like the sample listrc file
supplied in this directory.  This listrc file need only be present once,
it will cater for all the mailinglists you like to create.


7. Now, what does the above all do?
   --------------------------------

If mail arrives at mylist, first of all procmail will be started using
/some/path/listrc as the rcfile.  Then it will grep the header to check if
it could be a bounced message after all (from postmaster or mailer-daemon),
or if it probably is a request message.	 If neither applies, procmail will
filter just the header of the message through formail.

formail will remove any "Return-Receipt-To:" fields, and will provide plain
substitutes for "Errors-To:" and "Sender:".  Then it will look for
any "Reply-To:" fields which are already in the header and rewrite them
as "Old-Reply-To:";  after having done this it will add your "Reply-To:"
field.	BTW, the "Return-Receipt-To:" and "Errors-To:" fields are not
recommended by the RFC-822, they are however commonly supported by most
sendmails;  if they are not supported however, they won't hurt, they will
simply be ignored.

Then, the mail is piped into $SENDMAIL which receives, as command line
arguments, the addresses of all subscribers.  The option -f will only
take effect if sendmail is running under daemon privileges; this only
occurs if the sender of the mail is *not* a local user; if the sender
is a local user, then sendmail (and procmail) runs as the local user.

*********************************** WARNING **********************************
*									     *
* For this reason it might be wise to allow writing of the memberlist file   *
* only (to a list maintainer), keep the listrc file under *root supervision* *
* (i.e. owned by a very reliable person (e.g. root), world readable, but NOT *
* world writeable).							     *
*									     *
******************************************************************************


8. The result of this exercise
   ---------------------------

As you can see, we have addressed and solved every single one of the original
seven problems (well, ok, except problem f, that one is left as an excercise
to the reader; shouldn't be too difficult).


P.S. Any suggestions/corrections/improvements on this document are welcome.
