head     1.18;
branch   ;
access   ;
symbols  lastpatch:1.12;
locks    rsalz:1.18; strict;
comment  @# @;


1.18
date     91.07.22.10.26.48;  author rsalz;  state Exp;
branches ;
next     1.17;

1.17
date     91.07.18.20.13.11;  author rsalz;  state Exp;
branches ;
next     1.16;

1.16
date     91.07.18.09.55.44;  author rsalz;  state Exp;
branches ;
next     1.15;

1.15
date     91.07.17.16.27.01;  author rsalz;  state Exp;
branches ;
next     1.14;

1.14
date     91.07.17.16.15.48;  author rsalz;  state Exp;
branches ;
next     1.13;

1.13
date     91.04.01.09.43.16;  author rsalz;  state Exp;
branches ;
next     1.12;

1.12
date     91.03.15.16.42.15;  author rsalz;  state Exp;
branches ;
next     1.11;

1.11
date     91.02.12.15.08.18;  author rsalz;  state Exp;
branches ;
next     1.10;

1.10
date     91.02.12.14.42.49;  author rsalz;  state Exp;
branches ;
next     1.9;

1.9
date     90.03.12.10.33.57;  author rsalz;  state Exp;
branches ;
next     1.8;

1.8
date     89.11.10.14.50.30;  author rsalz;  state Exp;
branches ;
next     1.7;

1.7
date     89.10.24.22.09.18;  author rsalz;  state Exp;
branches ;
next     1.6;

1.6
date     89.07.10.12.19.22;  author rsalz;  state Exp;
branches ;
next     1.5;

1.5
date     88.08.10.12.24.01;  author rsalz;  state Exp;
branches ;
next     1.4;

1.4
date     87.12.18.21.26.34;  author rsalz;  state Release1;
branches ;
next     1.3;

1.3
date     87.09.09.17.01.41;  author rsalz;  state Alpha;
branches ;
next     1.2;

1.2
date     87.09.08.18.30.25;  author rsalz;  state Exp;
branches ;
next     1.1;

1.1
date     87.08.26.11.34.30;  author rsalz;  state Exp;
branches ;
next     ;


desc
@Internal documentation and installation instructions.
@


1.18
log
@Add comments and #ifdef to use C News getabsdate routines.
@
text
@INTRODUCTION
------------
This kit provides two programs for "linking" RFC822 Mail messages and
RFC1036 Usenet News articles.  Each half of the conversion is handled by a
different program, mail2news or news2mail.  A few utility programs are
also included.

With these programs and the right set of mail aliases and news sys and
active file entries, it is possible to build any set of moderated,
unmoderated, one-way, or bi-directional gateways between any set of news
and mail groups and lists that your little heart desires.

If you run a small site with a couple of mostly-local mailing lists, you
probably don't want to bother setting this stuff up.  Instead, either
convert everything directly to News, or set up local moderated groups.  On
the other hand, if you provide gateway service to the Internet (e.g., UCB)
or large organization, then this stuff is for you.  News, especially with
the proliferation of NNTP (RFC977) and related clients, is generally more
efficient than mail for disk space, CPU cycles, and network usage.

The programs work with Sendmail or MMDF and News 2.11.8 or later.  I don't
know of anything off-hand that would prevent this from working with C news
(did Erik ever say thank you, Henry?).  We've only run the programs on BSD
hosts, but a start toward a System V port has been made.  Who knows, it
might even work right now.

Erik Fair <fair@@apple.com> wrote the original version of this package a
couple of years ago as "nrecnews" distributed with 2.11 and as "gateway", a
very tricky awk/shell/sed script.  I got copies, and recoded it all in C.
I also completely overhauled nrecnews, changed the names, added some
utility programs, and wrote the documentation.  It seems pretty solid now,
and is processing several hundred messages a week at BBN and elsewhere.

INSTALLATION
------------
Mail2news uses the date parser from the 2.11 netnews distribution; edit
the Makefile to point to where you keep your source.  If you want to use
the better C News date parsing routines, edit the Makefile as directed
therein.  Next, edit gate.h as appropriate for your system.  There are
some fairly detailed comments explaining the meaning of the #define's, but
good luck.  Unfortunately, this is one of those things that you have to
already know a fair bit about before you can use it.  As one way up the
learning curve, play with gag and test-gag -- the hardest part is almost
always getting the right aliases and news entries set up.

The "signoff" code is pulled out into a separate program and is also part
of mail2news.  This is so that a properly paranoid administrator can trap
the exit code, and forward rejected postings to a human to verify.  If
you trust the heuristics in that program, just turn on the -F flag.

It would be great if someone would send me a draft of an installation
document, including a description of how to set up various types of
lists.  Until that happens, the next couple of paragraphs (and the
comments in gate.h) will have to suffice.

News uses the Message-ID header to reject an incoming duplicate.  This
will also prevent it from sending the duplicate back out, obviously.  Be
careful if you feed mail that doesn't have Message-ID's into mail2news.
MMDF doesn't require one, for example.  Under these circumstances you have
to make sure that your sys file doesn't send the message back out the
gateway.  It will have a Message-ID so *you* don't see the loop, but
everyone else on the mailing list will, and they'll yell out you.  One way
to fix this is to have mail2news use the "-x" flag to put some fake site
in the header, then put /fakesite in the sys file entry that calls
news2mail.  Or, compile with FIXED_PATH or GATEWAY_NAME enabled and use
the /GATEWAY_NAME sub-field in your sys file so that the incoming article
doesn't get sent back out.  Another option is careful use of the L flag to
send only local articles back out.

The biggest problem with news2mail gatewaying is that you usually need
a separate entry for every group in your sys file.  One option is to create
a single sys file entry that matches all the groups you want to gateway,
like this:
    mail-gateway:...groups...::/usr/lib/news/mailgate %s
mailgate strips the newsgroup name out and looks it up in a file.  If
there is cross-posting you might have to be much more clever, but this
might work for simple things:
	#! /bin/sh
	# Turn /usr/spool/news/foo/bar/123 into foo.bar
	NG=`echo $1 | sed -e s@@/usr/spool/news@@@@ -e 's@@/[0-9]*$@@@@' -e 'y@@.@@/@@`
	LINE=`grep "^$1" </usr/lib/news/gateways | sed -e 's/^$1[ 	]*'`
	eval /usr/lib/news/newsmail $LINE $1
The /usr/lib/news/gateways file looks like this:
	# news/mail gateway lines.
	#newsgroup	the other three params
	rec.music.cd	cd-lovers-list usenet cd-lovers-list-request

Another way to do large gatewaying is to create a fake host and send all
mail for that host to a special mailer.  The mailer can that examines its
argument list, builds a suitable argument list for mail2news and then
execs to mail2news.  At least in theory, this allows a mail message with
several news recipient addresses to be cross-posted to several
newsgroups.  The mailer uses a database to determine what distribution to
use and whether to add an Approved header.

It can be a good idea to put the mailing list address on the Organization
header.  If the mailing list is moderated, make the newsgroup moderated so
that postings get mailed back to the person preparing the digest.  You can
also merge related lists into a single newsgroup, use the "n" flag in the
active file, and force people to send mail using the address found in the
Organization header.

If you run C News, you want to pipe into relaynews rather than inews
for three reasons:
    1.  A mailing to a moderated newsgroup will be rejected, rather than
	mailed to the moderator, if no Approved header is present.
    2.  Mailings are possible to newsgroups with the "n" flag in the active
	file entry, while inews will reject them.
    3.  Henry Spencer says that's what you should do.

Another method is to define a class for all top-level groups (this often
changes much less frequently then the list of groups to gateway), and
define a "news" mailer to receive mail sent to such addresses.  If the
following class is defined:
    CNcomp misc news rec sci soc talk alt CNunix\-pc ddn gnu u3b pubnet
then these lines can be added to the "mailer resolver" part of ruleset zero:
    R$=N.$*   $#news$:$1.$2   Posting to news
with the following mailer defined:
    Mnews,  P=/usr/lib/news/mail2news,
	    F=DFMls, M=131071, A=mail2news -n $u

(Thanks to Karl Kleinpaste and Neil Rickert for many of the good ideas
in this section.)

PROBLEM AREAS AND THINGS STILL TO BE DONE
-----------------------------------------
Cross-posting and multiple mail recipients are not handled very well.
Cross-post means each mailing list gets a separate message.  Can this be
worked-around with B news's multi-cast?

Someone sending mail to lista and listb means the message only goes into
news.list.a, or news.list.b, depending on which gateway alias gets it
first, which isn't great.  One hack, to avoid losing messages, is to munge
the message-id to put the newsgroup name in there.

How do you bypass the "more included than new text" check in inews?

A news2mail option that specifies an Errors-To header.

A mail2news option that specifies where errors get mailed to.

A news2mail option that specifies a Sender header.

It would be nice to automatically generate (or update) References lines.
This could be done by parsing In-Reply-To lines (which aren't
standardized, but have a few formats).  Also, keep a database of Subject
lines and Message-ID's and use that to track down "Re:" messages.

If mail2news gets a message without a Message-ID, generate one.  Use a CRC
on the first ten lines of message body counting only alphanumeric
characters.  In news2mail recompute the Message-ID and send out a cancel
for it.  This would help break loops involving mailers that strip out
Message-ID headers.  (From Neil Rickert.)

FINALLY
-------
here is a mailing list for sites doing this in a big way, primarily on the
Internet.  Write to news-n-mail-request@@bbn.com to be added.  Please don't
ask to join if you're only doing local gateways of a couple of groups.

If you don't send e-mail, then use comp.mail.misc and news.software.b;
cross-post.

Rich $alz
rsalz@@bbn.com
@


1.17
log
@Merge in Neil's ideas, and move Karl's from mkmailpost.1 into the README.
@
text
@d37 8
a44 7
the Makefile to point to where you keep your source.  Next, edit gate.h as
appropriate for your system.  There are some fairly detailed comments
explaining the meaning of the #define's, but good luck.  Unfortunately,
this is one of those things that you have to already know a fair bit about
before you can use it.  As one way up the learning curve, play with gag
and test-gag -- the hardest part is almost always getting the right
aliases and news entries set up.
@


1.16
log
@Replace "I'm burnt out comment" with different words.
@
text
@d1 1
a1 1
Introduction
d34 1
a34 1
Installation
d69 4
a72 3
The biggest problem with news2mail gatewaying is that you need a separate
entry for every group in your sys file.  One option is to create a single
sys file entry that matches all the groups you want to gateway, like this:
d87 38
a124 1
Problem areas and things still to be done
d154 1
a154 1
Finally
@


1.15
log
@Another slight tweak about using -x.
@
text
@d50 4
a53 4
At this point I should be telling you how to set up the different types of
groups, but I'm too burnt out on this to write a great deal.  If anyone
can send me something to include, I'd appreciate it.  At any rate, here a
a few random operational notes.
a68 1

d86 2
a87 2
Flaws
-----
d89 2
a90 2
Cross-post means each mailing list gets a separate message.  Can this
be worked-around with B news's multi-cast?
a98 2
To-do list
----------
d105 2
a106 2
It would be nice to automatically generate (or update) References
lines.  This could be done by parsing In-Reply-To lines (which aren't
d110 2
a111 2
If mail2news gets a message without a Message-ID, generate one.  Use a
CRC on the first ten lines of message body counting only alphanumeric
@


1.14
log
@Add WHOAMI #define case.
    Brendan Kehoe <brendan@@cs.widener.edu>
Split didn't make sure final string was terminated.
    Neil Rickert <rickert@@cs.niu.edu>
TrimEnvironment was not quite right.
    Neil Rickert <rickert@@cs.niu.edu>
Handle quoted characters and a lot of other stuff in the
From parser.
Invalid dates weren't caught in the RFC822-parser.
    Neil Rickert <rickert@@cs.niu.edu>
Make Received lines RFC822 conformant in news2mail.
Change Makefile to be more clear about how to get getdate.o
Don't turn foo.trw.com into foo.TRw.COM in Casify.
Add clobber to Makefile.
    Rich $alz <rsalz@@bbn.com>
@
text
@d58 11
a68 8
MMDF doesn't require one, for example.  Under these circumstances you
have to make sure that your sys file doesn't send the message back out
the gateway.  It will have a Message-ID so *you* don't see the loop, but
everyone else on the mailing list will, and they'll yell out you.
The best cure for this is to compile with FIXED_PATH or GATEWAY_NAME enabled
and use the /GATEWAY_NAME sub-field in your sys file so that the incoming
article doesn't get sent back out.  Another option is careful use of the
L flag to send only local articles back out.
@


1.13
log
@patch01:  README had wrong RFC number.
patch01:	Mark Moraes <moraes@@cs.toronto.edu>
patch01:  Not all systems have <netdb.h>.
patch01:  Have TrimEnvironment set environ to empty array, not a null pointer.
patch01:  	Gordon L. Burditt <gordon@@sneaky.lonestar.org>
patch01:  Closefile in gag.y was closing the wrong file.
patch01:	Mike Pelletier <stealth@@engin.umich.edu>
patch01:  There is a multi-word typo in news2mail.1
patch01:  	Brian Kanton <brian@@ucsd.edu>
patch01:  Was getting coredumps if UUNAME and L_SYS are #undef'd.
patch01:	Kannan Varadhan <kannan@@oar.net>
patch01:  News2mail was putting out an extra space between the header name
patch01:  and the colon.
patch01:	Michael Gengenbach <gengenba@@forwiss.uni-passau.de>
patch01:  Fixed GATEWAY/GATEWAY_NAME confusion; added more explanatory
patch01:  comments about what it (and FIXED_PATH) do.
patch01:  Added C News and InterNetNews support to gag.  (-i is currently
patch01:  about as useful to the outside world as -b, but I need it.)
patch01:  Added warning that gag overwrites files and doesn't edit them,
patch01:  and some other verbiage.
patch01:  Described how to prevent loops and a simple-minded way that uses
patch01:  one sys file entry when gatewaying many newsgroups into mail.
patch01:  	Rich $alz <rsalz@@bbn.com>
@
text
@d32 1
a32 1
and is processing a few hundred messages a week at BBN and elsewhere.
d41 3
a43 3
before you can use it.  As one way up the learning curve, play with the
gag -- the hardest part is almost always getting the right aliases and news
entries set up.
d52 1
a52 1
can send me something to include, I'd appreciate.  At any rate, here a
d56 2
a57 2
will also prevent it from sending the duplicate back out, obviously.  If
you feed mail into mail2news that doesn't have Message-ID's, be careful.
d62 1
a62 1
The best cure of this is to compile with FIXED_PATH or GATEWAY_NAME enabled
d76 1
a76 1
	NG=`echo $1 | sed -e s@@/usr/spool/news@@@@ -e 's@@/[0-9]*$@@@@' | tr . /`
d87 2
a88 2
Cross-post means each mailing list gets a separate message.  Does
B news's multi-cast fix this, perhaps?
d96 19
@


1.12
log
@Remove "don't release" notice.
@
text
@d4 1
a4 1
RFC1076 Usenet News articles.  Each half of the conversion is handled by a
d51 3
a53 2
groups, but I'm too burnt out on this to bother.  If someone has some
words here to get me started, I'd appreciate it.
d55 29
d99 3
a101 5
I am maintaining this release, but Erik is still involved in the issues
and might be interested in what you have to say.  There is a mailing list
for sites doing this in a big way, primarily on the Internet.  Write
to news-n-mail-request@@bbn.com to be added.  Please don't ask to join
if you're only doing local gateways of a couple of groups.
@


1.11
log
@Ask folks not to distribute too widely.
@
text
@a0 6

NOTE:	PLEASE DO NOT DISTRIBUTE THIS TOO WIDELY; AN OFFICIAL RELEASE,
	AT VERSION 2.0, WILL BE POSTED IN A FEW WEEKS.
	THANKS,
		/rich $alz

d28 1
a28 1
couple of years ago as "nrecnews" distributed with 2.11 and as gateway, a
d32 1
a32 1
and is processing a few hundred messages a week at BBN.
a66 2
The code in hdr.c is very ugly and could stand to be rewritten.

d72 2
a73 1
to news-n-mail-request@@bbn.com to be added.
@


1.10
log
@Document signoff, remove reference to Erik's m4 stuff.
@
text
@d2 5
@


1.9
log
@Update Fair's address, reformat parameter declarations.
@
text
@d43 2
a44 1
gag.
d46 4
a49 4
Once you've gotten the programs to compile, you'll want to play around
with the alias generator to build stuff.  Erik has a set of M4 macros and
Makefile that you can use if you prefer.  The hard part is in setting up
the right aliases and news entries to get the system up and working.
d59 1
a59 1
multi-cast fix this, perhaps?
d74 2
a75 1
for sites doing this in a big way, primarily on the Internet.
@


1.8
log
@Pretty much rewrote it.
@
text
@d28 6
a33 6
Erik Fair <fair@@ucbarpa.berkeley.edu> wrote the original version of this
package a couple of years ago as "nrecnews" distributed with 2.11 and as
gateway, a very tricky awk/shell/sed script.  I got copies, and recoded it
all in C.  I also completely overhauled nrecnews, changed the names, added
some utility programs, and wrote the documentation.  It seems pretty solid
now, and is processing a few hundred messages a week at BBN.
d42 1
a42 1
before you can use it.  As a way up the learning curve, play with the
d56 1
a56 1
Cross-posting and multiple mail recipients are handled very well.
d67 1
a67 2
The code in hdr.c is very ugly.

@


1.7
log
@Reword some stuff.
@
text
@d6 2
a7 1
different program, mail2news or news2mail.
d14 7
a20 9
If you run a small site with only a couple of machines and a few
mostly-local mailing lists, you probably don't want to bother setting this
stuff up.  Instead, either convert everything directly to News, or set up
local moderated groups.  On the other hand, if you provide a large-scale
gateway service to the ARPA Internet (e.g., UCB) or you're at a large
site where you can't control everything so tightly (e.g., BBN), then this
stuff is for you.  News, especially with the proliferation of NNTP
(RFC977) and related clients, is generally more efficient than mail for
disk space, CPU cycles, and network usage.
d22 5
a26 5
The programs work best with sendmail or MMDF and News 2.11.8 or later.  I
don't know of anything off-hand that would prevent this from working with
C news (did Erik ever say thank you, Henry?).  We've only run the programs
on BSD hosts, but a start toward a System V port has been made.  Who
knows, it might even work right now.
d28 6
a33 10
The original version of this package were written by Erik in xxxxx, as a
collection of scripts and programs.  Credit him for the initial design, and
accomodating that to the reality of the networks involved.  In the fall of
1988, Rich converted everything to C and rewrote the existing programs for
esthetics (boo) and efficiency (hooray).  Credit him for the documentation,
and any bugs you find.  The regular expressions routines (regex.c and
regex.3) were written by Ozan Yigit <yunexus!oz>, and published in Volume 6
of comp.sources.unix.  Dan Tappan <tappan@@bbn.com> was suckered into running
this stuff at BBN and made a "final" set of fixes.  The package seems
pretty solid now, and is processing a few hundred messages a week at BBN.
d40 4
a43 2
explaining the meaning of the #define's, but in general you should know
what you're doing.  Good luck. :-)
a44 4
By default, the install target in the Makefile is a no-op.  This is mainly
due to history, but see the caveat about the UUCP_INET macro in the
Makefile.

d50 3
a52 3
At this point I (R$) should be telling you how to set up the different
types of groups, but I'm too burnt out on this to bother.  If someone has
some words here to get me started, I'd appreciate it.
d72 9
a80 4
I am maintaining this release, but Erik probably knows more about how
the real world works.  Send your mail to him, or me, as approporiate:
	Rich $alz			Erik Fair
	rsalz@@bbn.com			fair@@ucbarpa.berkely.edu
@


1.6
log
@Minor tweaks, bug-fixes from alt.sources posting.
@
text
@d20 1
a20 1
(RFC977) and related clients, is generally more efficient than mail, for
d23 5
a27 3
The programs work best with sendmail or MMDF and News 2.11.8 or later.
We've only run the programs on BSD hosts, but a start toward a System V
port has been made.  Who knows, it might even work right now.
d33 1
a33 1
esthetics (boo) and efficiency (yeah).  Credit him for the documentation,
a39 1

d43 4
a46 3
the Makefile to point to where you keep your source.  There are a handful
of other parameters, all at the beginning of the gate.h header file.  If
should be obvious what values you'll need to put in there.
d64 1
a64 1
Cross-post means each person mailing list gets a separate message.  Does
d69 2
a70 2
first.  Not cool.  Can it be fixed?  One hack, to avoid losing messages,
is to munge the message-id to put the newsgroup name in there.
d72 1
d74 3
d79 4
a82 5
Please send any comments, kudo's, suggestions, or brick-bats to both of us:
	Erik Fair				Rich $alz
	fair@@ucbarpa.berkely.edu		rsalz@@bbn.com

	$Header: README,v 1.5 88/08/10 12:24:01 rsalz Locked $
@


1.5
log
@Turn yetti!oz into yunexus!oz
@
text
@d5 2
a6 2
RFCxxx Usenet News articles.  Each half of the conversion is handled by a
different program, mail2news or news2mail (you figure out who does what).
d17 1
a17 1
gateway service to the ARPA Internet (e.g., UCBVAX) or you're at a large
d20 1
a20 1
(RFCxxx) and related clients, is generally more efficient than mail, for
d23 3
a25 3
The programs work best with sendmail or MMDF and News 2.11.8 (or later,
presumably).  We've only run the programs on BSD hosts, but a start toward
a System V port has been made.  Who knows, it might even work right now.
d31 1
a31 1
(bad) esthetics and (good) efficiency.  Credit him for the documentation,
d35 2
a36 1
this stuff at BBN and made the final set of fixes.
d41 4
a44 6
Mail2news needs a few object and header files from the 2.11 distribution,
and must be compiled on a machine where you have built and are running
2.11 news.  The other parameters are in gate.h -- it should be obvious
what to edit.  One thing that is probably not obvious is that you
will have to copy the uucp-2-inet file to the location specified in the
header file.
d46 3
a48 4
The hard part is in setting up the right aliases and news entries to get
the system up and working.  Erik has a complicated set of M4 macros and
a Makefile that can build the complete Internet/Usenet gateway aliases,
etc., now in use at Berkeley.
d50 5
d59 5
d65 6
d77 1
a77 1
	$Header: README,v 1.4 87/12/18 21:26:34 rsalz Release1 $
@


1.4
log
@Add some more words.
@
text
@d33 1
a33 1
regex.3) were written by Ozan Yigit <yetti!oz>, and published in Volume 6
d63 1
a63 1
	$Header: README,v 1.3 87/09/09 17:01:41 rsalz Alpha $
@


1.3
log
@Incorporate changes made for bbn.com production environment.

@
text
@d6 1
a6 2
different program, mail2news or news2mail -- the direction should be
obvious from the program name.
d8 2
a9 2
With these programs, and a judicious set of mail aliases and news sys
and active file entries, it is possible to build any set of moderated,
d29 1
a29 1
accomodating that to the reality of the networks involved.  In the Fall of
d34 2
a35 1
of comp.sources.unix.
d52 3
a54 1
		**NEED MORE WORDS HERE**
d63 1
a63 1
	$Header: README,v 1.2 87/09/08 18:30:25 rsalz Exp $
@


1.2
log
@Credit Ozan.
@
text
@d4 1
a4 1
This kit provides two programs for linking RFC822 Mail messages with
d6 2
a7 2
different program -- mail2news or news2mail -- the direction should be
obvious from the name which is used for what.
d9 1
a9 1
With these two programs, and a judicious set of mail aliases and news sys
d24 3
a26 2
The system works best on a host running sendmail and 2.11.8 (or later,
presumably) news.  We've only run this stuff on BSD hosts.
d32 4
a35 4
esthetics and efficiency.  Credit him for the documentation, and any bugs
you find.  The regular expressions routines (regex.c and regex.3) were
written by Ozan Yigit <yetti!oz>, and published in comp.sources.unix,
Volume 6.
d42 4
a45 2
2.11 news; the other parameters are in gate.h -- it should be obvious
what to do.
d48 3
a50 1
the system up and working.
d52 1
d54 3
a56 2
Epilog
------
d61 1
a61 1
	$Header: README,v 1.1 87/08/26 11:34:30 rsalz Exp $
@


1.1
log
@Initial revision
@
text
@d25 1
a25 1
presumably) news.
d32 3
a34 1
you find.
d47 3
d54 1
a54 1
	$Header:$
@
