
Network Working Group                          John C. Klensin
Internet Draft                                       Ned Freed
						   Keith Moore
					       Jeroen Houttuin
                            <draft-ietf-mailext-smtpas-00.txt>

	 Requirements for Internet Mail Transport Agents
			January 24, 1995


                     Status of this Memo

This document is an Internet-Draft.  Internet-Drafts are
working documents of the Internet Engineering Task Force
(IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-
Drafts.

Internet-Drafts are draft documents valid for a maximum of six
months. Internet-Drafts may be updated, replaced, or obsoleted
by other documents at any time.  It is not appropriate to use
Internet-Drafts as reference material or to cite them other
than as a "working draft" or "work in progress".

To learn the current status of any Internet-Draft, please
check the 1id-abstracts.txt listing contained in the
Internet-Drafts Shadow Directories on ds.internic.net (US East
Coast), nic.nordu.net (Europe), ftp.isi.edu (US West Coast),
or munnari.oz.au (Pacific Rim).


1.  Abstract

This document reflects apparent ambiguities in RFC 821 that
remain after the clarifications in RFC 1123.  It also includes
some material from RFC 1123 that appeared to need amplification.
These have been identified in multiple ways, mostly by tracking
flaming on the header-people list and problems of unusual
readings or interpretations that have turned up as the SMTP
extensions have been deployed.  It is important to note that
everything here is in response to some identified confusion or
bad behavior, not just paranoia.  

Although SMTP was designed as a mail transport and delivery
protocol, this specification also contains information that is
important to its use as a "mail posting" protocol, as
recommended for POP3 [RFC-POP3] and IMAP4 [RFC-IMAP4].

Except when the historical terminology is necessary for clarity,
this document uses the current "client" and "server" terminology
to identify the sending and receiving SMTP processes,
respectively. 

A companion document discusses mail bodies and formats: RFC-822,
MIME, and their relationship. 


2. Clarifications, Corrections, and Requirements for SMTP
implementations.

2.1. Eight bit transmission.

The unextended SMTP service provides seven bit transport
only.  SMTP clients MUST NOT transmit messages with information
in the high-order bit of octets.   If such messages are
transmitted in violation of this rule, receiving SMTP servers
MAY clear the high-order bit or reject the message as invalid.
    Reference: RFC821, page 3, next-to-last paragraph.

Eight-bit transmission MAY be requested of the server by the
client using extended SMTP facilities and the "8BITMIME"
extension. 

2.2.  VRFY and EXPN issues

2.2.1  VRFY failure conditions

When a name that is the argument to VRFY could identify more
than one mailbox, the server MAY either note the ambiguity or
identify the alternatives.   In other words, either of the
following are legitimate response to VRFY:

	553 User ambiguous
   or
	553- Ambiguous;  Possibilities are
	553-Joe Smith <jsmith@host.domain>
	553-Harry Smith <hsmith@host.domain>
	553 Melvin Smith <dweep@host.domain>

   Under normal circumstances a client receiving a 533 reply
   would be expected to expose the result to the user.

   Reference: RFC821, page 8 and example on page 9.


2.2.2. VRFY normal response.

When normal (2yz or 551) responses are returned from a VRFY or
EXPN request, the reply MUST include the mailbox name, e.g.,
"<foo@bar>" (where "bar" is a fully qualified domain name) must
appear in the syntax.  EXPN and VRFY MUST return only legal
domain addresses that are usable in SMTP RCPT commands.

2.2.3 Meaning of VRFY or EXPN success response.

A server MUST NOT return a 220 code in response to a VRFY or
EXPN command unless it has actually verified the address.  In
particular, a server MUST NOT return 220 if all it has done is
to verify that the syntax given is valid.  In that case 502
(Command not implemented) or 500 (Syntax error, command
unrecognized) SHOULD be returned (note that implementation of
VRFY is required by RFC1123 and EXPN is strongly recommended;
this specification does not change that requirement and, hence,
implementations that return 500 or 502 for VRFY are not in
compliance with the specification).

2.2.4 VRFY, EXPN, and security.

As RFC-1123 points out, individual sites may want to disable one
or both of VRFY or EXPN for security reasons.  As a corollary to
the above, implementations that permit this MUST NOT appear to
have verified addresses that are not, in fact, verified.
Implementations disabling these commands for security reasons
SHOULD return a 252 response, rather than a code that could be
confused with successful or unsuccessful verification.

Returning a 250 reply code with the address listed in the VRFY
command after having checked it for syntax only violates this
rule.  Of course, an implementation that "supports" VRFY by
always returning 550 whether or not the address is valid is
equally not in conformance.

2.2.5.  Forwarding responses and other addresses.

RFC-821 permits an SMTP server to return a 551 reply (User not
local; please try...) to indicate that the client should send
the mail elsewhere.  Such a "forwarding address" SHOULD be
stable and MUST be usable from the worldwide Internet (e.g., it
must represent a mailbox at a fully qualified domain name).
Similarly, a 251 reply (User not local; will forward...) MAY be
returned to inform the client that the message has been (or will
be) forwarded.

SMTP client software that is designed to take advantage of 251
or 551 replies should note that, in practice, mail forwarding is
used for both "change of address" (both temporary and permanent)
and "centralized post office" purposes and that it is impossible
to differentiate between these two without additional
information.  Return of 251 or 551 replies is never mandatory.

2.2.6 Semantics and applications of EXPN.

While EXPN is often very useful in debugging and understanding
problems with mailing lists and multiple-target-address aliases,
some systems have attempted to use source expansion of mailing
lists as a means of eliminating duplicates.  The propagation of
aliasing systems with mail on the Internet--both for hosts
(typically with MX and CNAME DNS records) and for mailboxes
(various types of local host aliases) has made it nearly
impossible for these strategies to work, and mail systems SHOULD
NOT attempt them.

2.3. Typographical errors.

2.3.1. The next to last line on page 11 of RFC821 contains a
typographical error.  The line should end "...mail is delivered
to the"

2.3.2.  On page 18, third line from the end:
 "supporting be mail" -> "supporting a mail"


2.4. Termination of connections.

   A server-SMTP MUST NOT intentionally close the connection
   except: 
      o After receiving a QUIT connand and responding with a 221 reply.
      o After detecting the need to shutdown the SMTP service and
        returning a 451 reply to any command.

   In particular, a server that closes connections in response
   to commands that are not understood is in violation of this
   specification. 

   A server-SMTP which is forcibly shut down via external
   means SHOULD attempt to send a line containing 451 response
   code to the client-SMTP before exiting.  The client-SMTP will
   normally read the 451 response code after sending its next
   command.   
   
   Reference: RFC821, page 13 and the discussion of QUIT on
   RFC821, page 26. 

2.5.  SMTP server greeting message.

SMTP Servers SHOULD include identification of their software and
version information in the connection greeting reply after the
220 code. This practice permits much more efficient isolation
and repair of any problems.  While some systems also identify
their contact point for mail problems, this is not a substitute
for maintaining the required Postmaster address.

Reference: RFC821, page 19.

2.6. The Return-path header.

RFC1123 requires that the SMTP server system making final
delivery create a Return-Path header that contains the reverse
path.  It is sometimes difficult for an SMTP server to determine
whether or not it is making final delivery since forwarding or
other operations may occur after the message is delivered.
However any further (forwarding, gateway, or relay) systems
MAY remove the return path and rebuild the MAIL FROM command
as needed to ensure that exactly one such line appears in a
delivered message.  An SMTP server MUST NOT add a return path to
a message header that already contains one without removing the
original Return-path header.

The primary intent for the Return-path is that it designate the
address to which messages indicating non-delivery or other mail
system failures at to be sent.  For this to be unambigious,
exactly one return path should be present when the message is
delivered.  Systems using RFC822 syntax with non-SMTP transports
SHOULD preserve the intent.

Text in RFC 822 that appears to contradict the use of
Return-path (or the envelope MAIL FROM address) as the
destination of error messages is overridden by it.

In particular,

(i)  a gateway from SMTP->elsewhere SHOULD insert a return-path
    header, unless it is known that the "elsewhere" transport
    also uses Internet domain addresses and maintains the
    envelope sender address separately.

(ii)  a gateway from elsewhere->SMTP SHOULD delete any
    return-path header present in the message, and either copy
    that information to the SMTP envelope or combine it with
    information present in the envelope of the other transport
    system to construct the MAIL FROM part of the SMTP envelope.

2.7. Order of commands.

The text in RFC821 that describes the order in which commands
may be issued is misleading and has led to some confusion.
After adjustment for the SMTP extension mechanism, the
explanation should read: 

    There are restrictions on the order in which these commands may
    be used.

    A session that is to contain mail transactions MUST first be
    initialized by the use of the HELO or EHLO command although
    an SMTP server SHOULD accept commands for non-mail
    transactions (e.g., VRFY or EXPN) without this
    initialization.  HELO or EHLO commands MAY also be issued by 
    a client later in the session.   If they are issued later,
    the SMTP server MUST clear all buffers and state, i.e.,
    the sequence of RSET followed immediately by HELO is
    redundant, but not harmful (except, obviously, to
    performance). 

    If the HELO or EHLO commands are not acceptable to the
    server-SMTP, 501, 500, or 502 failure replies MUST be 
    returned as appropriate.  The server-SMTP must stay in the
    same state after transmitting these replies as it was in
    before the HELO or EHLO were received.

    RFC 1123 contains a discussion of arguments to HELO and 
    conditions under which the HELO command can be rejected.  In
    particular, HELO MUST NOT be rejected because the client's
    putative name does not match some criteria established by
    the server (e.g., verification of reverse DNS mapping).

    The NOOP, HELP, EXPN, VRFY, and RSET commands can be used at 
    any time during a session, or without previously initializing a 
    session.  SMTP servers SHOULD process these normally (i.e., not
    return a 503 code)  even if no HELO or EHLO command has yet
    been received; clients SHOULD open a session with HELO or
    EHLO before sending these commands. 

    If the above rules are followed, the example in RFC821 that
    shows "550 access denied to you" in response to an EXPN
    command is essentially meaningless unless a HELO or EHLO
    command preceeds the EXPN. 

    The MAIL, SEND, SOML, or SAML commands begin a mail
    transaction.  Once started a mail transaction consists of
    one of the transaction beginning commands, one or more RCPT
    commands, and a DATA command, in that order.  A mail
    transaction may be aborted by the RSET command.  There may
    be zero or more transactions in a session.

    If the transaction beginning command argument is not
    acceptable a 501 failure reply MUST be returned and the
    SMTP server must stay in the same state.  If the commands
    in a transaction are out of order to the degree that they
    cannot be processed by the server a 503 failure reply MUST
    be returned and the SMTP server must stay in the same
    state.

    The last command in a session must be the QUIT command.  The
    QUIT command can not be used at any other time in a session,
    but may be used by the client SMTP to request
    connection-closing even if no session-opening command has
    been sent and accepted. 

    Reference: RFC821, top of page 27, down to the beginning of
    4.1.2. 

2.8. Private-use commands

    Commands starting in "X" may be used by bilateral agreement
    between the client (sending) and server (receiving) SMTPs.
    An SMTP server that does not recognize such a command is
    expected to reply with "500 Command not recognized".   An
    extended SMTP server MAY list the feature names associated
    with these private commands in the response to the EHLO
    command. 

    As specified in the SMTP Service Extensions RFC [RFC-1651],
    commands sent or accepted by SMTP systems that do not start
    with "X" MUST be documented in published RFCs and be at
    least candidates for standardization. 


2.9. List of commands by category.

   The first line of page 29 of RFC821 should read: "...are the
   SMTP and Extended SMTP commands: EHLO".


2.10. Form of mailbox local-parts.

   For maximum interoperability, a host that expects to receive
   mail SHOULD avoid defining mailboxes where the <local-part>
   contains (or requires) either a quoted-string or backslash
   quoting characters. 

   Systems MUST NOT define mailboxes in such a way as to require
   the use of non-ASCII characters (octets with the high order
   bit set to one).   These characters MUST NOT be used in MAIL
   FROM or RCPT TO commands or other commands that require
   mailbox names.

   Reference: RFC821, page 31.

2.11.  Format and interpretation of date fields

2.11.1   Four digit years

   RFC1123 expands the definition of "<yy>" in dates to
   permit and encourage 4-digit years.  SMTP servers SHOULD use
   four-digit years whenever dates are specified.  SMTP servers
   MUST NOT reject mail on the basis of its containing
   four-digit years.

   Reference: RFC821, page 32; RFC 1123.

2.11.2  Time zones

   The use of the term "default" in the RFC 821 definition does
   not imply that zone information is optional.  A time zone
   designator MUST be supplied.

   As the Internet grows, comparability of Received fields is
   important for detecting problems, especially slow relays.
   SMTP servers that create Received fields SHOULD use explicit
   offsets (e.g., -0800), rather than names of any type.  If it
   is desired to also use a time zone name, it should be
   included in a commment.


2.11.3 Time semantics

   The Date specified in the RFC 822 header is normally presumed
   to be the date and time at which the user completed
   composition of the message.  As such, it SHOULD be supplied by
   the message composing system.  An SMTP server MUST NOT alter
   this date if it appears in a message received by it for 
   transmission or relaying.  

   In environments in which mail composition agents queue mail
   locally before transmitting it to a sending SMTP server (or
   the other component of a split-MUA arrangement), the RFC 822
   date SHOULD reflect the time at which message composition was 
   completed, not the time of transfer to the MTA.

   In environments in which Mail User Agent/ Mail Transfer Agent
   distinctions are made, SMTP servers normally SHOULD insert a
   Received field to reflect the receipt of the message for
   transmission from the MUA.


2.12.  Domains

RFC-821 and RFC-822 were developed before the domain name system
had fully stabilized.   The domain syntax for both (using
RFC-822's syntax) is:
       domain-ref = sub-domain *("." sub-domain)

       sub-domain = let-dig *(ldh-str)
       domain-literal = "[" IP-address-literal "]"
       IP-address-literal = snum 3*("." snum)
       snum = one, two, or three digits representing a decimal
           integer value in the range 0 through 255
       let-dig = Alpha / Digit
       ldh-str = *( Alpha / Digit / "-" ) 1*(let-dig)

(the '#' indicates a comma separated list.)

       Alpha = ASCII character in the range A-Z or a-z.  As specified in
           RFC1035, case is not significant in domain strings.
       Digit = 0 - 9

       Note that the IP-address-literal syntax above specifies four
       or more of these numbers to accomodate IPv6.


2.13. Reply codes.

2.13.1. Reply codes after DATA and the subsequent CRLF.CRLF.

When a SMTP server returns a positive completion status (2yz
code) after the DATA command is completed with CRLF.CRLF, it
accepts responsibity for:

+ delivering the message (if the recipient mailbox exists), or 

+ if attempts to deliver the message fail due to transient
  conditions, retrying delivery some reasonable number of times
  at intervals as specified in RFC 1123, or

+ if attempts to deliver the message fail due to permanent
  conditions, or if repeated attempts to deliver the message
  fail due to transient conditions, returning appropriate
  notification to the sender of the original message (using the
  address in the SMTP MAIL FROM command). 


When a SMTP server returns a transient error completion status
(4yz) code after the DATA command is completed with CRLF.CRLF,
it MUST NOT make any further attempt to deliver that message.
The client-SMTP retains responsibility for delivery of that
message.  The sending user should be able to interpret the
return of a transient or permanent failure status as a
non-delivery indication.


2.13.2.  Reply code 502

Questions have been raised as to when reply code 502 (Command
not implemented) should be returned in preference to other
codes.  502 SHOULD be used when the command is actually
recognized by the SMTP server, but not implemented.   If the
command is not recognized, code 500 SHOULD be returned.
Extended SMTP systems MUST NOT list capabilities in response to
EHLO for which they will return 502 (or 500) replies.


2.14. Generating SMTP commands from RFC822 headers

Some systems use RFC-822 headers (only) in a mail submission
protocol, or otherwise generate smtp commands from RFC 822
headers when such a message is handed to an MTA from a UA.
While the MTA-UA protocol is a private matter, not covered by
any Internet Standard, there are problems with this approach.
For example, there have been repeated problems with proper
handling of "bcc" copies and redistribution lists when
information that conceptually belongs to a mail envelopes is not
separated early in processing from header information (and kept
separate).  

It is recommended that the UA provide its initial MTA with an
envelope separate from the message itself.  However, if the
envelope is not supplied, SMTP commands should be generated as
follows:

(i) each recipient addresses from a TO, CC, or BCC header field
should be copied to a RCPT command (generating multiple message
copies if that is required for queuing or delivery).  This
includes any addresses listed in a rfc822 "group".  Any BCC
fields should then be removed from the headers.  Once this
process is completed, the remaining headers should be checked to
verify that at least one To:, Cc:, or Bcc: header remains.   If
none do, then a bcc: header with no additional information MUST
be inserted.

(ii) the return address in the MAIL command should be derived
from the system's identity for the submitting (local) user.
That return address should also be copied to the Sender header
field if it is different from the address in the From header
field.  (Any Sender field that was already there should be
removed.)  Systems may provide a way for submitters to override
the envelope return address, but may want to restrict its use to
priviliged users.  (This will not prevent mail forgery, but may
lessen its incidence.)

(iii) If a Return-path header is present in the message to be
sent, the MTA MUST reject the message.

A submission protocol based on RFC-822 information
alone MUST NOT be used to gateway a message from a foreign
(non-SMTP) mail system into an SMTP environment.  Attempts to
do so have repeatedly caused mail loops and other behavior
adverse to the proper functioning of the Internet mail
environment.  In gateway situations, the SMTP recipient
addresses (and, in general, the MAIL FROM address) MUST be
derived from the foreign system's envelope or external
information. 

 [[Note in draft: I changed my mind about Keith's suggestion
 here and put it back in, although significantly rephrased.]]


2.15 Other gateway issues.

In general, gateways between the Internet and other mail systems
SHOULD attempt to preserve any layering semantics across the
boundaries between the two mail systems involved.  Gateway-
translation approaches that attempt to take shortcuts by
mapping, e.g., envelope information from one system to the
message headers or body of another have generally proven to be
inadequate in important ways.   Systems translating between
environments that do not support both envelopes and headers and
Internet mail must be written with the understanding that some
information loss is almost inevitable.


2.16. Mail forging and "spoofing".

As knowledge of Internet mail increases, so does the knowledge
that SMTP mail inherently cannot be authenticated, or integrity
checks provided, at the transport level.  Real security lies only in
end-to-end methods involving the message bodies.

A corollary to this is that efforts to make it more difficult
for users to set envelope MAIL FROM and header "From" fields to
point to valid addresses other than their own are largely
misguided: they do not prevent any would-be mail spoofer from
doing so, and do frustrate legitimate applications in which mail
is sent by one user on behalf of another or in which error
replies should be directed to a special address.   On the other
hand, systems that provide convenient ways for users to alter
these fields on a per-message basis should attempt to establish
a primary and permanent mailbox address for the user so that
Sender fields can be generated correctly.


2.17.  Relays, forwarding, resending, and header interpretation

RFC-822 is not specific about the semantics of "resent-" fields,
and additional issues about them are addressed in the companion
document to this one on mail bodies.  However, there are
important issues for MTAs engaged in relays and forwarding as
well.  

2.17.1. Address rewrites.

A mail forwarder that is rewriting a message address that
belongs to a single user (e.g., instantiating a single alias)
MAY add resent- fields to identify that activity and the new
recipients.  If it does not, it SHOULD add a Received header to
reflect what it has done (in addition to, or as part of, the
Received header that it MUST add when it handles the message).
In particular, an SMTP server that returns a 251 response code
SHOULD add Resent- fields, a Received header that specifically
identifies the action taken, or both.

2.17.2.  Fields to be added.

When Resent- fields are added to a message, at least one
Resent-to or Resent-cc field, together with a Resent-date,
SHOULD be provided.   Resent-From MAY be provided and maybe be
useful if the information isn't obvious from context.

2.17.3  Valid messages.

In general, SMTP servers SHOULD avoid rejecting (bouncing)
messages because they believe the format of the bodies of those
messages are incorrect.  They MUST NOT reject messages on the
basis of trying to match numbers of Resent- fields. 


2.18.  "Blind" copies.

Addresses may appear in the RCPT TO commands to an SMTP server
that do not appear in the message headers for a number of
reasons.  The two most common of these involve the use of a
mailing address as a "list exploder" -- a single address that
resolves into multiple addresses -- and the appearance of "blind
copies".  In order to avoid defeating some of the purpose of
these mechanisms, SMTP clients and servers SHOULD NOT copy the
RCPT TO command arguments into the headers, even as
informational or private-extension headers.  Since this rule is 
often violated in practice, and cannot be enforced, sending SMTP
systems that are aware of "bcc" use MAY find it helpful to send
each blind copy as a separate message transaction containing
only a single RCPT TO command.

More generally, while there are often similarities, there is no
inherent relationship between either "reverse" (MAIL FROM, SAML
FROM, etc.) or "forward" (RCPT TO) addresses in the SMTP
transaction ("envelope") and the addresses in the headers.
Receiving systems SHOULD NOT attempt to deduce such
relationships and use them to alter the headers of the message
for delivery.  The popular "Apparently-to" header is a violation
of this principle and SHOULD NOT be used.


3.  Security Considerations

SMTP mail is inherently insecure in that it is feasible for even
fairly casual users to negotiate directly with receiving and
relaying SMTP servers and create messages that will trick a
naive recipient into believing that they came from somewhere
else.   Constructing such a message so that the "spoofed"
behavior cannot be detected by an expert is somewhat more
difficult, but not sufficiently so as to be a deterrent to
someone who is determined and knowledgeable.

This specification does not address the security issues
associated with SMTP other than to advocate that useful
functionality not be disabled in the hope of providing some
small margin of protection against an ignorant user who is
trying to fake mail.

4. References

[RFC-821]  J. Postel, "Simple Mail Transfer Protocol",
            08/01/1982 

[RFC-1123] R. Braden, "Requirements for Internet hosts -
            application and support", 10/01/1989 

[RFC-1651] J. Klensin, N. Freed, M. Rose, E. Stefferud, D.
            Crocker, "SMTP Service Extensions", 07/18/1994.


5. Authors' Addresses

John C. Klensin
MCI Data and Information Services
2100 Reston Parkway
Reston, VA 22091
USA
  Email: Klensin@mail1.reston.mci.net
  Phone: +1 703 715 7361
  Fax:   +1 703 715 7436

Ned Freed
Innosoft International, Inc.
1050 East Garvey Avenue South
West Covina, CA 91790
USA
  Email: ned@innosoft.com
  Phone: +1 818 919 3600
  Fax: +1 818 919 3614

Jeroen Houttuin
RARE Secretariat
   Singel 466-468
   NL-1017 AW Amsterdam
   Europe
   
   Tel +31 20 6391131
   Fax +31 20 6393289
   Email:   X.400 /S=houttuin/O=rare/PRMD=surf/ADMD=400net/C=nl/
            RFC 822     houttuin@rare.nl


Keith Moore
Computer Science Dept.
University of Tennessee
107 Ayres Hall
Knoxville, TN 37996-1301
USA
   EMail: moore@cs.utk.edu


6. Acknowledgements

The authors, who are co-initiators of this project, would like
to express appreciation to Dave Crocker and Marshall Rose for
comments on an intermediate draft, to participants in the
header-people mailing list (and the former comp.mail.headers
newsgroup) who raised some of the issues discussed in this
specification and helped work out a preliminary consensus on
others, and to the Bob Braden as Editor and the contributors to
RFC 1123 which laid the foundations for the present work.


