#ident	"@(#)smail:ToDo,v 1.103 2004/08/27 06:56:20 woods Exp"


Things that should be done before the next minor release (patches are,
of course, gratefully accepted!):

Important Bugs:
--------------------

- "MAIL TO:<postmaster>" seems to go to root, bypassing the smart_host
  router -- i.e. it should be canonicalized just as command-line params
  are.

- try to make sure check_sender() validates local addresses.

- do GC on xprintf() values passed to send_smtp_msg() in smtprecv.c

- there is some rare crash happening in smtpd after a HELO questionable operand

- maybe sometimes matching dead_bounce_senders from the dead-mail.egrep
  file don't make it into dead-mail.senders?

- RCS the dead-mail.senders list, if RCS commands can be found....

- get rid of/update more of the old NOTES/* files

- apparently we can get a NULL when the mail domain has a CNAME but no
  MX for the target:

	01/23/2004 08:33:22: [17724] DNS MX error: MX domain '(null)' has a target host 'extenda.net' which is an invalid CNAME (for 'ds2.domainspa.com')

	$ host -t mx  extenda.net 
	 *** extenda.net has no MX record (Authoritative answer)

	 $ host -a  extenda.net    
	 extenda.net             NS      dns1.name-services.com
	 extenda.net             NS      dns2.name-services.com
	 extenda.net             NS      dns3.name-services.com
	 extenda.net             NS      dns4.name-services.com
	 extenda.net             NS      dns5.name-services.com
	 extenda.net             SOA     dns1.name-services.com info.name-services.com (
	                        2002050701      ;serial number (version)
	                        3600    ;slave refresh period (1 hour)
	                        120     ;slave retry interval (2 minutes)
	                        86400   ;slave expire time (1 day)
	                        3600    ;negative response TTL (1 hour)
	                        )
	extenda.net             CNAME   ds2.domainspa.com

	$ host -t mx ds2.domainspa.com
	 *** ds2.domainspa.com has no MX record (Authoritative answer)

  this should of course be impossible in the first place (CNAME with
  other data), but some nameservers are just too stupid for words

  (dunno if this happens when the CNAME is valid, but still points to a
  name with no MX -- that should cause 974 processing, and if there's
  also no A RR then that's a hard error)

- use new flags, and maybe a new field or two, to mark when the retry
  file error and most recent msglog defer error have been added for a
  given address.  Otherwise we get two copies of the previous
  message, as below:

	|------------------------- Failed addresses follow: ---------------------|
	 address: hostmaster@[213.248.29.9] failed:
		transport inet_zone_bind_smtp: connect(mail.domainreservationsite.com:smtp): Connection timed out
		Previous retry error:  transport inet_zone_bind_smtp: connect(mail.domainreservationsite.com:smtp): Connection timed out
		Retry duration (1day) has been exceeded.
		No further delivery attempts will be made.
		Retry duration (1day) has been exceeded.
		No further delivery attempts will be made.

	|------------------------- Transaction log summary follows: -------------|
	defer: <hostmaster@domainreservationsite.com> reason: (ERR151) temporary failure from inet_zone_bind_smtp transport:
	454 <hostmaster@domainreservationsite.com>: Relay access denied
	[01/16/2004 14:14:34: [18044]]
	defer: <hostmaster@[213.248.29.9]> reason: (ERR148) transport inet_zone_smtp: connect([213.248.29.9]:smtp): Connection refused
	[01/16/2004 11:15:10: [7171]]
	defer: <hostmaster@mail1.domainreservationsite.com> reason: (ERR151) temporary failure from inet_zone_smtp transport:
	454 <hostmaster@mail.domainreservationsite.com>: Relay access denied
	[01/16/2004 14:14:34: [18044]]
	fail: <hostmaster@[213.248.29.9]> reason: (ERR148) transport inet_zone_bind_smtp: connect(mail.domainreservationsite.com:smtp): Connection timed out
		Previous retry error:  transport inet_zone_bind_smtp: connect(mail.domainreservationsite.com:smtp): Connection timed out
		Retry duration (1day) has been exceeded.
		No further delivery attempts will be made.
		Retry duration (1day) has been exceeded.
		No further delivery attempts will be made.
	[01/16/2004 14:14:30: [18044]]

  Note also the problem with retry files for domain literals "merging"
  with retry files for the domain itself.

  also there's a chance the retry duration is never reported:

	|------------------------- Failed addresses follow: ---------------------|
	 address 'postmaster@e-mailanywhere.com' failed:
		remote storage full error reported by inet_zone_bind_smtp transport:
	452 4.2.2 Message would exceed quota for <postmaster@e-mailanywhere.com>
	
	 address 'postmaster@[149.99.7.10]' failed:
		remote storage full error reported by inet_zone_smtp transport:
	452 4.2.2 Message would exceed quota for <postmaster@[149.99.7.10]>
	
	|------------------------- Transaction log summary follows: -------------|
	defer: <postmaster@e-mailanywhere.com> reason: (ERR153) remote storage full error reported by inet_zone_bind_smtp transport:
	452 4.2.2 Message would exceed quota for <postmaster@e-mailanywhere.com>
	[02/09/2004 14:34:38: [5456]]
	defer: <postmaster@[149.99.7.10]> reason: (ERR153) remote storage full error reported by inet_zone_smtp transport:
	452 4.2.2 Message would exceed quota for <postmaster@[149.99.7.10]>
	[02/09/2004 14:34:38: [5456]]
	fail: <administrator@businetcomm.com> reason: (ERR156) inet_zone_bind_smtp transport reports unknown user:
	550 5.1.1 <administrator@businetcomm.com> is not a valid mailbox
	[02/04/2004 15:18:40: [8113]]
	succeed: <hostmaster@e-mailanywhere.com>
	[02/04/2004 15:18:42: [8113]]
	succeed: <hostmaster@[149.99.7.10]>
	[02/04/2004 15:18:44: [8113]]
	succeed: <hostmaster@smtp.e-mailanywhere.com>
	[02/04/2004 15:18:46: [8113]]
	succeed: <postmaster@smtp.e-mailanywhere.com>
	[02/04/2004 15:18:46: [8113]]
	sent_error: <administrator@businetcomm.com>
	[02/04/2004 15:18:47: [8113]]
	fail: <postmaster@e-mailanywhere.com> reason: (ERR153) remote storage full error reported by inet_zone_bind_smtp transport:
	452 4.2.2 Message would exceed quota for <postmaster@e-mailanywhere.com>
	[02/09/2004 15:04:57: [6726]]
	fail: <postmaster@[149.99.7.10]> reason: (ERR153) remote storage full error reported by inet_zone_smtp transport:
	452 4.2.2 Message would exceed quota for <postmaster@[149.99.7.10]>
	[02/09/2004 15:05:00: [6726]]
	

- what about retry-file errors for addresses that go to the same
  transport and next_host?  shouldn't they all have the same text
  merged/recorded?

- a double-bounce from the error director causes apparently spurious
  "error writing errfile" and "error linking" log and panic entries:

	12/12/2003 17:14:34: [16360] [m1AUr73-0002TBC] Failed TO:":fail:\"This user is currently over-quota.\"" ORIG-TO:amanda@proven.weird.com DIRECTOR:error ERROR:(ERR193) address amanda@proven.weird.com failed: This user is currently over-quota.
	12/12/2003 17:14:34: [16362] [m1AUvYQ-0003O2C] Received FROM:<+> HOST:proven.weird.com PROTOCOL:bsmtp PROGRAM:smail SIZE:2324
	12/12/2003 17:14:34: [16362] [m1AUvYQ-0003O2C] remote got QUIT from <no-ident>@proven.weird.com
	12/12/2003 17:14:34: [16363] [m1AUvYQ-0003O2C] Failed TO:":fail:\"This user is currently over-quota.\"" ORIG-TO:amanda@proven.weird.com DIRECTOR:error ERROR:(ERR193) address amanda@proven.weird.com failed: This user is currently over-quota.
	12/12/2003 17:14:34: [16363] [m1AUvYQ-0003O2C] notify: freezing a double-bounce message.
	12/12/2003 17:14:34: [16363] [m1AUvYQ-0003O2C] error writing errfile[2]: Bad file descriptor
	12/12/2003 17:14:34: [16363] [m1AUvYQ-0003O2C] freeze_message: error linking /var/spool/smail/input/1AUvYQ-0003O2C to /var/spool/smail/error/1AUvYQ-0003O2C: No such file or directory
	12/12/2003 17:14:34: [16360] [m1AUr73-0002TBC] Returned error FOR:amanda@proven.weird.com TO:amanda
	12/12/2003 17:14:34: [16360] [m1AUr73-0002TBC] Completed.

  the message is in fact linked into the error queue....

  This is probably because there are two calls to freeze_message(), the
  second in panic(), the panic for "error writing errfile", but how did
  errfile get closed without being set to NULL?


- smtp pipelining causes the wrong error to be displayed with '-v1', but
  luckily the right error message is logged:

        delivery FAILED: fatal error from inet_zone_bind_smtp transport:\n503-5.5.1 'DATA' command must be preceded by 'RCPT TO:' command.\n503-5.5.1\n503-5.5.1 If you are seeing this message in a bounce, or in an alert box\n503-5.5.1 from your mailer client, etc., then your mailer software\n503-5.5.1 is not showing you the correct and most meaningful error message.\n503 5.5.1 Please report this error to those responsible for your mailer software.
    unlocking retry/smtp/205.207.148.251 and unlinking.
    write_log(SYS): Failed TO:com1324@aci.on.ca ORIG-TO:jschemmer@naturalwhite.com ROUTER:bind_hosts TRANSPORT:inet_zone_bind_smtp ERROR:(ERR152) fatal error from inet_zone_bind_smtp transport:\n559-5.0.0 Attempts to send to the address '<com1324@aci.on.ca>' are being rejected.\n559-5.0.0 This address has not been accepted.\n559-5.0.0 Reason given: (ERR193) address '<com1324@aci.on.ca>' failed: This mailbox has exceeded its allotted storage quota.\n559 5.0.0 A permanent failure has been logged.


- we need a ${for: operator for the "for" subfield of recieved_field and
  other places where address lists should be iterated over.

- add "delivered-to:" and "x-original-to:" headers ala Postfix.  Can
  these both be done with an "append_header" in the local transports?
  Is that the right way to implement them?

	append_header="X-Original-To: ${top:input_addr}"

  NOTE:  postfix apparently prepends this header.....

  one would have to "remove_header=delivered-to", but doing it through
  this mechanism wouldn't allow for loop detection, which is the whole
  purpose of "delivered-to" in the first place.

	remove_header="delivered-to",
	append_header="Delivered-To: ${input_addr}"

- separate out the use of quote() into quote_value() and quote_addr(),
  the latter expecting in_addr and producing RFC 821 quoted addresses.

- invent "%a" for str_printf() for producing RFC 821 addresses.

- consider moving defer_delivery, fail_delivery, succeed_delivery, and
  error_delivery to log.c since they write log messages.

- why is "smail:" being added to paniclog entries, especially during a
  normal queue run?  It can not happen unless only_testing is set and
  that's only set in certain command-line driven operations!

- should we deprecate "localnet" support?  it's not "CIDR" compatible
  and is just too dangerous to use safely in most environments these
  days of widely used CIDR-ized class-A networks unless we can also get
  the netmask of the receiving interface.

- reconcile the differences between the interpretation of SMTP response
  codes in Smail and the way they're used by Smail:

	 address: postmaster@e3m.com failed:
		inet_zone_bind_smtp transport reports user over quota:
	552 5.1.3 reject sending to address '<postmaster@e3m.com>'.  That addressing form is not permitted.

	 address: hostmaster@mail.e3m.com failed:
		smtp transport reports user not local:
	551-5.1.2 reject sending to address '<hostmaster@relay.e3m.com>'.
	551-5.1.2 The address <hostmaster@relay.e3m.com> was not accepted.
	551-5.1.2 Reason given was: (ERR_168) relay.e3m.com is not a valid domain.
	551 5.1.2 Permanent failure logged.

  while doing that it might be good to look over RFC 3463 "Enhanced Mail
  System Status Codes".

- continue to make sure all foreign data is safely formatted wherever
  it's used:

  + is str_c_quote() sufficient to avoid violating SMTP message response
    rules and RFC 822 header text rules?

- with '-v -bv' directors should probably note the transport they'll
  use, just as routers do....

- try to find some way to print the transport hint and address hint(s)
  when the inet_hosts router is used instead of the bind_hosts:

	$ smail -v -bv postmaster@symatico.ca
	postmaster@symatico.ca: symatico.ca matched by inet_hosts router:
	    routed postmaster@symatico.ca --> postmaster@symatico.ca at symatico.ca
	    transport is smtp
	postmaster@symatico.ca at symatico.ca ... deliverable

	$ smail -v2 -bv postmaster@symatico.ca
	bindlib: DNS lookup for 'symatico.ca' returned NO_DATA: No address associated with name.
	full_mx_lookup: target 'symatico.ca' needs null MX.
	handle_null_mxs: no faking for symatico.ca with BIND_MX_ONLY!.
	router inet_hosts: driver gethostbyname: 'symatico.ca' is reachable via hostname 'symatico.ca'
	postmaster@symatico.ca: symatico.ca matched by inet_hosts router:
	    routed postmaster@symatico.ca --> postmaster@symatico.ca at symatico.ca
	    transport is smtp
	postmaster@symatico.ca at symatico.ca ... deliverable

- DO NOT RUN SMAIL AS ROOT!!!!

  + the daemon will setuid(nobody:smail) and re-exec itself after it has
    a file descriptor already bound to port 25 (it will have to be
    started by root on most systems, of course)

  + the queues will be writable by the group 'smail' (not "mail") and
    owned (and also writable) by the user 'smail'.

  + the main smail/sendmail binary will be setgid-smail (not 'mail') so
    that it can write to the queues (but owned by root, of course).

  + checkerr will be run as smail:smail

  + local delivery to spool files will be done only by a separate
    setgid-'mail' agent ala *BSD mail.local, with kernel locking where
    possible, *.lock files only where absolutely necessary.

  + initial local spool file creation, if necessary, will be done by a
    tiny setuid-root helper on systems that have a root-only chown(2)
    [one exists somewhere already -- search the net]; they will be owned
    by the user, group 'mail', and be mode 660.  The spool directory
    will be mode 555 if kernel locking is possible, else 575 & group
    'mail' if necessary for *.lock files (and in all cases owned by
    root, of course).  the helper will _only_ create an empty spool file
    if one does not already exist, and it will determine the pathname to
    use based on the user name given to it on its command line.

  + all mail readers will be expected to either use a small helper to
    safely copy the spool file to a private place (ala movemail, which
    will use kernel file locking when possible or be setgid-mail and use
    *.lock files if necessary), or to use kernel file locking to access
    the spool file (and hopefully copy it away to a private place while
    th user does his/her thing); mail readers (including movemail) will
    be "encouraged" to keep the zero-length spool file after emptying
    it; and use of setgid-mail for readers will be very Very VERY
    strongly warned against (anything more complex than the old V7
    /bin/mail has had many bugs in this regard for decades now).

  + .forward files will have to be world readable (or at least readable
    by the group 'smail'), *or* "Forward to" support can be used.

  + delivery to files and pipes done either as nobody:nogroup or
    nobody:smail regardless of where they're expanded from or who "owns"
    the address.  Anyone wanting more will be advised to deliver to an
    intermediate lock-protected spool and to run a collector
    daemon/periodic-job as with the ultimately desired identity, though
    nothing will prevent the delivery to a pipe from execing a setgid
    (or setuid) binary and about the only time that would be sensible is
    for something like Cyrus-IMAPd where the setgid-cyrus "deliver"
    wrapper would be protected in a directory accessible only by the
    "smail" group (and of course it would verify that it was invoked as
    "nobody:smail" before exec'ing "deliver").

  + access to locked files will only be attempted for a limited amount
    of time and messages will be left in the queue if delivery is
    unsuccessful because of the presence of any pre-existing a lock

- think about always truncating the bodies of bounces leaving only the
  message headers, or perhaps add a new option $truncate_bounce_size
  or similar with default of 10KB.

- deal with un-qualified local hostnames when there's no qualify file in
  some more sane way.... [is this still necessary now that the
  command-line parameters are turned into a "field" for processing?]

- fix "from_field" to never allow "From:" to go missing and if it's nil
  do something appropriate....

- flesh out the test_headers code in main.c

- use ftruncate() to remove partially written messages in appendfile.c
  if ERR_135 [if possible].

- investigate smail vs. MH and BCC/DCC.  Note also that MH uses 'Dcc'
  instead of 'Bcc' for normal (direct) blind carbon and that this header
  may not be stripped when '-t' is used!

- Apparently an address from the BCC header can end up in the initial
  Received header sent to a non-BCC'ed recipient.  The exact address
  that appears in the first Received header will vary if there are
  multiple destination addresses.

- fix aliasfile parser to allow case sensitive aliases.  [also keep in
  mind the lists director also uses "lists/${lc:user}"]

- fix expand_string() et al to always return newly allocated storage,
  and then make sure that storage is freed when it's done with.

- turn down the verbose logging of failed locks, if known other smail
  process holds lock....  eg:

	02/28/96 12:07:36: open_spool: /local/var/spool/smail/input/0trpIB-00076nC: lock failed: Permission denied

  Unfortunately this will probably require re-writing the spool locking
  functions to use pid-in-a-lock-file mechanisms.  [effectively fixed in
  3.2.1 for systems that return EAGAIN if lock_fd() meets another lock?]

- the logic in notify() (and error_delivery()) is far too hairy.
  classify_addr() seems to be more on the right track, but it's still a
  bit wonky.  For example there is no "Returned" log entry written when
  an error is sent for an alias that has an owner -- only the owner
  entry is noted (Error sent) entry is written.  [first write a spec on
  how address structures get linked together, then make sure the rest of
  the code adheres to the spec!]

- after "lost connection unexpectedly from" there's no EX_NOINPUT exit
  status message from the parent daemon....  [has this been fixed by the
  setting of errno=EIO in the same context?]

- fix expand_string_to() to ignore a backslash-escaped '$', but to
  remove the backslash....



Incomplete Features:
--------------------

- dead-mail.egrep support needs to be re-written to use the same
  specification and matching mechanisms as body_checks, and by default
  maybe body_checks and body_checks_always should be used (in addition
  to?) the dead-mail.egrep list.  Too bad pcregrep(1) doesn't have an
  option for doing whole-file matching and for taking a list of REs in
  the "/RE/OPTIONS" format....

- implement ":redirect:" for the error director to return a proper
  SMTP-level redirect response message.

- consider trying to detect input on the SMTP connection which is sent
  before the 220 greeting goes out (apparently Exim can do this and it
  drops the connection with a "protocol violation: synchronization error
  (input sent without waiting for greeting): rejected connection from
  ..." error being logged).

- consider implementing whole_header_checks{_always} so that each RE
  matches the whole header portion just as with body_checks, which would
  allow much more exacting patterns matching multiple fields to be used.

- think about an option, or another set of *_header_checks_* vars, that
  would match the canonical header entries, i.e. un-folded headers.

- think about how to pre-compile other RE lists.

- implement line-wrapping in format_pcre_list().

- write an expression pretty-printer for the likes of received_field.

- may need more testing of comment handling in bsearch and lsearch alias
  files, though so far so good!

- smtp_bad_mx_targets should match "faked" MXes too!

- don't even bother logging protocol violations such as this one if the
  sender used EHLO since they'll almost certainly be due to pipelining:

	12/22/2003 05:27:54: [26183] sent 503-5.5.1: ''DATA' command must be preceded by 'RCPT TO:' command (if not using ESMTP PIPELINING).' to benares.eva.ac.at source [143.130.18.130]

  [partly fixed?]

- pd/Makefile should check if the pathalias driver is included before
  diving into the pd/pathalias directory.

- consider making all file readers check that the file's last-modified
  time hasn't changed while the file was being read so as to prevent
  problems when people use software which modifies files in place
  instead of safely writing all changes to a new file and then using
  rename().

- implement an outbound ACL to prevent delivery to destinations listed
  in either smtp_output_reject_hostnames, smtp_output_reject_ipaddr,
  smtp_output_reject_dnsbl (addrs), or smtp_output_reject_rhsbl
  (hostnames).  Maybe these can be checked during verify too so that
  they'll even fail at RCPT TO:  time, though that would then require
  that at least the domain/host forms be excepted if they are
  "islocalhost()".  We may need smtp_output_reject_except_{host,ipaddr}
  lists too/anyway.  This will allow the like of inputs.relays.osirusoft
  to be used to block even command-line users from sending to a known
  open relay.

- implement a client-source-address based ACL to allow restricted EXPN
  usage (smtp_expn_allow?).

- implement more generic ACLs for at least directors (and maybe routers,
  and what about transports too?).  E.g. this could be used to only
  allow certain alias files, or lists, or the magic Cyrus noquota
  director, to be used only if the message originated from the local
  host, a local sender, or from some other trusted client address.

- implement smtp transport driver attributes to allow the sending_name
  (and thus the greeting name) to be specified uniquely for a given
  transport (overriding the global setting, if any).

- make sure RFC-[2]821 local-part quoting isn't too aggressive and
  re-quoting things that don't need re-quoting.

- check to be sure we use IANA registered protocol names, etc. in
  received headers and such places.

- don't allow bogus A RR's to "match" (0, 255.255.255.255, 127/8,
  RFC-1918 addresses etc.).  Probably need to provide a config variable
  that contains a list of "bogus" addresses, something like
  smtp_bad_mx_targets but more general for all A RRs.  Perhaps names
  given in HELO would skip this check if the client address matches in
  smtp_remote_allow or something.  What about names for internal MX
  hosts though?  How do we know if they're "internal"?

- When command-line recipients are given, but there's no "To:" header,
  add one like "To: undisclosed-recipients:;", just as Postfix does.

- don't generate a "From:" header that appears to be forged if the
  sender address is null.  Perhaps use the remote SMTP client name if
  there is one available, otherwise at least include a comment about it
  being some remote software which screwed up.

- think about making mailq's "-E" work for runq too.  [unfreezemail
  effectively does this though]

- add an option to 'mailq -E' that'll only match if the undelivered
  addresses are local or not to make it easier to find and unfreeze
  large batches of messages which might now be locally deliverable (this
  could be especially useful for Cyrus IMAP sites where many messages
  get frozen because of quota restrictions).  [this is partly done now
  in a hackish way by the tempfail retry logic in util/checkerr]

- implement client name loop detection [compare HELO name with what we
  give in the 220 startup message, and vice versa] (not that it ever
  seems to be needed, but perhaps some idiot will mis-configure things
  so badly that such a simple check will save their necks).

- modify the host retry locking mechanism to allow for multiple
  concurrent deliveries to a given target host.  Perhaps each sender
  locks the retry file only temporarily, not for the whole duration of
  its delivery attempt, and if the file is new (or empty?) or contains
  only fewer than the concurrency limit of "pid N" lines then it appends
  its own "pid N" line to the end of the file, unlocks it and goes on.
  If the file exists but contains an error number and message then it
  does the normal retry duration timeout first.  The concurrency limit
  should be (optionally) specified in the retry configuration so that
  different destinations can be given different concurrency limits.

- the retry file should allow hostnames for tcpsmtp driver targets,
  though this might require the PTR to be looked up for every address
  being tried (or maybe if there are no matching IP#s then we fall back
  to looking up the target name from which we resolved the IP we're
  about to try connection to? -- problem is this doesn't match the retry
  file name and we may have an API layering problem getting that name)

- the retry file should allow for CIDR subnets.  Currently the 'tcpsmtp'
  transport driver calls retry_host_lock() with a hostname that is
  concocted by converting the target IP address to a string with
  inet_ntoa().  Perhaps we could implement this by testing the hostname
  to see if it can be converted by inet_aton(), and if so then in
  match_retry_domain() instead of looking through the retry table with
  is_suffix(), use match_ip_net() instead.

- maybe the retry file should also allow for hostname REs too?  If so
  then are they all REs (how do we maintain the leading dot
  compatability?) or should we just use the quoting trick ala aliases?

- Normally we don't want to restrict users who are not using our mail
  server to relay their outgoing messages but rather only to receive
  (and probably re-route remotely again) their incoming messages
  (i.e. virtual domain users).  Such users will not usually be using
  clients listed in smtp_remote_allow.  For finer grained
  smtp_local_sender_restrict control we should be able to tell the
  difference between an address routed via $hostnames or $more_hostnames
  and one routed via some outside router like one using the 'rewrite'
  driver.  Maybe we need a full list of domains for which we do
  anti-spoof checks.  In the mean time if you host virtual domains like
  this then you'd best not enable smtp_local_sender_restrict.

- try to include the "ORIG-TO:" field in the logs when a message bounces
  (i.e. in the "Failed" log entry) -- otherwise it's almost impossible
  to see what the input address was.

- think about how to include the "ORIG-TO:" field from the "Delivered"
  log entry in the received header -- for SMTP this should be the
  original envelope recipient address for this particular delivery (what
  do we do when a delivery has collapsed multiple addresses when
  avoiding duplicate delivery?  It looks like only the "first" will
  appear in the log, whatever "first" means....)

- properly fix all the other director drivers to have RE-capable prefix
  and suffix attributes (like what was done recently for the user driver)

- add a "cmd" attribute to the smartuser driver (and a corresponding
  smart_user_cmd config variable) such that dynamic lookups can be
  done (e.g. forwarding address verification to an internal server).

- think about adding a verify_command to the pipe transport driver so
  that VRFY and RCPT TO: commands, as well as '-bv', can actually check
  if a delivery will succeed.  This is primarily most useful when local
  delivery is done via a pipe driver and where the delivery agent has
  some easy way to report if delivery might fail due to quota violations
  or other problems (eg. with Cyrus IMAP).  Note the "error" director in
  combination with the new "cmd" attribute for the smartuser driver
  driver may eliminate some need for this feature since lookups may now
  expand to the ":error:message text" form that this director will
  match.  E.g. perhaps a smartuser director could be configured with:

	smart_user_cmd="|/$lib_dir/cyrchkquota ${shquote:user}"

  and cyrchkquota would interactively check the current mailbox usage
  for $user and either return $user again, or return a form matched by
  the error director, e.g. ":error:$user is over quota!"  (note this
  would impose a process invocation on every "RCPT TO:" or similar
  verification activity)

- think about inventing a new router driver that is like the
  queryprogram director driver but also is able to rewrite the
  local-part of an address as well -- it would be just like the rewrite
  router driver, but instead of static file lookups it could do dynamic
  lookups.  It will definitely need to support the a "required" private
  attribute to allow its magic to be more efficiently restricted.

- add new "quota" and "quota_threshold_warn" attributes to the
  appendfile transport driver both of which would be run through
  expand_string() so they could do file lookups.  A non-zero "quota"
  would simply enforce the quota with a failure if file size would
  exceed the value after delivery.  "quota_threshold_warn" would trigger
  delivery of a warning message to the same in_addr that triggered the
  current delivery if the size of the file crosses the given threshold
  after delivery completes.  If "quota" is also set, the threshold may
  be specified as a percentage of "quota' by following the value with a
  percent sign.  The warning message could be configurable with a
  "quota_warn_msg" attribute or similar.  [idea from exim] [see BDB's
  contrib/patch.appendfile-maxsize for the partial implementation]

- this happens sometimes when multi-homed with multiple IP subnets on
  the same segment and connecting to a peer's "alternate" address....

	07/07/1999 17:47:12: [4931] remote EHLO: questionable operand: 'becoming.weird.com': from root@becoming.weird.com source [204.29.161.180]: Remote address PTR lookup failed (Unknown host).

   This would probably be fixed by always greeting with the name
   matching our actual source address [getsockname() and then
   gethostbyaddr() or getnameinfo()], and would re-invent/remove the
   meaning of 'primary_name'.

- think about allowing $listen_name to be set on command line too [if
  this is used for more than one domain then you'll need separate config
  files anyay, so just use -C; but if you are using this to avoid having
  SMTP on some interfaces then this info may be easier to manage in one
  place in the /etc/rc* files or whatever].

- do something to make aliasfile parsing identical across lookup protos.
  (related to 'db lookup parser' bug above?)

- Put the following in default.c for SVR4's local, pipe, & file transports:

	remove_header="Content-Length",
	append_header="${if !header:Content-Type :Content-Type: text}",
	append_header="Content-Length: $body_size",

- think about how to integrate checkerr and savelog so that security
  violations can be snarfed from logfile just after it is cycled.
  Perhaps a new over-all maintenance script (smailmaint?) could do the
  work and there would only be one crontab entry necessary.  Note that
  there's no need to use the antiquated savelog on systems that have a
  newsyslog(1) capable of not compressing the .0 file (eg. my version!).
  [syslog logging would also change all of this since then security
  violations will get higher priority from syslog if the admin so
  desires...]

- add an "always" attribute to the directors drivers, esp. aliasfile.

- add 'senders' and 'senders_except' attributes to directors and routers
  to implement restricted aliases, transports, etc.

- implement lookup drivers/protocols through drivertab.c [already partly
  supported in mkdrivtab.sh]

- think about allowing hostnames in match_ip() by doing a reverse lookup
  on the address and matching the resulting PTR(s) with any hostname
  patterns [regex's too, or just glob(3), or just domain suffixes?].
  Remember to always do the safe thing when no PTR is found --
  i.e. return a code saying that a test was not possible (either
  temporary error indicator if DNS times out, or permanent if
  authoritative NXDOMAIN) and let the calling code can do the "safe"
  thing (eg. reject a relay attempt).

- think about making smtp_remote_allow and other users of match_ip() and
  match_re_list() capable of specifying a file lookup mechanism in a
  list element:

       smtp_remote_allow="localnet:10/8:192.168/16:\
		${lookup:sender_host_addr:ipsearch{
				/etc/smail/remote.allow}:$value}"

  where "ipsearch" iterates the [new] match_ip() function over all the
  values in the file.  (does this mean keeping the double compare?)
  (the file should probably be cached in-core and treated as a list if
  it's not too big).  See next item too about how to specify the
  variable containing the value being searched for instead of magically
  knowing what it is as in the above example.

- think about adding a new magic variable name like "key" that the
  caller of expand_string() can set in a dummy addr structure so that
  expansion of ${lookup in a list-style variable can be done reliably
  without having to know what variable is used for matching (eg. with
  $sender_host_addr in the previous item and $sender in the next one)

- think about fixing parsing of all list-style variables so that they
  can all always optionally include an element that is run through
  expand_string(), in particular so that ${lookup can be used.  The
  first trick here is in making sure there's some way to always specify
  the value being searched for in the list, and making sure each
  variable's definition documents the expected lookup variable(s).  The
  second trick is making sure the expansion results in something useful
  so that the caller makes sense of the lookup result.  For hostname and
  IP# lookups this should be as simple as expanding the the searched-for
  key if the lookup succeeds, or the key prefixed by '!' if not.

  + eg. this would allow collapsing smtp_sender_reject and
    smtp_sender_reject_db into just the former.  In this particular
    example the caller would have to arrange to have the expected lookup
    variable set to the appropriate value:

	smtp_sender_reject:".*@[^@]*\\.localdomain;bogus domain!:\
		${lookup:sender:lsearch{dead-mail.senders}
		then {$sender;$value}
		else {!$sender}"

- think about adding 'DNS' and 'RDNS' db search protocols for ${lookup.

- think about how to get more detailed errors from ${lookup so that
  expand_string() doesn't just end up with an empty value.  This would
  help pass DB_AGAIN and FILE_AGAIN errors out to the caller so that
  messages can be deferred (or failed) just like when a lookup fails in
  a router or director.  Maybe a generic callback such as
  chk_expansion_error() would suffice?

- Think about splitting lsearch and USE_LSEARCH_REGEXCMP into a plain
  old lsearch and a new "re_search" (is this a bad name? ;-) [JPR
  suggests "grep", how how about "grepsearch"?] for straight RE linear
  searches.  We could turn off icase then too since PRCE has "(?i)".
  Think about not using double quotes to trigger the RE match in
  "grepsearch", but rather doing it for every key value.  Think about a
  combined lsearch+grepsearch that would do what lsearch+REGEXCMP does
  now with the double-quote trigger.

- fix the error messages in config file parsing to include at least the
  line number, and anything else helpful, not just:

	05/07/1997 15:40:59: [13914] /etc/smail/config: parse error: unexpected end of attribute

  and not even just:

	01/11/2004 11:27:39: [20640] /etc/smail/config: parse error: unexpected end of quoted string for variable more_hostnames

  The problem is that read_entry() can suck up more than one line at a
  time.  If we had a global (or callback) to record the line number of
  the last line read then at least we'd do as well as GCC and similar.

- the config file syntax needs some way to allow for appending to an
  existing value -- e.g. the default, or the value from the primary
  config when reading the secondary config, etc.

- think about changing the "var" portion of the eq{ et al condition
  operators to be a fully expanded value, not just a variable name
  (which would make the eqic{ et al operators effectively redundant).

- add support for Kiem-Phong Vo <kpv@research.att.com> Vmalloc library,
  particularly debugging support [partly done].  Also add hooks to build
  with sfio (i.e. without the stdio layer).

- document ${eval: if it turns out to be useful anywhere but with -bP.
  [should test to see if it happens to work in the variable part of eq{
  condition operators]

- re-write aliasfile.c in the style of the fwdfile.c with a finish_*()
  function, etc.

- think about allowing multiple recipients at RCPT_CMD time for SMTP
  bounce messages but then denying them at the DATA phase.  This may
  cause a lame sender to retry the bounce ad-infinitum though....

- make the startup log message more verbose (version, build, build date,
  release date, etc.) [use $smtp_banner ???]

- figure out how to do the configuration for per-transport (or
  even per-target?) relaying control.

- pass a flag to fill_attributes() so that it can print a more
  meaningful error message that indicates if an unknown attribute is
  expected to be either a generic attribute, or a driver-specific
  attribute (possibly either the word "generic" or the driver name).

- the error message string returned by parse_header() doesn't indicate
  which header line the problem was with, never mind which specific
  address in the case of address parsing problems.

- implement "queue_run_max" to limit the total number of queue_run child
  processes.

- add configurable reserve space for spooldirs ($min_spooldir_free?)

- be careful about never filling the logfile too (can we instantly defer
  connections if we're out of resources like this?)

- try to ensure all variables are run through expand_string().

- keep a static copy of all default settings, or at least all defaults
  for config attributes, so that a command-line option, or a magic
  parameter to '-bP' can be used to print ot just the changed values.

- for all string type attributes, especially those that are lists, allow
  a magic '$DEFAULT' expansion to include the original default value
  by referencing the static copy of the default setting.

- add a way to supress warnings for smtp_helo_broken_allow.

- Microsoft IDIOTS:

  220 exchange1.ACC.WORKFORCE.COM Microsoft ESMTP MAIL Service, Version: 5.0.2195.1600 ready at  Tue, 6 Feb 2001 13:05:01 -0800 
  EHLO proven.weird.com
  250-exchange1.ACC.WORKFORCE.COM Hello [204.92.254.15]
  250-TURN
  250-ATRN
  250-SIZE
  250-ETRN
  250-PIPELINING
  250-DSN
  250-ENHANCEDSTATUSCODES
  250-8bitmime
  250-BINARYMIME
  250-CHUNKING
  250-VRFY				# it's _NOT_ optional!
  250-X-EXPS GSSAPI NTLM LOGIN
  250-X-EXPS=LOGIN
  250-AUTH GSSAPI NTLM LOGIN
  250-AUTH=LOGIN			# bogus -- it's just "AUTH"
  250-XEXCH50
  250-X-LINK2STATE
  250 OK				# what's this BS!?!?!?
  quit
  221 2.0.0 exchange1.ACC.WORKFORCE.COM Service closing transmission channel

- Maillenium idiots too:

  220 prserv.net - Maillennium ESMTP/MULTIBOX in2 #13
  EHLO proven.weird.com
  250-prserv.net
  250-7BIT				# bogus -- that's the default!
  250-8BITMIME
  250-DSN
  250-HELP
  250-NOOP				# bogus -- this isn't an option!
  250-PIPELINING
  250-SIZE 10485760
  250-VERS V04.50c++			# what the hell?
  250 XMVP 2

- checkerr should maybe try to find the original message-id for double
  bounces and look for related log entries for it too, then we could see
  right in its report the original source of the failing message.

- look for more places where xprintf(), dprintf(), and str_printf()
  could use new '%S' (like %*s in printf(3)) could be used.

- add more to the API defined in list.c/list.h and make more use of it.

- think about how to safely and portably typedef uid_t and gid_t
  [autobuild?]

- consider implementing "HELP command".

- think about some way to support SysVr3.x's broken SIGCLD [really?].

- add new types to smailconf.c:  enums, arrays of int, char*, long, etc.
  (also validate format of arrays of int & long to make sure all values
  are allowable and all ranges are correctly specified).

- document the following useful remove_header transport attributes:

	Disposition-Notification-.*
	Read-Receipt-To
	Registered-Mail-Reply-Requested-By
	Return-Receipt-Requested
	Return-Receipt-To
	X-Confirm-Reading-To

- allow rlimit settings to be changed by config -- in particular we
  should at least always accommodate message_buf_size.

- we should probably not deliver any message which has no data
  (e.g. ".<CR><LF>" is sent immediately after "DATA" in an SMTP
  transaction).

- consider making use of ENHANCEDSTATUSCODES in the smtp transport.


New Features:
-------------

These are primarily things that should wait for the next major release.

- someone somewhere has AUTH patches for a version of smail that
  masquerades as "sendmail":  home.axman.com, gomer.august.net

	15:08 [10] $ telnet home.axman.com 25
	Trying 216.87.129.60...
	Connected to home.axman.com.
	Escape character is '^]'.
	220 axman.com sendmail ready at Wed, 21 Jul 2004 14:08:19 -0500 (CDT)
	HELP
	250-The following SMTP commands are recognized:
	250-
	250-   HELO hostname                   - startup and give your hostname
	250-   EHLO hostname                   - startup with extension info
	250-   MAIL FROM:<sender-address>      - start transaction from sender
	250-   RCPT TO:<recipient-address>     - name recipient for message
	250-   EXPN <address>                  - expand mailing list address
	250-   DATA                            - start text of mail message
	250-   RSET                            - reset state, drop transaction
	250-   NOOP                            - do nothing
	250-   DEBUG [level]                   - set debugging level, default 1
	250-   HELP                            - produce this help message
	250-   QUIT                            - close SMTP connection
	250-
	250-The normal sequence of events in sending a message is to state the
	250-sender address with a 'MAIL FROM:' command, give the recipients with
	250-as many 'RCPT TO:' commands as are required (one address per command)
	250-and then to specify the mail message text after the DATA command.
	250 Multiple messages may be specified.  End the last one with a QUIT.
	DEBUG
	250 Debugging level: 1
	EHLO building.weird.com
	250-home.axman.com Hello building.weird.com (building.weird.com from address [204.92.254.24]), here is what we support:
	250-SIZE 10240000
	250-8BITMIME
	250-PIPELINING
	250-EXPN
	250-AUTH LOGIN PLAIN
	250 HELP
	quit
	221 home.axman.com closing connection
	write_log:[8532] remote got QUIT from building.weird.com(building.weird.com) [204.92.254.24].
	Connection closed by foreign host.


- Implement LMTP support, especially for local delivery.

- With LMTP verification should probably go all the way to doing an LMTP
  VRFY command to hopefully get over-quota notification right away.

- Implement STARTTLS for port#25 and consider supporting SMTPS (SMTP
  over TLS/SSL on port 465) as well.

- Implement SASL (with SASL options to require SSL?).

- consider supporting the Postfix "XADDR" SMTP command

  - Easier debugging of SMTPD access restrictions.  The SMTP command
    "XADDR client-address client-hostname" changes Postfix's idea of
    the remote client name and address, so that you can pretend to
    connect from anywhere on the Internet.

- Think about a config variable that could (maybe $log_events?) that
  could control which items are logged and which are not [or wait for
  syslog support?]

- implement RFC 3461 ESMTP DSN (maybe not NOTIFY=DELAY though)

- implement RFC 3464 "MIME Delivery Status Notifications" for bounces.

- implement at least some of RFC 2852, DELIVERBY, to allow clients to
  specify their own retry duration and to request notifications and
  traces.  [how?  more queue stored command-line parameters?]

- write a minimal mailstats replacement (new log file format only)
  [real stats, not just what logsumm does]

- implement '-R'

     -Rstring	    Go through the  queue  of  pending	mail  and
		    attempt  to	 deliver any message with a reci-
		    pient containing the specified string.   This
		    is useful for clearing out mail directed to a
		    machine which has been down for awhile.

- implement ETRN from RFC 1985 ala the above (patch already available,
  but needs some performance enhancements and support for '-R').

- implement other standards-track SMTP extensions....

- possible make the daemon children change their ps command line text to
  show what they are currently doing (on systems where this is possible)

- teach substitute() to recognize the variable names listed in
  conf_attributes, etc.(?)

- consider deprecating the aliasinclude and forwardinclude director
  drivers in favour of a new "matchdriver" attribute in the lone
  genericinclude driver.  This new attribute would work like the
  "matchdirector" attribute but would match all directors using the
  specified driver.  The aliasinclude director entry would then use the
  genericinclude driver and have "matchdriver=aliasfile", etc.  The code
  savings would be tiny but the documentation would be much cleaner!

- it would be nice to have some kind of sanity checker that could work
  out conflicts between various directors -- i.e. some way to see if
  more than one director would match a given mailbox name.

- implement an LDAP DB lookup method.

- consider adding an option to checkerr to turn off statistics reporting



Miscellaneous:
--------------------

- remove nested includes from routers/bind.h and transports/tcpsmtp.h

- add documentation to each header describing what other headers it
  depends on (e.g. "log.h" needs "addr.h" for struct identify_addr)

- investigate this weird log message fragment:

	ORIG-ID:<199604230758.AA13625@post.tandem.com\POS,$ZNET^U5>

  (possibly related: what'll happen if a message-ID header has other
  crap, and even continued lines, in it too?)

- install ".so" (soelim) manual pages with their full longer names on
  systems with longnames (do we detect this feature dynamically, or do
  we rely on a configuration item?) [need to fix up xrefs too?]

- we should add IsValid*() checking?  from:
  <URL:ftp://ftp.cert.org/pub/cert_advisories/CA-96.04.corrupt_info_from_servers>
  [one place this should be done is in addr.c:check_target_and_remainder()]
  [[ or maybe re-implement it with a hand-written parser ]]

- what about checking syntax of names retrieved from PTRs too?

- read RFC-2821 and RFC-2822 even more carefully.

- re-check use of all RFC 3463 ENHANCEDSTATUSCODES

- consider implementing an "RFCS" command to list supported RFCs, etc.
  For example this result from Maillenium:

	RFCS
	214-  RFC -- description --
	214-  821 Simple Mail Transport Protocol
	214-  822 Standard for ARPA Internet Text Messages
	214- 1047 Duplicate Messages and SMTP
	214- 1321 MD5 Message-Digest Algorithm
	214- 1652 SMTP Service Extensions for 8bit-MIME transport
	214- 1869 SMTP Service Extensions
	214- 1870 SMTP Service Extensions for Message Size Declaration
	214- 1891 SMTP Service Extensions for Delivery Status Notifications
	214- 2195 IMAP/POP AUTHorize Extension for Simple Challenge/Response
	214- 2197 SMTP Service Extensions for Command Pipelining
	214- 2222 Simple Authentication and Security Layer (SASL)
	214- 2246 The TLS Protocol
	214- 2487 SMTP Service Extension for Secure SMTP over TLS
	214- 2505 Anti-Spam Recommendations for SMTP MTAs
	214- 2554 SMTP Service Extensions for Authentication
	214- 2595 Using TLS with IMAP, POP3 and ACAP ( Auth Plain )
	214- 2821 Simple Mail Transport Protocol ( updates rfc821 )
	214- 2822 Internet Message Format
	214  2852 Deliver By SMTP Extension

- consider implementing a "VERS[ion]" command like Maillenium:

	VERS
	250-version:      V04.50c++
	250-compiled:     13-Jan-03 14:35:17
	250-codebase:     cpu_rs6000.os_aix.comp_ibm
	250-developers:   AT&T Labs (Middletown) - Maillennium
	250-core_team:    Steve Spear, Michael McGroary, Al Robinson
	250-support:      ALGOR LDAP(Open) MTA(switch) STUB 
	250-active:       LDAP
	250-up_since:     Wed Jun 18 20:59:59 2003
	250 snmp:         compiled in / running

- think about not stripping comments from aliases, etc., and providing
  GCOS info; esp. for EXPN and VRFY, perhaps re-using smtp_info to
  control.

- Should the "real_user" director set ignore_alias_match?

- consider allowing multiple whitespace characters to act as one when
  speparating words in a string parsed by expand_string().

- consider ignoring trailing whitespace on config entries that don't
  have quoted value settings.  [already done?]

- think about the possible benefits of having separate DBG_DRIVER types
  for each of the different kinds of drivers (router, director,
  transport).

- clean up the duplication between COPY_STRING() and copy() -- maybe
  even call it smail_strdup()?

- there may still be some minor memory leakage in 'mailq -s'

- clean up remaining use of deprecated index(), bcopy(), whatever.

- an interval of '1y' prints as '52w1d5h45m36s'.

- consider a feature to have a dynamic list of hosts (i.e. a lookup db)
  that could be used when sorting MXs such that a 4xx response could be
  returned at RCPT time for any address in any domain that this host is
  secondary for.  The list could then be regularly updated by an
  external script that tested whether the primary host was up or not and
  thus this secondary would only accept mail for the domain if the
  primary was down.  Timing of the down check wouldn't have to be
  immediate since any messages deferred for a just recently down primary
  would still end up getting delivered eventually, either to this
  secondary if the next check happens before the next delivery attempt,
  or to the primary if the primary is up/reachable again before the next
  delivery attempt.  Transient reachability problems between the primary
  and the client-SMTP are irrelevant since they will presumably go away
  before the delivery times out.  This would allow one to implement true
  "backup"-MX service for longer-term customer outages without having to
  worry quite so much about filtering issues in these days of direct-to-
  secondary spamming.

- note that when a user has a ~/.forward file that explicitly forwards
  their mail to their "local" address (e.g. so that it can be shared
  across NFS such that mail sent to them on a workstation will go to the
  central mail hub) the owner address gets changed to real-$user, which
  of course still routes to the same local address and so bounces due to
  over-quota problems on the mail hub will end up in the error queue.
  Maybe this is a good thing?

- the "required" and "ignore" private attributes implemented by
  many/most router drivers should be deprecated and new common
  "only_match_domains" and "ignore_domains" attributes should be created
  in their place.  (Note the bind driver borks "required" to have a
  non-standard meaning and has its own private match_domains hack
  instead.)
