New Features in Exim
--------------------

This file contains full descriptions of new features that have been added to
Exim, but have not yet made it into the main manual (which is most conveniently
updated when there is a relatively large batch of changes). The toplevel file
CHANGES contains brief listings for recent releases. The doc/ChangeLog file
contains a listing of all changes, including bug fixes.

Version 3.16
------------

1. The handling of "freeze" and "fail" in system filter files has changed.
Previously, any deliveries set up by a filter that ended with "freeze" or
"fail" were discarded. This no longer happens; such deliveries are honoured.
Thus, it is now possible to use constructions such as

   mail .....
   freeze

to send a specified message when the system filter is freezing (or failing)
something. The same change has been made to non-system filters that have
the allow_system_actions option set so they can freeze or fail.

2. Berkeley DB 3.1 is now supported. You don't need to do anything special.

3. Previously, only the current group was tested against trusted_groups. Now,
the supplementary group list is also tested against trusted_groups.

4. The $body_linecount variable contains the number of lines in the body (for
Mutt users).


Version 3.15
------------

1. A terminology change is under way. Instead of saying that a router or
director "fails" an address when it cannot handle it, we now say it "declines"
an address. This avoids overloading the word "fail", which is now reserved for
describing an undeliverable address. In support of this change, some option
settings have been changed:

  . The generic router option "self" now uses "pass" for passing the address to
    the next router, and "fail" for failing the address. The previous values
    ("fail_soft" and "fail_hard") remain as synonyms, but are deprecated.

  . The host_find_failed option in the domainlist router has had the same
    change applied.

  . The output from the command run by the queryprogram router should now
    use the word "decline" instead of "fail", which is now deprecated.

2. Whenever a router is specified as passing the address to the following
router ("pass" in the "self" option, or domainlist or queryprogram without
an option for looking up a new domain), a setting of no_more in the current
router is overridden.

3. There is now an option called "timezone". When "timestamps_utc" is *not*
set, the value of timezone is used to set the environment variable TZ while
running Exim (if it is different on entry). This ensures that all timestamps
created by Exim (e.g. on log lines) are in the required timezone. The default
value of "timezone" is taken from TIMEZONE_DEFAULT in the Local/Makefile, or,
if that is not set, from the value of the TZ environment variable when Exim is
built. If timezone is set to the empty string, either at build or run time,
then any existing TZ variable is removed from the environment when Exim runs.
This is appropriate behaviour on some operating systems.


Version 3.14
------------

1. The values of $domain_data and $local_part_data are now preserved with the
address when it is queued for a transport, and are made available for use in
the transport.

2. A user filter file may now use the errors_to keyword to reset the envelope
sender after a "deliver" command, but only to change it to point to the address
that caused the filter to run. In other words, a user can use a filter file to
forward a message elsewhere, with bounce messages coming back to the forwarding
user instead of the original sender. Obviously, if this is done, the filter
file must not attempt to forward the bounce messages. Only a system filter can
supply an arbitrary address after errors_to.

3. By default, the appendfile transport uses non-blocking calls to fcntl() when
locking a file. If the call fails, it sleeps (for lock_interval) and tries
again (for lock_retries times). Non-blocking calls are used so that the file is
not kept open during the wait for the lock; the reason for this is to make it
as safe as possible for NFS deliveries in the case when processes might be
accessing an NFS mailbox without using a lock file. (This shouldn't be done,
but ...). On a busy system, however, the performance of a non-blocking lock
approach is not as good as using a blocking lock with a timeout. There is now a
new option for appendfile called lock_fcntl_timeout, defaulting to 0s. If it is
set to a non-zero time, blocking locks, with that timeout, are used. There may
still be some retrying: the maximum number of retries is

  (lock_retries * lock_interval) / lock_fcntl_timeout

rounded up to the next whole number. In other words, the total time during
which appendfile is trying to get a lock is roughly the same, unless
lock_fcntl_timeout is set very large.

4. The following new options have been added to the exim_lock utility:

  -retries  <number>   Set the number of times to retry (default 10)
  -interval <number>   Set the interval to sleep between retries (default 3)
  -timeout  <number>   Set a timeout value for fcntl() locking

Times are always in seconds, and are unqualified numbers.

5. The RBL-handling features have been extended. The syntax of an item in the
list in rbl_domains is now

  <domain>[=<ip-address>[,<ip-address>]...][/accept | /reject | /warn]

For example:

  rbl_domains = "white.list/accept : \
                 relays.orbs.org=127.0.0.2/reject :
                 rbl.maps.vix.com/warn"

In other words, the name of the DNS domain can be followed by an optional
address list (comma-separated), as well as by one of the three optional words.
If the address list is present, then one of the address records that is found
in the DNS must match one of the addresses in the list for it to "count". The
ORBS database, for example, uses different addresses to denote different types
of open relay, and you might want to block on some but not on others. The
current values are 127.0.0.2 for a confirmed open relay, 127.0.0.3 for a manual
entry, and 127.0.0.4 for a netblock. If the are no addresses given, then any A
record is treated as a match.

The new "accept" option allows RBL-type lookups to be used for "white lists" as
well as black lists. Relaying is accepted from a host that matches an "accept"
item, and no further RBL domains are considered.  Earlier /warn entries may
have already added warning headers.

6. In addition to the extension in (5), any item in rbl_domains may now also be
followed by /skiprelay. When this is set, checking of that particular RBL
domain is skipped if the calling host matches host_accept_relay.

7. The handling of maildir_tag has been changed. Any non-blank printing
characters other than '/' are now permitted, and a leading colon is inserted
only if the string begins with an alphanumeric character. This means that
maildir++ names that end in ,S=ddd can be generated. The expansion of
maildir_tag now happens after the file has been written, and $message_size is
updated to the accurate size of the file before the expansion.

8. If quota is set on an appendfile transport, and one of the delivery modes
that writes a separate file for each message is being used, then when Exim
wants to find the size of a file, it first checks quota_size_regex. If this is
set to a regular expression that matches the file name, and it captures one
string, then that string is interpreted as a representation of the file's size.
This feature is useful only when users have no shell access to their mailboxes
- otherwise they could defeat the quota simply be renaming the files.

This facility can be used with maildir deliveries, by setting maildir_tag to
add the file length to the file name. For example:

  maildir_tag = ,S=$message_size
  quota_size_regex = S=(\d+)$

The string is not expanded.

9. When auth_always_advertise is false, a AUTH is no longer advertised if the
host is in host_accept_relay, even if it is also in host_accept_auth_relay.
Thus one can use combinations like

  host_auth_accept_relay = *
  host_accept_relay = 10.9.8.0/24

without having to fill up host_auth_accept_relay with exceptions.

10. If a lookup defers during a search of a list such as local_domains, Exim no
longer panics and dies. What happens depends on the circumstances:

. local_domains, hold_domains, queue_remote_domains during delivery: the
  address it is checking is deferred; other addresses may succeed if they
  match something earlier in the list.

. domains, local_parts, senders, condition on a router or director: delivery
  is deferred.

. local_domains, percent_hack_domains, relay_domains while receiving SMTP:
  a 451 temporary error is given to the RCPT command.

. local_domains during verification: temporary error given.

. mx_domains during lookuphost: defer.

. mx_domains and in the smtp transport (i.e. for hosts specified on the
  transport): treat as not matching.

. queue_smtp_domains in the smtp transport: treat as not matching - otherwise
  all SMTP deliveries would be held up.

11. It appears that more and more DNS zones are breaking the rules and putting
IP addresses on the RHS of MX records. Exim follows the rules and rejects this,
but other MTAs do support it, so allow_mx_to_ip has been added to permit this
heinous activity.

12. All lines in the configuration file can now be continued by ending them
with backslash. The change applies to option settings where the value is not
given in quotes. Trailing white space after the backslash is ignored, and
leading white space at the start of continuation lines is also ignored. Comment
lines (those beginning with #, ignoreing leading white space) are entirely
ignored, and may appear in the middle of a sequence of continuation lines.

13. All lists, with the exception of log_file_path, can now use a different
character to colon as the separator. This is specified by starting the list
with <x where x is any punctuation character. For example:

  local_interfaces = <; 127.0.0.1 ; ::1

The new feature is provided to make life easier with IPv6 addresses. It is
recommended that its use be confined to circumstances where it really is
needed, and that colon be used in most cases.

14. Support for PostgreSQL (pgsql) is now included in Exim. It is exactly
analagous to MySQL support. Set LOOKUP_PGSQL in Local/Makefile to include it in
the binary, and set pgsql_servers to a list of host/database/user/password
tuples. The lookup type is "pgsql".

15. The forbid_reply option in forwardfile has been renamed as
forbid_filter_reply, to make it have the same form as the other forbid_filter_
xxx options. The old name remains as a synonym.

16. If timestamps_utc is set, all time stamps generated by Exim are in UTC
rather than in local time.

17. If no_local_from_check is set, checking the From: line of locally
submitted messages for matching the login id is disabled, so no Sender: header
is ever added. The envelope sender, however, is still forced to be the login id
at the qualify domain.

18. When Exim checks the From: line of locally submitted messages for matching
the login id, it can now be configured to ignore certain prefixes and suffixes
by setting local_from_prefix and/or local_from_suffix to appropriate lists, in
the same form as the prefix and suffix options of directors. For example, if

  local_from_prefix = *-

is set, then a From: line containing

  From: anything-user@your.domain

will not cause a Sender: header to be added if "user@your.domain" matches the
actual sender address that is constructed from the login name and qualify
domain.

19. In an aliasfile director that uses a query-style lookup, "optional"
(previously ignored) now causes the address to be passed to the next director
if no query can be completed (e.g. all databases down). Without "optional", the
delivery is deferred.

20. If hosts_randomize is set on a domainlist router, the order of the items in
the host list is randomized each time it is called.

21. If hosts_randomize is set on an smtp transport that uses its hosts list,
the order of items in the list is randomized each time it is used. This does
not apply to fallback_hosts.

22. When split_spool_directory is set, the behaviour of queue runner processes
has changed. Instead of creating a list of all messages in the queue, and then
trying to deliver each one in turn, it gets a list of those in one sub-
directory and tries to deliver them, before moving on to the next sub-
directory. This spreads out the scanning of the input directories and is
beneficial when there are lots of messages on the queue.

However, if queue_run_in_order is set, none of this new processing happens. The
entire queue is scanned and sorted before any deliveries start, as was the case
before.

This change has affected the way the random ordering of messages is done for
queue running. What happens now is that the list of sub-directories is sorted
randomly, and then the messages within each sub-directory are processed in a
random order.

The way partial queue runs specified by commands like

  exim -q 12XLCQ-0006p9-00 12XLCm-0006pF-00

are implemented has also changed. Previously, the queue was sorted, and
processing stopped after the last selected message. Now no sorting is done; the
entire queue is scanned (directory by directory, if queue_run_in_order is not
set), but messages whose ids are not in the selected range are skipped. Thus
the selected messages are now run in a random order by default.

23. The dnsdb lookup type has been extended:

  (a) If it finds multiple records, they are returned as a concatenation,
  separated by newlines. The order, of course, depends on the DNS resolver.

  (b) Record types other than TXT can now be looked up. The format of a query
  is   type=domain-name   where type can be  txt, a, aaaa (with IPv6 support),
  ptr, mx, ns, or cname, in either upper or lower case. For example:

    ${lookup dnsdb{mx=a.b.example}{$value}fail}

  When the type is PTR, the address should be given as normal; it gets
  converted to the necessary magic internally. For example:

    ${lookup dnsdb{ptr=192.168.4.5}{$value}fail}

  If the type is omitted, it defaults to TXT for backwards compatibility.

24. The aliasfile director now has forbid_include, to lock out the use of
:include:, and also forbid_specials, to lock out the use of :defer:, :fail:,
:blackhole: and :unknown:. If any are used, delivery is deferred.

25. There is now a "tr" operator for string expansions, for translating strings
in the usual way. It takes three arguments: the string to be translated, and
two translation strings. For example

    ${tr {abcde}{ac}{13}}    yields 1b3de

If there are duplicates in the second character string, the last occurrence is
used. If the third string is shorter than the second, its last character is
replicated. However, if it is empty, no translation takes place.

26. There is now an "sg" expansion operator which works like Perl's
substitution operator with a /g option (e.g.  s/xxx/yyyy/g). It takes three
arguments: the subject string, a regular expression, and a substitution string.
For example

  ${sg {abcdefabcdef}{abc}{xyz}}

yield "xyzdefxyzdef". Because all three arguments are expanded before use, if
any $ characters are required in the regex or in the substitution string, they
have to be escaped. For example

  ${sg {abcdef}{^(...)(...)\$}{\$2\$1}}

yields "defabc".

27. The appendfile transport now has a check_owner option (default true). If
turned off, the ownership of the file is not checked.

28. The hosts for which sender_address_relay is checked can now be controlled
by sender_address_relay_hosts, whose default is "*" (all hosts). This is useful
in a cluster where one host is delegated as a fallback to hold all the delayed
deliveries. It needs to be able to relay from the other hosts without sender
checking (e.g. for messages forwarded by local users) but might want to check
senders in messages relayed from other hosts.

29. A new security level, set by

  security = unprivileged

has been implemented. This is for use in installations that want run to Exim in
an unprivileged state almost all the time, for added security. When this is
done, all deliveries take place under the Exim user/group (which must be
defined), and there are restrictions on the features that can be used in the
configuration.

There are two possibilities if you want to run Exim in this way:

(a) Keep it setuid to root, as in other configurations, but set

      security = unprivileged

    In all cases except when starting the daemon, Exim gives up the root
    privilege and becomes the Exim user/group as soon as it has started, using
    setuid() and setgid(). This removes all privilege that might have been
    associated with the calling user. In the case of the daemon, root
    privilege is retained until it has bound its listening socket to the SMTP
    port, but then it gives it up in the same way. The daemon can respond
    correctly to SIGHUP because the re-invocation regains root privilege.

(b) Make Exim setuid/setgid to the Exim user and group. This means it cannot
    start up the daemon unless it is called by a root process, and
    consequently, the daemon cannot restart itself as a result of SIGHUP
    because it is no longer a root process at that point. It is still useful
    to set

      security = unprivileged

    in this case, because this setting stops Exim from trying to re-invoke
    itself to do a delivery after a message has been received. Such a re-
    invocation is a waste of time because it would have no effect.

Using security = unprivileged is more efficient than security = setuid or
security = setuid+seteuid because Exim no longer needs to re-exec itself when
starting a delivery process after receiving a message. However, to achieve this
extra efficiency you have to submit to the restrictions described below.

When using style (b), unless called by root (in which case it behaves as style
(a)), Exim is running with the real uid and gid set to those of the calling
process, and the effective uid/gid set to Exim's values. Ideally, any
association with the calling process' uid/gid should be dropped, that is, the
real uid/gid should be reset to the effective values so as to discard any
privileges that the caller may have. While some operating systems have a
function that permits this action for a non-root effective uid, quite a number
of them do not. Because of this lack of standardization, Exim does not address
this problem at this time. For this reason, (a) is perhaps the better approach
to take.

The restrictions on the configuration when running Exim unprivileged are as
follows:

You can deliver only as the Exim user/group. You should  explicitly use the
"user" and "group" options to override directors or transports that normally
deliver as the recipient. (This makes sure that configurations that work in
this mode function the same way in normal mode.) Any implicit or explicit
specification of another user causes an error.

Use of .forward files is severely restricted, such that it is usually
not worthwhile to include a forwardfile director in the configuration.

Users who wish to use .forward would have to make their home directory and the
file itself accessable to the Exim user. Pipe and append-to-file entries, and
their equivalents in Exim filters, cannot be used. While they could be enabled
in the Exim user's name, that would be insecure and not very useful.

Unless the user mailboxes are all owned by the Exim user (possible
in some POP3 or IMAP-only environments):

 .   They must be owned by the Exim group and be writable by that group.
     This implies you must set "mode" in the appendfile configuration,
     as well as the mode of the mailbox files themselves.

 .   You must set "no_check_owner", since most or all of the files will
     not be owned by the Exim user.

 .   You must set "file_must_exist", as Exim cannot set the owner
     correctly on a new mailbox when unpriviledged. This also implies
     that new mailboxes need to be created manually.

There are no additional restrictions on message reception or external (SMTP)
delivery.


Version 3.13
------------

1. There is a new option called auth_always_advertise, available when Exim is
compiled with AUTH support, which defaults TRUE. If it is set FALSE, then Exim
advertises the AUTH command in response to EHLO only if the calling host is in
auth_hosts or host_auth_accept_relay (assuming Exim is configured with some
authenticators). [See 3.14/9 above for later addition.] Otherwise it does not
advertise AUTH, though it is prepared to accept it. Certain mail clients (e.g.
Netscape) require to the user to provide a name and password for authentication
if AUTH is advertised, even though it may not be needed (the host may be in
host_accept_relay, for example). Unsetting auth_always_advertise makes these
clients more friendly in these circumstances.

2. The appendfile transport has a new option called allow_fifo. When set, it
permits delivery to named pipes (FIFOs) as well as to regular files. If no
process is reading the named pipe at delivery time, the delivery is deferred.

3. The smartuser now has qualify_preserve_domain, which works the same as it
does for aliasfile and forwardfile.

4. The exim_dbmbuild utility now has a -noduperr option. If set, duplicates in
the input do not cause the return code to be non-zero.


Version 3.10
------------

The documentation is up-to-date for the 3.10 release.
