From xemacs-m  Mon Feb  3 14:50:48 1997
Received: from venus.Sun.COM (venus.Sun.COM [192.9.25.5])
	by xemacs.org (8.8.5/8.8.5) with SMTP id OAA24617
	for <xemacs-beta@xemacs.org>; Mon, 3 Feb 1997 14:50:48 -0600 (CST)
Received: from Eng.Sun.COM ([129.146.1.25]) by venus.Sun.COM (SMI-8.6/mail.byaddr) with SMTP id MAA06836; Mon, 3 Feb 1997 12:50:17 -0800
Received: from kindra.eng.sun.com by Eng.Sun.COM (SMI-8.6/SMI-5.3)
	id MAA18447; Mon, 3 Feb 1997 12:50:14 -0800
Received: from xemacs.eng.sun.com by kindra.eng.sun.com (SMI-8.6/SMI-SVR4)
	id MAA27252; Mon, 3 Feb 1997 12:50:13 -0800
Received: by xemacs.eng.sun.com (SMI-8.6/SMI-SVR4)
	id MAA29644; Mon, 3 Feb 1997 12:50:12 -0800
Date: Mon, 3 Feb 1997 12:50:12 -0800
Message-Id: <199702032050.MAA29644@xemacs.eng.sun.com>
From: Martin Buchholz <mrb@Eng.Sun.COM>
To: Oscar Figueiredo <Oscar.Figueiredo@di.epfl.ch>,
        MORIOKA Tomohiko <morioka@jaist.ac.jp>,
        Kyle Jones <kyle_jones@wonderworks.com>
Cc: XEmacs Beta Test <xemacs-beta@xemacs.org>
Subject: State of VM + XEmacs/Mule
Reply-To: Martin Buchholz <mrb@Eng.Sun.COM>
Mime-Version: 1.0 (generated by tm-edit 7.100)
Content-Type: text/plain; charset=US-ASCII

Reading mail with VM+TM containing non-latin-1 characters is a
dangerous operation in XEmacs 20.0.  I realize the VM software
maintainers have already gone beyond the state of the code in XEmacs
20.0, but the issues probably still apply, so I will discuss the
problems at a high level.

When VM gets new mail, it creates a crashbox file, and then
incorporates it into the inbox via a process I don't understand.  What
is of interest is the encoding used.  For the sake of the discussion,
let's assume the user is Japanese and has (set-language-environment 'japanese).

Every time XEmacs visits a new file, it must guess the encoding.  The
default coding-system is 'autodetect, which means for a new message
containing iso2022-encoded japanese, that it might guess iso2022-jp for
the crashbox.  So the inbox buffer will contain japanese characters.
But it is likely to guess 'binary when it visited the inbox itself,
which means also that when the inbox is finally saved, that the
japanese characters will get turned into '~' characters, since there
is no way to encode those characters using the binary encoding.

Anyways, the Right Thing is likely that VM should force use of the
'binary encoding both for reading in new mail from the spool, and for
reading and saving a folder, so that the original encoding of the
messages are preserved (this also ensures compatibility with other
MUA's reading the same folder later).  When presenting a mail message
to the user, however, some kind of conversion must be performed to
display the appropriate Mule characters. VM+TM+vm-tm7 handles this
correctly for the RFC1522-encoded headers, but seems to get things
very wrong for the body of the message.

The bottom line is that VM+TM is currently unusable if you get
non-latin-1 mail, and this is something that needs to be addressed in
future VM software, hopefully in XEmacs 20.1, due out before summer
1997.  I don't know whether VM 6 sans TM, or VM+TM+vm-tm8 will be the
solution, but both must address the encoding problem.  (I have not
used either VM6 or tm-vm8 yet.)

Martin

