Bug#178194: should work more cooperative on latin1<->utf-8 recoding failures

[James Vega]

On Fri, Jan 24, 2003 at 12:55:06PM +0100, Eduard Bloch wrote:
> you are working in the Unicode environment
>  - you open a new file
>  - vim chooses latin1 as native file encoding, as described in
>    options.txt:1867ff. This is okay, if you store a file, it is better
>    to have it in the native encoding (latin1) for compatibility reasons.
>    _But_: only better for middle-europeans. Non-europeans should be able
>    to specify another encoding as default for empty (or pure-ascii)
>    files.
>  - As next, you insert some latin1 chars (say, a German signature) and a
>    text in Cyrrilic. Now you try to save the file. Vim only says:
>    file foo ... CONVERSION FAILED, written.

Current Vim warns "CONVERSION ERROR" and does NOT write out the file.

>    a) This information is completely insufficient for new users. It
>    should say something like: CONVERSION FAILED, unset "fileencoding"
>    setting to write in UTF-8.

Encodings aren't exactly a simple topic.  It's not easy to offer good
advice for what to do since Vim basically knows "There are
bytes/characters in this buffer which are not valid in the buffer's
encoding."  Those may be valid in more than one encoding (other than the
current one, obviously) and it's up to the user to tell Vim what to do.

>    b) I wish a new option to make VIM work more intelligent. If this new
>    option is set, VIM should store in the native encoding (latin1 for
>    German, or KOI8-R on Russian when specified, see above) as long as
>    conversion is possible, and fall-back to UTF-8 when it is not
>    possible, and warn then, of course.

Based on the "CONVERSION ERROR" message, Vim is attempting to convert to
the native encoding of the buffer.  Automatically changing the buffer to
another encoding is presumptuous of Vim which is why it simply fails.

The typical way to solve this is to tell Vim which encoding you actually
want to use when writing out the file:

  :set fenc=...
  :w

  or

  :w ++enc=...

This still relies on all the characters in the buffer being able to be
represented in the new encoding.

There's an item in Vim's todo list to improve the error message so that
it tells you which column in the line has the conversion error so the
user has more information on which to base their decision

I don't really see a better way of handling this.

-- 
James
GPG Key: 1024D/61326D40 2003-09-02 James Vega <jamessan at debian.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://lists.alioth.debian.org/pipermail/pkg-vim-maintainers/attachments/20080402/fc4d060b/attachment.pgp