Well, the comments confirm what I suspected: there's no easy answer.
Long story short: there is no way to reliably convert character sets by automatic processing, and I'm tired of doing it by hand.
The solution? Sorry, I haven't got one:
- If I change to UTF-8, then Digest contributors lose *some* "native mode" flexibility in using common phrases which have accented characters. Since ISO-8859-1 is "more or less" the Windows character set, I think that makes it easier to copy-and-paste quotes with accented characters from onlie sites.
- Even though the Digest is written in English, those with accented characters in their names are entitled to see those characters printed accurately. Especially in Canada, where French-speaking citizens have the right to use accented characters in legal records - something computer databases in various U.S. states do not allow to this day - the Digest needs to reflect the audience it is being
Quebec.
The Digest's FAQ spells out the policy:
- Your post must be written in English. Although it may contain words or phrases that are commonly used by non-English speaking peoples, such entries must be generally acceptable in the English-speaking online world.
I don't know if there's a perfect situtation, but ISO-8859-1 is showing its age, and I feel it's time for a change. What change will best reflect our readers' needs, I'm not sure. What I *AM* sure of is that no one solution will please everyone.
I'm going to wait and see if there are other opinions, and do some more research.
And, if you're thinking that I know all the answers, I'll just shrug
Bill Horne Moderator
P.S. Anyone who wants to see the current list of Frequently Asked Questions (FAQ) need only use the link below to send an email to snipped-for-privacy@telecom-digest.org, with the words "faq telecom" in the BODY of the message (the subject line will be ignored). If your mail client supports automatic conversion of plain-text, then the link here will be useful as-is, but otherwise you'll have to write the line yourself:
mailto: snipped-for-privacy@telecom-digest.org?body=faq%20telecom