Saturday, October 4, 2008

Chinese character encoding

In computing, Chinese character encodings can be used to represent text written in the CJK languages — , , — and obsolete , all of which use Chinese characters. Several general-purpose character encodings accommodate Chinese characters, and some of them were developed specifically for Chinese.

The following are common Chinese character encoding systems:

* is mainly used in Mainland China and Singapore. All Guobiao standards are prefixed by GB, the latest version is GB18030 which is a one, two or four byte encoding.
* Big5, used in Taiwan, Hong Kong and Macau, is a one or two byte encoding.
* Unicode.

Other encoding scheme, such as were also used in early days.

Guobiao is usually displayed using and Big5 is usually displayed using . There is however no mandated connection between the encoding system and the font used to display the characters; font and encoding are usually tied together for practical reasons.

The conversion between traditional and simplified Chinese is usually problematic, because the simplification of some traditional forms merged two or more different characters into one simplified form. The traditional to simplified conversion is technically simple. The opposite conversion often results in a data loss when converting to early forms of the GB character set : in mapping one-to-many when assigning traditional glyphs to the simplified glyphs, some characters will inevitably be the wrong choices in some of the usages. Thus simplified to traditional conversion often requires usage context or common phrases to resolve conflicts. This issue is less of a problem with newer standards such as GB18030 and Unicode which have separate code points for both simplified and traditional characters.

One other issue is that many of the encoding systems are missing characters. While the missing characters are often literary and not commonly used in ordinary text, this does become a problem because people's names often contain these characters. An example of the problem is the Taiwanese politician Wang Jian-Hsuan whose second given name is not in some character systems. But the newest GB standard, GB18030 has the complete character repertoire of Unicode 4.0, including the Unihan extensions in the Supplementary Ideographic Plane.

The issue of which encoding to use can also have political implications, as GB is the official standard of the People's Republic of China and Big5 is a ''de facto'' standard of Taiwan.

In contrast to the situation with , there has been relatively little overt opposition to Unicode, which solves many of the issues involved with GB and Big5. Unicode is widely regarded as politically neutral, has good support for both simplified and traditional characters, and can be easily converted to and from the GB and Big5. Furthermore Unicode has the advantage of not being limited only to Chinese, since it can also display many other character sets.

No comments: