In the last post I wrote that supplementary characters in the Unicode standard are in the range above U+FFFF, which means they need more than 16 bits to represent them. Since the char primitive type in Java is a 16 bit character, we will have to use 2 char's for them.
I just finished reading some stuff on supplementary character support in Java, and well, there are parts I understood right away and parts that are going to need further reading. I will try to share what I am learning on this blog. However, let us first clarify some terminology.
Character: Is an abstract minimal unit of text. It doesn't have a fixed shape (that would be a glyph), and it doesn't have a value. "A" is a character, and so is "€", the symbol for the common currency of Germany, France, and numerous other European countries.
Character Set: Is a collection of characters.
Unicode is a coded character set that assigns a unique number to every character defined in the Unicode standard.
A Code Point is the unique number assigned to every Unicode character. Valid Unicode code points are in the range of U+0000 to U+10FFFF. This range is capable of holding more than a million characters, out of which 96,382 have been assigned by the Unicode 4.0 standard.
Supplementary characters are those characters that have been assigned code points beyond U+FFFF. So essentially they lie in the range of U+10000 - U+10FFFF.
When these characters are to be stored in a computer system, they have to be stored as a sequence of bits (this is known as UTF-32 encoding). The simplest way store them is to store each character as a 4 byte sequence capable to addressing the entire unicode range. However this will waste a lot of space, because most of the time we deal with characters in the ASCII range of 00 - FF. Some other mechanism is needed to make better use of the computer's memory and storage. Other encodings that exist are UTF-8 and UTF-16, which as their names suggest, use 8-bit and 16-bit sequences.
A natural question that must have occurred to you is, how do we store characters that go beyond 8 bits or 16 bits in UTF-8 and UTF-16. This is made possible by using multiple blocks. Each block will also have to indicate whether it represents a single character or is part of a series of blocks that represent one character. UTF-8 and UTF-16 help us store characters using less space than UTF-32. The most widely used encoding standard is UTF-8.
In the next post I will discuss how Java supports the supplementary range in it's API's and in the Virtual Machine.
Note: This text was originally posted on my earlier blog at http://www.adaptivelearningonline.net
Comments