|
428
|
1 @c -*-texinfo-*-
|
|
|
2 @c This is part of the XEmacs Lisp Reference Manual.
|
|
|
3 @c Copyright (C) 1996 Ben Wing.
|
|
|
4 @c See the file lispref.texi for copying conditions.
|
|
|
5 @setfilename ../../info/internationalization.info
|
|
|
6 @node MULE, Tips, Internationalization, top
|
|
|
7 @chapter MULE
|
|
|
8
|
|
442
|
9 @dfn{MULE} is the name originally given to the version of GNU Emacs
|
|
428
|
10 extended for multi-lingual (and in particular Asian-language) support.
|
|
442
|
11 ``MULE'' is short for ``MUlti-Lingual Emacs''. It is an extension and
|
|
|
12 complete rewrite of Nemacs (``Nihon Emacs'' where ``Nihon'' is the
|
|
|
13 Japanese word for ``Japan''), which only provided support for Japanese.
|
|
|
14 XEmacs refers to its multi-lingual support as @dfn{MULE support} since
|
|
|
15 it is based on @dfn{MULE}.
|
|
428
|
16
|
|
|
17 @menu
|
|
|
18 * Internationalization Terminology::
|
|
|
19 Definition of various internationalization terms.
|
|
|
20 * Charsets:: Sets of related characters.
|
|
|
21 * MULE Characters:: Working with characters in XEmacs/MULE.
|
|
|
22 * Composite Characters:: Making new characters by overstriking other ones.
|
|
|
23 * Coding Systems:: Ways of representing a string of chars using integers.
|
|
|
24 * CCL:: A special language for writing fast converters.
|
|
|
25 * Category Tables:: Subdividing charsets into groups.
|
|
|
26 @end menu
|
|
|
27
|
|
442
|
28 @node Internationalization Terminology, Charsets, , MULE
|
|
428
|
29 @section Internationalization Terminology
|
|
|
30
|
|
442
|
31 In internationalization terminology, a string of text is divided up
|
|
428
|
32 into @dfn{characters}, which are the printable units that make up the
|
|
|
33 text. A single character is (for example) a capital @samp{A}, the
|
|
442
|
34 number @samp{2}, a Katakana character, a Hangul character, a Kanji
|
|
|
35 ideograph (an @dfn{ideograph} is a ``picture'' character, such as is
|
|
|
36 used in Japanese Kanji, Chinese Hanzi, and Korean Hanja; typically there
|
|
|
37 are thousands of such ideographs in each language), etc. The basic
|
|
|
38 property of a character is that it is the smallest unit of text with
|
|
|
39 semantic significance in text processing.
|
|
|
40
|
|
|
41 Human beings normally process text visually, so to a first approximation
|
|
|
42 a character may be identified with its shape. Note that the same
|
|
|
43 character may be drawn by two different people (or in two different
|
|
|
44 fonts) in slightly different ways, although the "basic shape" will be the
|
|
|
45 same. But consider the works of Scott Kim; human beings can recognize
|
|
|
46 hugely variant shapes as the "same" character. Sometimes, especially
|
|
|
47 where characters are extremely complicated to write, completely
|
|
|
48 different shapes may be defined as the "same" character in national
|
|
|
49 standards. The Taiwanese variant of Hanzi is generally the most
|
|
444
|
50 complicated; over the centuries, the Japanese, Koreans, and the People's
|
|
442
|
51 Republic of China have adopted simplifications of the shape, but the
|
|
|
52 line of descent from the original shape is recorded, and the meanings
|
|
|
53 and pronunciation of different forms of the same character are
|
|
|
54 considered to be identical within each language. (Of course, it may
|
|
|
55 take a specialist to recognize the related form; the point is that the
|
|
|
56 relations are standardized, despite the differing shapes.)
|
|
428
|
57
|
|
|
58 In some cases, the differences will be significant enough that it is
|
|
|
59 actually possible to identify two or more distinct shapes that both
|
|
|
60 represent the same character. For example, the lowercase letters
|
|
440
|
61 @samp{a} and @samp{g} each have two distinct possible shapes---the
|
|
428
|
62 @samp{a} can optionally have a curved tail projecting off the top, and
|
|
|
63 the @samp{g} can be formed either of two loops, or of one loop and a
|
|
|
64 tail hanging off the bottom. Such distinct possible shapes of a
|
|
|
65 character are called @dfn{glyphs}. The important characteristic of two
|
|
|
66 glyphs making up the same character is that the choice between one or
|
|
|
67 the other is purely stylistic and has no linguistic effect on a word
|
|
|
68 (this is the reason why a capital @samp{A} and lowercase @samp{a}
|
|
440
|
69 are different characters rather than different glyphs---e.g.
|
|
428
|
70 @samp{Aspen} is a city while @samp{aspen} is a kind of tree).
|
|
|
71
|
|
|
72 Note that @dfn{character} and @dfn{glyph} are used differently
|
|
|
73 here than elsewhere in XEmacs.
|
|
|
74
|
|
442
|
75 A @dfn{character set} is essentially a set of related characters. ASCII,
|
|
428
|
76 for example, is a set of 94 characters (or 128, if you count
|
|
|
77 non-printing characters). Other character sets are ISO8859-1 (ASCII
|
|
|
78 plus various accented characters and other international symbols),
|
|
442
|
79 JIS X 0201 (ASCII, more or less, plus half-width Katakana), JIS X 0208
|
|
|
80 (Japanese Kanji), JIS X 0212 (a second set of less-used Japanese Kanji),
|
|
428
|
81 GB2312 (Mainland Chinese Hanzi), etc.
|
|
|
82
|
|
442
|
83 The definition of a character set will implicitly or explicitly give
|
|
|
84 it an @dfn{ordering}, a way of assigning a number to each character in
|
|
|
85 the set. For many character sets, there is a natural ordering, for
|
|
|
86 example the ``ABC'' ordering of the Roman letters. But it is not clear
|
|
|
87 whether digits should come before or after the letters, and in fact
|
|
|
88 different European languages treat the ordering of accented characters
|
|
|
89 differently. It is useful to use the natural order where available, of
|
|
|
90 course. The number assigned to any particular character is called the
|
|
|
91 character's @dfn{code point}. (Within a given character set, each
|
|
|
92 character has a unique code point. Thus the word "set" is ill-chosen;
|
|
|
93 different orderings of the same characters are different character sets.
|
|
|
94 Identifying characters is simple enough for alphabetic character sets,
|
|
|
95 but the difference in ordering can cause great headaches when the same
|
|
|
96 thousands of characters are used by different cultures as in the Hanzi.)
|
|
428
|
97
|
|
442
|
98 A code point may be broken into a number of @dfn{position codes}. The
|
|
|
99 number of position codes required to index a particular character in a
|
|
|
100 character set is called the @dfn{dimension} of the character set. For
|
|
|
101 practical purposes, a position code may be thought of as a byte-sized
|
|
|
102 index. The printing characters of ASCII, being a relatively small
|
|
|
103 character set, is of dimension one, and each character in the set is
|
|
|
104 indexed using a single position code, in the range 1 through 94. Use of
|
|
|
105 this unusual range, rather than the familiar 33 through 126, is an
|
|
|
106 intentional abstraction; to understand the programming issues you must
|
|
|
107 break the equation between character sets and encodings.
|
|
|
108
|
|
|
109 JIS X 0208, i.e. Japanese Kanji, has thousands of characters, and is
|
|
|
110 of dimension two -- every character is indexed by two position codes,
|
|
|
111 each in the range 1 through 94. (This number ``94'' is not a
|
|
|
112 coincidence; we shall see that the JIS position codes were chosen so
|
|
|
113 that JIS kanji could be encoded without using codes that in ASCII are
|
|
|
114 associated with device control functions.) Note that the choice of the
|
|
|
115 range here is somewhat arbitrary. You could just as easily index the
|
|
|
116 printing characters in ASCII using numbers in the range 0 through 93, 2
|
|
|
117 through 95, 3 through 96, etc. In fact, the standardized
|
|
|
118 @emph{encoding} for the ASCII @emph{character set} uses the range 33
|
|
|
119 through 126.
|
|
428
|
120
|
|
|
121 An @dfn{encoding} is a way of numerically representing characters from
|
|
|
122 one or more character sets into a stream of like-sized numerical values
|
|
|
123 called @dfn{words}; typically these are 8-bit, 16-bit, or 32-bit
|
|
|
124 quantities. If an encoding encompasses only one character set, then the
|
|
|
125 position codes for the characters in that character set could be used
|
|
442
|
126 directly. (This is the case with the trivial cipher used by children,
|
|
|
127 assigning 1 to `A', 2 to `B', and so on.) However, even with ASCII,
|
|
|
128 other considerations intrude. For example, why are the upper- and
|
|
|
129 lowercase alphabets separated by 8 characters? Why do the digits start
|
|
|
130 with `0' being assigned the code 48? In both cases because semantically
|
|
|
131 interesting operations (case conversion and numerical value extraction)
|
|
|
132 become convenient masking operations. Other artificial aspects (the
|
|
|
133 control characters being assigned to codes 0--31 and 127) are historical
|
|
|
134 accidents. (The use of 127 for @samp{DEL} is an artifact of the "punch
|
|
|
135 once" nature of paper tape, for example.)
|
|
|
136
|
|
|
137 Naive use of the position code is not possible, however, if more than
|
|
|
138 one character set is to be used in the encoding. For example, printed
|
|
|
139 Japanese text typically requires characters from multiple character sets
|
|
|
140 -- ASCII, JIS X 0208, and JIS X 0212, to be specific. Each of these is
|
|
|
141 indexed using one or more position codes in the range 1 through 94, so
|
|
|
142 the position codes could not be used directly or there would be no way
|
|
|
143 to tell which character was meant. Different Japanese encodings handle
|
|
|
144 this differently -- JIS uses special escape characters to denote
|
|
|
145 different character sets; EUC sets the high bit of the position codes
|
|
|
146 for JIS X 0208 and JIS X 0212, and puts a special extra byte before each
|
|
|
147 JIS X 0212 character; etc. (JIS, EUC, and most of the other encodings
|
|
|
148 you will encounter in files are 7-bit or 8-bit encodings. There is one
|
|
|
149 common 16-bit encoding, which is Unicode; this strives to represent all
|
|
|
150 the world's characters in a single large character set. 32-bit
|
|
|
151 encodings are often used internally in programs, such as XEmacs with
|
|
|
152 MULE support, to simplify the code that manipulates them; however, they
|
|
|
153 are not used externally because they are not very space-efficient.)
|
|
|
154
|
|
|
155 A general method of handling text using multiple character sets
|
|
|
156 (whether for multilingual text, or simply text in an extremely
|
|
|
157 complicated single language like Japanese) is defined in the
|
|
|
158 international standard ISO 2022. ISO 2022 will be discussed in more
|
|
|
159 detail later (@pxref{ISO 2022}), but for now suffice it to say that text
|
|
|
160 needs control functions (at least spacing), and if escape sequences are
|
|
|
161 to be used, an escape sequence introducer. It was decided to make all
|
|
|
162 text streams compatible with ASCII in the sense that the codes 0--31
|
|
|
163 (and 128-159) would always be control codes, never graphic characters,
|
|
|
164 and where defined by the character set the @samp{SPC} character would be
|
|
|
165 assigned code 32, and @samp{DEL} would be assigned 127. Thus there are
|
|
|
166 94 code points remaining if 7 bits are used. This is the reason that
|
|
|
167 most character sets are defined using position codes in the range 1
|
|
|
168 through 94. Then ISO 2022 compatible encodings are produced by shifting
|
|
|
169 the position codes 1 to 94 into character codes 33 to 126, or (if 8 bit
|
|
|
170 codes are available) into character codes 161 to 254.
|
|
428
|
171
|
|
|
172 Encodings are classified as either @dfn{modal} or @dfn{non-modal}. In
|
|
442
|
173 a @dfn{modal encoding}, there are multiple states that the encoding can
|
|
|
174 be in, and the interpretation of the values in the stream depends on the
|
|
428
|
175 current global state of the encoding. Special values in the encoding,
|
|
|
176 called @dfn{escape sequences}, are used to change the global state.
|
|
|
177 JIS, for example, is a modal encoding. The bytes @samp{ESC $ B}
|
|
|
178 indicate that, from then on, bytes are to be interpreted as position
|
|
442
|
179 codes for JIS X 0208, rather than as ASCII. This effect is cancelled
|
|
428
|
180 using the bytes @samp{ESC ( B}, which mean ``switch from whatever the
|
|
442
|
181 current state is to ASCII''. To switch to JIS X 0212, the escape
|
|
|
182 sequence @samp{ESC $ ( D}. (Note that here, as is common, the escape
|
|
|
183 sequences do in fact begin with @samp{ESC}. This is not necessarily the
|
|
|
184 case, however. Some encodings use control characters called "locking
|
|
|
185 shifts" (effect persists until cancelled) to switch character sets.)
|
|
428
|
186
|
|
442
|
187 A @dfn{non-modal encoding} has no global state that extends past the
|
|
428
|
188 character currently being interpreted. EUC, for example, is a
|
|
442
|
189 non-modal encoding. Characters in JIS X 0208 are encoded by setting
|
|
|
190 the high bit of the position codes, and characters in JIS X 0212 are
|
|
428
|
191 encoded by doing the same but also prefixing the character with the
|
|
|
192 byte 0x8F.
|
|
|
193
|
|
|
194 The advantage of a modal encoding is that it is generally more
|
|
442
|
195 space-efficient, and is easily extendible because there are essentially
|
|
428
|
196 an arbitrary number of escape sequences that can be created. The
|
|
|
197 disadvantage, however, is that it is much more difficult to work with
|
|
|
198 if it is not being processed in a sequential manner. In the non-modal
|
|
|
199 EUC encoding, for example, the byte 0x41 always refers to the letter
|
|
|
200 @samp{A}; whereas in JIS, it could either be the letter @samp{A}, or
|
|
442
|
201 one of the two position codes in a JIS X 0208 character, or one of the
|
|
|
202 two position codes in a JIS X 0212 character. Determining exactly which
|
|
428
|
203 one is meant could be difficult and time-consuming if the previous
|
|
442
|
204 bytes in the string have not already been processed, or impossible if
|
|
|
205 they are drawn from an external stream that cannot be rewound.
|
|
428
|
206
|
|
|
207 Non-modal encodings are further divided into @dfn{fixed-width} and
|
|
|
208 @dfn{variable-width} formats. A fixed-width encoding always uses
|
|
|
209 the same number of words per character, whereas a variable-width
|
|
|
210 encoding does not. EUC is a good example of a variable-width
|
|
|
211 encoding: one to three bytes are used per character, depending on
|
|
|
212 the character set. 16-bit and 32-bit encodings are nearly always
|
|
|
213 fixed-width, and this is in fact one of the main reasons for using
|
|
|
214 an encoding with a larger word size. The advantages of fixed-width
|
|
|
215 encodings should be obvious. The advantages of variable-width
|
|
|
216 encodings are that they are generally more space-efficient and allow
|
|
442
|
217 for compatibility with existing 8-bit encodings such as ASCII. (For
|
|
|
218 example, in Unicode ASCII characters are simply promoted to a 16-bit
|
|
|
219 representation. That means that every ASCII character contains a
|
|
|
220 @samp{NUL} byte; evidently all of the standard string manipulation
|
|
|
221 functions will lose badly in a fixed-width Unicode environment.)
|
|
428
|
222
|
|
442
|
223 The bytes in an 8-bit encoding are often referred to as @dfn{octets}
|
|
|
224 rather than simply as bytes. This terminology dates back to the days
|
|
|
225 before 8-bit bytes were universal, when some computers had 9-bit bytes,
|
|
|
226 others had 10-bit bytes, etc.
|
|
428
|
227
|
|
442
|
228 @node Charsets, MULE Characters, Internationalization Terminology, MULE
|
|
428
|
229 @section Charsets
|
|
|
230
|
|
|
231 A @dfn{charset} in MULE is an object that encapsulates a
|
|
|
232 particular character set as well as an ordering of those characters.
|
|
|
233 Charsets are permanent objects and are named using symbols, like
|
|
|
234 faces.
|
|
|
235
|
|
|
236 @defun charsetp object
|
|
|
237 This function returns non-@code{nil} if @var{object} is a charset.
|
|
|
238 @end defun
|
|
|
239
|
|
|
240 @menu
|
|
|
241 * Charset Properties:: Properties of a charset.
|
|
|
242 * Basic Charset Functions:: Functions for working with charsets.
|
|
|
243 * Charset Property Functions:: Functions for accessing charset properties.
|
|
|
244 * Predefined Charsets:: Predefined charset objects.
|
|
|
245 @end menu
|
|
|
246
|
|
442
|
247 @node Charset Properties, Basic Charset Functions, , Charsets
|
|
428
|
248 @subsection Charset Properties
|
|
|
249
|
|
|
250 Charsets have the following properties:
|
|
|
251
|
|
|
252 @table @code
|
|
|
253 @item name
|
|
|
254 A symbol naming the charset. Every charset must have a different name;
|
|
|
255 this allows a charset to be referred to using its name rather than
|
|
|
256 the actual charset object.
|
|
|
257 @item doc-string
|
|
|
258 A documentation string describing the charset.
|
|
|
259 @item registry
|
|
|
260 A regular expression matching the font registry field for this character
|
|
|
261 set. For example, both the @code{ascii} and @code{latin-iso8859-1}
|
|
|
262 charsets use the registry @code{"ISO8859-1"}. This field is used to
|
|
|
263 choose an appropriate font when the user gives a general font
|
|
|
264 specification such as @samp{-*-courier-medium-r-*-140-*}, i.e. a
|
|
|
265 14-point upright medium-weight Courier font.
|
|
|
266 @item dimension
|
|
|
267 Number of position codes used to index a character in the character set.
|
|
|
268 XEmacs/MULE can only handle character sets of dimension 1 or 2.
|
|
|
269 This property defaults to 1.
|
|
|
270 @item chars
|
|
|
271 Number of characters in each dimension. In XEmacs/MULE, the only
|
|
|
272 allowed values are 94 or 96. (There are a couple of pre-defined
|
|
|
273 character sets, such as ASCII, that do not follow this, but you cannot
|
|
|
274 define new ones like this.) Defaults to 94. Note that if the dimension
|
|
|
275 is 2, the character set thus described is 94x94 or 96x96.
|
|
|
276 @item columns
|
|
|
277 Number of columns used to display a character in this charset.
|
|
|
278 Only used in TTY mode. (Under X, the actual width of a character
|
|
|
279 can be derived from the font used to display the characters.)
|
|
|
280 If unspecified, defaults to the dimension. (This is almost
|
|
|
281 always the correct value, because character sets with dimension 2
|
|
|
282 are usually ideograph character sets, which need two columns to
|
|
|
283 display the intricate ideographs.)
|
|
|
284 @item direction
|
|
|
285 A symbol, either @code{l2r} (left-to-right) or @code{r2l}
|
|
|
286 (right-to-left). Defaults to @code{l2r}. This specifies the
|
|
|
287 direction that the text should be displayed in, and will be
|
|
|
288 left-to-right for most charsets but right-to-left for Hebrew
|
|
|
289 and Arabic. (Right-to-left display is not currently implemented.)
|
|
|
290 @item final
|
|
|
291 Final byte of the standard ISO 2022 escape sequence designating this
|
|
|
292 charset. Must be supplied. Each combination of (@var{dimension},
|
|
|
293 @var{chars}) defines a separate namespace for final bytes, and each
|
|
|
294 charset within a particular namespace must have a different final byte.
|
|
|
295 Note that ISO 2022 restricts the final byte to the range 0x30 - 0x7E if
|
|
|
296 dimension == 1, and 0x30 - 0x5F if dimension == 2. Note also that final
|
|
|
297 bytes in the range 0x30 - 0x3F are reserved for user-defined (not
|
|
|
298 official) character sets. For more information on ISO 2022, see @ref{Coding
|
|
|
299 Systems}.
|
|
|
300 @item graphic
|
|
|
301 0 (use left half of font on output) or 1 (use right half of font on
|
|
|
302 output). Defaults to 0. This specifies how to convert the position
|
|
|
303 codes that index a character in a character set into an index into the
|
|
|
304 font used to display the character set. With @code{graphic} set to 0,
|
|
|
305 position codes 33 through 126 map to font indices 33 through 126; with
|
|
|
306 it set to 1, position codes 33 through 126 map to font indices 161
|
|
|
307 through 254 (i.e. the same number but with the high bit set). For
|
|
|
308 example, for a font whose registry is ISO8859-1, the left half of the
|
|
|
309 font (octets 0x20 - 0x7F) is the @code{ascii} charset, while the right
|
|
|
310 half (octets 0xA0 - 0xFF) is the @code{latin-iso8859-1} charset.
|
|
|
311 @item ccl-program
|
|
|
312 A compiled CCL program used to convert a character in this charset into
|
|
|
313 an index into the font. This is in addition to the @code{graphic}
|
|
|
314 property. If a CCL program is defined, the position codes of a
|
|
|
315 character will first be processed according to @code{graphic} and
|
|
|
316 then passed through the CCL program, with the resulting values used
|
|
|
317 to index the font.
|
|
|
318
|
|
442
|
319 This is used, for example, in the Big5 character set (used in Taiwan).
|
|
428
|
320 This character set is not ISO-2022-compliant, and its size (94x157) does
|
|
|
321 not fit within the maximum 96x96 size of ISO-2022-compliant character
|
|
|
322 sets. As a result, XEmacs/MULE splits it (in a rather complex fashion,
|
|
|
323 so as to group the most commonly used characters together) into two
|
|
|
324 charset objects (@code{big5-1} and @code{big5-2}), each of size 94x94,
|
|
|
325 and each charset object uses a CCL program to convert the modified
|
|
|
326 position codes back into standard Big5 indices to retrieve a character
|
|
|
327 from a Big5 font.
|
|
|
328 @end table
|
|
|
329
|
|
442
|
330 Most of the above properties can only be set when the charset is
|
|
|
331 initialized, and cannot be changed later.
|
|
|
332 @xref{Charset Property Functions}.
|
|
428
|
333
|
|
442
|
334 @node Basic Charset Functions, Charset Property Functions, Charset Properties, Charsets
|
|
428
|
335 @subsection Basic Charset Functions
|
|
|
336
|
|
|
337 @defun find-charset charset-or-name
|
|
|
338 This function retrieves the charset of the given name. If
|
|
|
339 @var{charset-or-name} is a charset object, it is simply returned.
|
|
|
340 Otherwise, @var{charset-or-name} should be a symbol. If there is no
|
|
|
341 such charset, @code{nil} is returned. Otherwise the associated charset
|
|
|
342 object is returned.
|
|
|
343 @end defun
|
|
|
344
|
|
|
345 @defun get-charset name
|
|
|
346 This function retrieves the charset of the given name. Same as
|
|
|
347 @code{find-charset} except an error is signalled if there is no such
|
|
|
348 charset instead of returning @code{nil}.
|
|
|
349 @end defun
|
|
|
350
|
|
|
351 @defun charset-list
|
|
|
352 This function returns a list of the names of all defined charsets.
|
|
|
353 @end defun
|
|
|
354
|
|
|
355 @defun make-charset name doc-string props
|
|
|
356 This function defines a new character set. This function is for use
|
|
442
|
357 with MULE support. @var{name} is a symbol, the name by which the
|
|
428
|
358 character set is normally referred. @var{doc-string} is a string
|
|
|
359 describing the character set. @var{props} is a property list,
|
|
|
360 describing the specific nature of the character set. The recognized
|
|
|
361 properties are @code{registry}, @code{dimension}, @code{columns},
|
|
|
362 @code{chars}, @code{final}, @code{graphic}, @code{direction}, and
|
|
|
363 @code{ccl-program}, as previously described.
|
|
|
364 @end defun
|
|
|
365
|
|
|
366 @defun make-reverse-direction-charset charset new-name
|
|
|
367 This function makes a charset equivalent to @var{charset} but which goes
|
|
|
368 in the opposite direction. @var{new-name} is the name of the new
|
|
|
369 charset. The new charset is returned.
|
|
|
370 @end defun
|
|
|
371
|
|
|
372 @defun charset-from-attributes dimension chars final &optional direction
|
|
|
373 This function returns a charset with the given @var{dimension},
|
|
|
374 @var{chars}, @var{final}, and @var{direction}. If @var{direction} is
|
|
|
375 omitted, both directions will be checked (left-to-right will be returned
|
|
|
376 if character sets exist for both directions).
|
|
|
377 @end defun
|
|
|
378
|
|
|
379 @defun charset-reverse-direction-charset charset
|
|
|
380 This function returns the charset (if any) with the same dimension,
|
|
|
381 number of characters, and final byte as @var{charset}, but which is
|
|
|
382 displayed in the opposite direction.
|
|
|
383 @end defun
|
|
|
384
|
|
442
|
385 @node Charset Property Functions, Predefined Charsets, Basic Charset Functions, Charsets
|
|
428
|
386 @subsection Charset Property Functions
|
|
|
387
|
|
442
|
388 All of these functions accept either a charset name or charset object.
|
|
428
|
389
|
|
|
390 @defun charset-property charset prop
|
|
|
391 This function returns property @var{prop} of @var{charset}.
|
|
|
392 @xref{Charset Properties}.
|
|
|
393 @end defun
|
|
|
394
|
|
442
|
395 Convenience functions are also provided for retrieving individual
|
|
428
|
396 properties of a charset.
|
|
|
397
|
|
|
398 @defun charset-name charset
|
|
|
399 This function returns the name of @var{charset}. This will be a symbol.
|
|
|
400 @end defun
|
|
|
401
|
|
444
|
402 @defun charset-description charset
|
|
|
403 This function returns the documentation string of @var{charset}.
|
|
428
|
404 @end defun
|
|
|
405
|
|
|
406 @defun charset-registry charset
|
|
|
407 This function returns the registry of @var{charset}.
|
|
|
408 @end defun
|
|
|
409
|
|
|
410 @defun charset-dimension charset
|
|
|
411 This function returns the dimension of @var{charset}.
|
|
|
412 @end defun
|
|
|
413
|
|
|
414 @defun charset-chars charset
|
|
|
415 This function returns the number of characters per dimension of
|
|
|
416 @var{charset}.
|
|
|
417 @end defun
|
|
|
418
|
|
444
|
419 @defun charset-width charset
|
|
428
|
420 This function returns the number of display columns per character (in
|
|
|
421 TTY mode) of @var{charset}.
|
|
|
422 @end defun
|
|
|
423
|
|
|
424 @defun charset-direction charset
|
|
440
|
425 This function returns the display direction of @var{charset}---either
|
|
428
|
426 @code{l2r} or @code{r2l}.
|
|
|
427 @end defun
|
|
|
428
|
|
444
|
429 @defun charset-iso-final-char charset
|
|
428
|
430 This function returns the final byte of the ISO 2022 escape sequence
|
|
|
431 designating @var{charset}.
|
|
|
432 @end defun
|
|
|
433
|
|
444
|
434 @defun charset-iso-graphic-plane charset
|
|
428
|
435 This function returns either 0 or 1, depending on whether the position
|
|
|
436 codes of characters in @var{charset} map to the left or right half
|
|
|
437 of their font, respectively.
|
|
|
438 @end defun
|
|
|
439
|
|
|
440 @defun charset-ccl-program charset
|
|
|
441 This function returns the CCL program, if any, for converting
|
|
|
442 position codes of characters in @var{charset} into font indices.
|
|
|
443 @end defun
|
|
|
444
|
|
442
|
445 The only property of a charset that can currently be set after
|
|
428
|
446 the charset has been created is the CCL program.
|
|
|
447
|
|
|
448 @defun set-charset-ccl-program charset ccl-program
|
|
|
449 This function sets the @code{ccl-program} property of @var{charset} to
|
|
|
450 @var{ccl-program}.
|
|
|
451 @end defun
|
|
|
452
|
|
442
|
453 @node Predefined Charsets, , Charset Property Functions, Charsets
|
|
428
|
454 @subsection Predefined Charsets
|
|
|
455
|
|
442
|
456 The following charsets are predefined in the C code.
|
|
428
|
457
|
|
|
458 @example
|
|
|
459 Name Type Fi Gr Dir Registry
|
|
|
460 --------------------------------------------------------------
|
|
|
461 ascii 94 B 0 l2r ISO8859-1
|
|
|
462 control-1 94 0 l2r ---
|
|
|
463 latin-iso8859-1 94 A 1 l2r ISO8859-1
|
|
|
464 latin-iso8859-2 96 B 1 l2r ISO8859-2
|
|
|
465 latin-iso8859-3 96 C 1 l2r ISO8859-3
|
|
|
466 latin-iso8859-4 96 D 1 l2r ISO8859-4
|
|
|
467 cyrillic-iso8859-5 96 L 1 l2r ISO8859-5
|
|
|
468 arabic-iso8859-6 96 G 1 r2l ISO8859-6
|
|
|
469 greek-iso8859-7 96 F 1 l2r ISO8859-7
|
|
|
470 hebrew-iso8859-8 96 H 1 r2l ISO8859-8
|
|
|
471 latin-iso8859-9 96 M 1 l2r ISO8859-9
|
|
|
472 thai-tis620 96 T 1 l2r TIS620
|
|
|
473 katakana-jisx0201 94 I 1 l2r JISX0201.1976
|
|
|
474 latin-jisx0201 94 J 0 l2r JISX0201.1976
|
|
|
475 japanese-jisx0208-1978 94x94 @@ 0 l2r JISX0208.1978
|
|
|
476 japanese-jisx0208 94x94 B 0 l2r JISX0208.19(83|90)
|
|
|
477 japanese-jisx0212 94x94 D 0 l2r JISX0212
|
|
|
478 chinese-gb2312 94x94 A 0 l2r GB2312
|
|
|
479 chinese-cns11643-1 94x94 G 0 l2r CNS11643.1
|
|
|
480 chinese-cns11643-2 94x94 H 0 l2r CNS11643.2
|
|
|
481 chinese-big5-1 94x94 0 0 l2r Big5
|
|
|
482 chinese-big5-2 94x94 1 0 l2r Big5
|
|
|
483 korean-ksc5601 94x94 C 0 l2r KSC5601
|
|
|
484 composite 96x96 0 l2r ---
|
|
|
485 @end example
|
|
|
486
|
|
442
|
487 The following charsets are predefined in the Lisp code.
|
|
428
|
488
|
|
|
489 @example
|
|
|
490 Name Type Fi Gr Dir Registry
|
|
|
491 --------------------------------------------------------------
|
|
|
492 arabic-digit 94 2 0 l2r MuleArabic-0
|
|
|
493 arabic-1-column 94 3 0 r2l MuleArabic-1
|
|
|
494 arabic-2-column 94 4 0 r2l MuleArabic-2
|
|
|
495 sisheng 94 0 0 l2r sisheng_cwnn\|OMRON_UDC_ZH
|
|
|
496 chinese-cns11643-3 94x94 I 0 l2r CNS11643.1
|
|
|
497 chinese-cns11643-4 94x94 J 0 l2r CNS11643.1
|
|
|
498 chinese-cns11643-5 94x94 K 0 l2r CNS11643.1
|
|
|
499 chinese-cns11643-6 94x94 L 0 l2r CNS11643.1
|
|
|
500 chinese-cns11643-7 94x94 M 0 l2r CNS11643.1
|
|
|
501 ethiopic 94x94 2 0 l2r Ethio
|
|
|
502 ascii-r2l 94 B 0 r2l ISO8859-1
|
|
|
503 ipa 96 0 1 l2r MuleIPA
|
|
|
504 vietnamese-lower 96 1 1 l2r VISCII1.1
|
|
|
505 vietnamese-upper 96 2 1 l2r VISCII1.1
|
|
|
506 @end example
|
|
|
507
|
|
|
508 For all of the above charsets, the dimension and number of columns are
|
|
|
509 the same.
|
|
|
510
|
|
442
|
511 Note that ASCII, Control-1, and Composite are handled specially.
|
|
428
|
512 This is why some of the fields are blank; and some of the filled-in
|
|
|
513 fields (e.g. the type) are not really accurate.
|
|
|
514
|
|
442
|
515 @node MULE Characters, Composite Characters, Charsets, MULE
|
|
428
|
516 @section MULE Characters
|
|
|
517
|
|
|
518 @defun make-char charset arg1 &optional arg2
|
|
|
519 This function makes a multi-byte character from @var{charset} and octets
|
|
|
520 @var{arg1} and @var{arg2}.
|
|
|
521 @end defun
|
|
|
522
|
|
444
|
523 @defun char-charset character
|
|
|
524 This function returns the character set of char @var{character}.
|
|
428
|
525 @end defun
|
|
|
526
|
|
444
|
527 @defun char-octet character &optional n
|
|
428
|
528 This function returns the octet (i.e. position code) numbered @var{n}
|
|
444
|
529 (should be 0 or 1) of char @var{character}. @var{n} defaults to 0 if omitted.
|
|
428
|
530 @end defun
|
|
|
531
|
|
|
532 @defun find-charset-region start end &optional buffer
|
|
|
533 This function returns a list of the charsets in the region between
|
|
|
534 @var{start} and @var{end}. @var{buffer} defaults to the current buffer
|
|
|
535 if omitted.
|
|
|
536 @end defun
|
|
|
537
|
|
|
538 @defun find-charset-string string
|
|
|
539 This function returns a list of the charsets in @var{string}.
|
|
|
540 @end defun
|
|
|
541
|
|
442
|
542 @node Composite Characters, Coding Systems, MULE Characters, MULE
|
|
428
|
543 @section Composite Characters
|
|
|
544
|
|
442
|
545 Composite characters are not yet completely implemented.
|
|
428
|
546
|
|
|
547 @defun make-composite-char string
|
|
|
548 This function converts a string into a single composite character. The
|
|
|
549 character is the result of overstriking all the characters in the
|
|
|
550 string.
|
|
|
551 @end defun
|
|
|
552
|
|
444
|
553 @defun composite-char-string character
|
|
428
|
554 This function returns a string of the characters comprising a composite
|
|
|
555 character.
|
|
|
556 @end defun
|
|
|
557
|
|
|
558 @defun compose-region start end &optional buffer
|
|
|
559 This function composes the characters in the region from @var{start} to
|
|
|
560 @var{end} in @var{buffer} into one composite character. The composite
|
|
|
561 character replaces the composed characters. @var{buffer} defaults to
|
|
|
562 the current buffer if omitted.
|
|
|
563 @end defun
|
|
|
564
|
|
|
565 @defun decompose-region start end &optional buffer
|
|
|
566 This function decomposes any composite characters in the region from
|
|
|
567 @var{start} to @var{end} in @var{buffer}. This converts each composite
|
|
|
568 character into one or more characters, the individual characters out of
|
|
|
569 which the composite character was formed. Non-composite characters are
|
|
|
570 left as-is. @var{buffer} defaults to the current buffer if omitted.
|
|
|
571 @end defun
|
|
|
572
|
|
442
|
573 @node Coding Systems, CCL, Composite Characters, MULE
|
|
|
574 @section Coding Systems
|
|
|
575
|
|
|
576 A coding system is an object that defines how text containing multiple
|
|
|
577 character sets is encoded into a stream of (typically 8-bit) bytes. The
|
|
|
578 coding system is used to decode the stream into a series of characters
|
|
|
579 (which may be from multiple charsets) when the text is read from a file
|
|
|
580 or process, and is used to encode the text back into the same format
|
|
|
581 when it is written out to a file or process.
|
|
|
582
|
|
|
583 For example, many ISO-2022-compliant coding systems (such as Compound
|
|
|
584 Text, which is used for inter-client data under the X Window System) use
|
|
|
585 escape sequences to switch between different charsets -- Japanese Kanji,
|
|
|
586 for example, is invoked with @samp{ESC $ ( B}; ASCII is invoked with
|
|
|
587 @samp{ESC ( B}; and Cyrillic is invoked with @samp{ESC - L}. See
|
|
|
588 @code{make-coding-system} for more information.
|
|
|
589
|
|
|
590 Coding systems are normally identified using a symbol, and the symbol is
|
|
|
591 accepted in place of the actual coding system object whenever a coding
|
|
|
592 system is called for. (This is similar to how faces and charsets work.)
|
|
|
593
|
|
|
594 @defun coding-system-p object
|
|
|
595 This function returns non-@code{nil} if @var{object} is a coding system.
|
|
|
596 @end defun
|
|
428
|
597
|
|
442
|
598 @menu
|
|
|
599 * Coding System Types:: Classifying coding systems.
|
|
|
600 * ISO 2022:: An international standard for
|
|
|
601 charsets and encodings.
|
|
|
602 * EOL Conversion:: Dealing with different ways of denoting
|
|
|
603 the end of a line.
|
|
|
604 * Coding System Properties:: Properties of a coding system.
|
|
|
605 * Basic Coding System Functions:: Working with coding systems.
|
|
|
606 * Coding System Property Functions:: Retrieving a coding system's properties.
|
|
|
607 * Encoding and Decoding Text:: Encoding and decoding text.
|
|
|
608 * Detection of Textual Encoding:: Determining how text is encoded.
|
|
|
609 * Big5 and Shift-JIS Functions:: Special functions for these non-standard
|
|
|
610 encodings.
|
|
|
611 * Predefined Coding Systems:: Coding systems implemented by MULE.
|
|
|
612 @end menu
|
|
428
|
613
|
|
442
|
614 @node Coding System Types, ISO 2022, , Coding Systems
|
|
|
615 @subsection Coding System Types
|
|
|
616
|
|
|
617 The coding system type determines the basic algorithm XEmacs will use to
|
|
|
618 decode or encode a data stream. Character encodings will be converted
|
|
|
619 to the MULE encoding, escape sequences processed, and newline sequences
|
|
|
620 converted to XEmacs's internal representation. There are three basic
|
|
|
621 classes of coding system type: no-conversion, ISO-2022, and special.
|
|
|
622
|
|
|
623 No conversion allows you to look at the file's internal representation.
|
|
|
624 Since XEmacs is basically a text editor, "no conversion" does convert
|
|
|
625 newline conventions by default. (Use the 'binary coding-system if this
|
|
|
626 is not desired.)
|
|
428
|
627
|
|
442
|
628 ISO 2022 (@pxref{ISO 2022}) is the basic international standard regulating
|
|
|
629 use of "coded character sets for the exchange of data", ie, text
|
|
|
630 streams. ISO 2022 contains functions that make it possible to encode
|
|
|
631 text streams to comply with restrictions of the Internet mail system and
|
|
|
632 de facto restrictions of most file systems (eg, use of the separator
|
|
|
633 character in file names). Coding systems which are not ISO 2022
|
|
|
634 conformant can be difficult to handle. Perhaps more important, they are
|
|
|
635 not adaptable to multilingual information interchange, with the obvious
|
|
|
636 exception of ISO 10646 (Unicode). (Unicode is partially supported by
|
|
|
637 XEmacs with the addition of the Lisp package ucs-conv.)
|
|
|
638
|
|
|
639 The special class of coding systems includes automatic detection, CCL (a
|
|
|
640 "little language" embedded as an interpreter, useful for translating
|
|
|
641 between variants of a single character set), non-ISO-2022-conformant
|
|
|
642 encodings like Unicode, Shift JIS, and Big5, and MULE internal coding.
|
|
|
643 (NB: this list is based on XEmacs 21.2. Terminology may vary slightly
|
|
|
644 for other versions of XEmacs and for GNU Emacs 20.)
|
|
|
645
|
|
|
646 @table @code
|
|
|
647 @item no-conversion
|
|
|
648 No conversion, for binary files, and a few special cases of non-ISO-2022
|
|
|
649 coding systems where conversion is done by hook functions (usually
|
|
|
650 implemented in CCL). On output, graphic characters that are not in
|
|
|
651 ASCII or Latin-1 will be replaced by a @samp{?}. (For a
|
|
|
652 no-conversion-encoded buffer, these characters will only be present if
|
|
|
653 you explicitly insert them.)
|
|
|
654 @item iso2022
|
|
|
655 Any ISO-2022-compliant encoding. Among others, this includes JIS (the
|
|
|
656 Japanese encoding commonly used for e-mail), national variants of EUC
|
|
|
657 (the standard Unix encoding for Japanese and other languages), and
|
|
|
658 Compound Text (an encoding used in X11). You can specify more specific
|
|
|
659 information about the conversion with the @var{flags} argument.
|
|
|
660 @item ucs-4
|
|
|
661 ISO 10646 UCS-4 encoding. A 31-bit fixed-width superset of Unicode.
|
|
|
662 @item utf-8
|
|
|
663 ISO 10646 UTF-8 encoding. A ``file system safe'' transformation format
|
|
|
664 that can be used with both UCS-4 and Unicode.
|
|
|
665 @item undecided
|
|
|
666 Automatic conversion. XEmacs attempts to detect the coding system used
|
|
|
667 in the file.
|
|
|
668 @item shift-jis
|
|
|
669 Shift-JIS (a Japanese encoding commonly used in PC operating systems).
|
|
|
670 @item big5
|
|
|
671 Big5 (the encoding commonly used for Taiwanese).
|
|
|
672 @item ccl
|
|
|
673 The conversion is performed using a user-written pseudo-code program.
|
|
|
674 CCL (Code Conversion Language) is the name of this pseudo-code. For
|
|
|
675 example, CCL is used to map KOI8-R characters (an encoding for Russian
|
|
|
676 Cyrillic) to ISO8859-5 (the form used internally by MULE).
|
|
|
677 @item internal
|
|
|
678 Write out or read in the raw contents of the memory representing the
|
|
|
679 buffer's text. This is primarily useful for debugging purposes, and is
|
|
|
680 only enabled when XEmacs has been compiled with @code{DEBUG_XEMACS} set
|
|
|
681 (the @samp{--debug} configure option). @strong{Warning}: Reading in a
|
|
|
682 file using @code{internal} conversion can result in an internal
|
|
|
683 inconsistency in the memory representing a buffer's text, which will
|
|
|
684 produce unpredictable results and may cause XEmacs to crash. Under
|
|
|
685 normal circumstances you should never use @code{internal} conversion.
|
|
428
|
686 @end table
|
|
|
687
|
|
442
|
688 @node ISO 2022, EOL Conversion, Coding System Types, Coding Systems
|
|
|
689 @section ISO 2022
|
|
|
690
|
|
|
691 This section briefly describes the ISO 2022 encoding standard. A more
|
|
|
692 thorough treatment is available in the original document of ISO
|
|
|
693 2022 as well as various national standards (such as JIS X 0202).
|
|
428
|
694
|
|
442
|
695 Character sets (@dfn{charsets}) are classified into the following four
|
|
|
696 categories, according to the number of characters in the charset:
|
|
|
697 94-charset, 96-charset, 94x94-charset, and 96x96-charset. This means
|
|
|
698 that although an ISO 2022 coding system may have variable width
|
|
|
699 characters, each charset used is fixed-width (in contrast to the MULE
|
|
|
700 character set and UTF-8, for example).
|
|
|
701
|
|
|
702 ISO 2022 provides for switching between character sets via escape
|
|
|
703 sequences. This switching is somewhat complicated, because ISO 2022
|
|
|
704 provides for both legacy applications like Internet mail that accept
|
|
444
|
705 only 7 significant bits in some contexts (RFC 822 headers, for example),
|
|
442
|
706 and more modern "8-bit clean" applications. It also provides for
|
|
|
707 compact and transparent representation of languages like Japanese which
|
|
|
708 mix ASCII and a national script (even outside of computer programs).
|
|
428
|
709
|
|
442
|
710 First, ISO 2022 codified prevailing practice by dividing the code space
|
|
|
711 into "control" and "graphic" regions. The code points 0x00-0x1F and
|
|
|
712 0x80-0x9F are reserved for "control characters", while "graphic
|
|
|
713 characters" must be assigned to code points in the regions 0x20-0x7F and
|
|
|
714 0xA0-0xFF. The positions 0x20 and 0x7F are special, and under some
|
|
|
715 circumstances must be assigned the graphic character "ASCII SPACE" and
|
|
|
716 the control character "ASCII DEL" respectively.
|
|
428
|
717
|
|
442
|
718 The various regions are given the name C0 (0x00-0x1F), GL (0x20-0x7F),
|
|
|
719 C1 (0x80-0x9F), and GR (0xA0-0xFF). GL and GR stand for "graphic left"
|
|
|
720 and "graphic right", respectively, because of the standard method of
|
|
|
721 displaying graphic character sets in tables with the high byte indexing
|
|
444
|
722 columns and the low byte indexing rows. I don't find it very intuitive,
|
|
442
|
723 but these are called "registers".
|
|
|
724
|
|
|
725 An ISO 2022-conformant encoding for a graphic character set must use a
|
|
|
726 fixed number of bytes per character, and the values must fit into a
|
|
|
727 single register; that is, each byte must range over either 0x20-0x7F, or
|
|
|
728 0xA0-0xFF. It is not allowed to extend the range of the repertoire of a
|
|
|
729 character set by using both ranges at the same. This is why a standard
|
|
|
730 character set such as ISO 8859-1 is actually considered by ISO 2022 to
|
|
|
731 be an aggregation of two character sets, ASCII and LATIN-1, and why it
|
|
|
732 is technically incorrect to refer to ISO 8859-1 as "Latin 1". Also, a
|
|
|
733 single character's bytes must all be drawn from the same register; this
|
|
|
734 is why Shift JIS (for Japanese) and Big 5 (for Chinese) are not ISO
|
|
|
735 2022-compatible encodings.
|
|
428
|
736
|
|
442
|
737 The reason for this restriction becomes clear when you attempt to define
|
|
|
738 an efficient, robust encoding for a language like Japanese. Like ISO
|
|
|
739 8859, Japanese encodings are aggregations of several character sets. In
|
|
|
740 practice, the vast majority of characters are drawn from the "JIS Roman"
|
|
|
741 character set (a derivative of ASCII; it won't hurt to think of it as
|
|
|
742 ASCII) and the JIS X 0208 standard "basic Japanese" character set
|
|
|
743 including not only ideographic characters ("kanji") but syllabic
|
|
|
744 Japanese characters ("kana"), a wide variety of symbols, and many
|
|
|
745 alphabetic characters (Roman, Greek, and Cyrillic) as well. Although
|
|
|
746 JIS X 0208 includes the whole Roman alphabet, as a 2-byte code it is not
|
|
|
747 suited to programming; thus the inclusion of ASCII in the standard
|
|
|
748 Japanese encodings.
|
|
428
|
749
|
|
442
|
750 For normal Japanese text such as in newspapers, a broad repertoire of
|
|
|
751 approximately 3000 characters is used. Evidently this won't fit into
|
|
|
752 one byte; two must be used. But much of the text processed by Japanese
|
|
|
753 computers is computer source code, nearly all of which is ASCII. A not
|
|
|
754 insignificant portion of ordinary text is English (as such or as
|
|
|
755 borrowed Japanese vocabulary) or other languages which can represented
|
|
|
756 at least approximately in ASCII, as well. It seems reasonable then to
|
|
|
757 represent ASCII in one byte, and JIS X 0208 in two. And this is exactly
|
|
|
758 what the Extended Unix Code for Japanese (EUC-JP) does. ASCII is
|
|
|
759 invoked to the GL register, and JIS X 0208 is invoked to the GR
|
|
|
760 register. Thus, each byte can be tested for its character set by
|
|
|
761 looking at the high bit; if set, it is Japanese, if clear, it is ASCII.
|
|
|
762 Furthermore, since control characters like newline can never be part of
|
|
|
763 a graphic character, even in the case of corruption in transmission the
|
|
|
764 stream will be resynchronized at every line break, on the order of 60-80
|
|
|
765 bytes. This coding system requires no escape sequences or special
|
|
|
766 control codes to represent 99.9% of all Japanese text.
|
|
428
|
767
|
|
442
|
768 Note carefully the distinction between the character sets (ASCII and JIS
|
|
|
769 X 0208), the encoding (EUC-JP), and the coding system (ISO 2022). The
|
|
|
770 JIS X 0208 character set is used in three different encodings for
|
|
|
771 Japanese, but in ISO-2022-JP it is invoked into GL (so the high bit is
|
|
|
772 always clear), in EUC-JP it is invoked into GR (setting the high bit in
|
|
|
773 the process), and in Shift JIS the high bit may be set or reset, and the
|
|
|
774 significant bits are shifted within the 16-bit character so that the two
|
|
|
775 main character sets can coexist with a third (the "halfwidth katakana"
|
|
|
776 of JIS X 0201). As the name implies, the ISO-2022-JP encoding is also a
|
|
|
777 version of the ISO-2022 coding system.
|
|
428
|
778
|
|
442
|
779 In order to systematically treat subsidiary character sets (like the
|
|
|
780 "halfwidth katakana" already mentioned, and the "supplementary kanji" of
|
|
|
781 JIS X 0212), four further registers are defined: G0, G1, G2, and G3.
|
|
|
782 Unlike GL and GR, they are not logically distinguished by internal
|
|
|
783 format. Instead, the process of "invocation" mentioned earlier is
|
|
|
784 broken into two steps: first, a character set is @dfn{designated} to one
|
|
|
785 of the registers G0-G3 by use of an @dfn{escape sequence} of the form:
|
|
428
|
786
|
|
|
787 @example
|
|
440
|
788 ESC [@var{I}] @var{I} @var{F}
|
|
428
|
789 @end example
|
|
|
790
|
|
442
|
791 where @var{I} is an intermediate character or characters in the range
|
|
|
792 0x20 - 0x3F, and @var{F}, from the range 0x30-0x7Fm is the final
|
|
|
793 character identifying this charset. (Final characters in the range
|
|
|
794 0x30-0x3F are reserved for private use and will never have a publicly
|
|
|
795 registered meaning.)
|
|
|
796
|
|
|
797 Then that register is @dfn{invoked} to either GL or GR, either
|
|
|
798 automatically (designations to G0 normally involve invocation to GL as
|
|
|
799 well), or by use of shifting (affecting only the following character in
|
|
|
800 the data stream) or locking (effective until the next designation or
|
|
|
801 locking) control sequences. An encoding conformant to ISO 2022 is
|
|
|
802 typically defined by designating the initial contents of the G0-G3
|
|
|
803 registers, specifying an 7 or 8 bit environment, and specifying whether
|
|
|
804 further designations will be recognized.
|
|
|
805
|
|
|
806 Some examples of character sets and the registered final characters
|
|
|
807 @var{F} used to designate them:
|
|
428
|
808
|
|
442
|
809 @need 1000
|
|
|
810 @table @asis
|
|
|
811 @item 94-charset
|
|
|
812 ASCII (B), left (J) and right (I) half of JIS X 0201, ...
|
|
|
813 @item 96-charset
|
|
|
814 Latin-1 (A), Latin-2 (B), Latin-3 (C), ...
|
|
|
815 @item 94x94-charset
|
|
|
816 GB2312 (A), JIS X 0208 (B), KSC5601 (C), ...
|
|
|
817 @item 96x96-charset
|
|
|
818 none for the moment
|
|
|
819 @end table
|
|
|
820
|
|
|
821 The meanings of the various characters in these sequences, where not
|
|
|
822 specified by the ISO 2022 standard (such as the ESC character), are
|
|
|
823 assigned by @dfn{ECMA}, the European Computer Manufacturers Association.
|
|
|
824
|
|
|
825 The meaning of intermediate characters are:
|
|
428
|
826
|
|
|
827 @example
|
|
|
828 @group
|
|
440
|
829 $ [0x24]: indicate charset of dimension 2 (94x94 or 96x96).
|
|
|
830 ( [0x28]: designate to G0 a 94-charset whose final byte is @var{F}.
|
|
|
831 ) [0x29]: designate to G1 a 94-charset whose final byte is @var{F}.
|
|
|
832 * [0x2A]: designate to G2 a 94-charset whose final byte is @var{F}.
|
|
|
833 + [0x2B]: designate to G3 a 94-charset whose final byte is @var{F}.
|
|
442
|
834 , [0x2C]: designate to G0 a 96-charset whose final byte is @var{F}.
|
|
440
|
835 - [0x2D]: designate to G1 a 96-charset whose final byte is @var{F}.
|
|
|
836 . [0x2E]: designate to G2 a 96-charset whose final byte is @var{F}.
|
|
|
837 / [0x2F]: designate to G3 a 96-charset whose final byte is @var{F}.
|
|
428
|
838 @end group
|
|
|
839 @end example
|
|
|
840
|
|
442
|
841 The comma may be used in files read and written only by MULE, as a MULE
|
|
|
842 extension, but this is illegal in ISO 2022. (The reason is that in ISO
|
|
|
843 2022 G0 must be a 94-member character set, with 0x20 assigned the value
|
|
|
844 SPACE, and 0x7F assigned the value DEL.)
|
|
428
|
845
|
|
442
|
846 Here are examples of designations:
|
|
428
|
847
|
|
|
848 @example
|
|
|
849 @group
|
|
440
|
850 ESC ( B : designate to G0 ASCII
|
|
|
851 ESC - A : designate to G1 Latin-1
|
|
|
852 ESC $ ( A or ESC $ A : designate to G0 GB2312
|
|
|
853 ESC $ ( B or ESC $ B : designate to G0 JISX0208
|
|
|
854 ESC $ ) C : designate to G1 KSC5601
|
|
428
|
855 @end group
|
|
|
856 @end example
|
|
|
857
|
|
442
|
858 (The short forms used to designate GB2312 and JIS X 0208 are for
|
|
|
859 backwards compatibility; the long forms are preferred.)
|
|
|
860
|
|
|
861 To use a charset designated to G2 or G3, and to use a charset designated
|
|
428
|
862 to G1 in a 7-bit environment, you must explicitly invoke G1, G2, or G3
|
|
|
863 into GL. There are two types of invocation, Locking Shift (forever) and
|
|
|
864 Single Shift (one character only).
|
|
|
865
|
|
442
|
866 Locking Shift is done as follows:
|
|
428
|
867
|
|
|
868 @example
|
|
440
|
869 LS0 or SI (0x0F): invoke G0 into GL
|
|
|
870 LS1 or SO (0x0E): invoke G1 into GL
|
|
|
871 LS2: invoke G2 into GL
|
|
|
872 LS3: invoke G3 into GL
|
|
|
873 LS1R: invoke G1 into GR
|
|
|
874 LS2R: invoke G2 into GR
|
|
|
875 LS3R: invoke G3 into GR
|
|
428
|
876 @end example
|
|
|
877
|
|
442
|
878 Single Shift is done as follows:
|
|
428
|
879
|
|
|
880 @example
|
|
|
881 @group
|
|
440
|
882 SS2 or ESC N: invoke G2 into GL
|
|
|
883 SS3 or ESC O: invoke G3 into GL
|
|
428
|
884 @end group
|
|
|
885 @end example
|
|
|
886
|
|
442
|
887 The shift functions (such as LS1R and SS3) are represented by control
|
|
|
888 characters (from C1) in 8 bit environments and by escape sequences in 7
|
|
|
889 bit environments.
|
|
|
890
|
|
428
|
891 (#### Ben says: I think the above is slightly incorrect. It appears that
|
|
|
892 SS2 invokes G2 into GR and SS3 invokes G3 into GR, whereas ESC N and
|
|
444
|
893 ESC O behave as indicated. The above definitions will not parse
|
|
428
|
894 EUC-encoded text correctly, and it looks like the code in mule-coding.c
|
|
|
895 has similar problems.)
|
|
|
896
|
|
442
|
897 Evidently there are a lot of ISO-2022-compliant ways of encoding
|
|
|
898 multilingual text. Now, in the world, there exist many coding systems
|
|
|
899 such as X11's Compound Text, Japanese JUNET code, and so-called EUC
|
|
|
900 (Extended UNIX Code); all of these are variants of ISO 2022.
|
|
428
|
901
|
|
442
|
902 In MULE, we characterize a version of ISO 2022 by the following
|
|
|
903 attributes:
|
|
428
|
904
|
|
|
905 @enumerate
|
|
|
906 @item
|
|
442
|
907 The character sets initially designated to G0 thru G3.
|
|
428
|
908 @item
|
|
442
|
909 Whether short form designations are allowed for Japanese and Chinese.
|
|
428
|
910 @item
|
|
442
|
911 Whether ASCII should be designated to G0 before control characters.
|
|
428
|
912 @item
|
|
442
|
913 Whether ASCII should be designated to G0 at the end of line.
|
|
428
|
914 @item
|
|
|
915 7-bit environment or 8-bit environment.
|
|
|
916 @item
|
|
442
|
917 Whether Locking Shifts are used or not.
|
|
428
|
918 @item
|
|
442
|
919 Whether to use ASCII or the variant JIS X 0201-1976-Roman.
|
|
428
|
920 @item
|
|
442
|
921 Whether to use JIS X 0208-1983 or the older version JIS X 0208-1976.
|
|
428
|
922 @end enumerate
|
|
|
923
|
|
|
924 (The last two are only for Japanese.)
|
|
|
925
|
|
442
|
926 By specifying these attributes, you can create any variant
|
|
428
|
927 of ISO 2022.
|
|
|
928
|
|
442
|
929 Here are several examples:
|
|
428
|
930
|
|
|
931 @example
|
|
|
932 @group
|
|
442
|
933 ISO-2022-JP -- Coding system used in Japanese email (RFC 1463 #### check).
|
|
440
|
934 1. G0 <- ASCII, G1..3 <- never used
|
|
|
935 2. Yes.
|
|
|
936 3. Yes.
|
|
|
937 4. Yes.
|
|
|
938 5. 7-bit environment
|
|
|
939 6. No.
|
|
|
940 7. Use ASCII
|
|
442
|
941 8. Use JIS X 0208-1983
|
|
428
|
942 @end group
|
|
|
943
|
|
|
944 @group
|
|
442
|
945 ctext -- X11 Compound Text
|
|
|
946 1. G0 <- ASCII, G1 <- Latin-1, G2,3 <- never used.
|
|
440
|
947 2. No.
|
|
|
948 3. No.
|
|
|
949 4. Yes.
|
|
442
|
950 5. 8-bit environment.
|
|
440
|
951 6. No.
|
|
442
|
952 7. Use ASCII.
|
|
|
953 8. Use JIS X 0208-1983.
|
|
428
|
954 @end group
|
|
|
955
|
|
|
956 @group
|
|
442
|
957 euc-china -- Chinese EUC. Often called the "GB encoding", but that is
|
|
|
958 technically incorrect.
|
|
|
959 1. G0 <- ASCII, G1 <- GB 2312, G2,3 <- never used.
|
|
440
|
960 2. No.
|
|
|
961 3. Yes.
|
|
|
962 4. Yes.
|
|
442
|
963 5. 8-bit environment.
|
|
440
|
964 6. No.
|
|
442
|
965 7. Use ASCII.
|
|
|
966 8. Use JIS X 0208-1983.
|
|
428
|
967 @end group
|
|
|
968
|
|
|
969 @group
|
|
442
|
970 ISO-2022-KR -- Coding system used in Korean email.
|
|
|
971 1. G0 <- ASCII, G1 <- KSC 5601, G2,3 <- never used.
|
|
440
|
972 2. No.
|
|
|
973 3. Yes.
|
|
|
974 4. Yes.
|
|
442
|
975 5. 7-bit environment.
|
|
440
|
976 6. Yes.
|
|
442
|
977 7. Use ASCII.
|
|
|
978 8. Use JIS X 0208-1983.
|
|
428
|
979 @end group
|
|
|
980 @end example
|
|
|
981
|
|
442
|
982 MULE creates all of these coding systems by default.
|
|
428
|
983
|
|
442
|
984 @node EOL Conversion, Coding System Properties, ISO 2022, Coding Systems
|
|
428
|
985 @subsection EOL Conversion
|
|
|
986
|
|
|
987 @table @code
|
|
|
988 @item nil
|
|
|
989 Automatically detect the end-of-line type (LF, CRLF, or CR). Also
|
|
|
990 generate subsidiary coding systems named @code{@var{name}-unix},
|
|
|
991 @code{@var{name}-dos}, and @code{@var{name}-mac}, that are identical to
|
|
|
992 this coding system but have an EOL-TYPE value of @code{lf}, @code{crlf},
|
|
|
993 and @code{cr}, respectively.
|
|
|
994 @item lf
|
|
|
995 The end of a line is marked externally using ASCII LF. Since this is
|
|
|
996 also the way that XEmacs represents an end-of-line internally,
|
|
|
997 specifying this option results in no end-of-line conversion. This is
|
|
|
998 the standard format for Unix text files.
|
|
|
999 @item crlf
|
|
|
1000 The end of a line is marked externally using ASCII CRLF. This is the
|
|
|
1001 standard format for MS-DOS text files.
|
|
|
1002 @item cr
|
|
|
1003 The end of a line is marked externally using ASCII CR. This is the
|
|
|
1004 standard format for Macintosh text files.
|
|
|
1005 @item t
|
|
|
1006 Automatically detect the end-of-line type but do not generate subsidiary
|
|
|
1007 coding systems. (This value is converted to @code{nil} when stored
|
|
|
1008 internally, and @code{coding-system-property} will return @code{nil}.)
|
|
|
1009 @end table
|
|
|
1010
|
|
442
|
1011 @node Coding System Properties, Basic Coding System Functions, EOL Conversion, Coding Systems
|
|
428
|
1012 @subsection Coding System Properties
|
|
|
1013
|
|
|
1014 @table @code
|
|
|
1015 @item mnemonic
|
|
|
1016 String to be displayed in the modeline when this coding system is
|
|
|
1017 active.
|
|
|
1018
|
|
|
1019 @item eol-type
|
|
|
1020 End-of-line conversion to be used. It should be one of the types
|
|
|
1021 listed in @ref{EOL Conversion}.
|
|
|
1022
|
|
442
|
1023 @item eol-lf
|
|
444
|
1024 The coding system which is the same as this one, except that it uses the
|
|
442
|
1025 Unix line-breaking convention.
|
|
|
1026
|
|
|
1027 @item eol-crlf
|
|
444
|
1028 The coding system which is the same as this one, except that it uses the
|
|
442
|
1029 DOS line-breaking convention.
|
|
|
1030
|
|
|
1031 @item eol-cr
|
|
444
|
1032 The coding system which is the same as this one, except that it uses the
|
|
442
|
1033 Macintosh line-breaking convention.
|
|
|
1034
|
|
428
|
1035 @item post-read-conversion
|
|
|
1036 Function called after a file has been read in, to perform the decoding.
|
|
444
|
1037 Called with two arguments, @var{start} and @var{end}, denoting a region of
|
|
428
|
1038 the current buffer to be decoded.
|
|
|
1039
|
|
|
1040 @item pre-write-conversion
|
|
|
1041 Function called before a file is written out, to perform the encoding.
|
|
444
|
1042 Called with two arguments, @var{start} and @var{end}, denoting a region of
|
|
428
|
1043 the current buffer to be encoded.
|
|
|
1044 @end table
|
|
|
1045
|
|
442
|
1046 The following additional properties are recognized if @var{type} is
|
|
428
|
1047 @code{iso2022}:
|
|
|
1048
|
|
|
1049 @table @code
|
|
|
1050 @item charset-g0
|
|
|
1051 @itemx charset-g1
|
|
|
1052 @itemx charset-g2
|
|
|
1053 @itemx charset-g3
|
|
|
1054 The character set initially designated to the G0 - G3 registers.
|
|
|
1055 The value should be one of
|
|
|
1056
|
|
|
1057 @itemize @bullet
|
|
|
1058 @item
|
|
|
1059 A charset object (designate that character set)
|
|
|
1060 @item
|
|
|
1061 @code{nil} (do not ever use this register)
|
|
|
1062 @item
|
|
|
1063 @code{t} (no character set is initially designated to the register, but
|
|
|
1064 may be later on; this automatically sets the corresponding
|
|
|
1065 @code{force-g*-on-output} property)
|
|
|
1066 @end itemize
|
|
|
1067
|
|
|
1068 @item force-g0-on-output
|
|
|
1069 @itemx force-g1-on-output
|
|
|
1070 @itemx force-g2-on-output
|
|
|
1071 @itemx force-g3-on-output
|
|
|
1072 If non-@code{nil}, send an explicit designation sequence on output
|
|
|
1073 before using the specified register.
|
|
|
1074
|
|
|
1075 @item short
|
|
|
1076 If non-@code{nil}, use the short forms @samp{ESC $ @@}, @samp{ESC $ A},
|
|
|
1077 and @samp{ESC $ B} on output in place of the full designation sequences
|
|
|
1078 @samp{ESC $ ( @@}, @samp{ESC $ ( A}, and @samp{ESC $ ( B}.
|
|
|
1079
|
|
|
1080 @item no-ascii-eol
|
|
|
1081 If non-@code{nil}, don't designate ASCII to G0 at each end of line on
|
|
|
1082 output. Setting this to non-@code{nil} also suppresses other
|
|
|
1083 state-resetting that normally happens at the end of a line.
|
|
|
1084
|
|
|
1085 @item no-ascii-cntl
|
|
|
1086 If non-@code{nil}, don't designate ASCII to G0 before control chars on
|
|
|
1087 output.
|
|
|
1088
|
|
|
1089 @item seven
|
|
|
1090 If non-@code{nil}, use 7-bit environment on output. Otherwise, use 8-bit
|
|
|
1091 environment.
|
|
|
1092
|
|
|
1093 @item lock-shift
|
|
|
1094 If non-@code{nil}, use locking-shift (SO/SI) instead of single-shift or
|
|
|
1095 designation by escape sequence.
|
|
|
1096
|
|
|
1097 @item no-iso6429
|
|
|
1098 If non-@code{nil}, don't use ISO6429's direction specification.
|
|
|
1099
|
|
|
1100 @item escape-quoted
|
|
444
|
1101 If non-@code{nil}, literal control characters that are the same as the
|
|
428
|
1102 beginning of a recognized ISO 2022 or ISO 6429 escape sequence (in
|
|
|
1103 particular, ESC (0x1B), SO (0x0E), SI (0x0F), SS2 (0x8E), SS3 (0x8F),
|
|
|
1104 and CSI (0x9B)) are ``quoted'' with an escape character so that they can
|
|
|
1105 be properly distinguished from an escape sequence. (Note that doing
|
|
|
1106 this results in a non-portable encoding.) This encoding flag is used for
|
|
|
1107 byte-compiled files. Note that ESC is a good choice for a quoting
|
|
|
1108 character because there are no escape sequences whose second byte is a
|
|
|
1109 character from the Control-0 or Control-1 character sets; this is
|
|
|
1110 explicitly disallowed by the ISO 2022 standard.
|
|
|
1111
|
|
|
1112 @item input-charset-conversion
|
|
|
1113 A list of conversion specifications, specifying conversion of characters
|
|
|
1114 in one charset to another when decoding is performed. Each
|
|
|
1115 specification is a list of two elements: the source charset, and the
|
|
|
1116 destination charset.
|
|
|
1117
|
|
|
1118 @item output-charset-conversion
|
|
|
1119 A list of conversion specifications, specifying conversion of characters
|
|
|
1120 in one charset to another when encoding is performed. The form of each
|
|
|
1121 specification is the same as for @code{input-charset-conversion}.
|
|
|
1122 @end table
|
|
|
1123
|
|
442
|
1124 The following additional properties are recognized (and required) if
|
|
428
|
1125 @var{type} is @code{ccl}:
|
|
|
1126
|
|
|
1127 @table @code
|
|
|
1128 @item decode
|
|
|
1129 CCL program used for decoding (converting to internal format).
|
|
|
1130
|
|
|
1131 @item encode
|
|
|
1132 CCL program used for encoding (converting to external format).
|
|
|
1133 @end table
|
|
|
1134
|
|
442
|
1135 The following properties are used internally: @var{eol-cr},
|
|
|
1136 @var{eol-crlf}, @var{eol-lf}, and @var{base}.
|
|
|
1137
|
|
|
1138 @node Basic Coding System Functions, Coding System Property Functions, Coding System Properties, Coding Systems
|
|
428
|
1139 @subsection Basic Coding System Functions
|
|
|
1140
|
|
|
1141 @defun find-coding-system coding-system-or-name
|
|
|
1142 This function retrieves the coding system of the given name.
|
|
|
1143
|
|
442
|
1144 If @var{coding-system-or-name} is a coding-system object, it is simply
|
|
428
|
1145 returned. Otherwise, @var{coding-system-or-name} should be a symbol.
|
|
|
1146 If there is no such coding system, @code{nil} is returned. Otherwise
|
|
|
1147 the associated coding system object is returned.
|
|
|
1148 @end defun
|
|
|
1149
|
|
|
1150 @defun get-coding-system name
|
|
|
1151 This function retrieves the coding system of the given name. Same as
|
|
|
1152 @code{find-coding-system} except an error is signalled if there is no
|
|
|
1153 such coding system instead of returning @code{nil}.
|
|
|
1154 @end defun
|
|
|
1155
|
|
|
1156 @defun coding-system-list
|
|
|
1157 This function returns a list of the names of all defined coding systems.
|
|
|
1158 @end defun
|
|
|
1159
|
|
|
1160 @defun coding-system-name coding-system
|
|
|
1161 This function returns the name of the given coding system.
|
|
|
1162 @end defun
|
|
|
1163
|
|
442
|
1164 @defun coding-system-base coding-system
|
|
|
1165 Returns the base coding system (undecided EOL convention)
|
|
|
1166 coding system.
|
|
|
1167 @end defun
|
|
|
1168
|
|
428
|
1169 @defun make-coding-system name type &optional doc-string props
|
|
|
1170 This function registers symbol @var{name} as a coding system.
|
|
|
1171
|
|
|
1172 @var{type} describes the conversion method used and should be one of
|
|
|
1173 the types listed in @ref{Coding System Types}.
|
|
|
1174
|
|
|
1175 @var{doc-string} is a string describing the coding system.
|
|
|
1176
|
|
|
1177 @var{props} is a property list, describing the specific nature of the
|
|
|
1178 character set. Recognized properties are as in @ref{Coding System
|
|
|
1179 Properties}.
|
|
|
1180 @end defun
|
|
|
1181
|
|
|
1182 @defun copy-coding-system old-coding-system new-name
|
|
|
1183 This function copies @var{old-coding-system} to @var{new-name}. If
|
|
|
1184 @var{new-name} does not name an existing coding system, a new one will
|
|
|
1185 be created.
|
|
|
1186 @end defun
|
|
|
1187
|
|
|
1188 @defun subsidiary-coding-system coding-system eol-type
|
|
|
1189 This function returns the subsidiary coding system of
|
|
|
1190 @var{coding-system} with eol type @var{eol-type}.
|
|
|
1191 @end defun
|
|
|
1192
|
|
442
|
1193 @node Coding System Property Functions, Encoding and Decoding Text, Basic Coding System Functions, Coding Systems
|
|
428
|
1194 @subsection Coding System Property Functions
|
|
|
1195
|
|
|
1196 @defun coding-system-doc-string coding-system
|
|
|
1197 This function returns the doc string for @var{coding-system}.
|
|
|
1198 @end defun
|
|
|
1199
|
|
|
1200 @defun coding-system-type coding-system
|
|
|
1201 This function returns the type of @var{coding-system}.
|
|
|
1202 @end defun
|
|
|
1203
|
|
|
1204 @defun coding-system-property coding-system prop
|
|
|
1205 This function returns the @var{prop} property of @var{coding-system}.
|
|
|
1206 @end defun
|
|
|
1207
|
|
442
|
1208 @node Encoding and Decoding Text, Detection of Textual Encoding, Coding System Property Functions, Coding Systems
|
|
428
|
1209 @subsection Encoding and Decoding Text
|
|
|
1210
|
|
|
1211 @defun decode-coding-region start end coding-system &optional buffer
|
|
|
1212 This function decodes the text between @var{start} and @var{end} which
|
|
|
1213 is encoded in @var{coding-system}. This is useful if you've read in
|
|
|
1214 encoded text from a file without decoding it (e.g. you read in a
|
|
|
1215 JIS-formatted file but used the @code{binary} or @code{no-conversion} coding
|
|
|
1216 system, so that it shows up as @samp{^[$B!<!+^[(B}). The length of the
|
|
|
1217 encoded text is returned. @var{buffer} defaults to the current buffer
|
|
|
1218 if unspecified.
|
|
|
1219 @end defun
|
|
|
1220
|
|
|
1221 @defun encode-coding-region start end coding-system &optional buffer
|
|
|
1222 This function encodes the text between @var{start} and @var{end} using
|
|
|
1223 @var{coding-system}. This will, for example, convert Japanese
|
|
|
1224 characters into stuff such as @samp{^[$B!<!+^[(B} if you use the JIS
|
|
|
1225 encoding. The length of the encoded text is returned. @var{buffer}
|
|
|
1226 defaults to the current buffer if unspecified.
|
|
|
1227 @end defun
|
|
|
1228
|
|
442
|
1229 @node Detection of Textual Encoding, Big5 and Shift-JIS Functions, Encoding and Decoding Text, Coding Systems
|
|
428
|
1230 @subsection Detection of Textual Encoding
|
|
|
1231
|
|
|
1232 @defun coding-category-list
|
|
|
1233 This function returns a list of all recognized coding categories.
|
|
|
1234 @end defun
|
|
|
1235
|
|
|
1236 @defun set-coding-priority-list list
|
|
|
1237 This function changes the priority order of the coding categories.
|
|
|
1238 @var{list} should be a list of coding categories, in descending order of
|
|
|
1239 priority. Unspecified coding categories will be lower in priority than
|
|
|
1240 all specified ones, in the same relative order they were in previously.
|
|
|
1241 @end defun
|
|
|
1242
|
|
|
1243 @defun coding-priority-list
|
|
|
1244 This function returns a list of coding categories in descending order of
|
|
|
1245 priority.
|
|
|
1246 @end defun
|
|
|
1247
|
|
|
1248 @defun set-coding-category-system coding-category coding-system
|
|
|
1249 This function changes the coding system associated with a coding category.
|
|
|
1250 @end defun
|
|
|
1251
|
|
|
1252 @defun coding-category-system coding-category
|
|
|
1253 This function returns the coding system associated with a coding category.
|
|
|
1254 @end defun
|
|
|
1255
|
|
|
1256 @defun detect-coding-region start end &optional buffer
|
|
|
1257 This function detects coding system of the text in the region between
|
|
|
1258 @var{start} and @var{end}. Returned value is a list of possible coding
|
|
|
1259 systems ordered by priority. If only ASCII characters are found, it
|
|
|
1260 returns @code{autodetect} or one of its subsidiary coding systems
|
|
|
1261 according to a detected end-of-line type. Optional arg @var{buffer}
|
|
|
1262 defaults to the current buffer.
|
|
|
1263 @end defun
|
|
|
1264
|
|
442
|
1265 @node Big5 and Shift-JIS Functions, Predefined Coding Systems, Detection of Textual Encoding, Coding Systems
|
|
428
|
1266 @subsection Big5 and Shift-JIS Functions
|
|
|
1267
|
|
442
|
1268 These are special functions for working with the non-standard
|
|
428
|
1269 Shift-JIS and Big5 encodings.
|
|
|
1270
|
|
|
1271 @defun decode-shift-jis-char code
|
|
442
|
1272 This function decodes a JIS X 0208 character of Shift-JIS coding-system.
|
|
428
|
1273 @var{code} is the character code in Shift-JIS as a cons of type bytes.
|
|
|
1274 The corresponding character is returned.
|
|
|
1275 @end defun
|
|
|
1276
|
|
444
|
1277 @defun encode-shift-jis-char character
|
|
|
1278 This function encodes a JIS X 0208 character @var{character} to
|
|
|
1279 SHIFT-JIS coding-system. The corresponding character code in SHIFT-JIS
|
|
|
1280 is returned as a cons of two bytes.
|
|
428
|
1281 @end defun
|
|
|
1282
|
|
|
1283 @defun decode-big5-char code
|
|
|
1284 This function decodes a Big5 character @var{code} of BIG5 coding-system.
|
|
|
1285 @var{code} is the character code in BIG5. The corresponding character
|
|
|
1286 is returned.
|
|
|
1287 @end defun
|
|
|
1288
|
|
444
|
1289 @defun encode-big5-char character
|
|
|
1290 This function encodes the Big5 character @var{character} to BIG5
|
|
428
|
1291 coding-system. The corresponding character code in Big5 is returned.
|
|
|
1292 @end defun
|
|
|
1293
|
|
442
|
1294 @node Predefined Coding Systems, , Big5 and Shift-JIS Functions, Coding Systems
|
|
|
1295 @subsection Coding Systems Implemented
|
|
|
1296
|
|
|
1297 MULE initializes most of the commonly used coding systems at XEmacs's
|
|
|
1298 startup. A few others are initialized only when the relevant language
|
|
|
1299 environment is selected and support libraries are loaded. (NB: The
|
|
444
|
1300 following list is based on XEmacs 21.2.19, the development branch at the
|
|
442
|
1301 time of writing. The list may be somewhat different for other
|
|
|
1302 versions. Recent versions of GNU Emacs 20 implement a few more rare
|
|
|
1303 coding systems; work is being done to port these to XEmacs.)
|
|
|
1304
|
|
444
|
1305 Unfortunately, there is not a consistent naming convention for character
|
|
|
1306 sets, and for practical purposes coding systems often take their name
|
|
442
|
1307 from their principal character sets (ASCII, KOI8-R, Shift JIS). Others
|
|
444
|
1308 take their names from the coding system (ISO-2022-JP, EUC-KR), and a few
|
|
|
1309 from their non-text usages (internal, binary). To provide for this, and
|
|
442
|
1310 for the fact that many coding systems have several common names, an
|
|
|
1311 aliasing system is provided. Finally, some effort has been made to use
|
|
|
1312 names that are registered as MIME charsets (this is why the name
|
|
|
1313 'shift_jis contains that un-Lisp-y underscore).
|
|
|
1314
|
|
|
1315 There is a systematic naming convention regarding end-of-line (EOL)
|
|
|
1316 conventions for different systems. A coding system whose name ends in
|
|
|
1317 "-unix" forces the assumptions that lines are broken by newlines (0x0A).
|
|
|
1318 A coding system whose name ends in "-mac" forces the assumptions that
|
|
|
1319 lines are broken by ASCII CRs (0x0D). A coding system whose name ends
|
|
|
1320 in "-dos" forces the assumptions that lines are broken by CRLF sequences
|
|
|
1321 (0x0D 0x0A). These subsidiary coding systems are automatically derived
|
|
|
1322 from a base coding system. Use of the base coding system implies
|
|
|
1323 autodetection of the text file convention. (The fact that the -unix,
|
|
|
1324 -mac, and -dos are derived from a base system results in them showing up
|
|
|
1325 as "aliases" in `list-coding-systems'.) These subsidiaries have a
|
|
|
1326 consistent modeline indicator as well. "-dos" coding systems have ":T"
|
|
|
1327 appended to their modeline indicator, while "-mac" coding systems have
|
|
|
1328 ":t" appended (eg, "ISO8:t" for iso-2022-8-mac).
|
|
|
1329
|
|
|
1330 In the following table, each coding system is given with its mode line
|
|
|
1331 indicator in parentheses. Non-textual coding systems are listed first,
|
|
|
1332 followed by textual coding systems and their aliases. (The coding system
|
|
|
1333 subsidiary modeline indicators ":T" and ":t" will be omitted from the
|
|
|
1334 table of coding systems.)
|
|
|
1335
|
|
|
1336 ### SJT 1999-08-23 Maybe should order these by language? Definitely
|
|
|
1337 need language usage for the ISO-8859 family.
|
|
|
1338
|
|
|
1339 Note that although true coding system aliases have been implemented for
|
|
444
|
1340 XEmacs 21.2, the coding system initialization has not yet been converted
|
|
442
|
1341 as of 21.2.19. So coding systems described as aliases have the same
|
|
|
1342 properties as the aliased coding system, but will not be equal as Lisp
|
|
|
1343 objects.
|
|
|
1344
|
|
|
1345 @table @code
|
|
|
1346
|
|
|
1347 @item automatic-conversion
|
|
|
1348 @itemx undecided
|
|
|
1349 @itemx undecided-dos
|
|
|
1350 @itemx undecided-mac
|
|
|
1351 @itemx undecided-unix
|
|
|
1352
|
|
|
1353 Modeline indicator: @code{Auto}. A type @code{undecided} coding system.
|
|
|
1354 Attempts to determine an appropriate coding system from file contents or
|
|
|
1355 the environment.
|
|
|
1356
|
|
|
1357 @item raw-text
|
|
|
1358 @itemx no-conversion
|
|
|
1359 @itemx raw-text-dos
|
|
|
1360 @itemx raw-text-mac
|
|
|
1361 @itemx raw-text-unix
|
|
|
1362 @itemx no-conversion-dos
|
|
|
1363 @itemx no-conversion-mac
|
|
|
1364 @itemx no-conversion-unix
|
|
|
1365
|
|
|
1366 Modeline indicator: @code{Raw}. A type @code{no-conversion} coding system,
|
|
|
1367 which converts only line-break-codes. An implementation quirk means
|
|
|
1368 that this coding system is also used for ISO8859-1.
|
|
|
1369
|
|
|
1370 @item binary
|
|
|
1371 Modeline indicator: @code{Binary}. A type @code{no-conversion} coding
|
|
|
1372 system which does no character coding or EOL conversions. An alias for
|
|
|
1373 @code{raw-text-unix}.
|
|
|
1374
|
|
|
1375 @item alternativnyj
|
|
|
1376 @itemx alternativnyj-dos
|
|
|
1377 @itemx alternativnyj-mac
|
|
|
1378 @itemx alternativnyj-unix
|
|
|
1379
|
|
|
1380 Modeline indicator: @code{Cy.Alt}. A type @code{ccl} coding system used for
|
|
|
1381 Alternativnyj, an encoding of the Cyrillic alphabet.
|
|
|
1382
|
|
|
1383 @item big5
|
|
|
1384 @itemx big5-dos
|
|
|
1385 @itemx big5-mac
|
|
|
1386 @itemx big5-unix
|
|
|
1387
|
|
|
1388 Modeline indicator: @code{Zh/Big5}. A type @code{big5} coding system used for
|
|
|
1389 BIG5, the most common encoding of traditional Chinese as used in Taiwan.
|
|
|
1390
|
|
|
1391 @item cn-gb-2312
|
|
|
1392 @itemx cn-gb-2312-dos
|
|
|
1393 @itemx cn-gb-2312-mac
|
|
|
1394 @itemx cn-gb-2312-unix
|
|
|
1395
|
|
|
1396 Modeline indicator: @code{Zh-GB/EUC}. A type @code{iso2022} coding system used
|
|
|
1397 for simplified Chinese (as used in the People's Republic of China), with
|
|
|
1398 the @code{ascii} (G0), @code{chinese-gb2312} (G1), and @code{sisheng}
|
|
|
1399 (G2) character sets initially designated. Chinese EUC (Extended Unix
|
|
|
1400 Code).
|
|
|
1401
|
|
|
1402 @item ctext-hebrew
|
|
|
1403 @itemx ctext-hebrew-dos
|
|
|
1404 @itemx ctext-hebrew-mac
|
|
|
1405 @itemx ctext-hebrew-unix
|
|
|
1406
|
|
|
1407 Modeline indicator: @code{CText/Hbrw}. A type @code{iso2022} coding system
|
|
|
1408 with the @code{ascii} (G0) and @code{hebrew-iso8859-8} (G1) character
|
|
|
1409 sets initially designated for Hebrew.
|
|
|
1410
|
|
|
1411 @item ctext
|
|
|
1412 @itemx ctext-dos
|
|
|
1413 @itemx ctext-mac
|
|
|
1414 @itemx ctext-unix
|
|
|
1415
|
|
|
1416 Modeline indicator: @code{CText}. A type @code{iso2022} 8-bit coding system
|
|
|
1417 with the @code{ascii} (G0) and @code{latin-iso8859-1} (G1) character
|
|
|
1418 sets initially designated. X11 Compound Text Encoding. Often
|
|
|
1419 mistakenly recognized instead of EUC encodings; usual cause is
|
|
|
1420 inappropriate setting of @code{coding-priority-list}.
|
|
|
1421
|
|
|
1422 @item escape-quoted
|
|
|
1423
|
|
|
1424 Modeline indicator: @code{ESC/Quot}. A type @code{iso2022} 8-bit coding
|
|
|
1425 system with the @code{ascii} (G0) and @code{latin-iso8859-1} (G1)
|
|
|
1426 character sets initially designated and escape quoting. Unix EOL
|
|
|
1427 conversion (ie, no conversion). It is used for .ELC files.
|
|
|
1428
|
|
|
1429 @item euc-jp
|
|
|
1430 @itemx euc-jp-dos
|
|
|
1431 @itemx euc-jp-mac
|
|
|
1432 @itemx euc-jp-unix
|
|
|
1433
|
|
|
1434 Modeline indicator: @code{Ja/EUC}. A type @code{iso2022} 8-bit coding system
|
|
|
1435 with @code{ascii} (G0), @code{japanese-jisx0208} (G1),
|
|
|
1436 @code{katakana-jisx0201} (G2), and @code{japanese-jisx0212} (G3)
|
|
|
1437 initially designated. Japanese EUC (Extended Unix Code).
|
|
|
1438
|
|
|
1439 @item euc-kr
|
|
|
1440 @itemx euc-kr-dos
|
|
|
1441 @itemx euc-kr-mac
|
|
|
1442 @itemx euc-kr-unix
|
|
|
1443
|
|
|
1444 Modeline indicator: @code{ko/EUC}. A type @code{iso2022} 8-bit coding system
|
|
|
1445 with @code{ascii} (G0) and @code{korean-ksc5601} (G1) initially
|
|
|
1446 designated. Korean EUC (Extended Unix Code).
|
|
|
1447
|
|
|
1448 @item hz-gb-2312
|
|
|
1449 Modeline indicator: @code{Zh-GB/Hz}. A type @code{no-conversion} coding
|
|
|
1450 system with Unix EOL convention (ie, no conversion) using
|
|
|
1451 post-read-decode and pre-write-encode functions to translate the Hz/ZW
|
|
|
1452 coding system used for Chinese.
|
|
|
1453
|
|
|
1454 @item iso-2022-7bit
|
|
|
1455 @itemx iso-2022-7bit-unix
|
|
|
1456 @itemx iso-2022-7bit-dos
|
|
|
1457 @itemx iso-2022-7bit-mac
|
|
|
1458 @itemx iso-2022-7
|
|
|
1459
|
|
|
1460 Modeline indicator: @code{ISO7}. A type @code{iso2022} 7-bit coding system
|
|
|
1461 with @code{ascii} (G0) initially designated. Other character sets must
|
|
|
1462 be explicitly designated to be used.
|
|
|
1463
|
|
|
1464 @item iso-2022-7bit-ss2
|
|
|
1465 @itemx iso-2022-7bit-ss2-dos
|
|
|
1466 @itemx iso-2022-7bit-ss2-mac
|
|
|
1467 @itemx iso-2022-7bit-ss2-unix
|
|
|
1468
|
|
|
1469 Modeline indicator: @code{ISO7/SS}. A type @code{iso2022} 7-bit coding system
|
|
|
1470 with @code{ascii} (G0) initially designated. Other character sets must
|
|
|
1471 be explicitly designated to be used. SS2 is used to invoke a
|
|
|
1472 96-charset, one character at a time.
|
|
|
1473
|
|
|
1474 @item iso-2022-8
|
|
|
1475 @itemx iso-2022-8-dos
|
|
|
1476 @itemx iso-2022-8-mac
|
|
|
1477 @itemx iso-2022-8-unix
|
|
|
1478
|
|
|
1479 Modeline indicator: @code{ISO8}. A type @code{iso2022} 8-bit coding system
|
|
|
1480 with @code{ascii} (G0) and @code{latin-iso8859-1} (G1) initially
|
|
|
1481 designated. Other character sets must be explicitly designated to be
|
|
|
1482 used. No single-shift or locking-shift.
|
|
|
1483
|
|
|
1484 @item iso-2022-8bit-ss2
|
|
|
1485 @itemx iso-2022-8bit-ss2-dos
|
|
|
1486 @itemx iso-2022-8bit-ss2-mac
|
|
|
1487 @itemx iso-2022-8bit-ss2-unix
|
|
|
1488
|
|
|
1489 Modeline indicator: @code{ISO8/SS}. A type @code{iso2022} 8-bit coding system
|
|
|
1490 with @code{ascii} (G0) and @code{latin-iso8859-1} (G1) initially
|
|
|
1491 designated. Other character sets must be explicitly designated to be
|
|
|
1492 used. SS2 is used to invoke a 96-charset, one character at a time.
|
|
|
1493
|
|
|
1494 @item iso-2022-int-1
|
|
|
1495 @itemx iso-2022-int-1-dos
|
|
|
1496 @itemx iso-2022-int-1-mac
|
|
|
1497 @itemx iso-2022-int-1-unix
|
|
|
1498
|
|
|
1499 Modeline indicator: @code{INT-1}. A type @code{iso2022} 7-bit coding system
|
|
|
1500 with @code{ascii} (G0) and @code{korean-ksc5601} (G1) initially
|
|
|
1501 designated. ISO-2022-INT-1.
|
|
|
1502
|
|
|
1503 @item iso-2022-jp-1978-irv
|
|
|
1504 @itemx iso-2022-jp-1978-irv-dos
|
|
|
1505 @itemx iso-2022-jp-1978-irv-mac
|
|
|
1506 @itemx iso-2022-jp-1978-irv-unix
|
|
|
1507
|
|
|
1508 Modeline indicator: @code{Ja-78/7bit}. A type @code{iso2022} 7-bit coding
|
|
|
1509 system. For compatibility with old Japanese terminals; if you need to
|
|
|
1510 know, look at the source.
|
|
|
1511
|
|
|
1512 @item iso-2022-jp
|
|
|
1513 @itemx iso-2022-jp-2 (ISO7/SS)
|
|
|
1514 @itemx iso-2022-jp-dos
|
|
|
1515 @itemx iso-2022-jp-mac
|
|
|
1516 @itemx iso-2022-jp-unix
|
|
|
1517 @itemx iso-2022-jp-2-dos
|
|
|
1518 @itemx iso-2022-jp-2-mac
|
|
|
1519 @itemx iso-2022-jp-2-unix
|
|
|
1520
|
|
|
1521 Modeline indicator: @code{MULE/7bit}. A type @code{iso2022} 7-bit coding
|
|
|
1522 system with @code{ascii} (G0) initially designated, and complex
|
|
|
1523 specifications to insure backward compatibility with old Japanese
|
|
|
1524 systems. Used for communication with mail and news in Japan. The "-2"
|
|
|
1525 versions also use SS2 to invoke a 96-charset one character at a time.
|
|
|
1526
|
|
|
1527 @item iso-2022-kr
|
|
|
1528 Modeline indicator: @code{Ko/7bit} A type @code{iso2022} 7-bit coding
|
|
|
1529 system with @code{ascii} (G0) and @code{korean-ksc5601} (G1) initially
|
|
|
1530 designated. Used for e-mail in Korea.
|
|
|
1531
|
|
|
1532 @item iso-2022-lock
|
|
|
1533 @itemx iso-2022-lock-dos
|
|
|
1534 @itemx iso-2022-lock-mac
|
|
|
1535 @itemx iso-2022-lock-unix
|
|
|
1536
|
|
|
1537 Modeline indicator: @code{ISO7/Lock}. A type @code{iso2022} 7-bit coding
|
|
|
1538 system with @code{ascii} (G0) initially designated, using Locking-Shift
|
|
|
1539 to invoke a 96-charset.
|
|
|
1540
|
|
|
1541 @item iso-8859-1
|
|
|
1542 @itemx iso-8859-1-dos
|
|
|
1543 @itemx iso-8859-1-mac
|
|
|
1544 @itemx iso-8859-1-unix
|
|
|
1545
|
|
|
1546 Due to implementation, this is not a type @code{iso2022} coding system,
|
|
|
1547 but rather an alias for the @code{raw-text} coding system.
|
|
|
1548
|
|
|
1549 @item iso-8859-2
|
|
|
1550 @itemx iso-8859-2-dos
|
|
|
1551 @itemx iso-8859-2-mac
|
|
|
1552 @itemx iso-8859-2-unix
|
|
|
1553
|
|
|
1554 Modeline indicator: @code{MIME/Ltn-2}. A type @code{iso2022} coding
|
|
|
1555 system with @code{ascii} (G0) and @code{latin-iso8859-2} (G1) initially
|
|
|
1556 invoked.
|
|
|
1557
|
|
|
1558 @item iso-8859-3
|
|
|
1559 @itemx iso-8859-3-dos
|
|
|
1560 @itemx iso-8859-3-mac
|
|
|
1561 @itemx iso-8859-3-unix
|
|
|
1562
|
|
|
1563 Modeline indicator: @code{MIME/Ltn-3}. A type @code{iso2022} coding system
|
|
|
1564 with @code{ascii} (G0) and @code{latin-iso8859-3} (G1) initially
|
|
|
1565 invoked.
|
|
|
1566
|
|
|
1567 @item iso-8859-4
|
|
|
1568 @itemx iso-8859-4-dos
|
|
|
1569 @itemx iso-8859-4-mac
|
|
|
1570 @itemx iso-8859-4-unix
|
|
|
1571
|
|
|
1572 Modeline indicator: @code{MIME/Ltn-4}. A type @code{iso2022} coding system
|
|
|
1573 with @code{ascii} (G0) and @code{latin-iso8859-4} (G1) initially
|
|
|
1574 invoked.
|
|
|
1575
|
|
|
1576 @item iso-8859-5
|
|
|
1577 @itemx iso-8859-5-dos
|
|
|
1578 @itemx iso-8859-5-mac
|
|
|
1579 @itemx iso-8859-5-unix
|
|
|
1580
|
|
|
1581 Modeline indicator: @code{ISO8/Cyr}. A type @code{iso2022} coding system with
|
|
|
1582 @code{ascii} (G0) and @code{cyrillic-iso8859-5} (G1) initially invoked.
|
|
|
1583
|
|
|
1584 @item iso-8859-7
|
|
|
1585 @itemx iso-8859-7-dos
|
|
|
1586 @itemx iso-8859-7-mac
|
|
|
1587 @itemx iso-8859-7-unix
|
|
|
1588
|
|
|
1589 Modeline indicator: @code{Grk}. A type @code{iso2022} coding system with
|
|
|
1590 @code{ascii} (G0) and @code{greek-iso8859-7} (G1) initially invoked.
|
|
|
1591
|
|
|
1592 @item iso-8859-8
|
|
|
1593 @itemx iso-8859-8-dos
|
|
|
1594 @itemx iso-8859-8-mac
|
|
|
1595 @itemx iso-8859-8-unix
|
|
|
1596
|
|
|
1597 Modeline indicator: @code{MIME/Hbrw}. A type @code{iso2022} coding system with
|
|
|
1598 @code{ascii} (G0) and @code{hebrew-iso8859-8} (G1) initially invoked.
|
|
|
1599
|
|
|
1600 @item iso-8859-9
|
|
|
1601 @itemx iso-8859-9-dos
|
|
|
1602 @itemx iso-8859-9-mac
|
|
|
1603 @itemx iso-8859-9-unix
|
|
|
1604
|
|
|
1605 Modeline indicator: @code{MIME/Ltn-5}. A type @code{iso2022} coding system
|
|
|
1606 with @code{ascii} (G0) and @code{latin-iso8859-9} (G1) initially
|
|
|
1607 invoked.
|
|
|
1608
|
|
|
1609 @item koi8-r
|
|
|
1610 @itemx koi8-r-dos
|
|
|
1611 @itemx koi8-r-mac
|
|
|
1612 @itemx koi8-r-unix
|
|
|
1613
|
|
|
1614 Modeline indicator: @code{KOI8}. A type @code{ccl} coding-system used for
|
|
|
1615 KOI8-R, an encoding of the Cyrillic alphabet.
|
|
|
1616
|
|
|
1617 @item shift_jis
|
|
|
1618 @itemx shift_jis-dos
|
|
|
1619 @itemx shift_jis-mac
|
|
|
1620 @itemx shift_jis-unix
|
|
|
1621
|
|
|
1622 Modeline indicator: @code{Ja/SJIS}. A type @code{shift-jis} coding-system
|
|
|
1623 implementing the Shift-JIS encoding for Japanese. The underscore is to
|
|
|
1624 conform to the MIME charset implementing this encoding.
|
|
|
1625
|
|
|
1626 @item tis-620
|
|
|
1627 @itemx tis-620-dos
|
|
|
1628 @itemx tis-620-mac
|
|
|
1629 @itemx tis-620-unix
|
|
|
1630
|
|
|
1631 Modeline indicator: @code{TIS620}. A type @code{ccl} encoding for Thai. The
|
|
|
1632 external encoding is defined by TIS620, the internal encoding is
|
|
|
1633 peculiar to MULE, and called @code{thai-xtis}.
|
|
|
1634
|
|
|
1635 @item viqr
|
|
|
1636
|
|
|
1637 Modeline indicator: @code{VIQR}. A type @code{no-conversion} coding
|
|
|
1638 system with Unix EOL convention (ie, no conversion) using
|
|
|
1639 post-read-decode and pre-write-encode functions to translate the VIQR
|
|
|
1640 coding system for Vietnamese.
|
|
|
1641
|
|
|
1642 @item viscii
|
|
|
1643 @itemx viscii-dos
|
|
|
1644 @itemx viscii-mac
|
|
|
1645 @itemx viscii-unix
|
|
|
1646
|
|
|
1647 Modeline indicator: @code{VISCII}. A type @code{ccl} coding-system used
|
|
|
1648 for VISCII 1.1 for Vietnamese. Differs slightly from VSCII; VISCII is
|
|
|
1649 given priority by XEmacs.
|
|
|
1650
|
|
|
1651 @item vscii
|
|
|
1652 @itemx vscii-dos
|
|
|
1653 @itemx vscii-mac
|
|
|
1654 @itemx vscii-unix
|
|
|
1655
|
|
|
1656 Modeline indicator: @code{VSCII}. A type @code{ccl} coding-system used
|
|
|
1657 for VSCII 1.1 for Vietnamese. Differs slightly from VISCII, which is
|
|
|
1658 given priority by XEmacs. Use
|
|
|
1659 @code{(prefer-coding-system 'vietnamese-vscii)} to give priority to VSCII.
|
|
|
1660
|
|
|
1661 @end table
|
|
|
1662
|
|
428
|
1663 @node CCL, Category Tables, Coding Systems, MULE
|
|
|
1664 @section CCL
|
|
|
1665
|
|
442
|
1666 CCL (Code Conversion Language) is a simple structured programming
|
|
428
|
1667 language designed for character coding conversions. A CCL program is
|
|
|
1668 compiled to CCL code (represented by a vector of integers) and executed
|
|
|
1669 by the CCL interpreter embedded in Emacs. The CCL interpreter
|
|
|
1670 implements a virtual machine with 8 registers called @code{r0}, ...,
|
|
|
1671 @code{r7}, a number of control structures, and some I/O operators. Take
|
|
|
1672 care when using registers @code{r0} (used in implicit @dfn{set}
|
|
|
1673 statements) and especially @code{r7} (used internally by several
|
|
444
|
1674 statements and operations, especially for multiple return values and I/O
|
|
428
|
1675 operations).
|
|
|
1676
|
|
442
|
1677 CCL is used for code conversion during process I/O and file I/O for
|
|
428
|
1678 non-ISO2022 coding systems. (It is the only way for a user to specify a
|
|
|
1679 code conversion function.) It is also used for calculating the code
|
|
|
1680 point of an X11 font from a character code. However, since CCL is
|
|
|
1681 designed as a powerful programming language, it can be used for more
|
|
|
1682 generic calculation where efficiency is demanded. A combination of
|
|
|
1683 three or more arithmetic operations can be calculated faster by CCL than
|
|
|
1684 by Emacs Lisp.
|
|
|
1685
|
|
442
|
1686 @strong{Warning:} The code in @file{src/mule-ccl.c} and
|
|
428
|
1687 @file{$packages/lisp/mule-base/mule-ccl.el} is the definitive
|
|
|
1688 description of CCL's semantics. The previous version of this section
|
|
|
1689 contained several typos and obsolete names left from earlier versions of
|
|
|
1690 MULE, and many may remain. (I am not an experienced CCL programmer; the
|
|
|
1691 few who know CCL well find writing English painful.)
|
|
|
1692
|
|
442
|
1693 A CCL program transforms an input data stream into an output data
|
|
428
|
1694 stream. The input stream, held in a buffer of constant bytes, is left
|
|
|
1695 unchanged. The buffer may be filled by an external input operation,
|
|
|
1696 taken from an Emacs buffer, or taken from a Lisp string. The output
|
|
|
1697 buffer is a dynamic array of bytes, which can be written by an external
|
|
|
1698 output operation, inserted into an Emacs buffer, or returned as a Lisp
|
|
|
1699 string.
|
|
|
1700
|
|
442
|
1701 A CCL program is a (Lisp) list containing two or three members. The
|
|
428
|
1702 first member is the @dfn{buffer magnification}, which indicates the
|
|
|
1703 required minimum size of the output buffer as a multiple of the input
|
|
|
1704 buffer. It is followed by the @dfn{main block} which executes while
|
|
|
1705 there is input remaining, and an optional @dfn{EOF block} which is
|
|
|
1706 executed when the input is exhausted. Both the main block and the EOF
|
|
|
1707 block are CCL blocks.
|
|
|
1708
|
|
442
|
1709 A @dfn{CCL block} is either a CCL statement or list of CCL statements.
|
|
444
|
1710 A @dfn{CCL statement} is either a @dfn{set statement} (either an integer
|
|
428
|
1711 or an @dfn{assignment}, which is a list of a register to receive the
|
|
444
|
1712 assignment, an assignment operator, and an expression) or a @dfn{control
|
|
428
|
1713 statement} (a list starting with a keyword, whose allowable syntax
|
|
|
1714 depends on the keyword).
|
|
|
1715
|
|
|
1716 @menu
|
|
|
1717 * CCL Syntax:: CCL program syntax in BNF notation.
|
|
|
1718 * CCL Statements:: Semantics of CCL statements.
|
|
|
1719 * CCL Expressions:: Operators and expressions in CCL.
|
|
|
1720 * Calling CCL:: Running CCL programs.
|
|
|
1721 * CCL Examples:: The encoding functions for Big5 and KOI-8.
|
|
|
1722 @end menu
|
|
|
1723
|
|
442
|
1724 @node CCL Syntax, CCL Statements, , CCL
|
|
428
|
1725 @comment Node, Next, Previous, Up
|
|
|
1726 @subsection CCL Syntax
|
|
|
1727
|
|
442
|
1728 The full syntax of a CCL program in BNF notation:
|
|
428
|
1729
|
|
|
1730 @format
|
|
|
1731 CCL_PROGRAM :=
|
|
|
1732 (BUFFER_MAGNIFICATION
|
|
|
1733 CCL_MAIN_BLOCK
|
|
|
1734 [ CCL_EOF_BLOCK ])
|
|
|
1735
|
|
|
1736 BUFFER_MAGNIFICATION := integer
|
|
|
1737 CCL_MAIN_BLOCK := CCL_BLOCK
|
|
|
1738 CCL_EOF_BLOCK := CCL_BLOCK
|
|
|
1739
|
|
|
1740 CCL_BLOCK :=
|
|
|
1741 STATEMENT | (STATEMENT [STATEMENT ...])
|
|
|
1742 STATEMENT :=
|
|
|
1743 SET | IF | BRANCH | LOOP | REPEAT | BREAK | READ | WRITE
|
|
|
1744 | CALL | END
|
|
|
1745
|
|
|
1746 SET :=
|
|
|
1747 (REG = EXPRESSION)
|
|
|
1748 | (REG ASSIGNMENT_OPERATOR EXPRESSION)
|
|
|
1749 | integer
|
|
|
1750
|
|
|
1751 EXPRESSION := ARG | (EXPRESSION OPERATOR ARG)
|
|
|
1752
|
|
|
1753 IF := (if EXPRESSION CCL_BLOCK [CCL_BLOCK])
|
|
|
1754 BRANCH := (branch EXPRESSION CCL_BLOCK [CCL_BLOCK ...])
|
|
|
1755 LOOP := (loop STATEMENT [STATEMENT ...])
|
|
|
1756 BREAK := (break)
|
|
|
1757 REPEAT :=
|
|
|
1758 (repeat)
|
|
|
1759 | (write-repeat [REG | integer | string])
|
|
|
1760 | (write-read-repeat REG [integer | ARRAY])
|
|
|
1761 READ :=
|
|
|
1762 (read REG ...)
|
|
|
1763 | (read-if (REG OPERATOR ARG) CCL_BLOCK CCL_BLOCK)
|
|
|
1764 | (read-branch REG CCL_BLOCK [CCL_BLOCK ...])
|
|
|
1765 WRITE :=
|
|
|
1766 (write REG ...)
|
|
|
1767 | (write EXPRESSION)
|
|
|
1768 | (write integer) | (write string) | (write REG ARRAY)
|
|
|
1769 | string
|
|
|
1770 CALL := (call ccl-program-name)
|
|
|
1771 END := (end)
|
|
|
1772
|
|
|
1773 REG := r0 | r1 | r2 | r3 | r4 | r5 | r6 | r7
|
|
|
1774 ARG := REG | integer
|
|
|
1775 OPERATOR :=
|
|
|
1776 + | - | * | / | % | & | '|' | ^ | << | >> | <8 | >8 | //
|
|
|
1777 | < | > | == | <= | >= | != | de-sjis | en-sjis
|
|
|
1778 ASSIGNMENT_OPERATOR :=
|
|
|
1779 += | -= | *= | /= | %= | &= | '|=' | ^= | <<= | >>=
|
|
|
1780 ARRAY := '[' integer ... ']'
|
|
|
1781 @end format
|
|
|
1782
|
|
|
1783 @node CCL Statements, CCL Expressions, CCL Syntax, CCL
|
|
|
1784 @comment Node, Next, Previous, Up
|
|
|
1785 @subsection CCL Statements
|
|
|
1786
|
|
442
|
1787 The Emacs Code Conversion Language provides the following statement
|
|
428
|
1788 types: @dfn{set}, @dfn{if}, @dfn{branch}, @dfn{loop}, @dfn{repeat},
|
|
|
1789 @dfn{break}, @dfn{read}, @dfn{write}, @dfn{call}, and @dfn{end}.
|
|
|
1790
|
|
|
1791 @heading Set statement:
|
|
|
1792
|
|
442
|
1793 The @dfn{set} statement has three variants with the syntaxes
|
|
428
|
1794 @samp{(@var{reg} = @var{expression})},
|
|
|
1795 @samp{(@var{reg} @var{assignment_operator} @var{expression})}, and
|
|
|
1796 @samp{@var{integer}}. The assignment operator variation of the
|
|
|
1797 @dfn{set} statement works the same way as the corresponding C expression
|
|
|
1798 statement does. The assignment operators are @code{+=}, @code{-=},
|
|
|
1799 @code{*=}, @code{/=}, @code{%=}, @code{&=}, @code{|=}, @code{^=},
|
|
|
1800 @code{<<=}, and @code{>>=}, and they have the same meanings as in C. A
|
|
|
1801 "naked integer" @var{integer} is equivalent to a @var{set} statement of
|
|
|
1802 the form @code{(r0 = @var{integer})}.
|
|
|
1803
|
|
|
1804 @heading I/O statements:
|
|
|
1805
|
|
442
|
1806 The @dfn{read} statement takes one or more registers as arguments. It
|
|
444
|
1807 reads one byte (a C char) from the input into each register in turn.
|
|
428
|
1808
|
|
442
|
1809 The @dfn{write} takes several forms. In the form @samp{(write @var{reg}
|
|
428
|
1810 ...)} it takes one or more registers as arguments and writes each in
|
|
|
1811 turn to the output. The integer in a register (interpreted as an
|
|
|
1812 Emchar) is encoded to multibyte form (ie, Bufbytes) and written to the
|
|
|
1813 current output buffer. If it is less than 256, it is written as is.
|
|
|
1814 The forms @samp{(write @var{expression})} and @samp{(write
|
|
|
1815 @var{integer})} are treated analogously. The form @samp{(write
|
|
|
1816 @var{string})} writes the constant string to the output. A
|
|
|
1817 "naked string" @samp{@var{string}} is equivalent to the statement @samp{(write
|
|
|
1818 @var{string})}. The form @samp{(write @var{reg} @var{array})} writes
|
|
|
1819 the @var{reg}th element of the @var{array} to the output.
|
|
|
1820
|
|
|
1821 @heading Conditional statements:
|
|
|
1822
|
|
442
|
1823 The @dfn{if} statement takes an @var{expression}, a @var{CCL block}, and
|
|
428
|
1824 an optional @var{second CCL block} as arguments. If the
|
|
|
1825 @var{expression} evaluates to non-zero, the first @var{CCL block} is
|
|
|
1826 executed. Otherwise, if there is a @var{second CCL block}, it is
|
|
|
1827 executed.
|
|
|
1828
|
|
442
|
1829 The @dfn{read-if} variant of the @dfn{if} statement takes an
|
|
428
|
1830 @var{expression}, a @var{CCL block}, and an optional @var{second CCL
|
|
|
1831 block} as arguments. The @var{expression} must have the form
|
|
|
1832 @code{(@var{reg} @var{operator} @var{operand})} (where @var{operand} is
|
|
|
1833 a register or an integer). The @code{read-if} statement first reads
|
|
|
1834 from the input into the first register operand in the @var{expression},
|
|
|
1835 then conditionally executes a CCL block just as the @code{if} statement
|
|
|
1836 does.
|
|
|
1837
|
|
442
|
1838 The @dfn{branch} statement takes an @var{expression} and one or more CCL
|
|
428
|
1839 blocks as arguments. The CCL blocks are treated as a zero-indexed
|
|
|
1840 array, and the @code{branch} statement uses the @var{expression} as the
|
|
|
1841 index of the CCL block to execute. Null CCL blocks may be used as
|
|
|
1842 no-ops, continuing execution with the statement following the
|
|
|
1843 @code{branch} statement in the containing CCL block. Out-of-range
|
|
444
|
1844 values for the @var{expression} are also treated as no-ops.
|
|
428
|
1845
|
|
442
|
1846 The @dfn{read-branch} variant of the @dfn{branch} statement takes an
|
|
428
|
1847 @var{register}, a @var{CCL block}, and an optional @var{second CCL
|
|
|
1848 block} as arguments. The @code{read-branch} statement first reads from
|
|
|
1849 the input into the @var{register}, then conditionally executes a CCL
|
|
|
1850 block just as the @code{branch} statement does.
|
|
|
1851
|
|
|
1852 @heading Loop control statements:
|
|
|
1853
|
|
442
|
1854 The @dfn{loop} statement creates a block with an implied jump from the
|
|
444
|
1855 end of the block back to its head. The loop is exited on a @code{break}
|
|
428
|
1856 statement, and continued without executing the tail by a @code{repeat}
|
|
|
1857 statement.
|
|
|
1858
|
|
442
|
1859 The @dfn{break} statement, written @samp{(break)}, terminates the
|
|
428
|
1860 current loop and continues with the next statement in the current
|
|
444
|
1861 block.
|
|
428
|
1862
|
|
442
|
1863 The @dfn{repeat} statement has three variants, @code{repeat},
|
|
428
|
1864 @code{write-repeat}, and @code{write-read-repeat}. Each continues the
|
|
|
1865 current loop from its head, possibly after performing I/O.
|
|
|
1866 @code{repeat} takes no arguments and does no I/O before jumping.
|
|
444
|
1867 @code{write-repeat} takes a single argument (a register, an
|
|
428
|
1868 integer, or a string), writes it to the output, then jumps.
|
|
|
1869 @code{write-read-repeat} takes one or two arguments. The first must
|
|
|
1870 be a register. The second may be an integer or an array; if absent, it
|
|
|
1871 is implicitly set to the first (register) argument.
|
|
|
1872 @code{write-read-repeat} writes its second argument to the output, then
|
|
|
1873 reads from the input into the register, and finally jumps. See the
|
|
|
1874 @code{write} and @code{read} statements for the semantics of the I/O
|
|
|
1875 operations for each type of argument.
|
|
|
1876
|
|
|
1877 @heading Other control statements:
|
|
|
1878
|
|
442
|
1879 The @dfn{call} statement, written @samp{(call @var{ccl-program-name})},
|
|
428
|
1880 executes a CCL program as a subroutine. It does not return a value to
|
|
|
1881 the caller, but can modify the register status.
|
|
|
1882
|
|
442
|
1883 The @dfn{end} statement, written @samp{(end)}, terminates the CCL
|
|
428
|
1884 program successfully, and returns to caller (which may be a CCL
|
|
|
1885 program). It does not alter the status of the registers.
|
|
|
1886
|
|
|
1887 @node CCL Expressions, Calling CCL, CCL Statements, CCL
|
|
|
1888 @comment Node, Next, Previous, Up
|
|
|
1889 @subsection CCL Expressions
|
|
|
1890
|
|
442
|
1891 CCL, unlike Lisp, uses infix expressions. The simplest CCL expressions
|
|
428
|
1892 consist of a single @var{operand}, either a register (one of @code{r0},
|
|
|
1893 ..., @code{r0}) or an integer. Complex expressions are lists of the
|
|
|
1894 form @code{( @var{expression} @var{operator} @var{operand} )}. Unlike
|
|
|
1895 C, assignments are not expressions.
|
|
|
1896
|
|
442
|
1897 In the following table, @var{X} is the target resister for a @dfn{set}.
|
|
428
|
1898 In subexpressions, this is implicitly @code{r7}. This means that
|
|
|
1899 @code{>8}, @code{//}, @code{de-sjis}, and @code{en-sjis} cannot be used
|
|
|
1900 freely in subexpressions, since they return parts of their values in
|
|
|
1901 @code{r7}. @var{Y} may be an expression, register, or integer, while
|
|
|
1902 @var{Z} must be a register or an integer.
|
|
|
1903
|
|
|
1904 @multitable @columnfractions .22 .14 .09 .55
|
|
|
1905 @item Name @tab Operator @tab Code @tab C-like Description
|
|
|
1906 @item CCL_PLUS @tab @code{+} @tab 0x00 @tab X = Y + Z
|
|
|
1907 @item CCL_MINUS @tab @code{-} @tab 0x01 @tab X = Y - Z
|
|
|
1908 @item CCL_MUL @tab @code{*} @tab 0x02 @tab X = Y * Z
|
|
|
1909 @item CCL_DIV @tab @code{/} @tab 0x03 @tab X = Y / Z
|
|
|
1910 @item CCL_MOD @tab @code{%} @tab 0x04 @tab X = Y % Z
|
|
|
1911 @item CCL_AND @tab @code{&} @tab 0x05 @tab X = Y & Z
|
|
|
1912 @item CCL_OR @tab @code{|} @tab 0x06 @tab X = Y | Z
|
|
|
1913 @item CCL_XOR @tab @code{^} @tab 0x07 @tab X = Y ^ Z
|
|
|
1914 @item CCL_LSH @tab @code{<<} @tab 0x08 @tab X = Y << Z
|
|
|
1915 @item CCL_RSH @tab @code{>>} @tab 0x09 @tab X = Y >> Z
|
|
|
1916 @item CCL_LSH8 @tab @code{<8} @tab 0x0A @tab X = (Y << 8) | Z
|
|
|
1917 @item CCL_RSH8 @tab @code{>8} @tab 0x0B @tab X = Y >> 8, r[7] = Y & 0xFF
|
|
|
1918 @item CCL_DIVMOD @tab @code{//} @tab 0x0C @tab X = Y / Z, r[7] = Y % Z
|
|
|
1919 @item CCL_LS @tab @code{<} @tab 0x10 @tab X = (X < Y)
|
|
|
1920 @item CCL_GT @tab @code{>} @tab 0x11 @tab X = (X > Y)
|
|
|
1921 @item CCL_EQ @tab @code{==} @tab 0x12 @tab X = (X == Y)
|
|
|
1922 @item CCL_LE @tab @code{<=} @tab 0x13 @tab X = (X <= Y)
|
|
|
1923 @item CCL_GE @tab @code{>=} @tab 0x14 @tab X = (X >= Y)
|
|
|
1924 @item CCL_NE @tab @code{!=} @tab 0x15 @tab X = (X != Y)
|
|
|
1925 @item CCL_ENCODE_SJIS @tab @code{en-sjis} @tab 0x16 @tab X = HIGHER_BYTE (SJIS (Y, Z))
|
|
|
1926 @item @tab @tab @tab r[7] = LOWER_BYTE (SJIS (Y, Z)
|
|
|
1927 @item CCL_DECODE_SJIS @tab @code{de-sjis} @tab 0x17 @tab X = HIGHER_BYTE (DE-SJIS (Y, Z))
|
|
|
1928 @item @tab @tab @tab r[7] = LOWER_BYTE (DE-SJIS (Y, Z))
|
|
|
1929 @end multitable
|
|
|
1930
|
|
442
|
1931 The CCL operators are as in C, with the addition of CCL_LSH8, CCL_RSH8,
|
|
428
|
1932 CCL_DIVMOD, CCL_ENCODE_SJIS, and CCL_DECODE_SJIS. The CCL_ENCODE_SJIS
|
|
|
1933 and CCL_DECODE_SJIS treat their first and second bytes as the high and
|
|
|
1934 low bytes of a two-byte character code. (SJIS stands for Shift JIS, an
|
|
|
1935 encoding of Japanese characters used by Microsoft. CCL_ENCODE_SJIS is a
|
|
|
1936 complicated transformation of the Japanese standard JIS encoding to
|
|
|
1937 Shift JIS. CCL_DECODE_SJIS is its inverse.) It is somewhat odd to
|
|
|
1938 represent the SJIS operations in infix form.
|
|
|
1939
|
|
442
|
1940 @node Calling CCL, CCL Examples, CCL Expressions, CCL
|
|
428
|
1941 @comment Node, Next, Previous, Up
|
|
|
1942 @subsection Calling CCL
|
|
|
1943
|
|
442
|
1944 CCL programs are called automatically during Emacs buffer I/O when the
|
|
428
|
1945 external representation has a coding system type of @code{shift-jis},
|
|
|
1946 @code{big5}, or @code{ccl}. The program is specified by the coding
|
|
|
1947 system (@pxref{Coding Systems}). You can also call CCL programs from
|
|
|
1948 other CCL programs, and from Lisp using these functions:
|
|
|
1949
|
|
|
1950 @defun ccl-execute ccl-program status
|
|
|
1951 Execute @var{ccl-program} with registers initialized by
|
|
|
1952 @var{status}. @var{ccl-program} is a vector of compiled CCL code
|
|
444
|
1953 created by @code{ccl-compile}. It is an error for the program to try to
|
|
428
|
1954 execute a CCL I/O command. @var{status} must be a vector of nine
|
|
|
1955 values, specifying the initial value for the R0, R1 .. R7 registers and
|
|
|
1956 for the instruction counter IC. A @code{nil} value for a register
|
|
|
1957 initializer causes the register to be set to 0. A @code{nil} value for
|
|
|
1958 the IC initializer causes execution to start at the beginning of the
|
|
|
1959 program. When the program is done, @var{status} is modified (by
|
|
|
1960 side-effect) to contain the ending values for the corresponding
|
|
444
|
1961 registers and IC.
|
|
428
|
1962 @end defun
|
|
|
1963
|
|
444
|
1964 @defun ccl-execute-on-string ccl-program status string &optional continue
|
|
428
|
1965 Execute @var{ccl-program} with initial @var{status} on
|
|
|
1966 @var{string}. @var{ccl-program} is a vector of compiled CCL code
|
|
|
1967 created by @code{ccl-compile}. @var{status} must be a vector of nine
|
|
|
1968 values, specifying the initial value for the R0, R1 .. R7 registers and
|
|
|
1969 for the instruction counter IC. A @code{nil} value for a register
|
|
|
1970 initializer causes the register to be set to 0. A @code{nil} value for
|
|
|
1971 the IC initializer causes execution to start at the beginning of the
|
|
444
|
1972 program. An optional fourth argument @var{continue}, if non-@code{nil}, causes
|
|
428
|
1973 the IC to
|
|
|
1974 remain on the unsatisfied read operation if the program terminates due
|
|
|
1975 to exhaustion of the input buffer. Otherwise the IC is set to the end
|
|
444
|
1976 of the program. When the program is done, @var{status} is modified (by
|
|
428
|
1977 side-effect) to contain the ending values for the corresponding
|
|
|
1978 registers and IC. Returns the resulting string.
|
|
|
1979 @end defun
|
|
|
1980
|
|
442
|
1981 To call a CCL program from another CCL program, it must first be
|
|
428
|
1982 registered:
|
|
|
1983
|
|
|
1984 @defun register-ccl-program name ccl-program
|
|
444
|
1985 Register @var{name} for CCL program @var{ccl-program} in
|
|
|
1986 @code{ccl-program-table}. @var{ccl-program} should be the compiled form of
|
|
|
1987 a CCL program, or @code{nil}. Return index number of the registered CCL
|
|
428
|
1988 program.
|
|
|
1989 @end defun
|
|
|
1990
|
|
442
|
1991 Information about the processor time used by the CCL interpreter can be
|
|
428
|
1992 obtained using these functions:
|
|
|
1993
|
|
|
1994 @defun ccl-elapsed-time
|
|
|
1995 Returns the elapsed processor time of the CCL interpreter as cons of
|
|
|
1996 user and system time, as
|
|
|
1997 floating point numbers measured in seconds. If only one
|
|
|
1998 overall value can be determined, the return value will be a cons of that
|
|
|
1999 value and 0.
|
|
|
2000 @end defun
|
|
|
2001
|
|
|
2002 @defun ccl-reset-elapsed-time
|
|
|
2003 Resets the CCL interpreter's internal elapsed time registers.
|
|
|
2004 @end defun
|
|
|
2005
|
|
442
|
2006 @node CCL Examples, , Calling CCL, CCL
|
|
428
|
2007 @comment Node, Next, Previous, Up
|
|
|
2008 @subsection CCL Examples
|
|
|
2009
|
|
442
|
2010 This section is not yet written.
|
|
428
|
2011
|
|
|
2012 @node Category Tables, , CCL, MULE
|
|
|
2013 @section Category Tables
|
|
|
2014
|
|
|
2015 A category table is a type of char table used for keeping track of
|
|
|
2016 categories. Categories are used for classifying characters for use in
|
|
440
|
2017 regexps---you can refer to a category rather than having to use a
|
|
428
|
2018 complicated [] expression (and category lookups are significantly
|
|
|
2019 faster).
|
|
|
2020
|
|
|
2021 There are 95 different categories available, one for each printable
|
|
|
2022 character (including space) in the ASCII charset. Each category is
|
|
|
2023 designated by one such character, called a @dfn{category designator}.
|
|
|
2024 They are specified in a regexp using the syntax @samp{\cX}, where X is a
|
|
|
2025 category designator. (This is not yet implemented.)
|
|
|
2026
|
|
|
2027 A category table specifies, for each character, the categories that
|
|
|
2028 the character is in. Note that a character can be in more than one
|
|
|
2029 category. More specifically, a category table maps from a character to
|
|
|
2030 either the value @code{nil} (meaning the character is in no categories)
|
|
|
2031 or a 95-element bit vector, specifying for each of the 95 categories
|
|
|
2032 whether the character is in that category.
|
|
|
2033
|
|
|
2034 Special Lisp functions are provided that abstract this, so you do not
|
|
|
2035 have to directly manipulate bit vectors.
|
|
|
2036
|
|
444
|
2037 @defun category-table-p object
|
|
|
2038 This function returns @code{t} if @var{object} is a category table.
|
|
428
|
2039 @end defun
|
|
|
2040
|
|
|
2041 @defun category-table &optional buffer
|
|
|
2042 This function returns the current category table. This is the one
|
|
|
2043 specified by the current buffer, or by @var{buffer} if it is
|
|
|
2044 non-@code{nil}.
|
|
|
2045 @end defun
|
|
|
2046
|
|
|
2047 @defun standard-category-table
|
|
|
2048 This function returns the standard category table. This is the one used
|
|
|
2049 for new buffers.
|
|
|
2050 @end defun
|
|
|
2051
|
|
444
|
2052 @defun copy-category-table &optional category-table
|
|
|
2053 This function returns a new category table which is a copy of
|
|
|
2054 @var{category-table}, which defaults to the standard category table.
|
|
428
|
2055 @end defun
|
|
|
2056
|
|
444
|
2057 @defun set-category-table category-table &optional buffer
|
|
|
2058 This function selects @var{category-table} as the new category table for
|
|
|
2059 @var{buffer}. @var{buffer} defaults to the current buffer if omitted.
|
|
428
|
2060 @end defun
|
|
|
2061
|
|
444
|
2062 @defun category-designator-p object
|
|
|
2063 This function returns @code{t} if @var{object} is a category designator (a
|
|
428
|
2064 char in the range @samp{' '} to @samp{'~'}).
|
|
|
2065 @end defun
|
|
|
2066
|
|
444
|
2067 @defun category-table-value-p object
|
|
|
2068 This function returns @code{t} if @var{object} is a category table value.
|
|
428
|
2069 Valid values are @code{nil} or a bit vector of size 95.
|
|
|
2070 @end defun
|
|
|
2071
|
