Mercurial > hg > xemacs-beta
annotate lisp/unicode.el @ 5396:75469840109b
Drop support for Thai-XTIS, which was always non-standard and never used.
lisp/ChangeLog addition:
2011-04-08 Aidan Kehoe <kehoea@parhasard.net>
* unicode.el (load-unicode-tables):
No longer include thai-xtis in the default Unicode precedence list.
* mule/thai.el:
* mule/thai.el (tis-620):
* mule/thai.el (windows-874):
* mule/thai.el ("Thai"):
Move the Thai language environment and the TIS-620 coding system
to this file; add support for Microsoft's code page 874.
* mule/thai-util.el:
* mule/thai-xtis.el:
Remove these two files; XTIS was always non-standard and was never
widely implemented, and we've never supported the character
composition necessary for thai-util.el.
* dumped-lisp.el (preloaded-file-list):
Drop thai-xtis, dump thai.el instead.
| author | Aidan Kehoe <kehoea@parhasard.net> |
|---|---|
| date | Fri, 08 Apr 2011 14:44:20 +0100 |
| parents | c0934cef10c6 |
| children | 248176c74e6b |
| rev | line source |
|---|---|
|
4993
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
1 ;;; unicode.el --- Unicode support -*- coding: utf-8; -*- |
| 771 | 2 |
| 778 | 3 ;; Copyright (C) 2001, 2002 Ben Wing. |
| 771 | 4 |
| 5 ;; Keywords: multilingual, Unicode | |
| 6 | |
| 7 ;; This file is part of XEmacs. | |
| 8 | |
| 9 ;; XEmacs is free software; you can redistribute it and/or modify it | |
| 10 ;; under the terms of the GNU General Public License as published by | |
| 11 ;; the Free Software Foundation; either version 2, or (at your option) | |
| 12 ;; any later version. | |
| 13 | |
| 14 ;; XEmacs is distributed in the hope that it will be useful, but | |
| 15 ;; WITHOUT ANY WARRANTY; without even the implied warranty of | |
| 16 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU | |
| 17 ;; General Public License for more details. | |
| 18 | |
| 19 ;; You should have received a copy of the GNU General Public License | |
| 20 ;; along with XEmacs; see the file COPYING. If not, write to the Free | |
| 21 ;; Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA | |
| 22 ;; 02111-1307, USA. | |
| 23 | |
| 24 ;;; Synched up with: Not in FSF. | |
| 25 | |
| 26 ;;; Commentary: | |
| 27 | |
| 28 ;; Lisp support for Unicode, e.g. initialize the translation tables. | |
| 29 | |
| 30 ;;; Code: | |
| 31 | |
| 3659 | 32 ;; GNU Emacs has the charsets: |
| 778 | 33 |
| 3659 | 34 ;; mule-unicode-2500-33ff |
| 35 ;; mule-unicode-e000-ffff | |
| 36 ;; mule-unicode-0100-24ff | |
| 778 | 37 |
| 3659 | 38 ;; built-in. This is hack--and an incomplete hack at that--against the |
| 39 ;; spirit and the letter of standard ISO 2022 character sets. Instead of | |
| 40 ;; this, we have the jit-ucs-charset-N Mule character sets, created in | |
| 41 ;; unicode.c on encountering a Unicode code point that we don't recognise, | |
| 42 ;; and saved in ISO 2022 coding systems using the UTF-8 escape described in | |
| 43 ;; ISO-IR 196. | |
| 778 | 44 |
| 4083 | 45 (eval-when-compile (when (featurep 'mule) (require 'ccl))) |
| 46 | |
| 2367 | 47 ;; accessed in loadup.el, mule-cmds.el; see discussion in unicode.c |
| 48 (defvar load-unicode-tables-at-dump-time (eq system-type 'windows-nt) | |
| 49 "[INTERNAL] Whether to load the Unicode tables at dump time. | |
| 50 Setting this at run-time does nothing.") | |
| 51 | |
| 771 | 52 ;; NOTE: This takes only a fraction of a second on my Pentium III |
| 53 ;; 700Mhz even with a totally optimization-disabled XEmacs. | |
| 54 (defun load-unicode-tables () | |
| 55 "Initialize the Unicode translation tables for all standard charsets." | |
| 780 | 56 (let ((parse-args |
| 57 '(("unicode/unicode-consortium" | |
| 877 | 58 ;; Due to the braindamaged way Mule treats the ASCII and Control-1 |
| 59 ;; charsets' types, trying to load them results in out-of-range | |
| 60 ;; warnings at unicode.c:1439. They're no-ops anyway, they're | |
| 61 ;; hardwired in unicode.c (unicode_to_ichar, ichar_to_unicode). | |
| 62 ;; ("8859-1.TXT" ascii #x00 #x7F #x0) | |
| 63 ;; ("8859-1.TXT" control-1 #x80 #x9F #x-80) | |
| 64 ;; The 8859-1.TXT G1 assignments are half no-ops, hardwired in | |
| 65 ;; unicode.c ichar_to_unicode, but not in unicode_to_ichar. | |
| 780 | 66 ("8859-1.TXT" latin-iso8859-1 #xA0 #xFF #x-80) |
| 67 ;; "8859-10.TXT" | |
| 68 ;; "8859-13.TXT" | |
| 69 ("8859-14.TXT" latin-iso8859-14 #xA0 #xFF #x-80) | |
| 70 ("8859-15.TXT" latin-iso8859-15 #xA0 #xFF #x-80) | |
| 2575 | 71 ("8859-16.TXT" latin-iso8859-16 #xA0 #xFF #x-80) |
| 780 | 72 ("8859-2.TXT" latin-iso8859-2 #xA0 #xFF #x-80) |
| 73 ("8859-3.TXT" latin-iso8859-3 #xA0 #xFF #x-80) | |
| 74 ("8859-4.TXT" latin-iso8859-4 #xA0 #xFF #x-80) | |
| 75 ("8859-5.TXT" cyrillic-iso8859-5 #xA0 #xFF #x-80) | |
|
4784
a67bfb29dd8b
Dump the arabic-iso8859-6 character set, again.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4783
diff
changeset
|
76 ("8859-6.TXT" arabic-iso8859-6 #xA0 #xFF #x-80) |
| 780 | 77 ("8859-7.TXT" greek-iso8859-7 #xA0 #xFF #x-80) |
| 78 ("8859-8.TXT" hebrew-iso8859-8 #xA0 #xFF #x-80) | |
| 79 ("8859-9.TXT" latin-iso8859-9 #xA0 #xFF #x-80) | |
| 80 ;; charset for Big5 does not matter; specifying `big5' will | |
| 81 ;; automatically make the right thing happen | |
| 82 ("BIG5.TXT" chinese-big5-1 nil nil nil big5) | |
| 83 ("CNS11643.TXT" chinese-cns11643-1 #x10000 #x1FFFF #x-10000) | |
| 84 ("CNS11643.TXT" chinese-cns11643-2 #x20000 #x2FFFF #x-20000) | |
| 85 ;; "CP1250.TXT" | |
| 86 ;; "CP1251.TXT" | |
| 87 ;; "CP1252.TXT" | |
| 88 ;; "CP1253.TXT" | |
| 89 ;; "CP1254.TXT" | |
| 90 ;; "CP1255.TXT" | |
| 91 ;; "CP1256.TXT" | |
| 92 ;; "CP1257.TXT" | |
| 93 ;; "CP1258.TXT" | |
| 94 ;; "CP874.TXT" | |
| 95 ;; "CP932.TXT" | |
| 96 ;; "CP936.TXT" | |
| 97 ;; "CP949.TXT" | |
| 98 ;; "CP950.TXT" | |
| 99 ;; "GB12345.TXT" | |
| 100 ("GB2312.TXT" chinese-gb2312) | |
| 2297 | 101 ;; "HANGUL.TXT" |
| 102 ;; #### shouldn't JIS X 0201's upper limit be 7f? | |
| 780 | 103 ("JIS0201.TXT" latin-jisx0201 #x21 #x80) |
| 104 ("JIS0201.TXT" katakana-jisx0201 #xA0 #xFF #x-80) | |
| 105 ("JIS0208.TXT" japanese-jisx0208 nil nil nil ignore-first-column) | |
| 106 ("JIS0212.TXT" japanese-jisx0212) | |
| 107 ;; "JOHAB.TXT" | |
| 108 ;; "KOI8-R.TXT" | |
| 109 ;; "KSC5601.TXT" | |
| 110 ;; note that KSC5601.TXT as currently distributed is NOT what | |
| 111 ;; it claims to be! see comments in KSX1001.TXT. | |
| 112 ("KSX1001.TXT" korean-ksc5601) | |
| 113 ;; "OLD5601.TXT" | |
| 114 ;; "SHIFTJIS.TXT" | |
| 115 ) | |
| 116 ("unicode/mule-ucs" | |
| 2297 | 117 ;; #### we don't support surrogates?!?? |
| 780 | 118 ;; use these instead of the above ones once we support surrogates |
| 119 ;;("chinese-cns11643-1.txt" chinese-cns11643-1) | |
| 120 ;;("chinese-cns11643-2.txt" chinese-cns11643-2) | |
| 121 ;;("chinese-cns11643-3.txt" chinese-cns11643-3) | |
| 122 ;;("chinese-cns11643-4.txt" chinese-cns11643-4) | |
| 123 ;;("chinese-cns11643-5.txt" chinese-cns11643-5) | |
| 124 ;;("chinese-cns11643-6.txt" chinese-cns11643-6) | |
| 125 ;;("chinese-cns11643-7.txt" chinese-cns11643-7) | |
| 126 ("chinese-sisheng.txt" chinese-sisheng) | |
| 127 ("ethiopic.txt" ethiopic) | |
| 128 ("indian-is13194.txt" indian-is13194) | |
| 129 ("ipa.txt" ipa) | |
| 130 ("thai-tis620.txt" thai-tis620) | |
| 131 ("tibetan.txt" tibetan) | |
| 132 ("vietnamese-viscii-lower.txt" vietnamese-viscii-lower) | |
| 133 ("vietnamese-viscii-upper.txt" vietnamese-viscii-upper) | |
| 134 ) | |
| 135 ("unicode/other" | |
| 136 ("lao.txt" lao) | |
| 137 ) | |
| 771 | 138 ))) |
|
4783
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
139 (mapc #'(lambda (tables) |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
140 (let ((undir |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
141 (expand-file-name (car tables) data-directory))) |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
142 (mapc #'(lambda (args) |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
143 (apply 'load-unicode-mapping-table |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
144 (expand-file-name (car args) undir) |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
145 (cdr args))) |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
146 (cdr tables)))) |
| 4145 | 147 parse-args) |
| 148 ;; The default-unicode-precedence-list. We set this here to default to | |
| 149 ;; *not* mapping various European characters to East Asian characters; | |
| 150 ;; otherwise the default-unicode-precedence-list is numerically ordered | |
| 151 ;; by charset ID. | |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
152 (declare-fboundp |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
153 (set-default-unicode-precedence-list |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
154 '(ascii control-1 latin-iso8859-1 latin-iso8859-2 latin-iso8859-15 |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
155 greek-iso8859-7 hebrew-iso8859-8 ipa cyrillic-iso8859-5 |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
156 latin-iso8859-16 latin-iso8859-3 latin-iso8859-4 latin-iso8859-9 |
|
4805
980575c76541
Move the arabic-iso8859-6 character set back to C, otherwise X11 lookup fails.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4784
diff
changeset
|
157 vietnamese-viscii-lower vietnamese-viscii-upper arabic-iso8859-6 |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
158 jit-ucs-charset-0 japanese-jisx0208 japanese-jisx0208-1978 |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
159 japanese-jisx0212 japanese-jisx0213-1 japanese-jisx0213-2 |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
160 chinese-gb2312 chinese-sisheng chinese-big5-1 chinese-big5-2 |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
161 indian-is13194 korean-ksc5601 chinese-cns11643-1 chinese-cns11643-2 |
|
4491
d402d7b18bd8
Revamp the Arabic support. Create greek-iso-8bit-with-esc, arabic-iso-8bit-with-esc.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4468
diff
changeset
|
162 chinese-isoir165 |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
163 composite ethiopic indian-1-column indian-2-column jit-ucs-charset-0 |
|
5396
75469840109b
Drop support for Thai-XTIS, which was always non-standard and never used.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4993
diff
changeset
|
164 katakana-jisx0201 lao thai-tis620 tibetan tibetan-1-column |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
165 latin-jisx0201 chinese-cns11643-3 chinese-cns11643-4 |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
166 chinese-cns11643-5 chinese-cns11643-6 chinese-cns11643-7))))) |
| 771 | 167 |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
168 (defconst ccl-encode-to-ucs-2 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
169 (eval-when-compile |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
170 (let ((pre-existing |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
171 ;; This is the compiled CCL program from the assert |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
172 ;; below. Since this file is dumped and ccl.el isn't (and |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
173 ;; even when it was, it was dumped much later than this |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
174 ;; one), we can't compile the program at dump time. We can |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
175 ;; check at byte compile time that the program is as |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
176 ;; expected, though. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
177 [1 16 131127 7 98872 65823 1307 5 -65536 65313 64833 1028 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
178 147513 8 82009 255 22])) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
179 (when (featurep 'mule) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
180 ;; Check that the pre-existing constant reflects the intended |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
181 ;; CCL program. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
182 (assert |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
183 (equal pre-existing |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
184 (ccl-compile |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
185 `(1 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
186 ( ;; mule-to-unicode's first argument is the |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
187 ;; charset ID, the second its first byte |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
188 ;; left shifted by 7 bits masked with its |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
189 ;; second byte. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
190 (r1 = (r1 << 7)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
191 (r1 = (r1 | r2)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
192 (mule-to-unicode r0 r1) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
193 (if (r0 & ,(lognot #xFFFF)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
194 ;; Redisplay looks in r1 and r2 for the first |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
195 ;; and second bytes of the X11 font, |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
196 ;; respectively. For non-BMP characters we |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
197 ;; display U+FFFD. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
198 ((r1 = #xFF) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
199 (r2 = #xFD)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
200 ((r1 = (r0 >> 8)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
201 (r2 = (r0 & #xFF)))))))) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
202 nil |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
203 "The pre-compiled CCL program appears broken. ")) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
204 pre-existing)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
205 "CCL program to transform Mule characters to UCS-2.") |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
206 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
207 (when (featurep 'mule) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
208 (put 'ccl-encode-to-ucs-2 'ccl-program-idx |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
209 (declare-fboundp |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
210 (register-ccl-program 'ccl-encode-to-ucs-2 ccl-encode-to-ucs-2)))) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
211 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
212 (defun decode-char (quote-ucs code &optional restriction) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
213 "FSF compatibility--return Mule character with Unicode codepoint CODE. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
214 The second argument must be 'ucs, the third argument is ignored. " |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
215 ;; We're prepared to accept invalid Unicode in unicode-to-char, but not in |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
216 ;; this function, which is the API that should actually be used, since |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
217 ;; it's available in GNU and in Mule-UCS. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
218 (check-argument-range code #x0 #x10FFFF) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
219 (assert (eq quote-ucs 'ucs) t |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
220 "Sorry, decode-char doesn't yet support anything but the UCS. ") |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
221 (unicode-to-char code)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
222 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
223 (defun encode-char (char quote-ucs &optional restriction) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
224 "FSF compatibility--return the Unicode code point of CHAR. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
225 The second argument must be 'ucs, the third argument is ignored. " |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
226 (assert (eq quote-ucs 'ucs) t |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
227 "Sorry, encode-char doesn't yet support anything but the UCS. ") |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
228 (char-to-unicode char)) |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4619
diff
changeset
|
229 |
| 771 | 230 (make-coding-system |
| 231 'utf-16 'unicode | |
| 232 "UTF-16" | |
| 233 '(mnemonic "UTF-16" | |
| 3767 | 234 documentation |
| 771 | 235 "UTF-16 Unicode encoding -- the standard (almost-) fixed-width |
| 236 two-byte encoding, with surrogates. It will be fixed-width if all | |
| 237 characters are in the BMP (Basic Multilingual Plane -- first 65536 | |
| 238 codepoints). Cannot represent characters with codepoints above | |
| 239 0x10FFFF (a little more than 1,000,000). Unicode and ISO guarantee | |
| 240 never to encode any characters outside this range -- all the rest are | |
| 241 for private, corporate or internal use." | |
| 3767 | 242 unicode-type utf-16)) |
| 771 | 243 |
| 2574 | 244 (define-coding-system-alias 'utf-16-be 'utf-16) |
| 245 | |
| 771 | 246 (make-coding-system |
| 247 'utf-16-bom 'unicode | |
| 248 "UTF-16 w/BOM" | |
| 249 '(mnemonic "UTF16-BOM" | |
| 3767 | 250 documentation |
| 771 | 251 "UTF-16 Unicode encoding with byte order mark (BOM) at the beginning. |
| 252 The BOM is Unicode character U+FEFF -- i.e. the first two bytes are | |
| 253 0xFE and 0xFF, respectively, or reversed in a little-endian | |
| 254 representation. It has been sanctioned by the Unicode Consortium for | |
| 255 use at the beginning of a Unicode stream as a marker of the byte order | |
| 256 of the stream, and commonly appears in Unicode files under Microsoft | |
| 257 Windows, where it also functions as a magic cookie identifying a | |
| 258 Unicode file. The character is called \"ZERO WIDTH NO-BREAK SPACE\" | |
| 259 and is suitable as a byte-order marker because: | |
| 260 | |
| 261 -- it has no displayable representation | |
| 262 -- due to its semantics it never normally appears at the beginning | |
| 263 of a stream | |
| 264 -- its reverse U+FFFE is not a legal Unicode character | |
| 265 -- neither byte sequence is at all likely in any other standard | |
| 266 encoding, particularly at the beginning of a stream | |
| 267 | |
| 268 This coding system will insert a BOM at the beginning of a stream when | |
| 269 writing and strip it off when reading." | |
| 3767 | 270 unicode-type utf-16 |
| 771 | 271 need-bom t)) |
| 272 | |
| 273 (make-coding-system | |
| 274 'utf-16-little-endian 'unicode | |
| 275 "UTF-16 Little Endian" | |
| 276 '(mnemonic "UTF16-LE" | |
| 277 documentation | |
| 278 "Little-endian version of UTF-16 Unicode encoding. | |
| 279 See `utf-16' coding system." | |
| 3767 | 280 unicode-type utf-16 |
| 771 | 281 little-endian t)) |
| 282 | |
| 2574 | 283 (define-coding-system-alias 'utf-16-le 'utf-16-little-endian) |
| 284 | |
| 771 | 285 (make-coding-system |
| 286 'utf-16-little-endian-bom 'unicode | |
| 287 "UTF-16 Little Endian w/BOM" | |
| 288 '(mnemonic "MSW-Unicode" | |
| 289 documentation | |
| 290 "Little-endian version of UTF-16 Unicode encoding, with byte order mark. | |
| 291 Standard encoding for representing Unicode under MS Windows. See | |
| 292 `utf-16-bom' coding system." | |
| 3767 | 293 unicode-type utf-16 |
| 771 | 294 little-endian t |
| 295 need-bom t)) | |
| 296 | |
| 297 (make-coding-system | |
| 298 'ucs-4 'unicode | |
| 299 "UCS-4" | |
| 300 '(mnemonic "UCS4" | |
| 301 documentation | |
| 302 "UCS-4 Unicode encoding -- fully fixed-width four-byte encoding." | |
| 3767 | 303 unicode-type ucs-4)) |
| 771 | 304 |
| 305 (make-coding-system | |
| 306 'ucs-4-little-endian 'unicode | |
| 307 "UCS-4 Little Endian" | |
| 308 '(mnemonic "UCS4-LE" | |
| 309 documentation | |
| 2297 | 310 ;; #### I don't think this is permitted by ISO 10646, only Unicode. |
| 311 ;; Call it UTF-32 instead? | |
| 771 | 312 "Little-endian version of UCS-4 Unicode encoding. See `ucs-4' coding system." |
| 3767 | 313 unicode-type ucs-4 |
| 771 | 314 little-endian t)) |
| 315 | |
| 316 (make-coding-system | |
| 4096 | 317 'utf-32 'unicode |
| 318 "UTF-32" | |
| 319 '(mnemonic "UTF32" | |
| 320 documentation | |
| 321 "UTF-32 Unicode encoding -- fixed-width four-byte encoding, | |
| 322 characters less than #x10FFFF are not supported. " | |
| 323 unicode-type utf-32)) | |
| 324 | |
| 325 (make-coding-system | |
| 326 'utf-32-little-endian 'unicode | |
| 327 "UTF-32 Little Endian" | |
| 328 '(mnemonic "UTF32-LE" | |
| 329 documentation | |
| 330 "Little-endian version of UTF-32 Unicode encoding. | |
| 331 | |
| 332 A fixed-width four-byte encoding, characters less than #x10FFFF are not | |
| 333 supported. " | |
| 334 unicode-type ucs-4 little-endian t)) | |
| 335 | |
|
4834
b3ea9c582280
Use new cygwin_conv_path API with Cygwin 1.7 for converting names between Win32 and POSIX, UTF-8-aware, with attendant changes elsewhere
Ben Wing <ben@xemacs.org>
parents:
4805
diff
changeset
|
336 ;; Now defined in unicode.c. |
| 771 | 337 |
|
4834
b3ea9c582280
Use new cygwin_conv_path API with Cygwin 1.7 for converting names between Win32 and POSIX, UTF-8-aware, with attendant changes elsewhere
Ben Wing <ben@xemacs.org>
parents:
4805
diff
changeset
|
338 ;;(make-coding-system |
|
b3ea9c582280
Use new cygwin_conv_path API with Cygwin 1.7 for converting names between Win32 and POSIX, UTF-8-aware, with attendant changes elsewhere
Ben Wing <ben@xemacs.org>
parents:
4805
diff
changeset
|
339 ;; 'utf-8 'unicode |
|
b3ea9c582280
Use new cygwin_conv_path API with Cygwin 1.7 for converting names between Win32 and POSIX, UTF-8-aware, with attendant changes elsewhere
Ben Wing <ben@xemacs.org>
parents:
4805
diff
changeset
|
340 ;; "UTF-8" |
|
b3ea9c582280
Use new cygwin_conv_path API with Cygwin 1.7 for converting names between Win32 and POSIX, UTF-8-aware, with attendant changes elsewhere
Ben Wing <ben@xemacs.org>
parents:
4805
diff
changeset
|
341 ;; '(mnemonic "UTF8" |
|
b3ea9c582280
Use new cygwin_conv_path API with Cygwin 1.7 for converting names between Win32 and POSIX, UTF-8-aware, with attendant changes elsewhere
Ben Wing <ben@xemacs.org>
parents:
4805
diff
changeset
|
342 ;; documentation "..." |
|
b3ea9c582280
Use new cygwin_conv_path API with Cygwin 1.7 for converting names between Win32 and POSIX, UTF-8-aware, with attendant changes elsewhere
Ben Wing <ben@xemacs.org>
parents:
4805
diff
changeset
|
343 ;; unicode-type utf-8)) |
| 771 | 344 |
| 985 | 345 (make-coding-system |
| 346 'utf-8-bom 'unicode | |
| 347 "UTF-8 w/BOM" | |
| 348 '(mnemonic "MSW-UTF8" | |
| 349 documentation | |
| 350 "UTF-8 Unicode encoding, with byte order mark. | |
| 351 Standard encoding for representing UTF-8 under MS Windows." | |
| 3767 | 352 unicode-type utf-8 |
| 985 | 353 little-endian t |
| 354 need-bom t)) | |
| 355 | |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
356 ;; Now, create jit-ucs-charset-0 entries for those characters in Windows |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
357 ;; Glyph List 4 that would otherwise end up in East Asian character sets. |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
358 ;; |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
359 ;; WGL4 is a character repertoire from Microsoft that gives a guideline |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
360 ;; for font implementors as to what characters are sufficient for |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
361 ;; pan-European support. The intention of this code is to avoid the |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
362 ;; situation where these characters end up mapping to East Asian XEmacs |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
363 ;; characters, which generally clash strongly with European characters |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
364 ;; both in font choice and character width; jit-ucs-charset-0 is a |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
365 ;; single-width character set which comes before the East Asian character |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
366 ;; sets in the default-unicode-precedence-list above. |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
367 (loop for (ucs ascii-or-latin-1) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
368 in '((#x2013 ?-) ;; U+2013 EN DASH |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
369 (#x2014 ?-) ;; U+2014 EM DASH |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
370 (#x2105 ?%) ;; U+2105 CARE OF |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
371 (#x203e ?-) ;; U+203E OVERLINE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
372 (#x221f ?|) ;; U+221F RIGHT ANGLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
373 (#x2584 ?|) ;; U+2584 LOWER HALF BLOCK |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
374 (#x2588 ?|) ;; U+2588 FULL BLOCK |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
375 (#x258c ?|) ;; U+258C LEFT HALF BLOCK |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
376 (#x2550 ?|) ;; U+2550 BOX DRAWINGS DOUBLE HORIZONTAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
377 (#x255e ?|) ;; U+255E BOX DRAWINGS VERTICAL SINGLE AND RIGHT DOUBLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
378 (#x256a ?|) ;; U+256A BOX DRAWINGS VERTICAL SINGLE & HORIZONTAL DOUBLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
379 (#x2561 ?|) ;; U+2561 BOX DRAWINGS VERTICAL SINGLE AND LEFT DOUBLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
380 (#x2215 ?/) ;; U+2215 DIVISION SLASH |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
381 (#x02c9 ?`) ;; U+02C9 MODIFIER LETTER MACRON |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
382 (#x2211 ?s) ;; U+2211 N-ARY SUMMATION |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
383 (#x220f ?s) ;; U+220F N-ARY PRODUCT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
384 (#x2248 ?=) ;; U+2248 ALMOST EQUAL TO |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
385 (#x2264 ?=) ;; U+2264 LESS-THAN OR EQUAL TO |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
386 (#x2265 ?=) ;; U+2265 GREATER-THAN OR EQUAL TO |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
387 (#x201c ?') ;; U+201C LEFT DOUBLE QUOTATION MARK |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
388 (#x2026 ?.) ;; U+2026 HORIZONTAL ELLIPSIS |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
389 (#x2212 ?-) ;; U+2212 MINUS SIGN |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
390 (#x2260 ?=) ;; U+2260 NOT EQUAL TO |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
391 (#x221e ?=) ;; U+221E INFINITY |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
392 (#x2642 ?=) ;; U+2642 MALE SIGN |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
393 (#x2640 ?=) ;; U+2640 FEMALE SIGN |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
394 (#x2032 ?=) ;; U+2032 PRIME |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
395 (#x2033 ?=) ;; U+2033 DOUBLE PRIME |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
396 (#x25cb ?=) ;; U+25CB WHITE CIRCLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
397 (#x25cf ?=) ;; U+25CF BLACK CIRCLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
398 (#x25a1 ?=) ;; U+25A1 WHITE SQUARE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
399 (#x25a0 ?=) ;; U+25A0 BLACK SQUARE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
400 (#x25b2 ?=) ;; U+25B2 BLACK UP-POINTING TRIANGLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
401 (#x25bc ?=) ;; U+25BC BLACK DOWN-POINTING TRIANGLE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
402 (#x2192 ?=) ;; U+2192 RIGHTWARDS ARROW |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
403 (#x2190 ?=) ;; U+2190 LEFTWARDS ARROW |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
404 (#x2191 ?=) ;; U+2191 UPWARDS ARROW |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
405 (#x2193 ?=) ;; U+2193 DOWNWARDS ARROW |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
406 (#x2229 ?=) ;; U+2229 INTERSECTION |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
407 (#x2202 ?=) ;; U+2202 PARTIAL DIFFERENTIAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
408 (#x2261 ?=) ;; U+2261 IDENTICAL TO |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
409 (#x221a ?=) ;; U+221A SQUARE ROOT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
410 (#x222b ?=) ;; U+222B INTEGRAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
411 (#x2030 ?=) ;; U+2030 PER MILLE SIGN |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
412 (#x266a ?=) ;; U+266A EIGHTH NOTE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
413 (#x2020 ?*) ;; U+2020 DAGGER |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
414 (#x2021 ?*) ;; U+2021 DOUBLE DAGGER |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
415 (#x2500 ?|) ;; U+2500 BOX DRAWINGS LIGHT HORIZONTAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
416 (#x2502 ?|) ;; U+2502 BOX DRAWINGS LIGHT VERTICAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
417 (#x250c ?|) ;; U+250C BOX DRAWINGS LIGHT DOWN AND RIGHT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
418 (#x2510 ?|) ;; U+2510 BOX DRAWINGS LIGHT DOWN AND LEFT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
419 (#x2518 ?|) ;; U+2518 BOX DRAWINGS LIGHT UP AND LEFT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
420 (#x2514 ?|) ;; U+2514 BOX DRAWINGS LIGHT UP AND RIGHT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
421 (#x251c ?|) ;; U+251C BOX DRAWINGS LIGHT VERTICAL AND RIGHT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
422 (#x252c ?|) ;; U+252C BOX DRAWINGS LIGHT DOWN AND HORIZONTAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
423 (#x2524 ?|) ;; U+2524 BOX DRAWINGS LIGHT VERTICAL AND LEFT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
424 (#x2534 ?|) ;; U+2534 BOX DRAWINGS LIGHT UP AND HORIZONTAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
425 (#x253c ?|) ;; U+253C BOX DRAWINGS LIGHT VERTICAL AND HORIZONTAL |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
426 (#x02da ?^) ;; U+02DA RING ABOVE |
|
4993
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
427 (#x2122 ?\xa9) ;; U+2122 TRADE MARK SIGN, ?© |
| 4145 | 428 |
|
4993
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
429 (#x0132 ?\xe6) ;; U+0132 LATIN CAPITAL LIGATURE IJ, ?æ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
430 (#x013f ?\xe6) ;; U+013F LATIN CAPITAL LETTER L WITH MIDDLE DOT, ?æ |
| 4145 | 431 |
|
4993
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
432 (#x0133 ?\xe6) ;; U+0133 LATIN SMALL LIGATURE IJ, ?æ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
433 (#x0140 ?\xe6) ;; U+0140 LATIN SMALL LETTER L WITH MIDDLE DOT, ?æ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
434 (#x0149 ?\xe6) ;; U+0149 LATIN SMALL LETTER N PRECEDED BY APOSTROPH,?æ |
| 4145 | 435 |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
436 (#x2194 ?|) ;; U+2194 LEFT RIGHT ARROW |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
437 (#x2660 ?*) ;; U+2660 BLACK SPADE SUIT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
438 (#x2665 ?*) ;; U+2665 BLACK HEART SUIT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
439 (#x2663 ?*) ;; U+2663 BLACK CLUB SUIT |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
440 (#x2592 ?|) ;; U+2592 MEDIUM SHADE |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
441 (#x2195 ?|) ;; U+2195 UP DOWN ARROW |
| 4145 | 442 |
|
4993
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
443 (#x2113 ?\xb9) ;; U+2113 SCRIPT SMALL L, ?¹ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
444 (#x215b ?\xbe) ;; U+215B VULGAR FRACTION ONE EIGHTH, ?¾ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
445 (#x215c ?\xbe) ;; U+215C VULGAR FRACTION THREE EIGHTHS, ?¾ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
446 (#x215d ?\xbe) ;; U+215D VULGAR FRACTION FIVE EIGHTHS, ?¾ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
447 (#x215e ?\xbe) ;; U+215E VULGAR FRACTION SEVEN EIGHTHS, ?¾ |
|
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
448 (#x207f ?\xbe) ;; U+207F SUPERSCRIPT LATIN SMALL LETTER N, ?¾ |
| 4145 | 449 |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
450 ;; These are not in WGL 4, but are IPA characters that should not |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
451 ;; be double width. They are the only IPA characters that both |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
452 ;; occur in packages/mule-packages/leim/ipa.el and end up in East |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
453 ;; Asian character sets when that file is loaded in an XEmacs |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
454 ;; without packages. |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
455 (#x2197 ?|) ;; U+2197 NORTH EAST ARROW |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
456 (#x2199 ?|) ;; U+2199 SOUTH WEST ARROW |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
457 (#x2191 ?|) ;; U+2191 UPWARDS ARROW |
|
4993
c0934cef10c6
convert some source files to utf-8
Ben Wing <ben@xemacs.org>
parents:
4834
diff
changeset
|
458 (#x207f ?\xb9)) ;; U+207F SUPERSCRIPT LATIN SMALL LETTER N, ?¹ |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
459 with decoded = nil |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
460 with syntax-table = (standard-syntax-table) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
461 initially (unless (featurep 'mule) (return)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
462 ;; This creates jit-ucs-charset-0 entries because: |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
463 ;; |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
464 ;; 1. If the tables are dumped, it is run at dump time before they are |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
465 ;; dumped, and as such before the relevant conversions are available |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
466 ;; (they are made available in mule/general-late.el). |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
467 ;; |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
468 ;; 2. If the tables are not dumped, it is run at dump time, long before |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
469 ;; any of the other mappings are available. |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
470 ;; |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
471 do |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
472 (setq decoded (decode-char 'ucs ucs)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
473 (assert (eq (declare-fboundp (char-charset decoded)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
474 'jit-ucs-charset-0) nil |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
475 "Unexpected Unicode decoding behavior. ") |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
476 (modify-syntax-entry decoded |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
477 (string |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
478 (char-syntax ascii-or-latin-1)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
479 syntax-table)) |
| 4145 | 480 |
| 4268 | 481 ;; *Sigh*, declarations needs to be at the start of the line to be picked up |
| 482 ;; by make-docfile. Not so much an issue with ccl-encode-to-ucs-2, which we | |
| 483 ;; don't necessarily want to advertise, but the following are important. | |
| 484 | |
| 485 ;; Create all the Unicode error sequences, normally as jit-ucs-charset-0 | |
| 486 ;; characters starting at U+200000 (which isn't a valid Unicode code | |
| 487 ;; point). Make them available to user code. | |
| 488 (defvar unicode-error-default-translation-table | |
| 489 (loop | |
|
4468
a78d697ccd2c
Import and extend GNU's descr-text.el, supporting prefix argument for C-x =
Aidan Kehoe <kehoea@parhasard.net>
parents:
4317
diff
changeset
|
490 with char-table = (make-char-table 'generic) |
| 4268 | 491 for i from ?\x00 to ?\xFF |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
492 initially (unless (featurep 'mule) (return)) |
| 4268 | 493 do |
| 494 (put-char-table (aref | |
| 495 ;; #xd800 is the first leading surrogate; | |
| 496 ;; trailing surrogates must be in the range | |
| 497 ;; #xdc00-#xdfff. These examples are not, so we | |
| 498 ;; intentionally provoke an error sequence. | |
| 499 (decode-coding-string (format "\xd8\x00\x00%c" i) | |
| 500 'utf-16-be) | |
| 501 3) | |
| 502 i | |
| 503 char-table) | |
| 504 finally return char-table) | |
| 505 "Translation table mapping Unicode error sequences to Latin-1 chars. | |
| 4145 | 506 |
| 4202 | 507 To transform XEmacs Unicode error sequences to the Latin-1 characters that |
| 508 correspond to the octets on disk, you can use this variable. ") | |
| 4145 | 509 |
|
4490
67fbcaf3dbdc
error-sequence -> invalid-sequence
Aidan Kehoe <kehoea@parhasard.net>
parents:
4489
diff
changeset
|
510 (defvar unicode-invalid-sequence-regexp-range |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
511 (and (featurep 'mule) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
512 (format "%c%c-%c" |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
513 (aref (decode-coding-string "\xd8\x00\x00\x00" 'utf-16-be) 0) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
514 (aref (decode-coding-string "\xd8\x00\x00\x00" 'utf-16-be) 3) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
515 (aref (decode-coding-string "\xd8\x00\x00\xFF" 'utf-16-be) 3))) |
| 4268 | 516 "Regular expression range to match Unicode error sequences in XEmacs. |
| 4145 | 517 |
| 4202 | 518 Invalid Unicode sequences on input are represented as XEmacs |
| 519 characters with values stored as the keys in | |
| 520 `unicode-error-default-translation-table', one character for each | |
| 521 invalid octet. You can use this variable (with `re-search-forward' or | |
| 522 `skip-chars-forward') to search for such characters; see also | |
| 523 `unicode-error-translate-region'. ") | |
| 524 | |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
525 ;; Check that the lookup table is correct, and that all the actual error |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
526 ;; sequences are caught by the regexp. |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
527 (with-temp-buffer |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
528 (loop |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
529 for i from ?\x00 to ?\xFF |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
530 with to-check = (make-string 20 ?\x20) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
531 initially (unless (featurep 'mule) (return)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
532 do |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
533 (delete-region (point-min) (point-max)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
534 (insert to-check) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
535 (goto-char 10) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
536 (insert (decode-coding-string (format "\xd8\x00\x00%c" i) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
537 'utf-16-be)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
538 (backward-char) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
539 (assert (= i (get-char-table (char-after (point)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
540 unicode-error-default-translation-table)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
541 (format "Char ?\\x%x not the expected error sequence!" |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
542 i)) |
| 4202 | 543 |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
544 (goto-char (point-min)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
545 ;; Comment out until the issue in |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
546 ;; 18179.49815.622843.336527@parhasard.net is fixed. |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
547 (assert t ; (re-search-forward (concat "[" |
|
4490
67fbcaf3dbdc
error-sequence -> invalid-sequence
Aidan Kehoe <kehoea@parhasard.net>
parents:
4489
diff
changeset
|
548 ; unicode-invalid-sequence-regexp-range |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
549 ; "]")) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
550 nil |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
551 (format "Could not find char ?\\x%x in buffer" i)))) |
| 4202 | 552 |
| 4268 | 553 (defun frob-unicode-errors-region (frob-function begin end &optional buffer) |
| 554 "Call FROB-FUNCTION on the Unicode error sequences between BEGIN and END. | |
| 4202 | 555 |
| 556 Optional argument BUFFER specifies the buffer that should be examined for | |
| 557 such sequences. " | |
| 4268 | 558 (check-argument-type #'functionp frob-function) |
| 559 (check-argument-range begin (point-min buffer) (point-max buffer)) | |
| 560 (check-argument-range end (point-min buffer) (point-max buffer)) | |
| 4202 | 561 (save-excursion |
| 562 (save-restriction | |
| 563 (if buffer (set-buffer buffer)) | |
| 564 (narrow-to-region begin end) | |
| 565 (goto-char (point-min)) | |
| 566 (while end | |
| 567 (setq begin | |
| 568 (progn | |
| 569 (skip-chars-forward | |
|
4490
67fbcaf3dbdc
error-sequence -> invalid-sequence
Aidan Kehoe <kehoea@parhasard.net>
parents:
4489
diff
changeset
|
570 (concat "^" unicode-invalid-sequence-regexp-range)) |
| 4202 | 571 (point)) |
| 572 end (and (not (= (point) (point-max))) | |
| 573 (progn | |
| 574 (skip-chars-forward | |
|
4490
67fbcaf3dbdc
error-sequence -> invalid-sequence
Aidan Kehoe <kehoea@parhasard.net>
parents:
4489
diff
changeset
|
575 unicode-invalid-sequence-regexp-range) |
| 4202 | 576 (point)))) |
| 577 (if end | |
| 578 (funcall frob-function begin end)))))) | |
| 579 | |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
580 (defun unicode-error-translate-region (begin end &optional buffer table) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
581 "Translate the Unicode error sequences in BUFFER between BEGIN and END. |
| 4202 | 582 |
| 583 The error sequences are transformed, by default, into the ASCII, | |
| 584 control-1 and latin-iso8859-1 characters with the numeric values | |
| 585 corresponding to the incorrect octets encountered. This is achieved | |
| 586 by using `unicode-error-default-translation-table' (which see) for | |
| 587 TABLE; you can change this by supplying another character table, | |
| 588 mapping from the error sequences to the desired characters. " | |
| 589 (unless table (setq table unicode-error-default-translation-table)) | |
| 590 (frob-unicode-errors-region | |
| 591 (lambda (start finish) | |
| 592 (translate-region start finish table)) | |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
593 begin end buffer)) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
594 |
|
4489
b75b075a9041
Support displaying invalid UTF-8 in language-environment-specific ways.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4468
diff
changeset
|
595 ;; Sure would be nice to be able to use defface here. |
|
4490
67fbcaf3dbdc
error-sequence -> invalid-sequence
Aidan Kehoe <kehoea@parhasard.net>
parents:
4489
diff
changeset
|
596 (copy-face 'highlight 'unicode-invalid-sequence-warning-face) |
|
4489
b75b075a9041
Support displaying invalid UTF-8 in language-environment-specific ways.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4468
diff
changeset
|
597 |
|
4317
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
598 (unless (featurep 'mule) |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
599 ;; We do this in such a roundabout way--instead of having the above defun |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
600 ;; and defvar calls inside a (when (featurep 'mule) ...) form--to have |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
601 ;; make-docfile.c pick up symbol and function documentation correctly. An |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
602 ;; alternative approach would be to fix make-docfile.c to be able to read |
|
15d36164ebd7
Eliminate lost docstring warnings on 21.5.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4268
diff
changeset
|
603 ;; Lisp. |
|
4783
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
604 (mapc #'unintern |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
605 '(ccl-encode-to-ucs-2 unicode-error-default-translation-table |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
606 unicode-invalid-regexp-range frob-unicode-errors-region |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
607 unicode-error-translate-region unicode-query-coding-region |
|
e29fcfd8df5f
Eliminate most core code byte-compile warnings.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4690
diff
changeset
|
608 unicode-query-coding-skip-chars-arg))) |
| 3667 | 609 |
| 771 | 610 ;; #### UTF-7 is not yet implemented, and it's tricky to do. There's |
| 611 ;; an implementation in appendix A.1 of the Unicode Standard, Version | |
| 612 ;; 2.0, but I don't know its licensing characteristics. | |
| 613 | |
| 614 ; (make-coding-system | |
| 615 ; 'utf-7 'unicode | |
| 616 ; "UTF-7" | |
| 617 ; '(mnemonic "UTF7" | |
| 3659 | 618 ; documentation; "UTF-7 Unicode encoding -- 7-bit-ASCII modal Internet-mail-compatible |
| 771 | 619 ; encoding especially designed for headers, with the following |
| 620 ; properties: | |
| 621 | |
| 622 ; -- Only characters that are considered safe for passing through any mail | |
| 623 ; gateway without damage are used. | |
| 624 | |
| 625 ; -- This is a modal encoding, with two states. The first, default | |
| 626 ; state encodes the most common Unicode characters (upper and | |
| 627 ; lowercase letters, digits, and 9 common punctuation marks) as | |
| 628 ; themselves, and the second state, entered using '+' and | |
| 629 ; terminated with '-' or any character disallowed in state 2, | |
| 630 ; encodes any Unicode characters by first converting to UTF-16, | |
| 631 ; most significant byte first, and then to a slightly modified | |
| 632 ; Base64 encoding. (Thus, UTF-7 has the same limitations on the | |
| 633 ; characters it can encode as UTF-16.) | |
| 634 | |
| 635 ; -- The modified Base64 encoding deviates from standard Base64 in | |
| 636 ; that it omits the `=' pad character. This is eliminated so as to | |
| 637 ; avoid conflicts with the use of `=' as an escape in the | |
| 638 ; Quoted-Printable encoding and the related Q encoding for headers: | |
| 639 ; With this modification, non-whitespace chars in UTF-7 will be | |
| 640 ; represented in Quoted-Printable and in Q as-is, with no further | |
| 641 ; encoding. | |
| 642 | |
| 643 ; For more information, see Appendix A.1 of The Unicode Standard 2.0, or | |
| 644 ; wherever it is in v3.0." | |
| 3767 | 645 ; unicode-type utf-7)) |