Mercurial > hg > xemacs-beta
annotate lisp/mule/mule-coding.el @ 4690:257b468bf2ca
Move the #'query-coding-region implementation to C.
This is necessary because there is no reasonable way to access the
corresponding mswindows-multibyte functionality from Lisp, and we need such
functionality if we're going to have a reliable and portable
#'query-coding-region implementation. However, this change doesn't yet
provide #'query-coding-region for the mswindow-multibyte coding systems,
there should be no functional differences between an XEmacs with this change
and one without it.
src/ChangeLog addition:
2009-09-19 Aidan Kehoe <kehoea@parhasard.net>
Move the #'query-coding-region implementation to C.
This is necessary because there is no reasonable way to access the
corresponding mswindows-multibyte functionality from Lisp, and we
need such functionality if we're going to have a reliable and
portable #'query-coding-region implementation. However, this
change doesn't yet provide #'query-coding-region for the
mswindow-multibyte coding systems, there should be no functional
differences between an XEmacs with this change and one without it.
* mule-coding.c (struct fixed_width_coding_system):
Add a new coding system type, fixed_width, and implement it. It
uses the CCL infrastructure but has a much simpler creation API,
and its own query_method, formerly in lisp/mule/mule-coding.el.
* unicode.c:
Move the Unicode query method implementation here from
unicode.el.
* lisp.h: Declare Fmake_coding_system_internal, Fcopy_range_table
here.
* intl-win32.c (complex_vars_of_intl_win32):
Use Fmake_coding_system_internal, not Fmake_coding_system.
* general-slots.h: Add Qsucceeded, Qunencodable, Qinvalid_sequence
here.
* file-coding.h (enum coding_system_variant):
Add fixed_width_coding_system here.
(struct coding_system_methods):
Add query_method and query_lstream_method to the coding system
methods.
Provide flags for the query methods.
Declare the default query method; initialise it correctly in
INITIALIZE_CODING_SYSTEM_TYPE.
* file-coding.c (default_query_method):
New function, the default query method for coding systems that do
not set it. Moved from coding.el.
(make_coding_system_1):
Accept new elements in PROPS in #'make-coding-system; aliases, a
list of aliases; safe-chars and safe-charsets (these were
previously accepted but not saved); and category.
(Fmake_coding_system_internal):
New function, what used to be #'make-coding-system--on Mule
builds, we've now moved some of the functionality of this to
Lisp.
(Fcoding_system_canonical_name_p):
Move this earlier in the file, since it's now called from within
make_coding_system_1.
(Fquery_coding_region):
Move the implementation of this here, from coding.el.
(complex_vars_of_file_coding):
Call Fmake_coding_system_internal, not Fmake_coding_system;
specify safe-charsets properties when we're a mule build.
* extents.h (mouse_highlight_priority, Fset_extent_priority,
Fset_extent_face, Fmap_extents):
Make these available to other C files.
lisp/ChangeLog addition:
2009-09-19 Aidan Kehoe <kehoea@parhasard.net>
Move the #'query-coding-region implementation to C.
* coding.el:
Consolidate code that depends on the presence or absence of Mule
at the end of this file.
(default-query-coding-region, query-coding-region):
Move these functions to C.
(default-query-coding-region-safe-charset-skip-chars-map):
Remove this variable, the corresponding C variable is
Vdefault_query_coding_region_chartab_cache in file-coding.c.
(query-coding-string): Update docstring to reflect actual multiple
values, be more careful about not modifying a range table that
we're currently mapping over.
(encode-coding-char): Make the implementation of this simpler.
(featurep 'mule): Autoload #'make-coding-system from
mule/make-coding-system.el if we're a mule build; provide an
appropriate compiler macro.
Do various non-mule compatibility things if we're not a mule
build.
* update-elc.el (additional-dump-dependencies):
Add mule/make-coding-system as a dump time dependency if we're a
mule build.
* unicode.el (ccl-encode-to-ucs-2):
(decode-char):
(encode-char):
Move these earlier in the file, for the sake of some byte compile
warnings.
(unicode-query-coding-region):
Move this to unicode.c
* mule/make-coding-system.el:
New file, not dumped. Contains the functionality to rework the
arguments necessary for fixed-width coding systems, and contains
the implementation of #'make-coding-system, which now calls
#'make-coding-system-internal.
* mule/vietnamese.el (viscii):
* mule/latin.el (iso-8859-2):
(windows-1250):
(iso-8859-3):
(iso-8859-4):
(iso-8859-14):
(iso-8859-15):
(iso-8859-16):
(iso-8859-9):
(macintosh):
(windows-1252):
* mule/hebrew.el (iso-8859-8):
* mule/greek.el (iso-8859-7):
(windows-1253):
* mule/cyrillic.el (iso-8859-5):
(koi8-r):
(koi8-u):
(windows-1251):
(alternativnyj):
(koi8-ru):
(koi8-t):
(koi8-c):
(koi8-o):
* mule/arabic.el (iso-8859-6):
(windows-1256):
Move all these coding systems to being of type fixed-width, not of
type CCL. This allows the distinct query-coding-region for them to
be in C, something which will eventually allow us to implement
query-coding-region for the mswindows-multibyte coding systems.
* mule/general-late.el (posix-charset-to-coding-system-hash):
Document why we're pre-emptively persuading the byte compiler that
the ELC for this file needs to be written using escape-quoted.
Call #'set-unicode-query-skip-chars-args, now the Unicode
query-coding-region implementation is in C.
* mule/thai-xtis.el (tis-620):
Don't bother checking whether we're XEmacs or not here.
* mule/mule-coding.el:
Move the eight bit fixed-width functionality from this file to
make-coding-system.el.
tests/ChangeLog addition:
2009-09-19 Aidan Kehoe <kehoea@parhasard.net>
* automated/mule-tests.el:
Check a coding system's type, not an 8-bit-fixed property, for
whether that coding system should be treated as a fixed-width
coding system.
* automated/query-coding-tests.el:
Don't test the query coding functionality for mswindows-multibyte
coding systems, it's not yet implemented.
| author | Aidan Kehoe <kehoea@parhasard.net> |
|---|---|
| date | Sat, 19 Sep 2009 22:53:13 +0100 |
| parents | c786c3fd0740 |
| children | 308d34e9f07d |
| rev | line source |
|---|---|
| 502 | 1 ;;; mule-coding.el --- Coding-system functions for Mule. -*- coding: iso-2022-7bit; -*- |
| 333 | 2 |
| 3 ;; Copyright (C) 1995 Electrotechnical Laboratory, JAPAN. | |
| 4 ;; Licensed to the Free Software Foundation. | |
| 5 ;; Copyright (C) 1995 Amdahl Corporation. | |
| 6 ;; Copyright (C) 1995 Sun Microsystems. | |
| 7 ;; Copyright (C) 1997 MORIOKA Tomohiko | |
| 771 | 8 ;; Copyright (C) 2001 Ben Wing. |
| 333 | 9 |
| 10 ;; This file is part of XEmacs. | |
| 11 | |
| 12 ;; XEmacs is free software; you can redistribute it and/or modify it | |
| 13 ;; under the terms of the GNU General Public License as published by | |
| 14 ;; the Free Software Foundation; either version 2, or (at your option) | |
| 15 ;; any later version. | |
| 16 | |
| 17 ;; XEmacs is distributed in the hope that it will be useful, but | |
| 18 ;; WITHOUT ANY WARRANTY; without even the implied warranty of | |
| 19 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU | |
| 20 ;; General Public License for more details. | |
| 21 | |
| 22 ;; You should have received a copy of the GNU General Public License | |
| 444 | 23 ;; along with XEmacs; see the file COPYING. If not, write to the |
| 333 | 24 ;; Free Software Foundation, Inc., 59 Temple Place - Suite 330, |
| 25 ;; Boston, MA 02111-1307, USA. | |
| 26 | |
| 27 ;;; Commentary: | |
| 28 | |
| 29 ;;; split off of mule.el and mostly moved to coding.el | |
| 30 | |
| 31 ;;; Code: | |
| 32 | |
| 33 (defun coding-system-force-on-output (coding-system register) | |
| 34 "Return the 'force-on-output property of CODING-SYSTEM for the specified REGISTER." | |
| 444 | 35 (check-type register integer) |
| 333 | 36 (coding-system-property |
| 37 coding-system | |
| 38 (case register | |
| 39 (0 'force-g0-on-output) | |
| 40 (1 'force-g1-on-output) | |
| 41 (2 'force-g2-on-output) | |
| 42 (3 'force-g3-on-output) | |
| 43 (t (signal 'args-out-of-range (list register 0 3)))))) | |
| 44 | |
| 45 (defun coding-system-short (coding-system) | |
| 46 "Return the 'short property of CODING-SYSTEM." | |
| 47 (coding-system-property coding-system 'short)) | |
| 48 | |
| 49 (defun coding-system-no-ascii-eol (coding-system) | |
| 50 "Return the 'no-ascii-eol property of CODING-SYSTEM." | |
| 51 (coding-system-property coding-system 'no-ascii-eol)) | |
| 52 | |
| 53 (defun coding-system-no-ascii-cntl (coding-system) | |
| 54 "Return the 'no-ascii-cntl property of CODING-SYSTEM." | |
| 55 (coding-system-property coding-system 'no-ascii-cntl)) | |
| 56 | |
| 57 (defun coding-system-seven (coding-system) | |
| 58 "Return the 'seven property of CODING-SYSTEM." | |
| 59 (coding-system-property coding-system 'seven)) | |
| 60 | |
| 61 (defun coding-system-lock-shift (coding-system) | |
| 62 "Return the 'lock-shift property of CODING-SYSTEM." | |
| 63 (coding-system-property coding-system 'lock-shift)) | |
| 64 | |
| 65 ;;(defun coding-system-use-japanese-jisx0201-roman (coding-system) | |
| 66 ;; "Return the 'use-japanese-jisx0201-roman property of CODING-SYSTEM." | |
| 67 ;; (coding-system-property coding-system 'use-japanese-jisx0201-roman)) | |
| 68 | |
| 69 ;;(defun coding-system-use-japanese-jisx0208-1978 (coding-system) | |
| 70 ;; "Return the 'use-japanese-jisx0208-1978 property of CODING-SYSTEM." | |
| 71 ;; (coding-system-property coding-system 'use-japanese-jisx0208-2978)) | |
| 72 | |
| 73 (defun coding-system-no-iso6429 (coding-system) | |
| 74 "Return the 'no-iso6429 property of CODING-SYSTEM." | |
| 75 (coding-system-property coding-system 'no-iso6429)) | |
| 76 | |
| 77 (defun coding-system-ccl-encode (coding-system) | |
| 78 "Return the CCL 'encode property of CODING-SYSTEM." | |
| 79 (coding-system-property coding-system 'encode)) | |
| 80 | |
| 81 (defun coding-system-ccl-decode (coding-system) | |
| 82 "Return the CCL 'decode property of CODING-SYSTEM." | |
| 83 (coding-system-property coding-system 'decode)) | |
| 84 | |
| 771 | 85 (defun coding-system-iso2022-charset (coding-system register) |
| 86 "Return the charset initially designated to REGISTER in CODING-SYSTEM. | |
| 87 The allowable range of REGISTER is 0 through 3." | |
| 88 (if (or (< register 0) (> register 3)) | |
| 89 (error 'args-out-of-range "coding-system-charset REGISTER" register 0 3)) | |
| 90 (coding-system-property coding-system (nth register '(charset-g0 | |
| 91 charset-g1 | |
| 92 charset-g2 | |
| 93 charset-g3)))) | |
| 94 | |
| 333 | 95 |
| 96 ;;;; Definitions of predefined coding systems | |
| 97 | |
| 98 (make-coding-system | |
| 99 'ctext 'iso2022 | |
| 771 | 100 "Compound Text" |
| 333 | 101 '(charset-g0 ascii |
| 102 charset-g1 latin-iso8859-1 | |
| 103 eol-type nil | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
104 safe-charsets t ;; Reasonable |
| 333 | 105 mnemonic "CText")) |
| 106 | |
| 107 (make-coding-system | |
| 108 'iso-2022-8bit-ss2 'iso2022 | |
| 771 | 109 "ISO-2022 8-bit w/SS2" |
| 333 | 110 '(charset-g0 ascii |
| 111 charset-g1 latin-iso8859-1 | |
| 112 charset-g2 t ;; unspecified but can be used later. | |
| 113 short t | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
114 safe-charsets (ascii katakana-jisx0201 japanese-jisx0208-1978 |
|
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
115 japanese-jisx0208 japanese-jisx0212 japanese-jisx0213-1 |
|
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
116 japanese-jisx0213-2) |
| 333 | 117 mnemonic "ISO8/SS" |
| 771 | 118 documentation "ISO 2022 based 8-bit encoding using SS2 for 96-charset" |
| 333 | 119 )) |
| 120 | |
| 121 (make-coding-system | |
| 122 'iso-2022-7bit-ss2 'iso2022 | |
| 771 | 123 "ISO-2022 7-bit w/SS2" |
| 333 | 124 '(charset-g0 ascii |
| 125 charset-g2 t ;; unspecified but can be used later. | |
| 126 seven t | |
| 127 short t | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
128 safe-charsets t |
| 333 | 129 mnemonic "ISO7/SS" |
| 771 | 130 documentation "ISO 2022 based 7-bit encoding using SS2 for 96-charset" |
| 333 | 131 eol-type nil)) |
| 132 | |
| 133 ;; (copy-coding-system 'iso-2022-7bit-ss2 'iso-2022-jp-2) | |
| 134 (make-coding-system | |
| 135 'iso-2022-jp-2 'iso2022 | |
| 771 | 136 "ISO-2022-JP-2" |
| 333 | 137 '(charset-g0 ascii |
| 138 charset-g2 t ;; unspecified but can be used later. | |
| 139 seven t | |
| 140 short t | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
141 safe-charsets t |
| 333 | 142 mnemonic "ISO7/SS" |
| 143 eol-type nil)) | |
| 144 | |
| 145 (make-coding-system | |
| 146 'iso-2022-7bit 'iso2022 | |
| 771 | 147 "ISO 2022 7-bit" |
| 333 | 148 '(charset-g0 ascii |
| 149 seven t | |
| 150 short t | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
151 safe-charsets t |
| 771 | 152 mnemonic "ISO7" |
| 153 documentation "ISO-2022-based 7-bit encoding using only G0" | |
| 154 )) | |
| 333 | 155 |
| 156 ;; compatibility for old XEmacsen | |
| 771 | 157 (define-coding-system-alias 'iso-2022-7 'iso-2022-7bit) |
| 333 | 158 |
| 159 (make-coding-system | |
| 160 'iso-2022-8 'iso2022 | |
| 771 | 161 "ISO-2022 8-bit" |
| 333 | 162 '(charset-g0 ascii |
| 163 charset-g1 latin-iso8859-1 | |
| 164 short t | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
165 safe-charsets t |
| 333 | 166 mnemonic "ISO8" |
| 771 | 167 documentation "ISO-2022 eight-bit coding system. No single-shift or locking-shift." |
| 333 | 168 )) |
| 169 | |
| 170 (make-coding-system | |
| 171 'escape-quoted 'iso2022 | |
| 771 | 172 "Escape-Quoted (for .ELC files)" |
| 333 | 173 '(charset-g0 ascii |
| 174 charset-g1 latin-iso8859-1 | |
| 175 eol-type lf | |
| 176 escape-quoted t | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
177 safe-charsets t |
| 333 | 178 mnemonic "ESC/Quot" |
| 771 | 179 documentation "ISO-2022 eight-bit coding system with escape quoting; used for .ELC files." |
| 333 | 180 )) |
| 181 | |
| 182 (make-coding-system | |
| 183 'iso-2022-lock 'iso2022 | |
| 771 | 184 "ISO-2022 w/locking-shift" |
| 333 | 185 '(charset-g0 ascii |
| 186 charset-g1 t ;; unspecified but can be used later. | |
| 187 seven t | |
| 188 lock-shift t | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4567
diff
changeset
|
189 safe-charsets t |
| 333 | 190 mnemonic "ISO7/Lock" |
| 771 | 191 documentation "ISO-2022 coding system using Locking-Shift for 96-charset." |
| 333 | 192 )) |
| 4072 | 193 |
| 333 | 194 |
| 4072 | 195 ;; This is used by people writing CCL programs, but is called at runtime. |
| 196 (defun define-translation-hash-table (symbol table) | |
| 197 "Define SYMBOL as the name of the hash translation TABLE for use in CCL. | |
| 198 | |
| 199 Analogous to `define-translation-table', but updates | |
| 200 `translation-hash-table-vector' and the table is for use in the CCL | |
| 201 `lookup-integer' and `lookup-character' functions." | |
| 4145 | 202 (check-argument-type #'symbolp symbol) |
| 203 (check-argument-type #'hash-table-p table) | |
| 4072 | 204 (let ((len (length translation-hash-table-vector)) |
| 205 (id 0) | |
| 206 done) | |
| 207 (put symbol 'translation-hash-table table) | |
| 208 (while (not done) | |
| 209 (if (>= id len) | |
| 210 (setq translation-hash-table-vector | |
| 211 (vconcat translation-hash-table-vector [nil]))) | |
| 212 (let ((slot (aref translation-hash-table-vector id))) | |
| 213 (if (or (not slot) | |
| 214 (eq (car slot) symbol)) | |
| 215 (progn | |
| 216 (aset translation-hash-table-vector id (cons symbol table)) | |
| 217 (setq done t)) | |
| 218 (setq id (1+ id))))) | |
| 219 (put symbol 'translation-hash-table-id id) | |
| 220 id)) | |
| 221 | |
| 4299 | 222 ;; Ideally this would be in latin.el, but code-init.el uses it. |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
223 (make-coding-system |
| 4299 | 224 'iso-8859-1 |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
225 'fixed-width |
| 4299 | 226 "ISO-8859-1 (Latin-1)" |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
227 (eval-when-compile |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
228 `(unicode-map |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
229 ,(loop |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
230 for i from #x80 to #xff |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
231 collect (list i (int-char i))) ;; Identical to Latin-1. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
232 mnemonic "Latin 1" |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
233 documentation "The most used encoding of Western Europe and the Americas." |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4605
diff
changeset
|
234 aliases (iso-latin-1 latin-1)))) |