xemacs-beta: lisp/mule/make-coding-system.el annotate

annotate lisp/mule/make-coding-system.el @ 5068:c673987f5f3d

dump make-coding-system -------------------- ChangeLog entries follow: -------------------- lisp/ChangeLog addition: 2010-02-22 Ben Wing <ben@xemacs.org> * mule/make-coding-system.el: * mule/make-coding-system.el (fixed-width-generate-helper): * mule/make-coding-system.el (fixed-width-private-use-start): Removed. * mule/make-coding-system.el (fixed-width-create-decode-encode-tables): * coding.el: * coding.el (decode-char): New. * coding.el (featurep): * coding.el (encode-char): New. * dumped-lisp.el (preloaded-file-list): Dump make-coding-system. Aidan's hack to avoid dumping this file never really worked right -- with some configurations (not clear exactly which ones) `make-coding-system.el' gets dumped anyway due to calls to `make-coding-system' in unicode.el, with the result that the documentation of functions in make-coding-system.el gets lost. Also needed to remove defvar fixed-width-private-use-start and incorporate it inline, due to bootstrapping issues -- the call to decode-char introduced a cross-dependency between unicode.el and make-coding-system.el.

author	Ben Wing <ben@xemacs.org>
date	Mon, 22 Feb 2010 21:26:18 -0600
parents	c69aeb86b2a3
children	baffa6ca776a

rev	line source
4690 257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	1 ;;; make-coding-system.el; Provides the #'make-coding-system function and
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	2 ;;; much of the implementation of the fixed-width coding system type.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	3
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	4 ;; Copyright (C) 2009 Free Software Foundation
5068 c673987f5f3d dump make-coding-system Ben Wing <ben@xemacs.org> parents: 4777 diff changeset	5 ;; Copyright (C) 2010 Ben Wing.
4690 257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	6
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	7 ;; Author: Aidan Kehoe
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	8
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	9 ;; This file is part of XEmacs.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	10
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	11 ;; XEmacs is free software; you can redistribute it and/or modify it
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	12 ;; under the terms of the GNU General Public License as published by
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	13 ;; the Free Software Foundation; either version 2, or (at your option)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	14 ;; any later version.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	15
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	16 ;; XEmacs is distributed in the hope that it will be useful, but
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	17 ;; WITHOUT ANY WARRANTY; without even the implied warranty of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	18 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	19 ;; General Public License for more details.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	20
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	21 ;; You should have received a copy of the GNU General Public License
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	22 ;; along with XEmacs; see the file COPYING. If not, write to the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	23 ;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	24 ;; Boston, MA 02110-1301, USA.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	25
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	26 ;;; Commentary:
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	27
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	28 ;;; Code:
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	29
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	30 (defun fixed-width-generate-helper (decode-table encode-table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	31 encode-failure-octet)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	32 "Helper func, `fixed-width-generate-encode-program-and-skip-chars-strings',
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	33 which see.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	34
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	35 Deals with the case where ASCII and another character set can both be
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	36 encoded unambiguously and completely into the coding-system; if this is so,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	37 returns multiple values comprisig of such a ccl-program and the character
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	38 set in question. If not, it returns nil."
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	39 (let ((tentative-encode-program-parts
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	40 (eval-when-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	41 (let* ((vec-len 128)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	42 (compiled
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	43 (append
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	44 (ccl-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	45 `(1
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	46 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	47 (read-multibyte-character r0 r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	48 (if (r0 == ,(charset-id 'ascii))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	49 (write r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	50 ((if (r0 == #xABAB)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	51 ;; #xBFFE is a sentinel in the compiled
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	52 ;; program.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	53 ((r0 = r1 & #x7F)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	54 (write r0 ,(make-vector vec-len #xBFFE)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	55 ((mule-to-unicode r0 r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	56 (if (r0 == #xFFFD)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	57 (write #xBEEF)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	58 ((lookup-integer encode-table-sym r0 r3)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	59 (if r7
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	60 (write-multibyte-character r0 r3)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	61 (write #xBEEF))))))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	62 (repeat)))) nil))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	63 (first-part compiled)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	64 (last-part
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	65 (member-if-not (lambda (entr) (eq #xBFFE entr))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	66 (member-if
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	67 (lambda (entr) (eq #xBFFE entr))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	68 first-part))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	69 (while compiled
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	70 (when (eq #xBFFE (cadr compiled))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	71 (assert (= vec-len (search '(#xBFFE) (cdr compiled)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	72 :test #'/=)) nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	73 "Strange ccl vector length")
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	74 (setcdr compiled nil))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	75 (setq compiled (cdr compiled)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	76 ;; Is the generated code as we expect it to be?
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	77 (assert (and (memq #xABAB first-part)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	78 (memq #xBEEF14 last-part))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	79 nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	80 "This code assumes that the constant #xBEEF is #xBEEF14 in \
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	81 compiled CCL code,\nand that the constant #xABAB is #xABAB. If that is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	82 not the case, and it appears not to be--that's why you're getting this
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	83 message--it will not work. ")
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	84 (list first-part last-part vec-len))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	85 (charset-lower -1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	86 (charset-upper -1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	87 worth-trying known-charsets encode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	88 other-charset-vector ucs)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	89
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	90 (loop for char across decode-table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	91 do (pushnew (char-charset char) known-charsets))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	92 (setq known-charsets (delq 'ascii known-charsets))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	93
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	94 (loop for known-charset in known-charsets
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	95 do
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	96 ;; This is not possible for two dimensional charsets.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	97 (when (eq 1 (charset-dimension known-charset))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	98 (if (eq 'control-1 known-charset)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	99 (setq charset-lower 0
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	100 charset-upper 31)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	101 ;; There should be a nicer way to get the limits here.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	102 (condition-case args-out-of-range
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	103 (make-char known-charset #x100)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	104 (args-out-of-range
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	105 (setq charset-lower (third args-out-of-range)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	106 charset-upper (fourth args-out-of-range)))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	107 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	108 for i from charset-lower to charset-upper
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	109 always (and (setq ucs
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	110 (encode-char (make-char known-charset i) 'ucs))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	111 (gethash ucs encode-table))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	112 finally (setq worth-trying known-charset))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	113
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	114 ;; Only trying this for one charset at a time, the first find.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	115 (when worth-trying (return))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	116
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	117 ;; Okay, this charset is not worth trying, Try the next.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	118 (setq charset-lower -1
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	119 charset-upper -1
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	120 worth-trying nil)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	121
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	122 (when worth-trying
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	123 (setq other-charset-vector
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	124 (make-vector (third tentative-encode-program-parts)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	125 encode-failure-octet))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	126 (loop for i from charset-lower to charset-upper
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	127 do (aset other-charset-vector i
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	128 (gethash (encode-char (make-char worth-trying i)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	129 'ucs) encode-table)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	130 (setq encode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	131 (nsublis
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	132 (list (cons #xABAB (charset-id worth-trying)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	133 (nconc
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	134 (copy-list (first
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	135 tentative-encode-program-parts))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	136 (append other-charset-vector nil)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	137 (copy-tree (second
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	138 tentative-encode-program-parts))))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	139 (and encode-program (values encode-program worth-trying))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	140
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	141 (defun fixed-width-generate-encode-program-and-skip-chars-strings
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	142 (decode-table encode-table encode-failure-octet)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	143 "Generate a CCL program to encode a 8-bit fixed-width charset.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	144
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	145 DECODE-TABLE must have 256 non-cons entries, and will be regarded as
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	146 describing a map from the octet corresponding to an offset in the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	147 table to the that entry in the table. ENCODE-TABLE is a hash table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	148 map from unicode values to characters in the range [0,255].
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	149 ENCODE-FAILURE-OCTET describes an integer between 0 and 255
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	150 \(inclusive) to write in the event that a character cannot be encoded."
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	151 (check-argument-type #'vectorp decode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	152 (check-argument-range (length decode-table) #x100 #x100)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	153 (check-argument-type #'hash-table-p encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	154 (check-argument-type #'integerp encode-failure-octet)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	155 (check-argument-range encode-failure-octet #x00 #xFF)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	156 (let ((encode-program nil)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	157 (general-encode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	158 (eval-when-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	159 (let ((prog (append
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	160 (ccl-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	161 `(1
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	162 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	163 (read-multibyte-character r0 r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	164 (mule-to-unicode r0 r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	165 (if (r0 == #xFFFD)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	166 (write #xBEEF)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	167 ((lookup-integer encode-table-sym r0 r3)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	168 (if r7
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	169 (write-multibyte-character r0 r3)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	170 (write #xBEEF))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	171 (repeat)))) nil)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	172 (assert (memq #xBEEF14 prog)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	173 nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	174 "This code assumes that the constant #xBEEF is #xBEEF14 \
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	175 in compiled CCL code.\nIf that is not the case, and it appears not to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	176 be--that's why you're getting this message--it will not work. ")
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	177 prog)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	178 (encode-program-with-ascii-optimisation
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	179 (eval-when-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	180 (let ((prog (append
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	181 (ccl-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	182 `(1
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	183 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	184 (read-multibyte-character r0 r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	185 (if (r0 == ,(charset-id 'ascii))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	186 (write r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	187 ((mule-to-unicode r0 r1)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	188 (if (r0 == #xFFFD)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	189 (write #xBEEF)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	190 ((lookup-integer encode-table-sym r0 r3)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	191 (if r7
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	192 (write-multibyte-character r0 r3)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	193 (write #xBEEF))))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	194 (repeat)))) nil)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	195 (assert (memq #xBEEF14 prog)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	196 nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	197 "This code assumes that the constant #xBEEF is #xBEEF14 \
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	198 in compiled CCL code.\nIf that is not the case, and it appears not to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	199 be--that's why you're getting this message--it will not work. ")
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	200 prog)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	201 (ascii-encodes-as-itself nil)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	202 (control-1-encodes-as-itself t)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	203 (invalid-sequence-code-point-start
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	204 (eval-when-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	205 (char-to-unicode
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	206 (aref (decode-coding-string "\xd8\x00\x00\x00" 'utf-16-be) 3))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	207 further-char-set skip-chars invalid-sequences-skip-chars)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	208
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	209 ;; Is this coding system ASCII-compatible? If so, we can avoid the hash
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	210 ;; table lookup for those characters.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	211 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	212 for i from #x00 to #x7f
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	213 always (eq (int-to-char i) (gethash i encode-table))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	214 finally (setq ascii-encodes-as-itself t))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	215
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	216 ;; Note that this logic handles EBCDIC badly. For example, CP037,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	217 ;; MIME name ebcdic-na, has the entire repertoire of ASCII and
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	218 ;; Latin 1, and thus a more optimal ccl encode program would check
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	219 ;; for those character sets and use tables. But for now, we do a
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	220 ;; hash table lookup for every character.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	221 (if (null ascii-encodes-as-itself)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	222 ;; General encode program. Pros; general and correct. Cons;
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	223 ;; slow, a hash table lookup + mule-unicode conversion is done
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	224 ;; for every character encoding.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	225 (setq encode-program general-encode-program)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	226 (multiple-value-setq
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	227 (encode-program further-char-set)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	228 ;; Encode program with ascii-ascii mapping (based on a
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	229 ;; character's mule character set), and one other mule
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	230 ;; character set using table-based encoding, other
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	231 ;; character sets using hash table lookups.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	232 ;; fixed-width-non-ascii-completely-coveredp only returns
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	233 ;; such a mapping if some non-ASCII charset with
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	234 ;; characters in decode-table is entirely covered by
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	235 ;; encode-table.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	236 (fixed-width-generate-helper decode-table encode-table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	237 encode-failure-octet))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	238 (unless encode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	239 ;; If fixed-width-non-ascii-completely-coveredp returned nil,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	240 ;; but ASCII still encodes as itself, do one-to-one mapping
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	241 ;; for ASCII, and a hash table lookup for everything else.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	242 (setq encode-program encode-program-with-ascii-optimisation)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	243
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	244 (setq encode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	245 (nsublis
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	246 (list (cons #xBEEF14
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	247 (logior (lsh encode-failure-octet 8)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	248 #x14)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	249 (copy-tree encode-program)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	250 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	251 for i from #x80 to #x9f
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	252 do (unless (= i (aref decode-table i))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	253 (setq control-1-encodes-as-itself nil)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	254 (return)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	255 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	256 for i from #x00 to #xFF
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	257 initially (setq skip-chars
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	258 (cond
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	259 ((and ascii-encodes-as-itself
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	260 control-1-encodes-as-itself further-char-set)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	261 (concat "\x00-\x9f" (charset-skip-chars-string
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	262 further-char-set)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	263 ((and ascii-encodes-as-itself
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	264 control-1-encodes-as-itself)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	265 "\x00-\x9f")
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	266 ((null ascii-encodes-as-itself)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	267 (skip-chars-quote (apply #'string
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	268 (append decode-table nil))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	269 (further-char-set
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	270 (concat (charset-skip-chars-string 'ascii)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	271 (charset-skip-chars-string further-char-set)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	272 (t
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	273 (charset-skip-chars-string 'ascii)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	274 invalid-sequences-skip-chars "")
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	275 with decoded-ucs = nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	276 with decoded = nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	277 with no-ascii-transparency-skip-chars-list =
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	278 (unless ascii-encodes-as-itself (append decode-table nil))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	279 ;; Can't use #'match-string here, see:
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	280 ;; http://mid.gmane.org/18829.34118.709782.704574@parhasard.net
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	281 with skip-chars-test =
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	282 #'(lambda (skip-chars-string testing)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	283 (with-temp-buffer
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	284 (insert testing)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	285 (goto-char (point-min))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	286 (skip-chars-forward skip-chars-string)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	287 (= (point) (point-max))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	288 do
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	289 (setq decoded (aref decode-table i)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	290 decoded-ucs (char-to-unicode decoded))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	291 (cond
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	292 ((<= invalid-sequence-code-point-start decoded-ucs
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	293 (+ invalid-sequence-code-point-start #xFF))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	294 (setq invalid-sequences-skip-chars
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	295 (concat (string decoded)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	296 invalid-sequences-skip-chars))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	297 (assert (not (funcall skip-chars-test skip-chars decoded))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	298 "This char should only be skipped with \
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	299 `invalid-sequences-skip-chars', not by `skip-chars'"))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	300 ((not (funcall skip-chars-test skip-chars decoded))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	301 (if ascii-encodes-as-itself
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	302 (setq skip-chars (concat skip-chars (string decoded)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	303 (push decoded no-ascii-transparency-skip-chars-list))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	304 finally (unless ascii-encodes-as-itself
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	305 (setq skip-chars
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	306 (skip-chars-quote
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	307 (apply #'string
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	308 no-ascii-transparency-skip-chars-list)))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	309 (values encode-program skip-chars invalid-sequences-skip-chars)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	310
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	311 (defun fixed-width-create-decode-encode-tables (unicode-map)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	312 "Return multiple values \(DECODE-TABLE ENCODE-TABLE) given UNICODE-MAP.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	313 UNICODE-MAP should be an alist mapping from integer octet values to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	314 characters with UCS code points; DECODE-TABLE will be a 256-element
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	315 vector, and ENCODE-TABLE will be a hash table mapping from 256 numbers
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	316 to 256 distinct characters."
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	317 (check-argument-type #'listp unicode-map)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	318 (let ((decode-table (make-vector 256 nil))
4777 c69aeb86b2a3 Serialise non-default hash table rehash thresholds correctly; use this. Aidan Kehoe <kehoea@parhasard.net> parents: 4692 diff changeset	319 (encode-table (make-hash-table :size 256 :rehash-threshold 0.999))
5068 c673987f5f3d dump make-coding-system Ben Wing <ben@xemacs.org> parents: 4777 diff changeset	320 (private-use-start #xE000)
4690 257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	321 (invalid-sequence-code-point-start
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	322 (eval-when-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	323 (char-to-unicode
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	324 (aref (decode-coding-string "\xd8\x00\x00\x00" 'utf-16-be) 3))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	325 desired-ucs decode-table-entry)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	326
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	327 (loop for (external internal)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	328 in unicode-map
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	329 do
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	330 (aset decode-table external internal)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	331 (assert (not (eq (encode-char internal 'ucs) -1))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	332 nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	333 "Looks like you're creating a fixed-width coding system \
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	334 in a dumped file, \nand you're either not providing a literal unicode map
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	335 or PROPS. Don't do that; fixed-width coding systems rely on sensible
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	336 Unicode mappings being available, which they are at compile time for
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	337 dumped files (but this requires the mentioned literals), but not, for
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	338 most of them, at run time. ")
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	339
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	340 (puthash (encode-char internal 'ucs)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	341 ;; This is semantically an integer, but Dave Love's design
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	342 ;; for lookup-integer in CCL means we need to store it as a
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	343 ;; character.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	344 (int-to-char external)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	345 encode-table))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	346
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	347 ;; Now, go through the decode table. For octet values above #x7f, if the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	348 ;; decode table entry is nil, this means that they have an undefined
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	349 ;; mapping (= they map to XEmacs characters with keys in
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	350 ;; unicode-error-default-translation-table); for octet values below or
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	351 ;; equal to #x7f, it means that they map to ASCII.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	352
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	353 ;; If any entry (whether below or above #x7f) in the decode-table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	354 ;; already maps to some character with a key in
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	355 ;; unicode-error-default-translation-table, it is treated as an
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	356 ;; undefined octet by `query-coding-region'. That is, it is not
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	357 ;; necessary for an octet value to be above #x7f for this to happen.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	358
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	359 (dotimes (i 256)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	360 (setq decode-table-entry (aref decode-table i))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	361 (if decode-table-entry
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	362 (when (get-char-table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	363 decode-table-entry
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	364 unicode-error-default-translation-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	365 ;; The caller is explicitly specifying that this octet
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	366 ;; corresponds to an invalid sequence on disk:
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	367 (assert (= (get-char-table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	368 decode-table-entry
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	369 unicode-error-default-translation-table) i)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	370 "Bad argument for a fixed-width coding system.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	371 If you're going to designate an octet with value below #x80 as invalid
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	372 for this coding system, make sure to map it to the invalid sequence
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	373 character corresponding to its octet value on disk. "))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	374
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	375 ;; decode-table-entry is nil; either the octet is to be treated as
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	376 ;; contributing to an error sequence (when (> #x7f i)), or it should
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	377 ;; be attempted to treat it as ASCII-equivalent.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	378 (setq desired-ucs (or (and (< i #x80) i)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	379 (+ invalid-sequence-code-point-start i)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	380 (while (gethash desired-ucs encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	381 (assert (not (< i #x80))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	382 "UCS code point should not already be in encode-table!"
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	383 ;; There is one invalid sequence char per octet value;
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	384 ;; with fixed-width coding systems, it makes no sense
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	385 ;; for us to be multiply allocating them.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	386 (gethash desired-ucs encode-table))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	387 (setq desired-ucs (+ private-use-start desired-ucs)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	388 private-use-start (+ private-use-start 1)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	389 (puthash desired-ucs (int-to-char i) encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	390 (setq desired-ucs (if (> desired-ucs #xFF)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	391 (unicode-to-char desired-ucs)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	392 ;; So we get Latin-1 when run at dump time,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	393 ;; instead of JIT-allocated characters.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	394 (int-to-char desired-ucs)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	395 (aset decode-table i desired-ucs)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	396 (values decode-table encode-table)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	397
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	398 (defun fixed-width-generate-decode-program (decode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	399 "Given DECODE-TABLE, generate a CCL program to decode an 8-bit charset.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	400 DECODE-TABLE must have 256 non-cons entries, and will be regarded as
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	401 describing a map from the octet corresponding to an offset in the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	402 table to the that entry in the table. "
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	403 (check-argument-type #'vectorp decode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	404 (check-argument-range (length decode-table) #x100 #x100)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	405 (let ((decode-program-parts
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	406 (eval-when-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	407 (let* ((compiled
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	408 (append
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	409 (ccl-compile
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	410 `(3
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	411 ((read r0)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	412 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	413 (write-read-repeat r0 ,(make-vector
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	414 256 'sentinel)))))) nil))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	415 (first-part compiled)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	416 (last-part
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	417 (member-if-not #'symbolp
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	418 (member-if-not #'integerp first-part))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	419 ;; Chop off the sentinel sentinel sentinel [..] part.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	420 (while compiled
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	421 (if (symbolp (cadr compiled))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	422 (setcdr compiled nil))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	423 (setq compiled (cdr compiled)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	424 (list first-part last-part)))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	425 (nconc
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	426 ;; copy-list needed, because the structure of the literal provided
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	427 ;; by our eval-when-compile hangs around.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	428 (copy-list (first decode-program-parts))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	429 (append decode-table nil)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	430 (second decode-program-parts))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	431
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	432 (defun fixed-width-choose-category (decode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	433 "Given DECODE-TABLE, return an appropriate coding category.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	434 DECODE-TABLE is a 256-entry vector describing the mapping from octets on
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	435 disk to XEmacs characters for some fixed-width 8-bit coding system."
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	436 (check-argument-type #'vectorp decode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	437 (check-argument-range (length decode-table) #x100 #x100)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	438 (loop
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	439 named category
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	440 for i from #x80 to #x9F
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	441 do (unless (= i (aref decode-table i))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	442 (return-from category 'no-conversion))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	443 finally return 'iso-8-1))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	444
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	445 (defun fixed-width-rework-props-runtime (name props)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	446 "Rework PROPS to a form understood by `make-coding-system-internal'.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	447
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	448 NAME must be a symbol, describing a fixed-width coding system that is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	449 about to be created. Much of the implementation of the fixed-width
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	450 coding system is in Lisp, and this function allows us to rework the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	451 arguments that `make-coding-system-internal' sees accordingly.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	452
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	453 If you are calling this function from anywhere but
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	454 `make-coding-system', you're probably doing something wrong."
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	455 (check-argument-type #'symbolp name)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	456 (check-valid-plist props)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	457 (let ((encode-failure-octet (or (plist-get props 'encode-failure-octet)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	458 (char-to-int ?~)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	459 (unicode-map (plist-get props 'unicode-map))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	460 (hash-table-sym (gensym (format "%s-encode-table" name)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	461 encode-program decode-program decode-table encode-table skip-chars
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	462 invalid-sequences-skip-chars category)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	463
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	464 (check-argument-range encode-failure-octet 0 #xFF)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	465 ;; unicode-map must be a true list, and must be non-nil.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	466 (check-argument-type #'true-list-p unicode-map)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	467 (check-argument-type #'consp unicode-map)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	468
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	469 ;; Don't pass on our extra data to make-coding-system-internal.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	470 (setq props (plist-remprop props 'encode-failure-octet)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	471 props (plist-remprop props 'unicode-map))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	472
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	473 (multiple-value-setq
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	474 (decode-table encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	475 (fixed-width-create-decode-encode-tables unicode-map))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	476
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	477 ;; Register the decode-table.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	478 (define-translation-hash-table hash-table-sym encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	479
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	480 ;; Generate the programs and skip-chars strings.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	481 (setq decode-program (fixed-width-generate-decode-program decode-table))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	482 (multiple-value-setq
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	483 (encode-program skip-chars invalid-sequences-skip-chars)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	484 (fixed-width-generate-encode-program-and-skip-chars-strings
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	485 decode-table encode-table encode-failure-octet))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	486
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	487 (setq category (fixed-width-choose-category decode-table))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	488
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	489 (unless (vectorp encode-program)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	490 (setq encode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	491 (apply #'vector
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	492 (nsublis (list (cons 'encode-table-sym hash-table-sym))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	493 (copy-tree encode-program)))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	494 (unless (vectorp decode-program)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	495 (setq decode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	496 (apply #'vector decode-program)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	497
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	498 (loop for (symbol . value)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	499 in `((decode . ,decode-program)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	500 (encode . ,encode-program)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	501 (from-unicode . ,encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	502 (query-skip-chars . ,skip-chars)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	503 (invalid-sequences-skip-chars . ,invalid-sequences-skip-chars)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	504 (category . ,category))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	505 with default = (gensym)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	506 do
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	507 (unless (eq default (plist-get props symbol default))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	508 (error
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	509 'invalid-argument
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	510 "Explicit property not allowed for fixed-width coding systems"
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	511 symbol))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	512 (setq props (nconc (list symbol value) props)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	513 props))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	514
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	515 ;;;###autoload
4692 dc3c2f298857 Support last two arguments to #'make-coding-system being optional, again. Aidan Kehoe <kehoea@parhasard.net> parents: 4690 diff changeset	516 (defun make-coding-system (name type &optional description props)
4690 257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	517 "Register symbol NAME as a coding system.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	518
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	519 TYPE describes the conversion method used and should be one of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	520
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	521 nil or `undecided'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	522 Automatic conversion. XEmacs attempts to detect the coding system
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	523 used in the file.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	524 `chain'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	525 Chain two or more coding systems together to make a combination coding
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	526 system.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	527 `no-conversion'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	528 No conversion. Use this for binary files and such. On output,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	529 graphic characters that are not in ASCII or Latin-1 will be
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	530 replaced by a ?. (For a no-conversion-encoded buffer, these
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	531 characters will only be present if you explicitly insert them.)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	532 `convert-eol'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	533 Convert CRLF sequences or CR to LF.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	534 `shift-jis'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	535 Shift-JIS (a Japanese encoding commonly used in PC operating systems).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	536 `unicode'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	537 Any Unicode encoding (UCS-4, UTF-8, UTF-16, etc.).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	538 `mswindows-unicode-to-multibyte'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	539 (MS Windows only) Converts from Windows Unicode to Windows Multibyte
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	540 (any code page encoding) upon encoding, and the other way upon decoding.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	541 `mswindows-multibyte'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	542 Converts to or from Windows Multibyte (any code page encoding).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	543 This is resolved into a chain of `mswindows-unicode' and
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	544 `mswindows-unicode-to-multibyte'.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	545 `iso2022'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	546 Any ISO2022-compliant encoding. Among other things, this includes
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	547 JIS (the Japanese encoding commonly used for e-mail), EUC (the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	548 standard Unix encoding for Japanese and other languages), and
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	549 Compound Text (the encoding used in X11). You can specify more
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	550 specific information about the conversion with the PROPS argument.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	551 `fixed-width'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	552 A fixed-width eight bit encoding that is not necessarily compliant with
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	553 ISO 2022. This coding system assumes Unicode equivalency, that is, if
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	554 two given XEmacs characters have the same Unicode mapping, they will
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	555 always map to the same octet on disk.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	556 `big5'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	557 Big5 (the encoding commonly used for Mandarin Chinese in Taiwan).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	558 `ccl'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	559 The conversion is performed using a user-written pseudo-code
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	560 program. CCL (Code Conversion Language) is the name of this
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	561 pseudo-code.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	562 `gzip'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	563 GZIP compression format.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	564 `internal'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	565 Write out or read in the raw contents of the memory representing
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	566 the buffer's text. This is primarily useful for debugging
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	567 purposes, and is only enabled when XEmacs has been compiled with
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	568 DEBUG_XEMACS defined (via the --debug configure option).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	569 WARNING: Reading in a file using `internal' conversion can result
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	570 in an internal inconsistency in the memory representing a
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	571 buffer's text, which will produce unpredictable results and may
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	572 cause XEmacs to crash. Under normal circumstances you should
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	573 never use `internal' conversion.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	574
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	575 DESCRIPTION is a short English phrase describing the coding system,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	576 suitable for use as a menu item. (See also the `documentation' property
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	577 below.)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	578
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	579 PROPS is a property list, describing the specific nature of the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	580 character set. Recognized properties are:
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	581
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	582 `mnemonic'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	583 String to be displayed in the modeline when this coding system is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	584 active.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	585
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	586 `documentation'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	587 Detailed documentation on the coding system.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	588
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	589 `aliases'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	590 A list of aliases for the coding system. See
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	591 `define-coding-system-alias'.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	592
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	593 `eol-type'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	594 End-of-line conversion to be used. It should be one of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	595
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	596 nil
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	597 Automatically detect the end-of-line type (LF, CRLF,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	598 or CR). Also generate subsidiary coding systems named
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	599 `NAME-unix', `NAME-dos', and `NAME-mac', that are
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	600 identical to this coding system but have an EOL-TYPE
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	601 value of `lf', `crlf', and `cr', respectively.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	602 `lf'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	603 The end of a line is marked externally using ASCII LF.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	604 Since this is also the way that XEmacs represents an
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	605 end-of-line internally, specifying this option results
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	606 in no end-of-line conversion. This is the standard
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	607 format for Unix text files.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	608 `crlf'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	609 The end of a line is marked externally using ASCII
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	610 CRLF. This is the standard format for MS-DOS text
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	611 files.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	612 `cr'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	613 The end of a line is marked externally using ASCII CR.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	614 This is the standard format for Macintosh text files.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	615 t
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	616 Automatically detect the end-of-line type but do not
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	617 generate subsidiary coding systems. (This value is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	618 converted to nil when stored internally, and
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	619 `coding-system-property' will return nil.)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	620
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	621 `post-read-conversion'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	622 The value is a function to call after some text is inserted and
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	623 decoded by the coding system itself and before any functions in
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	624 `after-change-functions' are called. (#### Not actually true in
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	625 XEmacs. `after-change-functions' will be called twice if
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	626 `post-read-conversion' changes something.) The argument of this
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	627 function is the same as for a function in
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	628 `after-insert-file-functions', i.e. LENGTH of the text inserted,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	629 with point at the head of the text to be decoded.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	630
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	631 `pre-write-conversion'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	632 The value is a function to call after all functions in
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	633 `write-region-annotate-functions' and `buffer-file-format' are
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	634 called, and before the text is encoded by the coding system itself.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	635 The arguments to this function are the same as those of a function
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	636 in `write-region-annotate-functions', i.e. FROM and TO, specifying
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	637 a region of text.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	638
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	639 The following properties are used by `default-query-coding-region',
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	640 the default implementation of `query-coding-region'. This
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	641 implementation and these properties are not used by the Unicode coding
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	642 systems, nor by fixed-width coding systems.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	643
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	644 `safe-chars'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	645 The value is a char table. If a character has non-nil value in it,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	646 the character is safely supported by the coding system.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	647 This overrides the `safe-charsets' property.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	648
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	649 `safe-charsets'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	650 The value is a list of charsets safely supported by the coding
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	651 system. For coding systems based on ISO 2022, XEmacs may try to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	652 encode characters outside these character sets, but outside of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	653 East Asia and East Asian coding systems, it is unlikely that
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	654 consumers of the data will understand XEmacs' encoding.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	655 The value t means that all XEmacs character sets handles are supported.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	656
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	657 The following properties are allowed for FSF compatibility but currently
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	658 ignored:
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	659
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	660 `translation-table-for-decode'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	661 The value is a translation table to be applied on decoding. See
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	662 the function `make-translation-table' for the format of translation
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	663 table. This is not applicable to CCL-based coding systems.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	664
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	665 `translation-table-for-encode'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	666 The value is a translation table to be applied on encoding. This is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	667 not applicable to CCL-based coding systems.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	668
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	669 `mime-charset'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	670 The value is a symbol of which name is `MIME-charset' parameter of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	671 the coding system.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	672
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	673 `valid-codes' (meaningful only for a coding system based on CCL)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	674 The value is a list to indicate valid byte ranges of the encoded
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	675 file. Each element of the list is an integer or a cons of integer.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	676 In the former case, the integer value is a valid byte code. In the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	677 latter case, the integers specifies the range of valid byte codes.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	678
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	679 The following additional property is recognized if TYPE is `convert-eol':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	680
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	681 `subtype'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	682 One of `lf', `crlf', `cr' or nil (for autodetection). When decoding,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	683 the corresponding sequence will be converted to LF. When encoding,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	684 the opposite happens. This coding system converts characters to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	685 characters.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	686
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	687
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	688
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	689 The following additional properties are recognized if TYPE is `iso2022':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	690
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	691 `charset-g0'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	692 `charset-g1'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	693 `charset-g2'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	694 `charset-g3'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	695 The character set initially designated to the G0 - G3 registers.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	696 The value should be one of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	697
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	698 -- A charset object (designate that character set)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	699 -- nil (do not ever use this register)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	700 -- t (no character set is initially designated to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	701 the register, but may be later on; this automatically
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	702 sets the corresponding `force-g*-on-output' property)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	703
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	704 `force-g0-on-output'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	705 `force-g1-on-output'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	706 `force-g2-on-output'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	707 `force-g2-on-output'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	708 If non-nil, send an explicit designation sequence on output before
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	709 using the specified register.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	710
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	711 `short'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	712 If non-nil, use the short forms \"ESC $ @\", \"ESC $ A\", and
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	713 \"ESC $ B\" on output in place of the full designation sequences
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	714 \"ESC $ ( @\", \"ESC $ ( A\", and \"ESC $ ( B\".
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	715
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	716 `no-ascii-eol'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	717 If non-nil, don't designate ASCII to G0 at each end of line on output.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	718 Setting this to non-nil also suppresses other state-resetting that
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	719 normally happens at the end of a line.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	720
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	721 `no-ascii-cntl'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	722 If non-nil, don't designate ASCII to G0 before control chars on output.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	723
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	724 `seven'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	725 If non-nil, use 7-bit environment on output. Otherwise, use 8-bit
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	726 environment.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	727
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	728 `lock-shift'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	729 If non-nil, use locking-shift (SO/SI) instead of single-shift
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	730 or designation by escape sequence.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	731
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	732 `no-iso6429'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	733 If non-nil, don't use ISO6429's direction specification.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	734
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	735 `escape-quoted'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	736 If non-nil, literal control characters that are the same as
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	737 the beginning of a recognized ISO2022 or ISO6429 escape sequence
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	738 (in particular, ESC (0x1B), SO (0x0E), SI (0x0F), SS2 (0x8E),
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	739 SS3 (0x8F), and CSI (0x9B)) are \"quoted\" with an escape character
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	740 so that they can be properly distinguished from an escape sequence.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	741 (Note that doing this results in a non-portable encoding.) This
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	742 encoding flag is used for byte-compiled files. Note that ESC
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	743 is a good choice for a quoting character because there are no
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	744 escape sequences whose second byte is a character from the Control-0
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	745 or Control-1 character sets; this is explicitly disallowed by the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	746 ISO2022 standard.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	747
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	748 `input-charset-conversion'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	749 A list of conversion specifications, specifying conversion of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	750 characters in one charset to another when decoding is performed.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	751 Each specification is a list of two elements: the source charset,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	752 and the destination charset.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	753
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	754 `output-charset-conversion'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	755 A list of conversion specifications, specifying conversion of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	756 characters in one charset to another when encoding is performed.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	757 The form of each specification is the same as for
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	758 `input-charset-conversion'.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	759
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	760 The following additional properties are recognized if TYPE is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	761 `fixed-width':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	762
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	763 `unicode-map'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	764 Required. A plist describing a map from octets in the coding system
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	765 NAME (as integers) to XEmacs characters. Those XEmacs characters will
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	766 be used explicitly on decoding, but for encoding (most relevantly, on
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	767 writing to disk) XEmacs characters that map to the same Unicode code
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	768 point will be unified. This means that the ISO-8859-? characters that
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	769 map to the same Unicode code point will not be distinct when written to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	770 disk, which is normally what is intended; it also means that East Asian
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	771 Han characters from different XEmacs character sets will not be
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	772 distinct when written to disk, which is less often what is intended.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	773
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	774 Any octets not mapped, and with values above #x7f, will be decoded into
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	775 XEmacs characters that reflect that their values are undefined. These
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	776 characters will be displayed in a language-environment-specific
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	777 way. See `unicode-error-default-translation-table' and the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	778 `invalid-sequence-coding-system' argument to `set-language-info'.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	779
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	780 These characters will normally be treated as invalid when checking
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	781 whether text can be encoded with `query-coding-region'--see the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	782 IGNORE-INVALID-SEQUENCESP argument to that function to avoid this. It
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	783 is possible to specify that octets with values less than #x80 (or
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	784 indeed greater than it) be treated in this way, by specifying
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	785 explicitly that they correspond to the character mapping to that octet
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	786 in `unicode-error-default-translation-table'. Far fewer coding systems
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	787 override the ASCII mapping, though, so this is not the default.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	788
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	789 `encode-failure-octet'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	790 An integer between 0 and 255 to write in place of XEmacs characters
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	791 that cannot be encoded, defaulting to the code for tilde `~'.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	792
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	793 The following additional properties are recognized (and required)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	794 if TYPE is `ccl':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	795
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	796 `decode'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	797 CCL program used for decoding (converting to internal format).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	798
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	799 `encode'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	800 CCL program used for encoding (converting to external format).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	801
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	802
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	803 The following additional properties are recognized if TYPE is `chain':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	804
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	805 `chain'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	806 List of coding systems to be chained together, in decoding order.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	807
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	808 `canonicalize-after-coding'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	809 Coding system to be returned by the detector routines in place of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	810 this coding system.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	811
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	812
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	813
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	814 The following additional properties are recognized if TYPE is `unicode':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	815
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	816 `unicode-type'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	817 One of `utf-16', `utf-8', `ucs-4', or `utf-7' (the latter is not
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	818 yet implemented). `utf-16' is the basic two-byte encoding;
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	819 `ucs-4' is the four-byte encoding; `utf-8' is an ASCII-compatible
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	820 variable-width 8-bit encoding; `utf-7' is a 7-bit encoding using
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	821 only characters that will safely pass through all mail gateways.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	822 [[ This should be \"transformation format\". There should also be
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	823 `ucs-2' (or `bmp' -- no surrogates) and `utf-32' (range checked). ]]
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	824
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	825 `little-endian'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	826 If non-nil, `utf-16' and `ucs-4' will write out the groups of two
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	827 or four bytes little-endian instead of big-endian. This is required,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	828 for example, under Windows.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	829
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	830 `need-bom'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	831 If non-nil, a byte order mark (BOM, or Unicode FFFE) should be
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	832 written out at the beginning of the data. This serves both to
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	833 identify the endianness of the following data and to mark the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	834 data as Unicode (at least, this is how Windows uses it).
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	835 [[ The correct term is \"signature\", since this technique may also
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	836 be used with UTF-8. That is the term used in the standard. ]]
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	837
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	838
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	839 The following additional properties are recognized if TYPE is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	840 `mswindows-multibyte':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	841
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	842 `code-page'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	843 Either a number (specifying a particular code page) or one of the
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	844 symbols `ansi', `oem', `mac', or `ebcdic', specifying the ANSI,
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	845 OEM, Macintosh, or EBCDIC code page associated with a particular
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	846 locale (given by the `locale' property). NOTE: EBCDIC code pages
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	847 only exist in Windows 2000 and later.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	848
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	849 `locale'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	850 If `code-page' is a symbol, this specifies the locale whose code
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	851 page of the corresponding type should be used. This should be
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	852 one of the following: A cons of two strings, (LANGUAGE
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	853 . SUBLANGUAGE) (see `mswindows-set-current-locale'); a string (a
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	854 language; SUBLANG_DEFAULT, i.e. the default sublanguage, is
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	855 used); or one of the symbols `current', `user-default', or
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	856 `system-default', corresponding to the values of
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	857 `mswindows-current-locale', `mswindows-user-default-locale', or
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	858 `mswindows-system-default-locale', respectively.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	859
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	860
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	861 The following additional properties are recognized if TYPE is `undecided':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	862 \[[ Doesn't GNU use \"detect-*\" for the following two? ]]
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	863
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	864 `do-eol'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	865 Do EOL detection.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	866
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	867 `do-coding'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	868 Do encoding detection.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	869
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	870 `coding-system'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	871 If encoding detection is not done, use the specified coding system
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	872 to do decoding. This is used internally when implementing coding
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	873 systems with an EOL type that specifies autodetection (the default),
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	874 so that the detector routines return the proper subsidiary.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	875
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	876
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	877
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	878 The following additional property is recognized if TYPE is `gzip':
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	879
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	880 `level'
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	881 Compression level: 0 through 9, or `default' (currently 6)."
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	882 (when (eq 'fixed-width type)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	883 (setq props (fixed-width-rework-props-runtime name props)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	884 (make-coding-system-internal name type description props))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	885
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	886 (define-compiler-macro make-coding-system (&whole form name type
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	887 &optional description props)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	888 (if (equal '(quote fixed-width) type)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	889 (if (memq (car-safe props) '(quote eval-when-compile))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	890 (let* ((props (if (eq 'eval-when-compile (car props))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	891 (eval (cadr props))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	892 (cadr props)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	893 (encode-failure-octet
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	894 (or (plist-get props 'encode-failure-octet)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	895 (char-to-int ?~)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	896 (unicode-map (plist-get props 'unicode-map))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	897 (default-plist-entry (gensym))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	898 (encode-table-sym (gensym
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	899 (if (eq 'quote (car name))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	900 (format "%s-enc-" (second name)))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	901 encode-program decode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	902 decode-table encode-table
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	903 skip-chars invalid-sequences-skip-chars category)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	904
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	905 (check-argument-range encode-failure-octet 0 #xFF)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	906 ;; unicode-map must be a true list, and must be non-nil.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	907 (check-argument-type #'true-list-p unicode-map)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	908 (check-argument-type #'consp unicode-map)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	909
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	910 ;; Don't pass on our extra data to make-coding-system-internal.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	911 (setq props (plist-remprop props 'encode-failure-octet)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	912 props (plist-remprop props 'unicode-map))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	913
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	914 (multiple-value-setq
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	915 (decode-table encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	916 (fixed-width-create-decode-encode-tables unicode-map))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	917
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	918 ;; Generate the decode and encode programs, and the skip-chars
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	919 ;; arguments.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	920 (setq decode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	921 (fixed-width-generate-decode-program decode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	922 category (fixed-width-choose-category decode-table))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	923
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	924 (multiple-value-setq
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	925 (encode-program skip-chars invalid-sequences-skip-chars)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	926 (fixed-width-generate-encode-program-and-skip-chars-strings
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	927 decode-table encode-table encode-failure-octet))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	928
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	929 (unless (vectorp decode-program)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	930 (setq decode-program
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	931 (apply #'vector decode-program)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	932
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	933 (unless (eq default-plist-entry (plist-get props 'encode
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	934 default-plist-entry))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	935 (error
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	936 'invalid-argument
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	937 "Explicit property not allowed for fixed-width coding system"
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	938 'encode))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	939 (loop for (symbol . value)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	940 in `((decode . ,decode-program)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	941 (from-unicode . ,encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	942 (query-skip-chars . ,skip-chars)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	943 (invalid-sequences-skip-chars .
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	944 ,invalid-sequences-skip-chars)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	945 (category . ,category))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	946 do
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	947 (unless (eq default-plist-entry (plist-get props symbol
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	948 default-plist-entry))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	949 (error
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	950 'invalid-argument
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	951 "Explicit property not allowed for \
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	952 fixed-width coding systems"
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	953 symbol))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	954 (setq props (nconc (list symbol value) props)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	955 `(progn
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	956 (define-translation-hash-table ',encode-table-sym ,encode-table)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	957 (make-coding-system-internal
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	958 ,name ,type ,description
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	959 ',(nconc (list 'encode
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	960 (apply #'vector
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	961 (nsublis
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	962 (list (cons 'encode-table-sym
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	963 encode-table-sym))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	964 encode-program)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	965 props))))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	966 ;; The form does not use literals; call make-coding-system at
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	967 ;; run time.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	968 form)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	969 (if (byte-compile-constp type)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	970 ;; This is not a fixed-width call; compile it to a form that 21.4
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	971 ;; can also understand.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	972 `(funcall (or (and (fboundp 'make-coding-system-internal)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	973 'make-coding-system-internal)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	974 'make-coding-system)
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	975 ,@(cdr form))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	976 ;; TYPE is not literal; work things out at runtime.
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	977 form)))
257b468bf2ca Move the #'query-coding-region implementation to C. Aidan Kehoe <kehoea@parhasard.net> parents: diff changeset	978

Mercurial > hg > xemacs-beta

annotate lisp/mule/make-coding-system.el @ 5068:c673987f5f3d