annotate tests/automated/lisp-reader-tests.el @ 4604:e0a8715fdb1f

Support new IGNORE-INVALID-SEQUENCESP argument, #'query-coding-region. lisp/ChangeLog addition: 2009-02-07 Aidan Kehoe <kehoea@parhasard.net> * coding.el (query-coding-clear-highlights): Rename the BUFFER argument to BUFFER-OR-STRING, describe it as possibly being a string in its documentation. (default-query-coding-region): Add a new IGNORE-INVALID-SEQUENCESP argument, document that this function does not support it. Bind case-fold-search to nil, we don't want this to influence what the function thinks is encodable or not. (query-coding-region): Add a new IGNORE-INVALID-SEQUENCESP argument, document what it does; reflect this new argument in the associated compiler macro. (query-coding-string): Add a new IGNORE-INVALID-SEQUENCESP argument, document what it does. Support the HIGHLIGHT argument correctly. * unicode.el (unicode-query-coding-region): Add a new IGNORE-INVALID-SEQUENCESP argument, document what it does, implement this. Document a potential problem. Use #'query-coding-clear-highlights instead of reimplementing it ourselves. Remove some debugging messages. * mule/arabic.el (iso-8859-6): * mule/cyrillic.el (iso-8859-5): * mule/greek.el (iso-8859-7): * mule/hebrew.el (iso-8859-8): * mule/latin.el (iso-8859-2): * mule/latin.el (iso-8859-3): * mule/latin.el (iso-8859-4): * mule/latin.el (iso-8859-14): * mule/latin.el (iso-8859-15): * mule/latin.el (iso-8859-16): * mule/latin.el (iso-8859-9): * mule/latin.el (windows-1252): * mule/mule-coding.el (iso-8859-1): Avoid the assumption that characters not given an explicit mapping in these coding systems map to the ISO 8859-1 characters corresponding to the octets on disk; this makes it much more reasonable to implement the IGNORE-INVALID-SEQUENCESP argument to query-coding-region. * mule/mule-cmds.el (set-language-info): Correct the docstring. * mule/mule-cmds.el (finish-set-language-environment): Treat invalid Unicode sequences produced from invalid-sequence-coding-system and corresponding to control characters the same as control characters in redisplay. * mule/mule-cmds.el: Document that encode-coding-char is available in coding.el * mule/mule-coding.el (make-8-bit-generate-helper): Change to return the both the encode-program generated and the relevant non-ASCII charset; update the docstring to reflect this. * mule/mule-coding.el (make-8-bit-generate-encode-program-and-skip-chars-strings): Rename this function; have it return skip-chars-strings as well as the encode program. Have these skip-chars-strings use ranges for charsets, where possible. * mule/mule-coding.el (make-8-bit-create-decode-encode-tables): Revise this to allow people to specify explicitly characters that should be undefined (= corresponding to keys in unicode-error-default-translation-table), and treating unspecified octets above #x7f as undefined by default. * mule/mule-coding.el (8-bit-fixed-query-coding-region): Add a new IGNORE-INVALID-SEQUENCESP argument, implement support for it using the 8-bit-fixed-invalid-sequences-skip-chars coding system property; remove some debugging messages. * mule/mule-coding.el (make-8-bit-coding-system): This function is dumped, autoloading it makes no sense. Document what happens when characters above #x7f are not specified, implement this. * mule/vietnamese.el: Correct spelling. tests/ChangeLog addition: 2009-02-07 Aidan Kehoe <kehoea@parhasard.net> * automated/query-coding-tests.el: Add FAILING-CASE arguments to the Assert calls, making #'q-c-debug mostly unnecessary. Remove #'q-c-debug. Add new tests that use the IGNORE-INVALID-SEQUENCESP argument to #'query-coding-region; rework the existing ones to respect it.
author Aidan Kehoe <kehoea@parhasard.net>
date Sat, 07 Feb 2009 17:13:37 +0000
parents c136144fe765
children e6508b64ee08 308d34e9f07d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
3543
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
1 ;; Copyright (C) 2005 Martin Kuehl.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
2
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
3 ;; Author: Martin Kuehl <martin.kuehl@gmail.com>
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
4 ;; Maintainer: Martin Kuehl <martin.kuehl@gmail.com>
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
5 ;; Created: 2005
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
6 ;; Keywords: tests
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
7
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
8 ;; This file is NOT part of SXEmacs.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
9
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
10 ;; SXEmacs is free software; you can redistribute it and/or modify it
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
11 ;; under the terms of the GNU General Public License as published by
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
12 ;; the Free Software Foundation; either version 2, or (at your option)
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
13 ;; any later version.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
14
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
15 ;; SXEmacs is distributed in the hope that it will be useful, but
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
16 ;; WITHOUT ANY WARRANTY; without even the implied warranty of
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
17 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
18 ;; General Public License for more details.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
19
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
20 ;; You should have received a copy of the GNU General Public License
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
21 ;; along with SXEmacs; see the file COPYING. If not, write to the Free
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
22 ;; Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
23 ;; 02111-1307, USA.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
24
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
25 ;;; Synched up with: Not in FSF.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
26
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
27 ;;; Commentary:
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
28
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
29 ;; Test the lisp reader.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
30 ;; See test-harness.el for instructions on how to run these tests.
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
31
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
32 ;;; Raw Strings
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
33 ;;; ===========
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
34
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
35 ;; Equality to "traditional" strings
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
36 ;; ---------------------------------
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
37 (dolist (strings '((#r"xyz" "xyz") ; no backslashes
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
38 (#r"\xyz" "\\xyz") ; backslash at start
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
39 (#r"\\xyz" "\\\\xyz") ; backslashes at start
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
40 (#r"\nxyz" "\\nxyz") ; escape seq. at start
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
41 (#r"\"xyz" "\\\"xyz") ; quote at start
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
42 (#r"xy\z" "xy\\z") ; backslash in middle
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
43 (#r"xy\\z" "xy\\\\z") ; backslashes in middle
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
44 (#r"xy\nz" "xy\\nz") ; escape seq. in middle
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
45 (#r"xy\"z" "xy\\\"z") ; quote in middle
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
46 ;;(#r"xyz\" "xyz\\") ; backslash at end: error
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
47 (#r"xyz\\" "xyz\\\\") ; backslashes at end
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
48 (#r"xyz\n" "xyz\\n") ; escape seq. at end
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
49 (#r"xyz\"" "xyz\\\"") ; quote at end
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
50 (#ru"\u00ABxyz" "\u00ABxyz") ; one Unicode escape
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
51 (#rU"\U000000ABxyz" "\U000000ABxyz") ; another Unicode escape
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
52 (#rU"xyz\u00AB" "xyz\u00AB") ; one Unicode escape
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
53 ))
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
54 (Assert (apply #'string= strings)))
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
55
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
56 ;; Odd number of backslashes at the end
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
57 ;; ------------------------------------
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
58 (dolist (string '("#r\"xyz\\\"" ; `#r"abc\"': escaped delimiter
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
59 "#r\"xyz\\\\\\\"" ; `#r"abc\\\"': escaped delimiter
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
60 ))
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
61 (with-temp-buffer
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
62 (insert string)
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
63 (Check-Error end-of-file (eval-buffer))))
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
64
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
65 ;; Alternate string/regex delimiters
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
66 ;; ---------------------------------
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
67 (dolist (string '("#r/xyz/" ; Perl syntax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
68 "#r:ix/xyz/" ; Extended Perl syntax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
69 "#r|xyz|" ; TeX syntax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
70 "#r[xyz]" ; (uncommon) Perl syntax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
71 "#r<xyz>" ; Perl6 syntax?
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
72 "#r(xyz)" ; arbitrary santax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
73 "#r{xyz}" ; arbitrary santax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
74 "#r,xyz," ; arbitrary santax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
75 "#r!xyz!" ; arbitrary santax
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
76 ))
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
77 (with-temp-buffer
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
78 (insert string)
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
79 (Check-Error-Message invalid-read-syntax "unrecognized raw string"
c136144fe765 [xemacs-hg @ 2006-08-04 22:55:04 by aidan]
aidan
parents:
diff changeset
80 (eval-buffer))))