Mercurial > hg > xemacs-beta
comparison src/file-coding.h @ 4690:257b468bf2ca
Move the #'query-coding-region implementation to C.
This is necessary because there is no reasonable way to access the
corresponding mswindows-multibyte functionality from Lisp, and we need such
functionality if we're going to have a reliable and portable
#'query-coding-region implementation. However, this change doesn't yet
provide #'query-coding-region for the mswindow-multibyte coding systems,
there should be no functional differences between an XEmacs with this change
and one without it.
src/ChangeLog addition:
2009-09-19 Aidan Kehoe <kehoea@parhasard.net>
Move the #'query-coding-region implementation to C.
This is necessary because there is no reasonable way to access the
corresponding mswindows-multibyte functionality from Lisp, and we
need such functionality if we're going to have a reliable and
portable #'query-coding-region implementation. However, this
change doesn't yet provide #'query-coding-region for the
mswindow-multibyte coding systems, there should be no functional
differences between an XEmacs with this change and one without it.
* mule-coding.c (struct fixed_width_coding_system):
Add a new coding system type, fixed_width, and implement it. It
uses the CCL infrastructure but has a much simpler creation API,
and its own query_method, formerly in lisp/mule/mule-coding.el.
* unicode.c:
Move the Unicode query method implementation here from
unicode.el.
* lisp.h: Declare Fmake_coding_system_internal, Fcopy_range_table
here.
* intl-win32.c (complex_vars_of_intl_win32):
Use Fmake_coding_system_internal, not Fmake_coding_system.
* general-slots.h: Add Qsucceeded, Qunencodable, Qinvalid_sequence
here.
* file-coding.h (enum coding_system_variant):
Add fixed_width_coding_system here.
(struct coding_system_methods):
Add query_method and query_lstream_method to the coding system
methods.
Provide flags for the query methods.
Declare the default query method; initialise it correctly in
INITIALIZE_CODING_SYSTEM_TYPE.
* file-coding.c (default_query_method):
New function, the default query method for coding systems that do
not set it. Moved from coding.el.
(make_coding_system_1):
Accept new elements in PROPS in #'make-coding-system; aliases, a
list of aliases; safe-chars and safe-charsets (these were
previously accepted but not saved); and category.
(Fmake_coding_system_internal):
New function, what used to be #'make-coding-system--on Mule
builds, we've now moved some of the functionality of this to
Lisp.
(Fcoding_system_canonical_name_p):
Move this earlier in the file, since it's now called from within
make_coding_system_1.
(Fquery_coding_region):
Move the implementation of this here, from coding.el.
(complex_vars_of_file_coding):
Call Fmake_coding_system_internal, not Fmake_coding_system;
specify safe-charsets properties when we're a mule build.
* extents.h (mouse_highlight_priority, Fset_extent_priority,
Fset_extent_face, Fmap_extents):
Make these available to other C files.
lisp/ChangeLog addition:
2009-09-19 Aidan Kehoe <kehoea@parhasard.net>
Move the #'query-coding-region implementation to C.
* coding.el:
Consolidate code that depends on the presence or absence of Mule
at the end of this file.
(default-query-coding-region, query-coding-region):
Move these functions to C.
(default-query-coding-region-safe-charset-skip-chars-map):
Remove this variable, the corresponding C variable is
Vdefault_query_coding_region_chartab_cache in file-coding.c.
(query-coding-string): Update docstring to reflect actual multiple
values, be more careful about not modifying a range table that
we're currently mapping over.
(encode-coding-char): Make the implementation of this simpler.
(featurep 'mule): Autoload #'make-coding-system from
mule/make-coding-system.el if we're a mule build; provide an
appropriate compiler macro.
Do various non-mule compatibility things if we're not a mule
build.
* update-elc.el (additional-dump-dependencies):
Add mule/make-coding-system as a dump time dependency if we're a
mule build.
* unicode.el (ccl-encode-to-ucs-2):
(decode-char):
(encode-char):
Move these earlier in the file, for the sake of some byte compile
warnings.
(unicode-query-coding-region):
Move this to unicode.c
* mule/make-coding-system.el:
New file, not dumped. Contains the functionality to rework the
arguments necessary for fixed-width coding systems, and contains
the implementation of #'make-coding-system, which now calls
#'make-coding-system-internal.
* mule/vietnamese.el (viscii):
* mule/latin.el (iso-8859-2):
(windows-1250):
(iso-8859-3):
(iso-8859-4):
(iso-8859-14):
(iso-8859-15):
(iso-8859-16):
(iso-8859-9):
(macintosh):
(windows-1252):
* mule/hebrew.el (iso-8859-8):
* mule/greek.el (iso-8859-7):
(windows-1253):
* mule/cyrillic.el (iso-8859-5):
(koi8-r):
(koi8-u):
(windows-1251):
(alternativnyj):
(koi8-ru):
(koi8-t):
(koi8-c):
(koi8-o):
* mule/arabic.el (iso-8859-6):
(windows-1256):
Move all these coding systems to being of type fixed-width, not of
type CCL. This allows the distinct query-coding-region for them to
be in C, something which will eventually allow us to implement
query-coding-region for the mswindows-multibyte coding systems.
* mule/general-late.el (posix-charset-to-coding-system-hash):
Document why we're pre-emptively persuading the byte compiler that
the ELC for this file needs to be written using escape-quoted.
Call #'set-unicode-query-skip-chars-args, now the Unicode
query-coding-region implementation is in C.
* mule/thai-xtis.el (tis-620):
Don't bother checking whether we're XEmacs or not here.
* mule/mule-coding.el:
Move the eight bit fixed-width functionality from this file to
make-coding-system.el.
tests/ChangeLog addition:
2009-09-19 Aidan Kehoe <kehoea@parhasard.net>
* automated/mule-tests.el:
Check a coding system's type, not an 8-bit-fixed property, for
whether that coding system should be treated as a fixed-width
coding system.
* automated/query-coding-tests.el:
Don't test the query coding functionality for mswindows-multibyte
coding systems, it's not yet implemented.
| author | Aidan Kehoe <kehoea@parhasard.net> |
|---|---|
| date | Sat, 19 Sep 2009 22:53:13 +0100 |
| parents | 80e0588fb42f |
| children | e0db3c197671 |
comparison
equal
deleted
inserted
replaced
| 4689:0636c6ccb430 | 4690:257b468bf2ca |
|---|---|
| 228 mswindows_multibyte_coding_system, | 228 mswindows_multibyte_coding_system, |
| 229 iso2022_coding_system, | 229 iso2022_coding_system, |
| 230 ccl_coding_system, | 230 ccl_coding_system, |
| 231 shift_jis_coding_system, | 231 shift_jis_coding_system, |
| 232 big5_coding_system, | 232 big5_coding_system, |
| 233 unicode_coding_system | 233 unicode_coding_system, |
| 234 fixed_width_coding_system | |
| 234 }; | 235 }; |
| 235 | 236 |
| 236 struct coding_system_methods | 237 struct coding_system_methods |
| 237 { | 238 { |
| 238 Lisp_Object type; | 239 Lisp_Object type; |
| 316 Required. */ | 317 Required. */ |
| 317 Bytecount (*convert_method) (struct coding_stream *str, | 318 Bytecount (*convert_method) (struct coding_stream *str, |
| 318 const unsigned char *src, | 319 const unsigned char *src, |
| 319 unsigned_char_dynarr *dst, Bytecount n); | 320 unsigned_char_dynarr *dst, Bytecount n); |
| 320 | 321 |
| 322 /* Query method: Check whether the buffer text between point and END | |
| 323 can be encoded by this coding system. Returns | |
| 324 either nil (meaning the text can be encoded by the coding system) or a | |
| 325 range table object describing the stretches that the coding system | |
| 326 cannot encode. | |
| 327 | |
| 328 Possible values for flags are below, search for | |
| 329 QUERY_METHOD_IGNORE_INVALID_SEQUENCES. | |
| 330 | |
| 331 Coding systems are expected to be able to behave sensibly with all | |
| 332 possible octets on decoding, which is why this method is only available | |
| 333 for encoding. */ | |
| 334 Lisp_Object (*query_method) (Lisp_Object coding_system, struct buffer *buf, | |
| 335 Charbpos end, int flags); | |
| 336 | |
| 337 /* Same as the previous method, but this works in the context of | |
| 338 lstreams. (Where the data do need to be copied, unfortunately.) The | |
| 339 intention is to implement the query method for the mswindows-multibyte | |
| 340 coding systems in terms of a query_lstream method. */ | |
| 341 Lisp_Object (*query_lstream_method) (struct coding_stream *str, | |
| 342 const Ibyte *start, Bytecount n); | |
| 343 | |
| 321 /* Coding mark method: Mark any Lisp objects in the type-specific data | 344 /* Coding mark method: Mark any Lisp objects in the type-specific data |
| 322 attached to `struct coding_stream'. Optional. */ | 345 attached to `struct coding_stream'. Optional. */ |
| 323 void (*mark_coding_stream_method) (struct coding_stream *str); | 346 void (*mark_coding_stream_method) (struct coding_stream *str); |
| 324 | 347 |
| 325 /* Init coding stream method: Initialize the type-specific data attached | 348 /* Init coding stream method: Initialize the type-specific data attached |
| 385 lstream using this coding system), so we can't store this data in | 408 lstream using this coding system), so we can't store this data in |
| 386 the coding system object. */ | 409 the coding system object. */ |
| 387 int coding_data_size; | 410 int coding_data_size; |
| 388 }; | 411 }; |
| 389 | 412 |
| 413 /* Values for flags, as passed to query_method. */ | |
| 414 | |
| 415 #define QUERY_METHOD_IGNORE_INVALID_SEQUENCES 0x0001 | |
| 416 #define QUERY_METHOD_ERRORP 0x0002 | |
| 417 #define QUERY_METHOD_HIGHLIGHT 0x0004 | |
| 418 | |
| 419 enum query_coding_failure_reasons | |
| 420 { | |
| 421 query_coding_succeeded = 0, | |
| 422 query_coding_unencodable = 1, | |
| 423 query_coding_invalid_sequence = 2 | |
| 424 }; | |
| 425 | |
| 426 extern Lisp_Object Qquery_coding_warning_face; | |
| 427 | |
| 428 Lisp_Object default_query_method (Lisp_Object, struct buffer *, Charbpos, | |
| 429 int); | |
| 430 | |
| 390 /***** Calling a coding-system method *****/ | 431 /***** Calling a coding-system method *****/ |
| 391 | 432 |
| 392 #define RAW_CODESYSMETH(cs, m) ((cs)->methods->m##_method) | 433 #define RAW_CODESYSMETH(cs, m) ((cs)->methods->m##_method) |
| 393 #define HAS_CODESYSMETH_P(cs, m) (!!RAW_CODESYSMETH (cs, m)) | 434 #define HAS_CODESYSMETH_P(cs, m) (!!RAW_CODESYSMETH (cs, m)) |
| 394 #define CODESYSMETH(cs, m, args) (((cs)->methods->m##_method) args) | 435 #define CODESYSMETH(cs, m, args) (((cs)->methods->m##_method) args) |
| 410 CODESYSMETH (XCODING_SYSTEM (cs), m, args) | 451 CODESYSMETH (XCODING_SYSTEM (cs), m, args) |
| 411 #define MAYBE_XCODESYSMETH(cs, m, args) \ | 452 #define MAYBE_XCODESYSMETH(cs, m, args) \ |
| 412 MAYBE_CODESYSMETH (XCODING_SYSTEM (cs), m, args) | 453 MAYBE_CODESYSMETH (XCODING_SYSTEM (cs), m, args) |
| 413 #define XCODESYSMETH_OR_GIVEN(cs, m, args, given) \ | 454 #define XCODESYSMETH_OR_GIVEN(cs, m, args, given) \ |
| 414 CODESYSMETH_OR_GIVEN (XCODING_SYSTEM (cs), m, args, given) | 455 CODESYSMETH_OR_GIVEN (XCODING_SYSTEM (cs), m, args, given) |
| 415 | |
| 416 | 456 |
| 417 /***** Defining new coding-system types *****/ | 457 /***** Defining new coding-system types *****/ |
| 418 | 458 |
| 419 extern const struct sized_memory_description coding_system_empty_extra_description; | 459 extern const struct sized_memory_description coding_system_empty_extra_description; |
| 420 | 460 |
| 475 xnew_and_zero (struct coding_system_methods); \ | 515 xnew_and_zero (struct coding_system_methods); \ |
| 476 ty##_coding_system_methods->type = Q##ty; \ | 516 ty##_coding_system_methods->type = Q##ty; \ |
| 477 ty##_coding_system_methods->extra_description = \ | 517 ty##_coding_system_methods->extra_description = \ |
| 478 &coding_system_empty_extra_description; \ | 518 &coding_system_empty_extra_description; \ |
| 479 ty##_coding_system_methods->enumtype = ty##_coding_system; \ | 519 ty##_coding_system_methods->enumtype = ty##_coding_system; \ |
| 520 ty##_coding_system_methods->query_method = default_query_method; \ | |
| 480 defsymbol_nodump (&ty##_coding_system_methods->predicate_symbol, \ | 521 defsymbol_nodump (&ty##_coding_system_methods->predicate_symbol, \ |
| 481 pred_sym); \ | 522 pred_sym); \ |
| 482 add_entry_to_coding_system_type_list (ty##_coding_system_methods); \ | 523 add_entry_to_coding_system_type_list (ty##_coding_system_methods); \ |
| 483 dump_add_root_block_ptr (&ty##_coding_system_methods, \ | 524 dump_add_root_block_ptr (&ty##_coding_system_methods, \ |
| 484 &coding_system_methods_description); \ | 525 &coding_system_methods_description); \ |
| 1027 #endif | 1068 #endif |
| 1028 | 1069 |
| 1029 #ifdef MULE | 1070 #ifdef MULE |
| 1030 DECLARE_CODING_SYSTEM_TYPE (iso2022); | 1071 DECLARE_CODING_SYSTEM_TYPE (iso2022); |
| 1031 DECLARE_CODING_SYSTEM_TYPE (ccl); | 1072 DECLARE_CODING_SYSTEM_TYPE (ccl); |
| 1073 DECLARE_CODING_SYSTEM_TYPE (fixed_width); | |
| 1032 DECLARE_CODING_SYSTEM_TYPE (shift_jis); | 1074 DECLARE_CODING_SYSTEM_TYPE (shift_jis); |
| 1033 DECLARE_CODING_SYSTEM_TYPE (big5); | 1075 DECLARE_CODING_SYSTEM_TYPE (big5); |
| 1034 #endif | 1076 #endif |
| 1035 | 1077 |
| 1036 #ifdef HAVE_ZLIB | 1078 #ifdef HAVE_ZLIB |