Mercurial > hg > xemacs-beta
annotate src/file-coding.h @ 5776:65d65b52d608
Pass character count from coding systems to buffer insertion code.
src/ChangeLog addition:
2014-01-16 Aidan Kehoe <kehoea@parhasard.net>
Pass character count information from the no-conversion and
unicode coding systems to the buffer insertion code, making
#'find-file on large buffers a little snappier (if
ERROR_CHECK_TEXT is not defined).
* file-coding.c:
* file-coding.c (coding_character_tell): New.
* file-coding.c (conversion_coding_stream_description): New.
* file-coding.c (no_conversion_convert):
Update characters_seen when decoding.
* file-coding.c (no_conversion_character_tell): New.
* file-coding.c (lstream_type_create_file_coding): Create the
no_conversion type with data.
* file-coding.c (coding_system_type_create):
Make the character_tell method available here.
* file-coding.h:
* file-coding.h (struct coding_system_methods):
Add a new character_tell() method, passing charcount information
from the coding systems to the buffer code, avoiding duplicate
bytecount-to-charcount work especially with large buffers.
* fileio.c (Finsert_file_contents_internal):
Update this to pass charcount information to
buffer_insert_string_1(), if that is available from the lstream code.
* insdel.c:
* insdel.c (buffer_insert_string_1):
Add a new CCLEN argument, giving the character count of the string
to insert. It can be -1 to indicate that te function should work
it out itself using bytecount_to_charcount(), as it used to.
* insdel.c (buffer_insert_raw_string_1):
* insdel.c (buffer_insert_lisp_string_1):
* insdel.c (buffer_insert_ascstring_1):
* insdel.c (buffer_insert_emacs_char_1):
* insdel.c (buffer_insert_from_buffer_1):
* insdel.c (buffer_replace_char):
Update these functions to use the new calling convention.
* insdel.h:
* insdel.h (buffer_insert_string):
Update this header to reflect the new buffer_insert_string_1()
argument.
* lstream.c (Lstream_character_tell): New.
Return the number of characters *read* and seen by the consumer so
far, taking into account the unget buffer, and buffered reading.
* lstream.c (Lstream_unread):
Update unget_character_count here as appropriate.
* lstream.c (Lstream_rewind):
Reset unget_character_count here too.
* lstream.h:
* lstream.h (struct lstream):
Provide the character_tell method, add a new field,
unget_character_count, giving the number of characters ever passed
to Lstream_unread().
Declare Lstream_character_tell().
Make Lstream_ungetc(), which happens to be unused, an inline
function rather than a macro, in the course of updating it to
modify unget_character_count.
* print.c (output_string):
Use the new argument to buffer_insert_string_1().
* tests.c:
* tests.c (Ftest_character_tell):
New test function.
* tests.c (syms_of_tests):
Make it available.
* unicode.c:
* unicode.c (struct unicode_coding_stream):
* unicode.c (unicode_character_tell):
New method.
* unicode.c (unicode_convert):
Update the character counter as appropriate.
* unicode.c (coding_system_type_create_unicode):
Make the character_tell method available.
| author | Aidan Kehoe <kehoea@parhasard.net> |
|---|---|
| date | Thu, 16 Jan 2014 16:27:52 +0000 |
| parents | 308d34e9f07d |
| children |
| rev | line source |
|---|---|
| 771 | 1 /* Header for encoding conversion functions; coding-system object. |
| 2 #### rename me to coding-system.h | |
| 428 | 3 Copyright (C) 1991, 1995 Free Software Foundation, Inc. |
| 4 Copyright (C) 1995 Sun Microsystems, Inc. | |
| 793 | 5 Copyright (C) 2000, 2001, 2002 Ben Wing. |
| 428 | 6 |
| 7 This file is part of XEmacs. | |
| 8 | |
|
5402
308d34e9f07d
Changed bulk of GPLv2 or later files identified by script
Mats Lidell <matsl@xemacs.org>
parents:
5127
diff
changeset
|
9 XEmacs is free software: you can redistribute it and/or modify it |
| 428 | 10 under the terms of the GNU General Public License as published by the |
|
5402
308d34e9f07d
Changed bulk of GPLv2 or later files identified by script
Mats Lidell <matsl@xemacs.org>
parents:
5127
diff
changeset
|
11 Free Software Foundation, either version 3 of the License, or (at your |
|
308d34e9f07d
Changed bulk of GPLv2 or later files identified by script
Mats Lidell <matsl@xemacs.org>
parents:
5127
diff
changeset
|
12 option) any later version. |
| 428 | 13 |
| 14 XEmacs is distributed in the hope that it will be useful, but WITHOUT | |
| 15 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or | |
| 16 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License | |
| 17 for more details. | |
| 18 | |
| 19 You should have received a copy of the GNU General Public License | |
|
5402
308d34e9f07d
Changed bulk of GPLv2 or later files identified by script
Mats Lidell <matsl@xemacs.org>
parents:
5127
diff
changeset
|
20 along with XEmacs. If not, see <http://www.gnu.org/licenses/>. */ |
| 428 | 21 |
| 22 /* Synched up with: Mule 2.3. Not in FSF. */ | |
| 23 | |
| 771 | 24 /* Authorship: |
| 25 | |
| 26 Current primary author: Ben Wing <ben@xemacs.org> | |
| 27 | |
| 28 Written by Ben Wing <ben@xemacs.org> for XEmacs, 1995, loosely based | |
| 29 on code written 91.10.09 by K.Handa <handa@etl.go.jp>. | |
| 30 Rewritten again 2000-2001 by Ben Wing to support properly | |
| 31 abstracted coding systems. | |
| 32 September 2001: Finished last part of abstraction, the detection | |
| 33 mechanism. | |
| 34 */ | |
| 428 | 35 |
| 440 | 36 #ifndef INCLUDED_file_coding_h_ |
| 37 #define INCLUDED_file_coding_h_ | |
| 428 | 38 |
| 771 | 39 /* Capsule description of the different structures, what their purpose is, |
| 40 how they fit together, and where various bits of data are stored. | |
| 41 | |
| 2297 | 42 A "coding system" is an algorithm for converting stream data in one format |
| 43 into stream data in another format. Currently most of the coding systems | |
| 44 we have created concern internationalized text, and convert between the | |
| 45 XEmacs internal format for multilingual text, and various external | |
| 771 | 46 representations of such text. However, any such conversion is possible, |
| 47 for example, compressing or uncompressing text using the gzip algorithm. | |
| 48 All coding systems provide both encode and decode routines, so that the | |
| 2297 | 49 conversion can go both ways. Unfortunately encoding and decoding may not |
| 50 be exact inverses, even for a specific instance of a coding system. Care | |
| 51 must be taken when this is not the case. | |
| 771 | 52 |
| 53 The way we handle this is by dividing the various potential coding | |
| 54 systems into types, analogous to classes in C++. Each coding system | |
| 55 type encompasses a series of related coding systems that it can | |
| 56 implement, and it has properties which control how exactly the encoding | |
| 57 works. A particular set of values for each of the properties makes up a | |
| 58 "coding system", and specifies one particular encoding. A `struct | |
| 59 Lisp_Coding_System' object encapsulates those settings -- its type, the | |
| 60 values chosen for all properties of that type, a name for the coding | |
| 61 system, some documentation. | |
| 62 | |
| 63 In addition, there are of course methods associated with a coding system | |
| 64 type, implementing the encoding, decoding, etc. These are stored in a | |
| 65 `struct coding_system_methods' object, one per coding-system type, which | |
| 66 contains mostly function pointers. This is retrievable from the | |
| 67 coding-system object (i.e. the struct Lisp_Coding_System), which has a | |
| 68 pointer to it. | |
| 69 | |
| 70 In order to actually use a coding system to do an encoding or decoding | |
| 71 operation, you need to use a coding Lstream. | |
| 72 | |
| 73 Now let's look more at attached data. All coding systems have certain | |
| 74 common data fields -- name, type, documentation, etc. -- as well as a | |
| 75 bunch more that are defined by the coding system type. To handle this | |
| 76 cleanly, each coding system type defines a structure that holds just the | |
| 77 fields of data particular to it, and calls it e.g. `struct | |
| 78 iso2022_coding_system' for coding system type `iso2022'. When the | |
| 79 memory block holding the coding system object is created, it is sized | |
| 80 such that it can hold both the struct Lisp_Coding_System and the struct | |
| 81 iso2022_coding_system (or whatever) directly following it. (This is a | |
| 82 common trick; another possibility is to have a void * pointer in the | |
| 83 struct Lisp_Coding_System, which points to another memory block holding | |
| 84 the struct iso2022_coding_system.) A macro is provided | |
| 85 (CODING_SYSTEM_TYPE_DATA) to retrieve a pointer of the right type to the | |
| 86 type-specific data contained within the overall `struct | |
| 87 Lisp_Coding_System' block. | |
| 88 | |
| 89 Lstreams, similarly, are objects of type `struct lstream' holding data | |
| 90 about the stream operation (how much data has been read or written, any | |
| 91 buffered data, any error conditions, etc.), and like coding systems have | |
| 92 different types. They have a structure called `Lstream_implementation', | |
| 93 one per lstream type, exactly analogous to `struct | |
| 94 coding_system_methods'. In addition, they have type-specific data | |
| 95 (specifying, e.g., the file number, FILE *, memory location, other | |
| 96 lstream, etc. to read the data from or write it to, and for conversion | |
| 97 processes, the current state of the process -- are we decoding ASCII or | |
| 98 Kanji characters? are we in the middle of a processing an escape | |
| 99 sequence? etc.). This type-specific data is stored in a structure | |
| 100 named `struct coding_stream'. Just like for coding systems, the | |
| 101 type-independent data in the `struct lstream' and the type-dependent | |
| 102 data in the `struct coding_stream' are stored together in the same | |
| 103 memory block. | |
| 428 | 104 |
| 771 | 105 Now things get a bit tricky. The `struct coding_stream' is |
| 106 type-specific from the point of view of an lstream, but not from the | |
| 107 point of view of a coding system. It contains only general data about | |
| 108 the conversion process, e.g. the name of the coding system used for | |
| 109 conversion, the lstream that we take data from or write it to (depending | |
| 110 on whether this was created as a read stream or a write stream), a | |
| 111 buffer to hold extra data we retrieved but can't send on yet, some | |
| 112 flags, etc. It also needs some data specific to the particular coding | |
| 113 system and thus to the particular operation going on. This data is held | |
| 114 in a structure named (e.g.) `struct iso2022_coding_stream', and it's | |
| 115 held in a separate memory block and pointed to by the generic `struct | |
| 116 coding_stream'. It's not glommed into a single memory block both | |
| 117 because that would require making changes to the generic lstream code | |
| 118 and more importantly because the coding system used in a particular | |
| 119 coding lstream can be changed at any point during the lifetime of the | |
| 120 lstream, and possibly multiple times. (For example, it can be set using | |
| 121 the Lisp primitives `set-process-input-coding-system' and | |
| 122 `set-console-tty-input-coding-system', as well as getting set when a | |
| 123 conversion operation was started with coding system `undecided' and the | |
| 2297 | 124 correct coding system was then detected.) #### This suggests implementing |
| 125 compound text extended segments by saving the state of the ctext stream, | |
| 126 and installing an appropriate for the duration of the segment. | |
| 428 | 127 |
| 771 | 128 IMPORTANT NOTE: There are at least two ancillary data structures |
| 129 associated with a coding system type. (There may also be detection data; | |
| 130 see elsewhere.) It's important, when writing a coding system type, to | |
| 131 keep straight which type of data goes where. In particular, `struct | |
| 132 foo_coding_system' is attached to the coding system object itself. This | |
| 133 is a permanent object and there's only one per coding system. It's | |
| 134 created once, usually at init time, and never destroyed. So, `struct | |
| 135 foo_coding_system' should in general not contain dynamic data! (Just | |
| 136 data describing the properties of the coding system.) In particular, | |
| 137 *NO* data about any conversion in progress. There may be many | |
| 138 conversions going on simultaneously using a particular coding system, | |
| 139 and by storing conversion data in the coding system, these conversions | |
| 140 will overwrite each other's data. | |
| 141 | |
| 142 Instead, use the lstream object, whose purpose is to encapsulate a | |
| 143 particular conversion and all associated data. From the lstream object, | |
| 144 you can get the struct coding_stream using something like | |
| 145 | |
| 146 struct coding_stream *str = LSTREAM_TYPE_DATA (lstr, coding); | |
| 147 | |
| 148 But usually this structure is already passed to you as one of the | |
| 149 parameters of the method being invoked. | |
| 150 | |
| 151 From the struct coding_stream, you can retrieve the | |
| 152 coding-system-type-specific data using something like | |
| 153 | |
| 154 struct foo_coding_stream *data = CODING_STREAM_TYPE_DATA (str, foo); | |
| 155 | |
| 156 Then, use this structure to hold all data relevant to the particular | |
| 157 conversion being done. | |
| 158 | |
| 159 Initialize this structure whenever init_coding_stream_method is called | |
| 160 (this may happen more than once), and finalize it (free resources, etc.) | |
| 161 when finalize_coding_stream_method is called. | |
| 162 */ | |
| 163 | |
| 164 struct coding_stream; | |
| 165 struct detection_state; | |
| 166 | |
| 1204 | 167 extern const struct sized_memory_description coding_system_methods_description; |
| 771 | 168 |
| 169 struct coding_system_methods; | |
| 170 | |
| 171 enum source_sink_type | |
| 428 | 172 { |
| 771 | 173 DECODES_CHARACTER_TO_BYTE, |
| 174 DECODES_BYTE_TO_BYTE, | |
| 175 DECODES_BYTE_TO_CHARACTER, | |
| 176 DECODES_CHARACTER_TO_CHARACTER | |
| 428 | 177 }; |
| 178 | |
| 179 enum eol_type | |
| 180 { | |
| 181 EOL_LF, | |
| 182 EOL_CRLF, | |
| 771 | 183 EOL_CR, |
| 1429 | 184 EOL_AUTODETECT |
| 428 | 185 }; |
| 186 | |
| 187 struct Lisp_Coding_System | |
| 188 { | |
|
5127
a9c41067dd88
more cleanups, terminology clarification, lots of doc work
Ben Wing <ben@xemacs.org>
parents:
5124
diff
changeset
|
189 NORMAL_LISP_OBJECT_HEADER header; |
| 771 | 190 struct coding_system_methods *methods; |
| 428 | 191 |
| 1204 | 192 #define CODING_SYSTEM_SLOT_DECLARATION |
| 193 #define MARKED_SLOT(x) Lisp_Object x; | |
| 194 #include "coding-system-slots.h" | |
| 771 | 195 |
| 1204 | 196 /* Eol type requested by user. See comment about EOL junk in |
| 197 coding-system-slots.h. */ | |
| 771 | 198 enum eol_type eol_type; |
| 428 | 199 |
| 2132 | 200 /* If true, this is an internal coding system, which will not show up in |
| 201 coding-system-list unless a special parameter is given to it. */ | |
| 202 int internal_p; | |
| 203 | |
| 771 | 204 /* type-specific extra data attached to a coding_system */ |
| 205 char data[1]; | |
| 428 | 206 }; |
| 207 typedef struct Lisp_Coding_System Lisp_Coding_System; | |
| 208 | |
|
5118
e0db3c197671
merge up to latest default branch, doesn't compile yet
Ben Wing <ben@xemacs.org>
parents:
4690
diff
changeset
|
209 DECLARE_LISP_OBJECT (coding_system, Lisp_Coding_System); |
| 440 | 210 #define XCODING_SYSTEM(x) XRECORD (x, coding_system, Lisp_Coding_System) |
| 617 | 211 #define wrap_coding_system(p) wrap_record (p, coding_system) |
| 428 | 212 #define CODING_SYSTEMP(x) RECORDP (x, coding_system) |
| 213 #define CHECK_CODING_SYSTEM(x) CHECK_RECORD (x, coding_system) | |
| 214 #define CONCHECK_CODING_SYSTEM(x) CONCHECK_RECORD (x, coding_system) | |
| 215 | |
| 1204 | 216 enum coding_system_variant |
| 217 { | |
| 218 no_conversion_coding_system, | |
| 219 convert_eol_coding_system, | |
| 220 undecided_coding_system, | |
| 221 chain_coding_system, | |
| 222 text_file_wrapper_coding_system, | |
| 223 internal_coding_system, | |
| 224 gzip_coding_system, | |
| 225 mswindows_multibyte_to_unicode_coding_system, | |
| 226 mswindows_multibyte_coding_system, | |
| 227 iso2022_coding_system, | |
| 228 ccl_coding_system, | |
| 229 shift_jis_coding_system, | |
| 230 big5_coding_system, | |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
231 unicode_coding_system, |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
232 fixed_width_coding_system |
| 1204 | 233 }; |
| 234 | |
| 771 | 235 struct coding_system_methods |
| 236 { | |
| 237 Lisp_Object type; | |
| 238 Lisp_Object predicate_symbol; | |
| 239 | |
| 1204 | 240 /* Type expressed as an enum, needed for KKCC marking of the |
| 241 type-specific lstream data; copied into the struct coding_stream. */ | |
| 242 | |
| 243 enum coding_system_variant enumtype; | |
| 244 | |
| 771 | 245 /* Implementation specific methods: */ |
| 246 | |
| 247 /* Init method: Initialize coding-system data. Optional. */ | |
| 248 void (*init_method) (Lisp_Object coding_system); | |
| 249 | |
| 250 /* Mark method: Mark any Lisp objects in the type-specific data | |
| 251 attached to the coding-system object. Optional. */ | |
| 252 void (*mark_method) (Lisp_Object coding_system); | |
| 253 | |
| 254 /* Print method: Print the type-specific properties of this coding | |
| 255 system, as part of `print'-ing the object. If this method is defined | |
| 256 and prints anything, it should print a space as the first thing it | |
| 257 does. Optional. */ | |
| 258 void (*print_method) (Lisp_Object cs, Lisp_Object printcharfun, | |
| 259 int escapeflag); | |
| 260 | |
| 261 /* Canonicalize method: Convert this coding system to another one; called | |
| 262 once, at creation time, after all properties have been parsed. The | |
| 263 returned value should be a coding system created with | |
| 264 make_internal_coding_system() (passing the existing coding system as the | |
| 265 first argument), and will become the coding system returned by | |
| 266 `make-coding-system'. Optional. | |
| 267 | |
| 268 NOTE: There are *three* different uses of "canonical" or "canonicalize" | |
| 269 w.r.t. coding systems, and it's important to keep them straight. | |
| 270 | |
| 271 1. The canonicalize method. Used to specify a different coding | |
| 272 system, used when doing conversions, in place of the actual coding | |
| 273 system itself. Stored in the CANONICAL field of a coding system. | |
| 274 | |
| 275 2. The canonicalize-after-coding method. Used to return the encoding | |
| 276 that was "actually" used to decode some text, such that this | |
| 277 particular encoding can be used to encode the text again with the | |
| 278 expectation that the result will be the same as the original encoding. | |
| 279 Particularly important with auto-detecting coding systems. | |
| 280 | |
| 281 3. From the perspective of aliases, a "canonical" coding system is one | |
| 282 that's not an alias to some other coding system, and "canonicalization" | |
| 283 is the process of traversing the alias pointers to find the canonical | |
| 284 coding system that's equivalent to the alias. | |
| 285 */ | |
| 286 Lisp_Object (*canonicalize_method) (Lisp_Object coding_system); | |
| 287 | |
| 288 /* Canonicalize after coding method: Convert this coding system to | |
| 289 another one, after coding (usually decoding) has finished. This is | |
| 290 meant to be used by auto-detecting coding systems, which should return | |
| 291 the actually detected coding system. Optional. */ | |
| 292 Lisp_Object (*canonicalize_after_coding_method) | |
| 293 (struct coding_stream *str); | |
| 294 | |
| 295 /* Convert method: Decode or encode the data in SRC of size N, writing | |
| 296 the results into the Dynarr DST. If the conversion_end_type method | |
| 297 indicates that the source is characters (as opposed to bytes), you are | |
| 298 guaranteed to get only whole characters in the data in SRC/N. STR, a | |
| 299 struct coding_stream, stores all necessary state and other info about | |
| 300 the conversion. Coding-specific state (struct TYPE_coding_stream) can | |
| 301 be retrieved from STR using CODING_STREAM_TYPE_DATA(). Return value | |
| 302 indicates the number of bytes of the *INPUT* that were converted (not | |
| 303 the number of bytes written to the Dynarr!). This can be less than | |
| 304 the total amount of input passed in; if so, the remainder is | |
| 305 considered "rejected" and will appear again at the beginning of the | |
| 306 data passed in the next time the convert method is called. When EOF | |
| 307 is returned on the other end and there's no more data, the convert | |
| 308 method will be called one last time, STR->eof set and the passed-in | |
| 309 data will consist only of any rejected data from the previous | |
| 310 call. (At this point, file handles and similar resources can be | |
| 311 closed, but do NOT arbitrarily free data structures in the | |
| 312 type-specific data, because there are operations that can be done on | |
| 313 closed streams to query the results of the processing -- specifically, | |
| 314 for coding streams, there's the canonicalize_after_coding() method.) | |
| 315 Required. */ | |
| 316 Bytecount (*convert_method) (struct coding_stream *str, | |
| 317 const unsigned char *src, | |
| 318 unsigned_char_dynarr *dst, Bytecount n); | |
| 319 | |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
320 /* Query method: Check whether the buffer text between point and END |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
321 can be encoded by this coding system. Returns |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
322 either nil (meaning the text can be encoded by the coding system) or a |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
323 range table object describing the stretches that the coding system |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
324 cannot encode. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
325 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
326 Possible values for flags are below, search for |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
327 QUERY_METHOD_IGNORE_INVALID_SEQUENCES. |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
328 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
329 Coding systems are expected to be able to behave sensibly with all |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
330 possible octets on decoding, which is why this method is only available |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
331 for encoding. */ |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
332 Lisp_Object (*query_method) (Lisp_Object coding_system, struct buffer *buf, |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
333 Charbpos end, int flags); |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
334 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
335 /* Same as the previous method, but this works in the context of |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
336 lstreams. (Where the data do need to be copied, unfortunately.) The |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
337 intention is to implement the query method for the mswindows-multibyte |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
338 coding systems in terms of a query_lstream method. */ |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
339 Lisp_Object (*query_lstream_method) (struct coding_stream *str, |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
340 const Ibyte *start, Bytecount n); |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
341 |
| 771 | 342 /* Coding mark method: Mark any Lisp objects in the type-specific data |
| 343 attached to `struct coding_stream'. Optional. */ | |
| 344 void (*mark_coding_stream_method) (struct coding_stream *str); | |
| 345 | |
| 346 /* Init coding stream method: Initialize the type-specific data attached | |
| 347 to the coding stream (i.e. in struct TYPE_coding_stream), when the | |
| 348 coding stream is opened. The type-specific data will be zeroed out. | |
| 349 Optional. */ | |
| 350 void (*init_coding_stream_method) (struct coding_stream *str); | |
| 351 | |
| 352 /* Rewind coding stream method: Reset any necessary type-specific data as | |
| 353 a result of the stream being rewound. Optional. */ | |
| 354 void (*rewind_coding_stream_method) (struct coding_stream *str); | |
| 355 | |
|
5776
65d65b52d608
Pass character count from coding systems to buffer insertion code.
Aidan Kehoe <kehoea@parhasard.net>
parents:
5402
diff
changeset
|
356 /* Return the number of characters *decoded*. Optional. */ |
|
65d65b52d608
Pass character count from coding systems to buffer insertion code.
Aidan Kehoe <kehoea@parhasard.net>
parents:
5402
diff
changeset
|
357 Charcount (*character_tell_method) (struct coding_stream *str); |
|
65d65b52d608
Pass character count from coding systems to buffer insertion code.
Aidan Kehoe <kehoea@parhasard.net>
parents:
5402
diff
changeset
|
358 |
| 771 | 359 /* Finalize coding stream method: Clean up the type-specific data |
| 360 attached to the coding stream (i.e. in struct TYPE_coding_stream). | |
| 361 Happens when the Lstream is deleted using Lstream_delete() or is | |
| 362 garbage-collected. Most streams are deleted after they've been used, | |
| 363 so it's less likely (but still possible) that allocated data will | |
| 364 stick around until GC time. (File handles can also be closed when EOF | |
| 365 is signalled; but some data must stick around after this point, for | |
| 366 the benefit of canonicalize_after_coding. See the convert method.) | |
|
5124
623d57b7fbe8
separate regular and disksave finalization, print method fixes.
Ben Wing <ben@xemacs.org>
parents:
5120
diff
changeset
|
367 Called only once. Optional. */ |
| 771 | 368 void (*finalize_coding_stream_method) (struct coding_stream *str); |
| 369 | |
| 370 /* Finalize method: Clean up type-specific data (e.g. free allocated | |
| 371 data) attached to the coding system (i.e. in struct | |
| 372 TYPE_coding_system), when the coding system is about to be garbage | |
|
5124
623d57b7fbe8
separate regular and disksave finalization, print method fixes.
Ben Wing <ben@xemacs.org>
parents:
5120
diff
changeset
|
373 collected. (Currently not called.) Called only once. Optional. */ |
| 771 | 374 void (*finalize_method) (Lisp_Object codesys); |
| 375 | |
| 376 /* Conversion end type method: Does this coding system encode bytes -> | |
| 377 characters, characters -> characters, bytes -> bytes, or | |
| 378 characters -> bytes?. Default is characters -> bytes. Optional. */ | |
| 379 enum source_sink_type (*conversion_end_type_method) (Lisp_Object codesys); | |
| 380 | |
| 381 /* Putprop method: Set the value of a type-specific property. If | |
| 382 the property name is unrecognized, return 0. If the value is disallowed | |
| 383 or erroneous, signal an error. Currently called only at creation time. | |
| 384 Optional. */ | |
| 385 int (*putprop_method) (Lisp_Object codesys, | |
| 386 Lisp_Object key, | |
| 387 Lisp_Object value); | |
| 388 | |
| 389 /* Getprop method: Return the value of a type-specific property. If | |
| 390 the property name is unrecognized, return Qunbound. Optional. | |
| 391 */ | |
| 392 Lisp_Object (*getprop_method) (Lisp_Object coding_system, | |
| 393 Lisp_Object prop); | |
| 394 | |
| 395 /* These next three are set as part of the call to | |
| 396 INITIALIZE_CODING_SYSTEM_TYPE_WITH_DATA. */ | |
| 397 | |
| 398 /* Description of the extra data (struct foo_coding_system) attached to a | |
| 1204 | 399 coding system, for pdump purposes. */ |
| 400 const struct sized_memory_description *extra_description; | |
| 771 | 401 /* size of struct foo_coding_system -- extra data associated with |
| 402 the coding system */ | |
| 403 int extra_data_size; | |
| 404 /* size of struct foo_coding_stream -- extra data associated with the | |
| 405 struct coding_stream, needed for each active coding process | |
| 406 using this coding system. note that we can have more than one | |
| 407 process active at once (simply by creating more than one coding | |
| 408 lstream using this coding system), so we can't store this data in | |
| 409 the coding system object. */ | |
| 410 int coding_data_size; | |
| 411 }; | |
| 412 | |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
413 /* Values for flags, as passed to query_method. */ |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
414 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
415 #define QUERY_METHOD_IGNORE_INVALID_SEQUENCES 0x0001 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
416 #define QUERY_METHOD_ERRORP 0x0002 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
417 #define QUERY_METHOD_HIGHLIGHT 0x0004 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
418 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
419 enum query_coding_failure_reasons |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
420 { |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
421 query_coding_succeeded = 0, |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
422 query_coding_unencodable = 1, |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
423 query_coding_invalid_sequence = 2 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
424 }; |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
425 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
426 extern Lisp_Object Qquery_coding_warning_face; |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
427 |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
428 Lisp_Object default_query_method (Lisp_Object, struct buffer *, Charbpos, |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
429 int); |
|
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
430 |
| 771 | 431 /***** Calling a coding-system method *****/ |
| 432 | |
| 433 #define RAW_CODESYSMETH(cs, m) ((cs)->methods->m##_method) | |
| 434 #define HAS_CODESYSMETH_P(cs, m) (!!RAW_CODESYSMETH (cs, m)) | |
| 435 #define CODESYSMETH(cs, m, args) (((cs)->methods->m##_method) args) | |
| 436 | |
| 437 /* Call a void-returning coding-system method, if it exists. */ | |
| 438 #define MAYBE_CODESYSMETH(cs, m, args) do { \ | |
| 439 Lisp_Coding_System *maybe_codesysmeth_cs = (cs); \ | |
| 440 if (HAS_CODESYSMETH_P (maybe_codesysmeth_cs, m)) \ | |
| 441 CODESYSMETH (maybe_codesysmeth_cs, m, args); \ | |
| 442 } while (0) | |
| 443 | |
| 444 /* Call a coding-system method, if it exists, or return GIVEN. | |
| 445 NOTE: Multiply-evaluates CS. */ | |
| 446 #define CODESYSMETH_OR_GIVEN(cs, m, args, given) \ | |
| 447 (HAS_CODESYSMETH_P (cs, m) ? \ | |
| 448 CODESYSMETH (cs, m, args) : (given)) | |
| 449 | |
| 450 #define XCODESYSMETH(cs, m, args) \ | |
| 451 CODESYSMETH (XCODING_SYSTEM (cs), m, args) | |
| 452 #define MAYBE_XCODESYSMETH(cs, m, args) \ | |
| 453 MAYBE_CODESYSMETH (XCODING_SYSTEM (cs), m, args) | |
| 454 #define XCODESYSMETH_OR_GIVEN(cs, m, args, given) \ | |
| 455 CODESYSMETH_OR_GIVEN (XCODING_SYSTEM (cs), m, args, given) | |
| 456 | |
| 457 /***** Defining new coding-system types *****/ | |
| 458 | |
| 1204 | 459 extern const struct sized_memory_description coding_system_empty_extra_description; |
| 771 | 460 |
| 800 | 461 #ifdef ERROR_CHECK_TYPES |
| 771 | 462 #define DECLARE_CODING_SYSTEM_TYPE(type) \ |
| 463 \ | |
| 464 extern struct coding_system_methods * type##_coding_system_methods; \ | |
| 826 | 465 DECLARE_INLINE_HEADER ( \ |
| 466 struct type##_coding_system * \ | |
| 771 | 467 error_check_##type##_coding_system_data (Lisp_Coding_System *cs) \ |
| 826 | 468 ) \ |
| 771 | 469 { \ |
| 470 assert (CODING_SYSTEM_TYPE_P (cs, type)); \ | |
| 471 /* Catch accidental use of INITIALIZE_CODING_SYSTEM_TYPE in place \ | |
| 472 of INITIALIZE_CODING_SYSTEM_TYPE_WITH_DATA. */ \ | |
| 473 assert (cs->methods->extra_data_size > 0); \ | |
| 474 return (struct type##_coding_system *) cs->data; \ | |
| 475 } \ | |
| 476 \ | |
| 826 | 477 DECLARE_INLINE_HEADER ( \ |
| 478 struct type##_coding_stream * \ | |
| 771 | 479 error_check_##type##_coding_stream_data (struct coding_stream *s) \ |
| 826 | 480 ) \ |
| 771 | 481 { \ |
| 482 assert (XCODING_SYSTEM_TYPE_P (s->codesys, type)); \ | |
| 483 return (struct type##_coding_stream *) s->data; \ | |
| 484 } \ | |
| 485 \ | |
| 826 | 486 DECLARE_INLINE_HEADER ( \ |
| 487 Lisp_Coding_System * \ | |
| 771 | 488 error_check_##type##_coding_system_type (Lisp_Object obj) \ |
| 826 | 489 ) \ |
| 771 | 490 { \ |
| 491 Lisp_Coding_System *cs = XCODING_SYSTEM (obj); \ | |
| 492 assert (CODING_SYSTEM_TYPE_P (cs, type)); \ | |
| 493 return cs; \ | |
| 494 } \ | |
| 495 \ | |
| 496 DECLARE_NOTHING | |
| 497 #else | |
| 498 #define DECLARE_CODING_SYSTEM_TYPE(type) \ | |
| 499 extern struct coding_system_methods * type##_coding_system_methods | |
| 800 | 500 #endif /* ERROR_CHECK_TYPES */ |
| 771 | 501 |
| 502 #define DEFINE_CODING_SYSTEM_TYPE(type) \ | |
| 503 struct coding_system_methods * type##_coding_system_methods | |
| 504 | |
| 1204 | 505 #define DEFINE_CODING_SYSTEM_TYPE_WITH_DATA(type) \ |
| 506 struct coding_system_methods * type##_coding_system_methods; \ | |
| 507 static const struct sized_memory_description \ | |
| 508 type##_coding_system_description_0 = { \ | |
| 509 sizeof (struct type##_coding_system), \ | |
| 510 type##_coding_system_description \ | |
| 511 } | |
| 512 | |
| 771 | 513 #define INITIALIZE_CODING_SYSTEM_TYPE(ty, pred_sym) do { \ |
| 514 ty##_coding_system_methods = \ | |
| 515 xnew_and_zero (struct coding_system_methods); \ | |
| 516 ty##_coding_system_methods->type = Q##ty; \ | |
| 517 ty##_coding_system_methods->extra_description = \ | |
| 1204 | 518 &coding_system_empty_extra_description; \ |
| 519 ty##_coding_system_methods->enumtype = ty##_coding_system; \ | |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
520 ty##_coding_system_methods->query_method = default_query_method; \ |
| 771 | 521 defsymbol_nodump (&ty##_coding_system_methods->predicate_symbol, \ |
| 522 pred_sym); \ | |
| 523 add_entry_to_coding_system_type_list (ty##_coding_system_methods); \ | |
| 2367 | 524 dump_add_root_block_ptr (&ty##_coding_system_methods, \ |
| 771 | 525 &coding_system_methods_description); \ |
| 526 } while (0) | |
| 527 | |
| 528 #define REINITIALIZE_CODING_SYSTEM_TYPE(type) do { \ | |
| 529 staticpro_nodump (&type##_coding_system_methods->predicate_symbol); \ | |
| 530 } while (0) | |
| 531 | |
| 532 /* This assumes the existence of two structures: | |
| 533 | |
| 534 struct foo_coding_system (attached to the coding system) | |
| 535 struct foo_coding_stream (per coding process, attached to the | |
| 536 struct coding_stream) | |
| 1204 | 537 const struct memory_description foo_coding_system_description[] |
| 538 (data description of struct foo_coding_system) | |
| 771 | 539 |
| 1204 | 540 For an example of how to do the description, see |
| 771 | 541 chain_coding_system_description. |
| 542 */ | |
| 543 #define INITIALIZE_CODING_SYSTEM_TYPE_WITH_DATA(type, pred_sym) \ | |
| 544 do { \ | |
| 545 INITIALIZE_CODING_SYSTEM_TYPE (type, pred_sym); \ | |
| 546 type##_coding_system_methods->extra_data_size = \ | |
| 547 sizeof (struct type##_coding_system); \ | |
| 548 type##_coding_system_methods->extra_description = \ | |
| 1204 | 549 &type##_coding_system_description_0; \ |
| 771 | 550 type##_coding_system_methods->coding_data_size = \ |
| 551 sizeof (struct type##_coding_stream); \ | |
| 552 } while (0) | |
| 553 | |
| 554 /* Declare that coding-system-type TYPE has method METH; used in | |
| 555 initialization routines */ | |
| 556 #define CODING_SYSTEM_HAS_METHOD(type, meth) \ | |
| 557 (type##_coding_system_methods->meth##_method = type##_##meth) | |
| 558 | |
| 559 /***** Macros for accessing coding-system types *****/ | |
| 560 | |
| 561 #define CODING_SYSTEM_TYPE_P(cs, type) \ | |
| 562 ((cs)->methods == type##_coding_system_methods) | |
| 563 #define XCODING_SYSTEM_TYPE_P(cs, type) \ | |
| 564 CODING_SYSTEM_TYPE_P (XCODING_SYSTEM (cs), type) | |
| 565 | |
| 800 | 566 #ifdef ERROR_CHECK_TYPES |
| 771 | 567 # define CODING_SYSTEM_TYPE_DATA(cs, type) \ |
| 568 error_check_##type##_coding_system_data (cs) | |
| 569 #else | |
| 570 # define CODING_SYSTEM_TYPE_DATA(cs, type) \ | |
| 571 ((struct type##_coding_system *) \ | |
| 572 (cs)->data) | |
| 573 #endif | |
| 574 | |
| 575 #define XCODING_SYSTEM_TYPE_DATA(cs, type) \ | |
| 576 CODING_SYSTEM_TYPE_DATA (XCODING_SYSTEM_OF_TYPE (cs, type), type) | |
| 577 | |
| 800 | 578 #ifdef ERROR_CHECK_TYPES |
| 771 | 579 # define XCODING_SYSTEM_OF_TYPE(x, type) \ |
| 580 error_check_##type##_coding_system_type (x) | |
| 581 # define XSETCODING_SYSTEM_OF_TYPE(x, p, type) do \ | |
| 582 { \ | |
| 793 | 583 x = wrap_coding_system (p); \ |
| 584 assert (CODING_SYSTEM_TYPEP (XCODING_SYSTEM (x), type)); \ | |
| 771 | 585 } while (0) |
| 586 #else | |
| 587 # define XCODING_SYSTEM_OF_TYPE(x, type) XCODING_SYSTEM (x) | |
| 793 | 588 # define XSETCODING_SYSTEM_OF_TYPE(x, p, type) do \ |
| 589 { \ | |
| 590 x = wrap_coding_system (p); \ | |
| 591 } while (0) | |
| 771 | 592 #endif /* ERROR_CHECK_TYPE_CHECK */ |
| 593 | |
| 594 #define CODING_SYSTEM_TYPEP(x, type) \ | |
| 595 (CODING_SYSTEMP (x) && CODING_SYSTEM_TYPE_P (XCODING_SYSTEM (x), type)) | |
| 596 #define CHECK_CODING_SYSTEM_OF_TYPE(x, type) do { \ | |
| 597 CHECK_CODING_SYSTEM (x); \ | |
| 598 if (!CODING_SYSTEM_TYPE_P (XCODING_SYSTEM (x), type)) \ | |
| 599 dead_wrong_type_argument \ | |
| 600 (type##_coding_system_methods->predicate_symbol, x); \ | |
| 601 } while (0) | |
| 602 #define CONCHECK_CODING_SYSTEM_OF_TYPE(x, type) do { \ | |
| 603 CONCHECK_CODING_SYSTEM (x); \ | |
| 604 if (!(CODING_SYSTEM_TYPEP (x, type))) \ | |
| 605 x = wrong_type_argument \ | |
| 606 (type##_coding_system_methods->predicate_symbol, x); \ | |
| 607 } while (0) | |
| 608 | |
| 609 #define CODING_SYSTEM_METHODS(codesys) ((codesys)->methods) | |
| 428 | 610 #define CODING_SYSTEM_NAME(codesys) ((codesys)->name) |
| 771 | 611 #define CODING_SYSTEM_DESCRIPTION(codesys) ((codesys)->description) |
| 612 #define CODING_SYSTEM_TYPE(codesys) ((codesys)->methods->type) | |
| 428 | 613 #define CODING_SYSTEM_MNEMONIC(codesys) ((codesys)->mnemonic) |
| 771 | 614 #define CODING_SYSTEM_DOCUMENTATION(codesys) ((codesys)->documentation) |
| 428 | 615 #define CODING_SYSTEM_POST_READ_CONVERSION(codesys) \ |
| 616 ((codesys)->post_read_conversion) | |
| 617 #define CODING_SYSTEM_PRE_WRITE_CONVERSION(codesys) \ | |
| 618 ((codesys)->pre_write_conversion) | |
| 619 #define CODING_SYSTEM_EOL_TYPE(codesys) ((codesys)->eol_type) | |
| 771 | 620 #define CODING_SYSTEM_EOL_LF(codesys) ((codesys)->eol[EOL_LF]) |
| 621 #define CODING_SYSTEM_EOL_CRLF(codesys) ((codesys)->eol[EOL_CRLF]) | |
| 622 #define CODING_SYSTEM_EOL_CR(codesys) ((codesys)->eol[EOL_CR]) | |
| 623 #define CODING_SYSTEM_TEXT_FILE_WRAPPER(codesys) ((codesys)->text_file_wrapper) | |
| 624 #define CODING_SYSTEM_AUTO_EOL_WRAPPER(codesys) ((codesys)->auto_eol_wrapper) | |
| 625 #define CODING_SYSTEM_SUBSIDIARY_PARENT(codesys) ((codesys)->subsidiary_parent) | |
| 626 #define CODING_SYSTEM_CANONICAL(codesys) ((codesys)->canonical) | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
3017
diff
changeset
|
627 #define CODING_SYSTEM_SAFE_CHARSETS(codesys) ((codesys)->safe_charsets) |
|
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
3017
diff
changeset
|
628 #define CODING_SYSTEM_SAFE_CHARS(codesys) ((codesys)->safe_chars) |
| 428 | 629 |
| 771 | 630 #define CODING_SYSTEM_CHAIN_CHAIN(codesys) \ |
| 631 (CODING_SYSTEM_TYPE_DATA (codesys, chain)->chain) | |
| 632 #define CODING_SYSTEM_CHAIN_COUNT(codesys) \ | |
| 633 (CODING_SYSTEM_TYPE_DATA (codesys, chain)->count) | |
| 634 #define CODING_SYSTEM_CHAIN_CANONICALIZE_AFTER_CODING(codesys) \ | |
| 635 (CODING_SYSTEM_TYPE_DATA (codesys, chain)->canonicalize_after_coding) | |
| 428 | 636 |
| 771 | 637 #define XCODING_SYSTEM_METHODS(codesys) \ |
| 638 CODING_SYSTEM_METHODS (XCODING_SYSTEM (codesys)) | |
| 428 | 639 #define XCODING_SYSTEM_NAME(codesys) \ |
| 640 CODING_SYSTEM_NAME (XCODING_SYSTEM (codesys)) | |
| 771 | 641 #define XCODING_SYSTEM_DESCRIPTION(codesys) \ |
| 642 CODING_SYSTEM_DESCRIPTION (XCODING_SYSTEM (codesys)) | |
| 428 | 643 #define XCODING_SYSTEM_TYPE(codesys) \ |
| 644 CODING_SYSTEM_TYPE (XCODING_SYSTEM (codesys)) | |
| 645 #define XCODING_SYSTEM_MNEMONIC(codesys) \ | |
| 646 CODING_SYSTEM_MNEMONIC (XCODING_SYSTEM (codesys)) | |
| 771 | 647 #define XCODING_SYSTEM_DOCUMENTATION(codesys) \ |
| 648 CODING_SYSTEM_DOCUMENTATION (XCODING_SYSTEM (codesys)) | |
| 428 | 649 #define XCODING_SYSTEM_POST_READ_CONVERSION(codesys) \ |
| 650 CODING_SYSTEM_POST_READ_CONVERSION (XCODING_SYSTEM (codesys)) | |
| 651 #define XCODING_SYSTEM_PRE_WRITE_CONVERSION(codesys) \ | |
| 652 CODING_SYSTEM_PRE_WRITE_CONVERSION (XCODING_SYSTEM (codesys)) | |
| 653 #define XCODING_SYSTEM_EOL_TYPE(codesys) \ | |
| 654 CODING_SYSTEM_EOL_TYPE (XCODING_SYSTEM (codesys)) | |
| 655 #define XCODING_SYSTEM_EOL_LF(codesys) \ | |
| 656 CODING_SYSTEM_EOL_LF (XCODING_SYSTEM (codesys)) | |
| 657 #define XCODING_SYSTEM_EOL_CRLF(codesys) \ | |
| 658 CODING_SYSTEM_EOL_CRLF (XCODING_SYSTEM (codesys)) | |
| 659 #define XCODING_SYSTEM_EOL_CR(codesys) \ | |
| 660 CODING_SYSTEM_EOL_CR (XCODING_SYSTEM (codesys)) | |
| 771 | 661 #define XCODING_SYSTEM_TEXT_FILE_WRAPPER(codesys) \ |
| 662 CODING_SYSTEM_TEXT_FILE_WRAPPER (XCODING_SYSTEM (codesys)) | |
| 663 #define XCODING_SYSTEM_AUTO_EOL_WRAPPER(codesys) \ | |
| 664 CODING_SYSTEM_AUTO_EOL_WRAPPER (XCODING_SYSTEM (codesys)) | |
| 665 #define XCODING_SYSTEM_SUBSIDIARY_PARENT(codesys) \ | |
| 666 CODING_SYSTEM_SUBSIDIARY_PARENT (XCODING_SYSTEM (codesys)) | |
| 667 #define XCODING_SYSTEM_CANONICAL(codesys) \ | |
| 668 CODING_SYSTEM_CANONICAL (XCODING_SYSTEM (codesys)) | |
|
4568
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
3017
diff
changeset
|
669 #define XCODING_SYSTEM_SAFE_CHARSETS(codesys) \ |
|
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
3017
diff
changeset
|
670 CODING_SYSTEM_SAFE_CHARSETS (XCODING_SYSTEM (codesys)) |
|
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
3017
diff
changeset
|
671 #define XCODING_SYSTEM_SAFE_CHARS(codesys) \ |
|
1d74a1d115ee
Add #'query-coding-region tests; do the work necessary to get them running.
Aidan Kehoe <kehoea@parhasard.net>
parents:
3017
diff
changeset
|
672 CODING_SYSTEM_SAFE_CHARS (XCODING_SYSTEM (codesys)) |
| 428 | 673 |
| 771 | 674 #define XCODING_SYSTEM_CHAIN_CHAIN(codesys) \ |
| 675 CODING_SYSTEM_CHAIN_CHAIN (XCODING_SYSTEM (codesys)) | |
| 676 #define XCODING_SYSTEM_CHAIN_COUNT(codesys) \ | |
| 677 CODING_SYSTEM_CHAIN_COUNT (XCODING_SYSTEM (codesys)) | |
| 678 #define XCODING_SYSTEM_CHAIN_CANONICALIZE_AFTER_CODING(codesys) \ | |
| 679 CODING_SYSTEM_CHAIN_CANONICALIZE_AFTER_CODING (XCODING_SYSTEM (codesys)) | |
| 428 | 680 |
| 771 | 681 /**************************************************/ |
| 682 /* Detection */ | |
| 683 /**************************************************/ | |
| 428 | 684 |
| 771 | 685 #define MAX_DETECTOR_CATEGORIES 256 |
| 686 #define MAX_DETECTORS 64 | |
| 428 | 687 |
| 771 | 688 #define MAX_BYTES_PROCESSED_FOR_DETECTION 65536 |
| 428 | 689 |
| 771 | 690 struct detection_state |
| 428 | 691 { |
| 771 | 692 int seen_non_ascii; |
| 693 Bytecount bytes_seen; | |
| 428 | 694 |
| 771 | 695 char categories[MAX_DETECTOR_CATEGORIES]; |
| 696 Bytecount data_offset[MAX_DETECTORS]; | |
| 697 /* ... more data follows; data_offset[detector_##TYPE] points to | |
| 698 the data for that type */ | |
| 428 | 699 }; |
| 700 | |
| 771 | 701 #define DETECTION_STATE_DATA(st, type) \ |
| 702 ((struct type##_detector *) \ | |
| 703 ((char *) (st) + (st)->data_offset[detector_##type])) | |
| 428 | 704 |
| 448 | 705 /* Distinguishable categories of encodings. |
| 706 | |
| 707 This list determines the initial priority of the categories. | |
| 708 | |
| 709 For better or worse, currently Mule files are encoded in 7-bit ISO 2022. | |
| 710 For this reason, under Mule ISO_7 gets highest priority. | |
| 711 | |
| 712 Putting NO_CONVERSION second prevents "binary corruption" in the | |
| 713 default case in all but the (presumably) extremely rare case of a | |
| 714 binary file which contains redundant escape sequences but no 8-bit | |
| 715 characters. | |
| 716 | |
| 717 The remaining priorities are based on perceived "internationalization | |
| 718 political correctness." An exception is UCS-4 at the bottom, since | |
| 719 basically everything is compatible with UCS-4, but it is likely to | |
| 720 be very rare as an external encoding. */ | |
| 721 | |
| 771 | 722 /* Macros to define code of control characters for ISO2022's functions. */ |
| 723 /* Used by the detection routines of other coding system types as well. */ | |
| 724 /* code */ /* function */ | |
| 725 #define ISO_CODE_LF 0x0A /* line-feed */ | |
| 726 #define ISO_CODE_CR 0x0D /* carriage-return */ | |
| 727 #define ISO_CODE_SO 0x0E /* shift-out */ | |
| 728 #define ISO_CODE_SI 0x0F /* shift-in */ | |
| 729 #define ISO_CODE_ESC 0x1B /* escape */ | |
| 730 #define ISO_CODE_DEL 0x7F /* delete */ | |
| 731 #define ISO_CODE_SS2 0x8E /* single-shift-2 */ | |
| 732 #define ISO_CODE_SS3 0x8F /* single-shift-3 */ | |
| 733 #define ISO_CODE_CSI 0x9B /* control-sequence-introduce */ | |
| 734 | |
| 735 enum detection_result | |
| 736 { | |
| 737 /* Basically means a magic cookie was seen indicating this type, or | |
| 738 something similar. */ | |
| 739 DET_NEAR_CERTAINTY = 4, | |
| 740 DET_HIGHEST = 4, | |
| 741 /* Characteristics seen that are unlikely to be other coding system types | |
| 742 -- e.g. ISO-2022 escape sequences, or perhaps a consistent pattern of | |
| 743 alternating zero bytes in UTF-16, along with Unicode LF or CRLF | |
| 744 sequences at regular intervals. (Zero bytes are unlikely or impossible | |
| 745 in most text encodings.) */ | |
| 746 DET_QUITE_PROBABLE = 3, | |
| 747 /* Strong or medium statistical likelihood. At least some | |
| 748 characteristics seen that match what's normally found in this encoding | |
| 749 -- e.g. in Shift-JIS, a number of two-byte Japanese character | |
| 750 sequences in the right range, and nothing out of range; or in Unicode, | |
| 751 much higher statistical variance in the odd bytes than in the even | |
| 752 bytes, or vice-versa (perhaps the presence of regular EOL sequences | |
| 753 would bump this too to DET_QUITE_PROBABLE). This is quite often a | |
| 754 statistical test. */ | |
| 755 DET_SOMEWHAT_LIKELY = 2, | |
| 756 /* Weak statistical likelihood. Pretty much any features at all that | |
| 757 characterize this encoding, and nothing that rules against it. */ | |
| 758 DET_SLIGHTLY_LIKELY = 1, | |
| 759 /* Default state. Perhaps it indicates pure ASCII or something similarly | |
| 760 vague seen in Shift-JIS, or, exactly as the level says, it might mean | |
| 761 in a statistical-based detector that the pros and cons are balanced | |
| 762 out. This is also the lowest level that will be accepted by the | |
| 763 auto-detector without asking the user: If all available detectors | |
| 764 report lower levels for all categories with attached coding systems, | |
| 765 the user will be shown the results and explicitly prompted for action. | |
| 766 The user will also be prompted if this is the highest available level | |
| 767 and more than one detector reports the level. (See below about the | |
| 768 consequent necessity of an "ASCII" detector, which will return level 1 | |
| 769 or higher for most plain text files.) */ | |
| 770 DET_AS_LIKELY_AS_UNLIKELY = 0, | |
| 771 /* Some characteristics seen that are unusual for this encoding -- | |
| 772 e.g. unusual control characters in a plain-text encoding, lots of | |
| 773 8-bit characters, or little statistical variance in the odd and even | |
| 774 bytes in UTF-16. */ | |
| 775 DET_SOMEWHAT_UNLIKELY = -1, | |
| 776 /* This indicates that there is very little chance the data is in the | |
| 777 right format; this is probably the lowest level you can get when | |
| 778 presenting random binary data to a text file, because there are no | |
| 779 "specific sequences" you can see that would totally rule out | |
| 780 recognition. */ | |
| 781 DET_QUITE_IMPROBABLE = -2, | |
| 782 /* An erroneous sequence was seen. */ | |
| 783 DET_NEARLY_IMPOSSIBLE = -3, | |
| 1429 | 784 DET_LOWEST = -3 |
| 771 | 785 }; |
| 786 | |
| 787 extern int coding_detector_count; | |
| 788 extern int coding_detector_category_count; | |
| 789 | |
| 790 struct detector_category | |
| 428 | 791 { |
| 771 | 792 int id; |
| 793 Lisp_Object sym; | |
| 794 }; | |
| 795 | |
| 796 typedef struct | |
| 797 { | |
| 798 Dynarr_declare (struct detector_category); | |
| 799 } detector_category_dynarr; | |
| 800 | |
| 801 struct detector | |
| 802 { | |
| 803 int id; | |
| 804 detector_category_dynarr *cats; | |
| 805 Bytecount data_size; | |
| 806 /* Detect method: Required. */ | |
| 807 void (*detect_method) (struct detection_state *st, | |
| 808 const unsigned char *src, Bytecount n); | |
| 809 /* Finalize detection state method: Clean up any allocated data in the | |
|
5124
623d57b7fbe8
separate regular and disksave finalization, print method fixes.
Ben Wing <ben@xemacs.org>
parents:
5120
diff
changeset
|
810 detection state. Called only once. Optional. */ |
| 771 | 811 void (*finalize_detection_state_method) (struct detection_state *st); |
| 428 | 812 }; |
| 813 | |
| 771 | 814 /* Lvalue for a particular detection result -- detection state ST, |
| 815 category CAT */ | |
| 816 #define DET_RESULT(st, cat) ((st)->categories[detector_category_##cat]) | |
| 817 /* In state ST, set all detection results associated with detector DET to | |
| 818 RESULT. */ | |
| 819 #define SET_DET_RESULTS(st, det, result) \ | |
| 820 set_detection_results (st, detector_##det, result) | |
| 821 | |
| 822 typedef struct | |
| 823 { | |
| 824 Dynarr_declare (struct detector); | |
| 825 } detector_dynarr; | |
| 826 | |
| 827 extern detector_dynarr *all_coding_detectors; | |
| 828 | |
| 829 #define DEFINE_DETECTOR_CATEGORY(detector, cat) \ | |
| 830 int detector_category_##cat | |
| 831 #define DECLARE_DETECTOR_CATEGORY(detector, cat) \ | |
| 832 extern int detector_category_##cat | |
| 833 #define INITIALIZE_DETECTOR_CATEGORY(detector, cat) \ | |
| 834 do { \ | |
| 835 struct detector_category dog; \ | |
| 836 xzero (dog); \ | |
| 837 detector_category_##cat = coding_detector_category_count++; \ | |
| 838 dump_add_opaque_int (&detector_category_##cat); \ | |
| 839 dog.id = detector_category_##cat; \ | |
| 840 dog.sym = Q##cat; \ | |
| 841 Dynarr_add (Dynarr_at (all_coding_detectors, detector_##detector).cats, \ | |
| 842 dog); \ | |
| 843 } while (0) | |
| 844 | |
| 845 #define DEFINE_DETECTOR(Detector) \ | |
| 846 int detector_##Detector | |
| 847 #define DECLARE_DETECTOR(Detector) \ | |
| 848 extern int detector_##Detector | |
| 849 #define INITIALIZE_DETECTOR(Detector) \ | |
| 850 do { \ | |
| 851 struct detector det; \ | |
| 852 xzero (det); \ | |
| 853 detector_##Detector = coding_detector_count++; \ | |
| 854 dump_add_opaque_int (&detector_##Detector); \ | |
| 855 det.id = detector_##Detector; \ | |
| 856 det.cats = Dynarr_new2 (detector_category_dynarr, \ | |
| 857 struct detector_category); \ | |
| 858 det.data_size = sizeof (struct Detector##_detector); \ | |
| 859 Dynarr_add (all_coding_detectors, det); \ | |
| 860 } while (0) | |
| 861 #define DETECTOR_HAS_METHOD(Detector, Meth) \ | |
| 862 Dynarr_at (all_coding_detectors, detector_##Detector).Meth##_method = \ | |
| 802 | 863 Detector##_##Meth |
| 771 | 864 |
| 865 | |
| 866 /**************************************************/ | |
| 867 /* Decoding/Encoding */ | |
| 868 /**************************************************/ | |
| 869 | |
| 870 /* Is the source (SOURCEP == 1) or sink (SOURCEP == 0) when encoding specified | |
| 871 in characters? */ | |
| 872 | |
| 873 enum source_or_sink | |
| 874 { | |
| 875 CODING_SOURCE, | |
| 876 CODING_SINK | |
| 877 }; | |
| 878 | |
| 879 enum encode_decode | |
| 880 { | |
| 881 CODING_ENCODE, | |
| 882 CODING_DECODE | |
| 883 }; | |
| 884 | |
| 885 /* Data structure attached to an lstream of type `coding', | |
| 886 containing values specific to the coding process. Additional | |
| 887 data is stored in the DATA field below; the exact form of that data | |
| 888 is controlled by the type of the coding system that governs the | |
| 889 conversion (field CODESYS). CODESYS may be set at any time | |
| 890 throughout the lifetime of the lstream and possibly more than once. | |
| 891 See long comment above for more info. */ | |
| 892 | |
| 893 struct coding_stream | |
| 894 { | |
| 1204 | 895 /* Enumerated constant listing which type of console this is (TTY, X, |
| 896 MS-Windows, etc.). This duplicates the method structure in | |
| 897 XCODING_SYSTEM (str->codesys)->methods->type, which formerly was the | |
| 898 only way to determine the coding system type. We need this constant | |
| 899 now for KKCC, so that it can be used in an XD_UNION clause to | |
| 900 determine the Lisp objects in the type-specific data. */ | |
| 901 enum coding_system_variant type; | |
| 902 | |
| 771 | 903 /* Coding system that governs the conversion. */ |
| 904 Lisp_Object codesys; | |
| 905 /* Original coding system, pre-canonicalization. */ | |
| 906 Lisp_Object orig_codesys; | |
| 907 | |
| 908 /* Back pointer to current stream. */ | |
| 909 Lstream *us; | |
| 910 | |
| 911 /* Stream that we read the unprocessed data from or write the processed | |
| 912 data to. */ | |
| 913 Lstream *other_end; | |
| 914 | |
| 915 /* In order to handle both reading to and writing from a coding stream, | |
| 916 we phrase the conversion methods like write methods -- we can | |
| 917 implement reading in terms of a write method but not vice-versa, | |
| 918 because the write method is forced to take only what it's given but | |
| 919 the read method can read more data from the other end if necessary. | |
| 920 On the other hand, the write method is free to generate all the data | |
| 2297 | 921 it wants (and just write it to the other end), but the read method |
| 771 | 922 can return only as much as was asked for, so we need to implement our |
| 923 own buffering. */ | |
| 924 | |
| 925 /* If we are reading, then we can return only a fixed amount of data, but | |
| 926 the converter is free to return as much as it wants, so we direct it | |
| 927 to store the data here and lop off chunks as we need them. If we are | |
| 928 writing, we use this because the converter takes a Dynarr but we are | |
| 929 supposed to write into a fixed buffer. (NOTE: This introduces an extra | |
| 930 memory copy.) */ | |
| 931 unsigned_char_dynarr *convert_to; | |
| 932 | |
| 933 /* The conversion method might reject some of the data -- this typically | |
| 934 includes partial characters, partial escape sequences, etc. When | |
| 935 writing, we just pass the rejection up to the Lstream module, and it | |
| 936 will buffer the data. When reading, however, we need to do the | |
| 937 buffering ourselves, and we put it here, combined with newly read | |
| 938 data. */ | |
| 939 unsigned_char_dynarr *convert_from; | |
| 940 | |
| 941 /* If set, this is the last chunk of data being processed. When this is | |
| 942 finished, output any necessary terminating control characters, escape | |
| 943 sequences, etc. */ | |
| 944 unsigned int eof:1; | |
| 945 | |
| 946 /* CH holds a partially built-up character. This is really part of the | |
| 947 state-dependent data and should be moved there. */ | |
| 948 unsigned int ch; | |
| 949 | |
| 950 /* Coding-system-specific data holding extra state about the | |
| 951 conversion. Logically a struct TYPE_coding_stream; a pointer | |
| 800 | 952 to such a struct, with (when ERROR_CHECK_TYPES is defined) |
| 771 | 953 error-checking that this is really a structure of that type |
| 954 (checking the corresponding coding system type) can be retrieved using | |
| 955 CODING_STREAM_TYPE_DATA(). Allocated at the same time that | |
| 956 CODESYS is set (which may occur at any time, even multiple times, | |
| 957 during the lifetime of the stream). The size comes from | |
| 958 methods->coding_data_size. */ | |
| 959 void *data; | |
| 960 | |
| 961 enum encode_decode direction; | |
| 962 | |
| 800 | 963 /* If set, don't close the stream at the other end when being closed. */ |
| 964 unsigned int no_close_other:1; | |
| 802 | 965 /* If set, read only one byte at a time from other end to avoid any |
| 966 possible blocking. */ | |
| 967 unsigned int one_byte_at_a_time:1; | |
| 814 | 968 /* If set, and we're a read stream, we init char mode on ourselves as |
| 969 necessary to prevent the caller from getting partial characters. (the | |
| 970 default) */ | |
| 971 unsigned int set_char_mode_on_us_when_reading:1; | |
| 800 | 972 |
| 771 | 973 /* #### Temporary test */ |
| 974 unsigned int finalized:1; | |
| 975 }; | |
| 976 | |
| 977 #define CODING_STREAM_DATA(stream) LSTREAM_TYPE_DATA (stream, coding) | |
| 978 | |
| 800 | 979 #ifdef ERROR_CHECK_TYPES |
| 771 | 980 # define CODING_STREAM_TYPE_DATA(s, type) \ |
| 981 error_check_##type##_coding_stream_data (s) | |
| 982 #else | |
| 983 # define CODING_STREAM_TYPE_DATA(s, type) \ | |
| 984 ((struct type##_coding_stream *) (s)->data) | |
| 985 #endif | |
| 986 | |
| 987 /* C should be a binary character in the range 0 - 255; convert | |
| 988 to internal format and add to Dynarr DST. */ | |
| 989 | |
| 428 | 990 #ifdef MULE |
| 771 | 991 |
| 992 #define DECODE_ADD_BINARY_CHAR(c, dst) \ | |
| 993 do { \ | |
| 826 | 994 if (byte_ascii_p (c)) \ |
| 771 | 995 Dynarr_add (dst, c); \ |
| 826 | 996 else if (byte_c1_p (c)) \ |
| 771 | 997 { \ |
| 998 Dynarr_add (dst, LEADING_BYTE_CONTROL_1); \ | |
| 999 Dynarr_add (dst, c + 0x20); \ | |
| 1000 } \ | |
| 1001 else \ | |
| 1002 { \ | |
| 1003 Dynarr_add (dst, LEADING_BYTE_LATIN_ISO8859_1); \ | |
| 1004 Dynarr_add (dst, c); \ | |
| 1005 } \ | |
| 1006 } while (0) | |
| 1007 | |
| 1008 #else /* not MULE */ | |
| 1009 | |
| 1010 #define DECODE_ADD_BINARY_CHAR(c, dst) \ | |
| 1011 do { \ | |
| 1012 Dynarr_add (dst, c); \ | |
| 1013 } while (0) | |
| 1014 | |
| 1015 #endif /* MULE */ | |
| 1016 | |
| 1017 #define DECODE_OUTPUT_PARTIAL_CHAR(ch, dst) \ | |
| 1018 do { \ | |
| 1019 if (ch) \ | |
| 1020 { \ | |
| 1021 DECODE_ADD_BINARY_CHAR (ch, dst); \ | |
| 1022 ch = 0; \ | |
| 1023 } \ | |
| 1024 } while (0) | |
| 428 | 1025 |
| 1026 #ifdef MULE | |
| 1027 /* Convert shift-JIS code (sj1, sj2) into internal string | |
| 1028 representation (c1, c2). (The leading byte is assumed.) */ | |
| 1029 | |
| 771 | 1030 #define DECODE_SHIFT_JIS(sj1, sj2, c1, c2) \ |
| 428 | 1031 do { \ |
| 1032 int I1 = sj1, I2 = sj2; \ | |
| 1033 if (I2 >= 0x9f) \ | |
| 1034 c1 = (I1 << 1) - ((I1 >= 0xe0) ? 0xe0 : 0x60), \ | |
| 1035 c2 = I2 + 2; \ | |
| 1036 else \ | |
| 1037 c1 = (I1 << 1) - ((I1 >= 0xe0) ? 0xe1 : 0x61), \ | |
| 1038 c2 = I2 + ((I2 >= 0x7f) ? 0x60 : 0x61); \ | |
| 1039 } while (0) | |
| 1040 | |
| 1041 /* Convert the internal string representation of a Shift-JIS character | |
| 1042 (c1, c2) into Shift-JIS code (sj1, sj2). The leading byte is | |
| 1043 assumed. */ | |
| 1044 | |
| 771 | 1045 #define ENCODE_SHIFT_JIS(c1, c2, sj1, sj2) \ |
| 428 | 1046 do { \ |
| 1047 int I1 = c1, I2 = c2; \ | |
| 1048 if (I1 & 1) \ | |
| 1049 sj1 = (I1 >> 1) + ((I1 < 0xdf) ? 0x31 : 0x71), \ | |
| 1050 sj2 = I2 - ((I2 >= 0xe0) ? 0x60 : 0x61); \ | |
| 1051 else \ | |
| 1052 sj1 = (I1 >> 1) + ((I1 < 0xdf) ? 0x30 : 0x70), \ | |
| 1053 sj2 = I2 - 2; \ | |
| 1054 } while (0) | |
| 1055 #endif /* MULE */ | |
| 1056 | |
| 771 | 1057 DECLARE_CODING_SYSTEM_TYPE (no_conversion); |
| 1058 DECLARE_CODING_SYSTEM_TYPE (convert_eol); | |
| 1059 #if 0 | |
| 1060 DECLARE_CODING_SYSTEM_TYPE (text_file_wrapper); | |
| 1061 #endif /* 0 */ | |
| 1062 DECLARE_CODING_SYSTEM_TYPE (undecided); | |
| 1063 DECLARE_CODING_SYSTEM_TYPE (chain); | |
| 1064 | |
| 1065 #ifdef DEBUG_XEMACS | |
| 1066 DECLARE_CODING_SYSTEM_TYPE (internal); | |
| 1067 #endif | |
| 1068 | |
| 1069 #ifdef MULE | |
| 1070 DECLARE_CODING_SYSTEM_TYPE (iso2022); | |
| 1071 DECLARE_CODING_SYSTEM_TYPE (ccl); | |
|
4690
257b468bf2ca
Move the #'query-coding-region implementation to C.
Aidan Kehoe <kehoea@parhasard.net>
parents:
4569
diff
changeset
|
1072 DECLARE_CODING_SYSTEM_TYPE (fixed_width); |
| 771 | 1073 DECLARE_CODING_SYSTEM_TYPE (shift_jis); |
| 1074 DECLARE_CODING_SYSTEM_TYPE (big5); | |
| 1075 #endif | |
| 1076 | |
| 1077 #ifdef HAVE_ZLIB | |
| 1078 DECLARE_CODING_SYSTEM_TYPE (gzip); | |
| 1079 #endif | |
| 428 | 1080 |
| 771 | 1081 DECLARE_CODING_SYSTEM_TYPE (unicode); |
| 428 | 1082 |
| 1315 | 1083 #ifdef WIN32_ANY |
| 771 | 1084 DECLARE_CODING_SYSTEM_TYPE (mswindows_multibyte_to_unicode); |
| 1085 DECLARE_CODING_SYSTEM_TYPE (mswindows_multibyte); | |
| 428 | 1086 #endif |
| 771 | 1087 |
| 1088 Lisp_Object coding_stream_detected_coding_system (Lstream *stream); | |
| 1089 Lisp_Object coding_stream_coding_system (Lstream *stream); | |
| 1090 void set_coding_stream_coding_system (Lstream *stream, | |
| 1091 Lisp_Object codesys); | |
| 1092 Lisp_Object detect_coding_stream (Lisp_Object stream); | |
| 867 | 1093 Ichar decode_big5_char (int o1, int o2); |
| 771 | 1094 void add_entry_to_coding_system_type_list (struct coding_system_methods *m); |
| 1095 Lisp_Object make_internal_coding_system (Lisp_Object existing, | |
|
4528
726060ee587c
First draft of g++ 4.3 warning removal patch. Builds. *Needs ChangeLogs.*
Stephen J. Turnbull <stephen@xemacs.org>
parents:
4522
diff
changeset
|
1096 const Ascbyte *prefix, |
| 771 | 1097 Lisp_Object type, |
| 1098 Lisp_Object description, | |
| 1099 Lisp_Object props); | |
| 802 | 1100 |
| 814 | 1101 #define LSTREAM_FL_NO_CLOSE_OTHER (1 << 16) |
| 1102 #define LSTREAM_FL_READ_ONE_BYTE_AT_A_TIME (1 << 17) | |
| 1103 #define LSTREAM_FL_NO_INIT_CHAR_MODE_WHEN_READING (1 << 18) | |
| 1104 | |
| 771 | 1105 Lisp_Object make_coding_input_stream (Lstream *stream, Lisp_Object codesys, |
| 800 | 1106 enum encode_decode direction, |
| 802 | 1107 int flags); |
| 771 | 1108 Lisp_Object make_coding_output_stream (Lstream *stream, Lisp_Object codesys, |
| 800 | 1109 enum encode_decode direction, |
| 802 | 1110 int flags); |
| 771 | 1111 void set_detection_results (struct detection_state *st, int detector, |
| 1112 int given); | |
| 428 | 1113 |
| 440 | 1114 #endif /* INCLUDED_file_coding_h_ */ |
| 1115 |