1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
3 2006 Free Software Foundation, Inc.
4 Copyright (C) 1995, 1997, 1998, 2002, 2003, 2004, 2005
5 National Institute of Advanced Industrial Science and Technology (AIST)
6 Registration Number H14PRO021
8 This file is part of GNU Emacs.
10 GNU Emacs is free software; you can redistribute it and/or modify
11 it under the terms of the GNU General Public License as published by
12 the Free Software Foundation; either version 2, or (at your option)
15 GNU Emacs is distributed in the hope that it will be useful,
16 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
18 GNU General Public License for more details.
20 You should have received a copy of the GNU General Public License
21 along with GNU Emacs; see the file COPYING. If not, write to
22 the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
23 Boston, MA 02110-1301, USA. */
25 /*** TABLE OF CONTENTS ***
29 2. Emacs' internal format (emacs-mule) handlers
31 4. Shift-JIS and BIG5 handlers
33 6. End-of-line handlers
34 7. C library functions
35 8. Emacs Lisp library functions
40 /*** 0. General comments ***/
43 /*** GENERAL NOTE on CODING SYSTEMS ***
45 A coding system is an encoding mechanism for one or more character
46 sets. Here's a list of coding systems which Emacs can handle. When
47 we say "decode", it means converting some other coding system to
48 Emacs' internal format (emacs-mule), and when we say "encode",
49 it means converting the coding system emacs-mule to some other
52 0. Emacs' internal format (emacs-mule)
54 Emacs itself holds a multi-lingual character in buffers and strings
55 in a special format. Details are described in section 2.
59 The most famous coding system for multiple character sets. X's
60 Compound Text, various EUCs (Extended Unix Code), and coding
61 systems used in Internet communication such as ISO-2022-JP are
62 all variants of ISO2022. Details are described in section 3.
64 2. SJIS (or Shift-JIS or MS-Kanji-Code)
66 A coding system to encode character sets: ASCII, JISX0201, and
67 JISX0208. Widely used for PC's in Japan. Details are described in
72 A coding system to encode the character sets ASCII and Big5. Widely
73 used for Chinese (mainly in Taiwan and Hong Kong). Details are
74 described in section 4. In this file, when we write "BIG5"
75 (all uppercase), we mean the coding system, and when we write
76 "Big5" (capitalized), we mean the character set.
80 A coding system for text containing random 8-bit code. Emacs does
81 no code conversion on such text except for end-of-line format.
85 If a user wants to read/write text encoded in a coding system not
86 listed above, he can supply a decoder and an encoder for it as CCL
87 (Code Conversion Language) programs. Emacs executes the CCL program
88 while reading/writing.
90 Emacs represents a coding system by a Lisp symbol that has a property
91 `coding-system'. But, before actually using the coding system, the
92 information about it is set in a structure of type `struct
93 coding_system' for rapid processing. See section 6 for more details.
97 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
99 How end-of-line of text is encoded depends on the operating system.
100 For instance, Unix's format is just one byte of `line-feed' code,
101 whereas DOS's format is two-byte sequence of `carriage-return' and
102 `line-feed' codes. MacOS's format is usually one byte of
105 Since text character encoding and end-of-line encoding are
106 independent, any coding system described above can have any
107 end-of-line format. So Emacs has information about end-of-line
108 format in each coding-system. See section 6 for more details.
112 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
114 These functions check if a text between SRC and SRC_END is encoded
115 in the coding system category XXX. Each returns an integer value in
116 which appropriate flag bits for the category XXX are set. The flag
117 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
118 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
119 of the range 0x80..0x9F are in multibyte form. */
122 detect_coding_emacs_mule (src
, src_end
, multibytep
)
123 unsigned char *src
, *src_end
;
130 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
132 These functions decode SRC_BYTES length of unibyte text at SOURCE
133 encoded in CODING to Emacs' internal format. The resulting
134 multibyte text goes to a place pointed to by DESTINATION, the length
135 of which should not exceed DST_BYTES.
137 These functions set the information about original and decoded texts
138 in the members `produced', `produced_char', `consumed', and
139 `consumed_char' of the structure *CODING. They also set the member
140 `result' to one of CODING_FINISH_XXX indicating how the decoding
143 DST_BYTES zero means that the source area and destination area are
144 overlapped, which means that we can produce a decoded text until it
145 reaches the head of the not-yet-decoded source text.
147 Below is a template for these functions. */
150 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
151 struct coding_system
*coding
;
152 const unsigned char *source
;
153 unsigned char *destination
;
154 int src_bytes
, dst_bytes
;
160 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
162 These functions encode SRC_BYTES length text at SOURCE from Emacs'
163 internal multibyte format to CODING. The resulting unibyte text
164 goes to a place pointed to by DESTINATION, the length of which
165 should not exceed DST_BYTES.
167 These functions set the information about original and encoded texts
168 in the members `produced', `produced_char', `consumed', and
169 `consumed_char' of the structure *CODING. They also set the member
170 `result' to one of CODING_FINISH_XXX indicating how the encoding
173 DST_BYTES zero means that the source area and destination area are
174 overlapped, which means that we can produce encoded text until it
175 reaches at the head of the not-yet-encoded source text.
177 Below is a template for these functions. */
180 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
181 struct coding_system
*coding
;
182 unsigned char *source
, *destination
;
183 int src_bytes
, dst_bytes
;
189 /*** COMMONLY USED MACROS ***/
191 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
192 get one, two, and three bytes from the source text respectively.
193 If there are not enough bytes in the source, they jump to
194 `label_end_of_loop'. The caller should set variables `coding',
195 `src' and `src_end' to appropriate pointer in advance. These
196 macros are called from decoding routines `decode_coding_XXX', thus
197 it is assumed that the source text is unibyte. */
199 #define ONE_MORE_BYTE(c1) \
201 if (src >= src_end) \
203 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
204 goto label_end_of_loop; \
209 #define TWO_MORE_BYTES(c1, c2) \
211 if (src + 1 >= src_end) \
213 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
214 goto label_end_of_loop; \
221 /* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
222 form if MULTIBYTEP is nonzero. */
224 #define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep) \
226 if (src >= src_end) \
228 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
229 goto label_end_of_loop; \
232 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
233 c1 = *src++ - 0x20; \
236 /* Set C to the next character at the source text pointed by `src'.
237 If there are not enough characters in the source, jump to
238 `label_end_of_loop'. The caller should set variables `coding'
239 `src', `src_end', and `translation_table' to appropriate pointers
240 in advance. This macro is used in encoding routines
241 `encode_coding_XXX', thus it assumes that the source text is in
242 multibyte form except for 8-bit characters. 8-bit characters are
243 in multibyte form if coding->src_multibyte is nonzero, else they
244 are represented by a single byte. */
246 #define ONE_MORE_CHAR(c) \
248 int len = src_end - src; \
252 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
253 goto label_end_of_loop; \
255 if (coding->src_multibyte \
256 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
257 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
259 c = *src, bytes = 1; \
260 if (!NILP (translation_table)) \
261 c = translate_char (translation_table, c, -1, 0, 0); \
266 /* Produce a multibyte form of character C to `dst'. Jump to
267 `label_end_of_loop' if there's not enough space at `dst'.
269 If we are now in the middle of a composition sequence, the decoded
270 character may be ALTCHAR (for the current composition). In that
271 case, the character goes to coding->cmp_data->data instead of
274 This macro is used in decoding routines. */
276 #define EMIT_CHAR(c) \
278 if (! COMPOSING_P (coding) \
279 || coding->composing == COMPOSITION_RELATIVE \
280 || coding->composing == COMPOSITION_WITH_RULE) \
282 int bytes = CHAR_BYTES (c); \
283 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
285 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
286 goto label_end_of_loop; \
288 dst += CHAR_STRING (c, dst); \
289 coding->produced_char++; \
292 if (COMPOSING_P (coding) \
293 && coding->composing != COMPOSITION_RELATIVE) \
295 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
296 coding->composition_rule_follows \
297 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
302 #define EMIT_ONE_BYTE(c) \
304 if (dst >= (dst_bytes ? dst_end : src)) \
306 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
307 goto label_end_of_loop; \
312 #define EMIT_TWO_BYTES(c1, c2) \
314 if (dst + 2 > (dst_bytes ? dst_end : src)) \
316 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
317 goto label_end_of_loop; \
319 *dst++ = c1, *dst++ = c2; \
322 #define EMIT_BYTES(from, to) \
324 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
326 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
327 goto label_end_of_loop; \
334 /*** 1. Preamble ***/
347 #include "composite.h"
351 #include "intervals.h"
353 #else /* not emacs */
357 #endif /* not emacs */
359 Lisp_Object Qcoding_system
, Qeol_type
;
360 Lisp_Object Qbuffer_file_coding_system
;
361 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
362 Lisp_Object Qno_conversion
, Qundecided
;
363 Lisp_Object Qcoding_system_history
;
364 Lisp_Object Qsafe_chars
;
365 Lisp_Object Qvalid_codes
;
367 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
368 Lisp_Object Qcall_process
, Qcall_process_region
;
369 Lisp_Object Qstart_process
, Qopen_network_stream
;
370 Lisp_Object Qtarget_idx
;
372 /* If a symbol has this property, evaluate the value to define the
373 symbol as a coding system. */
374 Lisp_Object Qcoding_system_define_form
;
376 Lisp_Object Vselect_safe_coding_system_function
;
378 int coding_system_require_warning
;
380 /* Mnemonic string for each format of end-of-line. */
381 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
382 /* Mnemonic string to indicate format of end-of-line is not yet
384 Lisp_Object eol_mnemonic_undecided
;
386 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
387 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
392 /* Information about which coding system is safe for which chars.
393 The value has the form (GENERIC-LIST . NON-GENERIC-ALIST).
395 GENERIC-LIST is a list of generic coding systems which can encode
398 NON-GENERIC-ALIST is an alist of non generic coding systems vs the
399 corresponding char table that contains safe chars. */
400 Lisp_Object Vcoding_system_safe_chars
;
402 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
404 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
406 /* Coding system emacs-mule and raw-text are for converting only
407 end-of-line format. */
408 Lisp_Object Qemacs_mule
, Qraw_text
;
412 /* Coding-systems are handed between Emacs Lisp programs and C internal
413 routines by the following three variables. */
414 /* Coding-system for reading files and receiving data from process. */
415 Lisp_Object Vcoding_system_for_read
;
416 /* Coding-system for writing files and sending data to process. */
417 Lisp_Object Vcoding_system_for_write
;
418 /* Coding-system actually used in the latest I/O. */
419 Lisp_Object Vlast_coding_system_used
;
421 /* A vector of length 256 which contains information about special
422 Latin codes (especially for dealing with Microsoft codes). */
423 Lisp_Object Vlatin_extra_code_table
;
425 /* Flag to inhibit code conversion of end-of-line format. */
426 int inhibit_eol_conversion
;
428 /* Flag to inhibit ISO2022 escape sequence detection. */
429 int inhibit_iso_escape_detection
;
431 /* Flag to make buffer-file-coding-system inherit from process-coding. */
432 int inherit_process_coding_system
;
434 /* Coding system to be used to encode text for terminal display. */
435 struct coding_system terminal_coding
;
437 /* Coding system to be used to encode text for terminal display when
438 terminal coding system is nil. */
439 struct coding_system safe_terminal_coding
;
441 /* Coding system of what is sent from terminal keyboard. */
442 struct coding_system keyboard_coding
;
444 /* Default coding system to be used to write a file. */
445 struct coding_system default_buffer_file_coding
;
447 Lisp_Object Vfile_coding_system_alist
;
448 Lisp_Object Vprocess_coding_system_alist
;
449 Lisp_Object Vnetwork_coding_system_alist
;
451 Lisp_Object Vlocale_coding_system
;
455 Lisp_Object Qcoding_category
, Qcoding_category_index
;
457 /* List of symbols `coding-category-xxx' ordered by priority. */
458 Lisp_Object Vcoding_category_list
;
460 /* Table of coding categories (Lisp symbols). */
461 Lisp_Object Vcoding_category_table
;
463 /* Table of names of symbol for each coding-category. */
464 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
465 "coding-category-emacs-mule",
466 "coding-category-sjis",
467 "coding-category-iso-7",
468 "coding-category-iso-7-tight",
469 "coding-category-iso-8-1",
470 "coding-category-iso-8-2",
471 "coding-category-iso-7-else",
472 "coding-category-iso-8-else",
473 "coding-category-ccl",
474 "coding-category-big5",
475 "coding-category-utf-8",
476 "coding-category-utf-16-be",
477 "coding-category-utf-16-le",
478 "coding-category-raw-text",
479 "coding-category-binary"
482 /* Table of pointers to coding systems corresponding to each coding
484 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
486 /* Table of coding category masks. Nth element is a mask for a coding
487 category of which priority is Nth. */
489 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
491 /* Flag to tell if we look up translation table on character code
493 Lisp_Object Venable_character_translation
;
494 /* Standard translation table to look up on decoding (reading). */
495 Lisp_Object Vstandard_translation_table_for_decode
;
496 /* Standard translation table to look up on encoding (writing). */
497 Lisp_Object Vstandard_translation_table_for_encode
;
499 Lisp_Object Qtranslation_table
;
500 Lisp_Object Qtranslation_table_id
;
501 Lisp_Object Qtranslation_table_for_decode
;
502 Lisp_Object Qtranslation_table_for_encode
;
504 /* Alist of charsets vs revision number. */
505 Lisp_Object Vcharset_revision_alist
;
507 /* Default coding systems used for process I/O. */
508 Lisp_Object Vdefault_process_coding_system
;
510 /* Char table for translating Quail and self-inserting input. */
511 Lisp_Object Vtranslation_table_for_input
;
513 /* Global flag to tell that we can't call post-read-conversion and
514 pre-write-conversion functions. Usually the value is zero, but it
515 is set to 1 temporarily while such functions are running. This is
516 to avoid infinite recursive call. */
517 static int inhibit_pre_post_conversion
;
519 Lisp_Object Qchar_coding_system
;
521 /* Return `safe-chars' property of CODING_SYSTEM (symbol). Don't check
525 coding_safe_chars (coding_system
)
526 Lisp_Object coding_system
;
528 Lisp_Object coding_spec
, plist
, safe_chars
;
530 coding_spec
= Fget (coding_system
, Qcoding_system
);
531 plist
= XVECTOR (coding_spec
)->contents
[3];
532 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
533 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
536 #define CODING_SAFE_CHAR_P(safe_chars, c) \
537 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
540 /*** 2. Emacs internal format (emacs-mule) handlers ***/
542 /* Emacs' internal format for representation of multiple character
543 sets is a kind of multi-byte encoding, i.e. characters are
544 represented by variable-length sequences of one-byte codes.
546 ASCII characters and control characters (e.g. `tab', `newline') are
547 represented by one-byte sequences which are their ASCII codes, in
548 the range 0x00 through 0x7F.
550 8-bit characters of the range 0x80..0x9F are represented by
551 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
554 8-bit characters of the range 0xA0..0xFF are represented by
555 one-byte sequences which are their 8-bit code.
557 The other characters are represented by a sequence of `base
558 leading-code', optional `extended leading-code', and one or two
559 `position-code's. The length of the sequence is determined by the
560 base leading-code. Leading-code takes the range 0x81 through 0x9D,
561 whereas extended leading-code and position-code take the range 0xA0
562 through 0xFF. See `charset.h' for more details about leading-code
565 --- CODE RANGE of Emacs' internal format ---
569 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
570 eight-bit-graphic 0xA0..0xBF
571 ELSE 0x81..0x9D + [0xA0..0xFF]+
572 ---------------------------------------------
574 As this is the internal character representation, the format is
575 usually not used externally (i.e. in a file or in a data sent to a
576 process). But, it is possible to have a text externally in this
577 format (i.e. by encoding by the coding system `emacs-mule').
579 In that case, a sequence of one-byte codes has a slightly different
582 Firstly, all characters in eight-bit-control are represented by
583 one-byte sequences which are their 8-bit code.
585 Next, character composition data are represented by the byte
586 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
588 METHOD is 0xF0 plus one of composition method (enum
591 BYTES is 0xA0 plus the byte length of these composition data,
593 CHARS is 0xA0 plus the number of characters composed by these
596 COMPONENTs are characters of multibyte form or composition
597 rules encoded by two-byte of ASCII codes.
599 In addition, for backward compatibility, the following formats are
600 also recognized as composition data on decoding.
603 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
606 MSEQ is a multibyte form but in these special format:
607 ASCII: 0xA0 ASCII_CODE+0x80,
608 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
609 RULE is a one byte code of the range 0xA0..0xF0 that
610 represents a composition rule.
613 enum emacs_code_class_type emacs_code_class
[256];
615 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
616 Check if a text is encoded in Emacs' internal format. If it is,
617 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
620 detect_coding_emacs_mule (src
, src_end
, multibytep
)
621 unsigned char *src
, *src_end
;
626 /* Dummy for ONE_MORE_BYTE. */
627 struct coding_system dummy_coding
;
628 struct coding_system
*coding
= &dummy_coding
;
632 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
640 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
649 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
652 else if (c
>= 0x80 && c
< 0xA0)
655 /* Old leading code for a composite character. */
659 unsigned char *src_base
= src
- 1;
662 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
665 src
= src_base
+ bytes
;
670 return CODING_CATEGORY_MASK_EMACS_MULE
;
674 /* Record the starting position START and METHOD of one composition. */
676 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
678 struct composition_data *cmp_data = coding->cmp_data; \
679 int *data = cmp_data->data + cmp_data->used; \
680 coding->cmp_data_start = cmp_data->used; \
682 data[1] = cmp_data->char_offset + start; \
683 data[3] = (int) method; \
684 cmp_data->used += 4; \
687 /* Record the ending position END of the current composition. */
689 #define CODING_ADD_COMPOSITION_END(coding, end) \
691 struct composition_data *cmp_data = coding->cmp_data; \
692 int *data = cmp_data->data + coding->cmp_data_start; \
693 data[0] = cmp_data->used - coding->cmp_data_start; \
694 data[2] = cmp_data->char_offset + end; \
697 /* Record one COMPONENT (alternate character or composition rule). */
699 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
701 coding->cmp_data->data[coding->cmp_data->used++] = component; \
702 if (coding->cmp_data->used - coding->cmp_data_start \
703 == COMPOSITION_DATA_MAX_BUNCH_LENGTH) \
705 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
706 coding->composing = COMPOSITION_NO; \
711 /* Get one byte from a data pointed by SRC and increment SRC. If SRC
712 is not less than SRC_END, return -1 without incrementing Src. */
714 #define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
717 /* Decode a character represented as a component of composition
718 sequence of Emacs 20 style at SRC. Set C to that character, store
719 its multibyte form sequence at P, and set P to the end of that
720 sequence. If no valid character is found, set C to -1. */
722 #define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
726 c = SAFE_ONE_MORE_BYTE (); \
729 if (CHAR_HEAD_P (c)) \
731 else if (c == 0xA0) \
733 c = SAFE_ONE_MORE_BYTE (); \
742 else if (BASE_LEADING_CODE_P (c - 0x20)) \
744 unsigned char *p0 = p; \
748 bytes = BYTES_BY_CHAR_HEAD (c); \
751 c = SAFE_ONE_MORE_BYTE (); \
756 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes) \
757 || (coding->flags /* We are recovering a file. */ \
758 && p0[0] == LEADING_CODE_8_BIT_CONTROL \
759 && ! CHAR_HEAD_P (p0[1]))) \
760 c = STRING_CHAR (p0, bytes); \
769 /* Decode a composition rule represented as a component of composition
770 sequence of Emacs 20 style at SRC. Set C to the rule. If not
771 valid rule is found, set C to -1. */
773 #define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
775 c = SAFE_ONE_MORE_BYTE (); \
777 if (c < 0 || c >= 81) \
781 gref = c / 9, nref = c % 9; \
782 c = COMPOSITION_ENCODE_RULE (gref, nref); \
787 /* Decode composition sequence encoded by `emacs-mule' at the source
788 pointed by SRC. SRC_END is the end of source. Store information
789 of the composition in CODING->cmp_data.
791 For backward compatibility, decode also a composition sequence of
792 Emacs 20 style. In that case, the composition sequence contains
793 characters that should be extracted into a buffer or string. Store
794 those characters at *DESTINATION in multibyte form.
796 If we encounter an invalid byte sequence, return 0.
797 If we encounter an insufficient source or destination, or
798 insufficient space in CODING->cmp_data, return 1.
799 Otherwise, return consumed bytes in the source.
803 decode_composition_emacs_mule (coding
, src
, src_end
,
804 destination
, dst_end
, dst_bytes
)
805 struct coding_system
*coding
;
806 const unsigned char *src
, *src_end
;
807 unsigned char **destination
, *dst_end
;
810 unsigned char *dst
= *destination
;
811 int method
, data_len
, nchars
;
812 const unsigned char *src_base
= src
++;
813 /* Store components of composition. */
814 int component
[COMPOSITION_DATA_MAX_BUNCH_LENGTH
];
816 /* Store multibyte form of characters to be composed. This is for
817 Emacs 20 style composition sequence. */
818 unsigned char buf
[MAX_COMPOSITION_COMPONENTS
* MAX_MULTIBYTE_LENGTH
];
819 unsigned char *bufp
= buf
;
820 int c
, i
, gref
, nref
;
822 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
823 >= COMPOSITION_DATA_SIZE
)
825 coding
->result
= CODING_FINISH_INSUFFICIENT_CMP
;
830 if (c
- 0xF0 >= COMPOSITION_RELATIVE
831 && c
- 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS
)
836 with_rule
= (method
== COMPOSITION_WITH_RULE
837 || method
== COMPOSITION_WITH_RULE_ALTCHARS
);
841 || src_base
+ data_len
> src_end
)
847 for (ncomponent
= 0; src
< src_base
+ data_len
; ncomponent
++)
849 /* If it is longer than this, it can't be valid. */
850 if (ncomponent
>= COMPOSITION_DATA_MAX_BUNCH_LENGTH
)
853 if (ncomponent
% 2 && with_rule
)
855 ONE_MORE_BYTE (gref
);
857 ONE_MORE_BYTE (nref
);
859 c
= COMPOSITION_ENCODE_RULE (gref
, nref
);
864 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
865 || (coding
->flags
/* We are recovering a file. */
866 && src
[0] == LEADING_CODE_8_BIT_CONTROL
867 && ! CHAR_HEAD_P (src
[1])))
868 c
= STRING_CHAR (src
, bytes
);
873 component
[ncomponent
] = c
;
878 /* This may be an old Emacs 20 style format. See the comment at
879 the section 2 of this file. */
880 while (src
< src_end
&& !CHAR_HEAD_P (*src
)) src
++;
882 && !(coding
->mode
& CODING_MODE_LAST_BLOCK
))
883 goto label_end_of_loop
;
889 method
= COMPOSITION_RELATIVE
;
890 for (ncomponent
= 0; ncomponent
< MAX_COMPOSITION_COMPONENTS
;)
892 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
895 component
[ncomponent
++] = c
;
903 method
= COMPOSITION_WITH_RULE
;
905 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
910 ncomponent
< MAX_COMPOSITION_COMPONENTS
* 2 - 1;)
912 DECODE_EMACS_MULE_COMPOSITION_RULE (c
);
915 component
[ncomponent
++] = c
;
916 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
919 component
[ncomponent
++] = c
;
923 nchars
= (ncomponent
+ 1) / 2;
931 if (buf
== bufp
|| dst
+ (bufp
- buf
) <= (dst_bytes
? dst_end
: src
))
933 CODING_ADD_COMPOSITION_START (coding
, coding
->produced_char
, method
);
934 for (i
= 0; i
< ncomponent
; i
++)
935 CODING_ADD_COMPOSITION_COMPONENT (coding
, component
[i
]);
936 CODING_ADD_COMPOSITION_END (coding
, coding
->produced_char
+ nchars
);
939 unsigned char *p
= buf
;
940 EMIT_BYTES (p
, bufp
);
941 *destination
+= bufp
- buf
;
942 coding
->produced_char
+= nchars
;
944 return (src
- src_base
);
950 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
953 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
954 struct coding_system
*coding
;
955 const unsigned char *source
;
956 unsigned char *destination
;
957 int src_bytes
, dst_bytes
;
959 const unsigned char *src
= source
;
960 const unsigned char *src_end
= source
+ src_bytes
;
961 unsigned char *dst
= destination
;
962 unsigned char *dst_end
= destination
+ dst_bytes
;
963 /* SRC_BASE remembers the start position in source in each loop.
964 The loop will be exited when there's not enough source code, or
965 when there's not enough destination area to produce a
967 const unsigned char *src_base
;
969 coding
->produced_char
= 0;
970 while ((src_base
= src
) < src_end
)
972 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
];
973 const unsigned char *p
;
980 if (coding
->eol_type
== CODING_EOL_CR
)
982 else if (coding
->eol_type
== CODING_EOL_CRLF
)
992 coding
->produced_char
++;
995 else if (*src
== '\n')
997 if ((coding
->eol_type
== CODING_EOL_CR
998 || coding
->eol_type
== CODING_EOL_CRLF
)
999 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1001 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1002 goto label_end_of_loop
;
1005 coding
->produced_char
++;
1008 else if (*src
== 0x80 && coding
->cmp_data
)
1010 /* Start of composition data. */
1011 int consumed
= decode_composition_emacs_mule (coding
, src
, src_end
,
1015 goto label_end_of_loop
;
1016 else if (consumed
> 0)
1021 bytes
= CHAR_STRING (*src
, tmp
);
1025 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
1026 || (coding
->flags
/* We are recovering a file. */
1027 && src
[0] == LEADING_CODE_8_BIT_CONTROL
1028 && ! CHAR_HEAD_P (src
[1])))
1037 bytes
= BYTES_BY_CHAR_HEAD (*src
);
1039 for (i
= 1; i
< bytes
; i
++)
1042 if (CHAR_HEAD_P (c
))
1047 bytes
= CHAR_STRING (*src_base
, tmp
);
1056 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
1058 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
1061 while (bytes
--) *dst
++ = *p
++;
1062 coding
->produced_char
++;
1065 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1066 coding
->produced
= dst
- destination
;
1070 /* Encode composition data stored at DATA into a special byte sequence
1071 starting by 0x80. Update CODING->cmp_data_start and maybe
1072 CODING->cmp_data for the next call. */
1074 #define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1076 unsigned char buf[1024], *p0 = buf, *p; \
1077 int len = data[0]; \
1081 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1082 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1084 if (data[3] == COMPOSITION_WITH_RULE \
1085 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1087 p += CHAR_STRING (data[4], p); \
1088 for (i = 5; i < len; i += 2) \
1091 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1092 *p++ = 0x20 + gref; \
1093 *p++ = 0x20 + nref; \
1094 p += CHAR_STRING (data[i + 1], p); \
1099 for (i = 4; i < len; i++) \
1100 p += CHAR_STRING (data[i], p); \
1102 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1104 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1106 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1107 goto label_end_of_loop; \
1111 coding->cmp_data_start += data[0]; \
1112 if (coding->cmp_data_start == coding->cmp_data->used \
1113 && coding->cmp_data->next) \
1115 coding->cmp_data = coding->cmp_data->next; \
1116 coding->cmp_data_start = 0; \
1121 static void encode_eol
P_ ((struct coding_system
*, const unsigned char *,
1122 unsigned char *, int, int));
1125 encode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
1126 struct coding_system
*coding
;
1127 const unsigned char *source
;
1128 unsigned char *destination
;
1129 int src_bytes
, dst_bytes
;
1131 const unsigned char *src
= source
;
1132 const unsigned char *src_end
= source
+ src_bytes
;
1133 unsigned char *dst
= destination
;
1134 unsigned char *dst_end
= destination
+ dst_bytes
;
1135 const unsigned char *src_base
;
1140 Lisp_Object translation_table
;
1142 translation_table
= Qnil
;
1144 /* Optimization for the case that there's no composition. */
1145 if (!coding
->cmp_data
|| coding
->cmp_data
->used
== 0)
1147 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
1151 char_offset
= coding
->cmp_data
->char_offset
;
1152 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1157 /* If SRC starts a composition, encode the information about the
1158 composition in advance. */
1159 if (coding
->cmp_data_start
< coding
->cmp_data
->used
1160 && char_offset
+ coding
->consumed_char
== data
[1])
1162 ENCODE_COMPOSITION_EMACS_MULE (coding
, data
);
1163 char_offset
= coding
->cmp_data
->char_offset
;
1164 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1168 if (c
== '\n' && (coding
->eol_type
== CODING_EOL_CRLF
1169 || coding
->eol_type
== CODING_EOL_CR
))
1171 if (coding
->eol_type
== CODING_EOL_CRLF
)
1172 EMIT_TWO_BYTES ('\r', c
);
1174 EMIT_ONE_BYTE ('\r');
1176 else if (SINGLE_BYTE_CHAR_P (c
))
1178 if (coding
->flags
&& ! ASCII_BYTE_P (c
))
1180 /* As we are auto saving, retain the multibyte form for
1182 unsigned char buf
[MAX_MULTIBYTE_LENGTH
];
1183 int bytes
= CHAR_STRING (c
, buf
);
1186 EMIT_ONE_BYTE (buf
[0]);
1188 EMIT_TWO_BYTES (buf
[0], buf
[1]);
1194 EMIT_BYTES (src_base
, src
);
1195 coding
->consumed_char
++;
1198 coding
->consumed
= src_base
- source
;
1199 coding
->produced
= coding
->produced_char
= dst
- destination
;
1204 /*** 3. ISO2022 handlers ***/
1206 /* The following note describes the coding system ISO2022 briefly.
1207 Since the intention of this note is to help understand the
1208 functions in this file, some parts are NOT ACCURATE or are OVERLY
1209 SIMPLIFIED. For thorough understanding, please refer to the
1210 original document of ISO2022. This is equivalent to the standard
1211 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1213 ISO2022 provides many mechanisms to encode several character sets
1214 in 7-bit and 8-bit environments. For 7-bit environments, all text
1215 is encoded using bytes less than 128. This may make the encoded
1216 text a little bit longer, but the text passes more easily through
1217 several types of gateway, some of which strip off the MSB (Most
1220 There are two kinds of character sets: control character sets and
1221 graphic character sets. The former contain control characters such
1222 as `newline' and `escape' to provide control functions (control
1223 functions are also provided by escape sequences). The latter
1224 contain graphic characters such as 'A' and '-'. Emacs recognizes
1225 two control character sets and many graphic character sets.
1227 Graphic character sets are classified into one of the following
1228 four classes, according to the number of bytes (DIMENSION) and
1229 number of characters in one dimension (CHARS) of the set:
1230 - DIMENSION1_CHARS94
1231 - DIMENSION1_CHARS96
1232 - DIMENSION2_CHARS94
1233 - DIMENSION2_CHARS96
1235 In addition, each character set is assigned an identification tag,
1236 unique for each set, called the "final character" (denoted as <F>
1237 hereafter). The <F> of each character set is decided by ECMA(*)
1238 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1239 (0x30..0x3F are for private use only).
1241 Note (*): ECMA = European Computer Manufacturers Association
1243 Here are examples of graphic character sets [NAME(<F>)]:
1244 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1245 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1246 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1247 o DIMENSION2_CHARS96 -- none for the moment
1249 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
1250 C0 [0x00..0x1F] -- control character plane 0
1251 GL [0x20..0x7F] -- graphic character plane 0
1252 C1 [0x80..0x9F] -- control character plane 1
1253 GR [0xA0..0xFF] -- graphic character plane 1
1255 A control character set is directly designated and invoked to C0 or
1256 C1 by an escape sequence. The most common case is that:
1257 - ISO646's control character set is designated/invoked to C0, and
1258 - ISO6429's control character set is designated/invoked to C1,
1259 and usually these designations/invocations are omitted in encoded
1260 text. In a 7-bit environment, only C0 can be used, and a control
1261 character for C1 is encoded by an appropriate escape sequence to
1262 fit into the environment. All control characters for C1 are
1263 defined to have corresponding escape sequences.
1265 A graphic character set is at first designated to one of four
1266 graphic registers (G0 through G3), then these graphic registers are
1267 invoked to GL or GR. These designations and invocations can be
1268 done independently. The most common case is that G0 is invoked to
1269 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1270 these invocations and designations are omitted in encoded text.
1271 In a 7-bit environment, only GL can be used.
1273 When a graphic character set of CHARS94 is invoked to GL, codes
1274 0x20 and 0x7F of the GL area work as control characters SPACE and
1275 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1278 There are two ways of invocation: locking-shift and single-shift.
1279 With locking-shift, the invocation lasts until the next different
1280 invocation, whereas with single-shift, the invocation affects the
1281 following character only and doesn't affect the locking-shift
1282 state. Invocations are done by the following control characters or
1285 ----------------------------------------------------------------------
1286 abbrev function cntrl escape seq description
1287 ----------------------------------------------------------------------
1288 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1289 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1290 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1291 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1292 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1293 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1294 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1295 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1296 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
1297 ----------------------------------------------------------------------
1298 (*) These are not used by any known coding system.
1300 Control characters for these functions are defined by macros
1301 ISO_CODE_XXX in `coding.h'.
1303 Designations are done by the following escape sequences:
1304 ----------------------------------------------------------------------
1305 escape sequence description
1306 ----------------------------------------------------------------------
1307 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1308 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1309 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1310 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1311 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1312 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1313 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1314 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1315 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1316 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1317 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1318 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1319 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1320 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1321 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1322 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1323 ----------------------------------------------------------------------
1325 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
1326 of dimension 1, chars 94, and final character <F>, etc...
1328 Note (*): Although these designations are not allowed in ISO2022,
1329 Emacs accepts them on decoding, and produces them on encoding
1330 CHARS96 character sets in a coding system which is characterized as
1331 7-bit environment, non-locking-shift, and non-single-shift.
1333 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1334 '(' can be omitted. We refer to this as "short-form" hereafter.
1336 Now you may notice that there are a lot of ways of encoding the
1337 same multilingual text in ISO2022. Actually, there exist many
1338 coding systems such as Compound Text (used in X11's inter client
1339 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1340 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
1341 localized platforms), and all of these are variants of ISO2022.
1343 In addition to the above, Emacs handles two more kinds of escape
1344 sequences: ISO6429's direction specification and Emacs' private
1345 sequence for specifying character composition.
1347 ISO6429's direction specification takes the following form:
1348 o CSI ']' -- end of the current direction
1349 o CSI '0' ']' -- end of the current direction
1350 o CSI '1' ']' -- start of left-to-right text
1351 o CSI '2' ']' -- start of right-to-left text
1352 The control character CSI (0x9B: control sequence introducer) is
1353 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1355 Character composition specification takes the following form:
1356 o ESC '0' -- start relative composition
1357 o ESC '1' -- end composition
1358 o ESC '2' -- start rule-base composition (*)
1359 o ESC '3' -- start relative composition with alternate chars (**)
1360 o ESC '4' -- start rule-base composition with alternate chars (**)
1361 Since these are not standard escape sequences of any ISO standard,
1362 the use of them with these meanings is restricted to Emacs only.
1364 (*) This form is used only in Emacs 20.5 and older versions,
1365 but the newer versions can safely decode it.
1366 (**) This form is used only in Emacs 21.1 and newer versions,
1367 and the older versions can't decode it.
1369 Here's a list of example usages of these composition escape
1370 sequences (categorized by `enum composition_method').
1372 COMPOSITION_RELATIVE:
1373 ESC 0 CHAR [ CHAR ] ESC 1
1374 COMPOSITION_WITH_RULE:
1375 ESC 2 CHAR [ RULE CHAR ] ESC 1
1376 COMPOSITION_WITH_ALTCHARS:
1377 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
1378 COMPOSITION_WITH_RULE_ALTCHARS:
1379 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
1381 enum iso_code_class_type iso_code_class
[256];
1383 #define CHARSET_OK(idx, charset, c) \
1384 (coding_system_table[idx] \
1385 && (charset == CHARSET_ASCII \
1386 || (safe_chars = coding_safe_chars (coding_system_table[idx]->symbol), \
1387 CODING_SAFE_CHAR_P (safe_chars, c))) \
1388 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1390 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
1392 #define SHIFT_OUT_OK(idx) \
1393 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1395 #define COMPOSITION_OK(idx) \
1396 (coding_system_table[idx]->composing != COMPOSITION_DISABLED)
1398 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1399 Check if a text is encoded in ISO2022. If it is, return an
1400 integer in which appropriate flag bits any of:
1401 CODING_CATEGORY_MASK_ISO_7
1402 CODING_CATEGORY_MASK_ISO_7_TIGHT
1403 CODING_CATEGORY_MASK_ISO_8_1
1404 CODING_CATEGORY_MASK_ISO_8_2
1405 CODING_CATEGORY_MASK_ISO_7_ELSE
1406 CODING_CATEGORY_MASK_ISO_8_ELSE
1407 are set. If a code which should never appear in ISO2022 is found,
1411 detect_coding_iso2022 (src
, src_end
, multibytep
)
1412 unsigned char *src
, *src_end
;
1415 int mask
= CODING_CATEGORY_MASK_ISO
;
1417 int reg
[4], shift_out
= 0, single_shifting
= 0;
1419 /* Dummy for ONE_MORE_BYTE. */
1420 struct coding_system dummy_coding
;
1421 struct coding_system
*coding
= &dummy_coding
;
1422 Lisp_Object safe_chars
;
1424 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
1425 while (mask
&& src
< src_end
)
1427 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1432 if (inhibit_iso_escape_detection
)
1434 single_shifting
= 0;
1435 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1436 if (c
>= '(' && c
<= '/')
1438 /* Designation sequence for a charset of dimension 1. */
1439 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1440 if (c1
< ' ' || c1
>= 0x80
1441 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
1442 /* Invalid designation sequence. Just ignore. */
1444 reg
[(c
- '(') % 4] = charset
;
1448 /* Designation sequence for a charset of dimension 2. */
1449 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1450 if (c
>= '@' && c
<= 'B')
1451 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1452 reg
[0] = charset
= iso_charset_table
[1][0][c
];
1453 else if (c
>= '(' && c
<= '/')
1455 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1456 if (c1
< ' ' || c1
>= 0x80
1457 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
1458 /* Invalid designation sequence. Just ignore. */
1460 reg
[(c
- '(') % 4] = charset
;
1463 /* Invalid designation sequence. Just ignore. */
1466 else if (c
== 'N' || c
== 'O')
1468 /* ESC <Fe> for SS2 or SS3. */
1469 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1472 else if (c
>= '0' && c
<= '4')
1474 /* ESC <Fp> for start/end composition. */
1475 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7
))
1476 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1478 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1479 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
))
1480 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1482 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1483 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_1
))
1484 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1486 mask
&= ~CODING_CATEGORY_MASK_ISO_8_1
;
1487 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_2
))
1488 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1490 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1491 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
))
1492 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1494 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1495 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
))
1496 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1498 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1502 /* Invalid escape sequence. Just ignore. */
1505 /* We found a valid designation sequence for CHARSET. */
1506 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
1507 c
= MAKE_CHAR (charset
, 0, 0);
1508 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
1509 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1511 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1512 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
1513 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1515 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1516 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
1517 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1519 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1520 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
1521 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1523 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1527 if (inhibit_iso_escape_detection
)
1529 single_shifting
= 0;
1532 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
1533 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
1535 /* Locking shift out. */
1536 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1537 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1542 if (inhibit_iso_escape_detection
)
1544 single_shifting
= 0;
1547 /* Locking shift in. */
1548 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1549 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1554 single_shifting
= 0;
1558 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1560 if (inhibit_iso_escape_detection
)
1562 if (c
!= ISO_CODE_CSI
)
1564 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1565 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1566 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1567 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1568 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1569 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1570 single_shifting
= 1;
1572 if (VECTORP (Vlatin_extra_code_table
)
1573 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1575 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1576 & CODING_FLAG_ISO_LATIN_EXTRA
)
1577 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1578 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1579 & CODING_FLAG_ISO_LATIN_EXTRA
)
1580 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1583 mask_found
|= newmask
;
1590 single_shifting
= 0;
1595 single_shifting
= 0;
1596 if (VECTORP (Vlatin_extra_code_table
)
1597 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1601 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1602 & CODING_FLAG_ISO_LATIN_EXTRA
)
1603 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1604 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1605 & CODING_FLAG_ISO_LATIN_EXTRA
)
1606 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1608 mask_found
|= newmask
;
1615 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1616 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1617 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1618 /* Check the length of succeeding codes of the range
1619 0xA0..0FF. If the byte length is odd, we exclude
1620 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1621 when we are not single shifting. */
1622 if (!single_shifting
1623 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1628 while (src
< src_end
)
1630 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1636 if (i
& 1 && src
< src_end
)
1637 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1639 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1641 /* This means that we have read one extra byte. */
1649 return (mask
& mask_found
);
1652 /* Decode a character of which charset is CHARSET, the 1st position
1653 code is C1, the 2nd position code is C2, and return the decoded
1654 character code. If the variable `translation_table' is non-nil,
1655 returned the translated code. */
1657 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1658 (NILP (translation_table) \
1659 ? MAKE_CHAR (charset, c1, c2) \
1660 : translate_char (translation_table, -1, charset, c1, c2))
1662 /* Set designation state into CODING. */
1663 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1667 if (final_char < '0' || final_char >= 128) \
1668 goto label_invalid_code; \
1669 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1670 make_number (chars), \
1671 make_number (final_char)); \
1672 c = MAKE_CHAR (charset, 0, 0); \
1674 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1675 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1677 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1679 && charset == CHARSET_ASCII) \
1681 /* We should insert this designation sequence as is so \
1682 that it is surely written back to a file. */ \
1683 coding->spec.iso2022.last_invalid_designation_register = -1; \
1684 goto label_invalid_code; \
1686 coding->spec.iso2022.last_invalid_designation_register = -1; \
1687 if ((coding->mode & CODING_MODE_DIRECTION) \
1688 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1689 charset = CHARSET_REVERSE_CHARSET (charset); \
1690 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1694 coding->spec.iso2022.last_invalid_designation_register = reg; \
1695 goto label_invalid_code; \
1699 /* Allocate a memory block for storing information about compositions.
1700 The block is chained to the already allocated blocks. */
1703 coding_allocate_composition_data (coding
, char_offset
)
1704 struct coding_system
*coding
;
1707 struct composition_data
*cmp_data
1708 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1710 cmp_data
->char_offset
= char_offset
;
1712 cmp_data
->prev
= coding
->cmp_data
;
1713 cmp_data
->next
= NULL
;
1714 if (coding
->cmp_data
)
1715 coding
->cmp_data
->next
= cmp_data
;
1716 coding
->cmp_data
= cmp_data
;
1717 coding
->cmp_data_start
= 0;
1718 coding
->composing
= COMPOSITION_NO
;
1721 /* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1722 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1723 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1724 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1725 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1728 #define DECODE_COMPOSITION_START(c1) \
1730 if (coding->composing == COMPOSITION_DISABLED) \
1732 *dst++ = ISO_CODE_ESC; \
1733 *dst++ = c1 & 0x7f; \
1734 coding->produced_char += 2; \
1736 else if (!COMPOSING_P (coding)) \
1738 /* This is surely the start of a composition. We must be sure \
1739 that coding->cmp_data has enough space to store the \
1740 information about the composition. If not, terminate the \
1741 current decoding loop, allocate one more memory block for \
1742 coding->cmp_data in the caller, then start the decoding \
1743 loop again. We can't allocate memory here directly because \
1744 it may cause buffer/string relocation. */ \
1745 if (!coding->cmp_data \
1746 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1747 >= COMPOSITION_DATA_SIZE)) \
1749 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1750 goto label_end_of_loop; \
1752 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1753 : c1 == '2' ? COMPOSITION_WITH_RULE \
1754 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1755 : COMPOSITION_WITH_RULE_ALTCHARS); \
1756 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1757 coding->composing); \
1758 coding->composition_rule_follows = 0; \
1762 /* We are already handling a composition. If the method is \
1763 the following two, the codes following the current escape \
1764 sequence are actual characters stored in a buffer. */ \
1765 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1766 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1768 coding->composing = COMPOSITION_RELATIVE; \
1769 coding->composition_rule_follows = 0; \
1774 /* Handle composition end sequence ESC 1. */
1776 #define DECODE_COMPOSITION_END(c1) \
1778 if (! COMPOSING_P (coding)) \
1780 *dst++ = ISO_CODE_ESC; \
1782 coding->produced_char += 2; \
1786 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1787 coding->composing = COMPOSITION_NO; \
1791 /* Decode a composition rule from the byte C1 (and maybe one more byte
1792 from SRC) and store one encoded composition rule in
1793 coding->cmp_data. */
1795 #define DECODE_COMPOSITION_RULE(c1) \
1799 if (c1 < 81) /* old format (before ver.21) */ \
1801 int gref = (c1) / 9; \
1802 int nref = (c1) % 9; \
1803 if (gref == 4) gref = 10; \
1804 if (nref == 4) nref = 10; \
1805 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1807 else if (c1 < 93) /* new format (after ver.21) */ \
1809 ONE_MORE_BYTE (c2); \
1810 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1812 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1813 coding->composition_rule_follows = 0; \
1817 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1820 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1821 struct coding_system
*coding
;
1822 const unsigned char *source
;
1823 unsigned char *destination
;
1824 int src_bytes
, dst_bytes
;
1826 const unsigned char *src
= source
;
1827 const unsigned char *src_end
= source
+ src_bytes
;
1828 unsigned char *dst
= destination
;
1829 unsigned char *dst_end
= destination
+ dst_bytes
;
1830 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1831 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1832 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1833 /* SRC_BASE remembers the start position in source in each loop.
1834 The loop will be exited when there's not enough source code
1835 (within macro ONE_MORE_BYTE), or when there's not enough
1836 destination area to produce a character (within macro
1838 const unsigned char *src_base
;
1840 Lisp_Object translation_table
;
1841 Lisp_Object safe_chars
;
1843 safe_chars
= coding_safe_chars (coding
->symbol
);
1845 if (NILP (Venable_character_translation
))
1846 translation_table
= Qnil
;
1849 translation_table
= coding
->translation_table_for_decode
;
1850 if (NILP (translation_table
))
1851 translation_table
= Vstandard_translation_table_for_decode
;
1854 coding
->result
= CODING_FINISH_NORMAL
;
1863 /* We produce no character or one character. */
1864 switch (iso_code_class
[c1
])
1866 case ISO_0x20_or_0x7F
:
1867 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1869 DECODE_COMPOSITION_RULE (c1
);
1872 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1874 /* This is SPACE or DEL. */
1875 charset
= CHARSET_ASCII
;
1878 /* This is a graphic character, we fall down ... */
1880 case ISO_graphic_plane_0
:
1881 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1883 DECODE_COMPOSITION_RULE (c1
);
1889 case ISO_0xA0_or_0xFF
:
1890 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1891 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1892 goto label_invalid_code
;
1893 /* This is a graphic character, we fall down ... */
1895 case ISO_graphic_plane_1
:
1897 goto label_invalid_code
;
1902 if (COMPOSING_P (coding
))
1903 DECODE_COMPOSITION_END ('1');
1905 /* All ISO2022 control characters in this class have the
1906 same representation in Emacs internal format. */
1908 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1909 && (coding
->eol_type
== CODING_EOL_CR
1910 || coding
->eol_type
== CODING_EOL_CRLF
))
1912 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1913 goto label_end_of_loop
;
1915 charset
= CHARSET_ASCII
;
1919 if (COMPOSING_P (coding
))
1920 DECODE_COMPOSITION_END ('1');
1921 goto label_invalid_code
;
1923 case ISO_carriage_return
:
1924 if (COMPOSING_P (coding
))
1925 DECODE_COMPOSITION_END ('1');
1927 if (coding
->eol_type
== CODING_EOL_CR
)
1929 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1932 if (c1
!= ISO_CODE_LF
)
1938 charset
= CHARSET_ASCII
;
1942 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1943 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1944 goto label_invalid_code
;
1945 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1946 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1950 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1951 goto label_invalid_code
;
1952 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1953 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1956 case ISO_single_shift_2_7
:
1957 case ISO_single_shift_2
:
1958 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1959 goto label_invalid_code
;
1960 /* SS2 is handled as an escape sequence of ESC 'N' */
1962 goto label_escape_sequence
;
1964 case ISO_single_shift_3
:
1965 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1966 goto label_invalid_code
;
1967 /* SS2 is handled as an escape sequence of ESC 'O' */
1969 goto label_escape_sequence
;
1971 case ISO_control_sequence_introducer
:
1972 /* CSI is handled as an escape sequence of ESC '[' ... */
1974 goto label_escape_sequence
;
1978 label_escape_sequence
:
1979 /* Escape sequences handled by Emacs are invocation,
1980 designation, direction specification, and character
1981 composition specification. */
1984 case '&': /* revision of following character set */
1986 if (!(c1
>= '@' && c1
<= '~'))
1987 goto label_invalid_code
;
1989 if (c1
!= ISO_CODE_ESC
)
1990 goto label_invalid_code
;
1992 goto label_escape_sequence
;
1994 case '$': /* designation of 2-byte character set */
1995 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1996 goto label_invalid_code
;
1998 if (c1
>= '@' && c1
<= 'B')
1999 { /* designation of JISX0208.1978, GB2312.1980,
2001 DECODE_DESIGNATION (0, 2, 94, c1
);
2003 else if (c1
>= 0x28 && c1
<= 0x2B)
2004 { /* designation of DIMENSION2_CHARS94 character set */
2006 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
2008 else if (c1
>= 0x2C && c1
<= 0x2F)
2009 { /* designation of DIMENSION2_CHARS96 character set */
2011 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
2014 goto label_invalid_code
;
2015 /* We must update these variables now. */
2016 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2017 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2020 case 'n': /* invocation of locking-shift-2 */
2021 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2022 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2023 goto label_invalid_code
;
2024 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
2025 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2028 case 'o': /* invocation of locking-shift-3 */
2029 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2030 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2031 goto label_invalid_code
;
2032 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
2033 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2036 case 'N': /* invocation of single-shift-2 */
2037 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2038 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2039 goto label_invalid_code
;
2040 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
2042 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2043 goto label_invalid_code
;
2046 case 'O': /* invocation of single-shift-3 */
2047 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2048 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2049 goto label_invalid_code
;
2050 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
2052 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2053 goto label_invalid_code
;
2056 case '0': case '2': case '3': case '4': /* start composition */
2057 DECODE_COMPOSITION_START (c1
);
2060 case '1': /* end composition */
2061 DECODE_COMPOSITION_END (c1
);
2064 case '[': /* specification of direction */
2065 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
2066 goto label_invalid_code
;
2067 /* For the moment, nested direction is not supported.
2068 So, `coding->mode & CODING_MODE_DIRECTION' zero means
2069 left-to-right, and nonzero means right-to-left. */
2073 case ']': /* end of the current direction */
2074 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2076 case '0': /* end of the current direction */
2077 case '1': /* start of left-to-right direction */
2080 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2082 goto label_invalid_code
;
2085 case '2': /* start of right-to-left direction */
2088 coding
->mode
|= CODING_MODE_DIRECTION
;
2090 goto label_invalid_code
;
2094 goto label_invalid_code
;
2099 if (COMPOSING_P (coding
))
2100 DECODE_COMPOSITION_END ('1');
2104 /* CTEXT extended segment:
2105 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
2106 We keep these bytes as is for the moment.
2107 They may be decoded by post-read-conversion. */
2112 ONE_MORE_BYTE (dim
);
2115 size
= ((M
- 128) * 128) + (L
- 128);
2116 required
= 8 + size
* 2;
2117 if (dst
+ required
> (dst_bytes
? dst_end
: src
))
2118 goto label_end_of_loop
;
2119 *dst
++ = ISO_CODE_ESC
;
2124 dst
+= CHAR_STRING (M
, dst
), produced_chars
++;
2125 dst
+= CHAR_STRING (L
, dst
), produced_chars
++;
2129 dst
+= CHAR_STRING (c1
, dst
), produced_chars
++;
2131 coding
->produced_char
+= produced_chars
;
2135 unsigned char *d
= dst
;
2138 /* XFree86 extension for embedding UTF-8 in CTEXT:
2139 ESC % G --UTF-8-BYTES-- ESC % @
2140 We keep these bytes as is for the moment.
2141 They may be decoded by post-read-conversion. */
2142 if (d
+ 6 > (dst_bytes
? dst_end
: src
))
2143 goto label_end_of_loop
;
2144 *d
++ = ISO_CODE_ESC
;
2148 while (d
+ 1 < (dst_bytes
? dst_end
: src
))
2151 if (c1
== ISO_CODE_ESC
2152 && src
+ 1 < src_end
2159 d
+= CHAR_STRING (c1
, d
), produced_chars
++;
2161 if (d
+ 3 > (dst_bytes
? dst_end
: src
))
2162 goto label_end_of_loop
;
2163 *d
++ = ISO_CODE_ESC
;
2167 coding
->produced_char
+= produced_chars
+ 3;
2170 goto label_invalid_code
;
2174 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
2175 goto label_invalid_code
;
2176 if (c1
>= 0x28 && c1
<= 0x2B)
2177 { /* designation of DIMENSION1_CHARS94 character set */
2179 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
2181 else if (c1
>= 0x2C && c1
<= 0x2F)
2182 { /* designation of DIMENSION1_CHARS96 character set */
2184 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
2187 goto label_invalid_code
;
2188 /* We must update these variables now. */
2189 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2190 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2195 /* Now we know CHARSET and 1st position code C1 of a character.
2196 Produce a multibyte sequence for that character while getting
2197 2nd position code C2 if necessary. */
2198 if (CHARSET_DIMENSION (charset
) == 2)
2201 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
2202 /* C2 is not in a valid range. */
2203 goto label_invalid_code
;
2205 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2211 if (COMPOSING_P (coding
))
2212 DECODE_COMPOSITION_END ('1');
2215 if (! NILP (translation_table
))
2216 c
= translate_char (translation_table
, c
, 0, 0, 0);
2221 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2222 coding
->produced
= dst
- destination
;
2227 /* ISO2022 encoding stuff. */
2230 It is not enough to say just "ISO2022" on encoding, we have to
2231 specify more details. In Emacs, each ISO2022 coding system
2232 variant has the following specifications:
2233 1. Initial designation to G0 through G3.
2234 2. Allows short-form designation?
2235 3. ASCII should be designated to G0 before control characters?
2236 4. ASCII should be designated to G0 at end of line?
2237 5. 7-bit environment or 8-bit environment?
2238 6. Use locking-shift?
2239 7. Use Single-shift?
2240 And the following two are only for Japanese:
2241 8. Use ASCII in place of JIS0201-1976-Roman?
2242 9. Use JISX0208-1983 in place of JISX0208-1978?
2243 These specifications are encoded in `coding->flags' as flag bits
2244 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
2248 /* Produce codes (escape sequence) for designating CHARSET to graphic
2249 register REG at DST, and increment DST. If <final-char> of CHARSET is
2250 '@', 'A', or 'B' and the coding system CODING allows, produce
2251 designation sequence of short-form. */
2253 #define ENCODE_DESIGNATION(charset, reg, coding) \
2255 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2256 char *intermediate_char_94 = "()*+"; \
2257 char *intermediate_char_96 = ",-./"; \
2258 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
2260 if (revision < 255) \
2262 *dst++ = ISO_CODE_ESC; \
2264 *dst++ = '@' + revision; \
2266 *dst++ = ISO_CODE_ESC; \
2267 if (CHARSET_DIMENSION (charset) == 1) \
2269 if (CHARSET_CHARS (charset) == 94) \
2270 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2272 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2277 if (CHARSET_CHARS (charset) == 94) \
2279 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2281 || final_char < '@' || final_char > 'B') \
2282 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2285 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2287 *dst++ = final_char; \
2288 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2291 /* The following two macros produce codes (control character or escape
2292 sequence) for ISO2022 single-shift functions (single-shift-2 and
2295 #define ENCODE_SINGLE_SHIFT_2 \
2297 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2298 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2300 *dst++ = ISO_CODE_SS2; \
2301 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2304 #define ENCODE_SINGLE_SHIFT_3 \
2306 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2307 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2309 *dst++ = ISO_CODE_SS3; \
2310 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2313 /* The following four macros produce codes (control character or
2314 escape sequence) for ISO2022 locking-shift functions (shift-in,
2315 shift-out, locking-shift-2, and locking-shift-3). */
2317 #define ENCODE_SHIFT_IN \
2319 *dst++ = ISO_CODE_SI; \
2320 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2323 #define ENCODE_SHIFT_OUT \
2325 *dst++ = ISO_CODE_SO; \
2326 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2329 #define ENCODE_LOCKING_SHIFT_2 \
2331 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2332 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2335 #define ENCODE_LOCKING_SHIFT_3 \
2337 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
2338 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2341 /* Produce codes for a DIMENSION1 character whose character set is
2342 CHARSET and whose position-code is C1. Designation and invocation
2343 sequences are also produced in advance if necessary. */
2345 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2347 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2349 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2350 *dst++ = c1 & 0x7F; \
2352 *dst++ = c1 | 0x80; \
2353 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2356 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2358 *dst++ = c1 & 0x7F; \
2361 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2363 *dst++ = c1 | 0x80; \
2367 /* Since CHARSET is not yet invoked to any graphic planes, we \
2368 must invoke it, or, at first, designate it to some graphic \
2369 register. Then repeat the loop to actually produce the \
2371 dst = encode_invocation_designation (charset, coding, dst); \
2374 /* Produce codes for a DIMENSION2 character whose character set is
2375 CHARSET and whose position-codes are C1 and C2. Designation and
2376 invocation codes are also produced in advance if necessary. */
2378 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2380 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2382 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2383 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2385 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2386 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2389 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2391 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2394 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2396 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2400 /* Since CHARSET is not yet invoked to any graphic planes, we \
2401 must invoke it, or, at first, designate it to some graphic \
2402 register. Then repeat the loop to actually produce the \
2404 dst = encode_invocation_designation (charset, coding, dst); \
2407 #define ENCODE_ISO_CHARACTER(c) \
2409 int charset, c1, c2; \
2411 SPLIT_CHAR (c, charset, c1, c2); \
2412 if (CHARSET_DEFINED_P (charset)) \
2414 if (CHARSET_DIMENSION (charset) == 1) \
2416 if (charset == CHARSET_ASCII \
2417 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2418 charset = charset_latin_jisx0201; \
2419 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2423 if (charset == charset_jisx0208 \
2424 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2425 charset = charset_jisx0208_1978; \
2426 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2438 /* Instead of encoding character C, produce one or two `?'s. */
2440 #define ENCODE_UNSAFE_CHARACTER(c) \
2442 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2443 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2444 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2448 /* Produce designation and invocation codes at a place pointed by DST
2449 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2453 encode_invocation_designation (charset
, coding
, dst
)
2455 struct coding_system
*coding
;
2458 int reg
; /* graphic register number */
2460 /* At first, check designations. */
2461 for (reg
= 0; reg
< 4; reg
++)
2462 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
2467 /* CHARSET is not yet designated to any graphic registers. */
2468 /* At first check the requested designation. */
2469 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2470 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
2471 /* Since CHARSET requests no special designation, designate it
2472 to graphic register 0. */
2475 ENCODE_DESIGNATION (charset
, reg
, coding
);
2478 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
2479 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
2481 /* Since the graphic register REG is not invoked to any graphic
2482 planes, invoke it to graphic plane 0. */
2485 case 0: /* graphic register 0 */
2489 case 1: /* graphic register 1 */
2493 case 2: /* graphic register 2 */
2494 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2495 ENCODE_SINGLE_SHIFT_2
;
2497 ENCODE_LOCKING_SHIFT_2
;
2500 case 3: /* graphic register 3 */
2501 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2502 ENCODE_SINGLE_SHIFT_3
;
2504 ENCODE_LOCKING_SHIFT_3
;
2512 /* Produce 2-byte codes for encoded composition rule RULE. */
2514 #define ENCODE_COMPOSITION_RULE(rule) \
2517 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2518 *dst++ = 32 + 81 + gref; \
2519 *dst++ = 32 + nref; \
2522 /* Produce codes for indicating the start of a composition sequence
2523 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2524 which specify information about the composition. See the comment
2525 in coding.h for the format of DATA. */
2527 #define ENCODE_COMPOSITION_START(coding, data) \
2529 coding->composing = data[3]; \
2530 *dst++ = ISO_CODE_ESC; \
2531 if (coding->composing == COMPOSITION_RELATIVE) \
2535 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2537 coding->cmp_data_index = coding->cmp_data_start + 4; \
2538 coding->composition_rule_follows = 0; \
2542 /* Produce codes for indicating the end of the current composition. */
2544 #define ENCODE_COMPOSITION_END(coding, data) \
2546 *dst++ = ISO_CODE_ESC; \
2548 coding->cmp_data_start += data[0]; \
2549 coding->composing = COMPOSITION_NO; \
2550 if (coding->cmp_data_start == coding->cmp_data->used \
2551 && coding->cmp_data->next) \
2553 coding->cmp_data = coding->cmp_data->next; \
2554 coding->cmp_data_start = 0; \
2558 /* Produce composition start sequence ESC 0. Here, this sequence
2559 doesn't mean the start of a new composition but means that we have
2560 just produced components (alternate chars and composition rules) of
2561 the composition and the actual text follows in SRC. */
2563 #define ENCODE_COMPOSITION_FAKE_START(coding) \
2565 *dst++ = ISO_CODE_ESC; \
2567 coding->composing = COMPOSITION_RELATIVE; \
2570 /* The following three macros produce codes for indicating direction
2572 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2574 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2575 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2577 *dst++ = ISO_CODE_CSI; \
2580 #define ENCODE_DIRECTION_R2L \
2581 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2583 #define ENCODE_DIRECTION_L2R \
2584 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2586 /* Produce codes for designation and invocation to reset the graphic
2587 planes and registers to initial state. */
2588 #define ENCODE_RESET_PLANE_AND_REGISTER \
2591 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2593 for (reg = 0; reg < 4; reg++) \
2594 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2595 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2596 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2597 ENCODE_DESIGNATION \
2598 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
2601 /* Produce designation sequences of charsets in the line started from
2602 SRC to a place pointed by DST, and return updated DST.
2604 If the current block ends before any end-of-line, we may fail to
2605 find all the necessary designations. */
2607 static unsigned char *
2608 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
2609 struct coding_system
*coding
;
2610 Lisp_Object translation_table
;
2611 const unsigned char *src
, *src_end
;
2614 int charset
, c
, found
= 0, reg
;
2615 /* Table of charsets to be designated to each graphic register. */
2618 for (reg
= 0; reg
< 4; reg
++)
2627 charset
= CHAR_CHARSET (c
);
2628 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2629 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
2639 for (reg
= 0; reg
< 4; reg
++)
2641 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
2642 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2648 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2651 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2652 struct coding_system
*coding
;
2653 const unsigned char *source
;
2654 unsigned char *destination
;
2655 int src_bytes
, dst_bytes
;
2657 const unsigned char *src
= source
;
2658 const unsigned char *src_end
= source
+ src_bytes
;
2659 unsigned char *dst
= destination
;
2660 unsigned char *dst_end
= destination
+ dst_bytes
;
2661 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2662 from DST_END to assure overflow checking is necessary only at the
2664 unsigned char *adjusted_dst_end
= dst_end
- 19;
2665 /* SRC_BASE remembers the start position in source in each loop.
2666 The loop will be exited when there's not enough source text to
2667 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2668 there's not enough destination area to produce encoded codes
2669 (within macro EMIT_BYTES). */
2670 const unsigned char *src_base
;
2672 Lisp_Object translation_table
;
2673 Lisp_Object safe_chars
;
2675 if (coding
->flags
& CODING_FLAG_ISO_SAFE
)
2676 coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
2678 safe_chars
= coding_safe_chars (coding
->symbol
);
2680 if (NILP (Venable_character_translation
))
2681 translation_table
= Qnil
;
2684 translation_table
= coding
->translation_table_for_encode
;
2685 if (NILP (translation_table
))
2686 translation_table
= Vstandard_translation_table_for_encode
;
2689 coding
->consumed_char
= 0;
2695 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2697 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2701 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2702 && CODING_SPEC_ISO_BOL (coding
))
2704 /* We have to produce designation sequences if any now. */
2705 dst
= encode_designation_at_bol (coding
, translation_table
,
2707 CODING_SPEC_ISO_BOL (coding
) = 0;
2710 /* Check composition start and end. */
2711 if (coding
->composing
!= COMPOSITION_DISABLED
2712 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2714 struct composition_data
*cmp_data
= coding
->cmp_data
;
2715 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2716 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2718 if (coding
->composing
== COMPOSITION_RELATIVE
)
2720 if (this_pos
== data
[2])
2722 ENCODE_COMPOSITION_END (coding
, data
);
2723 cmp_data
= coding
->cmp_data
;
2724 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2727 else if (COMPOSING_P (coding
))
2729 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2730 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2731 /* We have consumed components of the composition.
2732 What follows in SRC is the composition's base
2734 ENCODE_COMPOSITION_FAKE_START (coding
);
2737 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2738 if (coding
->composition_rule_follows
)
2740 ENCODE_COMPOSITION_RULE (c
);
2741 coding
->composition_rule_follows
= 0;
2745 if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2746 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2747 ENCODE_UNSAFE_CHARACTER (c
);
2749 ENCODE_ISO_CHARACTER (c
);
2750 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2751 coding
->composition_rule_follows
= 1;
2756 if (!COMPOSING_P (coding
))
2758 if (this_pos
== data
[1])
2760 ENCODE_COMPOSITION_START (coding
, data
);
2768 /* Now encode the character C. */
2769 if (c
< 0x20 || c
== 0x7F)
2773 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2775 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2776 ENCODE_RESET_PLANE_AND_REGISTER
;
2780 /* fall down to treat '\r' as '\n' ... */
2785 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2786 ENCODE_RESET_PLANE_AND_REGISTER
;
2787 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2788 bcopy (coding
->spec
.iso2022
.initial_designation
,
2789 coding
->spec
.iso2022
.current_designation
,
2790 sizeof coding
->spec
.iso2022
.initial_designation
);
2791 if (coding
->eol_type
== CODING_EOL_LF
2792 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2793 *dst
++ = ISO_CODE_LF
;
2794 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2795 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2797 *dst
++ = ISO_CODE_CR
;
2798 CODING_SPEC_ISO_BOL (coding
) = 1;
2802 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2803 ENCODE_RESET_PLANE_AND_REGISTER
;
2807 else if (ASCII_BYTE_P (c
))
2808 ENCODE_ISO_CHARACTER (c
);
2809 else if (SINGLE_BYTE_CHAR_P (c
))
2814 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2815 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2816 ENCODE_UNSAFE_CHARACTER (c
);
2818 ENCODE_ISO_CHARACTER (c
);
2820 coding
->consumed_char
++;
2824 coding
->consumed
= src_base
- source
;
2825 coding
->produced
= coding
->produced_char
= dst
- destination
;
2829 /*** 4. SJIS and BIG5 handlers ***/
2831 /* Although SJIS and BIG5 are not ISO coding systems, they are used
2832 quite widely. So, for the moment, Emacs supports them in the bare
2833 C code. But, in the future, they may be supported only by CCL. */
2835 /* SJIS is a coding system encoding three character sets: ASCII, right
2836 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2837 as is. A character of charset katakana-jisx0201 is encoded by
2838 "position-code + 0x80". A character of charset japanese-jisx0208
2839 is encoded in 2-byte but two position-codes are divided and shifted
2840 so that it fits in the range below.
2842 --- CODE RANGE of SJIS ---
2843 (character set) (range)
2845 KATAKANA-JISX0201 0xA1 .. 0xDF
2846 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2847 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2848 -------------------------------
2852 /* BIG5 is a coding system encoding two character sets: ASCII and
2853 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2854 character set and is encoded in two bytes.
2856 --- CODE RANGE of BIG5 ---
2857 (character set) (range)
2859 Big5 (1st byte) 0xA1 .. 0xFE
2860 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2861 --------------------------
2863 Since the number of characters in Big5 is larger than maximum
2864 characters in Emacs' charset (96x96), it can't be handled as one
2865 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2866 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2867 contains frequently used characters and the latter contains less
2868 frequently used characters. */
2870 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2871 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2872 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
2873 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2875 /* Number of Big5 characters which have the same code in 1st byte. */
2876 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2878 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2881 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2883 charset = charset_big5_1; \
2886 charset = charset_big5_2; \
2887 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2889 c1 = temp / (0xFF - 0xA1) + 0x21; \
2890 c2 = temp % (0xFF - 0xA1) + 0x21; \
2893 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2895 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2896 if (charset == charset_big5_2) \
2897 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2898 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2899 b2 = temp % BIG5_SAME_ROW; \
2900 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2903 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2904 Check if a text is encoded in SJIS. If it is, return
2905 CODING_CATEGORY_MASK_SJIS, else return 0. */
2908 detect_coding_sjis (src
, src_end
, multibytep
)
2909 unsigned char *src
, *src_end
;
2913 /* Dummy for ONE_MORE_BYTE. */
2914 struct coding_system dummy_coding
;
2915 struct coding_system
*coding
= &dummy_coding
;
2919 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2922 if (c
== 0x80 || c
== 0xA0 || c
> 0xEF)
2924 if (c
<= 0x9F || c
>= 0xE0)
2926 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2927 if (c
< 0x40 || c
== 0x7F || c
> 0xFC)
2932 return CODING_CATEGORY_MASK_SJIS
;
2935 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2936 Check if a text is encoded in BIG5. If it is, return
2937 CODING_CATEGORY_MASK_BIG5, else return 0. */
2940 detect_coding_big5 (src
, src_end
, multibytep
)
2941 unsigned char *src
, *src_end
;
2945 /* Dummy for ONE_MORE_BYTE. */
2946 struct coding_system dummy_coding
;
2947 struct coding_system
*coding
= &dummy_coding
;
2951 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2954 if (c
< 0xA1 || c
> 0xFE)
2956 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2957 if (c
< 0x40 || (c
> 0x7F && c
< 0xA1) || c
> 0xFE)
2961 return CODING_CATEGORY_MASK_BIG5
;
2964 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2965 Check if a text is encoded in UTF-8. If it is, return
2966 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2968 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2969 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2970 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2971 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2972 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2973 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2974 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2977 detect_coding_utf_8 (src
, src_end
, multibytep
)
2978 unsigned char *src
, *src_end
;
2982 int seq_maybe_bytes
;
2983 /* Dummy for ONE_MORE_BYTE. */
2984 struct coding_system dummy_coding
;
2985 struct coding_system
*coding
= &dummy_coding
;
2989 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2990 if (UTF_8_1_OCTET_P (c
))
2992 else if (UTF_8_2_OCTET_LEADING_P (c
))
2993 seq_maybe_bytes
= 1;
2994 else if (UTF_8_3_OCTET_LEADING_P (c
))
2995 seq_maybe_bytes
= 2;
2996 else if (UTF_8_4_OCTET_LEADING_P (c
))
2997 seq_maybe_bytes
= 3;
2998 else if (UTF_8_5_OCTET_LEADING_P (c
))
2999 seq_maybe_bytes
= 4;
3000 else if (UTF_8_6_OCTET_LEADING_P (c
))
3001 seq_maybe_bytes
= 5;
3007 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3008 if (!UTF_8_EXTRA_OCTET_P (c
))
3012 while (seq_maybe_bytes
> 0);
3016 return CODING_CATEGORY_MASK_UTF_8
;
3019 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3020 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
3021 Little Endian (otherwise). If it is, return
3022 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
3025 #define UTF_16_INVALID_P(val) \
3026 (((val) == 0xFFFE) \
3027 || ((val) == 0xFFFF))
3029 #define UTF_16_HIGH_SURROGATE_P(val) \
3030 (((val) & 0xD800) == 0xD800)
3032 #define UTF_16_LOW_SURROGATE_P(val) \
3033 (((val) & 0xDC00) == 0xDC00)
3036 detect_coding_utf_16 (src
, src_end
, multibytep
)
3037 unsigned char *src
, *src_end
;
3040 unsigned char c1
, c2
;
3041 /* Dummy for ONE_MORE_BYTE_CHECK_MULTIBYTE. */
3042 struct coding_system dummy_coding
;
3043 struct coding_system
*coding
= &dummy_coding
;
3045 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
3046 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2
, multibytep
);
3048 if ((c1
== 0xFF) && (c2
== 0xFE))
3049 return CODING_CATEGORY_MASK_UTF_16_LE
;
3050 else if ((c1
== 0xFE) && (c2
== 0xFF))
3051 return CODING_CATEGORY_MASK_UTF_16_BE
;
3057 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
3058 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
3061 decode_coding_sjis_big5 (coding
, source
, destination
,
3062 src_bytes
, dst_bytes
, sjis_p
)
3063 struct coding_system
*coding
;
3064 const unsigned char *source
;
3065 unsigned char *destination
;
3066 int src_bytes
, dst_bytes
;
3069 const unsigned char *src
= source
;
3070 const unsigned char *src_end
= source
+ src_bytes
;
3071 unsigned char *dst
= destination
;
3072 unsigned char *dst_end
= destination
+ dst_bytes
;
3073 /* SRC_BASE remembers the start position in source in each loop.
3074 The loop will be exited when there's not enough source code
3075 (within macro ONE_MORE_BYTE), or when there's not enough
3076 destination area to produce a character (within macro
3078 const unsigned char *src_base
;
3079 Lisp_Object translation_table
;
3081 if (NILP (Venable_character_translation
))
3082 translation_table
= Qnil
;
3085 translation_table
= coding
->translation_table_for_decode
;
3086 if (NILP (translation_table
))
3087 translation_table
= Vstandard_translation_table_for_decode
;
3090 coding
->produced_char
= 0;
3093 int c
, charset
, c1
, c2
= 0;
3100 charset
= CHARSET_ASCII
;
3105 if (coding
->eol_type
== CODING_EOL_CRLF
)
3111 /* To process C2 again, SRC is subtracted by 1. */
3114 else if (coding
->eol_type
== CODING_EOL_CR
)
3118 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3119 && (coding
->eol_type
== CODING_EOL_CR
3120 || coding
->eol_type
== CODING_EOL_CRLF
))
3122 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3123 goto label_end_of_loop
;
3131 if (c1
== 0x80 || c1
== 0xA0 || c1
> 0xEF)
3132 goto label_invalid_code
;
3133 if (c1
<= 0x9F || c1
>= 0xE0)
3135 /* SJIS -> JISX0208 */
3137 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
3138 goto label_invalid_code
;
3139 DECODE_SJIS (c1
, c2
, c1
, c2
);
3140 charset
= charset_jisx0208
;
3143 /* SJIS -> JISX0201-Kana */
3144 charset
= charset_katakana_jisx0201
;
3149 if (c1
< 0xA0 || c1
> 0xFE)
3150 goto label_invalid_code
;
3152 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
3153 goto label_invalid_code
;
3154 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
3158 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
3170 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3171 coding
->produced
= dst
- destination
;
3175 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
3176 This function can encode charsets `ascii', `katakana-jisx0201',
3177 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
3178 are sure that all these charsets are registered as official charset
3179 (i.e. do not have extended leading-codes). Characters of other
3180 charsets are produced without any encoding. If SJIS_P is 1, encode
3181 SJIS text, else encode BIG5 text. */
3184 encode_coding_sjis_big5 (coding
, source
, destination
,
3185 src_bytes
, dst_bytes
, sjis_p
)
3186 struct coding_system
*coding
;
3187 unsigned char *source
, *destination
;
3188 int src_bytes
, dst_bytes
;
3191 unsigned char *src
= source
;
3192 unsigned char *src_end
= source
+ src_bytes
;
3193 unsigned char *dst
= destination
;
3194 unsigned char *dst_end
= destination
+ dst_bytes
;
3195 /* SRC_BASE remembers the start position in source in each loop.
3196 The loop will be exited when there's not enough source text to
3197 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3198 there's not enough destination area to produce encoded codes
3199 (within macro EMIT_BYTES). */
3200 unsigned char *src_base
;
3201 Lisp_Object translation_table
;
3203 if (NILP (Venable_character_translation
))
3204 translation_table
= Qnil
;
3207 translation_table
= coding
->translation_table_for_encode
;
3208 if (NILP (translation_table
))
3209 translation_table
= Vstandard_translation_table_for_encode
;
3214 int c
, charset
, c1
, c2
;
3219 /* Now encode the character C. */
3220 if (SINGLE_BYTE_CHAR_P (c
))
3225 if (!(coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
3232 if (coding
->eol_type
== CODING_EOL_CRLF
)
3234 EMIT_TWO_BYTES ('\r', c
);
3237 else if (coding
->eol_type
== CODING_EOL_CR
)
3245 SPLIT_CHAR (c
, charset
, c1
, c2
);
3248 if (charset
== charset_jisx0208
3249 || charset
== charset_jisx0208_1978
)
3251 ENCODE_SJIS (c1
, c2
, c1
, c2
);
3252 EMIT_TWO_BYTES (c1
, c2
);
3254 else if (charset
== charset_katakana_jisx0201
)
3255 EMIT_ONE_BYTE (c1
| 0x80);
3256 else if (charset
== charset_latin_jisx0201
)
3258 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3260 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3261 if (CHARSET_WIDTH (charset
) > 1)
3262 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3265 /* There's no way other than producing the internal
3267 EMIT_BYTES (src_base
, src
);
3271 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
3273 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
3274 EMIT_TWO_BYTES (c1
, c2
);
3276 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3278 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3279 if (CHARSET_WIDTH (charset
) > 1)
3280 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3283 /* There's no way other than producing the internal
3285 EMIT_BYTES (src_base
, src
);
3288 coding
->consumed_char
++;
3292 coding
->consumed
= src_base
- source
;
3293 coding
->produced
= coding
->produced_char
= dst
- destination
;
3297 /*** 5. CCL handlers ***/
3299 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3300 Check if a text is encoded in a coding system of which
3301 encoder/decoder are written in CCL program. If it is, return
3302 CODING_CATEGORY_MASK_CCL, else return 0. */
3305 detect_coding_ccl (src
, src_end
, multibytep
)
3306 unsigned char *src
, *src_end
;
3309 unsigned char *valid
;
3311 /* Dummy for ONE_MORE_BYTE. */
3312 struct coding_system dummy_coding
;
3313 struct coding_system
*coding
= &dummy_coding
;
3315 /* No coding system is assigned to coding-category-ccl. */
3316 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
3319 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
3322 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3327 return CODING_CATEGORY_MASK_CCL
;
3331 /*** 6. End-of-line handlers ***/
3333 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3336 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3337 struct coding_system
*coding
;
3338 const unsigned char *source
;
3339 unsigned char *destination
;
3340 int src_bytes
, dst_bytes
;
3342 const unsigned char *src
= source
;
3343 unsigned char *dst
= destination
;
3344 const unsigned char *src_end
= src
+ src_bytes
;
3345 unsigned char *dst_end
= dst
+ dst_bytes
;
3346 Lisp_Object translation_table
;
3347 /* SRC_BASE remembers the start position in source in each loop.
3348 The loop will be exited when there's not enough source code
3349 (within macro ONE_MORE_BYTE), or when there's not enough
3350 destination area to produce a character (within macro
3352 const unsigned char *src_base
;
3355 translation_table
= Qnil
;
3356 switch (coding
->eol_type
)
3358 case CODING_EOL_CRLF
:
3373 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
3375 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3376 goto label_end_of_loop
;
3389 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3391 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3392 goto label_end_of_loop
;
3401 default: /* no need for EOL handling */
3411 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3412 coding
->produced
= dst
- destination
;
3416 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
3417 format of end-of-line according to `coding->eol_type'. It also
3418 convert multibyte form 8-bit characters to unibyte if
3419 CODING->src_multibyte is nonzero. If `coding->mode &
3420 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3421 also means end-of-line. */
3424 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3425 struct coding_system
*coding
;
3426 const unsigned char *source
;
3427 unsigned char *destination
;
3428 int src_bytes
, dst_bytes
;
3430 const unsigned char *src
= source
;
3431 unsigned char *dst
= destination
;
3432 const unsigned char *src_end
= src
+ src_bytes
;
3433 unsigned char *dst_end
= dst
+ dst_bytes
;
3434 Lisp_Object translation_table
;
3435 /* SRC_BASE remembers the start position in source in each loop.
3436 The loop will be exited when there's not enough source text to
3437 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3438 there's not enough destination area to produce encoded codes
3439 (within macro EMIT_BYTES). */
3440 const unsigned char *src_base
;
3443 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
3445 translation_table
= Qnil
;
3446 if (coding
->src_multibyte
3447 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3451 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
3454 if (coding
->eol_type
== CODING_EOL_CRLF
)
3456 while (src
< src_end
)
3462 else if (c
== '\n' || (c
== '\r' && selective_display
))
3463 EMIT_TWO_BYTES ('\r', '\n');
3473 if (!dst_bytes
|| src_bytes
<= dst_bytes
)
3475 safe_bcopy (src
, dst
, src_bytes
);
3481 if (coding
->src_multibyte
3482 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3484 safe_bcopy (src
, dst
, dst_bytes
);
3485 src_base
= src
+ dst_bytes
;
3486 dst
= destination
+ dst_bytes
;
3487 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
3489 if (coding
->eol_type
== CODING_EOL_CR
)
3491 for (tmp
= destination
; tmp
< dst
; tmp
++)
3492 if (*tmp
== '\n') *tmp
= '\r';
3494 else if (selective_display
)
3496 for (tmp
= destination
; tmp
< dst
; tmp
++)
3497 if (*tmp
== '\r') *tmp
= '\n';
3500 if (coding
->src_multibyte
)
3501 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
3503 coding
->consumed
= src_base
- source
;
3504 coding
->produced
= dst
- destination
;
3505 coding
->produced_char
= coding
->produced
;
3509 /*** 7. C library functions ***/
3511 /* In Emacs Lisp, a coding system is represented by a Lisp symbol which
3512 has a property `coding-system'. The value of this property is a
3513 vector of length 5 (called the coding-vector). Among elements of
3514 this vector, the first (element[0]) and the fifth (element[4])
3515 carry important information for decoding/encoding. Before
3516 decoding/encoding, this information should be set in fields of a
3517 structure of type `coding_system'.
3519 The value of the property `coding-system' can be a symbol of another
3520 subsidiary coding-system. In that case, Emacs gets coding-vector
3523 `element[0]' contains information to be set in `coding->type'. The
3524 value and its meaning is as follows:
3526 0 -- coding_type_emacs_mule
3527 1 -- coding_type_sjis
3528 2 -- coding_type_iso2022
3529 3 -- coding_type_big5
3530 4 -- coding_type_ccl encoder/decoder written in CCL
3531 nil -- coding_type_no_conversion
3532 t -- coding_type_undecided (automatic conversion on decoding,
3533 no-conversion on encoding)
3535 `element[4]' contains information to be set in `coding->flags' and
3536 `coding->spec'. The meaning varies by `coding->type'.
3538 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3539 of length 32 (of which the first 13 sub-elements are used now).
3540 Meanings of these sub-elements are:
3542 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3543 If the value is an integer of valid charset, the charset is
3544 assumed to be designated to graphic register N initially.
3546 If the value is minus, it is a minus value of charset which
3547 reserves graphic register N, which means that the charset is
3548 not designated initially but should be designated to graphic
3549 register N just before encoding a character in that charset.
3551 If the value is nil, graphic register N is never used on
3554 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3555 Each value takes t or nil. See the section ISO2022 of
3556 `coding.h' for more information.
3558 If `coding->type' is `coding_type_big5', element[4] is t to denote
3559 BIG5-ETen or nil to denote BIG5-HKU.
3561 If `coding->type' takes the other value, element[4] is ignored.
3563 Emacs Lisp's coding systems also carry information about format of
3564 end-of-line in a value of property `eol-type'. If the value is
3565 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3566 means CODING_EOL_CR. If it is not integer, it should be a vector
3567 of subsidiary coding systems of which property `eol-type' has one
3568 of the above values.
3572 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3573 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3574 is setup so that no conversion is necessary and return -1, else
3578 setup_coding_system (coding_system
, coding
)
3579 Lisp_Object coding_system
;
3580 struct coding_system
*coding
;
3582 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
3585 /* At first, zero clear all members. */
3586 bzero (coding
, sizeof (struct coding_system
));
3588 /* Initialize some fields required for all kinds of coding systems. */
3589 coding
->symbol
= coding_system
;
3590 coding
->heading_ascii
= -1;
3591 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
3592 coding
->composing
= COMPOSITION_DISABLED
;
3593 coding
->cmp_data
= NULL
;
3595 if (NILP (coding_system
))
3596 goto label_invalid_coding_system
;
3598 coding_spec
= Fget (coding_system
, Qcoding_system
);
3600 if (!VECTORP (coding_spec
)
3601 || XVECTOR (coding_spec
)->size
!= 5
3602 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
3603 goto label_invalid_coding_system
;
3605 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
3606 if (VECTORP (eol_type
))
3608 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3609 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
3610 if (system_eol_type
!= CODING_EOL_LF
)
3611 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3613 else if (XFASTINT (eol_type
) == 1)
3615 coding
->eol_type
= CODING_EOL_CRLF
;
3616 coding
->common_flags
3617 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3619 else if (XFASTINT (eol_type
) == 2)
3621 coding
->eol_type
= CODING_EOL_CR
;
3622 coding
->common_flags
3623 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3626 coding
->eol_type
= CODING_EOL_LF
;
3628 coding_type
= XVECTOR (coding_spec
)->contents
[0];
3629 /* Try short cut. */
3630 if (SYMBOLP (coding_type
))
3632 if (EQ (coding_type
, Qt
))
3634 coding
->type
= coding_type_undecided
;
3635 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
3638 coding
->type
= coding_type_no_conversion
;
3639 /* Initialize this member. Any thing other than
3640 CODING_CATEGORY_IDX_UTF_16_BE and
3641 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3642 special treatment in detect_eol. */
3643 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
3648 /* Get values of coding system properties:
3649 `post-read-conversion', `pre-write-conversion',
3650 `translation-table-for-decode', `translation-table-for-encode'. */
3651 plist
= XVECTOR (coding_spec
)->contents
[3];
3652 /* Pre & post conversion functions should be disabled if
3653 inhibit_eol_conversion is nonzero. This is the case that a code
3654 conversion function is called while those functions are running. */
3655 if (! inhibit_pre_post_conversion
)
3657 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
3658 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
3660 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
3662 val
= Fget (val
, Qtranslation_table_for_decode
);
3663 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3664 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
3666 val
= Fget (val
, Qtranslation_table_for_encode
);
3667 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3668 val
= Fplist_get (plist
, Qcoding_category
);
3671 val
= Fget (val
, Qcoding_category_index
);
3673 coding
->category_idx
= XINT (val
);
3675 goto label_invalid_coding_system
;
3678 goto label_invalid_coding_system
;
3680 /* If the coding system has non-nil `composition' property, enable
3681 composition handling. */
3682 val
= Fplist_get (plist
, Qcomposition
);
3684 coding
->composing
= COMPOSITION_NO
;
3686 switch (XFASTINT (coding_type
))
3689 coding
->type
= coding_type_emacs_mule
;
3690 coding
->common_flags
3691 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3692 if (!NILP (coding
->post_read_conversion
))
3693 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3694 if (!NILP (coding
->pre_write_conversion
))
3695 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3699 coding
->type
= coding_type_sjis
;
3700 coding
->common_flags
3701 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3705 coding
->type
= coding_type_iso2022
;
3706 coding
->common_flags
3707 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3709 Lisp_Object val
, temp
;
3711 int i
, charset
, reg_bits
= 0;
3713 val
= XVECTOR (coding_spec
)->contents
[4];
3715 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3716 goto label_invalid_coding_system
;
3718 flags
= XVECTOR (val
)->contents
;
3720 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3721 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3722 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3723 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3724 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3725 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3726 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3727 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3728 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3729 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3730 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3731 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3732 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3735 /* Invoke graphic register 0 to plane 0. */
3736 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3737 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3738 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3739 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3740 /* Not single shifting at first. */
3741 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3742 /* Beginning of buffer should also be regarded as bol. */
3743 CODING_SPEC_ISO_BOL (coding
) = 1;
3745 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3746 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3747 val
= Vcharset_revision_alist
;
3750 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3752 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3753 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3754 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3758 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3759 FLAGS[REG] can be one of below:
3760 integer CHARSET: CHARSET occupies register I,
3761 t: designate nothing to REG initially, but can be used
3763 list of integer, nil, or t: designate the first
3764 element (if integer) to REG initially, the remaining
3765 elements (if integer) is designated to REG on request,
3766 if an element is t, REG can be used by any charsets,
3767 nil: REG is never used. */
3768 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3769 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3770 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3771 for (i
= 0; i
< 4; i
++)
3773 if ((INTEGERP (flags
[i
])
3774 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
)))
3775 || (charset
= get_charset_id (flags
[i
])) >= 0)
3777 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3778 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3780 else if (EQ (flags
[i
], Qt
))
3782 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3784 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3786 else if (CONSP (flags
[i
]))
3791 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3792 if ((INTEGERP (XCAR (tail
))
3793 && (charset
= XINT (XCAR (tail
)),
3794 CHARSET_VALID_P (charset
)))
3795 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3797 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3798 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3801 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3803 while (CONSP (tail
))
3805 if ((INTEGERP (XCAR (tail
))
3806 && (charset
= XINT (XCAR (tail
)),
3807 CHARSET_VALID_P (charset
)))
3808 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3809 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3811 else if (EQ (XCAR (tail
), Qt
))
3817 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3819 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3820 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3823 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3825 /* REG 1 can be used only by locking shift in 7-bit env. */
3826 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3828 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3829 /* Without any shifting, only REG 0 and 1 can be used. */
3834 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3836 if (CHARSET_DEFINED_P (charset
)
3837 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3838 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3840 /* There exist some default graphic registers to be
3843 /* We had better avoid designating a charset of
3844 CHARS96 to REG 0 as far as possible. */
3845 if (CHARSET_CHARS (charset
) == 96)
3846 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3848 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3850 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3852 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3856 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3857 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3861 coding
->type
= coding_type_big5
;
3862 coding
->common_flags
3863 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3865 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3866 ? CODING_FLAG_BIG5_HKU
3867 : CODING_FLAG_BIG5_ETEN
);
3871 coding
->type
= coding_type_ccl
;
3872 coding
->common_flags
3873 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3875 val
= XVECTOR (coding_spec
)->contents
[4];
3877 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3879 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3881 goto label_invalid_coding_system
;
3883 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3884 val
= Fplist_get (plist
, Qvalid_codes
);
3889 for (; CONSP (val
); val
= XCDR (val
))
3893 && XINT (this) >= 0 && XINT (this) < 256)
3894 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3895 else if (CONSP (this)
3896 && INTEGERP (XCAR (this))
3897 && INTEGERP (XCDR (this)))
3899 int start
= XINT (XCAR (this));
3900 int end
= XINT (XCDR (this));
3902 if (start
>= 0 && start
<= end
&& end
< 256)
3903 while (start
<= end
)
3904 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3909 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3910 coding
->spec
.ccl
.cr_carryover
= 0;
3911 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
3915 coding
->type
= coding_type_raw_text
;
3919 goto label_invalid_coding_system
;
3923 label_invalid_coding_system
:
3924 coding
->type
= coding_type_no_conversion
;
3925 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3926 coding
->common_flags
= 0;
3927 coding
->eol_type
= NILP (coding_system
) ? system_eol_type
: CODING_EOL_LF
;
3928 if (coding
->eol_type
!= CODING_EOL_LF
)
3929 coding
->common_flags
3930 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3931 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3932 return NILP (coding_system
) ? 0 : -1;
3935 /* Free memory blocks allocated for storing composition information. */
3938 coding_free_composition_data (coding
)
3939 struct coding_system
*coding
;
3941 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3945 /* Memory blocks are chained. At first, rewind to the first, then,
3946 free blocks one by one. */
3947 while (cmp_data
->prev
)
3948 cmp_data
= cmp_data
->prev
;
3951 next
= cmp_data
->next
;
3955 coding
->cmp_data
= NULL
;
3958 /* Set `char_offset' member of all memory blocks pointed by
3959 coding->cmp_data to POS. */
3962 coding_adjust_composition_offset (coding
, pos
)
3963 struct coding_system
*coding
;
3966 struct composition_data
*cmp_data
;
3968 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3969 cmp_data
->char_offset
= pos
;
3972 /* Setup raw-text or one of its subsidiaries in the structure
3973 coding_system CODING according to the already setup value eol_type
3974 in CODING. CODING should be setup for some coding system in
3978 setup_raw_text_coding_system (coding
)
3979 struct coding_system
*coding
;
3981 if (coding
->type
!= coding_type_raw_text
)
3983 coding
->symbol
= Qraw_text
;
3984 coding
->type
= coding_type_raw_text
;
3985 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3987 Lisp_Object subsidiaries
;
3988 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3990 if (VECTORP (subsidiaries
)
3991 && XVECTOR (subsidiaries
)->size
== 3)
3993 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3995 setup_coding_system (coding
->symbol
, coding
);
4000 /* Emacs has a mechanism to automatically detect a coding system if it
4001 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
4002 it's impossible to distinguish some coding systems accurately
4003 because they use the same range of codes. So, at first, coding
4004 systems are categorized into 7, those are:
4006 o coding-category-emacs-mule
4008 The category for a coding system which has the same code range
4009 as Emacs' internal format. Assigned the coding-system (Lisp
4010 symbol) `emacs-mule' by default.
4012 o coding-category-sjis
4014 The category for a coding system which has the same code range
4015 as SJIS. Assigned the coding-system (Lisp
4016 symbol) `japanese-shift-jis' by default.
4018 o coding-category-iso-7
4020 The category for a coding system which has the same code range
4021 as ISO2022 of 7-bit environment. This doesn't use any locking
4022 shift and single shift functions. This can encode/decode all
4023 charsets. Assigned the coding-system (Lisp symbol)
4024 `iso-2022-7bit' by default.
4026 o coding-category-iso-7-tight
4028 Same as coding-category-iso-7 except that this can
4029 encode/decode only the specified charsets.
4031 o coding-category-iso-8-1
4033 The category for a coding system which has the same code range
4034 as ISO2022 of 8-bit environment and graphic plane 1 used only
4035 for DIMENSION1 charset. This doesn't use any locking shift
4036 and single shift functions. Assigned the coding-system (Lisp
4037 symbol) `iso-latin-1' by default.
4039 o coding-category-iso-8-2
4041 The category for a coding system which has the same code range
4042 as ISO2022 of 8-bit environment and graphic plane 1 used only
4043 for DIMENSION2 charset. This doesn't use any locking shift
4044 and single shift functions. Assigned the coding-system (Lisp
4045 symbol) `japanese-iso-8bit' by default.
4047 o coding-category-iso-7-else
4049 The category for a coding system which has the same code range
4050 as ISO2022 of 7-bit environment but uses locking shift or
4051 single shift functions. Assigned the coding-system (Lisp
4052 symbol) `iso-2022-7bit-lock' by default.
4054 o coding-category-iso-8-else
4056 The category for a coding system which has the same code range
4057 as ISO2022 of 8-bit environment but uses locking shift or
4058 single shift functions. Assigned the coding-system (Lisp
4059 symbol) `iso-2022-8bit-ss2' by default.
4061 o coding-category-big5
4063 The category for a coding system which has the same code range
4064 as BIG5. Assigned the coding-system (Lisp symbol)
4065 `cn-big5' by default.
4067 o coding-category-utf-8
4069 The category for a coding system which has the same code range
4070 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
4071 symbol) `utf-8' by default.
4073 o coding-category-utf-16-be
4075 The category for a coding system in which a text has an
4076 Unicode signature (cf. Unicode Standard) in the order of BIG
4077 endian at the head. Assigned the coding-system (Lisp symbol)
4078 `utf-16-be' by default.
4080 o coding-category-utf-16-le
4082 The category for a coding system in which a text has an
4083 Unicode signature (cf. Unicode Standard) in the order of
4084 LITTLE endian at the head. Assigned the coding-system (Lisp
4085 symbol) `utf-16-le' by default.
4087 o coding-category-ccl
4089 The category for a coding system of which encoder/decoder is
4090 written in CCL programs. The default value is nil, i.e., no
4091 coding system is assigned.
4093 o coding-category-binary
4095 The category for a coding system not categorized in any of the
4096 above. Assigned the coding-system (Lisp symbol)
4097 `no-conversion' by default.
4099 Each of them is a Lisp symbol and the value is an actual
4100 `coding-system' (this is also a Lisp symbol) assigned by a user.
4101 What Emacs does actually is to detect a category of coding system.
4102 Then, it uses a `coding-system' assigned to it. If Emacs can't
4103 decide a single possible category, it selects a category of the
4104 highest priority. Priorities of categories are also specified by a
4105 user in a Lisp variable `coding-category-list'.
4110 int ascii_skip_code
[256];
4112 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
4113 If it detects possible coding systems, return an integer in which
4114 appropriate flag bits are set. Flag bits are defined by macros
4115 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
4116 it should point the table `coding_priorities'. In that case, only
4117 the flag bit for a coding system of the highest priority is set in
4118 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
4119 range 0x80..0x9F are in multibyte form.
4121 How many ASCII characters are at the head is returned as *SKIP. */
4124 detect_coding_mask (source
, src_bytes
, priorities
, skip
, multibytep
)
4125 unsigned char *source
;
4126 int src_bytes
, *priorities
, *skip
;
4129 register unsigned char c
;
4130 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
4131 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
4134 /* At first, skip all ASCII characters and control characters except
4135 for three ISO2022 specific control characters. */
4136 ascii_skip_code
[ISO_CODE_SO
] = 0;
4137 ascii_skip_code
[ISO_CODE_SI
] = 0;
4138 ascii_skip_code
[ISO_CODE_ESC
] = 0;
4140 label_loop_detect_coding
:
4141 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
4142 *skip
= src
- source
;
4145 /* We found nothing other than ASCII. There's nothing to do. */
4149 /* The text seems to be encoded in some multilingual coding system.
4150 Now, try to find in which coding system the text is encoded. */
4153 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
4154 /* C is an ISO2022 specific control code of C0. */
4155 mask
= detect_coding_iso2022 (src
, src_end
, multibytep
);
4158 /* No valid ISO2022 code follows C. Try again. */
4160 if (c
== ISO_CODE_ESC
)
4161 ascii_skip_code
[ISO_CODE_ESC
] = 1;
4163 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
4164 goto label_loop_detect_coding
;
4168 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4170 if (mask
& priorities
[i
])
4171 return priorities
[i
];
4173 return CODING_CATEGORY_MASK_RAW_TEXT
;
4180 if (multibytep
&& c
== LEADING_CODE_8_BIT_CONTROL
)
4185 /* C is the first byte of SJIS character code,
4186 or a leading-code of Emacs' internal format (emacs-mule),
4187 or the first byte of UTF-16. */
4188 try = (CODING_CATEGORY_MASK_SJIS
4189 | CODING_CATEGORY_MASK_EMACS_MULE
4190 | CODING_CATEGORY_MASK_UTF_16_BE
4191 | CODING_CATEGORY_MASK_UTF_16_LE
);
4193 /* Or, if C is a special latin extra code,
4194 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4195 or is an ISO2022 control-sequence-introducer (CSI),
4196 we should also consider the possibility of ISO2022 codings. */
4197 if ((VECTORP (Vlatin_extra_code_table
)
4198 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
4199 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
4200 || (c
== ISO_CODE_CSI
4203 || ((*src
== '0' || *src
== '1' || *src
== '2')
4204 && src
+ 1 < src_end
4205 && src
[1] == ']')))))
4206 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4207 | CODING_CATEGORY_MASK_ISO_8BIT
);
4210 /* C is a character of ISO2022 in graphic plane right,
4211 or a SJIS's 1-byte character code (i.e. JISX0201),
4212 or the first byte of BIG5's 2-byte code,
4213 or the first byte of UTF-8/16. */
4214 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4215 | CODING_CATEGORY_MASK_ISO_8BIT
4216 | CODING_CATEGORY_MASK_SJIS
4217 | CODING_CATEGORY_MASK_BIG5
4218 | CODING_CATEGORY_MASK_UTF_8
4219 | CODING_CATEGORY_MASK_UTF_16_BE
4220 | CODING_CATEGORY_MASK_UTF_16_LE
);
4222 /* Or, we may have to consider the possibility of CCL. */
4223 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4224 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4225 ->spec
.ccl
.valid_codes
)[c
])
4226 try |= CODING_CATEGORY_MASK_CCL
;
4229 utf16_examined_p
= iso2022_examined_p
= 0;
4232 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4234 if (!iso2022_examined_p
4235 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
4237 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4238 iso2022_examined_p
= 1;
4240 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
4241 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4242 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
4243 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4244 else if (!utf16_examined_p
4245 && (priorities
[i
] & try &
4246 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
4248 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4249 utf16_examined_p
= 1;
4251 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
4252 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4253 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
4254 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4255 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
4256 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4257 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
4258 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
4259 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
4260 mask
|= CODING_CATEGORY_MASK_BINARY
;
4261 if (mask
& priorities
[i
])
4262 return priorities
[i
];
4264 return CODING_CATEGORY_MASK_RAW_TEXT
;
4266 if (try & CODING_CATEGORY_MASK_ISO
)
4267 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4268 if (try & CODING_CATEGORY_MASK_SJIS
)
4269 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4270 if (try & CODING_CATEGORY_MASK_BIG5
)
4271 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4272 if (try & CODING_CATEGORY_MASK_UTF_8
)
4273 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4274 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
4275 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4276 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
4277 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4278 if (try & CODING_CATEGORY_MASK_CCL
)
4279 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4281 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
4284 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4285 The information of the detected coding system is set in CODING. */
4288 detect_coding (coding
, src
, src_bytes
)
4289 struct coding_system
*coding
;
4290 const unsigned char *src
;
4297 val
= Vcoding_category_list
;
4298 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
,
4299 coding
->src_multibyte
);
4300 coding
->heading_ascii
= skip
;
4304 /* We found a single coding system of the highest priority in MASK. */
4306 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
4308 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
4310 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[idx
]);
4312 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4316 tmp
= Fget (val
, Qeol_type
);
4318 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
4321 /* Setup this new coding system while preserving some slots. */
4323 int src_multibyte
= coding
->src_multibyte
;
4324 int dst_multibyte
= coding
->dst_multibyte
;
4326 setup_coding_system (val
, coding
);
4327 coding
->src_multibyte
= src_multibyte
;
4328 coding
->dst_multibyte
= dst_multibyte
;
4329 coding
->heading_ascii
= skip
;
4333 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
4334 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4335 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4337 How many non-eol characters are at the head is returned as *SKIP. */
4339 #define MAX_EOL_CHECK_COUNT 3
4342 detect_eol_type (source
, src_bytes
, skip
)
4343 unsigned char *source
;
4344 int src_bytes
, *skip
;
4346 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4348 int total
= 0; /* How many end-of-lines are found so far. */
4349 int eol_type
= CODING_EOL_UNDECIDED
;
4354 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4357 if (c
== '\n' || c
== '\r')
4360 *skip
= src
- 1 - source
;
4363 this_eol_type
= CODING_EOL_LF
;
4364 else if (src
>= src_end
|| *src
!= '\n')
4365 this_eol_type
= CODING_EOL_CR
;
4367 this_eol_type
= CODING_EOL_CRLF
, src
++;
4369 if (eol_type
== CODING_EOL_UNDECIDED
)
4370 /* This is the first end-of-line. */
4371 eol_type
= this_eol_type
;
4372 else if (eol_type
!= this_eol_type
)
4374 /* The found type is different from what found before. */
4375 eol_type
= CODING_EOL_INCONSISTENT
;
4382 *skip
= src_end
- source
;
4386 /* Like detect_eol_type, but detect EOL type in 2-octet
4387 big-endian/little-endian format for coding systems utf-16-be and
4391 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
4392 unsigned char *source
;
4393 int src_bytes
, *skip
, big_endian_p
;
4395 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4396 unsigned int c1
, c2
;
4397 int total
= 0; /* How many end-of-lines are found so far. */
4398 int eol_type
= CODING_EOL_UNDECIDED
;
4409 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4411 c1
= (src
[msb
] << 8) | (src
[lsb
]);
4414 if (c1
== '\n' || c1
== '\r')
4417 *skip
= src
- 2 - source
;
4421 this_eol_type
= CODING_EOL_LF
;
4425 if ((src
+ 1) >= src_end
)
4427 this_eol_type
= CODING_EOL_CR
;
4431 c2
= (src
[msb
] << 8) | (src
[lsb
]);
4433 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
4435 this_eol_type
= CODING_EOL_CR
;
4439 if (eol_type
== CODING_EOL_UNDECIDED
)
4440 /* This is the first end-of-line. */
4441 eol_type
= this_eol_type
;
4442 else if (eol_type
!= this_eol_type
)
4444 /* The found type is different from what found before. */
4445 eol_type
= CODING_EOL_INCONSISTENT
;
4452 *skip
= src_end
- source
;
4456 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4457 is encoded. If it detects an appropriate format of end-of-line, it
4458 sets the information in *CODING. */
4461 detect_eol (coding
, src
, src_bytes
)
4462 struct coding_system
*coding
;
4463 const unsigned char *src
;
4470 switch (coding
->category_idx
)
4472 case CODING_CATEGORY_IDX_UTF_16_BE
:
4473 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
4475 case CODING_CATEGORY_IDX_UTF_16_LE
:
4476 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
4479 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
4483 if (coding
->heading_ascii
> skip
)
4484 coding
->heading_ascii
= skip
;
4486 skip
= coding
->heading_ascii
;
4488 if (eol_type
== CODING_EOL_UNDECIDED
)
4490 if (eol_type
== CODING_EOL_INCONSISTENT
)
4493 /* This code is suppressed until we find a better way to
4494 distinguish raw text file and binary file. */
4496 /* If we have already detected that the coding is raw-text, the
4497 coding should actually be no-conversion. */
4498 if (coding
->type
== coding_type_raw_text
)
4500 setup_coding_system (Qno_conversion
, coding
);
4503 /* Else, let's decode only text code anyway. */
4505 eol_type
= CODING_EOL_LF
;
4508 val
= Fget (coding
->symbol
, Qeol_type
);
4509 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4511 int src_multibyte
= coding
->src_multibyte
;
4512 int dst_multibyte
= coding
->dst_multibyte
;
4513 struct composition_data
*cmp_data
= coding
->cmp_data
;
4515 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
4516 coding
->src_multibyte
= src_multibyte
;
4517 coding
->dst_multibyte
= dst_multibyte
;
4518 coding
->heading_ascii
= skip
;
4519 coding
->cmp_data
= cmp_data
;
4523 #define CONVERSION_BUFFER_EXTRA_ROOM 256
4525 #define DECODING_BUFFER_MAG(coding) \
4526 (coding->type == coding_type_iso2022 \
4528 : (coding->type == coding_type_ccl \
4529 ? coding->spec.ccl.decoder.buf_magnification \
4532 /* Return maximum size (bytes) of a buffer enough for decoding
4533 SRC_BYTES of text encoded in CODING. */
4536 decoding_buffer_size (coding
, src_bytes
)
4537 struct coding_system
*coding
;
4540 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
4541 + CONVERSION_BUFFER_EXTRA_ROOM
);
4544 /* Return maximum size (bytes) of a buffer enough for encoding
4545 SRC_BYTES of text to CODING. */
4548 encoding_buffer_size (coding
, src_bytes
)
4549 struct coding_system
*coding
;
4554 if (coding
->type
== coding_type_ccl
)
4556 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
4557 if (coding
->eol_type
== CODING_EOL_CRLF
)
4560 else if (CODING_REQUIRE_ENCODING (coding
))
4565 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
4568 /* Working buffer for code conversion. */
4569 struct conversion_buffer
4571 int size
; /* size of data. */
4572 int on_stack
; /* 1 if allocated by alloca. */
4573 unsigned char *data
;
4576 /* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4577 #define allocate_conversion_buffer(buf, len) \
4579 if (len < MAX_ALLOCA) \
4581 buf.data = (unsigned char *) alloca (len); \
4586 buf.data = (unsigned char *) xmalloc (len); \
4592 /* Double the allocated memory for *BUF. */
4594 extend_conversion_buffer (buf
)
4595 struct conversion_buffer
*buf
;
4599 unsigned char *save
= buf
->data
;
4600 buf
->data
= (unsigned char *) xmalloc (buf
->size
* 2);
4601 bcopy (save
, buf
->data
, buf
->size
);
4606 buf
->data
= (unsigned char *) xrealloc (buf
->data
, buf
->size
* 2);
4611 /* Free the allocated memory for BUF if it is not on stack. */
4613 free_conversion_buffer (buf
)
4614 struct conversion_buffer
*buf
;
4621 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
4622 struct coding_system
*coding
;
4623 unsigned char *source
, *destination
;
4624 int src_bytes
, dst_bytes
, encodep
;
4626 struct ccl_program
*ccl
4627 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
4628 unsigned char *dst
= destination
;
4630 ccl
->suppress_error
= coding
->suppress_error
;
4631 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
4634 /* On encoding, EOL format is converted within ccl_driver. For
4635 that, setup proper information in the structure CCL. */
4636 ccl
->eol_type
= coding
->eol_type
;
4637 if (ccl
->eol_type
==CODING_EOL_UNDECIDED
)
4638 ccl
->eol_type
= CODING_EOL_LF
;
4639 ccl
->cr_consumed
= coding
->spec
.ccl
.cr_carryover
;
4640 ccl
->eight_bit_control
= coding
->dst_multibyte
;
4643 ccl
->eight_bit_control
= 1;
4644 ccl
->multibyte
= coding
->src_multibyte
;
4645 if (coding
->spec
.ccl
.eight_bit_carryover
[0] != 0)
4647 /* Move carryover bytes to DESTINATION. */
4648 unsigned char *p
= coding
->spec
.ccl
.eight_bit_carryover
;
4651 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4653 dst_bytes
-= dst
- destination
;
4656 coding
->produced
= (ccl_driver (ccl
, source
, dst
, src_bytes
, dst_bytes
,
4657 &(coding
->consumed
))
4658 + dst
- destination
);
4662 coding
->produced_char
= coding
->produced
;
4663 coding
->spec
.ccl
.cr_carryover
= ccl
->cr_consumed
;
4665 else if (!ccl
->eight_bit_control
)
4667 /* The produced bytes forms a valid multibyte sequence. */
4668 coding
->produced_char
4669 = multibyte_chars_in_text (destination
, coding
->produced
);
4670 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4674 /* On decoding, the destination should always multibyte. But,
4675 CCL program might have been generated an invalid multibyte
4676 sequence. Here we make such a sequence valid as
4679 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
4681 if ((coding
->consumed
< src_bytes
4682 || !ccl
->last_block
)
4683 && coding
->produced
>= 1
4684 && destination
[coding
->produced
- 1] >= 0x80)
4686 /* We should not convert the tailing 8-bit codes to
4687 multibyte form even if they doesn't form a valid
4688 multibyte sequence. They may form a valid sequence in
4692 if (destination
[coding
->produced
- 1] < 0xA0)
4694 else if (coding
->produced
>= 2)
4696 if (destination
[coding
->produced
- 2] >= 0x80)
4698 if (destination
[coding
->produced
- 2] < 0xA0)
4700 else if (coding
->produced
>= 3
4701 && destination
[coding
->produced
- 3] >= 0x80
4702 && destination
[coding
->produced
- 3] < 0xA0)
4708 BCOPY_SHORT (destination
+ coding
->produced
- carryover
,
4709 coding
->spec
.ccl
.eight_bit_carryover
,
4711 coding
->spec
.ccl
.eight_bit_carryover
[carryover
] = 0;
4712 coding
->produced
-= carryover
;
4715 coding
->produced
= str_as_multibyte (destination
, bytes
,
4717 &(coding
->produced_char
));
4720 switch (ccl
->status
)
4722 case CCL_STAT_SUSPEND_BY_SRC
:
4723 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
4725 case CCL_STAT_SUSPEND_BY_DST
:
4726 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
4729 case CCL_STAT_INVALID_CMD
:
4730 coding
->result
= CODING_FINISH_INTERRUPT
;
4733 coding
->result
= CODING_FINISH_NORMAL
;
4736 return coding
->result
;
4739 /* Decode EOL format of the text at PTR of BYTES length destructively
4740 according to CODING->eol_type. This is called after the CCL
4741 program produced a decoded text at PTR. If we do CRLF->LF
4742 conversion, update CODING->produced and CODING->produced_char. */
4745 decode_eol_post_ccl (coding
, ptr
, bytes
)
4746 struct coding_system
*coding
;
4750 Lisp_Object val
, saved_coding_symbol
;
4751 unsigned char *pend
= ptr
+ bytes
;
4754 /* Remember the current coding system symbol. We set it back when
4755 an inconsistent EOL is found so that `last-coding-system-used' is
4756 set to the coding system that doesn't specify EOL conversion. */
4757 saved_coding_symbol
= coding
->symbol
;
4759 coding
->spec
.ccl
.cr_carryover
= 0;
4760 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4762 /* Here, to avoid the call of setup_coding_system, we directly
4763 call detect_eol_type. */
4764 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
4765 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
4766 coding
->eol_type
= CODING_EOL_LF
;
4767 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4769 val
= Fget (coding
->symbol
, Qeol_type
);
4770 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4771 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
4773 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4776 if (coding
->eol_type
== CODING_EOL_LF
4777 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
4779 /* We have nothing to do. */
4782 else if (coding
->eol_type
== CODING_EOL_CRLF
)
4784 unsigned char *pstart
= ptr
, *p
= ptr
;
4786 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4787 && *(pend
- 1) == '\r')
4789 /* If the last character is CR, we can't handle it here
4790 because LF will be in the not-yet-decoded source text.
4791 Record that the CR is not yet processed. */
4792 coding
->spec
.ccl
.cr_carryover
= 1;
4794 coding
->produced_char
--;
4801 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4808 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4809 goto undo_eol_conversion
;
4813 else if (*ptr
== '\n'
4814 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4815 goto undo_eol_conversion
;
4820 undo_eol_conversion
:
4821 /* We have faced with inconsistent EOL format at PTR.
4822 Convert all LFs before PTR back to CRLFs. */
4823 for (p
--, ptr
--; p
>= pstart
; p
--)
4826 *ptr
-- = '\n', *ptr
-- = '\r';
4830 /* If carryover is recorded, cancel it because we don't
4831 convert CRLF anymore. */
4832 if (coding
->spec
.ccl
.cr_carryover
)
4834 coding
->spec
.ccl
.cr_carryover
= 0;
4836 coding
->produced_char
++;
4840 coding
->eol_type
= CODING_EOL_LF
;
4841 coding
->symbol
= saved_coding_symbol
;
4845 /* As each two-byte sequence CRLF was converted to LF, (PEND
4846 - P) is the number of deleted characters. */
4847 coding
->produced
-= pend
- p
;
4848 coding
->produced_char
-= pend
- p
;
4851 else /* i.e. coding->eol_type == CODING_EOL_CR */
4853 unsigned char *p
= ptr
;
4855 for (; ptr
< pend
; ptr
++)
4859 else if (*ptr
== '\n'
4860 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4862 for (; p
< ptr
; p
++)
4868 coding
->eol_type
= CODING_EOL_LF
;
4869 coding
->symbol
= saved_coding_symbol
;
4875 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4876 decoding, it may detect coding system and format of end-of-line if
4877 those are not yet decided. The source should be unibyte, the
4878 result is multibyte if CODING->dst_multibyte is nonzero, else
4882 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4883 struct coding_system
*coding
;
4884 const unsigned char *source
;
4885 unsigned char *destination
;
4886 int src_bytes
, dst_bytes
;
4890 if (coding
->type
== coding_type_undecided
)
4891 detect_coding (coding
, source
, src_bytes
);
4893 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4894 && coding
->type
!= coding_type_ccl
)
4896 detect_eol (coding
, source
, src_bytes
);
4897 /* We had better recover the original eol format if we
4898 encounter an inconsistent eol format while decoding. */
4899 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4902 coding
->produced
= coding
->produced_char
= 0;
4903 coding
->consumed
= coding
->consumed_char
= 0;
4905 coding
->result
= CODING_FINISH_NORMAL
;
4907 switch (coding
->type
)
4909 case coding_type_sjis
:
4910 decode_coding_sjis_big5 (coding
, source
, destination
,
4911 src_bytes
, dst_bytes
, 1);
4914 case coding_type_iso2022
:
4915 decode_coding_iso2022 (coding
, source
, destination
,
4916 src_bytes
, dst_bytes
);
4919 case coding_type_big5
:
4920 decode_coding_sjis_big5 (coding
, source
, destination
,
4921 src_bytes
, dst_bytes
, 0);
4924 case coding_type_emacs_mule
:
4925 decode_coding_emacs_mule (coding
, source
, destination
,
4926 src_bytes
, dst_bytes
);
4929 case coding_type_ccl
:
4930 if (coding
->spec
.ccl
.cr_carryover
)
4932 /* Put the CR which was not processed by the previous call
4933 of decode_eol_post_ccl in DESTINATION. It will be
4934 decoded together with the following LF by the call to
4935 decode_eol_post_ccl below. */
4936 *destination
= '\r';
4938 coding
->produced_char
++;
4940 extra
= coding
->spec
.ccl
.cr_carryover
;
4942 ccl_coding_driver (coding
, source
, destination
+ extra
,
4943 src_bytes
, dst_bytes
, 0);
4944 if (coding
->eol_type
!= CODING_EOL_LF
)
4946 coding
->produced
+= extra
;
4947 coding
->produced_char
+= extra
;
4948 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4953 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4956 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4957 && coding
->mode
& CODING_MODE_LAST_BLOCK
4958 && coding
->consumed
== src_bytes
)
4959 coding
->result
= CODING_FINISH_NORMAL
;
4961 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4962 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4964 const unsigned char *src
= source
+ coding
->consumed
;
4965 unsigned char *dst
= destination
+ coding
->produced
;
4967 src_bytes
-= coding
->consumed
;
4969 if (COMPOSING_P (coding
))
4970 DECODE_COMPOSITION_END ('1');
4974 dst
+= CHAR_STRING (c
, dst
);
4975 coding
->produced_char
++;
4977 coding
->consumed
= coding
->consumed_char
= src
- source
;
4978 coding
->produced
= dst
- destination
;
4979 coding
->result
= CODING_FINISH_NORMAL
;
4982 if (!coding
->dst_multibyte
)
4984 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4985 coding
->produced_char
= coding
->produced
;
4988 return coding
->result
;
4991 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4992 multibyteness of the source is CODING->src_multibyte, the
4993 multibyteness of the result is always unibyte. */
4996 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4997 struct coding_system
*coding
;
4998 const unsigned char *source
;
4999 unsigned char *destination
;
5000 int src_bytes
, dst_bytes
;
5002 coding
->produced
= coding
->produced_char
= 0;
5003 coding
->consumed
= coding
->consumed_char
= 0;
5005 coding
->result
= CODING_FINISH_NORMAL
;
5006 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5007 coding
->eol_type
= system_eol_type
;
5009 switch (coding
->type
)
5011 case coding_type_sjis
:
5012 encode_coding_sjis_big5 (coding
, source
, destination
,
5013 src_bytes
, dst_bytes
, 1);
5016 case coding_type_iso2022
:
5017 encode_coding_iso2022 (coding
, source
, destination
,
5018 src_bytes
, dst_bytes
);
5021 case coding_type_big5
:
5022 encode_coding_sjis_big5 (coding
, source
, destination
,
5023 src_bytes
, dst_bytes
, 0);
5026 case coding_type_emacs_mule
:
5027 encode_coding_emacs_mule (coding
, source
, destination
,
5028 src_bytes
, dst_bytes
);
5031 case coding_type_ccl
:
5032 ccl_coding_driver (coding
, source
, destination
,
5033 src_bytes
, dst_bytes
, 1);
5037 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
5040 if (coding
->mode
& CODING_MODE_LAST_BLOCK
5041 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
5043 const unsigned char *src
= source
+ coding
->consumed
;
5044 unsigned char *dst
= destination
+ coding
->produced
;
5046 if (coding
->type
== coding_type_iso2022
)
5047 ENCODE_RESET_PLANE_AND_REGISTER
;
5048 if (COMPOSING_P (coding
))
5049 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
5050 if (coding
->consumed
< src_bytes
)
5052 int len
= src_bytes
- coding
->consumed
;
5054 BCOPY_SHORT (src
, dst
, len
);
5055 if (coding
->src_multibyte
)
5056 len
= str_as_unibyte (dst
, len
);
5058 coding
->consumed
= src_bytes
;
5060 coding
->produced
= coding
->produced_char
= dst
- destination
;
5061 coding
->result
= CODING_FINISH_NORMAL
;
5064 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
5065 && coding
->consumed
== src_bytes
)
5066 coding
->result
= CODING_FINISH_NORMAL
;
5068 return coding
->result
;
5071 /* Scan text in the region between *BEG and *END (byte positions),
5072 skip characters which we don't have to decode by coding system
5073 CODING at the head and tail, then set *BEG and *END to the region
5074 of the text we actually have to convert. The caller should move
5075 the gap out of the region in advance if the region is from a
5078 If STR is not NULL, *BEG and *END are indices into STR. */
5081 shrink_decoding_region (beg
, end
, coding
, str
)
5083 struct coding_system
*coding
;
5086 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
5088 Lisp_Object translation_table
;
5090 if (coding
->type
== coding_type_ccl
5091 || coding
->type
== coding_type_undecided
5092 || coding
->eol_type
!= CODING_EOL_LF
5093 || !NILP (coding
->post_read_conversion
)
5094 || coding
->composing
!= COMPOSITION_DISABLED
)
5096 /* We can't skip any data. */
5099 if (coding
->type
== coding_type_no_conversion
5100 || coding
->type
== coding_type_raw_text
5101 || coding
->type
== coding_type_emacs_mule
)
5103 /* We need no conversion, but don't have to skip any data here.
5104 Decoding routine handles them effectively anyway. */
5108 translation_table
= coding
->translation_table_for_decode
;
5109 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5110 translation_table
= Vstandard_translation_table_for_decode
;
5111 if (CHAR_TABLE_P (translation_table
))
5114 for (i
= 0; i
< 128; i
++)
5115 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5118 /* Some ASCII character should be translated. We give up
5123 if (coding
->heading_ascii
>= 0)
5124 /* Detection routine has already found how much we can skip at the
5126 *beg
+= coding
->heading_ascii
;
5130 begp_orig
= begp
= str
+ *beg
;
5131 endp_orig
= endp
= str
+ *end
;
5135 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5136 endp_orig
= endp
= begp
+ *end
- *beg
;
5139 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5140 || coding
->eol_type
== CODING_EOL_CRLF
);
5142 switch (coding
->type
)
5144 case coding_type_sjis
:
5145 case coding_type_big5
:
5146 /* We can skip all ASCII characters at the head. */
5147 if (coding
->heading_ascii
< 0)
5150 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
5152 while (begp
< endp
&& *begp
< 0x80) begp
++;
5154 /* We can skip all ASCII characters at the tail except for the
5155 second byte of SJIS or BIG5 code. */
5157 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
5159 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5160 /* Do not consider LF as ascii if preceded by CR, since that
5161 confuses eol decoding. */
5162 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5164 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
5168 case coding_type_iso2022
:
5169 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5170 /* We can't skip any data. */
5172 if (coding
->heading_ascii
< 0)
5174 /* We can skip all ASCII characters at the head except for a
5175 few control codes. */
5176 while (begp
< endp
&& (c
= *begp
) < 0x80
5177 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
5178 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
5179 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
5182 switch (coding
->category_idx
)
5184 case CODING_CATEGORY_IDX_ISO_8_1
:
5185 case CODING_CATEGORY_IDX_ISO_8_2
:
5186 /* We can skip all ASCII characters at the tail. */
5188 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
5190 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5191 /* Do not consider LF as ascii if preceded by CR, since that
5192 confuses eol decoding. */
5193 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5197 case CODING_CATEGORY_IDX_ISO_7
:
5198 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
5200 /* We can skip all characters at the tail except for 8-bit
5201 codes and ESC and the following 2-byte at the tail. */
5202 unsigned char *eight_bit
= NULL
;
5206 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
5208 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5213 && (c
= endp
[-1]) != ISO_CODE_ESC
)
5215 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5218 /* Do not consider LF as ascii if preceded by CR, since that
5219 confuses eol decoding. */
5220 if (begp
< endp
&& endp
< endp_orig
5221 && endp
[-1] == '\r' && endp
[0] == '\n')
5223 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
5225 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
5226 /* This is an ASCII designation sequence. We can
5227 surely skip the tail. But, if we have
5228 encountered an 8-bit code, skip only the codes
5230 endp
= eight_bit
? eight_bit
: endp
+ 2;
5232 /* Hmmm, we can't skip the tail. */
5244 *beg
+= begp
- begp_orig
;
5245 *end
+= endp
- endp_orig
;
5249 /* Like shrink_decoding_region but for encoding. */
5252 shrink_encoding_region (beg
, end
, coding
, str
)
5254 struct coding_system
*coding
;
5257 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
5259 Lisp_Object translation_table
;
5261 if (coding
->type
== coding_type_ccl
5262 || coding
->eol_type
== CODING_EOL_CRLF
5263 || coding
->eol_type
== CODING_EOL_CR
5264 || (coding
->eol_type
== CODING_EOL_UNDECIDED
5265 && system_eol_type
!= CODING_EOL_LF
)
5266 || (coding
->cmp_data
&& coding
->cmp_data
->used
> 0))
5268 /* We can't skip any data. */
5271 if (coding
->type
== coding_type_no_conversion
5272 || coding
->type
== coding_type_raw_text
5273 || coding
->type
== coding_type_emacs_mule
5274 || coding
->type
== coding_type_undecided
)
5276 /* We need no conversion, but don't have to skip any data here.
5277 Encoding routine handles them effectively anyway. */
5281 translation_table
= coding
->translation_table_for_encode
;
5282 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5283 translation_table
= Vstandard_translation_table_for_encode
;
5284 if (CHAR_TABLE_P (translation_table
))
5287 for (i
= 0; i
< 128; i
++)
5288 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5291 /* Some ASCII character should be translated. We give up
5298 begp_orig
= begp
= str
+ *beg
;
5299 endp_orig
= endp
= str
+ *end
;
5303 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5304 endp_orig
= endp
= begp
+ *end
- *beg
;
5307 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5308 || coding
->eol_type
== CODING_EOL_CRLF
);
5310 /* Here, we don't have to check coding->pre_write_conversion because
5311 the caller is expected to have handled it already. */
5312 switch (coding
->type
)
5314 case coding_type_iso2022
:
5315 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5316 /* We can't skip any data. */
5318 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
5320 unsigned char *bol
= begp
;
5321 while (begp
< endp
&& *begp
< 0x80)
5324 if (begp
[-1] == '\n')
5328 goto label_skip_tail
;
5332 case coding_type_sjis
:
5333 case coding_type_big5
:
5334 /* We can skip all ASCII characters at the head and tail. */
5336 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
5338 while (begp
< endp
&& *begp
< 0x80) begp
++;
5341 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
5343 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
5350 *beg
+= begp
- begp_orig
;
5351 *end
+= endp
- endp_orig
;
5355 /* As shrinking conversion region requires some overhead, we don't try
5356 shrinking if the length of conversion region is less than this
5358 static int shrink_conversion_region_threshhold
= 1024;
5360 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5362 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5364 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5365 else shrink_decoding_region (beg, end, coding, str); \
5369 /* ARG is (CODING BUFFER ...) where CODING is what to be set in
5370 Vlast_coding_system_used and the remaining elements are buffers to
5373 code_convert_region_unwind (arg
)
5376 struct gcpro gcpro1
;
5379 inhibit_pre_post_conversion
= 0;
5380 Vlast_coding_system_used
= XCAR (arg
);
5381 for (arg
= XCDR (arg
); ! NILP (arg
); arg
= XCDR (arg
))
5382 Fkill_buffer (XCAR (arg
));
5388 /* Store information about all compositions in the range FROM and TO
5389 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5390 buffer or a string, defaults to the current buffer. */
5393 coding_save_composition (coding
, from
, to
, obj
)
5394 struct coding_system
*coding
;
5401 if (coding
->composing
== COMPOSITION_DISABLED
)
5403 if (!coding
->cmp_data
)
5404 coding_allocate_composition_data (coding
, from
);
5405 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
5409 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
5412 coding
->composing
= COMPOSITION_NO
;
5415 if (COMPOSITION_VALID_P (start
, end
, prop
))
5417 enum composition_method method
= COMPOSITION_METHOD (prop
);
5418 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
5419 >= COMPOSITION_DATA_SIZE
)
5420 coding_allocate_composition_data (coding
, from
);
5421 /* For relative composition, we remember start and end
5422 positions, for the other compositions, we also remember
5424 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
5425 if (method
!= COMPOSITION_RELATIVE
)
5427 /* We must store a*/
5428 Lisp_Object val
, ch
;
5430 val
= COMPOSITION_COMPONENTS (prop
);
5434 ch
= XCAR (val
), val
= XCDR (val
);
5435 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5437 else if (VECTORP (val
) || STRINGP (val
))
5439 int len
= (VECTORP (val
)
5440 ? XVECTOR (val
)->size
: SCHARS (val
));
5442 for (i
= 0; i
< len
; i
++)
5445 ? Faref (val
, make_number (i
))
5446 : XVECTOR (val
)->contents
[i
]);
5447 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5450 else /* INTEGERP (val) */
5451 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
5453 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
5458 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
5461 /* Make coding->cmp_data point to the first memory block. */
5462 while (coding
->cmp_data
->prev
)
5463 coding
->cmp_data
= coding
->cmp_data
->prev
;
5464 coding
->cmp_data_start
= 0;
5467 /* Reflect the saved information about compositions to OBJ.
5468 CODING->cmp_data points to a memory block for the information. OBJ
5469 is a buffer or a string, defaults to the current buffer. */
5472 coding_restore_composition (coding
, obj
)
5473 struct coding_system
*coding
;
5476 struct composition_data
*cmp_data
= coding
->cmp_data
;
5481 while (cmp_data
->prev
)
5482 cmp_data
= cmp_data
->prev
;
5488 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
5489 i
+= cmp_data
->data
[i
])
5491 int *data
= cmp_data
->data
+ i
;
5492 enum composition_method method
= (enum composition_method
) data
[3];
5493 Lisp_Object components
;
5495 if (data
[0] < 0 || i
+ data
[0] > cmp_data
->used
)
5496 /* Invalid composition data. */
5499 if (method
== COMPOSITION_RELATIVE
)
5503 int len
= data
[0] - 4, j
;
5504 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
5506 if (method
== COMPOSITION_WITH_RULE_ALTCHARS
5510 /* Invalid composition data. */
5512 for (j
= 0; j
< len
; j
++)
5513 args
[j
] = make_number (data
[4 + j
]);
5514 components
= (method
== COMPOSITION_WITH_ALTCHARS
5515 ? Fstring (len
, args
)
5516 : Fvector (len
, args
));
5518 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
5520 cmp_data
= cmp_data
->next
;
5524 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
5525 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5526 coding system CODING, and return the status code of code conversion
5527 (currently, this value has no meaning).
5529 How many characters (and bytes) are converted to how many
5530 characters (and bytes) are recorded in members of the structure
5533 If REPLACE is nonzero, we do various things as if the original text
5534 is deleted and a new text is inserted. See the comments in
5535 replace_range (insdel.c) to know what we are doing.
5537 If REPLACE is zero, it is assumed that the source text is unibyte.
5538 Otherwise, it is assumed that the source text is multibyte. */
5541 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
5542 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
5543 struct coding_system
*coding
;
5545 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
5546 int nchars_del
= 0, nbytes_del
= 0;
5547 int require
, inserted
, inserted_byte
;
5548 int head_skip
, tail_skip
, total_skip
= 0;
5549 Lisp_Object saved_coding_symbol
;
5551 unsigned char *src
, *dst
;
5552 Lisp_Object deletion
;
5553 int orig_point
= PT
, orig_len
= len
;
5555 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
5558 saved_coding_symbol
= coding
->symbol
;
5560 if (from
< PT
&& PT
< to
)
5562 TEMP_SET_PT_BOTH (from
, from_byte
);
5568 int saved_from
= from
;
5569 int saved_inhibit_modification_hooks
;
5571 prepare_to_modify_buffer (from
, to
, &from
);
5572 if (saved_from
!= from
)
5575 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
5576 len_byte
= to_byte
- from_byte
;
5579 /* The code conversion routine can not preserve text properties
5580 for now. So, we must remove all text properties in the
5581 region. Here, we must suppress all modification hooks. */
5582 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
5583 inhibit_modification_hooks
= 1;
5584 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
5585 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
5588 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
5590 /* We must detect encoding of text and eol format. */
5592 if (from
< GPT
&& to
> GPT
)
5593 move_gap_both (from
, from_byte
);
5594 if (coding
->type
== coding_type_undecided
)
5596 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5597 if (coding
->type
== coding_type_undecided
)
5599 /* It seems that the text contains only ASCII, but we
5600 should not leave it undecided because the deeper
5601 decoding routine (decode_coding) tries to detect the
5602 encodings again in vain. */
5603 coding
->type
= coding_type_emacs_mule
;
5604 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5605 /* As emacs-mule decoder will handle composition, we
5606 need this setting to allocate coding->cmp_data
5608 coding
->composing
= COMPOSITION_NO
;
5611 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5612 && coding
->type
!= coding_type_ccl
)
5614 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5615 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5616 coding
->eol_type
= CODING_EOL_LF
;
5617 /* We had better recover the original eol format if we
5618 encounter an inconsistent eol format while decoding. */
5619 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5623 /* Now we convert the text. */
5625 /* For encoding, we must process pre-write-conversion in advance. */
5626 if (! inhibit_pre_post_conversion
5628 && SYMBOLP (coding
->pre_write_conversion
)
5629 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
5631 /* The function in pre-write-conversion may put a new text in a
5633 struct buffer
*prev
= current_buffer
;
5636 record_unwind_protect (code_convert_region_unwind
,
5637 Fcons (Vlast_coding_system_used
, Qnil
));
5638 /* We should not call any more pre-write/post-read-conversion
5639 functions while this pre-write-conversion is running. */
5640 inhibit_pre_post_conversion
= 1;
5641 call2 (coding
->pre_write_conversion
,
5642 make_number (from
), make_number (to
));
5643 inhibit_pre_post_conversion
= 0;
5644 /* Discard the unwind protect. */
5647 if (current_buffer
!= prev
)
5650 new = Fcurrent_buffer ();
5651 set_buffer_internal_1 (prev
);
5652 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
5653 TEMP_SET_PT_BOTH (from
, from_byte
);
5654 insert_from_buffer (XBUFFER (new), 1, len
, 0);
5656 if (orig_point
>= to
)
5657 orig_point
+= len
- orig_len
;
5658 else if (orig_point
> from
)
5662 from_byte
= CHAR_TO_BYTE (from
);
5663 to_byte
= CHAR_TO_BYTE (to
);
5664 len_byte
= to_byte
- from_byte
;
5665 TEMP_SET_PT_BOTH (from
, from_byte
);
5671 if (! EQ (current_buffer
->undo_list
, Qt
))
5672 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
5675 nchars_del
= to
- from
;
5676 nbytes_del
= to_byte
- from_byte
;
5680 if (coding
->composing
!= COMPOSITION_DISABLED
)
5683 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
5685 coding_allocate_composition_data (coding
, from
);
5688 /* Try to skip the heading and tailing ASCIIs. We can't skip them
5689 if we must run CCL program or there are compositions to
5691 if (coding
->type
!= coding_type_ccl
5692 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
5694 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
5696 if (from
< GPT
&& GPT
< to
)
5697 move_gap_both (from
, from_byte
);
5698 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
5699 if (from_byte
== to_byte
5700 && (encodep
|| NILP (coding
->post_read_conversion
))
5701 && ! CODING_REQUIRE_FLUSHING (coding
))
5703 coding
->produced
= len_byte
;
5704 coding
->produced_char
= len
;
5706 /* We must record and adjust for this new text now. */
5707 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
5708 coding_free_composition_data (coding
);
5712 head_skip
= from_byte
- from_byte_orig
;
5713 tail_skip
= to_byte_orig
- to_byte
;
5714 total_skip
= head_skip
+ tail_skip
;
5717 len
-= total_skip
; len_byte
-= total_skip
;
5720 /* For conversion, we must put the gap before the text in addition to
5721 making the gap larger for efficient decoding. The required gap
5722 size starts from 2000 which is the magic number used in make_gap.
5723 But, after one batch of conversion, it will be incremented if we
5724 find that it is not enough . */
5727 if (GAP_SIZE
< require
)
5728 make_gap (require
- GAP_SIZE
);
5729 move_gap_both (from
, from_byte
);
5731 inserted
= inserted_byte
= 0;
5733 GAP_SIZE
+= len_byte
;
5736 ZV_BYTE
-= len_byte
;
5739 if (GPT
- BEG
< BEG_UNCHANGED
)
5740 BEG_UNCHANGED
= GPT
- BEG
;
5741 if (Z
- GPT
< END_UNCHANGED
)
5742 END_UNCHANGED
= Z
- GPT
;
5744 if (!encodep
&& coding
->src_multibyte
)
5746 /* Decoding routines expects that the source text is unibyte.
5747 We must convert 8-bit characters of multibyte form to
5749 int len_byte_orig
= len_byte
;
5750 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
5751 if (len_byte
< len_byte_orig
)
5752 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
5754 coding
->src_multibyte
= 0;
5761 /* The buffer memory is now:
5762 +--------+converted-text+---------+-------original-text-------+---+
5763 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5764 |<---------------------- GAP ----------------------->| */
5765 src
= GAP_END_ADDR
- len_byte
;
5766 dst
= GPT_ADDR
+ inserted_byte
;
5769 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
5772 if (coding
->composing
!= COMPOSITION_DISABLED
)
5773 coding
->cmp_data
->char_offset
= from
+ inserted
;
5774 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
5777 /* The buffer memory is now:
5778 +--------+-------converted-text----+--+------original-text----+---+
5779 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5780 |<---------------------- GAP ----------------------->| */
5782 inserted
+= coding
->produced_char
;
5783 inserted_byte
+= coding
->produced
;
5784 len_byte
-= coding
->consumed
;
5786 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5788 coding_allocate_composition_data (coding
, from
+ inserted
);
5792 src
+= coding
->consumed
;
5793 dst
+= coding
->produced
;
5795 if (result
== CODING_FINISH_NORMAL
)
5800 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
5802 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
5803 Lisp_Object eol_type
;
5805 /* Encode LFs back to the original eol format (CR or CRLF). */
5806 if (coding
->eol_type
== CODING_EOL_CR
)
5808 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
5814 while (p
< pend
) if (*p
++ == '\n') count
++;
5815 if (src
- dst
< count
)
5817 /* We don't have sufficient room for encoding LFs
5818 back to CRLF. We must record converted and
5819 not-yet-converted text back to the buffer
5820 content, enlarge the gap, then record them out of
5821 the buffer contents again. */
5822 int add
= len_byte
+ inserted_byte
;
5825 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5826 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5827 make_gap (count
- GAP_SIZE
);
5829 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5830 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5831 /* Don't forget to update SRC, DST, and PEND. */
5832 src
= GAP_END_ADDR
- len_byte
;
5833 dst
= GPT_ADDR
+ inserted_byte
;
5837 inserted_byte
+= count
;
5838 coding
->produced
+= count
;
5839 p
= dst
= pend
+ count
;
5843 if (*p
== '\n') count
--, *--p
= '\r';
5847 /* Suppress eol-format conversion in the further conversion. */
5848 coding
->eol_type
= CODING_EOL_LF
;
5850 /* Set the coding system symbol to that for Unix-like EOL. */
5851 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5852 if (VECTORP (eol_type
)
5853 && XVECTOR (eol_type
)->size
== 3
5854 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
5855 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5857 coding
->symbol
= saved_coding_symbol
;
5863 if (coding
->type
!= coding_type_ccl
5864 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5866 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5869 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5871 /* The source text ends in invalid codes. Let's just
5872 make them valid buffer contents, and finish conversion. */
5875 unsigned char *start
= dst
;
5877 inserted
+= len_byte
;
5881 dst
+= CHAR_STRING (c
, dst
);
5884 inserted_byte
+= dst
- start
;
5888 inserted
+= len_byte
;
5889 inserted_byte
+= len_byte
;
5895 if (result
== CODING_FINISH_INTERRUPT
)
5897 /* The conversion procedure was interrupted by a user. */
5900 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5901 if (coding
->consumed
< 1)
5903 /* It's quite strange to require more memory without
5904 consuming any bytes. Perhaps CCL program bug. */
5909 /* We have just done the first batch of conversion which was
5910 stopped because of insufficient gap. Let's reconsider the
5911 required gap size (i.e. SRT - DST) now.
5913 We have converted ORIG bytes (== coding->consumed) into
5914 NEW bytes (coding->produced). To convert the remaining
5915 LEN bytes, we may need REQUIRE bytes of gap, where:
5916 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5917 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5918 Here, we are sure that NEW >= ORIG. */
5920 if (coding
->produced
<= coding
->consumed
)
5922 /* This happens because of CCL-based coding system with
5928 float ratio
= coding
->produced
- coding
->consumed
;
5929 ratio
/= coding
->consumed
;
5930 require
= len_byte
* ratio
;
5934 if ((src
- dst
) < (require
+ 2000))
5936 /* See the comment above the previous call of make_gap. */
5937 int add
= len_byte
+ inserted_byte
;
5940 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5941 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5942 make_gap (require
+ 2000);
5944 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5945 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5948 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5950 if (encodep
&& coding
->dst_multibyte
)
5952 /* The output is unibyte. We must convert 8-bit characters to
5954 if (inserted_byte
* 2 > GAP_SIZE
)
5956 GAP_SIZE
-= inserted_byte
;
5957 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5958 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5959 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5960 make_gap (inserted_byte
- GAP_SIZE
);
5961 GAP_SIZE
+= inserted_byte
;
5962 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5963 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5964 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5966 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5969 /* If we shrank the conversion area, adjust it now. */
5973 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5974 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5975 GAP_SIZE
+= total_skip
;
5976 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5977 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5978 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5979 from
-= head_skip
; from_byte
-= head_skip
;
5980 to
+= tail_skip
; to_byte
+= tail_skip
;
5984 if (! EQ (current_buffer
->undo_list
, Qt
))
5985 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5987 adjust_after_replace_noundo (from
, from_byte
, nchars_del
, nbytes_del
,
5988 inserted
, inserted_byte
);
5989 inserted
= Z
- prev_Z
;
5991 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5992 coding_restore_composition (coding
, Fcurrent_buffer ());
5993 coding_free_composition_data (coding
);
5995 if (! inhibit_pre_post_conversion
5996 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5999 Lisp_Object saved_coding_system
;
6002 TEMP_SET_PT_BOTH (from
, from_byte
);
6004 record_unwind_protect (code_convert_region_unwind
,
6005 Fcons (Vlast_coding_system_used
, Qnil
));
6006 saved_coding_system
= Vlast_coding_system_used
;
6007 Vlast_coding_system_used
= coding
->symbol
;
6008 /* We should not call any more pre-write/post-read-conversion
6009 functions while this post-read-conversion is running. */
6010 inhibit_pre_post_conversion
= 1;
6011 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
6012 inhibit_pre_post_conversion
= 0;
6013 coding
->symbol
= Vlast_coding_system_used
;
6014 Vlast_coding_system_used
= saved_coding_system
;
6015 /* Discard the unwind protect. */
6018 inserted
+= Z
- prev_Z
;
6021 if (orig_point
>= from
)
6023 if (orig_point
>= from
+ orig_len
)
6024 orig_point
+= inserted
- orig_len
;
6027 TEMP_SET_PT (orig_point
);
6032 signal_after_change (from
, to
- from
, inserted
);
6033 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
6037 coding
->consumed
= to_byte
- from_byte
;
6038 coding
->consumed_char
= to
- from
;
6039 coding
->produced
= inserted_byte
;
6040 coding
->produced_char
= inserted
;
6046 /* Name (or base name) of work buffer for code conversion. */
6047 static Lisp_Object Vcode_conversion_workbuf_name
;
6049 /* Set the current buffer to the working buffer prepared for
6050 code-conversion. MULTIBYTE specifies the multibyteness of the
6051 buffer. Return the buffer we set if it must be killed after use.
6052 Otherwise return Qnil. */
6055 set_conversion_work_buffer (multibyte
)
6058 Lisp_Object buffer
, buffer_to_kill
;
6061 buffer
= Fget_buffer_create (Vcode_conversion_workbuf_name
);
6062 buf
= XBUFFER (buffer
);
6063 if (buf
== current_buffer
)
6065 /* As we are already in the work buffer, we must generate a new
6066 buffer for the work. */
6069 name
= Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name
, Qnil
);
6070 buffer
= buffer_to_kill
= Fget_buffer_create (name
);
6071 buf
= XBUFFER (buffer
);
6074 buffer_to_kill
= Qnil
;
6076 delete_all_overlays (buf
);
6077 buf
->directory
= current_buffer
->directory
;
6078 buf
->read_only
= Qnil
;
6079 buf
->filename
= Qnil
;
6080 buf
->undo_list
= Qt
;
6081 eassert (buf
->overlays_before
== NULL
);
6082 eassert (buf
->overlays_after
== NULL
);
6083 set_buffer_internal (buf
);
6084 if (BEG
!= BEGV
|| Z
!= ZV
)
6086 del_range_2 (BEG
, BEG_BYTE
, Z
, Z_BYTE
, 0);
6087 buf
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
6088 return buffer_to_kill
;
6092 run_pre_post_conversion_on_str (str
, coding
, encodep
)
6094 struct coding_system
*coding
;
6097 int count
= SPECPDL_INDEX ();
6098 struct gcpro gcpro1
, gcpro2
;
6099 int multibyte
= STRING_MULTIBYTE (str
);
6100 Lisp_Object old_deactivate_mark
;
6101 Lisp_Object buffer_to_kill
;
6102 Lisp_Object unwind_arg
;
6104 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
6105 /* It is not crucial to specbind this. */
6106 old_deactivate_mark
= Vdeactivate_mark
;
6107 GCPRO2 (str
, old_deactivate_mark
);
6109 /* We must insert the contents of STR as is without
6110 unibyte<->multibyte conversion. For that, we adjust the
6111 multibyteness of the working buffer to that of STR. */
6112 buffer_to_kill
= set_conversion_work_buffer (multibyte
);
6113 if (NILP (buffer_to_kill
))
6114 unwind_arg
= Fcons (Vlast_coding_system_used
, Qnil
);
6116 unwind_arg
= list2 (Vlast_coding_system_used
, buffer_to_kill
);
6117 record_unwind_protect (code_convert_region_unwind
, unwind_arg
);
6119 insert_from_string (str
, 0, 0,
6120 SCHARS (str
), SBYTES (str
), 0);
6122 inhibit_pre_post_conversion
= 1;
6125 struct buffer
*prev
= current_buffer
;
6127 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
6128 if (prev
!= current_buffer
)
6129 /* We must kill the current buffer too. */
6130 Fsetcdr (unwind_arg
, Fcons (Fcurrent_buffer (), XCDR (unwind_arg
)));
6134 Vlast_coding_system_used
= coding
->symbol
;
6135 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
6136 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
6137 coding
->symbol
= Vlast_coding_system_used
;
6139 inhibit_pre_post_conversion
= 0;
6140 Vdeactivate_mark
= old_deactivate_mark
;
6141 str
= make_buffer_string (BEG
, Z
, 1);
6142 return unbind_to (count
, str
);
6146 /* Run pre-write-conversion function of CODING on NCHARS/NBYTES
6147 text in *STR. *SIZE is the allocated bytes for STR. As it
6148 is intended that this function is called from encode_terminal_code,
6149 the pre-write-conversion function is run by safe_call and thus
6150 "Error during redisplay: ..." is logged when an error occurs.
6152 Store the resulting text in *STR and set CODING->produced_char and
6153 CODING->produced to the number of characters and bytes
6154 respectively. If the size of *STR is too small, enlarge it by
6155 xrealloc and update *STR and *SIZE. */
6158 run_pre_write_conversin_on_c_str (str
, size
, nchars
, nbytes
, coding
)
6159 unsigned char **str
;
6160 int *size
, nchars
, nbytes
;
6161 struct coding_system
*coding
;
6163 struct gcpro gcpro1
, gcpro2
;
6164 struct buffer
*cur
= current_buffer
;
6165 struct buffer
*prev
;
6166 Lisp_Object old_deactivate_mark
, old_last_coding_system_used
;
6167 Lisp_Object args
[3];
6168 Lisp_Object buffer_to_kill
;
6170 /* It is not crucial to specbind this. */
6171 old_deactivate_mark
= Vdeactivate_mark
;
6172 old_last_coding_system_used
= Vlast_coding_system_used
;
6173 GCPRO2 (old_deactivate_mark
, old_last_coding_system_used
);
6175 /* We must insert the contents of STR as is without
6176 unibyte<->multibyte conversion. For that, we adjust the
6177 multibyteness of the working buffer to that of STR. */
6178 buffer_to_kill
= set_conversion_work_buffer (coding
->src_multibyte
);
6179 insert_1_both (*str
, nchars
, nbytes
, 0, 0, 0);
6181 inhibit_pre_post_conversion
= 1;
6182 prev
= current_buffer
;
6183 args
[0] = coding
->pre_write_conversion
;
6184 args
[1] = make_number (BEG
);
6185 args
[2] = make_number (Z
);
6186 safe_call (3, args
);
6187 inhibit_pre_post_conversion
= 0;
6188 Vdeactivate_mark
= old_deactivate_mark
;
6189 Vlast_coding_system_used
= old_last_coding_system_used
;
6190 coding
->produced_char
= Z
- BEG
;
6191 coding
->produced
= Z_BYTE
- BEG_BYTE
;
6192 if (coding
->produced
> *size
)
6194 *size
= coding
->produced
;
6195 *str
= xrealloc (*str
, *size
);
6197 if (BEG
< GPT
&& GPT
< Z
)
6199 bcopy (BEG_ADDR
, *str
, coding
->produced
);
6200 coding
->src_multibyte
6201 = ! NILP (current_buffer
->enable_multibyte_characters
);
6202 if (prev
!= current_buffer
)
6203 Fkill_buffer (Fcurrent_buffer ());
6204 set_buffer_internal (cur
);
6205 if (! NILP (buffer_to_kill
))
6206 Fkill_buffer (buffer_to_kill
);
6211 decode_coding_string (str
, coding
, nocopy
)
6213 struct coding_system
*coding
;
6217 struct conversion_buffer buf
;
6219 Lisp_Object saved_coding_symbol
;
6221 int require_decoding
;
6222 int shrinked_bytes
= 0;
6224 int consumed
, consumed_char
, produced
, produced_char
;
6227 to_byte
= SBYTES (str
);
6229 saved_coding_symbol
= coding
->symbol
;
6230 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6231 coding
->dst_multibyte
= 1;
6232 if (CODING_REQUIRE_DETECTION (coding
))
6234 /* See the comments in code_convert_region. */
6235 if (coding
->type
== coding_type_undecided
)
6237 detect_coding (coding
, SDATA (str
), to_byte
);
6238 if (coding
->type
== coding_type_undecided
)
6240 coding
->type
= coding_type_emacs_mule
;
6241 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
6242 /* As emacs-mule decoder will handle composition, we
6243 need this setting to allocate coding->cmp_data
6245 coding
->composing
= COMPOSITION_NO
;
6248 if (coding
->eol_type
== CODING_EOL_UNDECIDED
6249 && coding
->type
!= coding_type_ccl
)
6251 saved_coding_symbol
= coding
->symbol
;
6252 detect_eol (coding
, SDATA (str
), to_byte
);
6253 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
6254 coding
->eol_type
= CODING_EOL_LF
;
6255 /* We had better recover the original eol format if we
6256 encounter an inconsistent eol format while decoding. */
6257 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
6261 if (coding
->type
== coding_type_no_conversion
6262 || coding
->type
== coding_type_raw_text
)
6263 coding
->dst_multibyte
= 0;
6265 require_decoding
= CODING_REQUIRE_DECODING (coding
);
6267 if (STRING_MULTIBYTE (str
))
6269 /* Decoding routines expect the source text to be unibyte. */
6270 str
= Fstring_as_unibyte (str
);
6271 to_byte
= SBYTES (str
);
6273 coding
->src_multibyte
= 0;
6276 /* Try to skip the heading and tailing ASCIIs. */
6277 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
6279 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6281 if (from
== to_byte
)
6282 require_decoding
= 0;
6283 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6286 if (!require_decoding
6287 && !(SYMBOLP (coding
->post_read_conversion
)
6288 && !NILP (Ffboundp (coding
->post_read_conversion
))))
6290 coding
->consumed
= SBYTES (str
);
6291 coding
->consumed_char
= SCHARS (str
);
6292 if (coding
->dst_multibyte
)
6294 str
= Fstring_as_multibyte (str
);
6297 coding
->produced
= SBYTES (str
);
6298 coding
->produced_char
= SCHARS (str
);
6299 return (nocopy
? str
: Fcopy_sequence (str
));
6302 if (coding
->composing
!= COMPOSITION_DISABLED
)
6303 coding_allocate_composition_data (coding
, from
);
6304 len
= decoding_buffer_size (coding
, to_byte
- from
);
6305 allocate_conversion_buffer (buf
, len
);
6307 consumed
= consumed_char
= produced
= produced_char
= 0;
6310 result
= decode_coding (coding
, SDATA (str
) + from
+ consumed
,
6311 buf
.data
+ produced
, to_byte
- from
- consumed
,
6312 buf
.size
- produced
);
6313 consumed
+= coding
->consumed
;
6314 consumed_char
+= coding
->consumed_char
;
6315 produced
+= coding
->produced
;
6316 produced_char
+= coding
->produced_char
;
6317 if (result
== CODING_FINISH_NORMAL
6318 || result
== CODING_FINISH_INTERRUPT
6319 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6320 && coding
->consumed
== 0))
6322 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
6323 coding_allocate_composition_data (coding
, from
+ produced_char
);
6324 else if (result
== CODING_FINISH_INSUFFICIENT_DST
)
6325 extend_conversion_buffer (&buf
);
6326 else if (result
== CODING_FINISH_INCONSISTENT_EOL
)
6328 Lisp_Object eol_type
;
6330 /* Recover the original EOL format. */
6331 if (coding
->eol_type
== CODING_EOL_CR
)
6334 for (p
= buf
.data
; p
< buf
.data
+ produced
; p
++)
6335 if (*p
== '\n') *p
= '\r';
6337 else if (coding
->eol_type
== CODING_EOL_CRLF
)
6340 unsigned char *p0
, *p1
;
6341 for (p0
= buf
.data
, p1
= p0
+ produced
; p0
< p1
; p0
++)
6342 if (*p0
== '\n') num_eol
++;
6343 if (produced
+ num_eol
>= buf
.size
)
6344 extend_conversion_buffer (&buf
);
6345 for (p0
= buf
.data
+ produced
, p1
= p0
+ num_eol
; p0
> buf
.data
;)
6348 if (*p0
== '\n') *--p1
= '\r';
6350 produced
+= num_eol
;
6351 produced_char
+= num_eol
;
6353 /* Suppress eol-format conversion in the further conversion. */
6354 coding
->eol_type
= CODING_EOL_LF
;
6356 /* Set the coding system symbol to that for Unix-like EOL. */
6357 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
6358 if (VECTORP (eol_type
)
6359 && XVECTOR (eol_type
)->size
== 3
6360 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
6361 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
6363 coding
->symbol
= saved_coding_symbol
;
6369 coding
->consumed
= consumed
;
6370 coding
->consumed_char
= consumed_char
;
6371 coding
->produced
= produced
;
6372 coding
->produced_char
= produced_char
;
6374 if (coding
->dst_multibyte
)
6375 newstr
= make_uninit_multibyte_string (produced_char
+ shrinked_bytes
,
6376 produced
+ shrinked_bytes
);
6378 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6380 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6381 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6382 if (shrinked_bytes
> from
)
6383 STRING_COPYIN (newstr
, from
+ produced
,
6384 SDATA (str
) + to_byte
,
6385 shrinked_bytes
- from
);
6386 free_conversion_buffer (&buf
);
6388 coding
->consumed
+= shrinked_bytes
;
6389 coding
->consumed_char
+= shrinked_bytes
;
6390 coding
->produced
+= shrinked_bytes
;
6391 coding
->produced_char
+= shrinked_bytes
;
6393 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
6394 coding_restore_composition (coding
, newstr
);
6395 coding_free_composition_data (coding
);
6397 if (SYMBOLP (coding
->post_read_conversion
)
6398 && !NILP (Ffboundp (coding
->post_read_conversion
)))
6399 newstr
= run_pre_post_conversion_on_str (newstr
, coding
, 0);
6405 encode_coding_string (str
, coding
, nocopy
)
6407 struct coding_system
*coding
;
6411 struct conversion_buffer buf
;
6412 int from
, to
, to_byte
;
6414 int shrinked_bytes
= 0;
6416 int consumed
, consumed_char
, produced
, produced_char
;
6418 if (SYMBOLP (coding
->pre_write_conversion
)
6419 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
6421 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
6422 /* As STR is just newly generated, we don't have to copy it
6429 to_byte
= SBYTES (str
);
6431 /* Encoding routines determine the multibyteness of the source text
6432 by coding->src_multibyte. */
6433 coding
->src_multibyte
= SCHARS (str
) < SBYTES (str
);
6434 coding
->dst_multibyte
= 0;
6435 if (! CODING_REQUIRE_ENCODING (coding
))
6436 goto no_need_of_encoding
;
6438 if (coding
->composing
!= COMPOSITION_DISABLED
)
6439 coding_save_composition (coding
, from
, to
, str
);
6441 /* Try to skip the heading and tailing ASCIIs. We can't skip them
6442 if we must run CCL program or there are compositions to
6444 if (coding
->type
!= coding_type_ccl
6445 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
6447 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6449 if (from
== to_byte
)
6451 coding_free_composition_data (coding
);
6452 goto no_need_of_encoding
;
6454 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6457 len
= encoding_buffer_size (coding
, to_byte
- from
);
6458 allocate_conversion_buffer (buf
, len
);
6460 consumed
= consumed_char
= produced
= produced_char
= 0;
6463 result
= encode_coding (coding
, SDATA (str
) + from
+ consumed
,
6464 buf
.data
+ produced
, to_byte
- from
- consumed
,
6465 buf
.size
- produced
);
6466 consumed
+= coding
->consumed
;
6467 consumed_char
+= coding
->consumed_char
;
6468 produced
+= coding
->produced
;
6469 produced_char
+= coding
->produced_char
;
6470 if (result
== CODING_FINISH_NORMAL
6471 || result
== CODING_FINISH_INTERRUPT
6472 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6473 && coding
->consumed
== 0))
6475 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6476 extend_conversion_buffer (&buf
);
6479 coding
->consumed
= consumed
;
6480 coding
->consumed_char
= consumed_char
;
6481 coding
->produced
= produced
;
6482 coding
->produced_char
= produced_char
;
6484 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6486 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6487 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6488 if (shrinked_bytes
> from
)
6489 STRING_COPYIN (newstr
, from
+ produced
,
6490 SDATA (str
) + to_byte
,
6491 shrinked_bytes
- from
);
6493 free_conversion_buffer (&buf
);
6494 coding_free_composition_data (coding
);
6498 no_need_of_encoding
:
6499 coding
->consumed
= SBYTES (str
);
6500 coding
->consumed_char
= SCHARS (str
);
6501 if (STRING_MULTIBYTE (str
))
6504 /* We are sure that STR doesn't contain a multibyte
6506 STRING_SET_UNIBYTE (str
);
6509 str
= Fstring_as_unibyte (str
);
6513 coding
->produced
= SBYTES (str
);
6514 coding
->produced_char
= SCHARS (str
);
6515 return (nocopy
? str
: Fcopy_sequence (str
));
6520 /*** 8. Emacs Lisp library functions ***/
6522 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
6523 doc
: /* Return t if OBJECT is nil or a coding-system.
6524 See the documentation of `make-coding-system' for information
6525 about coding-system objects. */)
6533 if (! NILP (Fget (obj
, Qcoding_system_define_form
)))
6535 /* Get coding-spec vector for OBJ. */
6536 obj
= Fget (obj
, Qcoding_system
);
6537 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
6541 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
6542 Sread_non_nil_coding_system
, 1, 1, 0,
6543 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6550 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6551 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
6553 while (SCHARS (val
) == 0);
6554 return (Fintern (val
, Qnil
));
6557 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
6558 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6559 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
6560 (prompt
, default_coding_system
)
6561 Lisp_Object prompt
, default_coding_system
;
6564 if (SYMBOLP (default_coding_system
))
6565 default_coding_system
= SYMBOL_NAME (default_coding_system
);
6566 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6567 Qt
, Qnil
, Qcoding_system_history
,
6568 default_coding_system
, Qnil
);
6569 return (SCHARS (val
) == 0 ? Qnil
: Fintern (val
, Qnil
));
6572 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
6574 doc
: /* Check validity of CODING-SYSTEM.
6575 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6576 It is valid if it is nil or a symbol with a non-nil `coding-system' property.
6577 The value of this property should be a vector of length 5. */)
6579 Lisp_Object coding_system
;
6581 Lisp_Object define_form
;
6583 define_form
= Fget (coding_system
, Qcoding_system_define_form
);
6584 if (! NILP (define_form
))
6586 Fput (coding_system
, Qcoding_system_define_form
, Qnil
);
6587 safe_eval (define_form
);
6589 if (!NILP (Fcoding_system_p (coding_system
)))
6590 return coding_system
;
6592 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
6596 detect_coding_system (src
, src_bytes
, highest
, multibytep
)
6597 const unsigned char *src
;
6598 int src_bytes
, highest
;
6601 int coding_mask
, eol_type
;
6602 Lisp_Object val
, tmp
;
6605 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
, multibytep
);
6606 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
6607 if (eol_type
== CODING_EOL_INCONSISTENT
)
6608 eol_type
= CODING_EOL_UNDECIDED
;
6613 if (eol_type
!= CODING_EOL_UNDECIDED
)
6616 val2
= Fget (Qundecided
, Qeol_type
);
6618 val
= XVECTOR (val2
)->contents
[eol_type
];
6620 return (highest
? val
: Fcons (val
, Qnil
));
6623 /* At first, gather possible coding systems in VAL. */
6625 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
6627 Lisp_Object category_val
, category_index
;
6629 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
6630 category_val
= Fsymbol_value (XCAR (tmp
));
6631 if (!NILP (category_val
)
6632 && NATNUMP (category_index
)
6633 && (coding_mask
& (1 << XFASTINT (category_index
))))
6635 val
= Fcons (category_val
, val
);
6641 val
= Fnreverse (val
);
6643 /* Then, replace the elements with subsidiary coding systems. */
6644 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
6646 if (eol_type
!= CODING_EOL_UNDECIDED
6647 && eol_type
!= CODING_EOL_INCONSISTENT
)
6650 eol
= Fget (XCAR (tmp
), Qeol_type
);
6652 XSETCAR (tmp
, XVECTOR (eol
)->contents
[eol_type
]);
6655 return (highest
? XCAR (val
) : val
);
6658 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
6660 doc
: /* Detect how the byte sequence in the region is encoded.
6661 Return a list of possible coding systems used on decoding a byte
6662 sequence containing the bytes in the region between START and END when
6663 the coding system `undecided' is specified. The list is ordered by
6664 priority decided in the current language environment.
6666 If only ASCII characters are found, it returns a list of single element
6667 `undecided' or its subsidiary coding system according to a detected
6670 If optional argument HIGHEST is non-nil, return the coding system of
6671 highest priority. */)
6672 (start
, end
, highest
)
6673 Lisp_Object start
, end
, highest
;
6676 int from_byte
, to_byte
;
6677 int include_anchor_byte
= 0;
6679 CHECK_NUMBER_COERCE_MARKER (start
);
6680 CHECK_NUMBER_COERCE_MARKER (end
);
6682 validate_region (&start
, &end
);
6683 from
= XINT (start
), to
= XINT (end
);
6684 from_byte
= CHAR_TO_BYTE (from
);
6685 to_byte
= CHAR_TO_BYTE (to
);
6687 if (from
< GPT
&& to
>= GPT
)
6688 move_gap_both (to
, to_byte
);
6689 /* If we an anchor byte `\0' follows the region, we include it in
6690 the detecting source. Then code detectors can handle the tailing
6691 byte sequence more accurately.
6693 Fix me: This is not a perfect solution. It is better that we
6694 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6696 if (to
== Z
|| (to
== GPT
&& GAP_SIZE
> 0))
6697 include_anchor_byte
= 1;
6698 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
6699 to_byte
- from_byte
+ include_anchor_byte
,
6701 !NILP (current_buffer
6702 ->enable_multibyte_characters
));
6705 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
6707 doc
: /* Detect how the byte sequence in STRING is encoded.
6708 Return a list of possible coding systems used on decoding a byte
6709 sequence containing the bytes in STRING when the coding system
6710 `undecided' is specified. The list is ordered by priority decided in
6711 the current language environment.
6713 If only ASCII characters are found, it returns a list of single element
6714 `undecided' or its subsidiary coding system according to a detected
6717 If optional argument HIGHEST is non-nil, return the coding system of
6718 highest priority. */)
6720 Lisp_Object string
, highest
;
6722 CHECK_STRING (string
);
6724 return detect_coding_system (SDATA (string
),
6725 /* "+ 1" is to include the anchor byte
6726 `\0'. With this, code detectors can
6727 handle the tailing bytes more
6729 SBYTES (string
) + 1,
6731 STRING_MULTIBYTE (string
));
6734 /* Subroutine for Ffind_coding_systems_region_internal.
6736 Return a list of coding systems that safely encode the multibyte
6737 text between P and PEND. SAFE_CODINGS, if non-nil, is an alist of
6738 possible coding systems. If it is nil, it means that we have not
6739 yet found any coding systems.
6741 WORK_TABLE a char-table of which element is set to t once the
6742 element is looked up.
6744 If a non-ASCII single byte char is found, set
6745 *single_byte_char_found to 1. */
6748 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
6749 unsigned char *p
, *pend
;
6750 Lisp_Object safe_codings
, work_table
;
6751 int *single_byte_char_found
;
6754 Lisp_Object val
, ch
;
6755 Lisp_Object prev
, tail
;
6757 if (NILP (safe_codings
))
6758 goto done_safe_codings
;
6761 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6763 if (ASCII_BYTE_P (c
))
6764 /* We can ignore ASCII characters here. */
6766 if (SINGLE_BYTE_CHAR_P (c
))
6767 *single_byte_char_found
= 1;
6768 /* Check the safe coding systems for C. */
6769 ch
= make_number (c
);
6770 val
= Faref (work_table
, ch
);
6772 /* This element was already checked. Ignore it. */
6774 /* Remember that we checked this element. */
6775 Faset (work_table
, ch
, Qt
);
6777 for (prev
= tail
= safe_codings
; CONSP (tail
); tail
= XCDR (tail
))
6779 Lisp_Object elt
, translation_table
, hash_table
, accept_latin_extra
;
6783 if (CONSP (XCDR (elt
)))
6785 /* This entry has this format now:
6786 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6787 ACCEPT-LATIN-EXTRA ) */
6789 encodable
= ! NILP (Faref (XCAR (val
), ch
));
6793 translation_table
= XCAR (val
);
6794 hash_table
= XCAR (XCDR (val
));
6795 accept_latin_extra
= XCAR (XCDR (XCDR (val
)));
6800 /* This entry has this format now: ( CODING . SAFE-CHARS) */
6801 encodable
= ! NILP (Faref (XCDR (elt
), ch
));
6804 /* Transform the format to:
6805 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6806 ACCEPT-LATIN-EXTRA ) */
6807 val
= Fget (XCAR (elt
), Qcoding_system
);
6809 = Fplist_get (AREF (val
, 3),
6810 Qtranslation_table_for_encode
);
6811 if (SYMBOLP (translation_table
))
6812 translation_table
= Fget (translation_table
,
6813 Qtranslation_table
);
6815 = (CHAR_TABLE_P (translation_table
)
6816 ? XCHAR_TABLE (translation_table
)->extras
[1]
6819 = ((EQ (AREF (val
, 0), make_number (2))
6820 && VECTORP (AREF (val
, 4)))
6821 ? AREF (AREF (val
, 4), 16)
6823 XSETCAR (tail
, list5 (XCAR (elt
), XCDR (elt
),
6824 translation_table
, hash_table
,
6825 accept_latin_extra
));
6830 && ((CHAR_TABLE_P (translation_table
)
6831 && ! NILP (Faref (translation_table
, ch
)))
6832 || (HASH_TABLE_P (hash_table
)
6833 && ! NILP (Fgethash (ch
, hash_table
, Qnil
)))
6834 || (SINGLE_BYTE_CHAR_P (c
)
6835 && ! NILP (accept_latin_extra
)
6836 && VECTORP (Vlatin_extra_code_table
)
6837 && ! NILP (AREF (Vlatin_extra_code_table
, c
)))))
6843 /* Exclude this coding system from SAFE_CODINGS. */
6844 if (EQ (tail
, safe_codings
))
6846 safe_codings
= XCDR (safe_codings
);
6847 if (NILP (safe_codings
))
6848 goto done_safe_codings
;
6851 XSETCDR (prev
, XCDR (tail
));
6857 /* If the above loop was terminated before P reaches PEND, it means
6858 SAFE_CODINGS was set to nil. If we have not yet found an
6859 non-ASCII single-byte char, check it now. */
6860 if (! *single_byte_char_found
)
6863 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6865 if (! ASCII_BYTE_P (c
)
6866 && SINGLE_BYTE_CHAR_P (c
))
6868 *single_byte_char_found
= 1;
6872 return safe_codings
;
6875 DEFUN ("find-coding-systems-region-internal",
6876 Ffind_coding_systems_region_internal
,
6877 Sfind_coding_systems_region_internal
, 2, 2, 0,
6878 doc
: /* Internal use only. */)
6880 Lisp_Object start
, end
;
6882 Lisp_Object work_table
, safe_codings
;
6883 int non_ascii_p
= 0;
6884 int single_byte_char_found
= 0;
6885 const unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
6887 if (STRINGP (start
))
6889 if (!STRING_MULTIBYTE (start
))
6891 p1
= SDATA (start
), p1end
= p1
+ SBYTES (start
);
6893 if (SCHARS (start
) != SBYTES (start
))
6900 CHECK_NUMBER_COERCE_MARKER (start
);
6901 CHECK_NUMBER_COERCE_MARKER (end
);
6902 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
6903 args_out_of_range (start
, end
);
6904 if (NILP (current_buffer
->enable_multibyte_characters
))
6906 from
= CHAR_TO_BYTE (XINT (start
));
6907 to
= CHAR_TO_BYTE (XINT (end
));
6908 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
6909 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
6913 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
6914 if (XINT (end
) - XINT (start
) != to
- from
)
6920 /* We are sure that the text contains no multibyte character.
6921 Check if it contains eight-bit-graphic. */
6923 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
6926 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
6932 /* The text contains non-ASCII characters. */
6934 work_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
6935 safe_codings
= Fcopy_sequence (XCDR (Vcoding_system_safe_chars
));
6937 safe_codings
= find_safe_codings (p1
, p1end
, safe_codings
, work_table
,
6938 &single_byte_char_found
);
6940 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
6941 &single_byte_char_found
);
6942 if (EQ (safe_codings
, XCDR (Vcoding_system_safe_chars
)))
6946 /* Turn safe_codings to a list of coding systems... */
6949 if (single_byte_char_found
)
6950 /* ... and append these for eight-bit chars. */
6951 val
= Fcons (Qraw_text
,
6952 Fcons (Qemacs_mule
, Fcons (Qno_conversion
, Qnil
)));
6954 /* ... and append generic coding systems. */
6955 val
= Fcopy_sequence (XCAR (Vcoding_system_safe_chars
));
6957 for (; CONSP (safe_codings
); safe_codings
= XCDR (safe_codings
))
6958 val
= Fcons (XCAR (XCAR (safe_codings
)), val
);
6962 return safe_codings
;
6966 /* Search from position POS for such characters that are unencodable
6967 accoding to SAFE_CHARS, and return a list of their positions. P
6968 points where in the memory the character at POS exists. Limit the
6969 search at PEND or when Nth unencodable characters are found.
6971 If SAFE_CHARS is a char table, an element for an unencodable
6974 If SAFE_CHARS is nil, all non-ASCII characters are unencodable.
6976 Otherwise, SAFE_CHARS is t, and only eight-bit-contrl and
6977 eight-bit-graphic characters are unencodable. */
6980 unencodable_char_position (safe_chars
, pos
, p
, pend
, n
)
6981 Lisp_Object safe_chars
;
6983 unsigned char *p
, *pend
;
6986 Lisp_Object pos_list
;
6992 int c
= STRING_CHAR_AND_LENGTH (p
, MAX_MULTIBYTE_LENGTH
, len
);
6995 && (CHAR_TABLE_P (safe_chars
)
6996 ? NILP (CHAR_TABLE_REF (safe_chars
, c
))
6997 : (NILP (safe_chars
) || c
< 256)))
6999 pos_list
= Fcons (make_number (pos
), pos_list
);
7006 return Fnreverse (pos_list
);
7010 DEFUN ("unencodable-char-position", Funencodable_char_position
,
7011 Sunencodable_char_position
, 3, 5, 0,
7013 Return position of first un-encodable character in a region.
7014 START and END specfiy the region and CODING-SYSTEM specifies the
7015 encoding to check. Return nil if CODING-SYSTEM does encode the region.
7017 If optional 4th argument COUNT is non-nil, it specifies at most how
7018 many un-encodable characters to search. In this case, the value is a
7021 If optional 5th argument STRING is non-nil, it is a string to search
7022 for un-encodable characters. In that case, START and END are indexes
7024 (start
, end
, coding_system
, count
, string
)
7025 Lisp_Object start
, end
, coding_system
, count
, string
;
7028 Lisp_Object safe_chars
;
7029 struct coding_system coding
;
7030 Lisp_Object positions
;
7032 unsigned char *p
, *pend
;
7036 validate_region (&start
, &end
);
7037 from
= XINT (start
);
7039 if (NILP (current_buffer
->enable_multibyte_characters
))
7041 p
= CHAR_POS_ADDR (from
);
7045 pend
= CHAR_POS_ADDR (to
);
7049 CHECK_STRING (string
);
7050 CHECK_NATNUM (start
);
7052 from
= XINT (start
);
7055 || to
> SCHARS (string
))
7056 args_out_of_range_3 (string
, start
, end
);
7057 if (! STRING_MULTIBYTE (string
))
7059 p
= SDATA (string
) + string_char_to_byte (string
, from
);
7060 pend
= SDATA (string
) + string_char_to_byte (string
, to
);
7063 setup_coding_system (Fcheck_coding_system (coding_system
), &coding
);
7069 CHECK_NATNUM (count
);
7073 if (coding
.type
== coding_type_no_conversion
7074 || coding
.type
== coding_type_raw_text
)
7077 if (coding
.type
== coding_type_undecided
)
7080 safe_chars
= coding_safe_chars (coding_system
);
7082 if (STRINGP (string
)
7083 || from
>= GPT
|| to
<= GPT
)
7084 positions
= unencodable_char_position (safe_chars
, from
, p
, pend
, n
);
7087 Lisp_Object args
[2];
7089 args
[0] = unencodable_char_position (safe_chars
, from
, p
, GPT_ADDR
, n
);
7090 n
-= XINT (Flength (args
[0]));
7092 positions
= args
[0];
7095 args
[1] = unencodable_char_position (safe_chars
, GPT
, GAP_END_ADDR
,
7097 positions
= Fappend (2, args
);
7101 return (NILP (count
) ? Fcar (positions
) : positions
);
7106 code_convert_region1 (start
, end
, coding_system
, encodep
)
7107 Lisp_Object start
, end
, coding_system
;
7110 struct coding_system coding
;
7113 CHECK_NUMBER_COERCE_MARKER (start
);
7114 CHECK_NUMBER_COERCE_MARKER (end
);
7115 CHECK_SYMBOL (coding_system
);
7117 validate_region (&start
, &end
);
7118 from
= XFASTINT (start
);
7119 to
= XFASTINT (end
);
7121 if (NILP (coding_system
) && system_eol_type
== CODING_EOL_LF
)
7122 return make_number (to
- from
);
7124 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7125 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7127 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7128 coding
.src_multibyte
= coding
.dst_multibyte
7129 = !NILP (current_buffer
->enable_multibyte_characters
);
7130 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
7131 &coding
, encodep
, 1);
7132 Vlast_coding_system_used
= coding
.symbol
;
7133 return make_number (coding
.produced_char
);
7136 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
7137 3, 3, "r\nzCoding system: ",
7138 doc
: /* Decode the current region from the specified coding system.
7139 When called from a program, takes three arguments:
7140 START, END, and CODING-SYSTEM. START and END are buffer positions.
7141 This function sets `last-coding-system-used' to the precise coding system
7142 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7143 not fully specified.)
7144 It returns the length of the decoded text. */)
7145 (start
, end
, coding_system
)
7146 Lisp_Object start
, end
, coding_system
;
7148 return code_convert_region1 (start
, end
, coding_system
, 0);
7151 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
7152 3, 3, "r\nzCoding system: ",
7153 doc
: /* Encode the current region into the specified coding system.
7154 When called from a program, takes three arguments:
7155 START, END, and CODING-SYSTEM. START and END are buffer positions.
7156 This function sets `last-coding-system-used' to the precise coding system
7157 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7158 not fully specified.)
7159 It returns the length of the encoded text. */)
7160 (start
, end
, coding_system
)
7161 Lisp_Object start
, end
, coding_system
;
7163 return code_convert_region1 (start
, end
, coding_system
, 1);
7167 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
7168 Lisp_Object string
, coding_system
, nocopy
;
7171 struct coding_system coding
;
7173 CHECK_STRING (string
);
7174 CHECK_SYMBOL (coding_system
);
7176 if (NILP (coding_system
) && system_eol_type
== CODING_EOL_LF
)
7177 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
7179 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7180 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7182 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7184 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
7185 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
7186 Vlast_coding_system_used
= coding
.symbol
;
7191 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
7193 doc
: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7194 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7195 if the decoding operation is trivial.
7196 This function sets `last-coding-system-used' to the precise coding system
7197 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7198 not fully specified.) */)
7199 (string
, coding_system
, nocopy
)
7200 Lisp_Object string
, coding_system
, nocopy
;
7202 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
7205 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
7207 doc
: /* Encode STRING to CODING-SYSTEM, and return the result.
7208 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7209 if the encoding operation is trivial.
7210 This function sets `last-coding-system-used' to the precise coding system
7211 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7212 not fully specified.) */)
7213 (string
, coding_system
, nocopy
)
7214 Lisp_Object string
, coding_system
, nocopy
;
7216 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
7219 /* Encode or decode STRING according to CODING_SYSTEM.
7220 Do not set Vlast_coding_system_used.
7222 This function is called only from macros DECODE_FILE and
7223 ENCODE_FILE, thus we ignore character composition. */
7226 code_convert_string_norecord (string
, coding_system
, encodep
)
7227 Lisp_Object string
, coding_system
;
7230 struct coding_system coding
;
7232 CHECK_STRING (string
);
7233 CHECK_SYMBOL (coding_system
);
7235 if (NILP (coding_system
) && system_eol_type
== CODING_EOL_LF
)
7238 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7239 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7241 coding
.composing
= COMPOSITION_DISABLED
;
7242 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7244 ? encode_coding_string (string
, &coding
, 1)
7245 : decode_coding_string (string
, &coding
, 1));
7248 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
7249 doc
: /* Decode a Japanese character which has CODE in shift_jis encoding.
7250 Return the corresponding character. */)
7254 unsigned char c1
, c2
, s1
, s2
;
7257 CHECK_NUMBER (code
);
7258 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
7262 XSETFASTINT (val
, s2
);
7263 else if (s2
>= 0xA0 || s2
<= 0xDF)
7264 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
7266 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7270 if ((s1
< 0x80 || (s1
> 0x9F && s1
< 0xE0) || s1
> 0xEF)
7271 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
7272 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7273 DECODE_SJIS (s1
, s2
, c1
, c2
);
7274 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
7279 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
7280 doc
: /* Encode a Japanese character CHAR to shift_jis encoding.
7281 Return the corresponding code in SJIS. */)
7285 int charset
, c1
, c2
, s1
, s2
;
7289 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7290 if (charset
== CHARSET_ASCII
)
7294 else if (charset
== charset_jisx0208
7295 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
7297 ENCODE_SJIS (c1
, c2
, s1
, s2
);
7298 XSETFASTINT (val
, (s1
<< 8) | s2
);
7300 else if (charset
== charset_katakana_jisx0201
7301 && c1
> 0x20 && c2
< 0xE0)
7303 XSETFASTINT (val
, c1
| 0x80);
7306 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
7310 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
7311 doc
: /* Decode a Big5 character which has CODE in BIG5 coding system.
7312 Return the corresponding character. */)
7317 unsigned char b1
, b2
, c1
, c2
;
7320 CHECK_NUMBER (code
);
7321 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
7325 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7330 if ((b1
< 0xA1 || b1
> 0xFE)
7331 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
7332 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7333 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
7334 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
7339 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
7340 doc
: /* Encode the Big5 character CHAR to BIG5 coding system.
7341 Return the corresponding character code in Big5. */)
7345 int charset
, c1
, c2
, b1
, b2
;
7349 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7350 if (charset
== CHARSET_ASCII
)
7354 else if ((charset
== charset_big5_1
7355 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
7356 || (charset
== charset_big5_2
7357 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
7359 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
7360 XSETFASTINT (val
, (b1
<< 8) | b2
);
7363 error ("Can't encode to Big5: %d", XFASTINT (ch
));
7367 DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal
,
7368 Sset_terminal_coding_system_internal
, 1, 1, 0,
7369 doc
: /* Internal use only. */)
7371 Lisp_Object coding_system
;
7373 CHECK_SYMBOL (coding_system
);
7374 setup_coding_system (Fcheck_coding_system (coding_system
), &terminal_coding
);
7375 /* We had better not send unsafe characters to terminal. */
7376 terminal_coding
.mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
7377 /* Character composition should be disabled. */
7378 terminal_coding
.composing
= COMPOSITION_DISABLED
;
7379 /* Error notification should be suppressed. */
7380 terminal_coding
.suppress_error
= 1;
7381 terminal_coding
.src_multibyte
= 1;
7382 terminal_coding
.dst_multibyte
= 0;
7386 DEFUN ("set-safe-terminal-coding-system-internal", Fset_safe_terminal_coding_system_internal
,
7387 Sset_safe_terminal_coding_system_internal
, 1, 1, 0,
7388 doc
: /* Internal use only. */)
7390 Lisp_Object coding_system
;
7392 CHECK_SYMBOL (coding_system
);
7393 setup_coding_system (Fcheck_coding_system (coding_system
),
7394 &safe_terminal_coding
);
7395 /* Character composition should be disabled. */
7396 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
7397 /* Error notification should be suppressed. */
7398 safe_terminal_coding
.suppress_error
= 1;
7399 safe_terminal_coding
.src_multibyte
= 1;
7400 safe_terminal_coding
.dst_multibyte
= 0;
7404 DEFUN ("terminal-coding-system", Fterminal_coding_system
,
7405 Sterminal_coding_system
, 0, 0, 0,
7406 doc
: /* Return coding system specified for terminal output. */)
7409 return terminal_coding
.symbol
;
7412 DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal
,
7413 Sset_keyboard_coding_system_internal
, 1, 1, 0,
7414 doc
: /* Internal use only. */)
7416 Lisp_Object coding_system
;
7418 CHECK_SYMBOL (coding_system
);
7419 setup_coding_system (Fcheck_coding_system (coding_system
), &keyboard_coding
);
7420 /* Character composition should be disabled. */
7421 keyboard_coding
.composing
= COMPOSITION_DISABLED
;
7425 DEFUN ("keyboard-coding-system", Fkeyboard_coding_system
,
7426 Skeyboard_coding_system
, 0, 0, 0,
7427 doc
: /* Return coding system specified for decoding keyboard input. */)
7430 return keyboard_coding
.symbol
;
7434 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
7435 Sfind_operation_coding_system
, 1, MANY
, 0,
7436 doc
: /* Choose a coding system for an operation based on the target name.
7437 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
7438 DECODING-SYSTEM is the coding system to use for decoding
7439 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
7440 for encoding (in case OPERATION does encoding).
7442 The first argument OPERATION specifies an I/O primitive:
7443 For file I/O, `insert-file-contents' or `write-region'.
7444 For process I/O, `call-process', `call-process-region', or `start-process'.
7445 For network I/O, `open-network-stream'.
7447 The remaining arguments should be the same arguments that were passed
7448 to the primitive. Depending on which primitive, one of those arguments
7449 is selected as the TARGET. For example, if OPERATION does file I/O,
7450 whichever argument specifies the file name is TARGET.
7452 TARGET has a meaning which depends on OPERATION:
7453 For file I/O, TARGET is a file name.
7454 For process I/O, TARGET is a process name.
7455 For network I/O, TARGET is a service name or a port number
7457 This function looks up what specified for TARGET in,
7458 `file-coding-system-alist', `process-coding-system-alist',
7459 or `network-coding-system-alist' depending on OPERATION.
7460 They may specify a coding system, a cons of coding systems,
7461 or a function symbol to call.
7462 In the last case, we call the function with one argument,
7463 which is a list of all the arguments given to this function.
7465 usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
7470 Lisp_Object operation
, target_idx
, target
, val
;
7471 register Lisp_Object chain
;
7474 error ("Too few arguments");
7475 operation
= args
[0];
7476 if (!SYMBOLP (operation
)
7477 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
7478 error ("Invalid first argument");
7479 if (nargs
< 1 + XINT (target_idx
))
7480 error ("Too few arguments for operation: %s",
7481 SDATA (SYMBOL_NAME (operation
)));
7482 /* For write-region, if the 6th argument (i.e. VISIT, the 5th
7483 argument to write-region) is string, it must be treated as a
7484 target file name. */
7485 if (EQ (operation
, Qwrite_region
)
7487 && STRINGP (args
[5]))
7488 target_idx
= make_number (4);
7489 target
= args
[XINT (target_idx
) + 1];
7490 if (!(STRINGP (target
)
7491 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
7492 error ("Invalid argument %d", XINT (target_idx
) + 1);
7494 chain
= ((EQ (operation
, Qinsert_file_contents
)
7495 || EQ (operation
, Qwrite_region
))
7496 ? Vfile_coding_system_alist
7497 : (EQ (operation
, Qopen_network_stream
)
7498 ? Vnetwork_coding_system_alist
7499 : Vprocess_coding_system_alist
));
7503 for (; CONSP (chain
); chain
= XCDR (chain
))
7509 && ((STRINGP (target
)
7510 && STRINGP (XCAR (elt
))
7511 && fast_string_match (XCAR (elt
), target
) >= 0)
7512 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
7515 /* Here, if VAL is both a valid coding system and a valid
7516 function symbol, we return VAL as a coding system. */
7519 if (! SYMBOLP (val
))
7521 if (! NILP (Fcoding_system_p (val
)))
7522 return Fcons (val
, val
);
7523 if (! NILP (Ffboundp (val
)))
7525 val
= call1 (val
, Flist (nargs
, args
));
7528 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
7529 return Fcons (val
, val
);
7537 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
7538 Supdate_coding_systems_internal
, 0, 0, 0,
7539 doc
: /* Update internal database for ISO2022 and CCL based coding systems.
7540 When values of any coding categories are changed, you must
7541 call this function. */)
7546 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7550 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[i
]);
7553 if (! coding_system_table
[i
])
7554 coding_system_table
[i
] = ((struct coding_system
*)
7555 xmalloc (sizeof (struct coding_system
)));
7556 setup_coding_system (val
, coding_system_table
[i
]);
7558 else if (coding_system_table
[i
])
7560 xfree (coding_system_table
[i
]);
7561 coding_system_table
[i
] = NULL
;
7568 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
7569 Sset_coding_priority_internal
, 0, 0, 0,
7570 doc
: /* Update internal database for the current value of `coding-category-list'.
7571 This function is internal use only. */)
7577 val
= Vcoding_category_list
;
7579 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
7581 if (! SYMBOLP (XCAR (val
)))
7583 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
7584 if (idx
>= CODING_CATEGORY_IDX_MAX
)
7586 coding_priorities
[i
++] = (1 << idx
);
7589 /* If coding-category-list is valid and contains all coding
7590 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
7591 the following code saves Emacs from crashing. */
7592 while (i
< CODING_CATEGORY_IDX_MAX
)
7593 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
7598 DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal
,
7599 Sdefine_coding_system_internal
, 1, 1, 0,
7600 doc
: /* Register CODING-SYSTEM as a base coding system.
7601 This function is internal use only. */)
7603 Lisp_Object coding_system
;
7605 Lisp_Object safe_chars
, slot
;
7607 if (NILP (Fcheck_coding_system (coding_system
)))
7608 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
7609 safe_chars
= coding_safe_chars (coding_system
);
7610 if (! EQ (safe_chars
, Qt
) && ! CHAR_TABLE_P (safe_chars
))
7611 error ("No valid safe-chars property for %s",
7612 SDATA (SYMBOL_NAME (coding_system
)));
7613 if (EQ (safe_chars
, Qt
))
7615 if (NILP (Fmemq (coding_system
, XCAR (Vcoding_system_safe_chars
))))
7616 XSETCAR (Vcoding_system_safe_chars
,
7617 Fcons (coding_system
, XCAR (Vcoding_system_safe_chars
)));
7621 slot
= Fassq (coding_system
, XCDR (Vcoding_system_safe_chars
));
7623 XSETCDR (Vcoding_system_safe_chars
,
7624 nconc2 (XCDR (Vcoding_system_safe_chars
),
7625 Fcons (Fcons (coding_system
, safe_chars
), Qnil
)));
7627 XSETCDR (slot
, safe_chars
);
7635 /*** 9. Post-amble ***/
7642 /* Emacs' internal format specific initialize routine. */
7643 for (i
= 0; i
<= 0x20; i
++)
7644 emacs_code_class
[i
] = EMACS_control_code
;
7645 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
7646 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
7647 for (i
= 0x21 ; i
< 0x7F; i
++)
7648 emacs_code_class
[i
] = EMACS_ascii_code
;
7649 emacs_code_class
[0x7F] = EMACS_control_code
;
7650 for (i
= 0x80; i
< 0xFF; i
++)
7651 emacs_code_class
[i
] = EMACS_invalid_code
;
7652 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
7653 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
7654 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
7655 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
7657 /* ISO2022 specific initialize routine. */
7658 for (i
= 0; i
< 0x20; i
++)
7659 iso_code_class
[i
] = ISO_control_0
;
7660 for (i
= 0x21; i
< 0x7F; i
++)
7661 iso_code_class
[i
] = ISO_graphic_plane_0
;
7662 for (i
= 0x80; i
< 0xA0; i
++)
7663 iso_code_class
[i
] = ISO_control_1
;
7664 for (i
= 0xA1; i
< 0xFF; i
++)
7665 iso_code_class
[i
] = ISO_graphic_plane_1
;
7666 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
7667 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
7668 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
7669 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
7670 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
7671 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
7672 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
7673 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
7674 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
7675 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
7677 setup_coding_system (Qnil
, &keyboard_coding
);
7678 setup_coding_system (Qnil
, &terminal_coding
);
7679 setup_coding_system (Qnil
, &safe_terminal_coding
);
7680 setup_coding_system (Qnil
, &default_buffer_file_coding
);
7682 bzero (coding_system_table
, sizeof coding_system_table
);
7684 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
7685 for (i
= 0; i
< 128; i
++)
7686 ascii_skip_code
[i
] = 1;
7688 #if defined (MSDOS) || defined (WINDOWSNT)
7689 system_eol_type
= CODING_EOL_CRLF
;
7691 system_eol_type
= CODING_EOL_LF
;
7694 inhibit_pre_post_conversion
= 0;
7702 staticpro (&Vcode_conversion_workbuf_name
);
7703 Vcode_conversion_workbuf_name
= build_string (" *code-conversion-work*");
7705 Qtarget_idx
= intern ("target-idx");
7706 staticpro (&Qtarget_idx
);
7708 Qcoding_system_history
= intern ("coding-system-history");
7709 staticpro (&Qcoding_system_history
);
7710 Fset (Qcoding_system_history
, Qnil
);
7712 /* Target FILENAME is the first argument. */
7713 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
7714 /* Target FILENAME is the third argument. */
7715 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
7717 Qcall_process
= intern ("call-process");
7718 staticpro (&Qcall_process
);
7719 /* Target PROGRAM is the first argument. */
7720 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
7722 Qcall_process_region
= intern ("call-process-region");
7723 staticpro (&Qcall_process_region
);
7724 /* Target PROGRAM is the third argument. */
7725 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
7727 Qstart_process
= intern ("start-process");
7728 staticpro (&Qstart_process
);
7729 /* Target PROGRAM is the third argument. */
7730 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
7732 Qopen_network_stream
= intern ("open-network-stream");
7733 staticpro (&Qopen_network_stream
);
7734 /* Target SERVICE is the fourth argument. */
7735 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
7737 Qcoding_system
= intern ("coding-system");
7738 staticpro (&Qcoding_system
);
7740 Qeol_type
= intern ("eol-type");
7741 staticpro (&Qeol_type
);
7743 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
7744 staticpro (&Qbuffer_file_coding_system
);
7746 Qpost_read_conversion
= intern ("post-read-conversion");
7747 staticpro (&Qpost_read_conversion
);
7749 Qpre_write_conversion
= intern ("pre-write-conversion");
7750 staticpro (&Qpre_write_conversion
);
7752 Qno_conversion
= intern ("no-conversion");
7753 staticpro (&Qno_conversion
);
7755 Qundecided
= intern ("undecided");
7756 staticpro (&Qundecided
);
7758 Qcoding_system_p
= intern ("coding-system-p");
7759 staticpro (&Qcoding_system_p
);
7761 Qcoding_system_error
= intern ("coding-system-error");
7762 staticpro (&Qcoding_system_error
);
7764 Fput (Qcoding_system_error
, Qerror_conditions
,
7765 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
7766 Fput (Qcoding_system_error
, Qerror_message
,
7767 build_string ("Invalid coding system"));
7769 Qcoding_category
= intern ("coding-category");
7770 staticpro (&Qcoding_category
);
7771 Qcoding_category_index
= intern ("coding-category-index");
7772 staticpro (&Qcoding_category_index
);
7774 Vcoding_category_table
7775 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
7776 staticpro (&Vcoding_category_table
);
7779 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7781 XVECTOR (Vcoding_category_table
)->contents
[i
]
7782 = intern (coding_category_name
[i
]);
7783 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
7784 Qcoding_category_index
, make_number (i
));
7788 Vcoding_system_safe_chars
= Fcons (Qnil
, Qnil
);
7789 staticpro (&Vcoding_system_safe_chars
);
7791 Qtranslation_table
= intern ("translation-table");
7792 staticpro (&Qtranslation_table
);
7793 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (2));
7795 Qtranslation_table_id
= intern ("translation-table-id");
7796 staticpro (&Qtranslation_table_id
);
7798 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
7799 staticpro (&Qtranslation_table_for_decode
);
7801 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
7802 staticpro (&Qtranslation_table_for_encode
);
7804 Qsafe_chars
= intern ("safe-chars");
7805 staticpro (&Qsafe_chars
);
7807 Qchar_coding_system
= intern ("char-coding-system");
7808 staticpro (&Qchar_coding_system
);
7810 /* Intern this now in case it isn't already done.
7811 Setting this variable twice is harmless.
7812 But don't staticpro it here--that is done in alloc.c. */
7813 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
7814 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
7815 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (0));
7817 Qvalid_codes
= intern ("valid-codes");
7818 staticpro (&Qvalid_codes
);
7820 Qemacs_mule
= intern ("emacs-mule");
7821 staticpro (&Qemacs_mule
);
7823 Qraw_text
= intern ("raw-text");
7824 staticpro (&Qraw_text
);
7826 Qutf_8
= intern ("utf-8");
7827 staticpro (&Qutf_8
);
7829 Qcoding_system_define_form
= intern ("coding-system-define-form");
7830 staticpro (&Qcoding_system_define_form
);
7832 defsubr (&Scoding_system_p
);
7833 defsubr (&Sread_coding_system
);
7834 defsubr (&Sread_non_nil_coding_system
);
7835 defsubr (&Scheck_coding_system
);
7836 defsubr (&Sdetect_coding_region
);
7837 defsubr (&Sdetect_coding_string
);
7838 defsubr (&Sfind_coding_systems_region_internal
);
7839 defsubr (&Sunencodable_char_position
);
7840 defsubr (&Sdecode_coding_region
);
7841 defsubr (&Sencode_coding_region
);
7842 defsubr (&Sdecode_coding_string
);
7843 defsubr (&Sencode_coding_string
);
7844 defsubr (&Sdecode_sjis_char
);
7845 defsubr (&Sencode_sjis_char
);
7846 defsubr (&Sdecode_big5_char
);
7847 defsubr (&Sencode_big5_char
);
7848 defsubr (&Sset_terminal_coding_system_internal
);
7849 defsubr (&Sset_safe_terminal_coding_system_internal
);
7850 defsubr (&Sterminal_coding_system
);
7851 defsubr (&Sset_keyboard_coding_system_internal
);
7852 defsubr (&Skeyboard_coding_system
);
7853 defsubr (&Sfind_operation_coding_system
);
7854 defsubr (&Supdate_coding_systems_internal
);
7855 defsubr (&Sset_coding_priority_internal
);
7856 defsubr (&Sdefine_coding_system_internal
);
7858 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
7859 doc
: /* List of coding systems.
7861 Do not alter the value of this variable manually. This variable should be
7862 updated by the functions `make-coding-system' and
7863 `define-coding-system-alias'. */);
7864 Vcoding_system_list
= Qnil
;
7866 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
7867 doc
: /* Alist of coding system names.
7868 Each element is one element list of coding system name.
7869 This variable is given to `completing-read' as TABLE argument.
7871 Do not alter the value of this variable manually. This variable should be
7872 updated by the functions `make-coding-system' and
7873 `define-coding-system-alias'. */);
7874 Vcoding_system_alist
= Qnil
;
7876 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
7877 doc
: /* List of coding-categories (symbols) ordered by priority.
7879 On detecting a coding system, Emacs tries code detection algorithms
7880 associated with each coding-category one by one in this order. When
7881 one algorithm agrees with a byte sequence of source text, the coding
7882 system bound to the corresponding coding-category is selected.
7884 Don't modify this variable directly, but use `set-coding-priority'. */);
7888 Vcoding_category_list
= Qnil
;
7889 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
7890 Vcoding_category_list
7891 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
7892 Vcoding_category_list
);
7895 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
7896 doc
: /* Specify the coding system for read operations.
7897 It is useful to bind this variable with `let', but do not set it globally.
7898 If the value is a coding system, it is used for decoding on read operation.
7899 If not, an appropriate element is used from one of the coding system alists:
7900 There are three such tables, `file-coding-system-alist',
7901 `process-coding-system-alist', and `network-coding-system-alist'. */);
7902 Vcoding_system_for_read
= Qnil
;
7904 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
7905 doc
: /* Specify the coding system for write operations.
7906 Programs bind this variable with `let', but you should not set it globally.
7907 If the value is a coding system, it is used for encoding of output,
7908 when writing it to a file and when sending it to a file or subprocess.
7910 If this does not specify a coding system, an appropriate element
7911 is used from one of the coding system alists:
7912 There are three such tables, `file-coding-system-alist',
7913 `process-coding-system-alist', and `network-coding-system-alist'.
7914 For output to files, if the above procedure does not specify a coding system,
7915 the value of `buffer-file-coding-system' is used. */);
7916 Vcoding_system_for_write
= Qnil
;
7918 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
7919 doc
: /* Coding system used in the latest file or process I/O.
7920 Also set by `encode-coding-region', `decode-coding-region',
7921 `encode-coding-string' and `decode-coding-string'. */);
7922 Vlast_coding_system_used
= Qnil
;
7924 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
7925 doc
: /* *Non-nil means always inhibit code conversion of end-of-line format.
7926 See info node `Coding Systems' and info node `Text and Binary' concerning
7927 such conversion. */);
7928 inhibit_eol_conversion
= 0;
7930 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
7931 doc
: /* Non-nil means process buffer inherits coding system of process output.
7932 Bind it to t if the process output is to be treated as if it were a file
7933 read from some filesystem. */);
7934 inherit_process_coding_system
= 0;
7936 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
7937 doc
: /* Alist to decide a coding system to use for a file I/O operation.
7938 The format is ((PATTERN . VAL) ...),
7939 where PATTERN is a regular expression matching a file name,
7940 VAL is a coding system, a cons of coding systems, or a function symbol.
7941 If VAL is a coding system, it is used for both decoding and encoding
7943 If VAL is a cons of coding systems, the car part is used for decoding,
7944 and the cdr part is used for encoding.
7945 If VAL is a function symbol, the function must return a coding system
7946 or a cons of coding systems which are used as above. The function gets
7947 the arguments with which `find-operation-coding-system' was called.
7949 See also the function `find-operation-coding-system'
7950 and the variable `auto-coding-alist'. */);
7951 Vfile_coding_system_alist
= Qnil
;
7953 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
7954 doc
: /* Alist to decide a coding system to use for a process I/O operation.
7955 The format is ((PATTERN . VAL) ...),
7956 where PATTERN is a regular expression matching a program name,
7957 VAL is a coding system, a cons of coding systems, or a function symbol.
7958 If VAL is a coding system, it is used for both decoding what received
7959 from the program and encoding what sent to the program.
7960 If VAL is a cons of coding systems, the car part is used for decoding,
7961 and the cdr part is used for encoding.
7962 If VAL is a function symbol, the function must return a coding system
7963 or a cons of coding systems which are used as above.
7965 See also the function `find-operation-coding-system'. */);
7966 Vprocess_coding_system_alist
= Qnil
;
7968 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
7969 doc
: /* Alist to decide a coding system to use for a network I/O operation.
7970 The format is ((PATTERN . VAL) ...),
7971 where PATTERN is a regular expression matching a network service name
7972 or is a port number to connect to,
7973 VAL is a coding system, a cons of coding systems, or a function symbol.
7974 If VAL is a coding system, it is used for both decoding what received
7975 from the network stream and encoding what sent to the network stream.
7976 If VAL is a cons of coding systems, the car part is used for decoding,
7977 and the cdr part is used for encoding.
7978 If VAL is a function symbol, the function must return a coding system
7979 or a cons of coding systems which are used as above.
7981 See also the function `find-operation-coding-system'. */);
7982 Vnetwork_coding_system_alist
= Qnil
;
7984 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
7985 doc
: /* Coding system to use with system messages.
7986 Also used for decoding keyboard input on X Window system. */);
7987 Vlocale_coding_system
= Qnil
;
7989 /* The eol mnemonics are reset in startup.el system-dependently. */
7990 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
7991 doc
: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7992 eol_mnemonic_unix
= build_string (":");
7994 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
7995 doc
: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7996 eol_mnemonic_dos
= build_string ("\\");
7998 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
7999 doc
: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
8000 eol_mnemonic_mac
= build_string ("/");
8002 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
8003 doc
: /* *String displayed in mode line when end-of-line format is not yet determined. */);
8004 eol_mnemonic_undecided
= build_string (":");
8006 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
8007 doc
: /* *Non-nil enables character translation while encoding and decoding. */);
8008 Venable_character_translation
= Qt
;
8010 DEFVAR_LISP ("standard-translation-table-for-decode",
8011 &Vstandard_translation_table_for_decode
,
8012 doc
: /* Table for translating characters while decoding. */);
8013 Vstandard_translation_table_for_decode
= Qnil
;
8015 DEFVAR_LISP ("standard-translation-table-for-encode",
8016 &Vstandard_translation_table_for_encode
,
8017 doc
: /* Table for translating characters while encoding. */);
8018 Vstandard_translation_table_for_encode
= Qnil
;
8020 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
8021 doc
: /* Alist of charsets vs revision numbers.
8022 While encoding, if a charset (car part of an element) is found,
8023 designate it with the escape sequence identifying revision (cdr part of the element). */);
8024 Vcharset_revision_alist
= Qnil
;
8026 DEFVAR_LISP ("default-process-coding-system",
8027 &Vdefault_process_coding_system
,
8028 doc
: /* Cons of coding systems used for process I/O by default.
8029 The car part is used for decoding a process output,
8030 the cdr part is used for encoding a text to be sent to a process. */);
8031 Vdefault_process_coding_system
= Qnil
;
8033 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
8034 doc
: /* Table of extra Latin codes in the range 128..159 (inclusive).
8035 This is a vector of length 256.
8036 If Nth element is non-nil, the existence of code N in a file
8037 \(or output of subprocess) doesn't prevent it to be detected as
8038 a coding system of ISO 2022 variant which has a flag
8039 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
8040 or reading output of a subprocess.
8041 Only 128th through 159th elements has a meaning. */);
8042 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
8044 DEFVAR_LISP ("select-safe-coding-system-function",
8045 &Vselect_safe_coding_system_function
,
8046 doc
: /* Function to call to select safe coding system for encoding a text.
8048 If set, this function is called to force a user to select a proper
8049 coding system which can encode the text in the case that a default
8050 coding system used in each operation can't encode the text.
8052 The default value is `select-safe-coding-system' (which see). */);
8053 Vselect_safe_coding_system_function
= Qnil
;
8055 DEFVAR_BOOL ("coding-system-require-warning",
8056 &coding_system_require_warning
,
8057 doc
: /* Internal use only.
8058 If non-nil, on writing a file, `select-safe-coding-system-function' is
8059 called even if `coding-system-for-write' is non-nil. The command
8060 `universal-coding-system-argument' binds this variable to t temporarily. */);
8061 coding_system_require_warning
= 0;
8064 DEFVAR_BOOL ("inhibit-iso-escape-detection",
8065 &inhibit_iso_escape_detection
,
8066 doc
: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
8068 By default, on reading a file, Emacs tries to detect how the text is
8069 encoded. This code detection is sensitive to escape sequences. If
8070 the sequence is valid as ISO2022, the code is determined as one of
8071 the ISO2022 encodings, and the file is decoded by the corresponding
8072 coding system (e.g. `iso-2022-7bit').
8074 However, there may be a case that you want to read escape sequences in
8075 a file as is. In such a case, you can set this variable to non-nil.
8076 Then, as the code detection ignores any escape sequences, no file is
8077 detected as encoded in some ISO2022 encoding. The result is that all
8078 escape sequences become visible in a buffer.
8080 The default value is nil, and it is strongly recommended not to change
8081 it. That is because many Emacs Lisp source files that contain
8082 non-ASCII characters are encoded by the coding system `iso-2022-7bit'
8083 in Emacs's distribution, and they won't be decoded correctly on
8084 reading if you suppress escape sequence detection.
8086 The other way to read escape sequences in a file without decoding is
8087 to explicitly specify some coding system that doesn't use ISO2022's
8088 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
8089 inhibit_iso_escape_detection
= 0;
8091 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input
,
8092 doc
: /* Char table for translating self-inserting characters.
8093 This is applied to the result of input methods, not their input. See also
8094 `keyboard-translate-table'. */);
8095 Vtranslation_table_for_input
= Qnil
;
8099 emacs_strerror (error_number
)
8104 synchronize_system_messages_locale ();
8105 str
= strerror (error_number
);
8107 if (! NILP (Vlocale_coding_system
))
8109 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
8110 Vlocale_coding_system
,
8112 str
= (char *) SDATA (dec
);
8120 /* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
8121 (do not change this comment) */