xemacs-beta: man/internals/internals.texi comparison

comparison man/internals/internals.texi @ 380:8626e4521993 r21-2-5

Import from CVS: tag r21-2-5

author	cvs
date	Mon, 13 Aug 2007 11:07:10 +0200
parents	d883f39b8495
children	aabb7f5b1c81

comparison

equal deleted inserted replaced

-:76b7d63099ad
+:8626e4521993
 @ifinfo
 Copyright @copyright{} 1992 - 1996 Ben Wing.
 Copyright @copyright{} 1996, 1997 Sun Microsystems.
-Copyright @copyright{} 1994, 1995 Free Software Foundation.
+Copyright @copyright{} 1994 - 1998 Free Software Foundation.
 Copyright @copyright{} 1994, 1995 Board of Trustees, University of Illinois.
 Permission is granted to make and distribute verbatim copies of this
 manual provided the copyright notice and this permission notice are
 @setchapternewpage odd
 @finalout
 @titlepage
 @title XEmacs Internals Manual
-@subtitle Version 1.1, March 1997
+@subtitle Version 1.2, October 1998
 @author Ben Wing
 @author Martin Buchholz
+@author Hrvoje Niksic
 @page
 @vskip 0pt plus 1fill
 @noindent
 Copyright @copyright{} 1992 - 1996 Ben Wing. @*
-Copyright @copyright{} 1996 Sun Microsystems, Inc. @*
+Copyright @copyright{} 1996, 1997 Sun Microsystems, Inc. @*
-Copyright @copyright{} 1994 Free Software Foundation. @*
+Copyright @copyright{} 1994 - 1998 Free Software Foundation. @*
 Copyright @copyright{} 1994, 1995 Board of Trustees, University of Illinois.
 @sp 2
-Version 1.1 @*
+Version 1.2 @*
-March, 1997.@*
+October 1998.@*
 Permission is granted to make and distribute verbatim copies of this
 manual provided the copyright notice and this permission notice are
 preserved on all copies.
 * Vector::
 * Bit Vector::
 * Symbol::
 * Marker::
 * String::
-* Bytecode::
+* Compiled Function::
 Events and the Event Loop
 * Introduction to Events::
 * Main Loop::
 marked with their type, as in dynamic typing; but there is a hierarchy
 of types and functions are declared to accept only certain types, thus
 providing the increased compile-time error-checking of static typing.
 @end enumerate
+The Java language also has some negative attributes:
+@enumerate
+@item
+Java uses the edit/compile/run model of software development.  This
+makes it hard to use interactively.  For example, to use Java like
+@code{bc} it is necessary to write a special purpose, albeit tiny,
+application.  In Emacs Lisp, a calculator comes built-in without any
+effort - one can always just type an expression in the @code{*scratch*}
+buffer.
+@item
+Java tries too hard to enforce, not merely enable, portability, making
+ordinary access to standard OS facilities painful.  Java has an
+@dfn{agenda}.  I think this is why @code{chdir} is not part of standard
+Java, which is inexcusable.
+@end enumerate
+Unfortunately, there is no perfect language.  Static typing allows a
+compiler to catch programmer errors and produce more efficient code, but
+makes programming more tedious and less fun.  For the forseeable future,
+an Ideal Editing and Programming Environment (and that is what XEmacs
+aspires to) will be programmable in multiple languages: high level ones
+like Lisp for user customization and prototyping, and lower level ones
+for infrastructure and industrial strength applications.  If I had my
+way, XEmacs would be friendly towards the Python, Scheme, C++, ML,
+etc... communities.  But there are serious technical difficulties to
+achieving that goal.
+The word @dfn{application} in the previous paragraph was used
+intentionally.  XEmacs implements an API for programs written in Lisp
+that makes it a full-fledged application platform, very much like an OS
+inside the real OS.
 @node XEmacs From the Perspective of Building, XEmacs From the Inside, The Lisp Language, Top
 @chapter XEmacs From the Perspective of Building
 The heart of XEmacs is the Lisp environment, which is written in C.
 This is contained in the @file{src/} subdirectory.  Underneath
 @file{src/} are two subdirectories of header files: @file{s/} (header
 files for particular operating systems) and @file{m/} (header files for
 particular machine types).  In practice the distinction between the two
 types of header files is blurred.  These header files define or undefine
 characteristics of the associated machine or operating system.  As part
 of the configure process, one @file{s/} file and one @file{m/} file is
 identified for the particular environment in which XEmacs is being
 built.
 XEmacs also contains a great deal of Lisp code.  This implements the
-operations that make XEmacs useful as an editor as well as just a
+operations that make XEmacs useful as an editor as well as just a Lisp
-Lisp environment, and also contains many add-on packages that allow
+environment, and also contains many add-on packages that allow XEmacs to
-XEmacs to browse directories, act as a mail and Usenet news reader,
+browse directories, act as a mail and Usenet news reader, compile Lisp
-compile Lisp code, etc.  There is actually more Lisp code than
+code, etc.  There is actually more Lisp code than C code associated with
-C code associated with XEmacs, but much of the Lisp code is
+XEmacs, but much of the Lisp code is peripheral to the actual operation
-peripheral to the actual operation of the editor.  The Lisp code
+of the editor.  The Lisp code all lies in subdirectories underneath the
-all lies in subdirectories underneath the @file{lisp/} directory.
+@file{lisp/} directory.
 The @file{lwlib/} directory contains C code that implements a
 generalized interface onto different X widget toolkits and also
 implements some widgets of its own that behave like Motif widgets but
 are faster, free, and in some cases more powerful.  The code in this
 directory compiles into a library and is mostly independent from XEmacs.
 The @file{etc/} directory contains various data files associated with
 XEmacs.  Some of them are actually read by XEmacs at startup; others
 merely contain useful information of various sorts.
 The @file{lib-src/} directory contains C code for various auxiliary
 programs that are used in connection with XEmacs.  Some of them are used
 during the build process; others are used to perform certain functions
 that cannot conveniently be placed in the XEmacs executable (e.g. the
 @file{movemail} program for fetching mail out of @file{/var/spool/mail},
 which must be setgid to @file{mail} on many systems; and the
 @file{gnuclient} program, which allows an external script to communicate
 with a running XEmacs process).
 The @file{man/} directory contains the sources for the XEmacs
 documentation.  It is mostly in a form called Texinfo, which can be
 converted into either a printed document (by passing it through @TeX{})
 or into on-line documentation called @dfn{info files}.
-The @file{info/} directory contains the results of formatting the
+The @file{info/} directory contains the results of formatting the XEmacs
-XEmacs documentation as @dfn{info files}, for on-line use.  These files
+documentation as @dfn{info files}, for on-line use.  These files are
-are used when you enter the Info system using @kbd{C-h i} or through the
+used when you enter the Info system using @kbd{C-h i} or through the
 Help menu.
 The @file{dynodump/} directory contains auxiliary code used to build
 XEmacs on Solaris platforms.
-The other directories contain various miscellaneous code and
+The other directories contain various miscellaneous code and information
-information that is not normally used or needed.
+that is not normally used or needed.
-The first step of building involves running the @file{configure}
+The first step of building involves running the @file{configure} program
-program and passing it various parameters to specify any optional
+and passing it various parameters to specify any optional features you
-features you want and compiler arguments and such, as described in the
+want and compiler arguments and such, as described in the @file{INSTALL}
-@file{INSTALL} file.  This determines what the build environment is,
+file.  This determines what the build environment is, chooses the
-chooses the appropriate @file{s/} and @file{m/} file, and runs a series
+appropriate @file{s/} and @file{m/} file, and runs a series of tests to
-of tests to determine many details about your environment, such as which
+determine many details about your environment, such as which library
-library functions are available and exactly how they work. (The
+functions are available and exactly how they work.  The reason for
-@file{s/} and @file{m/} files only contain information that cannot be
+running these tests is that it allows XEmacs to be compiled on a much
-conveniently detected in this fashion.) The reason for running these
+wider variety of platforms than those that the XEmacs developers happen
-tests is that it allows XEmacs to be compiled on a much wider variety of
+to be familiar with, including various sorts of hybrid platforms.  This
-platforms than those that the XEmacs developers happen to be familiar
+is especially important now that many operating systems give you a great
-with, including various sorts of hybrid platforms.  This is especially
+deal of control over exactly what features you want installed, and allow
-important now that many operating systems give you a great deal of
+for easy upgrading of parts of a system without upgrading the rest.  It
-control over exactly what features you want installed, and allow for
-easy upgrading of parts of a system without upgrading the rest.  It
 would be impossible to pre-determine and pre-specify the information for
 all possible configurations.
-When configure is done running, it generates @file{Makefile}s and the
+In fact, the @file{s/} and @file{m/} files are basically @emph{evil},
-file @file{src/config.h} (which describes the features of your system)
+since they contain unmaintainable platform-specific hard-coded
-from template files.  You then run @file{make}, which compiles the
+information.  XEmacs has been moving in the direction of having all
-auxiliary code and programs in @file{lib-src/} and @file{lwlib/} and the
+system-specific information be determined dynamically by
-main XEmacs executable in @file{src/}.  The result of compiling and
+@file{configure}.  Perhaps someday we can @code{rm -rf src/s src/m}.
-linking is an executable called @file{temacs}, which is @emph{not} the
-final XEmacs executable.  @file{temacs} by itself is not intended to
+When configure is done running, it generates @file{Makefile}s and
-function as an editor or even display any windows on the screen, and if
+@file{GNUmakefile}s and the file @file{src/config.h} (which describes
-you simply run it, it will exit immediately.  The @file{Makefile} runs
+the features of your system) from template files.  You then run
-@file{temacs} with certain options that cause it to initialize itself,
+@file{make}, which compiles the auxiliary code and programs in
-read in a number of basic Lisp files, and then dump itself out into a
+@file{lib-src/} and @file{lwlib/} and the main XEmacs executable in
-new executable called @file{xemacs}.  This new executable has been
+@file{src/}.  The result of compiling and linking is an executable
-pre-initialized and contains pre-digested Lisp code that is necessary
+called @file{temacs}, which is @emph{not} the final XEmacs executable.
-for the editor to function (this includes most basic Lisp functions,
+@file{temacs} by itself is not intended to function as an editor or even
-e.g. @code{not}, that can be defined in terms of other Lisp primitives;
+display any windows on the screen, and if you simply run it, it will
-some initialization code that is called when certain objects, such as
+exit immediately.  The @file{Makefile} runs @file{temacs} with certain
-frames, are created; and all of the standard keybindings and code for
+options that cause it to initialize itself, read in a number of basic
-the actions they result in).  This executable, @file{xemacs}, is the
+Lisp files, and then dump itself out into a new executable called
-executable that you run to use the XEmacs editor.
+@file{xemacs}.  This new executable has been pre-initialized and
+contains pre-digested Lisp code that is necessary for the editor to
+function (this includes most basic editing functions,
+e.g. @code{kill-line}, that can be defined in terms of other Lisp
+primitives; some initialization code that is called when certain
+objects, such as frames, are created; and all of the standard
+keybindings and code for the actions they result in).  This executable,
+@file{xemacs}, is the executable that you run to use the XEmacs editor.
 Although @file{temacs} is not intended to be run as an editor, it can,
 by using the incantation @code{temacs -batch -l loadup.el run-temacs}.
 This is useful when the dumping procedure described above is broken, or
 when using certain program debugging tools such as Purify.  These tools
 such as allocation memory in one process, and freeing it in the next.
 @node XEmacs From the Inside, The XEmacs Object System (Abstractly Speaking), XEmacs From the Perspective of Building, Top
 @chapter XEmacs From the Inside
 Internally, XEmacs is quite complex, and can be very confusing.  To
 simplify things, it can be useful to think of XEmacs as containing an
 event loop that ``drives'' everything, and a number of other subsystems,
 such as a Lisp engine and a redisplay mechanism.  Each of these other
 subsystems exists simultaneously in XEmacs, and each has a certain
 state.  The flow of control continually passes in and out of these
 different subsystems in the course of normal operation of the editor.
 It is important to keep in mind that, most of the time, the editor is
 ``driven'' by the event loop.  Except during initialization and batch
 mode, all subsystems are entered directly or indirectly through the
 event loop, and ultimately, control exits out of all subsystems back up
 to the event loop.  This cycle of entering a subsystem, exiting back out
 to the event loop, and starting another iteration of the event loop
 occurs once each keystroke, mouse motion, etc.
 If you're trying to understand a particular subsystem (other than the
 event loop), think of it as a ``daemon'' process or ``servant'' that is
 responsible for one particular aspect of a larger system, and
 periodically receives commands or environment changes that cause it to
 do something.  Ultimately, these commands and environment changes are
 always triggered by the event loop.  For example:
 The basic Lisp objects are
 @table @code
 @item integer
-28 bits of precision, or 60 bits on 64-bit machines; the reason for this
+28 or 31 bits of precision, or 60 or 63 bits on 64-bit machines; the
-is described below when the internal Lisp object representation is
+reason for this is described below when the internal Lisp object
-described.
+representation is described.
 @item float
 Same precision as a double in C.
 @item cons
 A simple container for two Lisp objects, used to implement lists and
 most other data structures in Lisp.
 faster than for lists, but the operations that can be done on a vector
 are more limited.
 @item string
 Self-explanatory; behaves much like a vector of chars
 but has a different read syntax and is stored and manipulated
-more compactly and efficiently.
+more compactly.
 @item bit-vector
 A vector of bits; similar to a string in spirit.
 @item compiled-function
-An object describing compiled Lisp code, known as @dfn{byte code}.
+An object containing compiled Lisp code, known as @dfn{byte code}.
 @item subr
-An object describing a Lisp primitive.
+A Lisp primitive, i.e. a Lisp-callable function implemented in C.
 @end table
 @cindex closure
 Note that there is no basic ``function'' type, as in more powerful
 versions of Lisp (where it's called a @dfn{closure}).  XEmacs Lisp does
 not provide the closure semantics implemented by Common Lisp and Scheme.
 The guts of a function in XEmacs Lisp are represented in one of four
 ways: a symbol specifying another function (when one function is an
-alias for another), a list containing the function's source code, a
+alias for another), a list (whose first element must be the symbol
-bytecode object, or a subr object. (In other words, given a symbol
+@code{lambda}) containing the function's source code, a
-specifying the name of a function, calling @code{symbol-function} to
+compiled-function object, or a subr object. (In other words, given a
-retrieve the contents of the symbol's function cell will return one of
+symbol specifying the name of a function, calling @code{symbol-function}
-these types of objects.)
+to retrieve the contents of the symbol's function cell will return one
+of these types of objects.)
-XEmacs Lisp also contains numerous specialized objects used to
-implement the editor:
+XEmacs Lisp also contains numerous specialized objects used to implement
+the editor:
 @table @code
 @item buffer
 Stores text like a string, but is optimized for insertion and deletion
 and has certain other properties that can be set.
 @item device
 An object representing a screen on which frames can be displayed;
 equivalent to a @dfn{display} in the X Window System and a @dfn{TTY} in
 character mode.
 @item face
-An object specifying the appearance of text or graphics; it contains
+An object specifying the appearance of text or graphics; it has
-characteristics such as font, foreground color, and background color.
+properties such as font, foreground color, and background color.
 @item marker
 An object that refers to a particular position in a buffer and moves
 around as text is inserted and deleted to stay in the same relative
 position to the text around it.
 @item extent
 @end table
 There are some other, less-commonly-encountered general objects:
 @table @code
-@item hashtable
+@item hash-table
 An object that maps from an arbitrary Lisp object to another arbitrary
 Lisp object, using hashing for fast lookup.
 @item obarray
-A limited form of hashtable that maps from strings to symbols; obarrays
+A limited form of hash-table that maps from strings to symbols; obarrays
 are used to look up a symbol given its name and are not actually their
 own object type but are kludgily represented using vectors with hidden
 fields (this representation derives from GNU Emacs).
 @item specifier
 A complex object used to specify the value of a display property; a
 @itemx tooltalk-pattern
 Objects that represent resources used in the ToolTalk interprocess
 communication protocol.
 @item toolbar-button
 An object used in conjunction with the toolbar.
-@item x-resource
-An object that encapsulates certain miscellaneous resources in the X
-window system, used only when Epoch support is enabled.
 @end table
 And objects that are only used internally:
-@table @asis
+@table @code
 @item opaque
 A generic object for encapsulating arbitrary memory; this allows you the
 generality of @code{malloc()} and the convenience of the Lisp object
 system.
 @item lstream
 ?^[$(B#&^[(B
 @end example
 (where @samp{^[} actually is an @samp{ESC} character) converts to a
 particular Kanji character when using an ISO2022-based coding system for
-input. (To decode this gook: @samp{ESC} begins an escape sequence;
+input. (To decode this goo: @samp{ESC} begins an escape sequence;
 @samp{ESC $ (} is a class of escape sequences meaning ``switch to a
 94x94 character set''; @samp{ESC $ ( B} means ``switch to Japanese
 Kanji''; @samp{#} and @samp{&} collectively index into a 94-by-94 array
 of characters [subtract 33 from the ASCII value of each character to get
 the corresponding index]; @samp{ESC (} is a class of escape sequences
 converts to a symbol whose name is @code{"foobar"}.  This is done by
 looking up the string equivalent in the global variable
 @code{obarray}, whose contents should be an obarray.  If no symbol
 is found, a new symbol with the name @code{"foobar"} is automatically
 created and added to @code{obarray}; this process is called
 @dfn{interning} the symbol.
 @cindex interning
 @example
 (foo . bar)
 @end example
 @end example
 converts to a bit-vector.
 @example
+#s(hash-table ... ...)
+@end example
+converts to a hash table (the actual contents are not shown).
+@example
 #s(range-table ... ...)
 @end example
 converts to a range table (the actual contents are not shown).
 @example
 #s(char-table ... ...)
 @end example
 converts to a char table (the actual contents are not shown).
-(Note that the #s syntax is the general syntax for structures,
-which are not really implemented in XEmacs Lisp but should be.)
+Note that the @code{#s()} syntax is the general syntax for structures,
+which are not really implemented in XEmacs Lisp but should be.
-When an object is printed out (using @code{print} or a related
+When an object is printed out (using @code{print} or a related
 function), the read syntax is used, so that the same object can be read
 in again.
-The other objects do not have read syntaxes, usually because it does
+The other objects do not have read syntaxes, usually because it does not
-not really make sense to create them in this fashion (i.e.  processes,
+really make sense to create them in this fashion (i.e.  processes, where
-where it doesn't make sense to have a subprocess created as a side
+it doesn't make sense to have a subprocess created as a side effect of
-effect of reading some Lisp code), or because they can't be created at
+reading some Lisp code), or because they can't be created at all
-all (e.g. subrs).  Permanent objects, as a rule, do not have a read
+(e.g. subrs).  Permanent objects, as a rule, do not have a read syntax;
-syntax; nor do most complex objects, which contain too much state to be
+nor do most complex objects, which contain too much state to be easily
-easily initialized through a read syntax.
+initialized through a read syntax.
 @node How Lisp Objects Are Represented in C, Rules When Writing New C Code, The XEmacs Object System (Abstractly Speaking), Top
 @chapter How Lisp Objects Are Represented in C
-Lisp objects are represented in C using a 32- or 64-bit machine word
+Lisp objects are represented in C using a 32-bit or 64-bit machine word
 (depending on the processor; i.e. DEC Alphas use 64-bit Lisp objects and
 most other processors use 32-bit Lisp objects).  The representation
 stuffs a pointer together with a tag, as follows:
 @example
 [ 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 ]
 [ 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 ]
-^ <---> <------------------------------------------------------>
+<---> ^ <------------------------------------------------------>
-|  tag         a pointer to a structure, or an integer
+tag  |       a pointer to a structure, or an integer
 |
-`---> mark bit
+mark bit
 @end example
-The tag describes the type of the Lisp object.  For integers and
+The tag describes the type of the Lisp object.  For integers and chars,
-chars, the lower 28 bits contain the value of the integer or char; for
+the lower 28 bits contain the value of the integer or char; for all
-all others, the lower 28 bits contain a pointer.  The mark bit is used
+others, the lower 28 bits contain a pointer.  The mark bit is used
 during garbage-collection, and is always 0 when garbage collection is
-not happening.  Many macros that extract out parts of a Lisp object
+not happening. (The way that garbage collection works, basically, is that it
-expect that the mark bit is 0, and will produce incorrect results if
-it's not. (The way that garbage collection works, basically, is that it
 loops over all places where Lisp objects could exist -- this includes
 all global variables in C that contain Lisp objects [including
 @code{Vobarray}, the C equivalent of @code{obarray}; through this, all
 Lisp variables will get marked], plus various other places -- and
 recursively scans through the Lisp objects, marking each object it finds
 by setting the mark bit.  Then it goes through the lists of all objects
-allocated, freeing the ones that are not marked and turning off the
+allocated, freeing the ones that are not marked and turning off the mark
-mark bit of the ones that are marked.)
+bit of the ones that are marked.)
 Lisp objects use the typedef @code{Lisp_Object}, but the actual C type
 used for the Lisp object can vary.  It can be either a simple type
 (@code{long} on the DEC Alpha, @code{int} on other machines) or a
 structure whose fields are bit fields that line up properly (actually, a
-union of structures that's used).  Generally the simple integral type is
+union of structures is used).  Generally the simple integral type is
 preferable because it ensures that the compiler will actually use a
 machine word to represent the object (some compilers will use more
 general and less efficient code for unions and structs even if they can
 fit in a machine word).  The union type, however, has the advantage of
 stricter type checking (if you accidentally pass an integer where a Lisp
 object is desired, you get a compile error), and it makes it easier to
 decode Lisp objects when debugging.  The choice of which type to use is
-determined by the presence or absence of the preprocessor constant
+determined by the preprocessor constant @code{USE_UNION_TYPE} which is
-@code{USE_UNION_TYPE}.
+defined via the @code{--use-union-type} option to @code{configure}.
 @cindex record type
-Note that there are only eight types that the tag can represent,
-but many more actual types than this.  This is handled by having
+Note that there are only eight types that the tag can represent, but
-one of the tag types specify a meta-type called a @dfn{record};
+many more actual types than this.  This is handled by having one of the
-for all such objects, the first four bytes of the pointed-to
+tag types specify a meta-type called a @dfn{record}; for all such
-structure indicate what the actual type is.
+objects, the first four bytes of the pointed-to structure indicate what
+the actual type is.
-Note also that having 28 bits for pointers and integers restricts a
-lot of things to 256 megabytes of memory. (Basically, enough pointers
+Note also that having 28 bits for pointers and integers restricts a lot
-and indices and whatnot get stuffed into Lisp objects that the total
+of things to 256 megabytes of memory. (Basically, enough pointers and
-amount of memory used by XEmacs can't grow above 256 megabytes.  In
+indices and whatnot get stuffed into Lisp objects that the total amount
-older versions of XEmacs and GNU Emacs, the tag was 5 bits wide,
+of memory used by XEmacs can't grow above 256 megabytes.  In older
-allowing for 32 types, which was more than the actual number of types
+versions of XEmacs and GNU Emacs, the tag was 5 bits wide, allowing for
-that existed at the time, and no ``record'' type was necessary.
+32 types, which was more than the actual number of types that existed at
-However, this limited the editor to 64 megabytes total, which some users
+the time, and no ``record'' type was necessary.  However, this limited
-who edited large files might conceivably exceed.)
+the editor to 64 megabytes total, which some users who edited large
+files might conceivably exceed.)
-Also, note that there is an implicit assumption here that all pointers
+Also, note that there is an implicit assumption here that all pointers
 are low enough that the top bits are all zero and can just be chopped
 off.  On standard machines that allocate memory from the bottom up (and
 give each process its own address space), this works fine.  Some
 machines, however, put the data space somewhere else in memory
 (e.g. beginning at 0x80000000).  Those machines cope by defining
 @code{DATA_SEG_BITS} in the corresponding @file{m/} or @file{s/} file to
 the proper mask.  Then, pointers retrieved from Lisp objects are
 automatically OR'ed with this value prior to being used.
 A corollary of the previous paragraph is that @strong{(pointers to)
 stack-allocated structures cannot be put into Lisp objects}.  The stack
 is generally located near the top of memory; if you put such a pointer
 into a Lisp object, it will get its top bits chopped off, and you will
 lose.
-Various macros are used to construct Lisp objects and extract the
+Actually, there's an alternative representation of a @code{Lisp_Object},
+invented by Kyle Jones, that is used when the
+@code{--use-minimal-tagbits} option to @code{configure} is used.  In
+this case the 2 lower bits are used for the tag bits.  This
+representation assumes that pointers to structs are always aligned to
+multiples of 4, so the lower 2 bits are always zero.
+@example
+[ 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 ]
+[ 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 ]
+<---------------------------------------------------------> <->
+a pointer to a structure, or an integer            tag
+@end example
+A tag of 00 is used for all pointer object types, a tag of 10 is used
+for characters, and the other two tags 01 and 11 are joined together to
+form the integer object type.  The markbit is moved to part of the
+structure being pointed at (integers and chars do not need to be marked,
+since no memory is allocated).  This representation has these
+advantages:
+@enumerate
+@item
+31 bits can be used for Lisp Integers.
+@item
+@emph{Any} pointer can be represented directly, and no bit masking
+operations are necessary.
+@end enumerate
+The disadvantages are:
+@enumerate
+@item
+An extra level of indirection is needed when accessing the object types
+that were not record types.  So checking whether a Lisp object is a cons
+cell becomes a slower operation.
+@item
+Mark bits can no longer be stored directly in Lisp objects, so another
+place for them must be found.  This means that a cons cell requires more
+memory than merely room for 2 lisp objects, leading to extra memory use.
+@end enumerate
+Various macros are used to construct Lisp objects and extract the
 components.  Macros of the form @code{XINT()}, @code{XCHAR()},
 @code{XSTRING()}, @code{XSYMBOL()}, etc. mask out the pointer/integer
 field and cast it to the appropriate type.  All of the macros that
 construct pointers will @code{OR} with @code{DATA_SEG_BITS} if
 necessary.  @code{XINT()} needs to be a bit tricky so that negative
 that it mimics a divide-by-two even for negative numbers).  Not all
 machines/compilers do this, and on the ones that don't, a more
 complicated definition is selected by defining
 @code{EXPLICIT_SIGN_EXTEND}.
 Note that when @code{ERROR_CHECK_TYPECHECK} is defined, the extractor
 macros become more complicated -- they check the tag bits and/or the
 type field in the first four bytes of a record type to ensure that the
 object is really of the correct type.  This is great for catching places
 where an incorrect type is being dereferenced -- this typically results
 in a pointer being dereferenced as the wrong type of structure, with
 unpredictable (and sometimes not easily traceable) results.
-There are similar @code{XSET@var{TYPE}()} macros that construct a Lisp object.
+There are similar @code{XSET@var{TYPE}()} macros that construct a Lisp
-These macros are of the form @code{XSET@var{TYPE} (@var{lvalue}, @var{result})},
+object.  These macros are of the form @code{XSET@var{TYPE}
+(@var{lvalue}, @var{result})},
 i.e. they have to be a statement rather than just used in an expression.
 The reason for this is that standard C doesn't let you ``construct'' a
 structure (but GCC does).  Granted, this sometimes isn't too convenient;
 for the case of integers, at least, you can use the function
 @code{make_int()}, which constructs and @emph{returns} an integer
 Lisp object.  Note that the @code{XSET@var{TYPE}()} macros are also
 affected by @code{ERROR_CHECK_TYPECHECK} and make sure that the
 structure is of the right type in the case of record types, where the
 type is contained in the structure.
+The C programmer is responsible for @strong{guaranteeing} that a
+Lisp_Object is is the correct type before using the @code{X@var{TYPE}}
+macros.  This is especially important in the case of lists.  Use
+@code{XCAR} and @code{XCDR} if a Lisp_Object is certainly a cons cell,
+else use @code{Fcar()} and @code{Fcdr()}.  Trust other C code, but not
+Lisp code.  On the other hand, if XEmacs has an internal logic error,
+it's better to crash immediately, so sprinkle ``unreachable''
+@code{abort()}s liberally about the source code.
 @node Rules When Writing New C Code, A Summary of the Various XEmacs Modules, How Lisp Objects Are Represented in C, Top
 @chapter Rules When Writing New C Code
-The XEmacs C Code is extremely complex and intricate, and there are
+The XEmacs C Code is extremely complex and intricate, and there are many
-many rules that are more or less consistently followed throughout the code.
+rules that are more or less consistently followed throughout the code.
 Many of these rules are not obvious, so they are explained here.  It is
-of the utmost importance that you follow them.  If you don't, you may get
+of the utmost importance that you follow them.  If you don't, you may
-something that appears to work, but which will crash in odd situations,
+get something that appears to work, but which will crash in odd
-often in code far away from where the actual breakage is.
+situations, often in code far away from where the actual breakage is.
 @menu
 * General Coding Rules::
 * Writing Lisp Primitives::
 * Adding Global Lisp Variables::
 @end menu
 @node General Coding Rules
 @section General Coding Rules
-Almost every module contains a @code{syms_of_*()} function and a
+The C code is actually written in a dialect of C called @dfn{Clean C},
+meaning that it can be compiled, mostly warning-free, with either a C or
+C++ compiler.  Coding in Clean C has several advantages over plain C.
+C++ compilers are more nit-picking, and a number of coding errors have
+been found by compiling with C++.  The ability to use both C and C++
+tools means that a greater variety of development tools are available to
+the developer.
+Almost every module contains a @code{syms_of_*()} function and a
 @code{vars_of_*()} function.  The former declares any Lisp primitives
 you have defined and defines any symbols you will be using.  The latter
 declares any global Lisp variables you have added and initializes global
 C variables in the module.  For each such function, declare it in
 @file{symsinit.h} and make sure it's called in the appropriate place in
 you'll be sorry!  If you want to do anything that isn't allowed, create
 a @code{complex_vars_of_*()} function for it.  Doing this is tricky,
 though: You have to make sure your function is called at the right time
 so that all the initialization dependencies work out.
 Every module includes @file{<config.h>} (angle brackets so that
 @samp{--srcdir} works correctly; @file{config.h} may or may not be in
 the same directory as the C sources) and @file{lisp.h}.  @file{config.h}
-should always be included before any other header files (including
+must always be included before any other header files (including
 system header files) to ensure that certain tricks played by various
 @file{s/} and @file{m/} files work out correctly.
 @strong{All global and static variables that are to be modifiable must
-be declared uninitialized.}  This means that you may not use the ``declare
+be declared uninitialized.}  This means that you may not use the
-with initializer'' form for these variables, such as @code{int
+``declare with initializer'' form for these variables, such as @code{int
 some_variable = 0;}.  The reason for this has to do with some kludges
 done during the dumping process: If possible, the initialized data
 segment is re-mapped so that it becomes part of the (unmodifiable) code
 segment in the dumped executable.  This allows this memory to be shared
 among multiple running XEmacs processes.  XEmacs is careful to place as
 much constant data as possible into initialized variables (in
 particular, into what's called the @dfn{pure space} -- see below) during
 the @file{temacs} phase.
 @cindex copy-on-write
-@strong{Please note:} This kludge only works on a few systems
+@strong{Please note:} This kludge only works on a few systems nowadays,
-nowadays, and is rapidly becoming irrelevant because most modern
+and is rapidly becoming irrelevant because most modern operating systems
-operating systems provide @dfn{copy-on-write} semantics.  All data is
+provide @dfn{copy-on-write} semantics.  All data is initially shared
-initially shared between processes, and a private copy is automatically
+between processes, and a private copy is automatically made (on a
-made (on a page-by-page basis) when a process first attempts to write to
+page-by-page basis) when a process first attempts to write to a page of
-a page of memory.
+memory.
-Formerly, there was a requirement that static variables not be
+Formerly, there was a requirement that static variables not be declared
-declared inside of functions.  This had to do with another hack along
+inside of functions.  This had to do with another hack along the same
-the same vein as what was just described: old USG systems put
+vein as what was just described: old USG systems put statically-declared
-statically-declared variables in the initialized data space, so those
+variables in the initialized data space, so those header files had a
-header files had a @code{#define static} declaration. (That way, the
+@code{#define static} declaration. (That way, the data-segment remapping
-data-segment remapping described above could still work.) This fails
+described above could still work.) This fails badly on static variables
-badly on static variables inside of functions, which suddenly become
+inside of functions, which suddenly become automatic variables;
-automatic variables; therefore, you weren't supposed to have any of
+therefore, you weren't supposed to have any of them.  This awful kludge
-them.  This awful kludge has been removed in XEmacs because
+has been removed in XEmacs because
 @enumerate
 @item
 almost all of the systems that used this kludge ended up having
 to disable the data-segment remapping anyway;
 the only systems that didn't were extremely outdated ones;
 @item
 this hack completely messed up inline functions.
 @end enumerate
+The C source code makes heavy use of C preprocessor macros.  One popular
+macro style is:
+@example
+#define FOO(var, value) do @{		\
+Lisp_Object FOO_value = (value);	\
+... /* compute using FOO_value */	\
+(var) = bar;				\
+@} while (0)
+@end example
+The @code{do @{...@} while (0)} is a standard trick to allow FOO to have
+statement semantics, so that it can safely be used within an @code{if}
+statement in C, for example.  Multiple evaluation is prevented by
+copying a supplied argument into a local variable, so that
+@code{FOO(var,fun(1))} only calls @code{fun} once.
+Lisp lists are popular data structures in the C code as well as in
+Elisp.  There are two sets of macros that iterate over lists.
+@code{EXTERNAL_LIST_LOOP_@var{n}} should be used when the list has been
+supplied by the user, and cannot be trusted to be acyclic and
+nil-terminated.  A @code{malformed-list} or @code{circular-list} error
+will be generated if the list being iterated over is not entirely
+kosher.  @code{LIST_LOOP_@var{n}}, on the other hand, is faster and less
+safe, and can be used only on trusted lists.
+Related macros are @code{GET_EXTERNAL_LIST_LENGTH} and
+@code{GET_LIST_LENGTH}, which calculate the length of a list, and in the
+case of @code{GET_EXTERNAL_LIST_LENGTH}, validating the properness of
+the list.  The macros @code{EXTERNAL_LIST_LOOP_DELETE_IF} and
+@code{LIST_LOOP_DELETE_IF} delete elements from a lisp list satisfying some
+predicate.
 @node Writing Lisp Primitives
 @section Writing Lisp Primitives
 Lisp primitives are Lisp functions implemented in C.  The details of
 interfacing the C function so that Lisp can call it are handled by a few
 C macros.  The only way to really understand how to write new C code is
 to read the source, but we can explain some things here.
-An example of a special form is the definition of @code{or}, from
+An example of a special form is the definition of @code{prog1}, from
 @file{eval.c}.  (An ordinary function would have the same general
 appearance.)
 @cindex garbage collection protection
 @smallexample
 @group
-DEFUN ("or", For, 0, UNEVALLED, 0, /*
+DEFUN ("prog1", Fprog1, 1, UNEVALLED, 0, /*
-Eval args until one of them yields non-nil, then return that value.
+Similar to `progn', but the value of the first form is returned.
-The remaining args are not evalled at all.
+\(prog1 FIRST BODY...): All the arguments are evaluated sequentially.
-If all args return nil, return nil.
+The value of FIRST is saved during evaluation of the remaining args,
+whose values are discarded.
 */
 (args))
 @{
 /* This function can GC */
-Lisp_Object val = Qnil;
+REGISTER Lisp_Object val, form, tail;
 struct gcpro gcpro1;
-GCPRO1 (args);
+val = Feval (XCAR (args));
-while (!NILP (args))
+GCPRO1 (val);
-@{
-val = Feval (XCAR (args));
+LIST_LOOP_3 (form, XCDR (args), tail)
-if (!NILP (val))
+Feval (form);
-break;
-args = XCDR (args);
-@}
 UNGCPRO;
 return val;
 @}
 @end group
 Let's start with a precise explanation of the arguments to the
 @code{DEFUN} macro.  Here is a template for them:
 @example
-DEFUN (@var{lname}, @var{fname}, @var{min}, @var{max}, @var{interactive}, /*
+@group
-@var{docstring}
+DEFUN (@var{lname}, @var{fname}, @var{min_args}, @var{max_args}, @var{interactive}, /*
-*/
+@var{docstring}
-(@var{arglist}) )
+*/
+(@var{arglist}))
+@end group
 @end example
 @table @var
 @item lname
 This string is the name of the Lisp symbol to define as the function
-name; in the example above, it is @code{"or"}.
+name; in the example above, it is @code{"prog1"}.
 @item fname
 This is the C function name for this function.  This is the name that is
 used in C code for calling the function.  The name is, by convention,
 @samp{F} prepended to the Lisp name, with all dashes (@samp{-}) in the
 Lisp name changed to underscores.  Thus, to call this function from C
-code, call @code{For}.  Remember that the arguments are of type
+code, call @code{Fprog1}.  Remember that the arguments are of type
 @code{Lisp_Object}; various macros and functions for creating values of
 type @code{Lisp_Object} are declared in the file @file{lisp.h}.
 Primitives whose names are special characters (e.g. @code{+} or
 @code{<}) are named by spelling out, in some fashion, the special
 the subr object that represents the function in Lisp.  This structure
 conveys the Lisp symbol name to the initialization routine that will
 create the symbol and store the subr object as its definition.  The C
 variable name of this structure is always @samp{S} prepended to the
 @var{fname}.  You hardly ever need to be aware of the existence of this
-structure.
+structure, since @code{DEFUN} plus @code{DEFSUBR} takes care of all the
+details.
-@item min
+@item min_args
 This is the minimum number of arguments that the function requires.  The
-function @code{or} allows a minimum of zero arguments.
+function @code{prog1} allows a minimum of one argument.
-@item max
+@item max_args
 This is the maximum number of arguments that the function accepts, if
 there is a fixed maximum.  Alternatively, it can be @code{UNEVALLED},
 indicating a special form that receives unevaluated arguments, or
 @code{MANY}, indicating an unlimited number of evaluated arguments (the
-equivalent of @code{&rest}).  Both @code{UNEVALLED} and @code{MANY} are
+C equivalent of @code{&rest}).  Both @code{UNEVALLED} and @code{MANY}
-macros.  If @var{max} is a number, it may not be less than @var{min} and
+are macros.  If @var{max_args} is a number, it may not be less than
-it may not be greater than 8. (If you need to add a function with
+@var{min_args} and it may not be greater than 8. (If you need to add a
-more than 8 arguments, either use the @code{MANY} form or edit the
+function with more than 8 arguments, use the @code{MANY} form.  Resist
-definition of @code{DEFUN} in @file{lisp.h}.  If you do the latter,
+the urge to edit the definition of @code{DEFUN} in @file{lisp.h}.  If
-make sure to also add another clause to the switch statement in
+you do it anyways, make sure to also add another clause to the switch
-@code{primitive_funcall().})
+statement in @code{primitive_funcall().})
 @item interactive
 This is an interactive specification, a string such as might be used as
 the argument of @code{interactive} in a Lisp function.  In the case of
-@code{or}, it is 0 (a null pointer), indicating that @code{or} cannot be
+@code{prog1}, it is 0 (a null pointer), indicating that @code{prog1}
-called interactively.  A value of @code{""} indicates a function that
+cannot be called interactively.  A value of @code{""} indicates a
-should receive no arguments when called interactively.
+function that should receive no arguments when called interactively.
 @item docstring
 This is the documentation string.  It is written just like a
 documentation string for a function defined in Lisp; in particular, the
 first line should be a single sentence.  Note how the documentation
 comment-start characters are on the same line as the interactive
 specification.  @file{make-docfile}, which scans the C files for
 documentation strings, is very particular about what it looks for, and
 will not properly extract the doc string if it's not in this exact format.
-You are free to put the various arguments to @code{DEFUN} on separate
+In order to make both @file{etags} and @file{make-docfile} happy, make
-lines to avoid overly long lines.  However, make sure to put the
+sure that the @code{DEFUN} line contains the @var{lname} and
-comment-start characters for the doc string on the same line as the
+@var{fname}, and that the comment-start characters for the doc string
-interactive specification, and put a newline directly after them (and
+are on the same line as the interactive specification, and put a newline
-before the comment-end characters).
+directly after them (and before the comment-end characters).
 @item arglist
 This is the comma-separated list of arguments to the C function.  For a
 function with a fixed maximum number of arguments, provide a C argument
 for each Lisp argument.  In this case, unlike regular C functions, the
 types of the arguments are not declared; they are simply always of type
 @code{Lisp_Object}.
 The names of the C arguments will be used as the names of the arguments
 to the Lisp primitive as displayed in its documentation, modulo the same
 concerns described above for @code{F...} names (in particular,
 underscores in the C arguments become dashes in the Lisp arguments).
 discarded when forming the Lisp argument.  This allows C language
 reserved words (like @code{default}) or global symbols (like
 @code{dirname}) to be used as argument names without compiler warnings
 or errors.
-A Lisp function with @w{@var{max} = @code{UNEVALLED}} is a
+A Lisp function with @w{@var{max_args} = @code{UNEVALLED}} is a
 @w{@dfn{special form}}; its arguments are not evaluated.  Instead it
 receives one argument of type @code{Lisp_Object}, a (Lisp) list of the
 unevaluated arguments, conventionally named @code{(args)}.
 When a Lisp function has no upper limit on the number of arguments,
-specify @w{@var{max} = @code{MANY}}.  In this case its implementation in
+specify @w{@var{max_args} = @code{MANY}}.  In this case its implementation in
 C actually receives exactly two arguments: the number of Lisp arguments
 (an @code{int}) and the address of a block containing their values (a
 @w{@code{Lisp_Object *}}).  In this case only are the C types specified
 in the @var{arglist}: @w{@code{(int nargs, Lisp_Object *args)}}.
 @end table
-Within the function @code{For} itself, note the use of the macros
+Within the function @code{Fprog1} itself, note the use of the macros
 @code{GCPRO1} and @code{UNGCPRO}.  @code{GCPRO1} is used to ``protect''
 a variable from garbage collection---to inform the garbage collector
-that it must look in that variable and regard its contents as an
+that it must look in that variable and regard the object pointed at by
-accessible object.  This is necessary whenever you call @code{Feval} or
+its contents as an accessible object.  This is necessary whenever you
-anything that can directly or indirectly call @code{Feval} (this
+call @code{Feval} or anything that can directly or indirectly call
-includes the @code{QUIT} macro!).  At such a time, any Lisp object that
+@code{Feval} (this includes the @code{QUIT} macro!).  At such a time,
-you intend to refer to again must be protected somehow.  @code{UNGCPRO}
+any Lisp object that you intend to refer to again must be protected
-cancels the protection of the variables that are protected in the
+somehow.  @code{UNGCPRO} cancels the protection of the variables that
-current function.  It is necessary to do this explicitly.
+are protected in the current function.  It is necessary to do this
+explicitly.
-The macro @code{GCPRO1} protects just one local variable.  If you want
+The macro @code{GCPRO1} protects just one local variable.  If you want
 to protect two, use @code{GCPRO2} instead; repeating @code{GCPRO1} will
 not work.  Macros @code{GCPRO3} and @code{GCPRO4} also exist.
 These macros implicitly use local variables such as @code{gcpro1}; you
 must declare these explicitly, with type @code{struct gcpro}.  Thus, if
 you use @code{GCPRO2}, you must declare @code{gcpro1} and @code{gcpro2}.
 @cindex caller-protects (@code{GCPRO} rule)
-Note also that the general rule is @dfn{caller-protects}; i.e. you
+Note also that the general rule is @dfn{caller-protects}; i.e. you are
-are only responsible for protecting those Lisp objects that you create.
+only responsible for protecting those Lisp objects that you create.  Any
-Any objects passed to you as parameters should have been protected
+objects passed to you as arguments should have been protected by whoever
-by whoever created them, so you don't in general have to protect them.
+created them, so you don't in general have to protect them.
-@code{For} is an exception; it protects its parameters to provide
-extra assurance against Lisp primitives elsewhere that are incorrectly
+In particular, the arguments to any Lisp primitive are always
-written, and against malicious self-modifying code.  There are a few
+automatically @code{GCPRO}ed, when called ``normally'' from Lisp code or
-other standard functions that also do this.
+bytecode.  So only a few Lisp primitives that are called frequently from
+C code, such as @code{Fprogn} protect their arguments as a service to
-@code{GCPRO}ing is perhaps the trickiest and most error-prone part
+their caller.  You don't need to protect your arguments when writing a
-of XEmacs coding.  It is @strong{extremely} important that you get this
+new @code{DEFUN}.
+@code{GCPRO}ing is perhaps the trickiest and most error-prone part of
+XEmacs coding.  It is @strong{extremely} important that you get this
 right and use a great deal of discipline when writing this code.
 @xref{GCPROing, ,@code{GCPRO}ing}, for full details on how to do this.
-What @code{DEFUN} actually does is declare a global structure of
+What @code{DEFUN} actually does is declare a global structure of type
-type @code{Lisp_Subr} whose name begins with capital @samp{SF} and
+@code{Lisp_Subr} whose name begins with capital @samp{SF} and which
-which contains information about the primitive (e.g. a pointer to the
+contains information about the primitive (e.g. a pointer to the
 function, its minimum and maximum allowed arguments, a string describing
-its Lisp name); @code{DEFUN} then begins a normal C function
+its Lisp name); @code{DEFUN} then begins a normal C function declaration
-declaration using the @code{F...} name.  The Lisp subr object that is
+using the @code{F...} name.  The Lisp subr object that is the function
-the function definition of a primitive (i.e. the object in the function
+definition of a primitive (i.e. the object in the function slot of the
-slot of the symbol that names the primitive) actually points to this
+symbol that names the primitive) actually points to this @samp{SF}
-@samp{SF} structure; when @code{Feval} encounters a subr, it looks in the
+structure; when @code{Feval} encounters a subr, it looks in the
 structure to find out how to call the C function.
 Defining the C function is not enough to make a Lisp primitive
 available; you must also create the Lisp symbol for the primitive (the
 symbol is @dfn{interned}; @pxref{Obarrays}) and store a suitable subr
 object in its function cell. (If you don't do this, the primitive won't
 be seen by Lisp code.) The code looks like this:
 @example
 DEFSUBR (@var{fname});
 @end example
 @noindent
-Here @var{fname} is the name you used as the second argument to
+Here @var{fname} is the same name you used as the second argument to
 @code{DEFUN}.
-This call to @code{DEFSUBR} should go in the @code{syms_of_*()}
+This call to @code{DEFSUBR} should go in the @code{syms_of_*()} function
-function at the end of the module.  If no such function exists, create
+at the end of the module.  If no such function exists, create it and
-it and make sure to also declare it in @file{symsinit.h} and call it
+make sure to also declare it in @file{symsinit.h} and call it from the
-from the appropriate spot in @code{main()}.  @xref{General Coding
+appropriate spot in @code{main()}.  @xref{General Coding Rules}.
-Rules}.
+Note that C code cannot call functions by name unless they are defined
-Note that C code cannot call functions by name unless they are defined
 in C.  The way to call a function written in Lisp from C is to use
 @code{Ffuncall}, which embodies the Lisp function @code{funcall}.  Since
 the Lisp function @code{funcall} accepts an unlimited number of
 arguments, in C it takes two: the number of Lisp-level arguments, and a
 one-dimensional array containing their values.  The first Lisp-level
 argument is the Lisp function to call, and the rest are the arguments to
 pass to it.  Since @code{Ffuncall} can call the evaluator, you must
 protect pointers from garbage collection around the call to
 @code{Ffuncall}. (However, @code{Ffuncall} explicitly protects all of
-its parameters, so you don't have to protect any pointers passed
+its parameters, so you don't have to protect any pointers passed as
-as parameters to it.)
+parameters to it.)
 The C functions @code{call0}, @code{call1}, @code{call2}, and so on,
 provide handy ways to call a Lisp function conveniently with a fixed
 number of arguments.  They work by calling @code{Ffuncall}.
 @file{eval.c} is a very good file to look through for examples;
-@file{lisp.h} contains the definitions for some important macros and
+@file{lisp.h} contains the definitions for important macros and
 functions.
 @node Adding Global Lisp Variables
 @section Adding Global Lisp Variables
 Global variables whose names begin with @samp{Q} are constants whose
 value is a symbol of a particular name.  The name of the variable should
 be derived from the name of the symbol using the same rules as for Lisp
 primitives.  These variables are initialized using a call to
 @code{defsymbol()} in the @code{syms_of_*()} function. (This call
 interns a symbol, sets the C variable to the resulting Lisp object, and
 @{
 Charcount cclen;
 ...
 @{
 /* Allocate place for @var{cclen} characters. */
-Bufbyte *tmp_buf = (Bufbyte *)alloca (cclen * MAX_EMCHAR_LEN);
+Bufbyte *buf = (Bufbyte *)alloca (cclen * MAX_EMCHAR_LEN);
 ...
 @end group
 @end example
 If you followed the previous section, you can guess that, logically,
 multiplying a @code{Charcount} value with @code{MAX_EMCHAR_LEN} produces
 a @code{Bytecount} value.
 In the current Mule implementation, @code{MAX_EMCHAR_LEN} equals 4.
 Without Mule, it is 1.
 When an external function, such as a C library function, returns a
 @code{char} pointer, you should almost never treat it as @code{Bufbyte}.
 This is because these returned strings may contain 8bit characters which
 can be misinterpreted by XEmacs, and cause a crash.  Likewise, when
 exporting a piece of internal text to the outside world, you should
 always convert it to an appropriate external encoding, lest the internal
 stuff (such as the infamous \201 characters) leak out.
 The interface to conversion between the internal and external
 representations of text are the numerous conversion macros defined in
 @file{buffer.h}.  Before looking at them, we'll look at the external
 formats supported by these macros.
 Currently meaningful formats are @code{FORMAT_BINARY},
 @code{FORMAT_FILENAME}, @code{FORMAT_OS}, and @code{FORMAT_CTEXT}.  Here
 is a description of these.
 @table @code
 @item FORMAT_BINARY
 Binary format.  This is the simplest format and is what we use in the
 Compound--text format.  This is the standard X format used for data
 stored in properties, selections, and the like.  This is an 8-bit
 no-lock-shift ISO2022 coding system.
 @end table
 The macros to convert between these formats and the internal format, and
 vice versa, follow.
 @table @code
 @item GET_CHARPTR_INT_DATA_ALLOCA
 @itemx GET_CHARPTR_EXT_DATA_ALLOCA
 @code{Lisp_Object} and @code{make_char}.  If you want a pointer to move
 through the internal text, use @code{Bufbyte *}.  Also note that you
 almost certainly do not need @code{Emchar *}.
 @item Be careful not to confuse @code{Charcount}, @code{Bytecount}, and @code{Bufpos}.
 The whole point of using different types is to avoid confusion about the
 use of certain variables.  Lest this effect be nullified, you need to be
 careful about using the right types.
 @item Always convert external data
 It is extremely important to always convert external data, because
 XEmacs can crash if unexpected 8bit sequences are copied to its internal
 buffers literally.
 This means that when a system function, such as @code{readdir}, returns
 a string, you need to convert it using one of the conversion macros
 described in the previous chapter, before passing it further to Lisp.
 doing, @code{CHECK_CHAR} will also suffice.  @code{XCHAR (lisp_char)}
 extracts the @code{Emchar} from the @code{Lisp_Object}, and
 @code{set_charptr_emchar} stores it to storage, increasing @code{p} in
 the process.
-Other instructing examples of correct coding under Mule can be found all
+Other instructive examples of correct coding under Mule can be found all
-over XEmacs code.  For starters, I recommend
+over the XEmacs code.  For starters, I recommend
 @code{Fnormalize_menu_item_name} in @file{menubar.c}.  After you have
 understood this section of the manual and studied the examples, you can
 proceed writing new Mule-aware code.
 @node Techniques for XEmacs Developers
 @section Techniques for XEmacs Developers
 To make a quantified XEmacs, do: @code{make quantmacs}.
 You simply can't dump Quantified and Purified images.  Run the image
-like so:  @code{quantmacs -batch -l loadup.el run-temacs -q}.
+like so:  @code{quantmacs -batch -l loadup.el run-temacs @var{xemacs-args...}}.
 Before you go through the trouble, are you compiling with all
 debugging and error-checking off?  If not try that first.  Be warned
 that while Quantify is directly responsible for quite a few
 optimizations which have been made to XEmacs, doing a run which
 recording right before it shuts down (which generates enough bogus data
 to throw most results off).  It also enables three additional elisp
 commands: @code{quantify-start-recording-data},
 @code{quantify-stop-recording-data} and @code{quantify-clear-data}.
+If you want to make XEmacs faster, target your favorite slow benchmark,
+run a profiler like Quantify, @code{gprof}, or @code{tcov}, and figure
+out where the cycles are going.  Specific projects:
+@itemize @bullet
+@item
+Make the garbage collector faster.  Figure out how to write an
+incremental garbage collector.
+@item
+Write a compiler that takes bytecode and spits out C code.
+Unfortunately, you will then need a C compiler and a more fully
+developed module system.
+@item
+Speed up redisplay.
+@item
+Speed up syntax highlighting.  Maybe moving some of the syntax
+highlighting capabilities into C would make a difference.
+@item
+Implement tail recursion in Emacs Lisp (hard!).
+@end itemize
+Unfortunately, Emacs Lisp is slow, and is going to stay slow.  Function
+calls in elisp are especially expensive.  Iterating over a long list is
+going to be 30 times faster implemented in C than in Elisp.
 To get started debugging XEmacs, take a look at the @file{gdbinit} and
 @file{dbxrc} files in the @file{src} directory.
 @xref{Q2.1.15 - How to Debug an XEmacs problem with a debugger,,,
 xemacs-faq, XEmacs FAQ}.
+After making source code changes, run @code{make check} to ensure that
+you haven't introduced any regressions.  If you're feeling ambitious,
+you can try to improve the test suite in @file{tests/automated}.
 Here are things to know when you create a new source file:
 @itemize @bullet
 @item
-All .c files should @code{#include <config.h>} first.  Almost all .c
+All @file{.c} files should @code{#include <config.h>} first.  Almost all
-files should @code{#include "lisp.h"} second.
+@file{.c} files should @code{#include "lisp.h"} second.
 @item
-Generated header files should be included using the @code{<>} syntax,
+Generated header files should be included using the @code{#include <...>} syntax,
-not the @code{""} syntax.  The generated headers are:
+not the @code{#include "..."} syntax.  The generated headers are:
-config.h puresize-adjust.h sheap-adjust.h paths.h Emacs.ad.h
+@file{config.h puresize-adjust.h sheap-adjust.h paths.h Emacs.ad.h}
 The basic rule is that you should assume builds using @code{--srcdir}
-and the @code{<>} syntax needs to be used when the to-be-included
+and the @code{#include <...>} syntax needs to be used when the
-generated file is in a potentially different directory
+to-be-included generated file is in a potentially different directory
-@emph{at compile time}.
+@emph{at compile time}.  The non-obvious C rule is that @code{#include "..."}
+means to search for the included file in the same directory as the
-@item
+including file, @emph{not} in the current directory.
-Header files should not include <config.h> and "lisp.h".   It is the
-responsibility of the .c files that use it to do so.
+@item
+Header files should @emph{not} include @code{<config.h>} and
-@item
+@code{"lisp.h"}.  It is the responsibility of the @file{.c} files that
-If the header uses INLINE, either directly or though DECLARE_LRECORD,
+use it to do so.
-then it must be added to inline.c's includes.
+@item
-@item
+If the header uses @code{INLINE}, either directly or though
-Try compiling at least once with
+@code{DECLARE_LRECORD}, then it must be added to @file{inline.c}'s
+includes.
+@item
+Try compiling at least once with
 @example
 gcc --with-mule --with-union-type --error-checking=all
 @end example
+@item
+Did I mention that you should run the test suite?
+@example
+make check
+@end example
 @end itemize
 @node A Summary of the Various XEmacs Modules, Allocation of Objects in XEmacs Lisp, Rules When Writing New C Code, Top
 @chapter A Summary of the Various XEmacs Modules
 This is accurate as of XEmacs 20.0.
 @node Low-Level Modules
 @section Low-Level Modules
 @example
-size  name
+config.h
--------  ---------------------
-18150  config.h
 @end example
 This is automatically generated from @file{config.h.in} based on the
 results of configure tests and user-selected optional features and
 contains preprocessor definitions specifying the nature of the
 environment in which XEmacs is being compiled.
 @example
-2347  paths.h
+paths.h
 @end example
 This is automatically generated from @file{paths.h.in} based on supplied
 configure values, and allows for non-standard installed configurations
 of the XEmacs directories.  It's currently broken, though.
 @example
-47878  emacs.c
+emacs.c
-20239  signal.c
+signal.c
 @end example
 @file{emacs.c} contains @code{main()} and other code that performs the most
 basic environment initializations and handles shutting down the XEmacs
 process (this includes @code{kill-emacs}, the normal way that XEmacs is
 @file{syssignal.h} header file, described in section J below.
 @example
-23458  unexaix.c
+unexaix.c
-9893  unexalpha.c
+unexalpha.c
-11302  unexapollo.c
+unexapollo.c
-16544  unexconvex.c
+unexconvex.c
-31967  unexec.c
+unexec.c
-30959  unexelf.c
+unexelf.c
-35791  unexelfsgi.c
+unexelfsgi.c
-3207  unexencap.c
+unexencap.c
-7276  unexenix.c
+unexenix.c
-20539  unexfreebsd.c
+unexfreebsd.c
-1153  unexfx2800.c
+unexfx2800.c
-13432  unexhp9k3.c
+unexhp9k3.c
-11049  unexhp9k800.c
+unexhp9k800.c
-9165  unexmips.c
+unexmips.c
-8981  unexnext.c
+unexnext.c
-1673  unexsol2.c
+unexsol2.c
-19261  unexsunos4.c
+unexsunos4.c
 @end example
 These modules contain code dumping out the XEmacs executable on various
 different systems. (This process is highly machine-specific and
 requires intimate knowledge of the executable format and the memory map
 chosen by @file{configure}.
 @example
-15715  crt0.c
+crt0.c
-1484  lastfile.c
+lastfile.c
-1115  pre-crt0.c
+pre-crt0.c
 @end example
 These modules are used in conjunction with the dump mechanism.  On some
 systems, an alternative version of the C startup code (the actual code
 that receives control from the operating system when the process is
 data space when dumping.
 @example
-14786  alloca.c
+alloca.c
-16678  free-hook.c
+free-hook.c
-1692  getpagesize.h
+getpagesize.h
-41936  gmalloc.c
+gmalloc.c
-25141  malloc.c
+malloc.c
-3802  mem-limits.h
+mem-limits.h
-39011  ralloc.c
+ralloc.c
-3436  vm-limit.c
+vm-limit.c
 @end example
 These handle basic C allocation of memory.  @file{alloca.c} is an emulation of
 the stack allocation function @code{alloca()} on machines that lack
 this. (XEmacs makes extensive use of @code{alloca()} in its code.)
 and should always be preferred if it works. (At one point, @file{gmalloc.c}
 didn't work on some systems where @file{malloc.c} worked; but this should be
 fixed now.)
 @cindex relocating allocator
-@file{ralloc.c} is the @dfn{relocating allocator}.  It provides functions
+@file{ralloc.c} is the @dfn{relocating allocator}.  It provides
-similar to @code{malloc()}, @code{realloc()} and @code{free()} that allocate
+functions similar to @code{malloc()}, @code{realloc()} and @code{free()}
-memory that can be dynamically relocated in memory.  The advantage of
+that allocate memory that can be dynamically relocated in memory.  The
-this is that allocated memory can be shuffled around to place all the
+advantage of this is that allocated memory can be shuffled around to
-free memory at the end of the heap, and the heap can then be shrunk,
+place all the free memory at the end of the heap, and the heap can then
-releasing the memory back to the operating system.  The use of this can
+be shrunk, releasing the memory back to the operating system.  The use
-be controlled with the configure option @code{--rel-alloc}; if enabled, memory allocated for
+of this can be controlled with the configure option @code{--rel-alloc};
-buffers will be relocatable, so that if a very large file is visited and
+if enabled, memory allocated for buffers will be relocatable, so that if
-the buffer is later killed, the memory can be released to the operating
+a very large file is visited and the buffer is later killed, the memory
-system.  (The disadvantage of this mechanism is that it can be very
+can be released to the operating system.  (The disadvantage of this
-slow.  On systems with the @code{mmap()} system call, the XEmacs version
+mechanism is that it can be very slow.  On systems with the
-of @file{ralloc.c} uses this to move memory around without actually having to
+@code{mmap()} system call, the XEmacs version of @file{ralloc.c} uses
-block-copy it, which can speed things up; but it can still cause
+this to move memory around without actually having to block-copy it,
-noticeable performance degradation.)
+which can speed things up; but it can still cause noticeable performance
+degradation.)
 @file{free-hook.c} contains some debugging functions for checking for invalid
 arguments to @code{free()}.
 @file{vm-limit.c} contains some functions that warn the user when memory is
 similar in spirit to the @file{sys*.h} files described in section J, below.
 @example
-2659  blocktype.c
+blocktype.c
-1410  blocktype.h
+blocktype.h
-7194  dynarr.c
+dynarr.c
-2671  dynarr.h
 @end example
 These implement a couple of basic C data types to facilitate memory
 allocation.  The @code{Blocktype} type efficiently manages the
 allocation of fixed-size blocks by minimizing the number of times that
 mechanism.
 @example
-2058  inline.c
+inline.c
 @end example
 This module is used in connection with inline functions (available in
 some compilers).  Often, inline functions need to have a corresponding
 non-inline function that does the same thing.  This module is where they
 function definitions, so that each one gets a real function equivalent.
 @example
-6489  debug.c
+debug.c
-2267  debug.h
+debug.h
 @end example
 These functions provide a system for doing internal consistency checks
 during code development.  This system is not currently used; instead the
 simpler @code{assert()} macro is used along with the various checks
 provided by the @samp{--error-check-*} configuration options.
 @example
-1643  prefix-args.c
+prefix-args.c
 @end example
 This is actually the source for a small, self-contained program
 used during building.
 @example
-904  universe.h
+universe.h
 @end example
 This is not currently used.
 @node Basic Lisp Modules
 @section Basic Lisp Modules
 @example
-size  name
+emacsfns.h
--------  ---------------------
+lisp-disunion.h
-70167  emacsfns.h
+lisp-union.h
-6305  lisp-disunion.h
+lisp.h
-7086  lisp-union.h
+lrecord.h
-54929  lisp.h
+symsinit.h
-14235  lrecord.h
-10728  symsinit.h
 @end example
 These are the basic header files for all XEmacs modules.  Each module
 includes @file{lisp.h}, which brings the other header files in.
 @file{lisp.h} contains the definitions of the structures and extractor
 low-level macros.
 As a general rule, all typedefs should go into the typedefs section of
 @file{lisp.h} rather than into a module-specific header file even if the
 structure is defined elsewhere.  This allows function prototypes that
-use the typedef to be placed into @file{emacsfns.h}.  Forward structure
+use the typedef to placed into other header files.  Forward structure
 declarations (i.e. a simple declaration like @code{struct foo;} where
 the structure itself is defined elsewhere) should be placed into the
 typedefs section as necessary.
 @file{lrecord.h} contains the basic structures and macros that implement
 all record-type Lisp objects -- i.e. all objects whose type is a field
 in their C structure, which includes all objects except the few most
 basic ones.
-@file{emacsfns.h} contains prototypes for most of the exported functions
+@file{lisp.h} contains prototypes for most of the exported functions in
-in the various modules. (In particular, prototypes for Lisp primitives
+the various modules.  Lisp primitives defined using @code{DEFUN} that
-should always go into this header file.  Prototypes for other functions
+need to be called by C code should be declared using @code{EXFUN}.
-can either go here or in a module-specific header file, depending on how
+Other function prototypes should be placed either into the appropriate
-general-purpose the function is and whether it has special-purpose
+section of @code{lisp.h}, or into a module-specific header file,
-argument types requiring definitions not in @file{lisp.h}.)  All
+depending on how general-purpose the function is and whether it has
-initialization functions are prototyped in @file{symsinit.h}.
+special-purpose argument types requiring definitions not in
+@file{lisp.h}.)  All initialization functions are prototyped in
+@file{symsinit.h}.
-@example
-120478  alloc.c
-1029  pure.c
+@example
-2506  puresize.h
+alloc.c
+pure.c
+puresize.h
 @end example
 The large module @file{alloc.c} implements all of the basic allocation and
 garbage collection for Lisp objects.  The most commonly used Lisp
 objects are allocated in chunks, similar to the Blocktype data type
 pure space is needed.
 @example
-122243  eval.c
+eval.c
-2305  backtrace.h
+backtrace.h
 @end example
 This module contains all of the functions to handle the flow of control.
 This includes the mechanisms of defining functions, calling functions,
 traversing stack frames, and binding variables; the control primitives
 flow of control.
 @example
-64949  lread.c
+lread.c
 @end example
 This module implements the Lisp reader and the @code{read} function,
 which converts text into Lisp objects, according to the read syntax of
 the objects, as described above.  This is similar to the parser that is
 a part of all compilers.
 @example
-40900  print.c
+print.c
 @end example
 This module implements the Lisp print mechanism and the @code{print}
 function and related functions.  This is the inverse of the Lisp reader
 -- it converts Lisp objects to a printed, textual representation.
 an equivalent object.)
 @example
-4518  general.c
+general.c
-60220  symbols.c
+symbols.c
-9966  symeval.h
+symeval.h
 @end example
 @file{symbols.c} implements the handling of symbols, obarrays, and
 retrieving the values of symbols.  Much of the code is devoted to
 handling the special @dfn{symbol-value-magic} objects that define
 @code{DEFVAR_LISP()} and related macros for declaring variables.
 @example
-48973  data.c
+data.c
-25694  floatfns.c
+floatfns.c
-71049  fns.c
+fns.c
 @end example
 These modules implement the methods and standard Lisp primitives for all
 the basic Lisp object types other than symbols (which are described
 above).  @file{data.c} contains all the predicates (primitives that return
 arithmetic.
 @example
-23555  bytecode.c
+bytecode.c
-3358  bytecode.h
+bytecode.h
 @end example
-@file{bytecode.c} implements the byte-code interpreter, and @file{bytecode.h} contains
+@file{bytecode.c} implements the byte-code interpreter and
-associated structures.  Note that the byte-code @emph{compiler} is
+compiled-function objects, and @file{bytecode.h} contains associated
-written in Lisp.
+structures.  Note that the byte-code @emph{compiler} is written in Lisp.
 @node Modules for Standard Editing Operations
 @section Modules for Standard Editing Operations
 @example
-size  name
+buffer.c
--------  ---------------------
+buffer.h
-82900  buffer.c
+bufslots.h
-60964  buffer.h
-6059  bufslots.h
 @end example
 @file{buffer.c} implements the @dfn{buffer} Lisp object type.  This
 includes functions that create and destroy buffers; retrieve buffers by
 name or by other properties; manipulate lists of buffers (remember that
 the built-in buffer-local variables.
 @example
-79888  insdel.c
+insdel.c
-6103  insdel.h
+insdel.h
 @end example
 @file{insdel.c} contains low-level functions for inserting and deleting text in
 a buffer, keeping track of changed regions for use by redisplay, and
 calling any before-change and after-change functions that may have been
 @file{insdel.h} contains associated headers.
 @example
-10975  marker.c
+marker.c
 @end example
 This module implements the @dfn{marker} Lisp object type, which
 conceptually is a pointer to a text position in a buffer that moves
 around as text is inserted and deleted, so as to remain in the same
 current buffer position of the marker.
 @example
-193714  extents.c
+extents.c
-15686  extents.h
+extents.h
 @end example
 This module implements the @dfn{extent} Lisp object type, which is like
 a marker that works over a range of text rather than a single position.
 Extents are also much more complex and powerful than markers and have a
 cover.)
 @example
-60155  editfns.c
+editfns.c
 @end example
 @file{editfns.c} contains the standard Lisp primitives for working with
 a buffer's text, and calls the low-level functions in @file{insdel.c}.
 It also contains primitives for working with @code{point} (the default
 @file{editfns.c}.
 @example
-26081  callint.c
+callint.c
-12577  cmds.c
+cmds.c
-2749  commands.h
+commands.h
 @end example
 @cindex interactive
 These modules implement the basic @dfn{interactive} commands,
 i.e. user-callable functions.  Commands, as opposed to other functions,
 @file{commands.h} contains associated structure definitions and prototypes.
 @example
-194863  regex.c
+regex.c
-18968  regex.h
+regex.h
-79800  search.c
+search.c
 @end example
 @file{search.c} implements the Lisp primitives for searching for text in
 a buffer, and some of the low-level algorithms for doing this.  In
 particular, the fast fixed-string Boyer-Moore search algorithm is
 routines used in @file{grep} and other GNU utilities.
 @example
-20476  doprnt.c
+doprnt.c
 @end example
 @file{doprnt.c} implements formatted-string processing, similar to
 @code{printf()} command in C.
 @example
-15372  undo.c
+undo.c
 @end example
 This module implements the undo mechanism for tracking buffer changes.
 Most of this could be implemented in Lisp.
 @node Editor-Level Control Flow Modules
 @section Editor-Level Control Flow Modules
 @example
-size  name
+event-Xt.c
--------  ---------------------
+event-stream.c
-84546  event-Xt.c
+event-tty.c
-121483  event-stream.c
+events.c
-6658  event-tty.c
+events.h
-49271  events.c
-14459  events.h
 @end example
 These implement the handling of events (user input and other system
 notifications).
 events from all different kinds of frames.
 @example
-129583  keymap.c
+keymap.c
-2621  keymap.h
+keymap.h
 @end example
 @file{keymap.c} and @file{keymap.h} define the @dfn{keymap} Lisp object
 type and associated methods and primitives. (Remember that keymaps are
 objects that associate event descriptions with functions to be called to
 relevant keymaps.)
 @example
-25212  keyboard.c
+keyboard.c
 @end example
 @file{keyboard.c} contains functions that implement the actual editor
 command loop -- i.e. the event loop that cyclically retrieves and
 dispatches events.  This code is also rather tricky, just like
 @file{event-stream.c}.
 @example
-9973  macros.c
+macros.c
-1397  macros.h
+macros.h
 @end example
 These two modules contain the basic code for defining keyboard macros.
 These functions don't actually do much; most of the code that handles keyboard
 macros is mixed in with the event-handling code in @file{event-stream.c}.
 @example
-23234  minibuf.c
+minibuf.c
 @end example
 This contains some miscellaneous code related to the minibuffer (most of
 the minibuffer code was moved into Lisp by Richard Mlynarik).  This
 includes the primitives for completion (although filename completion is
 @node Modules for the Basic Displayable Lisp Objects
 @section Modules for the Basic Displayable Lisp Objects
 @example
-size  name
+device-ns.h
--------  ---------------------
+device-stream.c
-985  device-ns.h
+device-stream.h
-6454  device-stream.c
+device-tty.c
-1196  device-stream.h
+device-tty.h
-9526  device-tty.c
+device-x.c
-8660  device-tty.h
+device-x.h
-43798  device-x.c
+device.c
-11667  device-x.h
+device.h
-26056  device.c
-22993  device.h
 @end example
 These modules implement the @dfn{device} Lisp object type.  This
 abstracts a particular screen or connection on which frames are
 displayed.  As with Lisp objects, event interfaces, and other
 subtypes (X, TTY, NeXTstep, Microsoft Windows, etc.) as devices do.
 @example
-934  frame-ns.h
+frame-ns.h
-2303  frame-tty.c
+frame-tty.c
-69205  frame-x.c
+frame-x.c
-5976  frame-x.h
+frame-x.h
-68175  frame.c
+frame.c
-15080  frame.h
+frame.h
 @end example
 Each device contains one or more frames in which objects (e.g. text) are
 displayed.  A frame corresponds to a window in the window system;
 usually this is a top-level window but it could potentially be one of a
 (e.g. raising, lowering, resizing, moving, etc.).
 @example
-160783  window.c
+window.c
-15974  window.h
+window.h
 @end example
 @cindex window (in Emacs)
 @cindex pane
 Each frame consists of one or more non-overlapping @dfn{windows} (better
 @node Modules for other Display-Related Lisp Objects
 @section Modules for other Display-Related Lisp Objects
 @example
-size  name
+faces.c
--------  ---------------------
+faces.h
-54397  faces.c
+@end example
-15173  faces.h
-@end example
+@example
+bitmaps.h
-@example
+glyphs-ns.h
-4961  bitmaps.h
+glyphs-x.c
-954  glyphs-ns.h
+glyphs-x.h
-105345  glyphs-x.c
+glyphs.c
-4288  glyphs-x.h
+glyphs.h
-72102  glyphs.c
+@end example
-16356  glyphs.h
-@end example
+@example
+objects-ns.h
-@example
+objects-tty.c
-952  objects-ns.h
+objects-tty.h
-9971  objects-tty.c
+objects-x.c
-1465  objects-tty.h
+objects-x.h
-32326  objects-x.c
+objects.c
-2806  objects-x.h
+objects.h
-31944  objects.c
+@end example
-6809  objects.h
-@end example
+@example
+menubar-x.c
-@example
+menubar.c
-57511  menubar-x.c
+@end example
-11243  menubar.c
-@end example
+@example
+scrollbar-x.c
-@example
+scrollbar-x.h
-25012  scrollbar-x.c
+scrollbar.c
-2554  scrollbar-x.h
+scrollbar.h
-26954  scrollbar.c
+@end example
-2778  scrollbar.h
-@end example
+@example
+toolbar-x.c
-@example
+toolbar.c
-23117  toolbar-x.c
+toolbar.h
-43456  toolbar.c
+@end example
-4280  toolbar.h
-@end example
+@example
+font-lock.c
-@example
-25070  font-lock.c
 @end example
 This file provides C support for syntax highlighting -- i.e.
 highlighting different syntactic constructs of a source file in
 different colors, for easy reading.  The C support is provided so that
 this is fast.
 @example
-32180  dgif_lib.c
+dgif_lib.c
-3999  gif_err.c
+gif_err.c
-10697  gif_lib.h
+gif_lib.h
-9371  gifalloc.c
+gifalloc.c
 @end example
 These modules decode GIF-format image files, for use with glyphs.
 @node Modules for the Redisplay Mechanism
 @section Modules for the Redisplay Mechanism
 @example
-size  name
+redisplay-output.c
--------  ---------------------
+redisplay-tty.c
-38692  redisplay-output.c
+redisplay-x.c
-40835  redisplay-tty.c
+redisplay.c
-65069  redisplay-x.c
+redisplay.h
-234142  redisplay.c
-17026  redisplay.h
 @end example
 These files provide the redisplay mechanism.  As with many other
 subsystems in XEmacs, there is a clean separation between the general
 and device-specific support.
 respectively.
 @example
-14129  indent.c
+indent.c
 @end example
 This module contains various functions and Lisp primitives for
 converting between buffer positions and screen positions.  These
 functions call the redisplay mechanism to do most of the work, and then
 module needs work.
 @example
-14754  termcap.c
+termcap.c
-2141  terminfo.c
+terminfo.c
-7253  tparam.c
+tparam.c
 @end example
 These files contain functions for working with the termcap (BSD-style)
 and terminfo (System V style) databases of terminal capabilities and
 escape sequences, used when XEmacs is displaying in a TTY.
 @example
-10869  cm.c
+cm.c
-5876  cm.h
+cm.h
 @end example
 These files provide some miscellaneous TTY-output functions and should
 probably be merged into @file{redisplay-tty.c}.
 @node Modules for Interfacing with the File System
 @section Modules for Interfacing with the File System
 @example
-size  name
+lstream.c
--------  ---------------------
+lstream.h
-43362  lstream.c
-14240  lstream.h
 @end example
 These modules implement the @dfn{stream} Lisp object type.  This is an
 internal-only Lisp object that implements a generic buffering stream.
 The idea is to provide a uniform interface onto all sources and sinks of
 types of streams; others are provided, e.g., in @file{mule-coding.c}.
 @example
-126926  fileio.c
+fileio.c
 @end example
 This implements the basic primitives for interfacing with the file
 system.  This includes primitives for reading files into buffers,
 writing buffers into files, checking for the presence or accessibility
 @file{simple.el}.
 @example
-10960  filelock.c
+filelock.c
 @end example
 This file provides functions for detecting clashes between different
 processes (e.g. XEmacs and some external process, or two different
 XEmacs processes) modifying the same file.  (XEmacs can optionally use
 modified, the user is made aware of this so that the buffer can be
 synched up with the external changes if necessary.
 @example
-4527  filemode.c
+filemode.c
 @end example
 This file provides some miscellaneous functions that construct a
 @samp{rwxr-xr-x}-type permissions string (as might appear in an
 @file{ls}-style directory listing) given the information returned by the
 @code{stat()} system call.
 @example
-22855  dired.c
+dired.c
-2094  ndir.h
+ndir.h
 @end example
 These files implement the XEmacs interface to directory searching.  This
 includes a number of primitives for determining the files in a directory
 and for doing filename completion. (Remember that generic completion is
 those systems, directories can be read directly as files, and parsed.)
 @example
-4311  realpath.c
+realpath.c
 @end example
 This file provides an implementation of the @code{realpath()} function
 for expanding symbolic links, on systems that don't implement it or have
 a broken implementation.
 @node Modules for Other Aspects of the Lisp Interpreter and Object System
 @section Modules for Other Aspects of the Lisp Interpreter and Object System
 @example
-size  name
+elhash.c
--------  ---------------------
+elhash.h
-22290  elhash.c
+hash.c
-2454  elhash.h
+hash.h
-12169  hash.c
+@end example
-3369  hash.h
-@end example
+These files provide two implementations of hash tables.  Files
-These files implement the @dfn{hashtable} Lisp object type.
 @file{hash.c} and @file{hash.h} provide a generic C implementation of
-hash tables (which can stand independently of XEmacs), and
+hash tables which can stand independently of XEmacs.  Files
-@file{elhash.c} and @file{elhash.h} provide a Lisp interface onto the C
+@file{elhash.c} and @file{elhash.h} provide a separate implementation of
-hash tables using the hashtable Lisp object type.
+hash tables that can store only Lisp objects, and knows about Lispy
+things like garbage collection, and implement the @dfn{hash-table} Lisp
+object type.
-@example
-95691  specifier.c
+@example
-11167  specifier.h
+specifier.c
+specifier.h
 @end example
 This module implements the @dfn{specifier} Lisp object type.  This is
 primarily used for displayable properties, and allows for values that
 are specific to a particular buffer, window, frame, device, or device
 looks up a value given a window (from which a buffer, frame, and device
 can be derived).
 @example
-43058  chartab.c
+chartab.c
-6503  chartab.h
+chartab.h
-9918  casetab.c
+casetab.c
 @end example
 @file{chartab.c} and @file{chartab.h} implement the @dfn{char table}
 Lisp object type, which maps from characters or certain sorts of
 character ranges to Lisp objects.  The implementation of this object
 and to do case-insensitive searching.
 @example
-49593  syntax.c
+syntax.c
-10200  syntax.h
+syntax.h
 @end example
 @cindex scanner
 This module implements @dfn{syntax tables}, another sort of char table
 that maps characters into syntax classes that define the syntax of these
 comments, etc.
 @example
-10438  casefiddle.c
+casefiddle.c
 @end example
 This module implements various Lisp primitives for upcasing, downcasing
 and capitalizing strings or regions of buffers.
 @example
-20234  rangetab.c
+rangetab.c
 @end example
 This module implements the @dfn{range table} Lisp object type, which
 provides for a mapping from ranges of integers to arbitrary Lisp
 objects.
 @example
-3201  opaque.c
+opaque.c
-2206  opaque.h
+opaque.h
 @end example
 This module implements the @dfn{opaque} Lisp object type, an
 internal-only Lisp object that encapsulates an arbitrary block of memory
 so that it can be managed by the Lisp allocation system.  To create an
 create a new Lisp object type -- it's not hard.)
 @example
-8783  abbrev.c
+abbrev.c
 @end example
 This function provides a few primitives for doing dynamic abbreviation
 expansion.  In XEmacs, most of the code for this has been moved into
 Lisp.  Some C code remains for speed and because the primitive
 is itself in C only for speed.)
 @example
-21934  doc.c
+doc.c
 @end example
 This function provides primitives for retrieving the documentation
 strings of functions and variables.  These documentation strings contain
 certain special markers that get dynamically expanded (e.g. a
 the appropriate documentation string.)
 @example
-13197  md5.c
+md5.c
 @end example
 This function provides a Lisp primitive that implements the MD5 secure
 hashing scheme, used to create a large hash value of a string of data such that
 the data cannot be derived from the hash value.  This is used for
 @node Modules for Interfacing with the Operating System
 @section Modules for Interfacing with the Operating System
 @example
-size  name
+callproc.c
--------  ---------------------
+process.c
-33533  callproc.c
+process.h
-89697  process.c
-4663  process.h
 @end example
 These modules allow XEmacs to spawn and communicate with subprocesses
 and network connections.
 subprocesses.
 @example
-136029  sysdep.c
+sysdep.c
-5986  sysdep.h
+sysdep.h
 @end example
 These modules implement most of the low-level, messy operating-system
 interface code.  This includes various device control (ioctl) operations
 for file descriptors, TTY's, pseudo-terminals, etc. (usually this stuff
 provide them or have broken versions.
 @example
-3605  sysdir.h
+sysdir.h
-6708  sysfile.h
+sysfile.h
-2027  sysfloat.h
+sysfloat.h
-2918  sysproc.h
+sysproc.h
-745  syspwd.h
+syspwd.h
-7643  syssignal.h
+syssignal.h
-6892  systime.h
+systime.h
-12477  systty.h
+systty.h
-3487  syswait.h
+syswait.h
 @end example
 These header files provide consistent interfaces onto system-dependent
 header files and system calls.  The idea is that, instead of including a
 standard header file like @file{<sys/param.h>} (which may or may not
 an int).
 @example
-7940  hpplay.c
+hpplay.c
-10920  libsst.c
+libsst.c
-1480  libsst.h
+libsst.h
-3260  libst.h
+libst.h
-15355  linuxplay.c
+linuxplay.c
-15849  nas.c
+nas.c
-19133  sgiplay.c
+sgiplay.c
-15411  sound.c
+sound.c
-7358  sunplay.c
+sunplay.c
 @end example
 These files implement the ability to play various sounds on some types
 of computers.  You have to configure your XEmacs with sound support in
 order to get this capability.
 currently in use.
 @example
-44368  tooltalk.c
+tooltalk.c
-2137  tooltalk.h
+tooltalk.h
 @end example
 These two modules implement an interface to the ToolTalk protocol, which
 is an interprocess communication protocol implemented on some versions
 of Unix.  ToolTalk is a high-level protocol that allows processes to
 parts of the SPARCWorks development environment.
 @example
-22695  getloadavg.c
+getloadavg.c
 @end example
 This module provides the ability to retrieve the system's current load
 average. (The way to do this is highly system-specific, unfortunately,
 and requires a lot of special-case code.)
 @example
-148520  energize.c
+sunpro.c
-6896  energize.h
-@end example
-This module provides code to interface to an Energize server (when
-XEmacs is used as part of Lucid's Energize development environment) and
-provides some other Energize-specific functions.  Much of the code in
-this module should be made more general-purpose and moved elsewhere, but
-is no longer very relevant now that Lucid is defunct.  It also hasn't
-worked since version 19.12, since nobody has been maintaining it.
-@example
-2861  sunpro.c
 @end example
 This module provides a small amount of code used internally at Sun to
 keep statistics on the usage of XEmacs.
 @example
-5548  broken-sun.h
+broken-sun.h
-3468  strcmp.c
+strcmp.c
-2179  strcpy.c
+strcpy.c
-1650  sunOS-fix.c
+sunOS-fix.c
 @end example
 These files provide replacement functions and prototypes to fix numerous
 bugs in early releases of SunOS 4.1.
 @example
-11669  hftctl.c
+hftctl.c
 @end example
 This module provides some terminal-control code necessary on versions of
 AIX prior to 4.1.
 @example
-1776  acldef.h
+msdos.c
-1602  chpdef.h
+msdos.h
-9032  uaf.h
-105  vlimit.h
-7145  vms-pp.c
-1158  vms-pwd.h
-26532  vmsfns.c
-6038  vmsmap.c
-695  vmspaths.h
-17482  vmsproc.c
-469  vmsproc.h
-@end example
-All of these files are used for VMS support, which has never worked in
-XEmacs.
-@example
-28316  msdos.c
-1472  msdos.h
 @end example
 These modules are used for MS-DOS support, which does not work in
 XEmacs.
 @node Modules for Interfacing with X Windows
 @section Modules for Interfacing with X Windows
 @example
-size  name
+Emacs.ad.h
--------  ---------------------
-3196  Emacs.ad.h
 @end example
 A file generated from @file{Emacs.ad}, which contains XEmacs-supplied
 fallback resources (so that XEmacs has pretty defaults).
 @example
-24242  EmacsFrame.c
+EmacsFrame.c
-6979  EmacsFrame.h
+EmacsFrame.h
-3351  EmacsFrameP.h
+EmacsFrameP.h
 @end example
 These modules implement an Xt widget class that encapsulates a frame.
 This is for ease in integrating with Xt.  The EmacsFrame widget covers
 the entire X window except for the menubar; the scrollbars are
 mercilessly at the slightest change.  Such is life under Xt.
 @example
-8178  EmacsManager.c
+EmacsManager.c
-1967  EmacsManager.h
+EmacsManager.h
-1895  EmacsManagerP.h
+EmacsManagerP.h
 @end example
 These modules implement a simple Xt manager (i.e. composite) widget
 class that simply lets its children set whatever geometry they want.
 It's amazing that Xt doesn't provide this standardly, but on second
 thought, it makes sense, considering how amazingly broken Xt is.
 @example
-13188  EmacsShell-sub.c
+EmacsShell-sub.c
-4588  EmacsShell.c
+EmacsShell.c
-2180  EmacsShell.h
+EmacsShell.h
-3133  EmacsShellP.h
+EmacsShellP.h
 @end example
 These modules implement two Xt widget classes that are subclasses of
 the TopLevelShell and TransientShell classes.  This is necessary to deal
 with more brokenness that Xt has sadistically thrust onto the backs of
 developers.
 @example
-9673  xgccache.c
+xgccache.c
-1111  xgccache.h
+xgccache.h
 @end example
 These modules provide functions for maintenance and caching of GC's
 (graphics contexts) under the X Window System.  This code is junky and
 needs to be rewritten.
 @example
-69181  xselect.c
+xselect.c
 @end example
 @cindex selections
 This module provides an interface to the X Window System's concept of
 @dfn{selections}, the standard way for X applications to communicate
 with each other.
 @example
-929  xintrinsic.h
+xintrinsic.h
-1038  xintrinsicp.h
+xintrinsicp.h
-1579  xmmanagerp.h
+xmmanagerp.h
-1585  xmprimitivep.h
+xmprimitivep.h
 @end example
 These header files are similar in spirit to the @file{sys*.h} files and buffer
 against different implementations of Xt and Motif.
 @end itemize
 @example
-16930  xmu.c
+xmu.c
-936  xmu.h
+xmu.h
 @end example
 These files provide an emulation of the Xmu library for those systems
 (i.e. HPUX) that don't provide it as a standard part of X.
 @example
-4201  ExternalClient-Xlib.c
+ExternalClient-Xlib.c
-18083  ExternalClient.c
+ExternalClient.c
-2035  ExternalClient.h
+ExternalClient.h
-2104  ExternalClientP.h
+ExternalClientP.h
-22684  ExternalShell.c
+ExternalShell.c
-1709  ExternalShell.h
+ExternalShell.h
-1971  ExternalShellP.h
+ExternalShellP.h
-2478  extw-Xlib.c
+extw-Xlib.c
-1481  extw-Xlib.h
+extw-Xlib.h
-6565  extw-Xt.c
+extw-Xt.c
-1430  extw-Xt.h
+extw-Xt.h
 @end example
 @cindex external widget
 These files provide the @dfn{external widget} interface, which allows an
 XEmacs frame to appear as a widget in another application.  To do this,
 Don't touch this code; something is liable to break if you do.
-@example
-31014  epoch.c
-@end example
-This file provides some additional, Epoch-compatible, functionality for
-interfacing to the X Window System.
 @node Modules for Internationalization
 @section Modules for Internationalization
 @example
-size  name
+mule-canna.c
--------  ---------------------
+mule-ccl.c
-42836  mule-canna.c
+mule-charset.c
-16737  mule-ccl.c
+mule-charset.h
-41080  mule-charset.c
+mule-coding.c
-30176  mule-charset.h
+mule-coding.h
-146844  mule-coding.c
+mule-mcpath.c
-16588  mule-coding.h
+mule-mcpath.h
-6996  mule-mcpath.c
+mule-wnnfns.c
-2899  mule-mcpath.h
+mule.c
-57158  mule-wnnfns.c
-3351  mule.c
 @end example
 These files implement the MULE (Asian-language) support.  Note that MULE
 actually provides a general interface for all sorts of languages, not
 just Asian languages (although they are generally the most complicated
 @file{mule-charset.*} and @file{mule-coding.*} provide the heart of the
 XEmacs MULE support.  @file{mule-charset.*} implements the @dfn{charset}
 Lisp object type, which encapsulates a character set (an ordered one- or
 two-dimensional set of characters, such as US ASCII or JISX0208 Japanese
 Kanji).
 @file{mule-coding.*} implements the @dfn{coding-system} Lisp object
 type, which encapsulates a method of converting between different
 encodings.  An encoding is a representation of a stream of characters,
 possibly from multiple character sets, using a stream of bytes or words,
 be elsewhere.
 @example
-9400  intl.c
+intl.c
 @end example
 This provides some miscellaneous internationalization code for
 implementing message translation and interfacing to the Ximp input
 method.  None of this code is currently working.
 @example
-1764  iso-wide.h
+iso-wide.h
 @end example
 This contains leftover code from an earlier implementation of
 Asian-language support, and is not currently used.
 * Vector::
 * Bit Vector::
 * Symbol::
 * Marker::
 * String::
-* Bytecode::
+* Compiled Function::
 @end menu
 @node Introduction to Allocation
 @section Introduction to Allocation
 @itemize @bullet
 @item
 (a) Those for whom the value directly represents the contents of the
 Lisp object.  Only two types are in this category: integers and
 characters.  No special allocation or garbage collection is necessary
 for such objects.  Lisp objects of these types do not need to be
 @code{GCPRO}ed.
 @end itemize
 In the remaining three categories, the value is a pointer to a
 structure.
 @itemize @bullet
 @item
 (c) Those lrecords that are allocated in frob blocks (see above).  This
 includes the objects that are most common and relatively small, and
-includes floats, bytecodes, symbols (when not in category (b)), extents,
+includes floats, compiled functions, symbols (when not in category (b)),
-events, and markers.  With the cleanup of frob blocks done in 19.12,
+extents, events, and markers.  With the cleanup of frob blocks done in
-it's not terribly hard to add more objects to this category, but it's a
+19.12, it's not terribly hard to add more objects to this category, but
-bit trickier than adding an object type to type (d) (esp. if the object
+it's a bit trickier than adding an object type to type (d) (esp. if the
-needs a finalization method), and is not likely to save much space
+object needs a finalization method), and is not likely to save much
-unless the object is small and there are many of them. (In fact, if
+space unless the object is small and there are many of them. (In fact,
-there are very few of them, it might actually waste space.)
+if there are very few of them, it might actually waste space.)
 @item
 (d) Those lrecords that are individually @code{malloc()}ed.  These are
 called @dfn{lcrecords}.  All other types are in this category.  Adding a
 new type to this category is comparatively easy, and all types added
 since 19.8 (when the current allocation scheme was devised, by Richard
 string data (which would normally be obtained from the now-non-existent
 @code{struct Lisp_String}) at the beginning of the dead string data gap.
 The string compactor recognizes this special 0xFFFFFFFF marker and
 handles it correctly.
-@node Bytecode
+@node Compiled Function
-@section Bytecode
+@section Compiled Function
 Not yet documented.
 @node Events and the Event Loop, Evaluation; Stack Frames; Bindings, Allocation of Objects in XEmacs Lisp, Top
 @chapter Events and the Event Loop
 at work in XEmacs, under TTY's (TTY's are simpler than X
 so we'll look at this first):
 @noindent
 @example
-asynch.      asynch.    asynch.   asynch.                [Collectors in
+asynch.      asynch.    asynch.   asynch.             [Collectors in
-kbd events  kbd events   process   process                   the OS]
+kbd events  kbd events   process   process                the OS]
 |         |         output    output
 |         |           |         |
-|         |           |         |      SIGINT,      [signal handlers
+|         |           |         |      SIGINT,   [signal handlers
-|         |           |         |      SIGQUIT,        in XEmacs]
+|         |           |         |      SIGQUIT,     in XEmacs]
 V         V           V         V      SIGWINCH,
 file      file        file      file    SIGALRM
 desc.     desc.       desc.     desc.     |
 (TTY)     (TTY)       (pipe)    (pipe)    |
 |          |          |         |      fake    timeouts
 |          |          |         |        |         |
 |          |          |         |        |         |
 |          |          |         |        |         |
 V          V          V         V        V         V
 ------>-----------<----------------<----------------
 |
 |
-|   [collected using select() in emacs_tty_next_event()
+| [collected using select() in emacs_tty_next_event()
-|    and converted to the appropriate Emacs event]
+|  and converted to the appropriate Emacs event]
 |
 |
-V              (above this line is TTY-specific)
+V          (above this line is TTY-specific)
-Emacs    ------------------------------------------------
+Emacs -----------------------------------------------
-event    (below this line is the generic event mechanism)
+event (below this line is the generic event mechanism)
 |
 |
-was there      if not, call
+was there     if not, call
-a SIGINT?   emacs_tty_next_event()
+a SIGINT?  emacs_tty_next_event()
-|              |
+|             |
-|              |
+|             |
-|              |
+|             |
-V              V
+V             V
---->-------<----
+--->------<----
 |
-|        [collected in event_stream_next_event();
+|     [collected in event_stream_next_event();
-|         SIGINT is converted using maybe_read_quit_event()]
+|      SIGINT is converted using maybe_read_quit_event()]
 V
 Emacs
 event
 |
 \---->------>----- maybe_kbd_translate() ---->---\
 |
 |
 |
 command event queue                                    |
 if not from command
-(contains events that were                     event queue, call
+(contains events that were                   event queue, call
-read earlier but not processed,                event_stream_next_event()
+read earlier but not processed,              event_stream_next_event()
 typically when waiting in a                               |
 sit-for, sleep-for, etc. for                              |
 a particular event to be received)                         |
 |                                            |
 |                                            |
 V                                            V
 ---->------------------------------------<----
 |
-|   [collected in
+| [collected in
-|    next_event_internal()]
+|  next_event_internal()]
 |
 unread-     unread-       event from          |
 command-    command-       keyboard       else, call
 events      event           macro      next_event_internal()
 |           |               |               |
 it's the same as the above diagram):
 @example
 asynch. asynch. asynch. asynch.                 [Collectors in
 kbd     kbd    process process                    the OS]
 events  events  output  output
 |       |       |       |
-|       |       |       |     asynch. asynch.   [Collectors in the
+|       |       |       |     asynch. asynch. [Collectors in the
-|       |       |       |       X        X       OS and X Window System]
+|       |       |       |       X        X     OS and X Window System]
 |       |       |       |     events  events
 |       |       |       |       |        |
 |       |       |       |       |        |
-|       |       |       |       |        |    SIGINT,   [signal handlers
+|       |       |       |       |        |    SIGINT, [signal handlers
-|       |       |       |       |        |    SIGQUIT,     in XEmacs]
+|       |       |       |       |        |    SIGQUIT,   in XEmacs]
 |       |       |       |       |        |    SIGWINCH,
 |       |       |       |       |        |    SIGALRM
 |       |       |       |       |        |       |
 |       |       |       |       |        |       |
 |       |       |       |       |        |       |      timeouts
 |       |       |       |       |        |       |          |
 |       |       |       |       |        |       |          |
 |       |       |       |       |        |       V          |
 V       V       V       V       V        V      fake        |
 file    file    file    file    file     file    file        |
 desc.   desc.   desc.   desc.   desc.    desc.   desc.       |
 (TTY)   (TTY)   (pipe)  (pipe) (socket) (socket) (pipe)      |
 |       |       |       |       |        |       |          |
 |       |       |       |       |        |       |          |
 |       |       |       |       |        |       |          |
 V       V       V       V       V        V       V          V
 --->----------------------------------------<---------<------
 |              |               |
-|              |               |   [collected using select() in
+|              |               |[collected using select() in
-|              |               |   _XtWaitForSomething(), called
+|              |               | _XtWaitForSomething(), called
-|              |               |   from XtAppProcessEvent(), called
+|              |               | from XtAppProcessEvent(), called
-|              |               |   in emacs_Xt_next_event();
+|              |               | in emacs_Xt_next_event();
-|              |               |   dispatched to various callbacks]
+|              |               | dispatched to various callbacks]
 |              |               |
 |              |               |
-emacs_Xt_        p_s_callback(),    |   [popup_selection_callback]
+emacs_Xt_        p_s_callback(),    | [popup_selection_callback]
-event_handler()  x_u_v_s_callback(),|   [x_update_vertical_scrollbar_
+event_handler()  x_u_v_s_callback(),| [x_update_vertical_scrollbar_
-|           x_u_h_s_callback(),|    callback]
+|           x_u_h_s_callback(),|  callback]
-|           search_callback()  |   [x_update_horizontal_scrollbar_
+|           search_callback()  | [x_update_horizontal_scrollbar_
-|              |               |    callback]
+|              |               |  callback]
 |              |               |
 |              |               |
 enqueue_Xt_       signal_special_   |
 dispatch_event()  Xt_user_event()   |
 [maybe multiple     |               |
 |              |               |
 V              V               |
 -->----------<--               |
 |                       |
 |                       |
 dispatch             Xt_what_callback()
 event                  sets flags
 queue                      |
 |                       |
 |                       |
 |                       |
 |                       |
 ---->-----------<--------
 |
 |
 |     [collected and converted as appropriate in
 |            emacs_Xt_next_event()]
 |
 |
-V              (above this line is Xt-specific)
+V          (above this line is Xt-specific)
-Emacs    ------------------------------------------------
+Emacs ------------------------------------------------
-event    (below this line is the generic event mechanism)
+event (below this line is the generic event mechanism)
 |
 |
 was there      if not, call
 a SIGINT?   emacs_Xt_next_event()
 |              |
 \---->------>----- maybe_kbd_translate() -->-----\
 |
 |
 |
 command event queue                                    |
 if not from command
-(contains events that were                     event queue, call
+(contains events that were                  event queue, call
-read earlier but not processed,                event_stream_next_event()
+read earlier but not processed,             event_stream_next_event()
 typically when waiting in a                               |
 sit-for, sleep-for, etc. for                              |
 a particular event to be received)                         |
 |                                            |
 |                                            |
 V                                            V
 ---->----------------------------------<------
 |
-|   [collected in
+| [collected in
-|    next_event_internal()]
+|  next_event_internal()]
 |
 unread-     unread-       event from          |
 command-    command-       keyboard       else, call
 events      event           macro      next_event_internal()
 |           |               |               |
 @section Evaluation
 @code{Feval()} evaluates the form (a Lisp object) that is passed to
 it.  Note that evaluation is only non-trivial for two types of objects:
 symbols and conses.  A symbol is evaluated simply by calling
-symbol-value on it and returning the value.
+@code{symbol-value} on it and returning the value.
 Evaluating a cons means calling a function.  First, @code{eval} checks
 to see if garbage-collection is necessary, and calls
-@code{Fgarbage_collect()} if so.  It then increases the evaluation depth
+@code{garbage_collect_1()} if so.  It then increases the evaluation
-by 1 (@code{lisp_eval_depth}, which is always less than @code{max_lisp_eval_depth}) and adds an
+depth by 1 (@code{lisp_eval_depth}, which is always less than
-element to the linked list of @code{struct backtrace}'s
+@code{max_lisp_eval_depth}) and adds an element to the linked list of
-(@code{backtrace_list}).  Each such structure contains a pointer to the
+@code{struct backtrace}'s (@code{backtrace_list}).  Each such structure
-function being called plus a list of the function's arguments.
+contains a pointer to the function being called plus a list of the
-Originally these values are stored unevalled, and as they are evaluated,
+function's arguments.  Originally these values are stored unevalled, and
-the backtrace structure is updated.  Garbage collection pays attention
+as they are evaluated, the backtrace structure is updated.  Garbage
-to the objects pointed to in the backtrace structures (garbage
+collection pays attention to the objects pointed to in the backtrace
-collection might happen while a function is being called or while an
+structures (garbage collection might happen while a function is being
-argument is being evaluated, and there could easily be no other
+called or while an argument is being evaluated, and there could easily
-references to the arguments in the argument list; once an argument is
+be no other references to the arguments in the argument list; once an
-evaluated, however, the unevalled version is not needed by eval, and so
+argument is evaluated, however, the unevalled version is not needed by
-the backtrace structure is changed).
+eval, and so the backtrace structure is changed).
 At this point, the function to be called is determined by looking at
 the car of the cons (if this is a symbol, its function definition is
 retrieved and the process repeated).  The function should then consist
-of either a @code{Lisp_Subr} (built-in function), a
+of either a @code{Lisp_Subr} (built-in function written in C), a
-@code{Lisp_Compiled_Function} object, or a cons whose car is the symbol
+@code{Lisp_Compiled_Function} object, or a cons whose car is one of the
-@code{autoload}, @code{macro} or @code{lambda}.
+symbols @code{autoload}, @code{macro} or @code{lambda}.
 If the function is a @code{Lisp_Subr}, the lisp object points to a
 @code{struct Lisp_Subr} (created by @code{DEFUN()}), which contains a
 pointer to the C function, a minimum and maximum number of arguments
-(possibly the special constants @code{MANY} or @code{UNEVALLED}), a
+(or possibly the special constants @code{MANY} or @code{UNEVALLED}), a
 pointer to the symbol referring to that subr, and a couple of other
 things.  If the subr wants its arguments @code{UNEVALLED}, they are
 passed raw as a list.  Otherwise, an array of evaluated arguments is
 created and put into the backtrace structure, and either passed whole
 (@code{MANY}) or each argument is passed as a C argument.
-If the function is a @code{Lisp_Compiled_Function} object or a lambda,
+If the function is a @code{Lisp_Compiled_Function},
-@code{apply_lambda()} is called.  If the function is a macro,
+@code{funcall_compiled_function()} is called.  If the function is a
-[..... fill in] is done.  If the function is an autoload,
+lambda list, @code{funcall_lambda()} is called.  If the function is a
+macro, [..... fill in] is done.  If the function is an autoload,
 @code{do_autoload()} is called to load the definition and then eval
 starts over [explain this more].
-When @code{Feval} exits, the evaluation depth is reduced by one, the
+When @code{Feval()} exits, the evaluation depth is reduced by one, the
 debugger is called if appropriate, and the current backtrace structure
 is removed from the list.
-@code{apply_lambda()} is passed a function, a list of arguments, and a
+Both @code{funcall_compiled_function()} and @code{funcall_lambda()} need
-flag indicating whether to evaluate the arguments.  It creates an array
+to go through the list of formal parameters to the function and bind
-of (possibly) evaluated arguments and fixes up the backtrace structure,
+them to the actual arguments, checking for @code{&rest} and
-just like eval does.  Then it calls @code{funcall_lambda()}.
+@code{&optional} symbols in the formal parameters and making sure the
+number of actual arguments is correct.
-@code{funcall_lambda()} goes through the formal arguments to the
+@code{funcall_compiled_function()} can do this a little more
-function and binds them to the actual arguments, checking for
+efficiently, since the formal parameter list can be checked for sanity
-@code{&rest} and @code{&optional} symbols in the formal arguments and
+when the compiled function object is created.
-making sure the number of actual arguments is correct.  Then either
-@code{progn} or @code{byte-code} is called to actually execute the body
+@code{funcall_lambda()} simply calls @code{Fprogn} to execute the code
-and return a value.
+in the lambda list.
-@code{Ffuncall()} implements Lisp @code{funcall}.  @code{(funcall fun
+@code{funcall_compiled_function()} calls the real byte-code interpreter
+@code{execute_optimized_program()} on the byte-code instructions, which
+are converted into an internal form for faster execution.
+When a compiled function is executed for the first time by
+@code{funcall_compiled_function()}, or when it is @code{Fpurecopy()}ed
+during the dump phase of building XEmacs, the byte-code instructions are
+converted from a @code{Lisp_String} (which is inefficient to access,
+especially in the presence of MULE) into a @code{Lisp_Opaque} object
+containing an array of unsigned char, which can be directly executed by
+the byte-code interpreter.  At this time the byte code is also analyzed
+for validity and transformed into a more optimized form, so that
+@code{execute_optimized_program()} can really fly.
+Here are some of the optimizations performed by the internal byte-code
+transformer:
+@enumerate
+@item
+References to the @code{constants} array are checked for out-of-range
+indices, so that the byte interpreter doesn't have to.
+@item
+References to the @code{constants} array that will be used as a Lisp
+variable are checked for being correct non-constant (i.e. not @code{t},
+@code{nil}, or @code{keywordp}) symbols, so that the byte interpreter
+doesn't have to.
+@item
+The maxiumum number of variable bindings in the byte-code is
+pre-computed, so that space on the @code{specpdl} stack can be
+pre-reserved once for the whole function execution.
+@item
+All byte-code jumps are relative to the current program counter instead
+of the start of the program, thereby saving a register.
+@item
+One-byte relative jumps are converted from the byte-code form of unsigned
+chars offset by 127 to machine-friendly signed chars.
+@end enumerate
+Of course, this transformation of the @code{instructions} should not be
+visible to the user, so @code{Fcompiled_function_instructions()} needs
+to know how to convert the optimized opaque object back into a Lisp
+string that is identical to the original string from the @file{.elc}
+file.  (Actually, the resulting string may (rarely) contain slightly
+different, yet equivalent, byte code.)
+@code{Ffuncall()} implements Lisp @code{funcall}.  @code{(funcall fun
 x1 x2 x3 ...)} is equivalent to @code{(eval (list fun (quote x1) (quote
 x2) (quote x3) ...))}.  @code{Ffuncall()} contains its own code to do
-the evaluation, however, and is almost identical to eval.
+the evaluation, however, and is very similar to @code{Feval()}.
-@code{Fapply()} implements Lisp @code{apply}, which is very similar to
+From the performance point of view, it is worth knowing that most of the
+time in Lisp evaluation is spent executing @code{Lisp_Subr} and
+@code{Lisp_Compiled_Function} objects via @code{Ffuncall()} (not
+@code{Feval()}).
+@code{Fapply()} implements Lisp @code{apply}, which is very similar to
 @code{funcall} except that if the last argument is a list, the result is the
 same as if each of the arguments in the list had been passed separately.
 @code{Fapply()} does some business to expand the last argument if it's a
 list, then calls @code{Ffuncall()} to do the work.
 @code{apply1()}, @code{call0()}, @code{call1()}, @code{call2()}, and
 @code{call3()} call a function, passing it the argument(s) given (the
 arguments are given as separate C arguments rather than being passed as
-an array).  @code{apply1()} uses @code{apply} while the others use
+an array).  @code{apply1()} uses @code{Fapply()} while the others use
-@code{funcall}.
+@code{Ffuncall()} to do the real work.
 @node Dynamic Binding; The specbinding Stack; Unwind-Protects
 @section Dynamic Binding; The specbinding Stack; Unwind-Protects
 @example
 struct specbinding
 @{
-Lisp_Object symbol, old_value;
+Lisp_Object symbol;
+Lisp_Object old_value;
 Lisp_Object (*func) (Lisp_Object); /* for unwind-protect */
 @};
 @end example
 @code{struct specbinding} is used for local-variable bindings and
 @code{or}, @code{and}, @code{if}, @code{cond}, @code{progn},
 @code{prog1}, @code{prog2}, @code{setq}, @code{quote}, @code{function},
 @code{let*}, @code{let}, @code{while}
 All of these are very simple and work as expected, calling
 @code{Feval()} or @code{Fprogn()} as necessary and (in the case of
 @code{let} and @code{let*}) using @code{specbind()} to create bindings
-and @code{unbind_to()} to undo the bindings when finished.  Note that
+and @code{unbind_to()} to undo the bindings when finished.
-these functions do a lot of @code{GCPRO}ing to protect their arguments
-from garbage collection because they call @code{Feval()} (@pxref{Garbage
+Note that, with the exeption of @code{Fprogn}, these functions are
-Collection}).
+typically called in real life only in interpreted code, since the byte
+compiler knows how to convert calls to these functions directly into
+byte code.
 @node Catch and Throw
 @section Catch and Throw
 @example
 can temporarily change the current buffer using @code{set-buffer} (often
 enclosed in a @code{save-excursion} so that the former current buffer
 gets restored when the code is finished).  However, calling
 @code{set-buffer} will NOT cause a permanent change in the current
 buffer.  The reason for this is that the top-level event loop sets
 @code{current_buffer} to the buffer of the selected window, each time
 it finishes executing a user command.
 @end enumerate
 Make sure you understand the distinction between @dfn{current buffer}
 and @dfn{buffer of the selected window}, and the distinction between
 @end menu
 @node Japanese EUC (Extended Unix Code)
 @subsection Japanese EUC (Extended Unix Code)
-This encompasses the character sets Printing-ASCII, Japanese-JISSX0201,
+This encompasses the character sets Printing-ASCII, Japanese-JISX0201,
 and Japanese-JISX0208-Kana (half-width katakana, the right half of
 JISX0201).  It uses 8-bit bytes.
 Note that Printing-ASCII and Japanese-JISX0201-Kana are 94-character
 charsets, while Japanese-JISX0208 is a 94x94-character charset.
 @node CCL
 @section CCL
 @example
 CCL PROGRAM SYNTAX:
 CCL_PROGRAM := (CCL_MAIN_BLOCK
 [ CCL_EOF_BLOCK ])
 CCL_MAIN_BLOCK := CCL_BLOCK
 CCL_EOF_BLOCK := CCL_BLOCK
 CCL_BLOCK := STATEMENT | (STATEMENT [STATEMENT ...])
 STATEMENT :=
 SET | IF | BRANCH | LOOP | REPEAT | BREAK
 | READ | WRITE
 SET := (REG = EXPRESSION) | (REG SELF_OP EXPRESSION)
 | INT-OR-CHAR
 EXPRESSION := ARG | (EXPRESSION OP ARG)
 IF := (if EXPRESSION CCL_BLOCK CCL_BLOCK)
 BRANCH := (branch EXPRESSION CCL_BLOCK [CCL_BLOCK ...])
 LOOP := (loop STATEMENT [STATEMENT ...])
 BREAK := (break)
 REPEAT := (repeat)
 | (write-repeat [REG | INT-OR-CHAR | string])
 | (write-read-repeat REG [INT-OR-CHAR | string | ARRAY]?)
 READ := (read REG) | (read REG REG)
 | (read-if REG ARITH_OP ARG CCL_BLOCK CCL_BLOCK)
 | (read-branch REG CCL_BLOCK [CCL_BLOCK ...])
 WRITE := (write REG) | (write REG REG)
 | (write INT-OR-CHAR) | (write STRING) | STRING
 | (write REG ARRAY)
 END := (end)
 REG := r0 | r1 | r2 | r3 | r4 | r5 | r6 | r7
 ARG := REG | INT-OR-CHAR
 OP :=   + | - | * | / | % | & | '|' | ^ | << | >> | <8 | >8 | //
 | < | > | == | <= | >= | !=
 SELF_OP :=
 += | -= | *= | /= | %= | &= | '|=' | ^= | <<= | >>=
 ARRAY := '[' INT-OR-CHAR ... ']'
 INT-OR-CHAR := INT | CHAR
 MACHINE CODE:
 The machine code consists of a vector of 32-bit words.
 The first such word specifies the start of the EOF section of the code;
 RRR (3-bit): register number
 XXXXXXXXXXXXXXXX (15-bit):
 CCCCCCCCCCCCCCC: constant or address
 000000000000rrr: register number
 AAAA:   00000 +
 00001 -
 00010 *
 00011 /
 00100 %
 00101 &
 00110 |
 00111 ~
 01000 <<
 01001 >>
 01010 <8
 01100 //
 01101 not used
 01110 not used
 01111 not used
 10000 <
 10001 >
 10010 ==
 10011 <=
 10100 >=
 10101 !=
 OPERATORS:      TTTTT RRR XX..
-SetCS:          00000 RRR C...C         RRR = C...C
+SetCS:          00000 RRR C...C      RRR = C...C
-SetCL:          00001 RRR .....         RRR = c...c
+SetCL:          00001 RRR .....      RRR = c...c
 c.............c
-SetR:           00010 RRR ..rrr         RRR = rrr
+SetR:           00010 RRR ..rrr      RRR = rrr
-SetA:           00011 RRR ..rrr         RRR = array[rrr]
+SetA:           00011 RRR ..rrr      RRR = array[rrr]
-C.............C         size of array = C...C
+C.............C      size of array = C...C
-c.............c         contents = c...c
+c.............c      contents = c...c
-Jump:           00100 000 c...c         jump to c...c
+Jump:           00100 000 c...c      jump to c...c
-JumpCond:       00101 RRR c...c         if (!RRR) jump to c...c
+JumpCond:       00101 RRR c...c      if (!RRR) jump to c...c
-WriteJump:      00110 RRR c...c         Write1 RRR, jump to c...c
+WriteJump:      00110 RRR c...c      Write1 RRR, jump to c...c
-WriteReadJump:  00111 RRR c...c         Write1, Read1 RRR, jump to c...c
+WriteReadJump:  00111 RRR c...c      Write1, Read1 RRR, jump to c...c
-WriteCJump:     01000 000 c...c         Write1 C...C, jump to c...c
+WriteCJump:     01000 000 c...c      Write1 C...C, jump to c...c
 C...C
-WriteCReadJump: 01001 RRR c...c         Write1 C...C, Read1 RRR,
+WriteCReadJump: 01001 RRR c...c      Write1 C...C, Read1 RRR,
-C.............C         and jump to c...c
+C.............C      and jump to c...c
-WriteSJump:     01010 000 c...c         WriteS, jump to c...c
+WriteSJump:     01010 000 c...c      WriteS, jump to c...c
 C.............C
 S.............S
 ...
-WriteSReadJump: 01011 RRR c...c         WriteS, Read1 RRR, jump to c...c
+WriteSReadJump: 01011 RRR c...c      WriteS, Read1 RRR, jump to c...c
 C.............C
 S.............S
 ...
-WriteAReadJump: 01100 RRR c...c         WriteA, Read1 RRR, jump to c...c
+WriteAReadJump: 01100 RRR c...c      WriteA, Read1 RRR, jump to c...c
-C.............C         size of array = C...C
+C.............C      size of array = C...C
-c.............c         contents = c...c
+c.............c      contents = c...c
 ...
-Branch:         01101 RRR C...C         if (RRR >= 0 && RRR < C..)
+Branch:         01101 RRR C...C      if (RRR >= 0 && RRR < C..)
-c.............c         branch to (RRR+1)th address
+c.............c      branch to (RRR+1)th address
-Read1:          01110 RRR ...           read 1-byte to RRR
+Read1:          01110 RRR ...        read 1-byte to RRR
-Read2:          01111 RRR ..rrr         read 2-byte to RRR and rrr
+Read2:          01111 RRR ..rrr      read 2-byte to RRR and rrr
-ReadBranch:     10000 RRR C...C         Read1 and Branch
+ReadBranch:     10000 RRR C...C      Read1 and Branch
 c.............c
 ...
-Write1:         10001 RRR .....         write 1-byte RRR
+Write1:         10001 RRR .....      write 1-byte RRR
-Write2:         10010 RRR ..rrr         write 2-byte RRR and rrr
+Write2:         10010 RRR ..rrr      write 2-byte RRR and rrr
-WriteC:         10011 000 .....         write 1-char C...CC
+WriteC:         10011 000 .....      write 1-char C...CC
 C.............C
-WriteS:         10100 000 .....         write C..-byte of string
+WriteS:         10100 000 .....      write C..-byte of string
 C.............C
 S.............S
 ...
-WriteA:         10101 RRR .....         write array[RRR]
+WriteA:         10101 RRR .....      write array[RRR]
-C.............C         size of array = C...C
+C.............C      size of array = C...C
-c.............c         contents = c...c
+c.............c      contents = c...c
 ...
-End:            10110 000 .....         terminate the execution
+End:            10110 000 .....      terminate the execution
-SetSelfCS:      10111 RRR C...C         RRR AAAAA= C...C
+SetSelfCS:      10111 RRR C...C      RRR AAAAA= C...C
 ..........AAAAA
-SetSelfCL:      11000 RRR .....         RRR AAAAA= c...c
+SetSelfCL:      11000 RRR .....      RRR AAAAA= c...c
 c.............c
 ..........AAAAA
-SetSelfR:       11001 RRR ..Rrr         RRR AAAAA= rrr
+SetSelfR:       11001 RRR ..Rrr      RRR AAAAA= rrr
 ..........AAAAA
-SetExprCL:      11010 RRR ..Rrr         RRR = rrr AAAAA c...c
+SetExprCL:      11010 RRR ..Rrr      RRR = rrr AAAAA c...c
 c.............c
 ..........AAAAA
-SetExprR:       11011 RRR ..rrr         RRR = rrr AAAAA Rrr
+SetExprR:       11011 RRR ..rrr      RRR = rrr AAAAA Rrr
 ............Rrr
 ..........AAAAA
-JumpCondC:      11100 RRR c...c         if !(RRR AAAAA C..) jump to c...c
+JumpCondC:      11100 RRR c...c      if !(RRR AAAAA C..) jump to c...c
 C.............C
 ..........AAAAA
-JumpCondR:      11101 RRR c...c         if !(RRR AAAAA rrr) jump to c...c
+JumpCondR:      11101 RRR c...c      if !(RRR AAAAA rrr) jump to c...c
 ............rrr
 ..........AAAAA
-ReadJumpCondC:  11110 RRR c...c         Read1 and JumpCondC
+ReadJumpCondC:  11110 RRR c...c      Read1 and JumpCondC
 C.............C
 ..........AAAAA
-ReadJumpCondR:  11111 RRR c...c         Read1 and JumpCondR
+ReadJumpCondR:  11111 RRR c...c      Read1 and JumpCondR
 ............rrr
 ..........AAAAA
 @end example
 @node The Lisp Reader and Compiler, Lstreams, MULE Character Sets and Encodings, Top
 already-running XEmacs process to display another frame on your local
 TTY.
 Thus, there is a hierarchy console -> display -> frame -> window.
 There is a separate Lisp object type for each of these four concepts.
 Furthermore, there is logically a @dfn{selected console},
 @dfn{selected display}, @dfn{selected frame}, and @dfn{selected window}.
 Each of these objects is distinguished in various ways, such as being the
 default object for various functions that act on objects of that type.
 Note that every containing object rememembers the ``selected'' object
 among the objects that it contains: e.g. not only is there a selected
 Extents are compared using memory indices.  There are two orderings
 for extents and both orders are kept current at all times.  The normal
 or @dfn{display} order is as follows:
 @example
-Extent A is ``less than'' extent B, that is, earlier in the display order,
+Extent A is ``less than'' extent B,
-if:    A-start < B-start,
+that is, earlier in the display order,
-or if: A-start = B-start, and A-end > B-end
+if:    A-start < B-start,
+or if: A-start = B-start, and A-end > B-end
 @end example
 So if two extents begin at the same position, the larger of them is the
 earlier one in the display order (@code{EXTENT_LESS} is true).
 For the e-order, the same thing holds:
 @example
-Extent A is ``less than'' extent B in e-order, that is, later in the buffer,
+Extent A is ``less than'' extent B in e-order,
-if:    A-end < B-end,
+that is, later in the buffer,
-or if: A-end = B-end, and A-start > B-start
+if:    A-end < B-end,
+or if: A-end = B-end, and A-start > B-start
 @end example
 So if two extents end at the same position, the smaller of them is the
 earlier one in the e-order (@code{EXTENT_E_LESS} is true).

Mercurial > hg > xemacs-beta

comparison man/internals/internals.texi @ 380:8626e4521993 r21-2-5