Mercurial > hg > xemacs-beta
changeset 2393:2d4dd2ef74e7
[xemacs-hg @ 2004-11-16 07:37:29 by ben]
internals update
internals/internals.texi: Add sections on Basic Types and Low-Level Allocation. Move module
docs here. Incorporate dynamic array and blocktype docs from
source.
Add info on beta releases up to present.
Redo chapter on "Rules When Writing New C Code", grouping stuff
together properly. Put "Major Textual Changes" under this
chapter. Incorporate etc/CODING-STANDARDS.
Add discussion sections on "Instantiators and Generic Property
Accessors" and "Switching to C++". Fill out discussion on garbage
collection.
Incorporate backtraces showing crashes due to problems with
redisplay-critical-section protection.
| author | ben |
|---|---|
| date | Tue, 16 Nov 2004 07:37:31 +0000 |
| parents | 9c4b194e4122 |
| children | 1100ab30735c |
| files | man/ChangeLog man/internals/internals.texi |
| diffstat | 2 files changed, 3178 insertions(+), 1035 deletions(-) [+] |
line wrap: on
line diff
--- a/man/ChangeLog Mon Nov 15 22:51:09 2004 +0000 +++ b/man/ChangeLog Tue Nov 16 07:37:31 2004 +0000 @@ -1,3 +1,74 @@ +2004-11-16 Ben Wing <ben@xemacs.org> + + * internals/internals.texi (Top): + * internals/internals.texi (Introduction): + * internals/internals.texi (Authorship of XEmacs): + * internals/internals.texi (A History of Emacs): + * internals/internals.texi (Through Version 18): + * internals/internals.texi (Epoch): + * internals/internals.texi (Lucid Emacs): + * internals/internals.texi (GNU Emacs 19): + * internals/internals.texi (GNU Emacs 20): + * internals/internals.texi (XEmacs): + * internals/internals.texi (The XEmacs Split): + * internals/internals.texi (Build-Time Dependencies): + * internals/internals.texi (Low-Level Modules): + * internals/internals.texi (Modules for Interfacing with the Operating System): + * internals/internals.texi (Rules When Writing New C Code): + * internals/internals.texi (Introduction to Writing C Code): + * internals/internals.texi (Writing New Modules): + * internals/internals.texi (Working with Lisp Objects): + * internals/internals.texi (Writing Lisp Primitives): + * internals/internals.texi (Adding Global Lisp Variables): + * internals/internals.texi (Writing Macros): + * internals/internals.texi (Proper Use of Unsigned Types): + * internals/internals.texi (Major Textual Changes): + * internals/internals.texi (Great Integral Type Renaming): + * internals/internals.texi (Text/Char Type Renaming): + * internals/internals.texi (Debugging and Testing): + * internals/internals.texi (Modules for Regression Testing): + * internals/internals.texi (Merging a Branch into the Trunk): + * internals/internals.texi (XEmacs from the Inside): + * internals/internals.texi (Basic Types): + * internals/internals.texi (Low-Level Allocation): + * internals/internals.texi (Basic Heap Allocation): + * internals/internals.texi (Stack Allocation): + * internals/internals.texi (Dynamic Arrays): + * internals/internals.texi (Allocation by Blocks): + * internals/internals.texi (Modules for Allocation): + * internals/internals.texi (Critical Redisplay Sections): + * internals/internals.texi (Control-G (Quit) Checking): + * internals/internals.texi (Future Work Discussion): + * internals/internals.texi (Discussion -- Garbage Collection): + * internals/internals.texi (Discussion -- Pure Space): + * internals/internals.texi (Discussion -- Hashtable-Based Marking and Cleanup): + * internals/internals.texi (Discussion -- The Anti-Cons): + * internals/internals.texi (Discussion -- Glyphs): + * internals/internals.texi (Discussion -- Dialog Boxes): + * internals/internals.texi (Discussion -- Multilingual Issues): + * internals/internals.texi (Discussion -- Instantiators and Generic Property Accessors): + * internals/internals.texi (image-instance-property): New. + * internals/internals.texi (set-image-instance-property): New. + * internals/internals.texi (Discussion -- Switching to C++): + * internals/internals.texi (Discussion -- Windows External Widget): + + Add sections on Basic Types and Low-Level Allocation. Move module + docs here. Incorporate dynamic array and blocktype docs from + source. + + Add info on beta releases up to present. + + Redo chapter on "Rules When Writing New C Code", grouping stuff + together properly. Put "Major Textual Changes" under this + chapter. Incorporate etc/CODING-STANDARDS. + + Add discussion sections on "Instantiators and Generic Property + Accessors" and "Switching to C++". Fill out discussion on garbage + collection. + + Incorporate backtraces showing crashes due to problems with + redisplay-critical-section protection. + 2004-11-04 Ben Wing <ben@xemacs.org> * lispref/mule.texi (CCL Syntax):
--- a/man/internals/internals.texi Mon Nov 15 22:51:09 2004 +0000 +++ b/man/internals/internals.texi Tue Nov 16 07:37:31 2004 +0000 @@ -311,11 +311,12 @@ * XEmacs from the Perspective of Building:: * Build-Time Dependencies:: * The Modules of XEmacs:: -* Major Textual Changes:: * Rules When Writing New C Code:: * Regression Testing XEmacs:: * CVS Techniques:: * XEmacs from the Inside:: +* Basic Types:: +* Low-Level Allocation:: * The XEmacs Object System (Abstractly Speaking):: * How Lisp Objects Are Represented in C:: * Allocation of Objects in XEmacs Lisp:: @@ -350,6 +351,7 @@ A History of Emacs * Through Version 18:: Unification prevails. +* Epoch:: An early graphical split of GNU Emacs. * Lucid Emacs:: One version 19 Emacs. * GNU Emacs 19:: The other version 19 Emacs. * GNU Emacs 20:: The other version 20 Emacs. @@ -365,22 +367,23 @@ * Modules for Other Aspects of the Lisp Interpreter and Object System:: * Modules for Interfacing with the Operating System:: -Major Textual Changes - -* Great Integral Type Renaming:: -* Text/Char Type Renaming:: - Rules When Writing New C Code -* A Reader's Guide to XEmacs Coding Conventions:: -* General Coding Rules:: -* Object-Oriented Techniques for C:: +* Introduction to Writing C Code:: +* Writing New Modules:: +* Working with Lisp Objects:: * Writing Lisp Primitives:: * Writing Good Comments:: * Adding Global Lisp Variables:: * Writing Macros:: * Proper Use of Unsigned Types:: -* Techniques for XEmacs Developers:: +* Major Textual Changes:: +* Debugging and Testing:: + +Major Textual Changes + +* Great Integral Type Renaming:: +* Text/Char Type Renaming:: Regression Testing XEmacs @@ -391,6 +394,14 @@ * Merging a Branch into the Trunk:: +Low-Level Allocation + +* Basic Heap Allocation:: +* Stack Allocation:: +* Dynamic Arrays:: +* Allocation by Blocks:: +* Modules for Allocation:: + Allocation of Objects in XEmacs Lisp * Introduction to Allocation:: @@ -664,14 +675,22 @@ Future Work Discussion -* Discussion -- garbage collection:: -* Discussion -- glyphs:: +* Discussion -- Garbage Collection:: +* Discussion -- Glyphs:: * Discussion -- Dialog Boxes:: * Discussion -- Multilingual Issues:: +* Discussion -- Instantiators and Generic Property Accessors:: +* Discussion -- Switching to C++:: * Discussion -- Windows External Widget:: * Discussion -- Packages:: * Discussion -- Distribution Layout:: +Discussion -- Garbage Collection + +* Discussion -- Pure Space:: +* Discussion -- Hashtable-Based Marking and Cleanup:: +* Discussion -- The Anti-Cons:: + Old Future Work * Old Future Work -- A Portable Unexec Replacement:: @@ -720,14 +739,17 @@ First is the introduction, including this chapter and chapters on the history and authorship of XEmacs. @item -Next, starting with @ref{XEmacs from the Outside}, are some chapters -giving a broad overview of the internal workings of XEmacs and -documenting important information relevant to those working on the code. +Next, starting with @ref{XEmacs from the Outside}, are a couple of chapters +giving a broad overview of the internal workings of XEmacs. +@item +Afterwards, starting with @ref{XEmacs from the Perspective of +Building}, are some chapters documenting important information relevant to +those working on the code. @item The remaining divisions document the nitty-gritty details of the -internal workings. First, starting with @ref{XEmacs from the Outside}, -is a division on the workings of the Lisp interpreter that drives -XEmacs. +internal workings. First, starting with @ref{XEmacs from the Inside}, +is a division on the low-level types and allocation routines and the +workings of the Lisp interpreter that drives XEmacs. @item Next, starting with @ref{Buffers}, is a division on the parts of the code specifically devoted to text processing, including multilingual @@ -852,8 +874,8 @@ between Xemacs and GNU Emacs. @item Kyle Jones -Author of the minimal tag bits support in—minimal lisp support for lisp -objects which allows for 32-bit pointers and 31-bit integers. +Author of the minimal tag bits support, which allows for 32-bit +pointers and 31-bit integers. @item Olivier Galibert Author of the portable dumping mechanism. @@ -959,13 +981,14 @@ @menu * Through Version 18:: Unification prevails. +* Epoch:: An early graphical split of GNU Emacs. * Lucid Emacs:: One version 19 Emacs. * GNU Emacs 19:: The other version 19 Emacs. * GNU Emacs 20:: The other version 20 Emacs. * XEmacs:: The continuation of Lucid Emacs. @end menu -@node Through Version 18, Lucid Emacs, A History of Emacs, A History of Emacs +@node Through Version 18, Epoch, A History of Emacs, A History of Emacs @section Through Version 18 @cindex version 18, through @cindex Gosling, James @@ -1008,102 +1031,125 @@ July 15, 1985. All Gosling code was removed due to potential copyright problems with the code. @item -version 16.57: released on September 16, 1985. -@item -versions 16.58, 16.59: released on September 17, 1985. -@item -version 16.60: released on September 19, 1985. These later version 16's +Version 16.57: released on September 16, 1985. +@item +Versions 16.58, 16.59: released on September 17, 1985. +@item +Version 16.60: released on September 19, 1985. These later version 16's incorporated patches from the net, esp. for getting Emacs to work under System V. @item -version 17.36 (first official v17 release) released on December 20, +Version 17.36 (first official v17 release) released on December 20, 1985. Included a TeX-able user manual. First official unpatched version that worked on vanilla System V machines. @item -version 17.43 (second official v17 release) released on January 25, +Version 17.43 (second official v17 release) released on January 25, 1986. @item -version 17.45 released on January 30, 1986. -@item -version 17.46 released on February 4, 1986. -@item -version 17.48 released on February 10, 1986. -@item -version 17.49 released on February 12, 1986. -@item -version 17.55 released on March 18, 1986. -@item -version 17.57 released on March 27, 1986. -@item -version 17.58 released on April 4, 1986. -@item -version 17.61 released on April 12, 1986. -@item -version 17.63 released on May 7, 1986. -@item -version 17.64 released on May 12, 1986. -@item -version 18.24 (a beta version) released on October 2, 1986. -@item -version 18.30 (a beta version) released on November 15, 1986. -@item -version 18.31 (a beta version) released on November 23, 1986. -@item -version 18.32 (a beta version) released on December 7, 1986. -@item -version 18.33 (a beta version) released on December 12, 1986. -@item -version 18.35 (a beta version) released on January 5, 1987. -@item -version 18.36 (a beta version) released on January 21, 1987. +Version 17.45 released on January 30, 1986. +@item +Version 17.46 released on February 4, 1986. +@item +Version 17.48 released on February 10, 1986. +@item +Version 17.49 released on February 12, 1986. +@item +Version 17.55 released on March 18, 1986. +@item +Version 17.57 released on March 27, 1986. +@item +Version 17.58 released on April 4, 1986. +@item +Version 17.61 released on April 12, 1986. +@item +Version 17.63 released on May 7, 1986. +@item +Version 17.64 released on May 12, 1986. +@item +Version 18.24 (a beta version) released on October 2, 1986. +@item +Version 18.30 (a beta version) released on November 15, 1986. +@item +Version 18.31 (a beta version) released on November 23, 1986. +@item +Version 18.32 (a beta version) released on December 7, 1986. +@item +Version 18.33 (a beta version) released on December 12, 1986. +@item +Version 18.35 (a beta version) released on January 5, 1987. +@item +Version 18.36 (a beta version) released on January 21, 1987. @item January 27, 1987: The Great Usenet Renaming. net.emacs is now comp.emacs. @item -version 18.37 (a beta version) released on February 12, 1987. -@item -version 18.38 (a beta version) released on March 3, 1987. -@item -version 18.39 (a beta version) released on March 14, 1987. -@item -version 18.40 (a beta version) released on March 18, 1987. -@item -version 18.41 (the first ``official'' release) released on March 22, +Version 18.37 (a beta version) released on February 12, 1987. +@item +Version 18.38 (a beta version) released on March 3, 1987. +@item +Version 18.39 (a beta version) released on March 14, 1987. +@item +Version 18.40 (a beta version) released on March 18, 1987. +@item +Version 18.41 (the first ``official'' release) released on March 22, 1987. @item -version 18.45 released on June 2, 1987. -@item -version 18.46 released on June 9, 1987. -@item -version 18.47 released on June 18, 1987. -@item -version 18.48 released on September 3, 1987. -@item -version 18.49 released on September 18, 1987. -@item -version 18.50 released on February 13, 1988. -@item -version 18.51 released on May 7, 1988. -@item -version 18.52 released on September 1, 1988. -@item -version 18.53 released on February 24, 1989. -@item -version 18.54 released on April 26, 1989. -@item -version 18.55 released on August 23, 1989. This is the earliest version -that is still available by FTP. -@item -version 18.56 released on January 17, 1991. -@item -version 18.57 released late January, 1991. -@item -version 18.58 released ?????. -@item -version 18.59 released October 31, 1992. -@end itemize - -@node Lucid Emacs, GNU Emacs 19, Through Version 18, A History of Emacs +Version 18.45 released on June 2, 1987. +@item +Version 18.46 released on June 9, 1987. +@item +Version 18.47 released on June 18, 1987. +@item +Version 18.48 released on September 3, 1987. +@item +Version 18.49 released on September 18, 1987. +@item +Version 18.50 released on February 13, 1988. +@item +Version 18.51 released on May 7, 1988. +@item +Version 18.52 released on September 1, 1988. +@item +Version 18.53 released on February 24, 1989. +@item +Version 18.54 released on April 26, 1989. +@item +Version 18.55 released on August 23, 1989. This is the earliest version +that is still available by FTP. (Verified in November 2004.) +@item +Version 18.56 released on January 17, 1991. +@item +Version 18.57 released late January, 1991. +@item +Version 18.58 released sometime in 1991. +@item +Version 18.59 released October 31, 1992. +@end itemize + + +@node Epoch, Lucid Emacs, Through Version 18, A History of Emacs +@section Epoch +@cindex Epoch +@cindex UIUC + +#### Document Epoch + + A time line for Epoch is + +@itemize @bullet +@item +Epoch 1.0 released December 14, 1989. (by Simon Kaplan, Chris Love, et al.) +@item +Epoch 2.0 released December 23, 1989. +@item +Epoch 3.1 released February 6, 1990. +@item +Epoch 3.2 released December[????] 11, 1990. +@item +Epoch 4.0 released August 27, 1990. +@end itemize + +@node Lucid Emacs, GNU Emacs 19, Epoch, A History of Emacs @section Lucid Emacs @cindex Lucid Emacs @cindex Lucid Inc. @@ -1144,29 +1190,37 @@ @itemize @bullet @item -version 19.0 shipped with Energize 1.0, April 1992. -@item -version 19.1 released June 4, 1992. -@item -version 19.2 released June 19, 1992. -@item -version 19.3 released September 9, 1992. -@item -version 19.4 released January 21, 1993. -@item -version 19.5 was a repackaging of 19.4 with a few bug fixes and -shipped with Energize 2.0. Never released to the net. -@item -version 19.6 released April 9, 1993. -@item -version 19.7 was a repackaging of 19.6 with a few bug fixes and +Version 19.0 shipped with Energize 1.0, April 1992. +@item +Version 19.1 released June 4, 1992. +@item +Version 19.2 released June 19, 1992. +@item +Version 19.3 released September 9, 1992. +@item +Version 19.4 released January 21, 1993. +@item +Version 19.5 released February 5, 1993. This was a repackaging of 19.4 with a +few bug fixes and shipped with Energize 2.0. It was a trade-show giveaway +and never released to the net. +@item +Version 19.6 released April 9, 1993. +@item +Version 19.7 was a repackaging of 19.6 with a few bug fixes and shipped with Energize 2.1. Never released to the net. @item -version 19.8 released September 6, 1993. -@item -version 19.9 released January 12, 1994. -@item -version 19.10 released May 27, 1994. +Version 19.8 released September 6, 1993. (Epoch 4.0 merger of +redisplay code, preliminary I18N support, code merged from GNU Emacs +19.8 beta) +@item +Version 19.9 released January 12, 1994. (Scrollbars, Athena.) +@item +Version 19.10 released May 27, 1994. (Uses `configure'; code merged +from GNU Emacs 19.23 beta and further merging with Epoch 4.0) Known as +"Lucid Emacs" when shipped by Lucid, and as "XEmacs" when shipped by +Sun; but Lucid went out of business a few days later and it's unclear +very many copies of 19.10 were released by Lucid. (Last release by +Jamie Zawinski.) @end itemize @node GNU Emacs 19, GNU Emacs 20, Lucid Emacs, A History of Emacs @@ -1184,69 +1238,75 @@ @itemize @bullet @item -version 19.8 (beta) released May 27, 1993. -@item -version 19.9 (beta) released May 27, 1993. -@item -version 19.10 (beta) released May 30, 1993. -@item -version 19.11 (beta) released June 1, 1993. -@item -version 19.12 (beta) released June 2, 1993. -@item -version 19.13 (beta) released June 8, 1993. -@item -version 19.14 (beta) released June 17, 1993. -@item -version 19.15 (beta) released June 19, 1993. -@item -version 19.16 (beta) released July 6, 1993. -@item -version 19.17 (beta) released late July, 1993. -@item -version 19.18 (beta) released August 9, 1993. -@item -version 19.19 (beta) released August 15, 1993. -@item -version 19.20 (beta) released November 17, 1993. -@item -version 19.21 (beta) released November 17, 1993. -@item -version 19.22 (beta) released November 28, 1993. -@item -version 19.23 (beta) released May 17, 1994. -@item -version 19.24 (beta) released May 16, 1994. -@item -version 19.25 (beta) released June 3, 1994. -@item -version 19.26 (beta) released September 11, 1994. -@item -version 19.27 (beta) released September 14, 1994. -@item -version 19.28 (first ``official'' release) released November 1, 1994. -@item -version 19.29 released June 21, 1995. -@item -version 19.30 released November 24, 1995. -@item -version 19.31 released May 25, 1996. -@item -version 19.32 released July 31, 1996. -@item -version 19.33 released August 11, 1996. -@item -version 19.34 released August 21, 1996. -@item -version 19.34b released September 6, 1996. +Version 19.7 beta released May 22, 1993. First public beta v19 release. +@item +Version 19.8 beta released May 27, 1993. +@item +Version 19.9 beta released May 27, 1993. +@item +Version 19.10 beta released May 30, 1993. +@item +Version 19.11 beta released June 1, 1993. +@item +Version 19.12 beta released June 2, 1993. +@item +Version 19.13 beta released June 8, 1993. +@item +Version 19.14 beta released June 17, 1993. +@item +Version 19.15 beta released June 19, 1993. +@item +Version 19.16 beta released July 6, 1993. +@item +Version 19.17 beta released late July, 1993. +@item +Version 19.18 beta released August 9, 1993. +@item +Version 19.19 beta released August 15, 1993. +@item +Version 19.20 beta released November 17, 1993. +@item +Version 19.21 beta released November 17, 1993. +@item +Version 19.22 beta released November 28, 1993. +@item +Version 19.23 beta released May 17, 1994. +@item +Version 19.24 beta released May 16, 1994. +@item +Version 19.25 beta released June 3, 1994. +@item +Version 19.26 beta released September 11, 1994. +@item +Version 19.27 beta released September 14, 1994. +@item +Version 19.28 (first ``official'' release) released November 1, 1994. +@item +Version 19.29 released June 21, 1995. +@item +Version 19.30 released November 24, 1995. +@item +Version 19.31 released May 25, 1996. +@item +Version 19.32 released July 31, 1996. +@item +Version 19.33 released August 11, 1996. +@item +Version 19.34 released August 21, 1996. +@item +Version 19.34b released September 6, 1996. @end itemize @cindex Mlynarik, Richard +@cindex Baur, Steve In some ways, GNU Emacs 19 was better than Lucid Emacs; in some ways, worse. Lucid soon began incorporating features from GNU Emacs 19 into -Lucid Emacs; the work was mostly done by Richard Mlynarik, who had been -working on and using GNU Emacs for a long time (back as far as version -16 or 17). +Lucid Emacs; for the first year, the work was mostly done by Richard +Mlynarik, who had been working on and using GNU Emacs for a long time +(back as far as version 16 or 17). After that, Lucid folded and Sun +continued with XEmacs; further merging work has continued up through +the present, done mostly by Ben Wing but a good deal of synching was +done by Steve Baur in 1996 with GNU Emacs 19.34. @node GNU Emacs 20, XEmacs, GNU Emacs 19, A History of Emacs @section GNU Emacs 20 @@ -1258,15 +1318,34 @@ On February 2, 1997 work began on GNU Emacs to integrate Mule. The first release was made in September of that year. -A timeline for Emacs 20 is - -@itemize @bullet -@item -version 20.1 released September 17, 1997. -@item -version 20.2 released September 20, 1997. -@item -version 20.3 released August 19, 1998. +A timeline for GNU Emacs 20 is + +@itemize @bullet +@item +Version 20.1 released September 17, 1997. +@item +Version 20.2 released September 20, 1997. +@item +Version 20.3 released August 19, 1998. +@item +version 20.4 released July 12, 1999; on comp.emacs, July 27. +@item +version 20.5 released ???. +@item +version 20.6 released ???. +@item +version 20.7 released ???. +@end itemize + +A timeline for GNU Emacs 21 is + +@itemize @bullet +@item +version 21.1 released October 20, 2001. +@item +Version 21.2 released March 16, 2002. +@item +Version 21.3 released March 19, 2003. @end itemize @node XEmacs, , GNU Emacs 20, A History of Emacs @@ -1302,19 +1381,27 @@ involvement, punctuated by a six-month contract with Amdahl Corporation. @cindex rename to XEmacs +@cindex Thompson, Chuck +@cindex Wing, Ben In 1994, Sun and Lucid agreed to rename Lucid Emacs to XEmacs (a name not favorable to either company); the first release called XEmacs was version 19.11. In June 1994, Lucid folded and Jamie quit to work for the newly formed Mosaic Communications Corp., later Netscape Communications Corp. (co-founded by the same Marc Andreessen, who had quit his Epoch job to work on a graphical browser for the World Wide -Web). Chuck then become the primary maintainer of XEmacs, and put out -versions 19.11 through 19.14 in conjunction with Ben. For 19.12 and -19.13, Chuck added the new redisplay and many other display improvements -and Ben added MULE support (support for Asian and other languages) and -redesigned most of the internal Lisp subsystems to better support the -MULE work and the various other features being added to XEmacs. After -19.14 Chuck retired as primary maintainer and Steve Baur stepped in. +Web). Chuck and Ben then become the primary authors and maintainers +of XEmacs, with Chuck putting out versions 19.11 through 19.14 in +conjunction with Ben. For 19.12 through 19.14, Chuck added the new +redisplay and various other display improvements and Ben added MULE +support (support for Asian and other languages), multi-device support, +glyphs, specifiers, and GIF/JPG/PNG support, and redesigned most of +the internal Lisp subsystems to better support the MULE work, display +work and the various other features being added to XEmacs. After +19.14 Chuck retired from XEmacs and Steve Baur stepped in as release +engineer. Ben Wing continued on as the primary author and architect +of XEmacs and has remained, sometimes on-and-off, with XEmacs until +the present day (late 2004), being responsible for perhaps 75% of all +the non-FSF code in the core (i.e. not the packages) of XEmacs. @cindex MULE merged XEmacs appears Soon after 19.13 was released, work began in earnest on the MULE @@ -1329,15 +1416,91 @@ @cindex Baur, Steve @cindex Buchholz, Martin -@cindex Jones, Kyle -@cindex Niksic, Hrvoje @cindex XEmacs goes it alone In 1997, Sun finally dropped all pretense of support for XEmacs and Martin Buchholz left the company in November. Since then, and mostly for the previous year, because Steve Baur was never paid to work on XEmacs, XEmacs has existed solely on the contributions of volunteers -from the Free Software Community. Starting from 1997, Hrvoje Niksic and -Kyle Jones have figured prominently in XEmacs development. +from the Free Software Community. + +@cindex Jones, Kyle +@cindex Niksic, Hrvoje +@cindex Galibert, Olivier +@cindex Piper, Andy +@cindex Harris, Jonathan +@cindex Katsnelson, Kirill +@cindex Turnbull, Stephen +@cindex Shelton, Vin +@cindex Wing, Ben + Between 1997 and 2000, MS-Windows support was added and stabilized by +Jonathan Harris, Andy Piper, Ben Wing and Kirill Katsnelson. Hrvoje +Niksic and Kyle Jones figured prominently in XEmacs development during +these same years. Steve Baur added the package system in 1997 (?), +and Olivier Galibert also added the portable dumper support around +2000. Martin Buchholz took over from Steve Baur as release manager in +late 1998 (?), and continued in this position through to eary 2000 +(?), when Stephen Turnbull took it over. XEmacs has also been split +into stable and experimental branches since early 1999, and Vin +Shelton has been the release manager of the stable branches since the +beginning. Ben Wing suffered severe pain problems throughout much of +this time, making him unable to use his hands, but he contributed when +he could, especially in the form of dictated design documents. + +@cindex Sperber, Michael +@cindex Turnbull, Stephen +@cindex James, Jerry +@cindex Youngs, Steve +@cindex Aichner, Adrian +@cindex Wing, Ben +@cindex Crestani, Marcus +@cindex Perry, Bill +@cindex Purvis, Malcolm +@cindex Shelton, Vin + Starting around 2000, Kyle, Hrvoje, Martin and Kirill became less active. +Jonathan Harris had dropped out of the project around 1998, and Andy +Piper became mostly inactive by the year 2001 or 2002. New faces +appeared, however, and others continued strong: + +@itemize @bullet +@item +Michael Sperber, who had been in the background as a beta tester for a +fair amount of time, began to assume a more active role. He revamped +the path-searching code at initialization time, did some major work on +the CVS repositories, and is in the process of a major project to +replace the garbage collector, which he is overseeing with some of his +students (e.g. Marcus Crestani). +@item +Steve Youngs stepped in as package maintainer in late 1998 (?). +@item +Stephen Turnbull has contained to produce the experimental beta +releases, write code when he can, produce many design documents, and +generally oversee the managerial aspects of the project. +@item +Jerry James appeared on the scene in early 2002 and has contributed a +large amount of code, including the module subsystem, bignums, and +lots of other code cleanup. +@item +Bill Perry, who had been active on and off in XEmacs since the early +1990's (e.g. he did a fair amount of work on the JPG and PNG interface +and added the TIFF interface, in addition to writing the Emacs/W3 +browser), added GTK support for XEmacs, a major project for which he +received a multi-month contract through BeOpen (?). He has since +disappeared but Malcolm Purvis has taken up the GTK project again and +is keeping it going when he has time. +@item +Adrian Aichner is continuing to create and update the web site on +@uref{www.xemacs.org,XEmacs Web Site}, and is a particularly active +beta tester. +@item +Ben Wing has recovered somewhat from the bad years of 1997 - 1999 and +has resumed his position as Architect of XEmacs and chief code +contributor to the project. He added Mule on Windows support, Unicode +support, the Internals manual (originally written by him during his +last days at Sun) and many other projects, and is now working on a new +behaviors system and cleanups of various other subsystems. +@item +Vin Shelton continues to put out stable releases of XEmacs. +@end itemize @cindex merging attempts Many attempts have been made to merge XEmacs and GNU Emacs, but they @@ -1345,27 +1508,64 @@ A more detailed history is contained in the XEmacs About page. + For more detailed information about the features added to each version, +see the files @file{NEWS}, @file{ONEWS}, and @file{OONEWS} in the +@file{etc/} directory. + A time line for XEmacs is @itemize @bullet @item version 19.11 (first XEmacs) released September 13, 1994. @item -version 19.12 released June 23, 1995. -@item -version 19.13 released September 1, 1995. -@item -version 19.14 released June 23, 1996. +Initial work on Mule support begins September 1994 by both Ben Wing and +Stig. Both projects got bogged down in other issues. +@item +version 19.12 released June 23, 1995. (The Release Times 10. Included +rewritten redisplay, TTY support, multi-device support, device and +console objects, specifiers, glyphs, toolbars, horizontal scrollbars, +Lucid scrollbar widget, 3-d modeline, stay-up Lucid menus, resizable +minibuffer, echo area is a true buffer, MD5 hashing support, expanded +menubar, redone menu specification format (including menu filters), +rewritten extents, renamed "screen" to "frame", misc-user events, +rewritten face code, rewritten mouse code, warnings system, CL +backquote syntax, critical C-g, code merging with GNU Emacs 19.28. +New packages Hyperbole, OOBR, hm--html-menus, viper, lazy-lock, +ksh-mode, rsz-minibuf.) +@item +Mule work done in earnest from May through November, 1995 by Ben Wing. +Early on, much of the work involved Mule-izing and was incorporated +into 19.12 and 19.13. After the release of 19.13, further work was +forked onto a new development branch, which eventually became 20.0. +@item +version 19.13 released September 1, 1995. (Bug-fix release. Message +logging, background pixmaps, sticky modifiers, Linux audio support, +new Elisp manual, keyboard-translate-table. New packages ada-mode, +arc-mode, auto-show-mode, completion, dabbrev, easymenu, live-icon, +mailcrypt 3.2, two-column.) +@item +xemacs.org created, date ??? -- early 1996?. +@item +version 19.14 released June 23, 1996. (TTY colors, mousable/color +modeline, GIF/JPEG/PNG support, file dialog box, blinking cursor, +gnuattach, auto scrolling horizontally to keep point in view, major +code merging with GNU Emacs 19.30, key bindings from GNU Emacs 19.30, +surrogate minibuffers, function-key-map, key-translation-map. New +packages PSGML, Java/VRML modes, GNUS 5.2.) @item version 20.0 released February 9, 1997. @item -version 19.15 released March 28, 1997. +version 19.15 released March 28, 1997. (Custom, widget, new logo and +background color, introduction of `compatible' variables, major code +merging with GNU Emacs 19.30. New packages EFS, TM, AUC Tex, redo, +igrep, uniquify, many others.) @item version 20.1 (not released to the net) April 15, 1997. @item version 20.2 released May 16, 1997. @item -version 19.16 released October 31, 1997. +version 19.16 released October 31, 1997. (Bug-fix release. Faster +font-locking. Not much else.) @item version 20.3 (the first stable version of XEmacs 20.x) released November 30, 1997. @@ -1483,17 +1683,89 @@ @item version 21.2.40 released January 8, 2001. @item -version 21.2.41 released January 17, 2001. -@item -version 21.2.42 released January 20, 2001. -@item -version 21.2.43 released January 26, 2001. -@item -version 21.2.44 released February 8, 2001. -@item -version 21.2.45 released February 23, 2001. -@item -version 21.2.46 released March 21, 2001. +version 21.2.41 "Polyhymnia" released January 17, 2001. +@item +version 21.2.42 "Poseidon" released January 20, 2001. +@item +version 21.2.43 "Terspichore" released January 26, 2001. +@item +version 21.2.44 "Thalia" released February 8, 2001. +@item +version 21.2.45 "Thelxepeia" released February 23, 2001. +@item +version 21.2.46 "Urania" released March 21, 2001. +@item +version 21.2.47 "Zephir" released April 14, 2001. +@item +XEmacs 21.4.0 "Solid Vapor" released April 16, 2001. +@item +XEmacs 21.4.1 "Copyleft" released April 19, 2001. +@item +XEmacs 21.4.2 "Developer-Friendly Unix APIs" released May 10, 2001. +@item +XEmacs 21.4.3 "Academic Rigor" released May 17, 2001. +@item +XEmacs 21.4.4 "Artificial Intelligence" released July 28, 2001. +@item +XEmacs 21.4.5 "Civil Service" released October 23, 2001. +@item +XEmacs 21.4.6 "Common Lisp" released December 17, 2001. +@item +XEmacs 21.4.7 "Economic Science" released May 4, 2002. +@item +XEmacs 21.4.8 "Honest Recruiter" released May 9, 2002. +@item +XEmacs 21.4.9 "Informed Management" released August 23, 2002. +@item +XEmacs 21.4.10 "Military Intelligence" released November 2, 2002. +@item +XEmacs 21.4.11 "Native Windows TTY Support" released January 3, 2003. +@item +XEmacs 21.4.12 "Portable Code" released January 15, 2003. +@item +XEmacs 21.4.13 "Rational FORTRAN" released May 25, 2003. +@item +XEmacs 21.4.14 "Reasonable Discussion" released September 3, 2003. +@item +XEmacs 21.4.15 "Security Through Obscurity" released February 2, 2004. +@item +version 21.5.0 "alfalfa" released April 18, 2001. +@item +version 21.5.1 "anise" released May 9, 2001. +@item +version 21.5.2 "artichoke" released July 28, 2001. +@item +version 21.5.3 "asparagus" released September 7, 2001. +@item +version 21.5.4 "bamboo" released January 8, 2002. +@item +version 21.5.5 "beets" released March 5, 2002. +@item +version 21.5.6 "bok choi" released April 5, 2002. +@item +version 21.5.7 "broccoflower" released July 2, 2002. +@item +version 21.5.8 "broccoli" released July 27, 2002. +@item +version 21.5.9 "brussels sprouts" released August 30, 2002. +@item +version 21.5.10 "burdock" released January 4, 2003. +@item +version 21.5.11 "cabbage" released February 16, 2003. +@item +version 21.5.12 "carrot" released April 24, 2003. +@item +version 21.5.13 "cauliflower" released May 10, 2003. +@item +version 21.5.14 "cassava" released June 1, 2003. +@item +version 21.5.15 "celery" released September 3, 2003. +@item +version 21.5.16 "celeriac" released September 26, 2003. +@item +version 21.5.17 "chayote" released March 22, 2004. +@item +version 21.5.18 "chestnut" released October 22, 2004. @end itemize @node The XEmacs Split, XEmacs from the Outside, A History of Emacs, Top @@ -1502,6 +1774,8 @@ Author: @uref{mailto:ben@@xemacs.org,Ben Wing} +@subheading Ben Wing's attempts + @strong{NOTE NOTE NOTE}: The following is a @strong{highly} opinionated piece written by one of the main authors of XEmacs. This reflects his opinions, and his only! It is included here because it may help to @@ -1615,6 +1889,104 @@ his principles. The fact that few if any people share his principles is meaningless to him. +@subheading Jamie Zawinski's attempts + +In 1991, I was working at Lucid Inc., and our newest product, +Energize, was an integrated development environment for C and C++ on +Unix. The design of this development environment involved very tight +integration between the various tools: compilers, linkers, debuggers, +graphers, and editors. So of course we needed a powerful editor to tie +the whole thing together, and it was obvious to all of us that there +was only one editor that would do: Emacs. + +At the time, the current version of GNU Emacs from the FSF was Emacs +18. There was another version of GNU Emacs called Epoch, that had been +developed at NCSA, which was a set of patches to Emacs 18 that gave it +much better GUI support (Emacs 18 was very much a tty program, with +GUI support crudely grafted on as an afterthought.) + +For the last few years, Emacs 19 had been due to be released ``real +soon now,'' and was expected to integrate the various features of +Epoch in a cleaner way. The Epoch maintainers themselves saw Epoch as +an interim measure, awaiting the release of Emacs 19. + +So, at Lucid we didn't want to tie ourselves to Emacs 18 or on Epoch, +because those code bases were considered obsolete by their +maintainers. We wanted to use Emacs 19 with our product: the idea was +that our product would operate with the off-the-shelf version of Emacs +19, which most people would already have pre-installed on their system +anyway. That way, Energize would make use, to some extent, of tools +you already had and were already using. + +The only problem was, Emacs 19 wasn't done yet. So, we decided we +could help solve that problem, by providing money and resources to get +Emacs 19 finished. + +Even though Energize was a proprietary, commercial product, all of our +work on Emacs (and on GCC and GDB) was released under the GPL. We even +assigned the copyright on all of our work back to the FSF, because we +had no proprietary interest in Emacs per se: it was just a tool that +we wanted to use, and we wanted it to work well, and that was best +achieved by making our modifications to it be as freely available as +possible. (This was one of the earliest, if not the earliest, example +of a commercial product being built to a significant extent out of +open source software.) + +Well, our attempts to help the FSF complete their Emacs 19 project +were pretty much a disaster, and we reached the point where we just +couldn't wait any longer: we needed to ship our product to customers, +and our product needed to have an editor in it. So we bundled up our +work on GNU Emacs 19, called it Lucid Emacs, and released it to the +world. + +This incident has become famous as one of the most significant +``forks'' in a free software code base. + +When Lucid went out of business in 1994, and I came to Netscape, I +passed the torch for the maintenance of Lucid Emacs to Chuck Thompson +(at NCSA) and Ben Wing (at Sun), who renamed it from ``Lucid Emacs'' +to ``XEmacs.'' + +To this day, XEmacs is as popular as FSFmacs, because it still +provides features and a design that many people find superior to the +FSF's version. + +I attribute Lucid Emacs's success to two things, primarily: + + +First, that my focus was on user interface, and an attempt to both +make Emacs be a good citizen of modern GUI desktops, and to make it as +easy for new users to pick up Emacs as any other GUI editor; + +Second, that I ran the Lucid Emacs project in a much more open, +inclusive way than RMS ran his project. I was not just willing, but +eager, to delegate significant and critical pieces of the project to +other hackers once they had shown that they knew what they were +doing. RMS was basically never willing to do this with anybody. Other +things that helped Lucid Emacs's success, but were probably less +important than the above: + + +We gave the users what they wanted first. People had been anticipating +Emacs 19 for years, and we stopped dragging our feet and finished +it. So this got us a lot of users up front. However, XEmacs's current +popularity can't be attributed to this, not since 1993, anyway. + +Lucid Emacs was technically superior in many ways. This won us the +mindshare of many good developers, who preferred working with Lucid +Emacs to FSF Emacs. It would be nice if technical superiority was all +that mattered, but realistically, the other factors were probably more +important than this one, as far as number of users is concerned. The +following messages, from the Lucid Emacs mailing lists in 1992 and +1993, comprise the bulk (if not the entirety) of the public +discussions between the Lucid and FSF camps on why the split happened +and why a merger never did. + +The current XEmacs maintainers have a much more pusillanimous summary +of this history on their XEmacs versus GNU Emacs page. + +-- jwz, 11-Feb-2000. + @node XEmacs from the Outside, The Lisp Language, The XEmacs Split, Top @chapter XEmacs from the Outside @cindex XEmacs from the outside @@ -2056,7 +2428,7 @@ @code{custom-declare-variable-list} to prevent the @samp{void-variable} error. (Currently this is only needed for @file{make-docfile.el}.) -@node The Modules of XEmacs, Major Textual Changes, Build-Time Dependencies, Top +@node The Modules of XEmacs, Rules When Writing New C Code, Build-Time Dependencies, Top @chapter The Modules of XEmacs @cindex modules of XEmacs @@ -2659,96 +3031,6 @@ data space when dumping. - -@example -@file{alloca.c} -@file{free-hook.c} -@file{getpagesize.h} -@file{gmalloc.c} -@file{malloc.c} -@file{mem-limits.h} -@file{ralloc.c} -@file{vm-limit.c} -@end example - -These handle basic C allocation of memory. @file{alloca.c} is an emulation of -the stack allocation function @code{alloca()} on machines that lack -this. (XEmacs makes extensive use of @code{alloca()} in its code.) - -@file{gmalloc.c} and @file{malloc.c} are two implementations of the standard C -functions @code{malloc()}, @code{realloc()} and @code{free()}. They are -often used in place of the standard system-provided @code{malloc()} -because they usually provide a much faster implementation, at the -expense of additional memory use. @file{gmalloc.c} is a newer implementation -that is much more memory-efficient for large allocations than @file{malloc.c}, -and should always be preferred if it works. (At one point, @file{gmalloc.c} -didn't work on some systems where @file{malloc.c} worked; but this should be -fixed now.) - -@cindex relocating allocator -@file{ralloc.c} is the @dfn{relocating allocator}. It provides -functions similar to @code{malloc()}, @code{realloc()} and @code{free()} -that allocate memory that can be dynamically relocated in memory. The -advantage of this is that allocated memory can be shuffled around to -place all the free memory at the end of the heap, and the heap can then -be shrunk, releasing the memory back to the operating system. The use -of this can be controlled with the configure option @code{--rel-alloc}; -if enabled, memory allocated for buffers will be relocatable, so that if -a very large file is visited and the buffer is later killed, the memory -can be released to the operating system. (The disadvantage of this -mechanism is that it can be very slow. On systems with the -@code{mmap()} system call, the XEmacs version of @file{ralloc.c} uses -this to move memory around without actually having to block-copy it, -which can speed things up; but it can still cause noticeable performance -degradation.) - -On Linux systems using @samp{glibc 2}, these strategies are built in to -the so-called ``Doug Lea malloc.'' See, for example, Doug Lea's home -page, especially @uref{http://gee.cs.oswego.edu/dl/html/malloc.html,``A -Memory Allocator''}. The source file, @file{malloc.c} (available at the -same place) is copiously (and usefully!) commented. -@uref{http://www.malloc.de/,Wolfram Gloger's home page} may also be -useful. - -@file{free-hook.c} contains some debugging functions for checking for invalid -arguments to @code{free()}. - -@file{vm-limit.c} contains some functions that warn the user when memory is -getting low. These are callback functions that are called by @file{gmalloc.c} -and @file{malloc.c} at appropriate times. - -@file{getpagesize.h} provides a uniform interface for retrieving the size of a -page in virtual memory. @file{mem-limits.h} provides a uniform interface for -retrieving the total amount of available virtual memory. Both are -similar in spirit to the @file{sys*.h} files described in section J, below. - - -@example -@file{blocktype.c} -@file{blocktype.h} -@file{dynarr.c} -@end example - -These implement a couple of basic C data types to facilitate memory -allocation. The @code{Blocktype} type efficiently manages the -allocation of fixed-size blocks by minimizing the number of times that -@code{malloc()} and @code{free()} are called. It allocates memory in -large chunks, subdivides the chunks into blocks of the proper size, and -returns the blocks as requested. When blocks are freed, they are placed -onto a linked list, so they can be efficiently reused. This data type -is not much used in XEmacs currently, because it's a fairly new -addition. - -@cindex dynamic array -The @code{Dynarr} type implements a @dfn{dynamic array}, which is -similar to a standard C array but has no fixed limit on the number of -elements it can contain. Dynamic arrays can hold elements of any type, -and when you add a new element, the array automatically resizes itself -if it isn't big enough. Dynarrs are extensively used in the redisplay -mechanism. - - - @example @file{inline.c} @end example @@ -3682,335 +3964,7 @@ This module provides some terminal-control code necessary on versions of AIX prior to 4.1. -@node Major Textual Changes, Rules When Writing New C Code, The Modules of XEmacs, Top -@chapter Major Textual Changes -@cindex textual changes, major -@cindex major textual changes - -Sometimes major textual changes are made to the source. This means that -a search-and-replace is done to change type names and such. Some people -disagree with such changes, and certainly if done without good reason -will just lead to headaches. But it's important to keep the code clean -and understable, and consistent naming goes a long way towards this. - -An example of the right way to do this was the so-called "great integral -type renaming". - -@menu -* Great Integral Type Renaming:: -* Text/Char Type Renaming:: -@end menu - -@node Great Integral Type Renaming, Text/Char Type Renaming, Major Textual Changes, Major Textual Changes -@section Great Integral Type Renaming -@cindex Great Integral Type Renaming -@cindex integral type renaming, great -@cindex type renaming, integral -@cindex renaming, integral types - -The purpose of this is to rationalize the names used for various -integral types, so that they match their intended uses and follow -consist conventions, and eliminate types that were not semantically -different from each other. - -The conventions are: - -@itemize @bullet -@item -All integral types that measure quantities of anything are signed. Some -people disagree vociferously with this, but their arguments are mostly -theoretical, and are vastly outweighed by the practical headaches of -mixing signed and unsigned values, and more importantly by the far -increased likelihood of inadvertent bugs: Because of the broken "viral" -nature of unsigned quantities in C (operations involving mixed -signed/unsigned are done unsigned, when exactly the opposite is nearly -always wanted), even a single error in declaring a quantity unsigned -that should be signed, or even the even more subtle error of comparing -signed and unsigned values and forgetting the necessary cast, can be -catastrophic, as comparisons will yield wrong results. -Wsign-compare -is turned on specifically to catch this, but this tends to result in a -great number of warnings when mixing signed and unsigned, and the casts -are annoying. More has been written on this elsewhere. - -@item -All such quantity types just mentioned boil down to EMACS_INT, which is -32 bits on 32-bit machines and 64 bits on 64-bit machines. This is -guaranteed to be the same size as Lisp objects of type @code{int}, and (as -far as I can tell) of size_t (unsigned!) and ssize_t. The only type -below that is not an EMACS_INT is Hashcode, which is an unsigned value -of the same size as EMACS_INT. - -@item -Type names should be relatively short (no more than 10 characters or -so), with the first letter capitalized and no underscores if they can at -all be avoided. - -@item -"count" == a zero-based measurement of some quantity. Includes sizes, -offsets, and indexes. - -@item -"bpos" == a one-based measurement of a position in a buffer. "Charbpos" -and "Bytebpos" count text in the buffer, rather than bytes in memory; -thus Bytebpos does not directly correspond to the memory representation. -Use "Membpos" for this. - -@item -"Char" refers to internal-format characters, not to the C type "char", -which is really a byte. -@end itemize - -For the actual name changes, see the script below. - -I ran the following script to do the conversion. (NOTE: This script is -idempotent. You can safely run it multiple times and it will not screw -up previous results -- in fact, it will do nothing if nothing has -changed. Thus, it can be run repeatedly as necessary to handle patches -coming in from old workspaces, or old branches.) There are two tags, -just before and just after the change: @samp{pre-integral-type-rename} -and @samp{post-integral-type-rename}. When merging code from the main -trunk into a branch, the best thing to do is first merge up to -@samp{pre-integral-type-rename}, then apply the script and associated -changes, then merge from @samp{post-integral-type-change} to the -present. (Alternatively, just do the merging in one operation; but you -may then have a lot of conflicts needing to be resolved by hand.) - -Script @samp{fixtypes.sh} follows: - -@example ------------------------------------ cut ------------------------------------ -files="*.[ch] s/*.h m/*.h config.h.in ../configure.in Makefile.in.in ../lib-src/*.[ch] ../lwlib/*.[ch]" -gr Memory_Count Bytecount $files -gr Lstream_Data_Count Bytecount $files -gr Element_Count Elemcount $files -gr Hash_Code Hashcode $files -gr extcount bytecount $files -gr bufpos charbpos $files -gr bytind bytebpos $files -gr memind membpos $files -gr bufbyte intbyte $files -gr Extcount Bytecount $files -gr Bufpos Charbpos $files -gr Bytind Bytebpos $files -gr Memind Membpos $files -gr Bufbyte Intbyte $files -gr EXTCOUNT BYTECOUNT $files -gr BUFPOS CHARBPOS $files -gr BYTIND BYTEBPOS $files -gr MEMIND MEMBPOS $files -gr BUFBYTE INTBYTE $files -gr MEMORY_COUNT BYTECOUNT $files -gr LSTREAM_DATA_COUNT BYTECOUNT $files -gr ELEMENT_COUNT ELEMCOUNT $files -gr HASH_CODE HASHCODE $files ------------------------------------ cut ------------------------------------ -@end example - -The @samp{gr} script, and the scripts it uses, are documented in -@file{README.global-renaming}, because if placed in this file they would -need to have their @@ characters doubled, meaning you couldn't easily -cut and paste from the source. - -In addition to those programs, I needed to fix up a few other -things, particularly relating to the duplicate definitions of -types, now that some types merged with others. Specifically: - -@enumerate -@item -in @file{lisp.h}, removed duplicate declarations of Bytecount. The changed -code should now look like this: (In each code snippet below, the first -and last lines are the same as the original, as are all lines outside of -those lines. That allows you to locate the section to be replaced, and -replace the stuff in that section, verifying that there isn't anything -new added that would need to be kept.) - -@example ---------------------------------- snip ------------------------------------- -/* Counts of bytes or chars */ -typedef EMACS_INT Bytecount; -typedef EMACS_INT Charcount; - -/* Counts of elements */ -typedef EMACS_INT Elemcount; - -/* Hash codes */ -typedef unsigned long Hashcode; - -/* ------------------------ dynamic arrays ------------------- */ ---------------------------------- snip ------------------------------------- -@end example - -@item -in @file{lstream.h}, removed duplicate declaration of Bytecount. Rewrote the -comment about this type. The changed code should now look like this: - -@example ---------------------------------- snip ------------------------------------- -#endif - -/* The have been some arguments over the what the type should be that - specifies a count of bytes in a data block to be written out or read in, - using @code{Lstream_read()}, @code{Lstream_write()}, and related functions. - Originally it was long, which worked fine; Martin "corrected" these to - size_t and ssize_t on the grounds that this is theoretically cleaner and - is in keeping with the C standards. Unfortunately, this practice is - horribly error-prone due to design flaws in the way that mixed - signed/unsigned arithmetic happens. In fact, by doing this change, - Martin introduced a subtle but fatal error that caused the operation of - sending large mail messages to the SMTP server under Windows to fail. - By putting all values back to be signed, avoiding any signed/unsigned - mixing, the bug immediately went away. The type then in use was - Lstream_Data_Count, so that it be reverted cleanly if a vote came to - that. Now it is Bytecount. - - Some earlier comments about why the type must be signed: This MUST BE - SIGNED, since it also is used in functions that return the number of - bytes actually read to or written from in an operation, and these - functions can return -1 to signal error. - - Note that the standard Unix @code{read()} and @code{write()} functions define the - count going in as a size_t, which is UNSIGNED, and the count going - out as an ssize_t, which is SIGNED. This is a horrible design - flaw. Not only is it highly likely to lead to logic errors when a - -1 gets interpreted as a large positive number, but operations are - bound to fail in all sorts of horrible ways when a number in the - upper-half of the size_t range is passed in -- this number is - unrepresentable as an ssize_t, so code that checks to see how many - bytes are actually written (which is mandatory if you are dealing - with certain types of devices) will get completely screwed up. - - --ben -*/ - -typedef enum lstream_buffering ---------------------------------- snip ------------------------------------- -@end example - -@item -in @file{dumper.c}, there are four places, all inside of @code{switch()} statements, -where XD_BYTECOUNT appears twice as a case tag. In each case, the two -case blocks contain identical code, and you should *REMOVE THE SECOND* -and leave the first. -@end enumerate - -@node Text/Char Type Renaming, , Great Integral Type Renaming, Major Textual Changes -@section Text/Char Type Renaming -@cindex Text/Char Type Renaming -@cindex type renaming, text/char -@cindex renaming, text/char types - -The purpose of this was - -@enumerate -@item -To distinguish between ``charptr'' when it refers to operations on -the pointer itself and when it refers to operations on text -@item -To use consistent naming for everything referring to internal format, i.e. -@end enumerate - -@example - Itext == text in internal format - Ibyte == a byte in such text - Ichar == a char as represented in internal character format -@end example - -Thus e.g. - -@example - set_charptr_emchar -> set_itext_ichar -@end example - -This was done using a script like this: - -@example -files="*.[ch] s/*.h m/*.h config.h.in ../configure.in Makefile.in.in ../lib-src/*.[ch] ../lwlib/*.[ch]" -gr Intbyte Ibyte $files -gr INTBYTE IBYTE $files -gr intbyte ibyte $files -gr EMCHAR ICHAR $files -gr emchar ichar $files -gr Emchar Ichar $files -gr INC_CHARPTR INC_IBYTEPTR $files -gr DEC_CHARPTR DEC_IBYTEPTR $files -gr VALIDATE_CHARPTR VALIDATE_IBYTEPTR $files -gr valid_charptr valid_ibyteptr $files -gr CHARPTR ITEXT $files -gr charptr itext $files -gr Charptr Itext $files -@end example - -See above for the source to @samp{gr}. - -As in the integral-types change, there are pre and post tags before and -after the change: - -@example - pre-internal-format-textual-renaming - post-internal-format-textual-renaming -@end example - -When merging a large branch, follow the same sort of procedure -documented above, using these tags -- essentially sync up to the pre -tag, then apply the script yourself, then sync from the post tag to the -present. You can probably do the same if you don't have a separate -workspace, but do have lots of outstanding changes and you'd rather not -just merge all the textual changes directly. Use something like this: - -(WARNING: I'm not a CVS guru; before trying this, or any large operation -that might potentially mess things up, @strong{DEFINITELY} make a backup of -your existing workspace.) - -@example -cup -r pre-internal-format-textual-renaming -<apply script> -cup -A -j post-internal-format-textual-renaming -j HEAD -@end example - -This might also work: - -@example -cup -j pre-internal-format-textual-renaming -<apply script> -cup -j post-internal-format-textual-renaming -j HEAD -@end example - -ben - -The following is a script to go in the opposite direction: - -@example -files="*.[ch] s/*.h m/*.h config.h.in ../configure.in Makefile.in.in ../lib-src/*.[ch] ../lwlib/*.[ch]" - -# Evidently Perl considers _ to be a word char ala \b, even though XEmacs -# doesn't. We need to be careful here with ibyte/ichar because of words -# like Richard, @code{eicharlen()}, multibyte, HIBYTE, etc. - -gr Ibyte Intbyte $files -gr '\bIBYTE' INTBYTE $files -gr '\bibyte' intbyte $files -gr '\bICHAR' EMCHAR $files -gr '\bichar' emchar $files -gr '\bIchar' Emchar $files -gr '\bIBYTEPTR' CHARPTR $files -gr '\bibyteptr' charptr $files -gr '\bITEXT' CHARPTR $files -gr '\bitext' charptr $files -gr '\bItext' CHARPTR $files - -gr '_IBYTE' _INTBYTE $files -gr '_ibyte' _intbyte $files -gr '_ICHAR' _EMCHAR $files -gr '_ichar' _emchar $files -gr '_Ichar' _Emchar $files -gr '_IBYTEPTR' _CHARPTR $files -gr '_ibyteptr' _charptr $files -gr '_ITEXT' _CHARPTR $files -gr '_itext' _charptr $files -gr '_Itext' _CHARPTR $files -@end example - -@node Rules When Writing New C Code, Regression Testing XEmacs, Major Textual Changes, Top +@node Rules When Writing New C Code, Regression Testing XEmacs, The Modules of XEmacs, Top @chapter Rules When Writing New C Code @cindex writing new C code, rules when @cindex C code, rules when writing new @@ -4024,24 +3978,246 @@ situations, often in code far away from where the actual breakage is. @menu -* A Reader's Guide to XEmacs Coding Conventions:: -* General Coding Rules:: -* Object-Oriented Techniques for C:: +* Introduction to Writing C Code:: +* Writing New Modules:: +* Working with Lisp Objects:: * Writing Lisp Primitives:: * Writing Good Comments:: * Adding Global Lisp Variables:: * Writing Macros:: * Proper Use of Unsigned Types:: -* Techniques for XEmacs Developers:: +* Major Textual Changes:: +* Debugging and Testing:: @end menu See also @ref{Coding for Mule}. -@node A Reader's Guide to XEmacs Coding Conventions, General Coding Rules, Rules When Writing New C Code, Rules When Writing New C Code -@section A Reader's Guide to XEmacs Coding Conventions +@node Introduction to Writing C Code, Writing New Modules, Rules When Writing New C Code, Rules When Writing New C Code +@section Introduction to Writing C Code +@cindex introduction to writing c code @cindex coding conventions -@cindex reader's guide -@cindex coding rules, naming + +The C code is actually written in a dialect of C called @dfn{Clean C}, +meaning that it can be compiled, mostly warning-free, with either a C +or C++ compiler. Coding in Clean C has several advantages over plain +C. C++ compilers are more nit-picking, and a number of coding errors +have been found by compiling with C++. The ability to use both C and +C++ tools means that a greater variety of development tools are +available to the developer. In addition, the ability to overload +operators in C++ means it is possible, for error-checking purposes, to +redefine certain simple types (normally defined as aliases for simple +built-in types such as @code{unsigned char} or @code{long}) as +classes, strictly limiting the permissible operations and catching +illegal implicit casts and such. + +XEmacs follows the GNU coding standards, which are documented +separately in @xref{top,,, standards, GNU Coding Standards}. This +section mainly documents standards that are not included in that +document; typically this consists of standards that are specifically +relevant to the XEmacs code itself. + +First, a recap of the GNU standards: + +@itemize @bullet +@item +Put a space after every comma. +@item +Put a space before the parenthesis that begins a function call, +macro call, function declaration or definition, or control +statement (if, while, switch, for). (DO NOT do this for macro +definitions; this is invalid preprocessor syntax.) +@item +The brace that begins a control statement (if, while, for, switch, +do) or a function definition should go on a line by itself. +@item +In function definitions, put the return type and all other +qualifiers on a line before the function name. Thus, the function +name is always at the beginning of a line. +@item +Indentation level is two spaces. (However, the first and following +statements of a while/for/if/etc. block are indented four spaces +from the while/for/if keyword. The opening and closing braces are +indented two spaces.) +@item +Variable and function names should be all lowercase, with underscores +separating words, except for a prefixing tag, which may be in +uppercase. Do not use the mixed-case convention (e.g. +SetVariableToValue ()) and *especially* do not use Microsoft +Hungarian notation (char **rgszRedundantTag). +@item +preprocessor and enum constants should be all uppercase, and should +be prefixed with a tag that groups related constants together. +@end itemize + +Now, the XEmacs coding standards: + +@subheading Specially-prefixed functions/variables: + +@itemize @bullet +@item +All global C variables whose value is constant and is a symbol begin +with a capital Q, e.g. Qkey_press_event. (The type will always be +Lisp_Object.) +@item +All other global C variables whose value is a Lisp_Object (this +includes variables that forward into Lisp variables plus others like +Vselected_console) begin with a capital V. +@item +No C variables whose value is other than a Lisp_Object should begin +with a capital V. (This includes C variables that forward into +integer or boolean Lisp variables.) +@item +All global C variables whose value is a struct Lisp_Subr begin with a +capital S. (This only occurs in connection with DEFUN ()). +@item +All C functions that are Lisp primitives begin with a capital F, +and no others should begin this way. +@end itemize + +@subheading Functions for manipulating Lisp types: + +@itemize @bullet +@item +Any function that creates an empty or mostly empty Lisp object +should begin allocate_(). (*Not* make_().) (Except, of course, +for Lisp primitives, which usually begin Fmake_()). +@item +Any function that converts a pointer into an equivalent Lisp_Object +should begin make_(). +@item +Any function that converts a Lisp_Object into its equivalent pointer +and checks the type and validity of the object (e.g. making sure +it's not dead) should begin decode_(). +@item +Any function that looks up a Lisp object (e.g. buffer, face) given +a symbol or string should begin get_(). (Except, of course, for +Lisp primitives, which usually begin Fget_()). +@end itemize + +@subheading Other: + +@itemize @bullet +@item +Any header-file declarations of the sort + +struct foobar; + +go into the "types" section of lisp.h. +@end itemize + +@node Writing New Modules, Working with Lisp Objects, Introduction to Writing C Code, Rules When Writing New C Code +@section Writing New Modules +@cindex writing new modules + +Every module includes @file{<config.h>} (angle brackets so that +@samp{--srcdir} works correctly; @file{config.h} may or may not be in +the same directory as the C sources) and @file{lisp.h}. @file{config.h} +must always be included before any other header files (including +system header files) to ensure that certain tricks played by various +@file{s/} and @file{m/} files work out correctly. + +When including header files, always use angle brackets, not double +quotes, except when the file to be included is always in the same +directory as the including file. If either file is a generated file, +then that is not likely to be the case. In order to understand why we +have this rule, imagine what happens when you do a build in the source +directory using @samp{./configure} and another build in another +directory using @samp{../work/configure}. There will be two different +@file{config.h} files. Which one will be used if you @samp{#include +"config.h"}? + +Almost every module contains a @code{syms_of_*()} function and a +@code{vars_of_*()} function. The former declares any Lisp primitives +you have defined and defines any symbols you will be using. The latter +declares any global Lisp variables you have added and initializes global +C variables in the module. @strong{Important}: There are stringent +requirements on exactly what can go into these functions. See the +comment in @file{emacs.c}. The reason for this is to avoid obscure +unwanted interactions during initialization. If you don't follow these +rules, you'll be sorry! If you want to do anything that isn't allowed, +create a @code{complex_vars_of_*()} function for it. Doing this is +tricky, though: you have to make sure your function is called at the +right time so that all the initialization dependencies work out. + +Declare each function of these kinds in @file{symsinit.h}. Make sure +it's called in the appropriate place in @file{emacs.c}. You never need +to include @file{symsinit.h} directly, because it is included by +@file{lisp.h}. + +@strong{All global and static variables that are to be modifiable must +be declared uninitialized.} This means that you may not use the +``declare with initializer'' form for these variables, such as @code{int +some_variable = 0;}. The reason for this has to do with some kludges +done during the dumping process: If possible, the initialized data +segment is re-mapped so that it becomes part of the (unmodifiable) code +segment in the dumped executable. This allows this memory to be shared +among multiple running XEmacs processes. XEmacs is careful to place as +much constant data as possible into initialized variables during the +@file{temacs} phase. + +@cindex copy-on-write +@strong{Please note:} This kludge only works on a few systems nowadays, +and is rapidly becoming irrelevant because most modern operating systems +provide @dfn{copy-on-write} semantics. All data is initially shared +between processes, and a private copy is automatically made (on a +page-by-page basis) when a process first attempts to write to a page of +memory. + +Formerly, there was a requirement that static variables not be declared +inside of functions. This had to do with another hack along the same +vein as what was just described: old USG systems put statically-declared +variables in the initialized data space, so those header files had a +@code{#define static} declaration. (That way, the data-segment remapping +described above could still work.) This fails badly on static variables +inside of functions, which suddenly become automatic variables; +therefore, you weren't supposed to have any of them. This awful kludge +has been removed in XEmacs because + +@enumerate +@item +almost all of the systems that used this kludge ended up having +to disable the data-segment remapping anyway; +@item +the only systems that didn't were extremely outdated ones; +@item +this hack completely messed up inline functions. +@end enumerate + +Here are things to know when you create a new source file: + +@itemize @bullet +@item +All @file{.c} files should @code{#include <config.h>} first. Almost all +@file{.c} files should @code{#include "lisp.h"} second. + +@item +Generated header files should be included using the @samp{#include <...>} +syntax, not the @samp{#include "..."} syntax. The generated headers are: + +@file{config.h sheap-adjust.h paths.h Emacs.ad.h} + +The basic rule is that you should assume builds using @samp{--srcdir} +and the @samp{#include <...>} syntax needs to be used when the +to-be-included generated file is in a potentially different directory +@emph{at compile time}. The non-obvious C rule is that +@samp{#include "..."} means to search for the included file in the same +directory as the including file, @emph{not} in the current directory. +Normally this is not a problem but when building with @samp{--srcdir}, +@file{make} will search the @samp{VPATH} for you, while the C compiler +knows nothing about it. + +@item +Header files should @emph{not} include @samp{<config.h>} and +@samp{"lisp.h"}. It is the responsibility of the @file{.c} files that +use it to do so. + +@end itemize + +@node Working with Lisp Objects, Writing Lisp Primitives, Writing New Modules, Rules When Writing New C Code +@section Working with Lisp Objects +@cindex working with lisp objects + +@subheading Conventions involving Lisp objects Of course the low-level implementation language of XEmacs is C, but much of that uses the Lisp engine to do its work. However, because the code @@ -4129,113 +4305,7 @@ their return values must be Lisp_Objects. (This is hidden in the @code{DEFUN} macro.) - -@node General Coding Rules, Object-Oriented Techniques for C, A Reader's Guide to XEmacs Coding Conventions, Rules When Writing New C Code -@section General Coding Rules -@cindex coding rules, general - -The C code is actually written in a dialect of C called @dfn{Clean C}, -meaning that it can be compiled, mostly warning-free, with either a C or -C++ compiler. Coding in Clean C has several advantages over plain C. -C++ compilers are more nit-picking, and a number of coding errors have -been found by compiling with C++. The ability to use both C and C++ -tools means that a greater variety of development tools are available to -the developer. In addition, the ability to overload operators in C++ -means it is possible, for error-checking purposes, to redefine certain -simple types (normally defined as aliases for simple built-in types such -as @code{unsigned char} or @code{long}) as classes, strictly limiting the permissible -operations and catching illegal implicit casts and such. - -Every module includes @file{<config.h>} (angle brackets so that -@samp{--srcdir} works correctly; @file{config.h} may or may not be in -the same directory as the C sources) and @file{lisp.h}. @file{config.h} -must always be included before any other header files (including -system header files) to ensure that certain tricks played by various -@file{s/} and @file{m/} files work out correctly. - -When including header files, always use angle brackets, not double -quotes, except when the file to be included is always in the same -directory as the including file. If either file is a generated file, -then that is not likely to be the case. In order to understand why we -have this rule, imagine what happens when you do a build in the source -directory using @samp{./configure} and another build in another -directory using @samp{../work/configure}. There will be two different -@file{config.h} files. Which one will be used if you @samp{#include -"config.h"}? - -Almost every module contains a @code{syms_of_*()} function and a -@code{vars_of_*()} function. The former declares any Lisp primitives -you have defined and defines any symbols you will be using. The latter -declares any global Lisp variables you have added and initializes global -C variables in the module. @strong{Important}: There are stringent -requirements on exactly what can go into these functions. See the -comment in @file{emacs.c}. The reason for this is to avoid obscure -unwanted interactions during initialization. If you don't follow these -rules, you'll be sorry! If you want to do anything that isn't allowed, -create a @code{complex_vars_of_*()} function for it. Doing this is -tricky, though: you have to make sure your function is called at the -right time so that all the initialization dependencies work out. - -Declare each function of these kinds in @file{symsinit.h}. Make sure -it's called in the appropriate place in @file{emacs.c}. You never need -to include @file{symsinit.h} directly, because it is included by -@file{lisp.h}. - -@strong{All global and static variables that are to be modifiable must -be declared uninitialized.} This means that you may not use the -``declare with initializer'' form for these variables, such as @code{int -some_variable = 0;}. The reason for this has to do with some kludges -done during the dumping process: If possible, the initialized data -segment is re-mapped so that it becomes part of the (unmodifiable) code -segment in the dumped executable. This allows this memory to be shared -among multiple running XEmacs processes. XEmacs is careful to place as -much constant data as possible into initialized variables during the -@file{temacs} phase. - -@cindex copy-on-write -@strong{Please note:} This kludge only works on a few systems nowadays, -and is rapidly becoming irrelevant because most modern operating systems -provide @dfn{copy-on-write} semantics. All data is initially shared -between processes, and a private copy is automatically made (on a -page-by-page basis) when a process first attempts to write to a page of -memory. - -Formerly, there was a requirement that static variables not be declared -inside of functions. This had to do with another hack along the same -vein as what was just described: old USG systems put statically-declared -variables in the initialized data space, so those header files had a -@code{#define static} declaration. (That way, the data-segment remapping -described above could still work.) This fails badly on static variables -inside of functions, which suddenly become automatic variables; -therefore, you weren't supposed to have any of them. This awful kludge -has been removed in XEmacs because - -@enumerate -@item -almost all of the systems that used this kludge ended up having -to disable the data-segment remapping anyway; -@item -the only systems that didn't were extremely outdated ones; -@item -this hack completely messed up inline functions. -@end enumerate - -The C source code makes heavy use of C preprocessor macros. One popular -macro style is: - -@example -#define FOO(var, value) do @{ \ - Lisp_Object FOO_value = (value); \ - ... /* compute using FOO_value */ \ - (var) = bar; \ -@} while (0) -@end example - -The @code{do @{...@} while (0)} is a standard trick to allow FOO to have -statement semantics, so that it can safely be used within an @code{if} -statement in C, for example. Multiple evaluation is prevented by -copying a supplied argument into a local variable, so that -@code{FOO(var,fun(1))} only calls @code{fun} once. +@subheading Working with Lisp lists Lisp lists are popular data structures in the C code as well as in Elisp. There are two sets of macros that iterate over lists. @@ -4253,10 +4323,7 @@ @code{LIST_LOOP_DELETE_IF} delete elements from a lisp list satisfying some predicate. -@node Object-Oriented Techniques for C, Writing Lisp Primitives, General Coding Rules, Rules When Writing New C Code -@section Object-Oriented Techniques for C -@cindex coding rules, object-oriented -@cindex object-oriented techniques +@subheading Implementation of Lisp objects At the lowest levels, XEmacs makes heavy use of object-oriented techniques to promote code-sharing and uniform interfaces for different @@ -4352,8 +4419,36 @@ that method for instances of other subtypes of @samp{<Type>}, and there may be a rather large number of them. - -@node Writing Lisp Primitives, Writing Good Comments, Object-Oriented Techniques for C, Rules When Writing New C Code +@cindex Lisp object types, creating +@cindex creating Lisp object types +@cindex object types, creating Lisp +Here is a checklist of things to do when creating a new lisp object type +named @var{foo}: + +@enumerate +@item +create @var{foo}.h +@item +create @var{foo}.c +@item +add definitions of @code{syms_of_@var{foo}}, etc. to @file{@var{foo}.c} +@item +add declarations of @code{syms_of_@var{foo}}, etc. to @file{symsinit.h} +@item +add calls to @code{syms_of_@var{foo}}, etc. to @file{emacs.c} +@item +add definitions of macros like @code{CHECK_@var{FOO}} and +@code{@var{FOO}P} to @file{@var{foo}.h} +@item +add the new type index to @code{enum lrecord_type} +@item +add a DEFINE_LRECORD_IMPLEMENTATION call to @file{@var{foo}.c} +@item +add an INIT_LRECORD_IMPLEMENTATION call to @code{syms_of_@var{foo}.c} +@end enumerate + + +@node Writing Lisp Primitives, Writing Good Comments, Working with Lisp Objects, Rules When Writing New C Code @section Writing Lisp Primitives @cindex writing Lisp primitives @cindex Lisp primitives, writing @@ -4577,7 +4672,7 @@ This call to @code{DEFSUBR} should go in the @code{syms_of_*()} function at the end of the module. If no such function exists, create it and make sure to also declare it in @file{symsinit.h} and call it from the -appropriate spot in @code{main()}. @xref{General Coding Rules}. +appropriate spot in @code{main()}. @xref{Writing New Modules}. Note that C code cannot call functions by name unless they are defined in C. The way to call a function written in Lisp from C is to use @@ -4737,163 +4832,6 @@ @cindex writing macros @cindex macros, writing -The three golden rules of macros: - -@enumerate -@item -Anything that's an lvalue can be evaluated more than once. -@item -Macros where anything else can be evaluated more than once should -have the word "unsafe" in their name (exceptions may be made for -large sets of macros that evaluate arguments of certain types more -than once, e.g. struct buffer * arguments, when clearly indicated in -the macro documentation). These macros are generally meant to be -called only by other macros that have already stored the calling -values in temporary variables. -@item -Nothing else can be evaluated more than once. Use inline -functions, if necessary, to prevent multiple evaluation. -@end enumerate - -NOTE: The functions and macros below are given full prototypes in their -docs, even when the implementation is a macro. In such cases, passing -an argument of a type other than expected will produce undefined -results. Also, given that macros can do things functions can't (in -particular, directly modify arguments as if they were passed by -reference), the declaration syntax has been extended to include the -call-by-reference syntax from C++, where an & after a type indicates -that the argument is an lvalue and is passed by reference, i.e. the -function can modify its value. (This is equivalent in C to passing a -pointer to the argument, but without the need to explicitly worry about -pointers.) - -When to capitalize macros: - -@itemize @bullet -@item -Capitalize macros doing stuff obviously impossible with (C) -functions, e.g. directly modifying arguments as if they were passed by -reference. -@item -Capitalize macros that evaluate @strong{any} argument more than once regardless -of whether that's "allowed" (e.g. buffer arguments). -@item -Capitalize macros that directly access a field in a Lisp_Object or -its equivalent underlying structure. In such cases, access through the -Lisp_Object precedes the macro with an X, and access through the underlying -structure doesn't. -@item -Capitalize certain other basic macros relating to Lisp_Objects; e.g. -FRAMEP, CHECK_FRAME, etc. -@item -Try to avoid capitalizing any other macros. -@end itemize - -@node Proper Use of Unsigned Types, Techniques for XEmacs Developers, Writing Macros, Rules When Writing New C Code -@section Proper Use of Unsigned Types -@cindex unsigned types, proper use of -@cindex types, proper use of unsigned - -Avoid using @code{unsigned int} and @code{unsigned long} whenever -possible. Unsigned types are viral -- any arithmetic or comparisons -involving mixed signed and unsigned types are automatically converted to -unsigned, which is almost certainly not what you want. Many subtle and -hard-to-find bugs are created by careless use of unsigned types. In -general, you should almost @emph{never} use an unsigned type to hold a -regular quantity of any sort. The only exceptions are - -@enumerate -@item -When there's a reasonable possibility you will actually need all 32 or -64 bits to store the quantity. -@item -When calling existing API's that require unsigned types. In this case, -you should still do all manipulation using signed types, and do the -conversion at the very threshold of the API call. -@item -In existing code that you don't want to modify because you don't -maintain it. -@item -In bit-field structures. -@end enumerate - -Other reasonable uses of @code{unsigned int} and @code{unsigned long} -are representing non-quantities -- e.g. bit-oriented flags and such. - -@node Techniques for XEmacs Developers, , Proper Use of Unsigned Types, Rules When Writing New C Code -@section Techniques for XEmacs Developers -@cindex techniques for XEmacs developers -@cindex developers, techniques for XEmacs - -@cindex Purify -@cindex Quantify -To make a purified XEmacs, do: @code{make puremacs}. -To make a quantified XEmacs, do: @code{make quantmacs}. - -You simply can't dump Quantified and Purified images (unless using the -portable dumper). Purify gets confused when xemacs frees memory in one -process that was allocated in a @emph{different} process on a different -machine! Run it like so: -@example -temacs -batch -l loadup.el run-temacs @var{xemacs-args...} -@end example - -@cindex error checking -Before you go through the trouble, are you compiling with all -debugging and error-checking off? If not, try that first. Be warned -that while Quantify is directly responsible for quite a few -optimizations which have been made to XEmacs, doing a run which -generates results which can be acted upon is not necessarily a trivial -task. - -Also, if you're still willing to do some runs make sure you configure -with the @samp{--quantify} flag. That will keep Quantify from starting -to record data until after the loadup is completed and will shut off -recording right before it shuts down (which generates enough bogus data -to throw most results off). It also enables three additional elisp -commands: @code{quantify-start-recording-data}, -@code{quantify-stop-recording-data} and @code{quantify-clear-data}. - -If you want to make XEmacs faster, target your favorite slow benchmark, -run a profiler like Quantify, @code{gprof}, or @code{tcov}, and figure -out where the cycles are going. In many cases you can localize the -problem (because a particular new feature or even a single patch -elicited it). Don't hesitate to use brute force techniques like a -global counter incremented at strategic places, especially in -combination with other performance indications (@emph{e.g.}, degree of -buffer fragmentation into extents). - -Specific projects: - -@itemize @bullet -@item -Make the garbage collector faster. Figure out how to write an -incremental garbage collector. -@item -Write a compiler that takes bytecode and spits out C code. -Unfortunately, you will then need a C compiler and a more fully -developed module system. -@item -Speed up redisplay. -@item -Speed up syntax highlighting. It was suggested that ``maybe moving some -of the syntax highlighting capabilities into C would make a -difference.'' Wrong idea, I think. When processing one 400kB file a -particular low-level routine was being called 40 @emph{million} times -simply for @emph{one} call to @code{newline-and-indent}. Syntax -highlighting needs to be rewritten to use a reliable, fast parser, then -to trust the pre-parsed structure, and only do re-highlighting locally -to a text change. Modern machines are fast enough to implement such -parsers in Lisp; but no machine will ever be fast enough to deal with -quadratic (or worse) algorithms! -@item -Implement tail recursion in Emacs Lisp (hard!). -@end itemize - -Unfortunately, Emacs Lisp is slow, and is going to stay slow. Function -calls in elisp are especially expensive. Iterating over a long list is -going to be 30 times faster implemented in C than in Elisp. - Heavily used small code fragments need to be fast. The traditional way to implement such code fragments in C is with macros. But macros in C are known to be broken. @@ -4905,9 +4843,10 @@ Variable names used in macros may collide with caller's variables, causing (at least) unwanted compiler warnings. -In order to solve these problems, and maintain statement semantics, one -should use the @code{do @{ ... @} while (0)} trick while trying to -reference macro arguments exactly once using local variables. +In order to solve these problems, and maintain statement semantics, +one should use the @code{do @{ ... @} while (0)} trick (which safely +works inside of if statements) while trying to reference macro +arguments exactly once using local variables. Let's take a look at this poor macro definition: @@ -4994,6 +4933,490 @@ functions are in fact inlined in all translation units, then the linker can just discard @code{inline.o}, since it contains only unreferenced code). +The three golden rules of macros: + +@enumerate +@item +Anything that's an lvalue can be evaluated more than once. +@item +Macros where anything else can be evaluated more than once should +have the word "unsafe" in their name (exceptions may be made for +large sets of macros that evaluate arguments of certain types more +than once, e.g. struct buffer * arguments, when clearly indicated in +the macro documentation). These macros are generally meant to be +called only by other macros that have already stored the calling +values in temporary variables. +@item +Nothing else can be evaluated more than once. Use inline +functions, if necessary, to prevent multiple evaluation. +@end enumerate + +NOTE: The functions and macros below are given full prototypes in their +docs, even when the implementation is a macro. In such cases, passing +an argument of a type other than expected will produce undefined +results. Also, given that macros can do things functions can't (in +particular, directly modify arguments as if they were passed by +reference), the declaration syntax has been extended to include the +call-by-reference syntax from C++, where an & after a type indicates +that the argument is an lvalue and is passed by reference, i.e. the +function can modify its value. (This is equivalent in C to passing a +pointer to the argument, but without the need to explicitly worry about +pointers.) + +When to capitalize macros: + +@itemize @bullet +@item +Capitalize macros doing stuff obviously impossible with (C) +functions, e.g. directly modifying arguments as if they were passed by +reference. +@item +Capitalize macros that evaluate @strong{any} argument more than once regardless +of whether that's "allowed" (e.g. buffer arguments). +@item +Capitalize macros that directly access a field in a Lisp_Object or +its equivalent underlying structure. In such cases, access through the +Lisp_Object precedes the macro with an X, and access through the underlying +structure doesn't. +@item +Capitalize certain other basic macros relating to Lisp_Objects; e.g. +FRAMEP, CHECK_FRAME, etc. +@item +Try to avoid capitalizing any other macros. +@end itemize + +@node Proper Use of Unsigned Types, Major Textual Changes, Writing Macros, Rules When Writing New C Code +@section Proper Use of Unsigned Types +@cindex unsigned types, proper use of +@cindex types, proper use of unsigned + +Avoid using @code{unsigned int} and @code{unsigned long} whenever +possible. Unsigned types are viral -- any arithmetic or comparisons +involving mixed signed and unsigned types are automatically converted to +unsigned, which is almost certainly not what you want. Many subtle and +hard-to-find bugs are created by careless use of unsigned types. In +general, you should almost @emph{never} use an unsigned type to hold a +regular quantity of any sort. The only exceptions are + +@enumerate +@item +When there's a reasonable possibility you will actually need all 32 or +64 bits to store the quantity. +@item +When calling existing API's that require unsigned types. In this case, +you should still do all manipulation using signed types, and do the +conversion at the very threshold of the API call. +@item +In existing code that you don't want to modify because you don't +maintain it. +@item +In bit-field structures. +@end enumerate + +Other reasonable uses of @code{unsigned int} and @code{unsigned long} +are representing non-quantities -- e.g. bit-oriented flags and such. + +@node Major Textual Changes, Debugging and Testing, Proper Use of Unsigned Types, Rules When Writing New C Code +@section Major Textual Changes +@cindex textual changes, major +@cindex major textual changes + +Sometimes major textual changes are made to the source. This means that +a search-and-replace is done to change type names and such. Some people +disagree with such changes, and certainly if done without good reason +will just lead to headaches. But it's important to keep the code clean +and understable, and consistent naming goes a long way towards this. + +An example of the right way to do this was the so-called "great integral +type renaming". + +@menu +* Great Integral Type Renaming:: +* Text/Char Type Renaming:: +@end menu + +@node Great Integral Type Renaming, Text/Char Type Renaming, Major Textual Changes, Major Textual Changes +@subsection Great Integral Type Renaming +@cindex Great Integral Type Renaming +@cindex integral type renaming, great +@cindex type renaming, integral +@cindex renaming, integral types + +The purpose of this is to rationalize the names used for various +integral types, so that they match their intended uses and follow +consist conventions, and eliminate types that were not semantically +different from each other. + +The conventions are: + +@itemize @bullet +@item +All integral types that measure quantities of anything are signed. Some +people disagree vociferously with this, but their arguments are mostly +theoretical, and are vastly outweighed by the practical headaches of +mixing signed and unsigned values, and more importantly by the far +increased likelihood of inadvertent bugs: Because of the broken "viral" +nature of unsigned quantities in C (operations involving mixed +signed/unsigned are done unsigned, when exactly the opposite is nearly +always wanted), even a single error in declaring a quantity unsigned +that should be signed, or even the even more subtle error of comparing +signed and unsigned values and forgetting the necessary cast, can be +catastrophic, as comparisons will yield wrong results. -Wsign-compare +is turned on specifically to catch this, but this tends to result in a +great number of warnings when mixing signed and unsigned, and the casts +are annoying. More has been written on this elsewhere. + +@item +All such quantity types just mentioned boil down to EMACS_INT, which is +32 bits on 32-bit machines and 64 bits on 64-bit machines. This is +guaranteed to be the same size as Lisp objects of type @code{int}, and (as +far as I can tell) of size_t (unsigned!) and ssize_t. The only type +below that is not an EMACS_INT is Hashcode, which is an unsigned value +of the same size as EMACS_INT. + +@item +Type names should be relatively short (no more than 10 characters or +so), with the first letter capitalized and no underscores if they can at +all be avoided. + +@item +"count" == a zero-based measurement of some quantity. Includes sizes, +offsets, and indexes. + +@item +"bpos" == a one-based measurement of a position in a buffer. "Charbpos" +and "Bytebpos" count text in the buffer, rather than bytes in memory; +thus Bytebpos does not directly correspond to the memory representation. +Use "Membpos" for this. + +@item +"Char" refers to internal-format characters, not to the C type "char", +which is really a byte. +@end itemize + +For the actual name changes, see the script below. + +I ran the following script to do the conversion. (NOTE: This script is +idempotent. You can safely run it multiple times and it will not screw +up previous results -- in fact, it will do nothing if nothing has +changed. Thus, it can be run repeatedly as necessary to handle patches +coming in from old workspaces, or old branches.) There are two tags, +just before and just after the change: @samp{pre-integral-type-rename} +and @samp{post-integral-type-rename}. When merging code from the main +trunk into a branch, the best thing to do is first merge up to +@samp{pre-integral-type-rename}, then apply the script and associated +changes, then merge from @samp{post-integral-type-change} to the +present. (Alternatively, just do the merging in one operation; but you +may then have a lot of conflicts needing to be resolved by hand.) + +Script @samp{fixtypes.sh} follows: + +@example +----------------------------------- cut ------------------------------------ +files="*.[ch] s/*.h m/*.h config.h.in ../configure.in Makefile.in.in ../lib-src/*.[ch] ../lwlib/*.[ch]" +gr Memory_Count Bytecount $files +gr Lstream_Data_Count Bytecount $files +gr Element_Count Elemcount $files +gr Hash_Code Hashcode $files +gr extcount bytecount $files +gr bufpos charbpos $files +gr bytind bytebpos $files +gr memind membpos $files +gr bufbyte intbyte $files +gr Extcount Bytecount $files +gr Bufpos Charbpos $files +gr Bytind Bytebpos $files +gr Memind Membpos $files +gr Bufbyte Intbyte $files +gr EXTCOUNT BYTECOUNT $files +gr BUFPOS CHARBPOS $files +gr BYTIND BYTEBPOS $files +gr MEMIND MEMBPOS $files +gr BUFBYTE INTBYTE $files +gr MEMORY_COUNT BYTECOUNT $files +gr LSTREAM_DATA_COUNT BYTECOUNT $files +gr ELEMENT_COUNT ELEMCOUNT $files +gr HASH_CODE HASHCODE $files +----------------------------------- cut ------------------------------------ +@end example + +The @samp{gr} script, and the scripts it uses, are documented in +@file{README.global-renaming}, because if placed in this file they would +need to have their @@ characters doubled, meaning you couldn't easily +cut and paste from the source. + +In addition to those programs, I needed to fix up a few other +things, particularly relating to the duplicate definitions of +types, now that some types merged with others. Specifically: + +@enumerate +@item +in @file{lisp.h}, removed duplicate declarations of Bytecount. The changed +code should now look like this: (In each code snippet below, the first +and last lines are the same as the original, as are all lines outside of +those lines. That allows you to locate the section to be replaced, and +replace the stuff in that section, verifying that there isn't anything +new added that would need to be kept.) + +@example +--------------------------------- snip ------------------------------------- +/* Counts of bytes or chars */ +typedef EMACS_INT Bytecount; +typedef EMACS_INT Charcount; + +/* Counts of elements */ +typedef EMACS_INT Elemcount; + +/* Hash codes */ +typedef unsigned long Hashcode; + +/* ------------------------ dynamic arrays ------------------- */ +--------------------------------- snip ------------------------------------- +@end example + +@item +in @file{lstream.h}, removed duplicate declaration of Bytecount. Rewrote the +comment about this type. The changed code should now look like this: + +@example +--------------------------------- snip ------------------------------------- +#endif + +/* The have been some arguments over the what the type should be that + specifies a count of bytes in a data block to be written out or read in, + using @code{Lstream_read()}, @code{Lstream_write()}, and related functions. + Originally it was long, which worked fine; Martin "corrected" these to + size_t and ssize_t on the grounds that this is theoretically cleaner and + is in keeping with the C standards. Unfortunately, this practice is + horribly error-prone due to design flaws in the way that mixed + signed/unsigned arithmetic happens. In fact, by doing this change, + Martin introduced a subtle but fatal error that caused the operation of + sending large mail messages to the SMTP server under Windows to fail. + By putting all values back to be signed, avoiding any signed/unsigned + mixing, the bug immediately went away. The type then in use was + Lstream_Data_Count, so that it be reverted cleanly if a vote came to + that. Now it is Bytecount. + + Some earlier comments about why the type must be signed: This MUST BE + SIGNED, since it also is used in functions that return the number of + bytes actually read to or written from in an operation, and these + functions can return -1 to signal error. + + Note that the standard Unix @code{read()} and @code{write()} functions define the + count going in as a size_t, which is UNSIGNED, and the count going + out as an ssize_t, which is SIGNED. This is a horrible design + flaw. Not only is it highly likely to lead to logic errors when a + -1 gets interpreted as a large positive number, but operations are + bound to fail in all sorts of horrible ways when a number in the + upper-half of the size_t range is passed in -- this number is + unrepresentable as an ssize_t, so code that checks to see how many + bytes are actually written (which is mandatory if you are dealing + with certain types of devices) will get completely screwed up. + + --ben +*/ + +typedef enum lstream_buffering +--------------------------------- snip ------------------------------------- +@end example + +@item +in @file{dumper.c}, there are four places, all inside of @code{switch()} statements, +where XD_BYTECOUNT appears twice as a case tag. In each case, the two +case blocks contain identical code, and you should *REMOVE THE SECOND* +and leave the first. +@end enumerate + +@node Text/Char Type Renaming, , Great Integral Type Renaming, Major Textual Changes +@subsection Text/Char Type Renaming +@cindex Text/Char Type Renaming +@cindex type renaming, text/char +@cindex renaming, text/char types + +The purpose of this was + +@enumerate +@item +To distinguish between ``charptr'' when it refers to operations on +the pointer itself and when it refers to operations on text +@item +To use consistent naming for everything referring to internal format, i.e. +@end enumerate + +@example + Itext == text in internal format + Ibyte == a byte in such text + Ichar == a char as represented in internal character format +@end example + +Thus e.g. + +@example + set_charptr_emchar -> set_itext_ichar +@end example + +This was done using a script like this: + +@example +files="*.[ch] s/*.h m/*.h config.h.in ../configure.in Makefile.in.in ../lib-src/*.[ch] ../lwlib/*.[ch]" +gr Intbyte Ibyte $files +gr INTBYTE IBYTE $files +gr intbyte ibyte $files +gr EMCHAR ICHAR $files +gr emchar ichar $files +gr Emchar Ichar $files +gr INC_CHARPTR INC_IBYTEPTR $files +gr DEC_CHARPTR DEC_IBYTEPTR $files +gr VALIDATE_CHARPTR VALIDATE_IBYTEPTR $files +gr valid_charptr valid_ibyteptr $files +gr CHARPTR ITEXT $files +gr charptr itext $files +gr Charptr Itext $files +@end example + +See above for the source to @samp{gr}. + +As in the integral-types change, there are pre and post tags before and +after the change: + +@example + pre-internal-format-textual-renaming + post-internal-format-textual-renaming +@end example + +When merging a large branch, follow the same sort of procedure +documented above, using these tags -- essentially sync up to the pre +tag, then apply the script yourself, then sync from the post tag to the +present. You can probably do the same if you don't have a separate +workspace, but do have lots of outstanding changes and you'd rather not +just merge all the textual changes directly. Use something like this: + +(WARNING: I'm not a CVS guru; before trying this, or any large operation +that might potentially mess things up, @strong{DEFINITELY} make a backup of +your existing workspace.) + +@example +cup -r pre-internal-format-textual-renaming +<apply script> +cup -A -j post-internal-format-textual-renaming -j HEAD +@end example + +This might also work: + +@example +cup -j pre-internal-format-textual-renaming +<apply script> +cup -j post-internal-format-textual-renaming -j HEAD +@end example + +ben + +The following is a script to go in the opposite direction: + +@example +files="*.[ch] s/*.h m/*.h config.h.in ../configure.in Makefile.in.in ../lib-src/*.[ch] ../lwlib/*.[ch]" + +# Evidently Perl considers _ to be a word char ala \b, even though XEmacs +# doesn't. We need to be careful here with ibyte/ichar because of words +# like Richard, @code{eicharlen()}, multibyte, HIBYTE, etc. + +gr Ibyte Intbyte $files +gr '\bIBYTE' INTBYTE $files +gr '\bibyte' intbyte $files +gr '\bICHAR' EMCHAR $files +gr '\bichar' emchar $files +gr '\bIchar' Emchar $files +gr '\bIBYTEPTR' CHARPTR $files +gr '\bibyteptr' charptr $files +gr '\bITEXT' CHARPTR $files +gr '\bitext' charptr $files +gr '\bItext' CHARPTR $files + +gr '_IBYTE' _INTBYTE $files +gr '_ibyte' _intbyte $files +gr '_ICHAR' _EMCHAR $files +gr '_ichar' _emchar $files +gr '_Ichar' _Emchar $files +gr '_IBYTEPTR' _CHARPTR $files +gr '_ibyteptr' _charptr $files +gr '_ITEXT' _CHARPTR $files +gr '_itext' _charptr $files +gr '_Itext' _CHARPTR $files +@end example + +@node Debugging and Testing, , Major Textual Changes, Rules When Writing New C Code +@section Debugging and Testing +@cindex debugging and testing + +@cindex Purify +@cindex Quantify +To make a purified XEmacs, do: @code{make puremacs}. +To make a quantified XEmacs, do: @code{make quantmacs}. + +You simply can't dump Quantified and Purified images (unless using the +portable dumper). Purify gets confused when xemacs frees memory in one +process that was allocated in a @emph{different} process on a different +machine! Run it like so: +@example +temacs -batch -l loadup.el run-temacs @var{xemacs-args...} +@end example + +@cindex error checking +Before you go through the trouble, are you compiling with all +debugging and error-checking off? If not, try that first. Be warned +that while Quantify is directly responsible for quite a few +optimizations which have been made to XEmacs, doing a run which +generates results which can be acted upon is not necessarily a trivial +task. + +Also, if you're still willing to do some runs make sure you configure +with the @samp{--quantify} flag. That will keep Quantify from starting +to record data until after the loadup is completed and will shut off +recording right before it shuts down (which generates enough bogus data +to throw most results off). It also enables three additional elisp +commands: @code{quantify-start-recording-data}, +@code{quantify-stop-recording-data} and @code{quantify-clear-data}. + +If you want to make XEmacs faster, target your favorite slow benchmark, +run a profiler like Quantify, @code{gprof}, or @code{tcov}, and figure +out where the cycles are going. In many cases you can localize the +problem (because a particular new feature or even a single patch +elicited it). Don't hesitate to use brute force techniques like a +global counter incremented at strategic places, especially in +combination with other performance indications (@emph{e.g.}, degree of +buffer fragmentation into extents). + +Specific projects: + +@itemize @bullet +@item +Make the garbage collector faster. Figure out how to write an +incremental garbage collector. +@item +Write a compiler that takes bytecode and spits out C code. +Unfortunately, you will then need a C compiler and a more fully +developed module system. +@item +Speed up redisplay. +@item +Speed up syntax highlighting. It was suggested that ``maybe moving some +of the syntax highlighting capabilities into C would make a +difference.'' Wrong idea, I think. When processing one 400kB file a +particular low-level routine was being called 40 @emph{million} times +simply for @emph{one} call to @code{newline-and-indent}. Syntax +highlighting needs to be rewritten to use a reliable, fast parser, then +to trust the pre-parsed structure, and only do re-highlighting locally +to a text change. Modern machines are fast enough to implement such +parsers in Lisp; but no machine will ever be fast enough to deal with +quadratic (or worse) algorithms! +@item +Implement tail recursion in Emacs Lisp (hard!). +@end itemize + +Unfortunately, Emacs Lisp is slow, and is going to stay slow. Function +calls in elisp are especially expensive. Iterating over a long list is +going to be 30 times faster implemented in C than in Elisp. + To get started debugging XEmacs, take a look at the @file{.gdbinit} and @file{.dbxrc} files in the @file{src} directory. See the section in the XEmacs FAQ on How to Debug an XEmacs problem with a debugger. @@ -5010,64 +5433,6 @@ configure --with-mule --use-union-type --error-checking=all @end example -Here are things to know when you create a new source file: - -@itemize @bullet -@item -All @file{.c} files should @code{#include <config.h>} first. Almost all -@file{.c} files should @code{#include "lisp.h"} second. - -@item -Generated header files should be included using the @samp{#include <...>} -syntax, not the @samp{#include "..."} syntax. The generated headers are: - -@file{config.h sheap-adjust.h paths.h Emacs.ad.h} - -The basic rule is that you should assume builds using @samp{--srcdir} -and the @samp{#include <...>} syntax needs to be used when the -to-be-included generated file is in a potentially different directory -@emph{at compile time}. The non-obvious C rule is that -@samp{#include "..."} means to search for the included file in the same -directory as the including file, @emph{not} in the current directory. -Normally this is not a problem but when building with @samp{--srcdir}, -@file{make} will search the @samp{VPATH} for you, while the C compiler -knows nothing about it. - -@item -Header files should @emph{not} include @samp{<config.h>} and -@samp{"lisp.h"}. It is the responsibility of the @file{.c} files that -use it to do so. - -@end itemize - -@cindex Lisp object types, creating -@cindex creating Lisp object types -@cindex object types, creating Lisp -Here is a checklist of things to do when creating a new lisp object type -named @var{foo}: - -@enumerate -@item -create @var{foo}.h -@item -create @var{foo}.c -@item -add definitions of @code{syms_of_@var{foo}}, etc. to @file{@var{foo}.c} -@item -add declarations of @code{syms_of_@var{foo}}, etc. to @file{symsinit.h} -@item -add calls to @code{syms_of_@var{foo}}, etc. to @file{emacs.c} -@item -add definitions of macros like @code{CHECK_@var{FOO}} and -@code{@var{FOO}P} to @file{@var{foo}.h} -@item -add the new type index to @code{enum lrecord_type} -@item -add a DEFINE_LRECORD_IMPLEMENTATION call to @file{@var{foo}.c} -@item -add an INIT_LRECORD_IMPLEMENTATION call to @code{syms_of_@var{foo}.c} -@end enumerate - @node Regression Testing XEmacs, CVS Techniques, Rules When Writing New C Code, Top @chapter Regression Testing XEmacs @cindex testing, regression @@ -5333,7 +5698,6 @@ @code{Check-Message}. The other files are test files, testing various XEmacs facilities. @xref{Regression Testing XEmacs}. - @node CVS Techniques, XEmacs from the Inside, Regression Testing XEmacs, Top @chapter CVS Techniques @cindex CVS techniques @@ -5424,7 +5788,7 @@ @end enumerate -@node XEmacs from the Inside, The XEmacs Object System (Abstractly Speaking), CVS Techniques, Top +@node XEmacs from the Inside, Basic Types, CVS Techniques, Top @chapter XEmacs from the Inside @cindex XEmacs from the inside @cindex inside, XEmacs from the @@ -5560,24 +5924,23 @@ doing this makes XEmacs run noticeably slower. Notice how much of the initialization is done in Lisp, not in C. -In general, XEmacs tries to move as much code as is possible -into Lisp. Code that remains in C is code that implements the -Lisp interpreter itself, or code that needs to be very fast, or -code that needs to do system calls or other such stuff that -needs to be done in C, or code that needs to have access to -``forbidden'' structures. (One conscious aspect of the design of -Lisp under XEmacs is a clean separation between the external -interface to a Lisp object's functionality and its internal -implementation. Part of this design is that Lisp programs -are forbidden from accessing the contents of the object other -than through using a standard API. In this respect, XEmacs Lisp -is similar to modern Lisp dialects but differs from GNU Emacs, -which tends to expose the implementation and allow Lisp -programs to look at it directly. The major advantage of -hiding the implementation is that it allows the implementation -to be redesigned without affecting any Lisp programs, including -those that might want to be ``clever'' by looking directly at -the object's contents and possibly manipulating them.) +In general, XEmacs tries to move as much code as is possible into +Lisp. Code that remains in C is code that implements the Lisp +interpreter itself, or code that needs to be very fast, or code that +needs to do system calls or other such stuff that needs to be done in +C, or code that needs to have access to ``forbidden'' structures. (One +conscious aspect of the design of Lisp under XEmacs is a clean +separation between the external interface to a Lisp object's +functionality and its internal implementation. Part of this design is +that Lisp programs are forbidden from accessing the contents of the +object other than through using a standard API. In this respect, +XEmacs Lisp is similar to modern Lisp dialects but differs from GNU +Emacs, which tends to expose the implementation and allow Lisp +programs to look at it directly. The major advantage of hiding the +implementation is that it allows the implementation to be redesigned +without affecting any Lisp programs, including those that might want +to be ``clever'' by looking directly at the object's contents and +possibly manipulating them.) Moving code into Lisp makes the code easier to debug and maintain and makes it much easier for people who are not XEmacs developers to @@ -5585,7 +5948,268 @@ of obscure and unwanted interactions occurring than if they were to change the C code. -@node The XEmacs Object System (Abstractly Speaking), How Lisp Objects Are Represented in C, XEmacs from the Inside, Top +@node Basic Types, Low-Level Allocation, XEmacs from the Inside, Top +@chapter Basic Types +@cindex basic types +@cindex types, basic + +Not yet documented. + +@node Low-Level Allocation, The XEmacs Object System (Abstractly Speaking), Basic Types, Top +@chapter Low-Level Allocation +@cindex low-level allocation +@cindex allocation, low-level + +@menu +* Basic Heap Allocation:: +* Stack Allocation:: +* Dynamic Arrays:: +* Allocation by Blocks:: +* Modules for Allocation:: +@end menu + +@node Basic Heap Allocation, Stack Allocation, Low-Level Allocation, Low-Level Allocation +@section Basic Heap Allocation +@cindex basic heap allocation + +@node Stack Allocation, Dynamic Arrays, Basic Heap Allocation, Low-Level Allocation +@section Stack Allocation +@cindex stack allocation + +@node Dynamic Arrays, Allocation by Blocks, Stack Allocation, Low-Level Allocation +@section Dynamic Arrays +@cindex dynamic arrays + +@cindex dynamic array +The @code{Dynarr} type implements a @dfn{dynamic array}, which is +similar to a standard C array but has no fixed limit on the number of +elements it can contain. Dynamic arrays can hold elements of any type, +and when you add a new element, the array automatically resizes itself +if it isn't big enough. Dynarrs are extensively used in the redisplay +mechanism. + + +A "dynamic array" is a contiguous array of fixed-size elements where there +is no upper limit (except available memory) on the number of elements in the +array. Because the elements are maintained contiguously, space is used +efficiently (no per-element pointers necessary) and random access to a +particular element is in constant time. At any one point, the block of memory +that holds the array has an upper limit; if this limit is exceeded, the +memory is realloc()ed into a new array that is twice as big. Assuming that +the time to grow the array is on the order of the new size of the array +block, this scheme has a provably constant amortized time (i.e. average +time over all additions). + +When you add elements or retrieve elements, pointers are used. Note that +the element itself (of whatever size it is), and not the pointer to it, +is stored in the array; thus you do not have to allocate any heap memory +on your own. Also, returned pointers are only guaranteed to be valid +until the next operation that changes the length of the array. + +This is a container object. Declare a dynamic array of a specific type +as follows: + + typedef struct + @{ + Dynarr_declare (mytype); + @} mytype_dynarr; + +Use the following functions/macros: + +@example +void *Dynarr_new(type) + [MACRO] Create a new dynamic-array object, with each element of the + specified type. The return value is cast to (type##_dynarr). + This requires following the convention that types are declared in + such a way that this type concatenation works. In particular, TYPE + must be a symbol, not an arbitrary C type. + +Dynarr_add(d, el) + [MACRO] Add an element to the end of a dynamic array. EL is a pointer + to the element; the element itself is stored in the array, however. + No function call is performed unless the array needs to be resized. + +Dynarr_add_many(d, base, len) + [MACRO] Add LEN elements to the end of the dynamic array. The elements + should be contiguous in memory, starting at BASE. If BASE if NULL, + just make space for the elements; don't actually add them. + +Dynarr_insert_many_at_start(d, base, len) + [MACRO] Append LEN elements to the beginning of the dynamic array. + The elements should be contiguous in memory, starting at BASE. + If BASE if NULL, just make space for the elements; don't actually + add them. + +Dynarr_insert_many(d, base, len, start) + Insert LEN elements to the dynamic array starting at position + START. The elements should be contiguous in memory, starting at BASE. + If BASE if NULL, just make space for the elements; don't actually + add them. + +Dynarr_delete(d, i) + [MACRO] Delete an element from the dynamic array at position I. + +Dynarr_delete_many(d, start, len) + Delete LEN elements from the dynamic array starting at position + START. + +Dynarr_delete_by_pointer(d, p) + [MACRO] Delete an element from the dynamic array at pointer P, + which must point within the block of memory that stores the data. + P should be obtained using Dynarr_atp(). + +int Dynarr_length(d) + [MACRO] Return the number of elements currently in a dynamic array. + +int Dynarr_largest(d) + [MACRO] Return the maximum value that Dynarr_length(d) would + ever have returned. + +type Dynarr_at(d, i) + [MACRO] Return the element at the specified index (no bounds checking + done on the index). The element itself is returned, not a pointer + to it. + +type *Dynarr_atp(d, i) + [MACRO] Return a pointer to the element at the specified index (no + bounds checking done on the index). The pointer may not be valid + after an element is added to or removed from the array. + +Dynarr_reset(d) + [MACRO] Reset the length of a dynamic array to 0. + +Dynarr_free(d) + Destroy a dynamic array and the memory allocated to it. +@end example + +Use the following global variable: + +@example + Dynarr_min_size + Minimum allowable size for a dynamic array when it is resized. +@end example + +@node Allocation by Blocks, Modules for Allocation, Dynamic Arrays, Low-Level Allocation +@section Allocation by Blocks +@cindex allocation by blocks + + The @code{Blocktype} type efficiently manages the +allocation of fixed-size blocks by minimizing the number of times that +@code{malloc()} and @code{free()} are called. It allocates memory in +large chunks, subdivides the chunks into blocks of the proper size, and +returns the blocks as requested. When blocks are freed, they are placed +onto a linked list, so they can be efficiently reused. This data type +is not much used in XEmacs currently, because it's a fairly new +addition. + + +A "block-type object" is used to efficiently allocate and free blocks +of a particular size. Freed blocks are remembered in a free list and +are reused as necessary to allocate new blocks, so as to avoid as +much as possible making calls to malloc() and free(). + +This is a container object. Declare a block-type object of a specific type +as follows: + +struct mytype_blocktype @{ + Blocktype_declare (mytype); +@}; + +Use the following functions/macros: + +@example +structype *Blocktype_new(structype) + [MACRO] Create a new block-type object of the specified type. + The argument to this call should be the type of object to be + created, e.g. foobar_blocktype. +type *Blocktype_alloc(b) + [MACRO] Allocate a block of the proper type for the specified + block-type object and return a pointer to it. +Blocktype_free(b, block) + Free a block of the type corresponding to the specified block-type + object. +Blocktype_delete(b) + Destroy a block-type object and the memory allocated to it. +@end example + + +@node Modules for Allocation, , Allocation by Blocks, Low-Level Allocation +@section Modules for Allocation +@cindex modules for allocation + +@example +@file{alloca.c} +@file{free-hook.c} +@file{getpagesize.h} +@file{gmalloc.c} +@file{malloc.c} +@file{mem-limits.h} +@file{ralloc.c} +@file{vm-limit.c} +@end example + +These handle basic C allocation of memory. @file{alloca.c} is an emulation of +the stack allocation function @code{alloca()} on machines that lack +this. (XEmacs makes extensive use of @code{alloca()} in its code.) + +@file{gmalloc.c} and @file{malloc.c} are two implementations of the standard C +functions @code{malloc()}, @code{realloc()} and @code{free()}. They are +often used in place of the standard system-provided @code{malloc()} +because they usually provide a much faster implementation, at the +expense of additional memory use. @file{gmalloc.c} is a newer implementation +that is much more memory-efficient for large allocations than @file{malloc.c}, +and should always be preferred if it works. (At one point, @file{gmalloc.c} +didn't work on some systems where @file{malloc.c} worked; but this should be +fixed now.) + +@cindex relocating allocator +@file{ralloc.c} is the @dfn{relocating allocator}. It provides +functions similar to @code{malloc()}, @code{realloc()} and @code{free()} +that allocate memory that can be dynamically relocated in memory. The +advantage of this is that allocated memory can be shuffled around to +place all the free memory at the end of the heap, and the heap can then +be shrunk, releasing the memory back to the operating system. The use +of this can be controlled with the configure option @code{--rel-alloc}; +if enabled, memory allocated for buffers will be relocatable, so that if +a very large file is visited and the buffer is later killed, the memory +can be released to the operating system. (The disadvantage of this +mechanism is that it can be very slow. On systems with the +@code{mmap()} system call, the XEmacs version of @file{ralloc.c} uses +this to move memory around without actually having to block-copy it, +which can speed things up; but it can still cause noticeable performance +degradation.) + +On Linux systems using @samp{glibc 2}, these strategies are built in to +the so-called ``Doug Lea malloc.'' See, for example, Doug Lea's home +page, especially @uref{http://gee.cs.oswego.edu/dl/html/malloc.html,``A +Memory Allocator''}. The source file, @file{malloc.c} (available at the +same place) is copiously (and usefully!) commented. +@uref{http://www.malloc.de/,Wolfram Gloger's home page} may also be +useful. + +@file{free-hook.c} contains some debugging functions for checking for invalid +arguments to @code{free()}. + +@file{vm-limit.c} contains some functions that warn the user when memory is +getting low. These are callback functions that are called by @file{gmalloc.c} +and @file{malloc.c} at appropriate times. + +@file{getpagesize.h} provides a uniform interface for retrieving the size of a +page in virtual memory. @file{mem-limits.h} provides a uniform interface for +retrieving the total amount of available virtual memory. Both are +similar in spirit to the @file{sys*.h} files described in section J, below. + + +@example +@file{blocktype.c} +@file{blocktype.h} +@file{dynarr.c} +@end example + +These implement a couple of basic C data types to facilitate memory +allocation. + +@node The XEmacs Object System (Abstractly Speaking), How Lisp Objects Are Represented in C, Low-Level Allocation, Top @chapter The XEmacs Object System (Abstractly Speaking) @cindex XEmacs object system (abstractly speaking), the @cindex object system (abstractly speaking), the XEmacs @@ -12861,6 +13485,9 @@ @cindex redisplay sections, critical @cindex critical redisplay sections + +@strong{The following paragraphs are way out-of-date and inaccurate.} + Within this section, we are defenseless and assume that the following cannot happen: @@ -12889,6 +13516,319 @@ #### If a frame-size change does occur we should probably actually be preempting redisplay. +@strong{Begin up-to-date stuff} + +@subsection Nasty Bugs due to Reentrancy in Redisplay Structures handling QUIT + +These are now fixed as of November 10, 2004. + +@subheading Crash -- reentrant @code{regenerate_window()} + +Here is a crash I (ben) just got -- November 9, 2004: +It can sort of be reproduced by creating a bunch of frames, opening a bunch of +large files (which may be fontlocking for awhile). and immediately start +Alt-TAB-ing back and forth quickly and constantly scrolling up and down using +the scrolling dial on your mouse. + +@example + +Fatal error: assertion failed, file c:\xemacs\build\src\redisplay.c, line 5532, +!dy->locked + +C backtrace: + +assert_failed(const char * 0x012a4ff0 `string', int 5532, const char * 0x0127bea4 `string') line 3839 +Dynarr_verify_mod_1(void * 0x023ad2b0, const char * 0x012a4ff0 `string', int 5532) line 1306 + 36 bytes +regenerate_window(window * 0x02f2ca88, long 40372, long 40372, int 2) line 5532 + 25 bytes +update_line_start_cache(window * 0x02f2ca88, long 40372, long 40401, long 40372, int 0) line 8543 + 19 bytes +point_in_line_start_cache(window * 0x02f2ca88, long 40372, int 0) line 7850 + 23 bytes +start_end_of_last_line(window * 0x02f2ca88, long 40372, int 1, int 1) line 8121 + 15 bytes +end_of_last_line_may_error(window * 0x02f2ca88, long 40372) line 8203 + 17 bytes +pixel_to_glyph_translation(frame * 0x02f2c900, int 291, int 317, int * 0x0082bb04, int * 0x0082bb00, int * 0x0082bafc, int * 0x0082baf8, window * * 0x0082bae8, long * 0x0082baf4, long * 0x0082baf0, long * 0x0082baec, long * 0x0082bb10, long * 0x0082bb0c) line 9336 + 32 bytes +mswindows_handle_mousewheel_event(long 49465600, int 0, int -240, tagPOINTS @{...@}) line 360 + 82 bytes +mswindows_wnd_proc(HWND__ * 0x00260a42, unsigned int 522, unsigned int 4279238656, long 29885130) line 3561 + 36 bytes +intercepted_wnd_proc(HWND__ * 0x00260a42, unsigned int 522, unsigned int 4279238656, long 29885130) line 2376 +USER32! 77e11ef0() +USER32! 77e1204c() +USER32! 77e121af() +mswindows_drain_windows_queue(int 0) line 1330 + 9 bytes +emacs_mswindows_drain_queue() line 1339 + 7 bytes +event_stream_drain_queue() line 1785 +event_stream_quit_p() line 1893 +check_quit() line 938 +check_what_happened() line 459 +internal_equal(long 22180468, long 22180468, int 0) line 2823 + 14 bytes +update_image_instance(long 83498640, long 22180468) line 2121 + 18 bytes +image_instantiate(long 21418616, long 20663624, long 54932896, long 22180468, long 3) line 3403 + 13 bytes +va_call_trapping_problems_1(void * 0x0082cf94) line 5220 + 221 bytes +call_trapping_problems_2(long 83160440) line 4867 + 13 bytes +call_with_condition_handler(long (long, long, long)* 0x010cc4c0 flagged_a_squirmer(long, long, long), long 83160440, long (long)* 0x010cc440 call_trapping_problems_2(long), long 83160440) line 2129 + 7 bytes +call_trapping_problems_1(long 83160440) line 4874 + 23 bytes +internal_catch(long 21399864, long (long)* 0x010cc490 call_trapping_problems_1(long), long 83160440, int * volatile 0x0082ce4c, long * volatile 0x0082ce54) line 1527 + 7 bytes +call_trapping_problems(long 20908160, const char * 0x00000000, int 98315, call_trapping_problems_result * 0x00000000, long (void *)* 0x010cca30 va_call_trapping_problems_1(void *), void * 0x0082cf94) line 5147 + 32 bytes +call_with_suspended_errors(long (void)* 0x011448c0 image_instantiate(long, long, long, long, long), long 20663624, long 20908160, _error_behavior_struct_ @{...@}, int 5) line 5314 + 26 bytes +specifier_instance_from_inst_list(long 21418616, long 20663624, long 54932896, long 21673760, _error_behavior_struct_ @{...@}, int 1, long 3) line 2501 + 54 bytes +specifier_instance(long 21418616, long 20663624, long 54932896, _error_behavior_struct_ @{...@}, int 1, int 0, long 3) line 2614 + 64 bytes +glyph_image_instance(long 22692176, long 54932896, _error_behavior_struct_ @{...@}, int 1) line 3955 + 31 bytes +add_glyph_rune(position_redisplay_data_type * 0x0082d52c, glyph_block * 0x0082d454, int 0, int 0, glyph_cachel * 0x04f4e518) line 1972 + 26 bytes +create_text_block(window * 0x034635a0, display_line * 0x033bfb28, long 29860, prop_block_dynarr * * 0x0082d7b8, int 2) line 2827 + 30 bytes +generate_display_line(window * 0x034635a0, display_line * 0x033bfb28, int 1, long 29860, prop_block_dynarr * * 0x0082d7b8, int 2) line 979 + 38 bytes +regenerate_window(window * 0x034635a0, long 29860, long 25012, int 2) line 5607 + 30 bytes +update_line_start_cache(window * 0x034635a0, long 25012, long 28767, long 25012, int 0) line 8614 + 19 bytes +point_in_line_start_cache(window * 0x034635a0, long 25012, int 0) line 7850 + 23 bytes +start_end_of_last_line(window * 0x034635a0, long 25012, int 1, int 0) line 8121 + 15 bytes +end_of_last_line(window * 0x034635a0, long 25012) line 8197 + 17 bytes +Fwindow_end(long 54932896, long 20926544) line 1848 + 13 bytes +Ffuncall(int 3, long * 0x0082dbb8) line 3841 + 93 bytes +execute_optimized_program(const unsigned char * 0x032ceee8, int 7, long * 0x03289f40) line 823 + 16 bytes +funcall_compiled_function(long 52991916, int 1, long * 0x0082dfb0) line 3454 + 85 bytes +Ffuncall(int 2, long * 0x0082dfac) line 3880 + 17 bytes +execute_optimized_program(const unsigned char * 0x02f667d8, int 6, long * 0x01558748) line 823 + 16 bytes +funcall_compiled_function(long 22579576, int 3, long * 0x0082e3ac) line 3454 + 85 bytes +Ffuncall(int 4, long * 0x0082e3a8) line 3880 + 17 bytes +execute_optimized_program(const unsigned char * 0x03209c98, int 5, long * 0x03288c68) line 823 + 16 bytes +funcall_compiled_function(long 51656320, int 1, long * 0x0082e7a4) line 3454 + 85 bytes +Ffuncall(int 2, long * 0x0082e7a0) line 3880 + 17 bytes +execute_optimized_program(const unsigned char * 0x0082e9ec, int 4, long * 0x03224990) line 823 + 16 bytes +Fbyte_code(long 37927380, long 52578688, long 9) line 2564 + 70 bytes +Feval(long 51505420) line 3601 + 187 bytes +internal_catch(long 51959412, long (long)* 0x010c6f40 Feval(long), long 51505420, int * volatile 0x00000000, long * volatile 0x00000000) line 1527 + 7 bytes +execute_rare_opcode(long * 0x0082eee8, const unsigned char * 0x03248365, Opcode Bcatch) line 1380 + 24 bytes +execute_optimized_program(const unsigned char * 0x03248340, int 2, long * 0x02f3c0a0) line 715 + 17 bytes +funcall_compiled_function(long 51656276, int 0, long * 0x0082f444) line 3454 + 85 bytes +Ffuncall(int 1, long * 0x0082f440) line 3880 + 17 bytes +run_hook_with_args_in_buffer(buffer * 0x04ee9060, int 1, long * 0x0082f440, run_hooks_condition RUN_HOOKS_TO_COMPLETION) line 4361 + 13 bytes +run_hook_with_args(int 1, long * 0x0082f440, run_hooks_condition RUN_HOOKS_TO_COMPLETION) line 4374 + 23 bytes +run_hook(long 51959028) line 4443 + 13 bytes +safe_run_hook_trapping_problems_1(void * 0x013c73c0) line 5517 + 9 bytes +call_trapping_problems_2(long 83157920) line 4867 + 13 bytes +call_with_condition_handler(long (long, long, long)* 0x010cc4c0 flagged_a_squirmer(long, long, long), long 83157920, long (long)* 0x010cc440 call_trapping_problems_2(long), long 83157920) line 2129 + 7 bytes +call_trapping_problems_1(long 83157920) line 4874 + 23 bytes +internal_catch(long 21399864, long (long)* 0x010cc490 call_trapping_problems_1(long), long 83157920, int * volatile 0x0082f700, long * volatile 0x0082f708) line 1527 + 7 bytes +call_trapping_problems(long 20925944, const char * 0x00000000, int 131235, call_trapping_problems_result * 0x0082f830, long (void *)* 0x010cd990 safe_run_hook_trapping_problems_1(void *), void * 0x013c73c0) line 5147 + 32 bytes +safe_run_hook_trapping_problems(long 20741312, long 20739008, int 160) line 5543 + 36 bytes +run_pre_idle_hook() line 2084 + 24 bytes +redisplay() line 7224 +Fnext_event(long 37363732, long 20928056) line 2263 +Fcommand_loop_1() line 600 + 15 bytes +command_loop_1(long 20928056) line 512 +condition_case_1(long 20925944, long (long)* 0x01096a80 command_loop_1(long), long 20928056, long (long, long)* 0x01096630 cmd_error(long, long), long 20928056) line 1918 + 7 bytes +command_loop_3() line 262 + 35 bytes +command_loop_2(long 20928056) line 277 +internal_catch(long 20683712, long (long)* 0x010967a0 command_loop_2(long), long 20928056, int * volatile 0x00000000, long * volatile 0x00000000) line 1527 + 7 bytes +initial_command_loop(long 20928056) line 313 + 28 bytes +xemacs_21_5_b18_i586_pc_win32(int 1, unsigned short * * 0x0082fed0, unsigned short * * 0x00000000, int 0) line 2551 +main(int 1, char * * 0x00e52610, char * * 0x00e52bb0) line 2992 +mainCRTStartup() line 338 + 17 bytes +KERNEL32! 7c59893d() + +Lisp backtrace: + + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (catch #<INTERNAL OBJECT (XEmacs bug?) (opaque, size=0) 0x1468938> ...) + # (unwind-protect ...) + # bind (inhibit-quit) + window-end(#<window on "signal.c<2>" 0x5e4a> t) + # (unwind-protect ...) + # bind (buffer we-are-screwed check-text-props window) + lazy-lock-fontify-window(#<window on "signal.c<2>" 0x5e4a>) + # bind (walk-windows-current walk-windows-start arg which-devices which-frames + minibuf function) + walk-windows(lazy-lock-fontify-window no-minibuf #<mswindows-frame "emacs" 0x5 +e49>) + # (unwind-protect ...) + # bind (ssf65112 tick frame) + lazy-lock-maybe-fontify-frame(#<mswindows-frame "emacs" 0x5e49>) + # bind (frame starting-frame) + byte-code("..." [starting-frame frame selected-frame frame-visible-p frame-min +ibuffer-only-p next-frame visible-nomini throw lazy-lock-frame-loop-done t lazy- +lock-maybe-fontify-frame] 4) + # (catch lazy-lock-frame-loop-done ...) + lazy-lock-pre-idle-fontify-windows() + # (unwind-protect ...) + # (catch #<INTERNAL OBJECT (XEmacs bug?) (opaque, size=0) 0x1468938> ...) + # (unwind-protect ...) + # (unwind-protect ...) + # bind (inhibit-quit) + # (unwind-protect ...) + # (unwind-protect ...) + # bind (inhibit-quit) + (next-event "[internal]") + # (condition-case ... . error) + # (catch top-level ...) +@end example + +@subsubheading Another Lisp trace of a similar situation (C stack trace not available): + +@example +Fatal error: assertion failed, file c:\xemacs\build\src\redisplay.c, line 5532, +!dy->locked + +Lisp backtrace follows: + + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (unwind-protect ...) + scrollbar-page-down((#<window on "*grep*" 0x1a5f9>)) + (dispatch-event "[internal]") + # (unwind-protect ...) + # (catch #<INTERNAL OBJECT (XEmacs bug?) (opaque, size=0) 0x1468270> ...) + # (unwind-protect ...) + # (unwind-protect ...) + # (catch #<INTERNAL OBJECT (XEmacs bug?) (opaque, size=0) 0x1468270> ...) + # (unwind-protect ...) + # bind (inhibit-quit) + window-end(#<window on "*grep*" 0x1a5f9> t) + # (unwind-protect ...) + # bind (buffer we-are-screwed check-text-props window) + lazy-lock-fontify-window(#<window on "*grep*" 0x1a5f9>) + # bind (walk-windows-current walk-windows-start arg which-devices which-frames + minibuf function) + walk-windows(lazy-lock-fontify-window no-minibuf #<mswindows-frame "emacs" 0x1 +9f64>) + # (unwind-protect ...) + # bind (ssf65112 tick frame) + lazy-lock-maybe-fontify-frame(#<mswindows-frame "emacs" 0x19f64>) + # bind (frame starting-frame) + byte-code("..." [starting-frame frame selected-frame frame-visible-p frame-min +ibuffer-only-p next-frame visible-nomini throw lazy-lock-frame-loop-done t lazy- +lock-maybe-fontify-frame] 4) + # (catch lazy-lock-frame-loop-done ...) + lazy-lock-pre-idle-fontify-windows() + # (unwind-protect ...) + # (catch #<INTERNAL OBJECT (XEmacs bug?) (opaque, size=0) 0x1468270> ...) + # (unwind-protect ...) + # (unwind-protect ...) + # bind (inhibit-quit) + # (unwind-protect ...) + # (unwind-protect ...) + # bind (inhibit-quit) + (next-event "[internal]") + # (condition-case ... . error) + # (catch top-level ...) +@end example + +@subheading Crash -- reentrant @code{generate_displayable_area()} + +Original code said [Tricky tricky tricky. +@code{generate_displayable_area()} can (could) be called reentrantly, +and redisplay is not prepared to handle this:]. + +assert_failed(const char * 0x0129c8c8 `string', int 5328, const char * 0x01274068 `string') line 3620 +Dynarr_verify_mod_1(void * 0x0250f228, const char * 0x0129c8c8 `string', int 5328) line 1256 + 36 bytes +generate_displayable_area(window * 0x02480028, long 38776292, int 0, int 0, int 265, int 169, display_line_dynarr * 0x0250f228, long 0, int 2) line 5328 + 25 bytes +output_gutter(frame * 0x0228ad90, gutter_pos TOP_GUTTER, int 1) line 409 + 69 bytes +redraw_exposed_gutter(frame * 0x0228ad90, gutter_pos TOP_GUTTER, int 8, int 23, int 249, int 127) line 687 + 15 bytes +redraw_exposed_gutters(frame * 0x0228ad90, int 8, int 23, int 249, int 127) line 703 + 29 bytes +mswindows_redraw_exposed_area(frame * 0x0228ad90, int 8, int 23, int 249, int 127) line 862 + 25 bytes +mswindows_handle_paint(frame * 0x0228ad90) line 2176 + 25 bytes +mswindows_wnd_proc(HWND__ * 0x001003e2, unsigned int 15, unsigned int 0, long 0) line 3233 + 45 bytes +intercepted_wnd_proc(HWND__ * 0x001003e2, unsigned int 15, unsigned int 0, long 0) line 2488 +USER32! 77e3a244() +USER32! 77e14730() +USER32! 77e1558a() +NTDLL! KiUserCallbackDispatcher@@12 + 19 bytes +USER32! 77e14680() +USER32! 77e1a792() +qxeIsDialogMessage(HWND__ * 0x001003e2, tagMSG * 0x0082a93c @{msg=0x0000000f wp=0x00000000 lp=0x00000000@}) line 2298 + 14 bytes +mswindows_is_dialog_msg(tagMSG * 0x0082a93c @{msg=0x0000000f wp=0x00000000 lp=0x00000000@}) line 165 + 13 bytes +mswindows_drain_windows_queue(int 0) line 1282 + 9 bytes +emacs_mswindows_drain_queue() line 1326 + 7 bytes +event_stream_drain_queue() line 1887 +event_stream_quit_p() line 1992 +check_quit() line 993 +unbind_to_hairy(int 35) line 5963 +unbind_to_1(int 35, long 20888208) line 5945 + 200 bytes +specifier_instance_from_inst_list(long 21379344, long 38135616, long 36220304, long 20888208, _error_behavior_struct_ @{...@}, int 1, long 3) line 2522 + 16 bytes +specifier_instance(long 21379344, long 38135616, long 36220304, _error_behavior_struct_ @{...@}, int 1, int 0, long 3) line 2625 + 65 bytes +specifier_instance_no_quit(long 21379344, long 38135616, long 36220304, _error_behavior_struct_ @{...@}, int 0, long 1) line 2658 + 31 bytes +face_property_matching_instance(long 22612340, long 20860632, long 22530956, long 36220304, _error_behavior_struct_ @{...@}, int 0, long 1) line 565 + 48 bytes +ensure_face_cachel_contains_charset(face_cachel * 0x0082b014, long 36220304, long 22530956) line 1104 + 35 bytes +update_face_cachel_data(face_cachel * 0x0082b014, long 36220304, long 22612340) line 1304 + 19 bytes +query_string_geometry(long 21110576, long 22612340, int * 0x00000000, int * 0x0082b5b4, int * 0x00000000, long 38852960) line 2370 + 23 bytes +mswindows_widget_query_string_geometry(long 21110576, long 22612340, int * 0x0082b5b8, int * 0x0082b5b4, long 38852960) line 2914 + 25 bytes +widget_query_string_geometry(long 21110576, long 22612340, int * 0x0082b5b8, int * 0x0082b5b4, long 38852960) line 514 + 32 bytes +edit_field_query_geometry(long 38857648, int * 0x0082b7b4, int * 0x0082b7b8, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38852960) line 920 + 390 bytes +widget_query_geometry(long 38857648, int * 0x0082b7b4, int * 0x0082b7b8, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38852960) line 567 + 26 bytes +image_instance_query_geometry(long 38857648, int * 0x0082b7b4, int * 0x0082b7b8, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38852960) line 2015 + 26 bytes +glyph_query_geometry(long 38853384, int * 0x0082b7b4, int * 0x0082b7b8, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38852960) line 4197 + 25 bytes +layout_query_geometry(long 38852960, int * 0x0082b9cc, int * 0x0082b9d0, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38404624) line 1351 + 25 bytes +widget_query_geometry(long 38852960, int * 0x0082b9cc, int * 0x0082b9d0, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38404624) line 567 + 26 bytes +image_instance_query_geometry(long 38852960, int * 0x0082b9cc, int * 0x0082b9d0, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38404624) line 2015 + 26 bytes +glyph_query_geometry(long 38537976, int * 0x0082b9cc, int * 0x0082b9d0, image_instance_geometry IMAGE_DESIRED_GEOMETRY, long 38404624) line 4197 + 25 bytes +layout_layout(long 38404624, int 265, int 156, int -2, int -2, long 38273064) line 1468 + 23 bytes +widget_layout(long 38404624, int 265, int 156, int -2, int -2, long 38273064) line 626 + 30 bytes +image_instance_layout(long 38404624, int 265, int 156, int -2, int -2, long 38273064) line 2102 + 51 bytes +glyph_ascent(long 38404624, long 38273064) line 4009 + 21 bytes +update_glyph_cachel_data(window * 0x02480028, long 36201168, glyph_cachel * 0x0248c3d8) line 4272 + 13 bytes +get_glyph_cachel_index(window * 0x02480028, long 36201168) line 4306 + 17 bytes +add_glyph_rune(position_redisplay_data_type * 0x0082bf2c, glyph_block * 0x024bd028, int 0, int 0, glyph_cachel * 0x00000000) line 1800 + 15 bytes +add_glyph_runes(position_redisplay_data_type * 0x0082bf2c, int 0) line 2085 + 31 bytes +create_string_text_block(window * 0x02480028, long 38776292, display_line * 0x02514500, long 0, prop_block_dynarr * * 0x0082c13c, int 2) line 4907 + 14 bytes +generate_string_display_line(window * 0x02480028, long 38776292, display_line * 0x02514500, long 0, prop_block_dynarr * * 0x0082c13c, int 2) line 5293 + 29 bytes +generate_displayable_area(window * 0x02480028, long 38776292, int 0, int 0, int 265, int 169, display_line_dynarr * 0x0250f228, long 0, int 2) line 5372 + 29 bytes +output_gutter(frame * 0x0228ad90, gutter_pos TOP_GUTTER, int 0) line 409 + 69 bytes +update_frame_gutters(frame * 0x0228ad90) line 639 + 15 bytes +redisplay_frame(frame * 0x0228ad90, int 1) line 6792 + 9 bytes +redisplay_device(device * 0x0171df00, int 1) line 6911 + 11 bytes +redisplay_without_hooks() line 6957 + 11 bytes +redisplay_no_pre_idle_hook() line 7029 +redisplay() line 7011 +mswindows_wnd_proc(HWND__ * 0x001003e2, unsigned int 5, unsigned int 0, long 10223881) line 3424 +intercepted_wnd_proc(HWND__ * 0x001003e2, unsigned int 5, unsigned int 0, long 10223881) line 2488 +USER32! 77e3a244() +USER32! 77e16362() +USER32! 77e14c1a() +USER32! 77e1dd30() +mswindows_wnd_proc(HWND__ * 0x001003e2, unsigned int 71, unsigned int 0, long 8578308) line 3926 + 21 bytes +intercepted_wnd_proc(HWND__ * 0x001003e2, unsigned int 71, unsigned int 0, long 8578308) line 2488 +USER32! 77e3a244() +USER32! 77e14730() +USER32! 77e174b4() +NTDLL! KiUserCallbackDispatcher@@12 + 19 bytes +mswindows_set_frame_size(frame * 0x0228ad90, int 265, int 156) line 355 +internal_set_frame_size(frame * 0x0228ad90, int 265, int 156, int 0) line 2754 + 24 bytes +Fset_frame_displayable_pixel_size(long 36220304, long 531, long 313, long 20888208) line 3004 + 32 bytes +Ffuncall(int 4, long * 0x0082e778) line 3844 + 168 bytes +execute_optimized_program(const unsigned char * 0x02286e48, int 40, long * 0x01529b80) line 609 + 16 bytes +funcall_compiled_function(long 22433308, int 0, long * 0x0082ec08) line 3452 + 85 bytes +Ffuncall(int 1, long * 0x0082ec04) line 3883 + 17 bytes +execute_optimized_program(const unsigned char * 0x02286d40, int 6, long * 0x01548ddc) line 609 + 16 bytes +funcall_compiled_function(long 22505864, int 11, long * 0x0082f00c) line 3452 + 85 bytes +Ffuncall(int 12, long * 0x0082f008) line 3883 + 17 bytes +execute_optimized_program(const unsigned char * 0x02503e38, int 47, long * 0x0152dc48) line 609 + 16 bytes +funcall_compiled_function(long 22436784, int 0, long * 0x0082f534) line 3452 + 85 bytes +Ffuncall(int 1, long * 0x0082f530) line 3883 + 17 bytes +apply1(long 22436784, long 20888208) line 4458 + 11 bytes +Fcall_interactively(long 20742816, long 20888208, long 20888208) line 460 + 13 bytes +Ffuncall(int 2, long * 0x0082f8ec) line 3844 + 127 bytes +call1(long 20854392, long 20742816) line 4489 + 11 bytes +execute_command_event(command_builder * 0x01798f98, long 24439276) line 4198 + 69 bytes +Fdispatch_event(long 24439276) line 4569 + 13 bytes +Fcommand_loop_1() line 569 + 9 bytes +command_loop_1(long 20888208) line 489 +condition_case_1(long 20886024, long (long)* 0x010955a0 command_loop_1(long), long 20888208, long (long, long)* 0x01095150 cmd_error(long, long), long 20888208) line 1917 + 7 bytes +command_loop_3() line 251 + 35 bytes +command_loop_2(long 20888208) line 264 +internal_catch(long 20650992, long (long)* 0x010952c0 command_loop_2(long), long 20888208, int * volatile 0x00000000, long * volatile 0x00000000) line 1527 + 7 bytes +initial_command_loop(long 20888208) line 300 + 28 bytes +xemacs_21_5_b10_i586_pc_win32(int 1, char * * 0x00e52620, char * * 0x00e52bb0, int 0) line 2356 +main(int 1, char * * 0x00e52620, char * * 0x00e52bb0) line 2733 +mainCRTStartup() line 338 + 17 bytes +KERNEL32! 77ea847c() + @node Line Start Cache, Redisplay Piece by Piece, Critical Redisplay Sections, The Redisplay Mechanism @section Line Start Cache @cindex line start cache @@ -14688,6 +15628,171 @@ anywhere else, because it's blocked and will never signal. @end enumerate +@subsection Reentrancy Problems due to QUIT Checking + +Checking for QUIT can do quite a long of things -- since it pumps the +event loop, this may cause arbitrary code to get executed, garbage collection +to happen. etc. (In fact, garbage collection cannot happen because it is inhibited.) This has led to crashes when functions get called reentrantly when not expecting it. Example: + +@subheading Crash -- reentrant @code{re_match_2()} + +@example + /* dont_check_for_quit is set in three circumstances: + + (1) when we are in the process of changing the window + configuration. The frame might be in an inconsistent state, + which will cause assertion failures if we check for QUIT. + + (2) when we are reading events, and want to read the C-g + as an event. The normal check for quit will discard the C-g, + which would be bad. + + (3) when we're going down with a fatal error. we're most likely + in an inconsistent state, and we definitely don't want to be + interrupted. */ + + /* We should *not* conditionalize on Vinhibit_quit, or + critical-quit (Control-Shift-G) won't work right. */ + + /* WARNING: Even calling check_quit(), without actually dispatching + a quit signal, can result in arbitrary Lisp code getting executed + -- at least under Windows. (Not to mention obvious Lisp + invocations like asynchronous timer callbacks.) Here's a sample + stack trace to demonstrate: + + NTDLL! DbgBreakPoint@@0 address 0x77f9eea9 +assert_failed(const char * 0x012d036c, int 4596, const char * 0x012d0354) line 3478 +re_match_2_internal(re_pattern_buffer * 0x012d6780, const unsigned char * 0x00000000, int 0, const unsigned char * 0x022f9328, int 34, int 0, re_registers * 0x012d53d0 search_regs, int 34) line 4596 + 41 bytes +re_search_2(re_pattern_buffer * 0x012d6780, const char * 0x00000000, int 0, const char * 0x022f9328, int 34, int 0, int 34, re_registers * 0x012d53d0 search_regs, int 34) line 4269 + 37 bytes +re_search(re_pattern_buffer * 0x012d6780, const char * 0x022f9328, int 34, int 0, int 34, re_registers * 0x012d53d0 search_regs) line 4031 + 37 bytes +string_match_1(long 31222628, long 30282164, long 28377092, buffer * 0x022fde00, int 0) line 413 + 69 bytes +Fstring_match(long 31222628, long 30282164, long 28377092, long 28377092) line 436 + 34 bytes +Ffuncall(int 3, long * 0x008297f8) line 3488 + 168 bytes +execute_optimized_program(const unsigned char * 0x020ddc50, int 6, long * 0x020ddf50) line 744 + 16 bytes +funcall_compiled_function(long 34407748, int 1, long * 0x00829aec) line 516 + 53 bytes +Ffuncall(int 2, long * 0x00829ae8) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x020ddc90, int 4, long * 0x020ddf90) line 744 + 16 bytes +funcall_compiled_function(long 34407720, int 1, long * 0x00829e28) line 516 + 53 bytes +Ffuncall(int 2, long * 0x00829e24) line 3523 + 17 bytes +mapcar1(long 15, long * 0x00829e48, long 34447820, long 34187868) line 2929 + 11 bytes +Fmapcar(long 34447820, long 34187868) line 3035 + 21 bytes +Ffuncall(int 3, long * 0x00829f20) line 3488 + 93 bytes +execute_optimized_program(const unsigned char * 0x020c2b70, int 7, long * 0x020dd010) line 744 + 16 bytes +funcall_compiled_function(long 34407580, int 2, long * 0x0082a210) line 516 + 53 bytes +Ffuncall(int 3, long * 0x0082a20c) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x020cf810, int 6, long * 0x020cfb10) line 744 + 16 bytes +funcall_compiled_function(long 34407524, int 0, long * 0x0082a580) line 516 + 53 bytes +Ffuncall(int 1, long * 0x0082a57c) line 3523 + 17 bytes +run_hook_with_args_in_buffer(buffer * 0x022fde00, int 1, long * 0x0082a57c, int 0) line 3980 + 13 bytes +run_hook_with_args(int 1, long * 0x0082a57c, int 0) line 3993 + 23 bytes +Frun_hooks(int 1, long * 0x0082a57c) line 3847 + 19 bytes +run_hook(long 34447484) line 4094 + 11 bytes +unsafe_handle_wm_initmenu_1(frame * 0x01dbb000) line 736 + 11 bytes +unsafe_handle_wm_initmenu(long 28377092) line 807 + 11 bytes +condition_case_1(long 28377116, long (long)* 0x0101c827 unsafe_handle_wm_initmenu(long), long 28377092, long (long, long)* 0x01005fa4 mswindows_modal_loop_error_handler(long, long), long 28377092) line 1692 + 7 bytes +mswindows_protect_modal_loop(long (long)* 0x0101c827 unsafe_handle_wm_initmenu(long), long 28377092) line 1194 + 32 bytes +mswindows_handle_wm_initmenu(HMENU__ * 0x00010199, frame * 0x01dbb000) line 826 + 17 bytes +mswindows_wnd_proc(HWND__ * 0x000501da, unsigned int 278, unsigned int 65945, long 0) line 3089 + 31 bytes +USER32! UserCallWinProc@@20 + 24 bytes +USER32! DispatchClientMessage@@20 + 47 bytes +USER32! __fnDWORD@@4 + 34 bytes +NTDLL! KiUserCallbackDispatcher@@12 + 19 bytes +USER32! DispatchClientMessage@@20 address 0x77e163cc +USER32! DefWindowProcW@@16 + 34 bytes +qxeDefWindowProc(HWND__ * 0x000501da, unsigned int 274, unsigned int 61696, long 98) line 1188 + 22 bytes +mswindows_wnd_proc(HWND__ * 0x000501da, unsigned int 274, unsigned int 61696, long 98) line 3362 + 21 bytes +USER32! UserCallWinProc@@20 + 24 bytes +USER32! DispatchClientMessage@@20 + 47 bytes +USER32! __fnDWORD@@4 + 34 bytes +NTDLL! KiUserCallbackDispatcher@@12 + 19 bytes +USER32! DispatchClientMessage@@20 address 0x77e163cc +USER32! DefWindowProcW@@16 + 34 bytes +qxeDefWindowProc(HWND__ * 0x000501da, unsigned int 262, unsigned int 98, long 540016641) line 1188 + 22 bytes +mswindows_wnd_proc(HWND__ * 0x000501da, unsigned int 262, unsigned int 98, long 540016641) line 3362 + 21 bytes +USER32! UserCallWinProc@@20 + 24 bytes +USER32! DispatchMessageWorker@@8 + 244 bytes +USER32! DispatchMessageW@@4 + 11 bytes +qxeDispatchMessage(const tagMSG * 0x0082c684 @{msg=0x00000106 wp=0x00000062 lp=0x20300001@}) line 989 + 10 bytes +mswindows_drain_windows_queue() line 1345 + 9 bytes +emacs_mswindows_quit_p() line 3947 +event_stream_quit_p() line 666 +check_quit() line 686 +check_what_happened() line 437 +re_match_2_internal(re_pattern_buffer * 0x012d5a18, const unsigned char * 0x00000000, int 0, const unsigned char * 0x02235000, int 23486, int 14645, re_registers * 0x012d53d0 search_regs, int 23486) line 4717 + 14 bytes +re_search_2(re_pattern_buffer * 0x012d5a18, const char * 0x02235000, int 23486, const char * 0x0223b38e, int 0, int 14645, int 8841, re_registers * 0x012d53d0 search_regs, int 23486) line 4269 + 37 bytes +search_buffer(buffer * 0x022fde00, long 29077572, long 13789, long 23487, long 1, int 1, long 28377092, long 28377092, int 0) line 1224 + 89 bytes +search_command(long 29077572, long 46975, long 28377116, long 28377092, long 28377092, int 1, int 1, int 0) line 1054 + 151 bytes +Fre_search_forward(long 29077572, long 46975, long 28377116, long 28377092, long 28377092) line 2147 + 31 bytes +Ffuncall(int 4, long * 0x0082ceb0) line 3488 + 216 bytes +execute_optimized_program(const unsigned char * 0x02047810, int 13, long * 0x02080c10) line 744 + 16 bytes +funcall_compiled_function(long 34187208, int 3, long * 0x0082d1b8) line 516 + 53 bytes +Ffuncall(int 4, long * 0x0082d1b4) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x01e96a10, int 6, long * 0x020ae510) line 744 + 16 bytes +funcall_compiled_function(long 34186676, int 3, long * 0x0082d4a0) line 516 + 53 bytes +Ffuncall(int 4, long * 0x0082d49c) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x02156b50, int 4, long * 0x020c2db0) line 744 + 16 bytes +funcall_compiled_function(long 34186564, int 2, long * 0x0082d780) line 516 + 53 bytes +Ffuncall(int 3, long * 0x0082d77c) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x0082d964, int 3, long * 0x020c2d70) line 744 + 16 bytes +Fbyte_code(long 29405156, long 34352480, long 7) line 2392 + 38 bytes +Feval(long 34354440) line 3290 + 187 bytes +condition_case_1(long 34354572, long (long)* 0x01087232 Feval(long), long 34354440, long (long, long)* 0x01084764 run_condition_case_handlers(long, long), long 28377092) line 1692 + 7 bytes +condition_case_3(long 34354440, long 28377092, long 34354572) line 1779 + 27 bytes +execute_rare_opcode(long * 0x0082dc7c, const unsigned char * 0x01b090af, int 143) line 1269 + 19 bytes +execute_optimized_program(const unsigned char * 0x01b09090, int 6, long * 0x020ae590) line 654 + 17 bytes +funcall_compiled_function(long 34186620, int 0, long * 0x0082df68) line 516 + 53 bytes +Ffuncall(int 1, long * 0x0082df64) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x02195470, int 1, long * 0x020c2df0) line 744 + 16 bytes +funcall_compiled_function(long 34186508, int 0, long * 0x0082e23c) line 516 + 53 bytes +Ffuncall(int 1, long * 0x0082e238) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x01e5d410, int 6, long * 0x0207d410) line 744 + 16 bytes +funcall_compiled_function(long 34186312, int 1, long * 0x0082e524) line 516 + 53 bytes +Ffuncall(int 2, long * 0x0082e520) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x02108fb0, int 2, long * 0x020c2e30) line 744 + 16 bytes +funcall_compiled_function(long 34186340, int 0, long * 0x0082e7fc) line 516 + 53 bytes +Ffuncall(int 1, long * 0x0082e7f8) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x020fe150, int 2, long * 0x01e6f510) line 744 + 16 bytes +funcall_compiled_function(long 31008124, int 0, long * 0x0082ebd8) line 516 + 53 bytes +Ffuncall(int 1, long * 0x0082ebd4) line 3523 + 17 bytes +run_hook_with_args_in_buffer(buffer * 0x022fde00, int 1, long * 0x0082ebd4, int 0) line 3980 + 13 bytes +run_hook_with_args(int 1, long * 0x0082ebd4, int 0) line 3993 + 23 bytes +Frun_hooks(int 1, long * 0x0082ebd4) line 3847 + 19 bytes +Ffuncall(int 2, long * 0x0082ebd0) line 3509 + 14 bytes +execute_optimized_program(const unsigned char * 0x01ef2210, int 5, long * 0x01da8e10) line 744 + 16 bytes +funcall_compiled_function(long 31020440, int 2, long * 0x0082eeb8) line 516 + 53 bytes +Ffuncall(int 3, long * 0x0082eeb4) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x0082f09c, int 3, long * 0x01d89390) line 744 + 16 bytes +Fbyte_code(long 31102388, long 30970752, long 7) line 2392 + 38 bytes +Feval(long 31087568) line 3290 + 187 bytes +condition_case_1(long 30961240, long (long)* 0x01087232 Feval(long), long 31087568, long (long, long)* 0x01084764 run_condition_case_handlers(long, long), long 28510180) line 1692 + 7 bytes +condition_case_3(long 31087568, long 28510180, long 30961240) line 1779 + 27 bytes +execute_rare_opcode(long * 0x0082f450, const unsigned char * 0x01ef23ec, int 143) line 1269 + 19 bytes +execute_optimized_program(const unsigned char * 0x01ef2310, int 6, long * 0x01da8f10) line 654 + 17 bytes +funcall_compiled_function(long 31020412, int 1, long * 0x0082f740) line 516 + 53 bytes +Ffuncall(int 2, long * 0x0082f73c) line 3523 + 17 bytes +execute_optimized_program(const unsigned char * 0x020fe650, int 3, long * 0x01d8c490) line 744 + 16 bytes +funcall_compiled_function(long 31020020, int 2, long * 0x0082fa14) line 516 + 53 bytes +Ffuncall(int 3, long * 0x0082fa10) line 3523 + 17 bytes +Fcall_interactively(long 29685180, long 28377092, long 28377092) line 1008 + 22 bytes +Fcommand_execute(long 29685180, long 28377092, long 28377092) line 2929 + 17 bytes +execute_command_event(command_builder * 0x01be1900, long 36626492) line 4048 + 25 bytes +Fdispatch_event(long 36626492) line 4341 + 70 bytes +Fcommand_loop_1() line 582 + 9 bytes +command_loop_1(long 28377092) line 495 +condition_case_1(long 28377188, long (long)* 0x01064fb9 command_loop_1(long), long 28377092, long (long, long)* 0x010649d0 cmd_error(long, long), long 28377092) line 1692 + 7 bytes +command_loop_3() line 256 + 35 bytes +command_loop_2(long 28377092) line 269 +internal_catch(long 28457612, long (long)* 0x01064b20 command_loop_2(long), long 28377092, int * volatile 0x00000000) line 1317 + 7 bytes +initial_command_loop(long 28377092) line 305 + 25 bytes +STACK_TRACE_EYE_CATCHER(int 1, char * * 0x01b63ff0, char * * 0x01ca5300, int 0) line 2501 +main(int 1, char * * 0x01b63ff0, char * * 0x01ca5300) line 2938 +XEMACS! mainCRTStartup + 180 bytes +_start() line 171 +KERNEL32! BaseProcessStart@@4 + 115547 bytes +@end example + +[explain dont_check_for_quit() et al] + @node Profiling, Asynchronous Timeouts, Control-G (Quit) Checking, Asynchronous Events; Quit Checking @section Profiling @cindex profiling @@ -21879,20 +22984,33 @@ into the normal Future Work section. @menu -* Discussion -- garbage collection:: -* Discussion -- glyphs:: +* Discussion -- Garbage Collection:: +* Discussion -- Glyphs:: * Discussion -- Dialog Boxes:: * Discussion -- Multilingual Issues:: +* Discussion -- Instantiators and Generic Property Accessors:: +* Discussion -- Switching to C++:: * Discussion -- Windows External Widget:: * Discussion -- Packages:: * Discussion -- Distribution Layout:: @end menu -@node Discussion -- garbage collection, Discussion -- glyphs, Future Work Discussion, Future Work Discussion -@section Discussion -- garbage collection +@node Discussion -- Garbage Collection, Discussion -- Glyphs, Future Work Discussion, Future Work Discussion +@section Discussion -- Garbage Collection @cindex discussion, garbage collection @cindex garbage collection, discussion +@menu +* Discussion -- Pure Space:: +* Discussion -- Hashtable-Based Marking and Cleanup:: +* Discussion -- The Anti-Cons:: +@end menu + +@node Discussion -- Pure Space, Discussion -- Hashtable-Based Marking and Cleanup, Discussion -- Garbage Collection, Discussion -- Garbage Collection +@subsection Discussion -- Pure Space +@cindex discussion, pure space +@cindex pure space, discussion + On Tue, Oct 12, 1999 at 03:36:59AM -0700, Ben Wing wrote: So what am I missing here? @@ -21968,9 +23086,12 @@ think, really cool. @end enumerate +@node Discussion -- Hashtable-Based Marking and Cleanup, Discussion -- The Anti-Cons, Discussion -- Pure Space, Discussion -- Garbage Collection +@subsection Discussion -- Hashtable-Based Marking and Cleanup +@cindex discussion, hashtable-based marking and cleanup +@cindex hashtable-based marking and cleanup, discussion + On 10/12/1999 5:49 PM Ben Wing wrote: - -Subject: Re: hashtable-based marking and cleanups OK, I can see the advantages. But: @@ -22015,8 +23136,105 @@ http://www.amazon.com/exec/obidos/ASIN/0471941484/qid=939775572/sr=1-1/002-3092633-2509405 @end example -@node Discussion -- glyphs, Discussion -- Dialog Boxes, Discussion -- garbage collection, Future Work Discussion -@section Discussion -- glyphs +@node Discussion -- The Anti-Cons, , Discussion -- Hashtable-Based Marking and Cleanup, Discussion -- Garbage Collection +@subsection Discussion -- The Anti-Cons +@cindex discussion, the anti-cons +@cindex the anti-cons, discussion + +From: "Ben Wing" <ben@@666.com> +Date: Tue, 14 May 2002 06:48:09 -0700 + +i was thinking about the proliferating types of weak hash tables -- +e.g. now we have "key-car-value weak" hash tables due to a need in the +glyphs code. i realized there should be a general solution, that lets +you control exactly how the weakness of such hash tables work. + +and, assuming we implement a simple "reference" type, a simple +container whose object is a weak reference and thus gets converted to +nil (and a flag set on the reference) when the object is collected, it +would be useful for more precisely controlling the reference, too. + +it's called an "anti-cons". it behaves somewhat like a cons in that +it boxes two items, but its marking properties are very different -- +in fact, backwards. normally, a cons, if marked, marks its children. +in this case, if the children of an anti-cons are marked, it marks +itself! you'd need a few different kinds of anti-cons -- probably the +following: + +@example +and [marks itself if both children marked] +or [...] +left [marks itself if left is marked, and then marks the right] +right [...] +not-left +not-right +@end example + +by putting such an object inside of a weak reference -- e.g. in a weak +hash table -- we can set up a tree of arbitrary complexity which +implements any boolean formula of markedness over any number of +objects. this would easily handle key-car, and key-cadr, and +key-car-or-cdr, and key-((caar or cadr) and cdr) etc. etc. + +implementing this in the current xemacs framework is mostly trivial. + +michael, would such an object get in the way of your new gc? + +From: sperber@@informatik.uni-tuebingen.de (Michael Sperber [Mr. Preprocessor]) +Date: Tue, 14 May 2002 16:04:01 +0200 + +You might want to look at + +http://research.microsoft.com/Users/simonpj/Papers/weak.htm + +for a pretty comprehensive survey of what you could want in terms of +weakness. Its weak pointers are very similar to your anti-cons. +However, there are some problems in doing the same in a Lisp settings, +mainly because of symbols. I intend to elaborate on this next week; +this week is full, unfortunately. + +Ben> implementing this in the current xemacs framework is mostly +Ben> trivial. + +Ben> michael, would such an object get in the way of your new gc? + +Well, our first commit will be an implementation of vanilla weak +boxes (ready within the next few days, I hope), and we'll then try to +replace most other instances of weakness with uses of those. We'll +then try to find a more general solution for the rest. (Richard +Reingruber has already done a comprehensive survey of the trouble +spot. + +Can you wait until next week? I'll try to come up with a battle plan +then. + +From: sperber@@informatik.uni-tuebingen.de (Michael Sperber [Mr. Preprocessor]) +Date: Tue, 28 May 2002 16:14:20 +0200 + +We've now started implementing ephemerons as a building block for the +more involved weakness-involving data structures: + +The relevant reference is + +Barry Hayes. Ephemerons: A New Finalization Mechanism. +OOPSLA 1997. 176--183 + +The idea is this: + +an ephemeron consists of a key and a value. Through the ephemeron, +the key is not reachable. The value is only reachable if both the +ephemeron is reachable and the key is reachable. If the ephemeron is +reachable and the key becomes unreachable, the value slot of the +ephemeron will be tombstoned, i.e. overwritten with NIL or something. + +This allows implementing, AFAICS, the other data structures involving +weakness, such as weak hash tables and their various mutants. + +We're also planning to come up with a more comprehensive solution for +finalization, but some design snags remain to be worked out. + +@node Discussion -- Glyphs, Discussion -- Dialog Boxes, Discussion -- Garbage Collection, Future Work Discussion +@section Discussion -- Glyphs @cindex discussion, glyphs @cindex glyphs, discussion @@ -22243,7 +23461,7 @@ way, of course, to get at the actual widget that you were invoked from. would you propose adding widget-callback-current-widget? -@node Discussion -- Dialog Boxes, Discussion -- Multilingual Issues, Discussion -- glyphs, Future Work Discussion +@node Discussion -- Dialog Boxes, Discussion -- Multilingual Issues, Discussion -- Glyphs, Future Work Discussion @section Discussion -- Dialog Boxes @cindex discussion, dialog boxes @cindex dialog boxes, discussion @@ -22643,7 +23861,7 @@ @end example -@node Discussion -- Multilingual Issues, Discussion -- Windows External Widget, Discussion -- Dialog Boxes, Future Work Discussion +@node Discussion -- Multilingual Issues, Discussion -- Instantiators and Generic Property Accessors, Discussion -- Dialog Boxes, Future Work Discussion @section Discussion -- Multilingual Issues @cindex discussion, multilingual issues @cindex multilingual issues, discussion @@ -23500,7 +24718,856 @@ @end example -@node Discussion -- Windows External Widget, Discussion -- Packages, Discussion -- Multilingual Issues, Future Work Discussion +@node Discussion -- Instantiators and Generic Property Accessors, Discussion -- Switching to C++, Discussion -- Multilingual Issues, Future Work Discussion +@section Discussion -- Instantiators and Generic Property Accessors +@cindex discussion, instantiators and generic property accessors +@cindex instantiators and generic property accessors, discussion + +From: Ben Wing <ben@@666.com> +Date: Sun, 05 May 2002 05:40:07 -0700 +Subject: generic functions, new instantiator API + +I've been reading the C++ manual and getting polymorphism, inheritance, +generic functions, etc. in my head. + +We have our own "generic function" already in terms of `get', `put', +etc. which accept various objects. i'm thinking of extending them so +they can accept, as well as objects, lists (either alists or plists) +or plist-style vectors, and manipulate their properties. what do +people think of this? + +Also, i'm designing a new API for "instantiators", which are objects +whose main purpose is to hold properties and provide a way of notifying +their containing specifiers when they change. Instantiator objects are +used when the instantiator gets sufficiently complicated that using +lists and vectors gets unwieldy -- e.g. when creating widget trees, such +as would appear in dialog boxes. you want the ability to +programmatically traipse up and down the tree and dynamically modify a +part of the tree -- e.g. a property on a single widget -- as necessary, +and have the internal code automatically notice this change and performs +any necessary updates. lists and vectors are too low-level for this -- +no way to get their parent, no way for internal code to be notified when +changes occur, can't always maintain object identity when making +property changes, no way to error-check illegal changes, etc. + +You could also extend this api to cover toolbars; it would probably make +toolbar manipulation significantly easier. but you'd have to think +about backward compatibility in such cases. + +here is what the api looks like so far -- making use of a newly-added +facility for keyword args in primitives. comments are welcome. + +@example +DEFUN ("make-instantiator", Fmake_instantiator, 1, MANY, 0, /* +Create a new instantiator object from TYPE and PROPS. +TYPE should be one of the image instantiator formats described in +`make-glyph'. +The rest of the arguments should be keyword properties and associated +values, +as also described in `make-glyph'. + +TYPE can also be an old-style vector instantiator. + +Instantiator objects can be used as instantiators (see `make-specifier') in +glyphs in place of old-style vector instantiators. They are especially +used for complicated, nested graphical elements such as widgets (buttons, +text fields, etc.) -- in fact, widget instantiators will automatically be +converted into instantiator objects if they are given in vector format. + +Individual properties on instantiators can be manipulated using +`set-instantiator-property'. If the property's value is a list (for +example, a list of children), you can also use `add-instantiator-item' +to add or insert individual elements in the list. + +`delete-instantiator-item' can be used to delete individual items in the +list; +`get-instantiator-item' to locate individual items in the list; and +`get-instantiator-item-position' to return the position of individual +items in +the list. + +`map-instantiator' can be used to (recursively or not) map over an +instantiator and its children. + +`find-instantiator' can be used to (recursively or not) locate an +instantiator +in a tree composed of an instantiator and its descendants. +*/ + /* (type &rest props) */ + (int nargs, Lisp_Object *args)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN ("set-instantiator-property", Fset_instantiator_property, 3, 3, 0, /* +Set property PROP to VALUE in INSTANTIATOR. +INSTANTIATOR should have been created with `make-instantiator'. +Valid properties depend on the instantiator type and are described in +`make-glyph'. For properties that are lists of items, individual items +can be added or deleted using `add-instantiator-item' and +`delete-instantiator-item'. + +For compatibility, this also accepts an old-style vector instantiator, and +destructively modifies it; in this case, adding a property requires +creating a new vector, which is returned. You need to use +`set-glyph-image' on glyphs, or `set-specifier-dirty-flag' on the result of +`glyph-image', to register instantiator changes to vector +instantiators. (New-style instantiators automatically convey property +changes to any glyphs they have been attached to.) +*/ + (instantiator, prop, value)) +@{ + Lisp_Object *elt; + int len; + + /* ^^#### */ + CHECK_VECTOR (instantiator); + if (!KEYWORDP (prop)) + invalid_argument ("instantiator property must be a keyword", prop); + + elt = XVECTOR_DATA (instantiator); + len = XVECTOR_LENGTH (instantiator); + + for (len -= 2; len >= 1; len -= 2) + @{ + if (EQ (elt[len], prop)) + @{ + elt[len + 1] = value; + break; + @} + @} + + /* Didn't find it so add it. */ + if (len < 1) + @{ + Lisp_Object alist = Qnil, result; + struct gcpro gcpro1; + + GCPRO1 (alist); + alist = tagged_vector_to_alist (instantiator); + alist = Fcons (Fcons (prop, value), alist); + result = alist_to_tagged_vector (elt[0], alist); + free_alist (alist); + RETURN_UNGCPRO (result); + @} + + return instantiator; +@} + +DEFUN ("instantiator-property", Finstantiator_property, 2, 3, 0, /* +Return the property PROP of INSTANTIATOR, or DEFAULT if PROP has no value. +INSTANTIATOR should have been created with `make-instantiator'. +*/ + (instantiator, prop, default_)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN ("instantiator-properties", Finstantiator_properties, 1, 1, 0, /* +Return a plist of all defined properties in INSTANTIATOR. +INSTANTIATOR should have been created with `make-instantiator'. +*/ + (instantiator)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN ("instantiator-type", Finstantiator_type, 1, 1, 0, /* +Return the type of INSTANTIATOR. +INSTANTIATOR should have been created with `make-instantiator'. +Valid types are the instantiator formats described in `make-glyph'. +*/ + (instantiator)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN ("instantiator-parent", Finstantiator_parent, 1, 1, 0, /* +Return the parent of INSTANTIATOR. +INSTANTIATOR should have been created with `make-instantiator'. +*/ + (instantiator)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN_WITH_KEYWORDS ("map-instantiator", Fmap_instantiator, 2, 2, 1, 0, +0, /* +Map FUN recursively over INSTANTIATOR and its descendants. +FUN is called with one argument, the INSTANTIATOR. +If:norecurse is non-nil, don't recurse, just map over the direct +children (not including the instantiator itself). +*/ + (fun, instantiator), + (norecurse)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN_WITH_KEYWORDS ("find-instantiator", Ffind_instantiator, 3, 3, + 1, 0, 0, /* +Find an instantiator by PROP and VALUE in INSTANTIATOR and its descendants. +Returns first item which has PROP set to VALUE. +If:norecurse is non-nil, don't recurse, just look through the direct +children (not including the instantiator itself). +*/ + (instantiator, prop, value), + (norecurse)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN_WITH_KEYWORDS ("add-instantiator-item", Fadd_instantiator_item, 3, +3, 7, + 0, 0, /* +Add an item to an instantiator property that's a list of items. +\(E.g. the children of an instantiator). PROP is the property whose list of +items is being modified, and ITEM is the item to add. To insert somewhere +before the end, use one of the keywords: + +--:position specifies a zero-based index of an item, and the new item +will be +inserted just before the item indicated by the position. Negative numbers +count from the end -- thus -1 will cause insertion before the last item, -2 +before the second-to-last item, etc. + +--:before-item and :after-item specify items to insert before or after. +:test (defaults to `eq') can be used to specify the way to compare the given +item with existing items. + +--:before-property and :after-property search for an item to insert +before or +after by looking for an item with the given property. If :value is +given, the +property must have that value; otherwise, it simply must exist. This method +of insertion works if the items in PROP's list are anything that can have or +hold properties. \("To have and to hold, for ever and ever ...") This +includes: + +-- any object for which `get' works +-- else, if object is a vector, assume it's a plist-style vector +-- else, if object is a cons, and its first element is also a cons, + assume it's an alist +-- else, if object is a cons, assume it's a plist +*/ + (instantiator, prop, item), + (position, before_item, after_item, test, + before_property, after_property, value)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN_WITH_KEYWORDS ("delete-instantiator-item", Fdelete_instantiator_item, + 2, 2, 5, 0, 0, /* +Delete an item in an instantiator property that's a list of items. + +\(E.g. the children of an instantiator). PROP is the property whose list is +being searched. One of these keywords should be given: + +--:position specifies a zero-based index of an item. Negative numbers +count from the end -- thus -1 will cause insertion before the last item, -2 +before the second-to-last item, etc. + +--:item specifies the item to delete. :test (defaults to `eq') can be +used to +specify the way to compare the given item with existing items. + +--:property searches for an item with the given property. If :value is +given, the property must have that value; otherwise, it simply must exist. +This method of insertion works if the items in PROP's list are anything that +can have or hold properties -- see `add-instantiator-item'. +*/ + (instantiator, prop), + (item, test, position, property, value)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN_WITH_KEYWORDS ("get-instantiator-item", Fget_instantiator_item, + 2, 2, 3, 0, 0, /* +Get an item in an instantiator property that's a list of items. + +\(E.g. the children of an instantiator). PROP is the property whose list is +being searched. One of these keywords should be given: + +--:position specifies a zero-based index of an item. Negative numbers +count +from the end -- thus -1 will cause insertion before the last item, -2 before +the second-to-last item, etc. + +--:property searches for an item with the given property. If :value is +given, the property must have that value; otherwise, it simply must exist. +This method of insertion works if the items in PROP's list are anything that +can have or hold properties -- see `add-instantiator-item'. +*/ + (instantiator, prop), + (position, property, value)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN_WITH_KEYWORDS ("get-instantiator-item-position", + Fget_instantiator_item_position, + 2, 2, 4, 0, 0, /* +Return an item's position in an instantiator property that's a list of +items. + +\(E.g. the children of an instantiator). PROP is the property whose list is +being searched. One of these keywords should be given: + +--:item specifies the item to search for. :test (defaults to `eq') can be +used to specify the way to compare the given item with existing items. + +--:property searches for an item with the given property. If :value is +given, the property must have that value; otherwise, it simply must exist. +This method of insertion works if the items in PROP's list are anything that +can have or hold properties -- see `add-instantiator-item'. +*/ + (instantiator, prop), + (item, test, property, value)) +@{ + /* ^^#### */ return Qnil; +@} + +DEFUN ("image-instance-instantiator", Fimage_instance_instantiator, 1, +1, 0, /* +Return the instantiator from which IMAGE-INSTANCE was created. +*/ + (image_instance)) +@{ + /* ^^#### */ return Qnil; +@} +@end example + +some other useful stuff: + +@example +DEFUN ("make-image-instance", Fmake_image_instance, 1, 4, 0, /* +Return a new `image-instance' object. + +Image-instance objects encapsulate the way a particular glyph (pixmap, +widget, etc.) is displayed on a particular device. In most circumstances, +you do not need to directly create image instances; instead, you create a +glyph using `make-glyph' and add settings (or "instantiators") onto it +using `set-glyph-image', and XEmacs creates the image instances as +necessary. However, it may occasionally be useful to explicitly create +image instances, if you want more control over the instantiation process. + +For more information on instantiators and instances, see `make-specifier'. + +DATA is an image instantiator, which describes the image; see `make-glyph' +for a description of the allowed values. + +The most likely circumstance where you need to deal directly with image +instances is in widget callbacks -- e.g. the callback that's executed when +a button is pressed in a dialog box of type `general' (see +`make-dialog-box'). In this case, the widget that was activated is +described by an image instance. (The callback is usually be written as an +interactive function with an interactive spec of (interactive \"e\"), and a +single `event' argument. The event will be an activate event, describing +the user action that trigged the callback. The image instance is +retrievable from the event using `event-image-instance'. Handling the +action may involve setting properties on the image instance or other image +instances in the dialog box in which the widget is usually contained -- or +changing the instantiator that generated the image instance, if you want +permanent changes that will be reflected the next time the dialog box is +popped up. Properties on an image instance are set using +`set-image-instance-property'. If the widget is part of a hierarchy of +widgets (as is usually the case in a dialog box, but may not apply if the +widget was inserted by itself in a buffer [by creating a glyph and +attaching it to an extent -- see `make-glyph']), there will be a +corresponding hierarchy of image instances to describe this particular +instance of the dialog box. You can retrieve other image instances in the +hierarchy using primitives such as `image-instance-parent', +`image-instance-children', and `find-image-instance'. +@end example + +... + + + + + +@example +(defun image-instance-property (image-instance property &optional default) + "Return the given property of the given image instance. +Returns DEFAULT if the property or the property method do not exist for +the image instance in the domain." + (check-argument-type 'image-instance-p image-instance) + (get image-instance property default)) + +(defun set-image-instance-property (image-instance prop value) + "Set the property PROP on IMAGE-INSTANCE to VALUE. +Only certain properties of the image instance can be changed, and they +represent \"temporary\" changes. If you want to make permanent changes, +you need to change the instantiator that generated the instance -- +retrieve the instantiator with `image-instance-instantiator', and change +its properties with `set-instantiator-property'. + +This applies mostly to widgets. For example, you can set a property on +a widget image instance to change the state of a radio or checkbox button, +set the text currently in an edit field, etc. However, those changes apply +only to the *currently* displayed widgets. If these widgets are in a dialog +box, and you want to change the way the widgets in the dialog box appear +*each* time the dialog box is displayed, you need to change the +instantiator. + +Make sure you understand the difference between instantiators and +instances. An \"instantiator\" is a specification, indicating how to +determine the value of a setting whose value can vary in different +circumstances or \"locales\" (buffers, frames, etc.). An \"instance\" +is the +resulting value in a particular circumstance. For more information, see +`make-specifier'." + (check-argument-type 'image-instance-p image-instance) + (put image-instance prop value)) +@end example + +From: "Stephen J. Turnbull" <stephen@@xemacs.org> +Date: 06 May 2002 16:40:46 +0900 + +>>>>> "Ben" == Ben Wing <ben@@666.com> writes: + + Ben> We have our own "generic function" already in terms of `get', + Ben> `put', etc. which accept various objects. + +I proposed extending the class to stuff like charsets about two years +ago, and I think you were one of the folks who objected. + + Ben> i'm thinking of extending them so they can accept lists + Ben> (either alists or plists) or plist-style vectors, and + Ben> manipulate their properties. what do people think of this? + +I think extending to lists is something we should approach +cautiously. For one thing, if "get" is polymorphic, "put" would have +to be too. But how does it decide when dealing with "nil"? + + Ben> you want the ability to programmatically traipse up and down + Ben> the tree and dynamically modify a part of the tree -- e.g. a + Ben> property on a single widget -- as necessary, and have the + Ben> internal code automatically notice this change and performs + Ben> any necessary updates. + +I like this. + +From: "Stephen J. Turnbull" <stephen@@xemacs.org> +Date: 07 May 2002 11:17:05 +0900 + +>>>>> "Neal" == Neal D Becker <nbecker@@hns.com> writes: + + Neal> I thought that generic polymorphism was inherent in lisp, as + Neal> it is dynamically evaluated. Why would you need anything + Neal> special in the way functions are written to support generic + Neal> programming? + +I think it's basically a technical matter. We have a number of +objects that have property lists besides symbols. Many of them have +special functions (coding-system-get, coding-system-property, +charset-property are examples I find particularly obnoxious). I would +like to make these obsolete by allowing `get' on charsets, coding +systems, etc. + +And currently we have + +(let ((p (symbol-plist symbol))) (plist-get p prop)) + +Ben would like to allow + +(let ((p (symbol-plist symbol))) (get p prop)) + +with `get' determining whether P is a plist or an alist. And where +Michael says "why not use hash tables?", I see `(get hash key)' +(probably to Michael's horror ;-). + +This isn't Lisp any more, though, in some sense. But then we haven't +been that for years. AFAIK all real Lisps restrict `get' to symbols. + +From: sperber@@informatik.uni-tuebingen.de (Michael Sperber [Mr. Preprocessor]) +Date: Tue, 07 May 2002 08:52:53 +0200 + +Indeed. I'll just say "goosebumps." But I don't see why it has to be +GET that accesses the plist. You just build more dispatch into GET +with no immediate benefit to the API. Ad-hoc genericity gets you +something when there's some place in the code you don't know what the +underlying object is. I don't see this being the case here. Why do +you find them "obnoxious?" + +Stephen> This isn't Lisp any more, though, in some sense. But then we haven't +Stephen> been that for years. AFAIK all real Lisps restrict `get' to symbols. + +Actually, Scheme (which admittedly isn't a real Lisp by many +standards) doesn't have get/put at all. And good riddance, I might +add:-) + +From: "Stephen J. Turnbull" <stephen@@xemacs.org> +Date: 07 May 2002 20:04:50 +0900 + +>>>>> "ms" == Michael Sperber <sperber@@informatik.uni-tuebingen.de> writes: + +Stephen> special functions (coding-system-get, coding-system-property, +Stephen> charset-property are examples I find particularly obnoxious). I would +Stephen> like to make these obsolete by allowing `get' on charsets, coding +Stephen> systems, etc. + + ms> But I don't see why it has to be GET that accesses the plist. + ms> You just build more dispatch into GET with no immediate + ms> benefit to the API. Ad-hoc genericity gets you something when + ms> there's some place in the code you don't know what the + ms> underlying object is. I don't see this being the case here. + ms> Why do you find them "obnoxious?" + +Their semantics are basically `get'. Why not use that name? Of +course I agree that it doesn't have to be `get', but why clutter +things up? + +But those are particularly obnoxious because of the object/name +confusion they have built in. Ie, my real problem with them is more +ancient Mule idiom than the *-get or *-property names for the API. + + ms> Actually, Scheme (which admittedly isn't a real Lisp by many + ms> standards) doesn't have get/put at all. + +What does it use instead? (And no, you can't bait _me_ with Lisp +definition trolls, I think of XML as "declarative LISP with fat, +flavored, fuzzy parentheses.") + +From: sperber@@informatik.uni-tuebingen.de (Michael Sperber [Mr. Preprocessor]) +Date: Tue, 07 May 2002 13:26:13 +0200 + +Stephen> Their semantics are basically `get'. Why not use that name? + +Because it doesn't convey as much information in the source code as it +could, and because it provides less type checking than it could. + +Stephen> What does it use instead? + +What for? I've never felt the desire to use them, and it seems to me +that in Lisp, properties are usually used for one of two purposes: + +- As a poor man's replacement for hash tables. + +- To store data which should really be stored inside the object + itself. + +In the former case, I use a hash table. In the latter case, I store +the data in the object itself. + +@node Discussion -- Switching to C++, Discussion -- Windows External Widget, Discussion -- Instantiators and Generic Property Accessors, Future Work Discussion +@section Discussion -- Switching to C++ +@cindex discussion, switching to c++ +@cindex switching to c++, discussion + +From: "Ben Wing" <ben@@666.com> +Date: Fri, 10 May 2002 19:42:53 -0700 + +i know i'm opening up a bag of worms by suggesting this, but what +about moving to C++? I know others advocate this (Jan, Martin), and +the more I read Stroustrup's 3rd edition, the more I realize that +*HUGE* armounts of code in XEmacs, and in particular most of the +really hairy and hard-to-understand stuff -- lots of weird macros, +faux object-oriented stuff implemented in multiple places, each +differently (Lisp objects; methods on consoles/devices/etc; specifier +sub-types; coding-system sub-types; image-instance device methods; +image-instance format methds; etc.), all the GCPROS (which could go +entirely), dynarrs, eistring, etc. etc. -- is simply superseded by +stuff already built into C++ or supplied by the standard libraries. +Just now, I was going through the redisplay code, and noticing the +huge amount of duplication between gtk and X, something that's hard to +fix [except through super ad-hoc ways like using a .c file as a .h +file to "generate" lots of similar but slightly different code] in C, +but is extremely easy in C++ using inheritance [and/or templates]. +for example, instead of having just one layer of device methods, you'd +have + +@example +general -> tty + -> windowing -> mswindows + -> xlike -> x, gtk +@end example + +which would nicely and naturally encapsulate lots of duplicated [and +thus, hard to maintain] code. + +even more of a win would be the GCPRO's. Taking advantage of +constructors and destructors, we could simply do away [COMPLETELY!] +with explicitly gcproing, and still have everything gcpro'd. [in fact, +much more reliably -- none of the dreaded "temporary" problem, and +every reference is always gcpro'd so we have greater flexibility for +GC work -- take note, Michael :-) -- e.g. we could safely garbage +collect when allocating, and we could even implement a relocating +garbage collector. in the few places where performance might be an +issue [i seriously doubt there'd be many of them], we simply use a +separate Lisp_Object_No_GCPRO class (presumably a base class of +Lisp_Object), and manually handle the GCPRO's ourselves. If we needed +to distinguish here between static and dynamic objects, or static +vs. local vs. heap, we could do so easily with bit flags in the object +pointed to -- we have space for lots of them. + +code reliablity and maintainability would likely substantially +increase due to the ability to express most things in a natural C++ +way instead of lots of weird hackish hard-to-understand C stuff +implementing stuff the language wasn't really designed for. +Furthermore, there are even some possibilities for increased speed -- +many operations that can only reasonably be done now using Lisp +objects (and the associated gc overhead and such) could be done using +the high-level built-in facilities of C++, which in their ease of use +approach Lisp; and C++ has `inline' built-in, so we could easily add +various container classes to improve the understandability of the code +without loss of performance. + +finally, making the "switch" is trivial, since martin did the initial +work making XEmacs C++-safe and I've been keeping it that way -- I +regularly build under C++ and fix any problems. All we'd need to do +is switch the compiler and start gradually introducing C++ constructs +as we feel like it. + +for those concerned that dumping might stop working, [a] i don't think +it would, [b] the portable dumper has come of age -- i use it almost +all the time, and it's rock-solid and not obviously slower than +unexec. + +the only major concern that i see is the quality of the C++ +implementations out there, in particular G++, which is the most widely +available. I know that 6 years ago G++ was a bit rocky -- I went to +interview for Netscape, and they mentioned having to rely on various +vendor implementations of C++, whereas they would have preferred G++ +if it was reliable, due to the sameness of environment. But that was +*SIX YEARS* ago! Stroustrup 3rd Edition has been out for 5 years now, +and it defines, as far as I know, ANSI Standard C++ -- so that's at +least 5 years to implement a standard. It's hard to believe that G++ +isn't completely reliable now; but I do not have as much experience as +others. + +What do you think? I would *really* like to make this change, as it +would immensely facilitate lots of code I'm working on and will be +working on, plus of course add all the above benefits once we get +around to converting the code. + +From: "Stephen J. Turnbull" <stephen@@xemacs.org> +Date: 11 May 2002 15:34:08 +0900 + +I don't have a real problem with it, as long as we're very +conservative about it, ie, using C++ as "clean C with classes", and +introducing things slowly. Implement everything ourselves, avoid the +standard class libraries. + +I've been following the Python lists recently, and although the bias +is easy to guess, it's interesting to note that the people who are +most anti-C++ are typically the ones who are world-class C++ +programmers with big projects under their belts. Many of them +actually advocate using C rather than C++. + +I do worry that with Martin currently out of the picture we don't have +an active C++ standards bigot and implementation collector to deal +with compiler-specific issues. We do OK with C, but C++ is a much +more complex, subtle language. Is there anybody else to plausibly +take on that role? + +From: Hrvoje Niksic <hniksic@@arsdigita.com> +Date: Sun, 12 May 2002 20:58:50 +0200 + +I'm strongly opposed to this. Here are some reasons: + +* C++ may fix some problems, but it will introduce others, some of + which may be much harder to fix. XEmacs is already a large program, + hard to understand. C++ will not improve things. + +* XEmacs will suddenly become uncompilable and unusable in + many environments where it used to build perfectly fine -- for + example, those that don't ship with a C++ compiler at all. We could + "make GCC 3 a requirement", but I don't like that idea. + +* People without C++ experience will no longer be able to hack XEmacs. + I'd be the first one to leave. For example, I know quite a few + programmers who don't care for Qt and KDE simply because it's C++. + +* C is the /lingua franca/ of free software development. If we're + switching languages, it should be for a good reason and to something + we agree is an improvement (e.g. Common Lisp). + +* C++ is not the be-all end-all to everything. People who undestand + it well are usually the first ones to warn against it. It's + possible that they were scarred by using C++ at a bad time, but I'd + think twice before discounting their advice or blindly believing + that C++ is now all better. + +If you were writing a new project, I'd say go for it. But at this +point, this seems like a needless tweak. Do we really need *more* +internal reorganizations? Shouldn't we work on user-visible features? +Wasn't that what you yourself advocated when I talked to you? + +From: Didier Verna <didier@@xemacs.org> +Date: Tue, 14 May 2002 11:21:32 +0200 + + Switching to C++ has been suggested for the first time at the M17n'99 +conference in Japan IIRC. Although it was around a table with many empty +bottles of beer on it :-), I've kept some hope from that time. I'm happy to +see Ben in favor of this today. This idea coming from him is likely to have +more impact than when it comes from me or Yan of whoever else. + + There are several points that make me in favor of this change: + +- C++ support is already there thanks to Martin. + +- The amount of OO simulation code written in C in XEmacs is *HUGE*. But more + important, this code simulates *BASIC* OO features that are not any more a + problem for any C++ compiler. I mean, by just using the basic features of + C++ in terms of OO and data abstraction (classes, inheritance, methods (with + inlining), operators overloading), we'll win big time in code size, + readability, maintainability, and correctness. + +- the fact that *basic* OO support is already a major gain is very important + to me. You don't have to go generic programming with templates everywhere to + write an OO XEmacs[1]. Switching to C++ can be completely gradual, and we + can even stop early in the C++ features we use. That will already be a big + gain. That's also the advantage over the idea of using another more modern + language to rewrite the core; something completely unrealistic. + +- there is another important aspect on the design issue: many people + (including from the industry) have worked on abstracting common problems in + an OO philosophy. Some people claim that the concepts that emerged from this + kind of work of just C++ specific hackery, and they're probably right, but + anyway that's obviously not a problem for us. Any C++ writer should have the + "Design Patterns" book in hand. It already has good design solutions for + many problems that we're facing in XEmacs (like, supporting more than one + widget set), because these problems are so *common*. By using C++ we can + directly benefit from the experience of other large applications designers. + + +Footnotes: +[1] We're working on GP in C++ in our lab here and we trigger bugs in gcc 3. + But you should see the code in question, it's pure template and static + programming. Things that XEmacs will never need. + +From: Daniel Pittman <daniel@@rimspace.net> +Date: Sat, 11 May 2002 19:15:04 +1000 + + I've been following the Python lists recently, and although the bias +> is easy to guess, it's interesting to note that the people who are +> most anti-C++ are typically the ones who are world-class C++ +> programmers with big projects under their belts. Many of them +> actually advocate using C rather than C++. + +I wouldn't class myself as "world-class", but I can understand this +perspective based on my experiences with large projects that aim for +portability to vendor compilers, not just gcc. + +The biggest problem, assuming that you are willing to ignore platforms +like Sinix-PC[1] and their poor compiler support[2] is that it's easy to +shoot yourself in the foot with C++. + + +The biggest portability problems are namespaces, the standard C++ +library and template support, in about that order, followed by exception +handling. + +Very few things get namespaces right, even today, with gcc being one of +the worst. Tempting as they are, they are best avoided where possible, +except in compiler and platform specific code.[3] + + +The standard C++ library, which supports RTTI and a few other things +including the [io]stream tools, is less than totally reliable although +it can be used with relative safety most places. + +What you really need to watch out for with that is the extensions that +every vendor in the universe has added to their collections because +there isn't any standard way of doing common things in most of these +areas. + + +The Standard Template Library isn't. Aside from a tendency to expose +limitations of symbol name lengths[4] the library tends to be unreliable +in behavior between compilers and platforms. Not enough to make simple +things fail, though, just enough to make it occasionally do odd things +or show up obscure bugs in your code... + +It's also not very well designed, I think, as libraries go. That's a +personal opinion, though, and not universal. + + +C++ exceptions are an interesting issue. They can work extremely well as +a mechanism for managing errors and improve the reliability of the +system. + +They can also become an unending nightmare of epic proportions, causing +more pain and suffering than you can imagine. :/ + +The main difference between the two situations, so far as I can tell, +comes from two aspects of design that have ... far reaching +implications. + +If you try adding exceptions to code that isn't ready to deal with them, +things tend to go very badly wrong. I /think/ that the existing +exception mechanisms in XEmacs would be similar enough that this isn't +the case, though. + +The other is that you need to base your code very strongly around the +"construction acquires, destruction releases" model of resource +handling. This, of course, implies using exceptions everywhere because +you /can't/ use that model in C++ without them.[5] + +Again, I think that the existing XEmacs model will probably work well +with this, but I am hardly an expert at either; my only real-world +experience is the one project where I gained these impressions and the +knowledge of the suffering they can bring. :) + + +Oh, and finally, watch out for operator overloading -- including casts. +These are very easy to abuse into a position where your code is +impossible for others to understand. + +I would also advocate avoiding multiple inheritance, but that's because +my personal design experience says that it's almost always a sign of bad +design. Views there vary greatly. + +> I do worry that with Martin currently out of the picture we don't have +> an active C++ standards bigot and implementation collector to deal +> with compiler-specific issues. + +You probably have more need of the second than the first. There are not +many things you actually need a standards bigot for; just write good C +and don't use too many things other than classes. + +> We do OK with C, but C++ is a much more complex, subtle language. + +C with classes, or the limited subset of C++ that doesn't include +templates, exceptions or RTTI is not much more complex than standard C. + +If you add exceptions to that you will probably not notice anything but +a syntax change in the core, given their current standing. Er, they +probably don't work right in signal handlers, though, because they don't +know anything about them.[6] + +> Is there anybody else to plausibly take on that role? + +I would be happy to look at things that were publicly discussed on the +topic but I don't think I have the experience or the knowledge of the +XEmacs development process to do anything more than that. + +Not, I imagine, that anyone would ask. :) + + Daniel + +Footnotes: +[1] Archaic Unix ported to i386 from a minicomputer over a decade ago. + +[2] The vendor C++ would segfault on anything that had multiple + inheritance. :) + +[3] I found them invaluable in resolving a few Win9x vn WinNT symbol + conflicts, for example, but that's obviously target-specific. + +[4] The current record for STL-generated name length that I have seen + is a symbol 892 characters long... + +[5] The lack of a return code from a class constructor is the killer + issue here. + +[6] This, I believe, varies from vendor to vendor. :) + + +@node Discussion -- Windows External Widget, Discussion -- Packages, Discussion -- Switching to C++, Future Work Discussion @section Discussion -- Windows External Widget @cindex discussion, windows external widget @cindex windows external widget, discussion @@ -23523,29 +25590,34 @@ -Nothing is currently done for external widget support under XEmacs but it should -not be too hard to do and would be a great addition to XEmacs. What you would -probably want to do is create an XEmacs control that has an interface something -like the built-in edit control and which communicates to an existing XEmacs -process using DDE. (Basically you would modify XEmacs so that it registered -itself as a DDE server accepting external widget requests, and then the external -edit control would simply send a DDE request and the result would be a handle of -some sort used for future communication with that particular XEmacs process.) - -There are two basic issues in getting the external widget to work, which are -display and input. Although I am not completely sure, I have a feeling that it -is possible for one process to write into the window of another process, simply -by using that window's HWND handle. If so it should be extremely easy to get the -output working (this is exactly the approach used under Xt). For input, you -would probably again want to do what is done under Xt, which is that the client -widget simply passes all of the appropriate messages to the XEmacs server -process using whatever communication channel was set up, e.g. DDE, and the -XEmacs server processes them normally. Very few modifications would be needed to -the XEmacs source code and all of the necessary modifications could be done -simply by looking for existing external widget code in XEmacs. - -If you are interested in continuing this, I will certainly give you any support -you need along the way. This would be a great project to be added to XEmacs. +Nothing is currently done for external widget support under XEmacs but +it should not be too hard to do and would be a great addition to +XEmacs. What you would probably want to do is create an XEmacs control +that has an interface something like the built-in edit control and +which communicates to an existing XEmacs process using DDE. (Basically +you would modify XEmacs so that it registered itself as a DDE server +accepting external widget requests, and then the external edit control +would simply send a DDE request and the result would be a handle of +some sort used for future communication with that particular XEmacs +process.) + +There are two basic issues in getting the external widget to work, +which are display and input. Although I am not completely sure, I have +a feeling that it is possible for one process to write into the window +of another process, simply by using that window's HWND handle. If so +it should be extremely easy to get the output working (this is exactly +the approach used under Xt). For input, you would probably again want +to do what is done under Xt, which is that the client widget simply +passes all of the appropriate messages to the XEmacs server process +using whatever communication channel was set up, e.g. DDE, and the +XEmacs server processes them normally. Very few modifications would be +needed to the XEmacs source code and all of the necessary +modifications could be done simply by looking for existing external +widget code in XEmacs. + +If you are interested in continuing this, I will certainly give you +any support you need along the way. This would be a great project to +be added to XEmacs.