* doc/lispref/variables.texi (Scope): Mention the availability of lexbind.

(Lexical Binding): New node.
* doc/lispref/eval.texi (Eval): Add `eval's new `lexical' arg.
* lisp/emacs-lisp/cconv.el (cconv-liftwhen): Increase threshold.
(cconv-closure-convert-rec): Convert interactive spec in empty lexenv.
(cconv-analyse-use): Improve unused vars warnings.
(cconv-analyse-form): Analyze interactive spec in empty lexenv.
* lisp/emacs-lisp/bytecomp.el (byte-compile-lambda): Always byte-compile
the interactive spec in lexical-binding mode.
(byte-compile-refresh-preloaded): Don't reload byte-compiler files.
* lisp/custom.el (custom-initialize-default): Use defvar.
(custom-declare-variable): Set the special-variable-p flag.
* lisp/help-fns.el (help-make-usage): Drop leading underscores.
* lisp/dired.el (dired-revert, dired-make-relative): Mark unused args.
(dired-unmark-all-files): Remove unused var `query'.
(dired-overwrite-confirmed): Declare.
(dired-restore-desktop-buffer): Don't use dynamically scoped arg names.
* lisp/mpc.el: Mark unused args.
(mpc--faster-toggle): Remove unused var `songnb'.
* lisp/server.el (server-kill-buffer-running): Move before first use.
* lisp/minibuffer.el: Mark unused args.
* src/callint.c (quotify_arg): Simplify the logic.
(Fcall_interactively): Use lexical binding when evaluating the
interactive spec of a lexically bound function.

4 files changed, 97 insertions, 33 deletions

diff --git a/doc/lispref/ChangeLog b/doc/lispref/ChangeLog
index c5e445cec38..338dbb5e7fd 100644
--- a/doc/lispref/ChangeLog
+++ b/doc/lispref/ChangeLog
@@ -1,3 +1,9 @@
+2011-03-01  Stefan Monnier  <monnier@iro.umontreal.ca>
+
+	* variables.texi (Scope): Mention the availability of lexical scoping.
+	(Lexical Binding): New node.
+	* eval.texi (Eval): Add `eval's new `lexical' arg.
+
 2011-02-25  Stefan Monnier  <monnier@iro.umontreal.ca>
 
 	* vol2.texi (Top):
diff --git a/doc/lispref/elisp.texi b/doc/lispref/elisp.texi
index f7c1d55f6ae..cc3ceb8003c 100644
--- a/doc/lispref/elisp.texi
+++ b/doc/lispref/elisp.texi
@@ -466,7 +466,8 @@ Functions
 * Declaring Functions::     Telling the compiler that a function is defined.
 * Function Safety::         Determining whether a function is safe to call.
 * Related Topics::          Cross-references to specific Lisp primitives
-                              that have a special bearing on how functions work.
+                              that have a special bearing on how
+                              functions work.
 
 Lambda Expressions
 
diff --git a/doc/lispref/eval.texi b/doc/lispref/eval.texi
index d44fe5bb95b..74f3d9c48b9 100644
--- a/doc/lispref/eval.texi
+++ b/doc/lispref/eval.texi
@@ -585,6 +585,11 @@ occurrence in a program being run.  On rare occasions, you may need to
 write code that evaluates a form that is computed at run time, such as
 after reading a form from text being edited or getting one from a
 property list.  On these occasions, use the @code{eval} function.
+Often @code{eval} is not needed and something else should be used instead.
+For example, to get the value of a variable, while @code{eval} works,
+@code{symbol-value} is preferable; or rather than store expressions
+in a property list that then need to go through @code{eval}, it is better to
+store functions instead that are then passed to @code{funcall}.
 
   The functions and variables described in this section evaluate forms,
 specify limits to the evaluation process, or record recently returned
@@ -596,10 +601,13 @@ to store an expression in the data structure and evaluate it.  Using
 functions provides the ability to pass information to them as
 arguments.
 
-@defun eval form
+@defun eval form &optional lexical
 This is the basic function evaluating an expression.  It evaluates
 @var{form} in the current environment and returns the result.  How the
 evaluation proceeds depends on the type of the object (@pxref{Forms}).
+@var{lexical} if non-nil means to evaluate @var{form} using lexical scoping
+rules (@pxref{Lexical Binding}) instead of the default dynamic scoping used
+historically in Emacs Lisp.
 
 Since @code{eval} is a function, the argument expression that appears
 in a call to @code{eval} is evaluated twice: once as preparation before
diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index 0cdcaa84d58..edffb4742ec 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -25,22 +25,22 @@ textual Lisp program is written using the read syntax for the symbol
 representing the variable.
 
 @menu
-* Global Variables::      Variable values that exist permanently, everywhere.
-* Constant Variables::    Certain "variables" have values that never change.
-* Local Variables::       Variable values that exist only temporarily.
-* Void Variables::        Symbols that lack values.
-* Defining Variables::    A definition says a symbol is used as a variable.
-* Tips for Defining::     Things you should think about when you
+* Global Variables::            Variable values that exist permanently, everywhere.
+* Constant Variables::          Certain "variables" have values that never change.
+* Local Variables::             Variable values that exist only temporarily.
+* Void Variables::              Symbols that lack values.
+* Defining Variables::          A definition says a symbol is used as a variable.
+* Tips for Defining::           Things you should think about when you
                             define a variable.
-* Accessing Variables::   Examining values of variables whose names
+* Accessing Variables::         Examining values of variables whose names
                             are known only at run time.
-* Setting Variables::     Storing new values in variables.
-* Variable Scoping::      How Lisp chooses among local and global values.
-* Buffer-Local Variables::  Variable values in effect only in one buffer.
-* File Local Variables::  Handling local variable lists in files.
-* Directory Local Variables:: Local variables common to all files in a directory.
-* Frame-Local Variables:: Frame-local bindings for variables.
-* Variable Aliases::      Variables that are aliases for other variables.
+* Setting Variables::           Storing new values in variables.
+* Variable Scoping::            How Lisp chooses among local and global values.
+* Buffer-Local Variables::      Variable values in effect only in one buffer.
+* File Local Variables::        Handling local variable lists in files.
+* Directory Local Variables::   Local variables common to all files in a directory.
+* Frame-Local Variables::       Frame-local bindings for variables.
+* Variable Aliases::            Variables that are aliases for other variables.
 * Variables with Restricted Values::  Non-constant variables whose value can
                                         @emph{not} be an arbitrary Lisp object.
 @end menu
@@ -437,14 +437,18 @@ this reason, user options must be defined with @code{defvar}.
 This special form defines @var{symbol} as a variable and can also
 initialize and document it.  The definition informs a person reading
 your code that @var{symbol} is used as a variable that might be set or
-changed.  Note that @var{symbol} is not evaluated; the symbol to be
-defined must appear explicitly in the @code{defvar}.
+changed.  It also declares this variable as @dfn{special}, meaning that it
+should always use dynamic scoping rules.  Note that @var{symbol} is not
+evaluated; the symbol to be defined must appear explicitly in the
+@code{defvar}.
 
 If @var{symbol} is void and @var{value} is specified, @code{defvar}
 evaluates it and sets @var{symbol} to the result.  But if @var{symbol}
 already has a value (i.e., it is not void), @var{value} is not even
-evaluated, and @var{symbol}'s value remains unchanged.  If @var{value}
-is omitted, the value of @var{symbol} is not changed in any case.
+evaluated, and @var{symbol}'s value remains unchanged.
+If @var{value} is omitted, the value of @var{symbol} is not changed in any
+case; instead, the only effect of @code{defvar} is to declare locally that this
+variable exists elsewhere and should hence always use dynamic scoping rules.
 
 If @var{symbol} has a buffer-local binding in the current buffer,
 @code{defvar} operates on the default value, which is buffer-independent,
@@ -881,7 +885,7 @@ the others.
 @cindex extent
 @cindex dynamic scoping
 @cindex lexical scoping
-  Local bindings in Emacs Lisp have @dfn{indefinite scope} and
+  By default, local bindings in Emacs Lisp have @dfn{indefinite scope} and
 @dfn{dynamic extent}.  @dfn{Scope} refers to @emph{where} textually in
 the source code the binding can be accessed.  ``Indefinite scope'' means
 that any part of the program can potentially access the variable
@@ -893,6 +897,8 @@ lasts as long as the activation of the construct that established it.
 @dfn{dynamic scoping}.  By contrast, most programming languages use
 @dfn{lexical scoping}, in which references to a local variable must be
 located textually within the function or block that binds the variable.
+Emacs can also support lexical scoping, upon request (@pxref{Lexical
+Binding}).
 
 @cindex CL note---special variables
 @quotation
@@ -901,11 +907,12 @@ dynamically scoped, like all variables in Emacs Lisp.
 @end quotation
 
 @menu
-* Scope::          Scope means where in the program a value is visible.
+* Scope::                       Scope means where in the program a value is visible.
                      Comparison with other languages.
-* Extent::         Extent means how long in time a value exists.
-* Impl of Scope::  Two ways to implement dynamic scoping.
-* Using Scoping::  How to use dynamic scoping carefully and avoid problems.
+* Extent::                      Extent means how long in time a value exists.
+* Impl of Scope::               Two ways to implement dynamic scoping.
+* Using Scoping::               How to use dynamic scoping carefully and avoid problems.
+* Lexical Binding::             
 @end menu
 
 @node Scope
@@ -969,12 +976,12 @@ Here, when @code{foo} is called by @code{binder}, it binds @code{x}.
 by @code{foo} instead of the one bound by @code{binder}.
 @end itemize
 
-Emacs Lisp uses dynamic scoping because simple implementations of
+Emacs Lisp used dynamic scoping by default because simple implementations of
 lexical scoping are slow.  In addition, every Lisp system needs to offer
-dynamic scoping at least as an option; if lexical scoping is the norm,
-there must be a way to specify dynamic scoping instead for a particular
-variable.  It might not be a bad thing for Emacs to offer both, but
-implementing it with dynamic scoping only was much easier.
+dynamic scoping at least as an option; if lexical scoping is the norm, there
+must be a way to specify dynamic scoping instead for a particular variable.
+Nowadays, Emacs offers both, but the default is still to use exclusively
+dynamic scoping.
 
 @node Extent
 @subsection Extent
@@ -1088,6 +1095,48 @@ for inter-function usage.  It also avoids a warning from the byte
 compiler.  Choose the variable's name to avoid name conflicts---don't
 use short names like @code{x}.
 
+
+@node Lexical Binding
+@subsection Use of Lexical Scoping
+
+Emacs Lisp can be evaluated in two different modes: in dynamic binding mode or
+lexical binding mode.  In dynamic binding mode, all local variables use dynamic
+scoping, whereas in lexical binding mode variables that have been declared
+@dfn{special} (i.e., declared with @code{defvar} or @code{defconst}) use
+dynamic scoping and all others use lexical scoping.
+
+@defvar lexical-binding
+When non-nil, evaluation of Lisp code uses lexical scoping for non-special
+local variables instead of dynamic scoping.  If nil, dynamic scoping is used
+for all local variables.  This variable is typically set for a whole Elisp file
+via file local variables (@pxref{File Local Variables}).
+@end defvar
+
+@defun special-variable-p SYMBOL
+Return whether SYMBOL has been declared as a special variable, via
+@code{defvar} or @code{defconst}.
+@end defun
+
+The use of a special variable as a formal argument in a function is generally
+discouraged and its behavior in lexical binding mode is unspecified (it may use
+lexical scoping sometimes and dynamic scoping other times).
+
+Functions like @code{symbol-value}, @code{boundp}, or @code{set} only know
+about dynamically scoped variables, so you cannot get the value of a lexical
+variable via @code{symbol-value} and neither can you change it via @code{set}.
+Another particularity is that code in the body of a @code{defun} or
+@code{defmacro} cannot refer to surrounding lexical variables.
+
+Evaluation of a @code{lambda} expression in lexical binding mode will not just
+return that lambda expression unchanged, as in the dynamic binding case, but
+will instead construct a new object that remembers the current lexical
+environment in which that lambda expression was defined, so that the function
+body can later be evaluated in the proper context.  Those objects are called
+@dfn{closures}.  They are also functions, in the sense that they are accepted
+by @code{funcall}, and they are represented by a cons cell whose @code{car} is
+the symbol @code{closure}.
+
+
 @node Buffer-Local Variables
 @section Buffer-Local Variables
 @cindex variable, buffer-local
@@ -1103,9 +1152,9 @@ local to each terminal, or to each frame.  @xref{Multiple Terminals},
 and @xref{Frame-Local Variables}.)
 
 @menu
-* Intro to Buffer-Local::      Introduction and concepts.
-* Creating Buffer-Local::      Creating and destroying buffer-local bindings.
-* Default Value::              The default value is seen in buffers
+* Intro to Buffer-Local::       Introduction and concepts.
+* Creating Buffer-Local::       Creating and destroying buffer-local bindings.
+* Default Value::               The default value is seen in buffers
                                  that don't have their own buffer-local values.
 @end menu