rebase to 9.0

doc/expr.n

@@ -13,12 +13,11 @@
 .SH NAME
 expr \- Evaluate an expression
 .SH SYNOPSIS
-\fBexpr \fIarg \fR?\fIarg arg ...\fR?
+\fBexpr \fIexpression\fR
 .BE
 .SH DESCRIPTION
 .PP
-Concatenates \fIarg\fRs, separated by a space, into an expression, and evaluates
-that expression, returning its value.
+Evaluates the expression, \fIexpression\fR, returning its value.
 The operators permitted in an expression include a subset of
 the operators permitted in C expressions.  For those operators
 common to both Tcl and C, Tcl applies the same meaning and precedence
@@ -28,7 +27,7 @@ floating-point value, but may also be a non-numeric value.
 For example, the expression
 .PP
 .CS
-\fBexpr\fR 8.2 + 6
+\fBexpr\fR {8.2 + 6}
 .CE
 .PP
 evaluates to 14.2.
@@ -41,6 +40,12 @@ When the result of expression is an integer, it is in decimal form, and when
 the result is a floating-point number, it is in the form produced by the
 \fB%g\fR format specifier of \fBformat\fR.
 .PP
+.VS "9.0, TIP 526"
+Unlike with prior versions of Tcl, from Tcl 9.0 onwards, the \fBexpr\fR
+command only accepts a single argument instead of concatenating an arbitrary
+number of arguments to form an expression.
+See \fBCOMPATIBILITY WITH TCL 8.*\fR below for details.
+.VE "9.0, TIP 526"
 .VS "TIP 582"
 At any point in the expression except within double quotes or braces, \fB#\fR
 is the beginning of a comment, which lasts to the end of the line or
@@ -282,7 +287,7 @@ only one of \fB[a]\fR or \fB[b]\fR is evaluated,
 depending on the value of \fB$v\fR.  This is not true of the normal Tcl parser,
 so it is normally recommended to enclose the arguments to \fBexpr\fR in braces.
 Without braces, as in
-\fBexpr\fR $v ? [a] : [b]
+\fBexpr\fR $v?[a]:[b]
 both \fB[a]\fR and \fB[b]\fR are evaluated before \fBexpr\fR is even called.
 .PP
 For more details on the results
@@ -368,16 +373,16 @@ A floating-point result can be distinguished from an integer result by the
 presence of either
 .QW \fB.\fR
 or
-.QW \fBe\fR
+.QW \fBe\fR .
 .PP
-. For example,
+For example,
 .PP
 .CS
 \fBexpr\fR {20.0/5.0}
 .CE
 .PP
 returns \fB4.0\fR, not \fB4\fR.
-.SH "PERFORMANCE CONSIDERATIONS"
+.SH "PERFORMANCE AND COMPATIBILITY"
 .PP
 Where an expression contains syntax that Tcl would otherwise perform
 substitutions on, enclosing an expression in braces or otherwise quoting it
@@ -390,7 +395,7 @@ In the following example, the value of the expression is 11 because the Tcl pars
 substitutes \fB$b\fR and \fBexpr\fR then substitutes \fB$a\fR as part
 of evaluating the expression
 .QW "$a + 2*4" .
-Enclosing the
+Simply enclosing the
 expression in braces would result in a syntax error as \fB$b\fR does
 not evaluate to a numeric value.
 .PP
@@ -413,25 +418,68 @@ ensure that it contains no substitutions or that it is enclosed in braces or
 otherwise quoted to prevent Tcl from performing substitutions, allowing
 \fBexpr\fR to perform them instead.
 .PP
-If it is necessary to include a non-constant expression string within the
-wider context of an otherwise-constant expression, the most efficient
-technique is to put the varying part inside a recursive \fBexpr\fR, as this at
-least allows for the compilation of the outer part, though it does mean that
-the varying part must itself be evaluated as a separate expression. Thus, in
-this example the result is 20 and the outer expression benefits from fully
-cached bytecode compilation.
-.PP
-.CS
-set a 3
-set b {$a + 2}
-\fBexpr\fR {[\fBexpr\fR $b] * 4}
-.CE
-.PP
 In general, you should enclose your expression in braces wherever possible,
 and where not possible, the argument to \fBexpr\fR should be an expression
 defined elsewhere as simply as possible. It is usually more efficient and
 safer to use other techniques (e.g., the commands in the \fBtcl::mathop\fR
-namespace) than it is to do complex expression generation.
+namespace) than it is to do complex expression generation. Storing a
+complex expression in a variable and using that with \fBexpr\fR is a
+case which the bytecode compiler also supports well, provided that
+expression is always evaluated in the same stack context (though the
+expression could be conveyed across procedure calls and through
+\fBuplevel\fR calls in between).
+.SS "COMPATIBILITY WITH TCL 8.*"
+.PP
+.VS "9.0, TIP 526"
+In Tcl 8 and older, the \fBexpr\fR command would concatenate multiple
+arguments to form the expression; if this behavior is desired in Tcl 9.0
+onwards, explicit use of \fBconcat\fR is required (this is also compatible
+with earlier versions of Tcl) or a double-quoted argument could be used (as
+whitespace is not generally important in expressions). Note that this style of
+programming is often a source of bugs, including security bugs, so it is
+\fInot recommended\fR. For example, instead of this version (supported in Tcl
+8 and before):
+.PP
+.CS
+set a 3
+set b {$a + 2}
+\fBexpr\fR $b * 4
+.CE
+.PP
+you might write:
+.PP
+.CS
+# ...
+\fBexpr\fR [concat $b * 4]
+.CE
+.PP
+or:
+.PP
+.CS
+# ...
+\fBexpr\fR "$b * 4"
+.CE
+.PP
+Both of the above cases substitute \fB$b\fR prior to evaluating the
+expression, and do the building of the expression dynamically making it
+difficult to avoid compiling the expression on every call to \fBexpr\fR.
+.PP
+However, some use cases that \fIare\fR supported may be preferable, such as
+embedding the use of \fBexpr\fR with a variable argument inside an otherwise
+constant expression:
+.PP
+.CS
+# ...
+\fBexpr\fR {[\fBexpr\fR $b] * 4}
+.CE
+.PP
+This changes what value is calculated (in this case, producing \fB20\fR rather
+than \fB11\fR) because they force the inner expression to be evaluated
+independently of the outer one; such changes may actually fix more bugs than
+they cause! Note that this allows the outer expression to be fully compiled,
+and enables compilation of the inner expression too (which can be cached in
+some cases).
+.VE "9.0, TIP 526"
 .SH EXAMPLES
 .PP
 A numeric comparison whose result is 1:

generic/tclCmdAH.c

@@ -975,20 +975,15 @@ TclNRExprObjCmd(
 {
     Tcl_Obj *resultPtr, *objPtr;
 
-    if (objc < 2) {
-	Tcl_WrongNumArgs(interp, 1, objv, "arg ?arg ...?");
+    if (objc != 2) {
+	Tcl_WrongNumArgs(interp, 1, objv, "expression");
 	return TCL_ERROR;
     }
 
     TclNewObj(resultPtr);
     Tcl_IncrRefCount(resultPtr);
-    if (objc == 2) {
-	objPtr = objv[1];
-	TclNRAddCallback(interp, ExprCallback, resultPtr, NULL, NULL, NULL);
-    } else {
-	objPtr = Tcl_ConcatObj(objc-1, objv+1);
-	TclNRAddCallback(interp, ExprCallback, resultPtr, objPtr, NULL, NULL);
-    }
+    objPtr = objv[1];
+    TclNRAddCallback(interp, ExprCallback, resultPtr, NULL, NULL, NULL);
 
     return Tcl_NRExprObj(interp, objPtr, resultPtr);
 }
@@ -1000,11 +995,6 @@ ExprCallback(
     int result)
 {
     Tcl_Obj *resultPtr = (Tcl_Obj *)data[0];
-    Tcl_Obj *objPtr = (Tcl_Obj *)data[1];
-
-    if (objPtr != NULL) {
-	Tcl_DecrRefCount(objPtr);
-    }
 
     if (result == TCL_OK) {
 	Tcl_SetObjResult(interp, resultPtr);

generic/tclCompCmds.c

@@ -2427,7 +2427,7 @@ TclCompileExprCmd(
 {
     Tcl_Token *firstWordPtr;
 
-    if (parsePtr->numWords == 1) {
+    if (parsePtr->numWords != 2) {
 	return TCL_ERROR;
     }
 

tests/compExpr-old.test

@@ -134,16 +134,11 @@ catch {unset a b i x}
 
 test compExpr-old-1.1 {TclCompileExprCmd: no expression} {
     list [catch {expr  } msg] $msg
-} {1 {wrong # args: should be "expr arg ?arg ...?"}}
+} {1 {wrong # args: should be "expr expression"}}
 test compExpr-old-1.2 {TclCompileExprCmd: one expression word} {
     expr -25
 } -25
-test compExpr-old-1.3 {TclCompileExprCmd: two expression words} {
-    expr -8.2   -6
-} -14.2
-test compExpr-old-1.4 {TclCompileExprCmd: five expression words} {
-    expr 20 - 5 +10 -7
-} 18
+# 1.3 and 1.4 no longer apply
 test compExpr-old-1.5 {TclCompileExprCmd: quoted expression word} {
     expr "0005"
 } 5
@@ -181,7 +176,7 @@ test compExpr-old-1.13 {TclCompileExprCmd: second level of substitutions in expr
 } foo
 test compExpr-old-1.14 {TclCompileExprCmd: second level of substitutions in expr with comparison as top-level operator} {
     set a xxx
-    set x 2;  set b {$x};  set a [expr $b == 2]
+    set x 2;  set b {$x};  set a [expr $b==2]
     set a
 } 1
 

tests/expr.test

@@ -137,18 +137,16 @@ proc do_twelve_days {} {
 
 catch {unset a b i x}
 
-test expr-1.1 {TclCompileExprCmd: no expression} {
-    list [catch {expr  } msg] $msg
-} {1 {wrong # args: should be "expr arg ?arg ...?"}}
+test expr-1.1 {TclCompileExprCmd: no expression} -body {
+    expr
+} -returnCodes error -result {wrong # args: should be "expr expression"}
 test expr-1.2 {TclCompileExprCmd: one expression word} {
-    expr -25
-} -25
-test expr-1.3 {TclCompileExprCmd: two expression words} {
-    expr -8.2   -6
-} -14.2
-test expr-1.4 {TclCompileExprCmd: five expression words} {
-    expr 20 - 5 +10 -7
-} 18
+    expr +25
+} 25
+test expr-1.3 {TclCompileExprCmd: two arguments} -body {
+    expr +1 +2
+} -returnCodes error -result {wrong # args: should be "expr expression"}
+# 1.4 no longer applies
 test expr-1.5 {TclCompileExprCmd: quoted expression word} {
     expr "0005"
 } 5
@@ -186,12 +184,12 @@ test expr-1.13 {TclCompileExprCmd: second level of substitutions in expr not in
 } foo
 test expr-1.14 {TclCompileExprCmd: second level of substitutions in expr with comparison as top-level operator} {
     set a xxx
-    set x 2;  set b {$x};  set a [expr $b == 2]
+    set x 2;  set b {$x};  set a [expr $b==2]
     set a
 } 1
 test expr-1.15 {TclCompileExprCmd: second level of substitutions in expr with comparison as top-level operator} {
     set a xxx
-    set x 2;  set b {$x};  set a [expr $b eq 2]
+    set x 2;  set b {$x};  set a [expr "$b eq 2"]
     set a
 } 1
 
@@ -796,10 +794,10 @@ test expr-20.5 {proper double evaluation compilation, working case} {
     expr 1?{$a}:0
 } yellow
 test expr-20.6 {handling of compile error in trial compile} {
-    list [catch {expr + {[incr]}} msg] $msg
+    list [catch {expr {+[incr]}} msg] $msg
 } {1 {wrong # args: should be "incr varName ?increment?"}}
 test expr-20.7 {handling of compile error in runtime case} {
-    list [catch {expr + {[error foo]}} msg] $msg
+    list [catch {expr {+[error foo]}} msg] $msg
 } {1 foo}
 
 # Test for non-numeric boolean literal handling
@@ -5707,21 +5705,21 @@ foreach s {no false off} {
     }
     incr i
 }
-test expr-31.3.4.0 {boolean conversion} {expr bool(n)} 0
-test expr-31.3.5.0 {boolean conversion} {expr bool("n")} 0
-test expr-31.4.4.0 {boolean conversion} {expr bool(f)} 0
-test expr-31.4.5.0 {boolean conversion} {expr bool("f")} 0
-test expr-31.6  {boolean conversion} {expr bool(-1 + 1)} 0
-test expr-31.7  {boolean conversion} {expr bool(0 + 1)} 1
-test expr-31.8  {boolean conversion} {expr bool(0.0)} 0
-test expr-31.9  {boolean conversion} {expr bool(0x0)} 0
-test expr-31.10 {boolean conversion} {expr bool(wide(0))} 0
-test expr-31.11 {boolean conversion} {expr bool(5.0)} 1
-test expr-31.12 {boolean conversion} {expr bool(5)} 1
-test expr-31.13 {boolean conversion} {expr bool(0x5)} 1
-test expr-31.14 {boolean conversion} {expr bool(wide(5))} 1
+test expr-31.3.4.0 {boolean conversion} {expr {bool(n)}} 0
+test expr-31.3.5.0 {boolean conversion} {expr {bool("n")}} 0
+test expr-31.4.4.0 {boolean conversion} {expr {bool(f)}} 0
+test expr-31.4.5.0 {boolean conversion} {expr {bool("f")}} 0
+test expr-31.6  {boolean conversion} {expr {bool(-1 + 1)}} 0
+test expr-31.7  {boolean conversion} {expr {bool(0 + 1)}} 1
+test expr-31.8  {boolean conversion} {expr {bool(0.0)}} 0
+test expr-31.9  {boolean conversion} {expr {bool(0x0)}} 0
+test expr-31.10 {boolean conversion} {expr {bool(wide(0))}} 0
+test expr-31.11 {boolean conversion} {expr {bool(5.0)}} 1
+test expr-31.12 {boolean conversion} {expr {bool(5)}} 1
+test expr-31.13 {boolean conversion} {expr {bool(0x5)}} 1
+test expr-31.14 {boolean conversion} {expr {bool(wide(5))}} 1
 test expr-31.15 {boolean conversion} -body {
-    expr bool("fred")
+    expr {bool("fred")}
 } -returnCodes error -match glob -result *
 
 test expr-32.1 {expr mod basics} {