revert smartmatch to 5.27.6 behaviour

The pumpking has determined that the CPAN breakage caused by changing
smartmatch [perl #132594] is too great for the smartmatch changes to
stay in for 5.28.

This reverts most of the merge in commit
da4e040f42421764ef069371d77c008e6b801f45.  All core behaviour and
documentation is reverted.  The removal of use of smartmatch from a couple
of tests (that aren't testing smartmatch) remains.  Customisation of
a couple of CPAN modules to make them portable across smartmatch types
remains.  A small bugfix in scope.c also remains.

1 files changed, 414 insertions, 115 deletions

diff --git a/pod/perlsyn.pod b/pod/perlsyn.pod
index a081242c97..74e228d5d8 100644
--- a/pod/perlsyn.pod
+++ b/pod/perlsyn.pod
@@ -118,7 +118,7 @@ as the last item in a statement.
 
 =head2 Statement Modifiers
 X<statement modifier> X<modifier> X<if> X<unless> X<while>
-X<until> X<whereis> X<whereso> X<foreach> X<for>
+X<until> X<when> X<foreach> X<for>
 
 Any simple statement may optionally be followed by a I<SINGLE> modifier,
 just before the terminating semicolon (or block ending).  The possible
@@ -130,8 +130,7 @@ modifiers are:
     until EXPR
     for LIST
     foreach LIST
-    whereis EXPR
-    whereso EXPR
+    when EXPR
 
 The C<EXPR> following the modifier is referred to as the "condition".
 Its truth or falsehood determines how the modifier will behave.
@@ -214,15 +213,18 @@ it.  Future versions of perl might do something different from the
 version of perl you try it out on.  Here be dragons.
 X<my>
 
-The C<whereis> and C<whereso> modifiers are an experimental feature
-that first appeared with this spelling in Perl 5.28.  To use them, you
-should include a C<use feature 'switch'> declaration, or a declaration
-that implies it.  They behave like the full C<whereis> or C<whereso>
-statement with block, described in L</"Switch Statements"> below.
-They executes the statement only if the I<EXPR> is true for C<whereso>,
-or C<$_> smartmatches the I<EXPR> for C<whereis>.  If the statement
-executes, control then implicitly jumps to the end of the dynamically
-enclosing loop (usually a C<given> block).
+The C<when> modifier is an experimental feature that first appeared in Perl
+5.14.  To use it, you should include a C<use v5.14> declaration.
+(Technically, it requires only the C<switch> feature, but that aspect of it
+was not available before 5.14.)  Operative only from within a C<foreach>
+loop or a C<given> block, it executes the statement only if the smartmatch
+C<< $_ ~~ I<EXPR> >> is true.  If the statement executes, it is followed by
+a C<next> from inside a C<foreach> and C<break> from inside a C<given>.
+
+Under the current implementation, the C<foreach> loop can be
+anywhere within the C<when> modifier's dynamic scope, but must be
+within the C<given> block's lexical scope.  This restriction may
+be relaxed in a future release.  See L</"Switch Statements"> below.
 
 =head2 Compound Statements
 X<statement, compound> X<block> X<bracket, curly> X<curly bracket> X<brace>
@@ -258,9 +260,6 @@ The following compound statements may be used to control flow:
 
     given (EXPR) BLOCK
 
-    whereis (EXPR) BLOCK
-    whereso (EXPR) BLOCK
-
     LABEL while (EXPR) BLOCK
     LABEL while (EXPR) BLOCK continue BLOCK
 
@@ -280,9 +279,8 @@ The following compound statements may be used to control flow:
 
     PHASE BLOCK
 
-The experimental C<given>, C<whereis>, and C<whereso> statements are I<not
-automatically enabled>; see L</"Switch Statements"> below for how to do
-so, and the attendant caveats.
+The experimental C<given> statement is I<not automatically enabled>; see
+L</"Switch Statements"> below for how to do so, and the attendant caveats.
 
 Unlike in C and Pascal, in Perl these are all defined in terms of BLOCKs,
 not statements.  This means that the curly brackets are I<required>--no
@@ -610,8 +608,8 @@ The BLOCK construct can be used to emulate case structures.
         $nothing = 1;
     }
 
-You'll also find the C<foreach> loop used to establish a topic for
-a switch:
+You'll also find that C<foreach> loop used to create a topicalizer
+and a switch:
 
     SWITCH:
     for ($var) {
@@ -623,126 +621,96 @@ a switch:
 
 Such constructs are quite frequently used, both because older versions of
 Perl had no official C<switch> statement, and also because the new version
-described immediately below remains experimental.
+described immediately below remains experimental and can sometimes be confusing.
 
 =head2 Switch Statements
 
-X<switch> X<case> X<given> X<whereis> X<whereso>
+X<switch> X<case> X<given> X<when> X<default>
 
-C<given>, C<whereso>, and related keywords make up an experimental feature
-that first appeared in Perl 5.10, but behaved quite differently from
-its present form prior to Perl 5.28.  To use it, you should declare
+Starting from Perl 5.10.1 (well, 5.10.0, but it didn't work
+right), you can say
 
     use feature "switch";
 
-You also get the switch feature whenever you declare that your
+to enable an experimental switch feature.  This is loosely based on an
+old version of a Perl 6 proposal, but it no longer resembles the Perl 6
+construct.   You also get the switch feature whenever you declare that your
 code prefers to run under a version of Perl that is 5.10 or later.  For
 example:
 
     use v5.14;
 
-Under the "switch" feature, Perl gains the experimental keywords C<given>,
-C<whereis>, and C<whereso>.  Starting from Perl 5.16, one can
-prefix the switch keywords with C<CORE::> to access the feature without
-a C<use feature> statement.
-
-The "switch" feature is considered highly experimental; it is subject
-to change with little notice.  Uses of the C<given>, C<whereis>, and
-C<whereso> keywords will by default warn about their experimental status.
-These warnings are in the same category as warnings about the C<~~>
-(smartmatch) operator being experimental.
-
-The keywords C<given> and C<whereis> or C<whereso>
-are analogous to C<switch> and C<case>
-in C.  They're meant to be used together, but can actually be used
-independently and mixed with other kinds of compound statement.
-
-C<given> evaluates its argument in scalar context, and executes its block
-with the C<$_> variable locally aliased to the result of evaluating the
-argument expression.  It is much like a C<foreach> loop that always has
-exactly one item to iterate over.
-A C<given> construct even counts as a one-iteration loop for the purposes
-of loop control, so the C<redo> operator can be used to restart its block,
-and C<next> or C<last> can be used to exit the block early.
-
-C<whereso> evaluates its argument as a truth value.  If the argument
-was false then it does not execute its block, and proceeds to the
-following statement.  If the argument was true, it executes the block,
-then implicitly performs a C<next>, jumping to the end of the closest
-dynamically enclosing C<given> block or other kind of loop.
-
-C<whereis> evaluates its argument and uses it as a smartmatch object,
-checking whether C<$_> matches it.  If C<$_> did not match then it does
-not execute its block, and proceeds to the following statement.  If C<$_>
-did match, it executes the block, then implicitly performs a C<next>,
-jumping to the end of the closest dynamically enclosing C<given> block
-or other kind of loop.  This is exactly like C<whereso>, except for the
-implicit use of smartmatch.
-
-Putting this together, the code in the previous section could be
-rewritten as
+Under the "switch" feature, Perl gains the experimental keywords
+C<given>, C<when>, C<default>, C<continue>, and C<break>.
+Starting from Perl 5.16, one can prefix the switch
+keywords with C<CORE::> to access the feature without a C<use feature>
+statement.  The keywords C<given> and
+C<when> are analogous to C<switch> and
+C<case> in other languages -- though C<continue> is not -- so the code
+in the previous section could be rewritten as
 
     use v5.10.1;
-    given ($var) {
-        whereso (/^abc/) { $abc = 1 }
-        whereso (/^def/) { $def = 1 }
-        whereso (/^xyz/) { $xyz = 1 }
-        $nothing = 1;
+    for ($var) {
+        when (/^abc/) { $abc = 1 }
+        when (/^def/) { $def = 1 }
+        when (/^xyz/) { $xyz = 1 }
+        default       { $nothing = 1 }
     }
 
-Or if you prefer the modifier form of C<whereso>, it can be written with
-less punctuation as
+The C<foreach> is the non-experimental way to set a topicalizer.
+If you wish to use the highly experimental C<given>, that could be
+written like this:
 
-    use v5.14;
+    use v5.10.1;
     given ($var) {
-        $abc = 1 whereso /^abc/;
-        $def = 1 whereso /^def/;
-        $xyz = 1 whereso /^xyz/;
-        $nothing = 1;
+        when (/^abc/) { $abc = 1 }
+        when (/^def/) { $def = 1 }
+        when (/^xyz/) { $xyz = 1 }
+        default       { $nothing = 1 }
     }
 
-You can use the C<continue> keyword to exit a C<whereis> or C<whereso>
-block, proceeding to the following statement.  This is most commonly
-done last thing inside the block, to override the implicit C<next>.
-For example
+As of 5.14, that can also be written this way:
 
-    given($foo) {
-        whereso (/x/) { say '$foo contains an x'; continue }
-        whereso (/y/) { say '$foo contains a y'            }
-        say '$foo does not contain a y';
+    use v5.14;
+    for ($var) {
+        $abc = 1 when /^abc/;
+        $def = 1 when /^def/;
+        $xyz = 1 when /^xyz/;
+        default { $nothing = 1 }
     }
 
-When a C<given> statement is executed in a position where it will provide
-a value, for example when it's the last statement of a subroutine and
-so providing the subroutine's return value, it evaluates to:
+Or if you don't care to play it safe, like this:
 
-=over 4
-
-=item *
-
-An empty list as soon as an explicit C<next> or C<last> is encountered.
-
-=item *
-
-The value of the last evaluated expression of the successful
-C<whereis> or C<whereso> clause, if there happens to be one.
-
-=item *
-
-The value of the last evaluated expression of the C<given> block if no
-condition is true.
-
-=back
-
-In both last cases, the last expression is evaluated in the context that
-was applied to the C<given> block.
-Note that, unlike C<if> and C<unless>,
-failed C<whereis>/C<whereso> statements always
-evaluate to an empty list.
+    use v5.14;
+    given ($var) {
+        $abc = 1 when /^abc/;
+        $def = 1 when /^def/;
+        $xyz = 1 when /^xyz/;
+        default { $nothing = 1 }
+    }
 
-On versions of Perl preceding Perl 5.28, C<given> and the related keywords
-behave quite differently from their present behaviour.  If your code needs
-to run on older versions, avoid C<given>, C<whereis>, and C<whereso>.
+The arguments to C<given> and C<when> are in scalar context,
+and C<given> assigns the C<$_> variable its topic value.
+
+Exactly what the I<EXPR> argument to C<when> does is hard to describe
+precisely, but in general, it tries to guess what you want done.  Sometimes
+it is interpreted as C<< $_ ~~ I<EXPR> >>, and sometimes it is not.  It
+also behaves differently when lexically enclosed by a C<given> block than
+it does when dynamically enclosed by a C<foreach> loop.  The rules are far
+too difficult to understand to be described here.  See L</"Experimental Details
+on given and when"> later on.
+
+Due to an unfortunate bug in how C<given> was implemented between Perl 5.10
+and 5.16, under those implementations the version of C<$_> governed by
+C<given> is merely a lexically scoped copy of the original, not a
+dynamically scoped alias to the original, as it would be if it were a
+C<foreach> or under both the original and the current Perl 6 language
+specification.  This bug was fixed in Perl 5.18 (and lexicalized C<$_> itself
+was removed in Perl 5.24).
+
+If your code still needs to run on older versions,
+stick to C<foreach> for your topicalizer and
+you will be less unhappy.
 
 =head2 Goto
 X<goto>
@@ -953,4 +921,335 @@ shell:
     __END__
     foo at goop line 345.
 
+=head2 Experimental Details on given and when
+
+As previously mentioned, the "switch" feature is considered highly
+experimental; it is subject to change with little notice.  In particular,
+C<when> has tricky behaviours that are expected to change to become less
+tricky in the future.  Do not rely upon its current (mis)implementation.
+Before Perl 5.18, C<given> also had tricky behaviours that you should still
+beware of if your code must run on older versions of Perl.
+
+Here is a longer example of C<given>:
+
+    use feature ":5.10";
+    given ($foo) {
+        when (undef) {
+            say '$foo is undefined';
+        }
+        when ("foo") {
+            say '$foo is the string "foo"';
+        }
+        when ([1,3,5,7,9]) {
+            say '$foo is an odd digit';
+            continue; # Fall through
+        }
+        when ($_ < 100) {
+            say '$foo is numerically less than 100';
+        }
+        when (\&complicated_check) {
+            say 'a complicated check for $foo is true';
+        }
+        default {
+            die q(I don't know what to do with $foo);
+        }
+    }
+
+Before Perl 5.18, C<given(EXPR)> assigned the value of I<EXPR> to
+merely a lexically scoped I<B<copy>> (!) of C<$_>, not a dynamically
+scoped alias the way C<foreach> does.  That made it similar to
+
+        do { my $_ = EXPR; ... }
+
+except that the block was automatically broken out of by a successful
+C<when> or an explicit C<break>.  Because it was only a copy, and because
+it was only lexically scoped, not dynamically scoped, you could not do the
+things with it that you are used to in a C<foreach> loop.  In particular,
+it did not work for arbitrary function calls if those functions might try
+to access $_.  Best stick to C<foreach> for that.
+
+Most of the power comes from the implicit smartmatching that can
+sometimes apply.  Most of the time, C<when(EXPR)> is treated as an
+implicit smartmatch of C<$_>, that is, C<$_ ~~ EXPR>.  (See
+L<perlop/"Smartmatch Operator"> for more information on smartmatching.)
+But when I<EXPR> is one of the 10 exceptional cases (or things like them)
+listed below, it is used directly as a boolean.
+
+=over 4
+
+=item Z<>1.
+
+A user-defined subroutine call or a method invocation.
+
+=item Z<>2.
+
+A regular expression match in the form of C</REGEX/>, C<$foo =~ /REGEX/>,
+or C<$foo =~ EXPR>.  Also, a negated regular expression match in
+the form C<!/REGEX/>, C<$foo !~ /REGEX/>, or C<$foo !~ EXPR>.
+
+=item Z<>3.
+
+A smart match that uses an explicit C<~~> operator, such as C<EXPR ~~ EXPR>.
+
+B<NOTE:> You will often have to use C<$c ~~ $_> because the default case
+uses C<$_ ~~ $c> , which is frequentlythe opposite of what you want.
+
+=item Z<>4.
+
+A boolean comparison operator such as C<$_ E<lt> 10> or C<$x eq "abc">.  The
+relational operators that this applies to are the six numeric comparisons
+(C<< < >>, C<< > >>, C<< <= >>, C<< >= >>, C<< == >>, and C<< != >>), and
+the six string comparisons (C<lt>, C<gt>, C<le>, C<ge>, C<eq>, and C<ne>).
+
+=item Z<>5.
+
+At least the three builtin functions C<defined(...)>, C<exists(...)>, and
+C<eof(...)>.  We might someday add more of these later if we think of them.
+
+=item Z<>6.
+
+A negated expression, whether C<!(EXPR)> or C<not(EXPR)>, or a logical
+exclusive-or, C<(EXPR1) xor (EXPR2)>.  The bitwise versions (C<~> and C<^>)
+are not included.
+
+=item Z<>7.
+
+A filetest operator, with exactly 4 exceptions: C<-s>, C<-M>, C<-A>, and
+C<-C>, as these return numerical values, not boolean ones.  The C<-z>
+filetest operator is not included in the exception list.
+
+=item Z<>8.
+
+The C<..> and C<...> flip-flop operators.  Note that the C<...> flip-flop
+operator is completely different from the C<...> elliptical statement
+just described.
+
+=back
+
+In those 8 cases above, the value of EXPR is used directly as a boolean, so
+no smartmatching is done.  You may think of C<when> as a smartsmartmatch.
+
+Furthermore, Perl inspects the operands of logical operators to
+decide whether to use smartmatching for each one by applying the
+above test to the operands:
+
+=over 4
+
+=item Z<>9.
+
+If EXPR is C<EXPR1 && EXPR2> or C<EXPR1 and EXPR2>, the test is applied
+I<recursively> to both EXPR1 and EXPR2.
+Only if I<both> operands also pass the
+test, I<recursively>, will the expression be treated as boolean.  Otherwise,
+smartmatching is used.
+
+=item Z<>10.
+
+If EXPR is C<EXPR1 || EXPR2>, C<EXPR1 // EXPR2>, or C<EXPR1 or EXPR2>, the
+test is applied I<recursively> to EXPR1 only (which might itself be a
+higher-precedence AND operator, for example, and thus subject to the
+previous rule), not to EXPR2.  If EXPR1 is to use smartmatching, then EXPR2
+also does so, no matter what EXPR2 contains.  But if EXPR2 does not get to
+use smartmatching, then the second argument will not be either.  This is
+quite different from the C<&&> case just described, so be careful.
+
+=back
+
+These rules are complicated, but the goal is for them to do what you want
+(even if you don't quite understand why they are doing it).  For example:
+
+    when (/^\d+$/ && $_ < 75) { ... }
+
+will be treated as a boolean match because the rules say both
+a regex match and an explicit test on C<$_> will be treated
+as boolean.
+
+Also:
+
+    when ([qw(foo bar)] && /baz/) { ... }
+
+will use smartmatching because only I<one> of the operands is a boolean:
+the other uses smartmatching, and that wins.
+
+Further:
+
+    when ([qw(foo bar)] || /^baz/) { ... }
+
+will use smart matching (only the first operand is considered), whereas
+
+    when (/^baz/ || [qw(foo bar)]) { ... }
+
+will test only the regex, which causes both operands to be
+treated as boolean.  Watch out for this one, then, because an
+arrayref is always a true value, which makes it effectively
+redundant.  Not a good idea.
+
+Tautologous boolean operators are still going to be optimized
+away.  Don't be tempted to write
+
+    when ("foo" or "bar") { ... }
+
+This will optimize down to C<"foo">, so C<"bar"> will never be considered (even
+though the rules say to use a smartmatch
+on C<"foo">).  For an alternation like
+this, an array ref will work, because this will instigate smartmatching:
+
+    when ([qw(foo bar)] { ... }
+
+This is somewhat equivalent to the C-style switch statement's fallthrough
+functionality (not to be confused with I<Perl's> fallthrough
+functionality--see below), wherein the same block is used for several
+C<case> statements.
+
+Another useful shortcut is that, if you use a literal array or hash as the
+argument to C<given>, it is turned into a reference.  So C<given(@foo)> is
+the same as C<given(\@foo)>, for example.
+
+C<default> behaves exactly like C<when(1 == 1)>, which is
+to say that it always matches.
+
+=head3 Breaking out
+
+You can use the C<break> keyword to break out of the enclosing
+C<given> block.  Every C<when> block is implicitly ended with
+a C<break>.
+
+=head3 Fall-through
+
+You can use the C<continue> keyword to fall through from one
+case to the next immediate C<when> or C<default>:
+
+    given($foo) {
+        when (/x/) { say '$foo contains an x'; continue }
+        when (/y/) { say '$foo contains a y'            }
+        default    { say '$foo does not contain a y'    }
+    }
+
+=head3 Return value
+
+When a C<given> statement is also a valid expression (for example,
+when it's the last statement of a block), it evaluates to:
+
+=over 4
+
+=item *
+
+An empty list as soon as an explicit C<break> is encountered.
+
+=item *
+
+The value of the last evaluated expression of the successful
+C<when>/C<default> clause, if there happens to be one.
+
+=item *
+
+The value of the last evaluated expression of the C<given> block if no
+condition is true.
+
+=back
+
+In both last cases, the last expression is evaluated in the context that
+was applied to the C<given> block.
+
+Note that, unlike C<if> and C<unless>, failed C<when> statements always
+evaluate to an empty list.
+
+    my $price = do {
+        given ($item) {
+            when (["pear", "apple"]) { 1 }
+            break when "vote";      # My vote cannot be bought
+            1e10  when /Mona Lisa/;
+            "unknown";
+        }
+    };
+
+Currently, C<given> blocks can't always
+be used as proper expressions.  This
+may be addressed in a future version of Perl.
+
+=head3 Switching in a loop
+
+Instead of using C<given()>, you can use a C<foreach()> loop.
+For example, here's one way to count how many times a particular
+string occurs in an array:
+
+    use v5.10.1;
+    my $count = 0;
+    for (@array) {
+        when ("foo") { ++$count }
+    }
+    print "\@array contains $count copies of 'foo'\n";
+
+Or in a more recent version:
+
+    use v5.14;
+    my $count = 0;
+    for (@array) {
+        ++$count when "foo";
+    }
+    print "\@array contains $count copies of 'foo'\n";
+
+At the end of all C<when> blocks, there is an implicit C<next>.
+You can override that with an explicit C<last> if you're
+interested in only the first match alone.
+
+This doesn't work if you explicitly specify a loop variable, as
+in C<for $item (@array)>.  You have to use the default variable C<$_>.
+
+=head3 Differences from Perl 6
+
+The Perl 5 smartmatch and C<given>/C<when> constructs are not compatible
+with their Perl 6 analogues.  The most visible difference and least
+important difference is that, in Perl 5, parentheses are required around
+the argument to C<given()> and C<when()> (except when this last one is used
+as a statement modifier).  Parentheses in Perl 6 are always optional in a
+control construct such as C<if()>, C<while()>, or C<when()>; they can't be
+made optional in Perl 5 without a great deal of potential confusion,
+because Perl 5 would parse the expression
+
+    given $foo {
+        ...
+    }
+
+as though the argument to C<given> were an element of the hash
+C<%foo>, interpreting the braces as hash-element syntax.
+
+However, their are many, many other differences.  For example,
+this works in Perl 5:
+
+    use v5.12;
+    my @primary = ("red", "blue", "green");
+
+    if (@primary ~~ "red") {
+        say "primary smartmatches red";
+    }
+
+    if ("red" ~~ @primary) {
+        say "red smartmatches primary";
+    }
+
+    say "that's all, folks!";
+
+But it doesn't work at all in Perl 6.  Instead, you should
+use the (parallelizable) C<any> operator:
+
+   if any(@primary) eq "red" {
+       say "primary smartmatches red";
+   }
+
+   if "red" eq any(@primary) {
+       say "red smartmatches primary";
+   }
+
+The table of smartmatches in L<perlop/"Smartmatch Operator"> is not
+identical to that proposed by the Perl 6 specification, mainly due to
+differences between Perl 6's and Perl 5's data models, but also because
+the Perl 6 spec has changed since Perl 5 rushed into early adoption.
+
+In Perl 6, C<when()> will always do an implicit smartmatch with its
+argument, while in Perl 5 it is convenient (albeit potentially confusing) to
+suppress this implicit smartmatch in various rather loosely-defined
+situations, as roughly outlined above.  (The difference is largely because
+Perl 5 does not have, even internally, a boolean type.)
+
 =cut