Some bugs in Perl regexp (core Perl issues)

"Hugo van der Sanden via RT" <perlbug-followup@perl.org> wrote:
:This is caused by a failure of the start_class optimization in the case
:of lookahead, as per the attached comment.
:
:In more detail: at the point study_chunk() attempts to deal with the
:start_class discovered for the lookahead chunk, we have
:SCF_DO_STCLASS_OR set, and_withp has the starting value of ANYOF_EOS |
:ANYOF_UNICODE_ALL, and data->start_class has [a] | ANYOF_EOS.
[...]
:In other words, we need to stack an alternation of ANDs and ORs to cope
:with this situation, and we don't have a mechanism to do that except to
:recurse into study_chunk() some more.
:
:A simpler short-term fix is instead to throw up our hands in this
:situation, and just nullify start_class. I'm not sure exactly how to do
:that, but it seems the more likely to be achievable for 5.10.1.

This patch implements the simple fix, and passes all tests including
Abigail's test cases for the bug.

Yves: note that I've preserved the 'was' code in this chunk, introduced
by you in the patch [1], discussed in the thread [2]. As far as I can
see the 3 lines propagating ANYOF_EOS via 'was' (and the copy of those
3 lines a little later) are simply doing the wrong thing - they seem
to be saying "when we combine two start classes using SCF_DO_STCLASS_AND,
claim that end-of-string is valid if the first class says it would be
even though the second says it wouldn't be". Removing those lines doesn't
cause any test failures - can you remember why you introduced those lines,
and maybe add a test case that fails without them?

Hugo

[1] http://perl5.git.perl.org/perl.git/commit/b515a41db88584b4fd1c30cf890c92d3f9697760
[2] http://groups.google.co.uk/group/perl.perl5.porters/browse_thread/thread/436187077ef96918/f11c3268394abf89

Message-Id: <200907021036.n62Aa8rv029500@zen.crypt.org>
rt.perl.org #56690

2 files changed, 18 insertions, 7 deletions

diff --git a/regcomp.c b/regcomp.c
index 7e80041d0b..50b0632529 100644
--- a/regcomp.c
+++ b/regcomp.c
@@ -3727,11 +3727,22 @@ S_study_chunk(pTHX_ RExC_state_t *pRExC_state, regnode **scanp,
                     data->whilem_c = data_fake.whilem_c;
                 }
                 if (f & SCF_DO_STCLASS_AND) {
-                    const int was = (data->start_class->flags & ANYOF_EOS);
-
-                    cl_and(data->start_class, &intrnl);
-                    if (was)
-                        data->start_class->flags |= ANYOF_EOS;
+		    if (flags & SCF_DO_STCLASS_OR) {
+			/* OR before, AND after: ideally we would recurse with
+			 * data_fake to get the AND applied by study of the
+			 * remainder of the pattern, and then derecurse;
+			 * *** HACK *** for now just treat as "no information".
+			 * See [perl #56690].
+			 */
+			cl_init(pRExC_state, data->start_class);
+		    }  else {
+			/* AND before and after: combine and continue */
+			const int was = (data->start_class->flags & ANYOF_EOS);
+
+			cl_and(data->start_class, &intrnl);
+			if (was)
+			    data->start_class->flags |= ANYOF_EOS;
+		    }
                 }
 	    }
 #if PERL_ENABLE_POSITIVE_ASSERTION_STUDY
diff --git a/t/op/re_tests b/t/op/re_tests
index 8c7381f900..10bee20f94 100644
--- a/t/op/re_tests
+++ b/t/op/re_tests
@@ -1371,8 +1371,8 @@ foo(\h)bar	foo\tbar	y	$1	\t
 .*?(?:(\w)|(\w))x	abx	y	$1-$2	b-
 
 0{50}	000000000000000000000000000000000000000000000000000	y	-	-
-^a?(?=b)b	ab	B	$&	ab	# Bug #56690
-^a*(?=b)b	ab	B	$&	ab	# Bug #56690
+^a?(?=b)b	ab	y	$&	ab	# Bug #56690
+^a*(?=b)b	ab	y	$&	ab	# Bug #56690
 />\d+$ \n/ix	>10\n	y	$&	>10
 />\d+$ \n/ix	>1\n	y	$&	>1
 /\d+$ \n/ix	>10\n	y	$&	10