diff options
| author | Karl Williamson <public@khwilliamson.com> | 2013-09-22 21:36:29 -0600 |
|---|---|---|
| committer | Karl Williamson <public@khwilliamson.com> | 2013-09-24 11:36:19 -0600 |
| commit | cdd87c1d4df41f9a54cccff996fa64d291adcee8 (patch) | |
| tree | 433feb19de18b9fb38c9a4dc8c1830ad3ae4be6c /embed.fnc | |
| parent | fb38762fa113a105b623d0eb7681d2cc03b0c161 (diff) | |
| download | perl-cdd87c1d4df41f9a54cccff996fa64d291adcee8.tar.gz | |
Teach regex optimizer to handle above-Latin1
Until this commit, the regular expression optimizer has essentially
punted on above-Latin1 code points. Under some circumstances, they
would be taken into account, more or less, but often, the generated
synthetic start class would end up matching all above-Latin1 code
points. With the advent of inversion lists, it becomes feasible to
actually fully handle such code points, as inversion lists are a
convenient way to express arbitrary lists of code points and take their
union, intersection, etc. This commit changes the optimizer to use
inversion lists for operating on the code points the synthetic start
class can match.
I don't much understand the overall operation of the optimizer. I'm
told that previous porters found that perturbing it caused unexpected
behaviors. I had promised to get this change in 5.18, but didn't. I'm
trying to get it in early enough into the 5.20 preliminary series that
any problems will surface before 5.20 ships.
This commit doesn't change the macro level logic, but does significantly
change various micro level things. Thus the 'and' and 'or' subroutines
have been rewritten to use inversion lists. I'm pretty confident that
they do what their names suggest. I re-derived the equations for what
these operations should do, getting the same results in some cases, but
extending others where the previous code mostly punted. The derivations
are given in comments in the respective routines.
Some of the code is greatly simplified, as it no longer has to treat
above-Latin1 specially.
It is now feasible for /i matching of above-Latin1 code points to know
explicitly the folds that should be in the synthetic start class. But
more prepatory work needs to be done before putting that into place.
...
Diffstat (limited to 'embed.fnc')
| -rw-r--r-- | embed.fnc | 8 |
1 files changed, 4 insertions, 4 deletions
@@ -2054,10 +2054,10 @@ Es |void |scan_commit |NN const RExC_state_t *pRExC_state \ |NN SSize_t *minlenp \ |int is_inf Es |void |populate_ANYOF_from_invlist|NN regnode *node|NN SV** invlist_ptr -Esn |void |ssc_anything |NN const RExC_state_t *pRExC_state \ +Es |void |ssc_anything |NN const RExC_state_t *pRExC_state \ |NN regnode_ssc *ssc -EsRn |int |ssc_is_anything|NN const regnode_ssc *ssc -Esn |void |ssc_init |NN const RExC_state_t *pRExC_state \ +EsR |int |ssc_is_anything|NN const regnode_ssc *ssc +Es |void |ssc_init |NN const RExC_state_t *pRExC_state \ |NN regnode_ssc *ssc EsR |int |ssc_is_cp_posixl_init|NN const RExC_state_t *pRExC_state \ |NN const regnode_ssc *ssc @@ -2065,7 +2065,7 @@ Es |void |ssc_and |NN const RExC_state_t *pRExC_state \ |NN regnode_ssc *ssc \ |NN const regnode_ssc *and_with Esn |void |ssc_flags_and |NN regnode_ssc *ssc|const U8 and_with -Esn |void |ssc_or |NN const RExC_state_t *pRExC_state \ +Es |void |ssc_or |NN const RExC_state_t *pRExC_state \ |NN regnode_ssc *ssc \ |NN const regnode_ssc *or_with Es |SV* |get_ANYOF_cp_list_for_ssc \ |