diff options
author | Márcio Almada <marcio3w@gmail.com> | 2015-03-09 02:24:50 -0300 |
---|---|---|
committer | Márcio Almada <marcio3w@gmail.com> | 2015-04-30 03:03:29 -0300 |
commit | a75decd452caea67a8afc175510ea5cf5c4c3aca (patch) | |
tree | 53e5bca3ea5e68324dec55357ba1a43eb7ff6c0c /ext/tokenizer/tokenizer.c | |
parent | a54e1237ec7d51e063f8bbf7c94241245573dd9e (diff) | |
download | php-git-a75decd452caea67a8afc175510ea5cf5c4c3aca.tar.gz |
implement context sensitive language with lexical feedback
The implementation has no regression risks, has an even smaller footprint
compared to the previous attempt involving a pure lexical approach, is higly
predictable and higly configurable.
To turn a word semi-reserved you only need to edit the "SEMI_RESERVED" parser rule,
it's an inclusive list of all the words that should be matched as T_STRING on specific contexts.
Example:
```
method_modifiers function returns_ref indentifier '(' parameter_list ')' ...
```
instead of:
```
method_modifiers function returns_ref T_STRING '(' parameter_list ')' ...
```
TODO: port ext tokenizer
Diffstat (limited to 'ext/tokenizer/tokenizer.c')
0 files changed, 0 insertions, 0 deletions