diff options
| author | Joseph Langley <mightyjo@gmail.com> | 2021-06-02 20:17:38 -0400 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-06-02 20:17:38 -0400 |
| commit | a297d520ae8f0d718a7aab5975550eb8dc1f2cae (patch) | |
| tree | c2acd3a13e89d47ff7018a6914bbfda6db13108b | |
| parent | 04c5b7c9209801aa1bdbf279ccdcde0d57874a55 (diff) | |
| parent | 5d93876e2249adec9e3a1e8f02e28bdf9a64e4c1 (diff) | |
| download | flex-git-a297d520ae8f0d718a7aab5975550eb8dc1f2cae.tar.gz | |
Merge pull request #16 from eric-s-raymond/retargeting
Reviewing ESR's Retargeting branch from westes/flex
134 files changed, 15868 insertions, 9369 deletions
diff --git a/Makefile.am b/Makefile.am index 8dac6f5..409ab7d 100644 --- a/Makefile.am +++ b/Makefile.am @@ -89,5 +89,8 @@ install-exec-hook: cd $(DESTDIR)$(bindir) && \ $(LN_S) -f flex$(EXEEXT) flex++$(EXEEXT) +lint: + shellcheck -f gcc tests/*.sh src/*.sh *.sh + .PHONY: libfl install-libfl uninstall-libfl clean-libfl \ - ChangeLog indent + ChangeLog indent lint @@ -0,0 +1,9 @@ +Things to be worked on: + +* Tests for %option user-init, %option pre-action, %option post-action. + +* integrate examples directory into tests so that normal testing + proves the examples are up to date. + +* Do away with any need for -lfl by generating a default yywrap + if the user doesn't specify one. diff --git a/doc/Makefile.am b/doc/Makefile.am index 523aa97..4d15da7 100644 --- a/doc/Makefile.am +++ b/doc/Makefile.am @@ -7,7 +7,7 @@ MAINTAINERCLEANFILES = flex.1 CLEANFILES = *.aux *.cp *.cps *.fn *.fns *.hk *.hks *.ky *.log \ *.op *.ops *.pg *.toc *.tp *.tps *.vr *.vrs -flex.1: $(top_srcdir)/configure.ac $(top_srcdir)/src/flex.skl $(top_srcdir)/src/options.c $(top_srcdir)/src/options.h | $(FLEX) +flex.1: $(top_srcdir)/configure.ac $(top_srcdir)/src/cpp-flex.skl $(top_srcdir)/src/options.c $(top_srcdir)/src/options.h | $(FLEX) $(HELP2MAN) --name='$(PACKAGE_NAME)' --section=1 \ --source='The Flex Project' --manual='Programming' \ --output=$@ $(FLEX) \ diff --git a/doc/flex.texi b/doc/flex.texi index 9087622..30d98dd 100644 --- a/doc/flex.texi +++ b/doc/flex.texi @@ -4,7 +4,7 @@ @include version.texi @settitle Lexical Analysis With Flex, for Flex @value{VERSION} @set authors Vern Paxson, Will Estes and John Millaway -@c "Macro Hooks" index +@c "User Hooks" index @defindex hk @c "Options" index @defindex op @@ -92,7 +92,7 @@ This manual was written by @value{authors}. * Start Conditions:: * Multiple Input Buffers:: * EOF:: -* Misc Macros:: +* Misc Controls:: * User Values:: * Yacc:: * Scanner Options:: @@ -170,7 +170,7 @@ FAQ * How can I have multiple input sources feed into the same scanner at the same time?:: * Can I build nested parsers that work with the same input file?:: * How can I match text only at the end of a file?:: -* How can I make REJECT cascade across start condition boundaries?:: +* How can I make yyreject() cascade across start condition boundaries?:: * Why cant I use fast or full tables with interactive mode?:: * How much faster is -F or -f than -C?:: * If I have a simple grammar cant I just parse it with flex?:: @@ -192,7 +192,7 @@ FAQ * How can I build a two-pass scanner?:: * How do I match any string not matched in the preceding rules?:: * I am trying to port code from AT&T lex that uses yysptr and yysbuf.:: -* Is there a way to make flex treat NULL like a regular character?:: +* Is there a way to make flex treat NUL like a regular character?:: * Whenever flex can not match the input it says "flex scanner jammed".:: * Why doesn't flex have non-greedy operators like perl does?:: * Memory leak - 16386 bytes allocated by malloc.:: @@ -205,7 +205,7 @@ FAQ * Can I fake multi-byte character support?:: * deleteme01:: * Can you discuss some flex internals?:: -* unput() messes up yy_at_bol:: +* yyunput() messes up yyatbol:: * The | operator is not doing what I want:: * Why can't flex understand this variable trailing context pattern?:: * The ^ operator isn't working:: @@ -259,7 +259,6 @@ FAQ * unnamed-faq-99:: * unnamed-faq-100:: * unnamed-faq-101:: -* What is the difference between YYLEX_PARAM and YY_DECL?:: * Why do I get "conflicting types for yylex" error?:: * How do I access the values set in a Flex action from within a Bison action?:: @@ -269,11 +268,12 @@ Appendices * Bison Bridge:: * M4 Dependency:: * Common Patterns:: +* Adding More Target Languages Indices * Concept Index:: -* Index of Functions and Macros:: +* Index of Functions:: * Index of Variables:: * Index of Data Types:: * Index of Hooks:: @@ -306,13 +306,22 @@ GitHub's issue tracking facility at @url{https://github.com/westes/flex/issues/} program which recognizes lexical patterns in text. The @code{flex} program reads the given input files, or its standard input if no file names are given, for a description of a scanner to generate. The -description is in the form of pairs of regular expressions and C code, -called @dfn{rules}. @code{flex} generates as output a C source file, -@file{lex.yy.c} by default, which defines a routine @code{yylex()}. -This file can be compiled and linked with the flex runtime library to +description is in the form of pairs of regular expressions and +fragments of source code +called @dfn{rules}. @code{flex} generates as output a source file +in your target language which defines a routine @code{yylex()}. +This file can be compiled and (if you are using the C/C++ back end) +optionally linked with the flex runtime library to produce an executable. When the executable is run, it analyzes its input for occurrences of the regular expressions. Whenever it finds -one, it executes the corresponding C code. +one, it executes the corresponding rule code. + +When your target language is C, the name of the generated scanner +@file{lex.yy.c} by default. Other languages will glue the suffix they +normally use for source-code files to the prefix @file{lex.yy}. + +The examples in this manual are in C, which is Flex's default target +language and until release 4.6.2 its only one. @node Simple Examples, Format, Introduction, Top @chapter Some Simple Examples @@ -343,29 +352,22 @@ beginning of the rules. Here's another simple example: -@cindex counting characters and lines +@cindex counting characters and lines; reentrant @example -@verbatim - int num_lines = 0, num_chars = 0; - - %% - \n ++num_lines; ++num_chars; - . ++num_chars; +@verbatiminclude ../examples/manual/example_r.lex +@end example - %% +If you have looked at older versions of the Flex nanual, you might +have seen a version of the above example that looked more like this: - int main() - { - yylex(); - printf( "# of lines = %d, # of chars = %d\n", - num_lines, num_chars ); - } -@end verbatim +@cindex counting characters and lines; non-reentrant +@example +@verbatiminclude ../examples/manual/example_nr.lex @end example -This scanner counts the number of characters and the number of lines in -its input. It produces no output other than the final report on the -character and line counts. The first line declares two globals, +Both versions count the number of characters and the number of lines in +its input. Both produces no output other than the final report on the +character and line counts. The first code line declares two globals, @code{num_lines} and @code{num_chars}, which are accessible both inside @code{yylex()} and in the @code{main()} routine declared after the second @samp{%%}. There are two rules, one which matches a newline @@ -373,59 +375,90 @@ second @samp{%%}. There are two rules, one which matches a newline and one which matches any character other than a newline (indicated by the @samp{.} regular expression). +The difference between these two variants is that the first uses +Flex's @emph{reentrant} interface, which bundles the scanner state +into a yyscan_t structure; the second uses the @emph{non-reentrant} +interface, in which the scanner's state is exposed through global +variables. + +The non-reentrant interface is a relic from the early 1970s when Lex, +the ancestor of Flex, was designed. Modern programming practice frowns +on hidden global variables; thus when Flex generates a scanner in any +language other than the original C/C++ non-reentrancy is not even an +option. Most likely it will make you some kind of scanner class +that you instantiate, with methods and fields rather than exposed globals. + +Thus it's a good idea to get used to not relying on the exposed +globals of the original interface from the beginning of your Flex +programming. This is so even though the reentrant example above is a +rather poor one; it avoids exposing the scanner state in globals but +creates globals of its own. There is a mechanism for including +user-defined fields in the scanner structure which will be explained +in detail at @xref{Extra Data}. For now, consider this: + +@example +@verbatiminclude ../examples/manual/example_er.lex +@end example + +While it requires a bit more ceremony, several instances of this +scanner can be run concurrently without stepping on each others' +storage. + +(The @code{%option noyywrap} in these examples is helpful in +making them run standalone, but does not change the behavior of the scsnner.) + A somewhat more complicated example: @cindex Pascal-like language @example @verbatim - /* scanner for a toy Pascal-like language */ +/* scanner for a toy Pascal-like language */ - %{ - /* need this for the call to atof() below */ - #include <math.h> - %} - - DIGIT [0-9] - ID [a-z][a-z0-9]* +%{ +/* need this for the call to atof() below */ +#include <math.h> +%} - %% +DIGIT [0-9] +ID [a-z][a-z0-9]* - {DIGIT}+ { - printf( "An integer: %s (%d)\n", yytext, - atoi( yytext ) ); - } +%% - {DIGIT}+"."{DIGIT}* { - printf( "A float: %s (%g)\n", yytext, - atof( yytext ) ); - } +{DIGIT}+ { + printf( "An integer: %s (%d)\n", yytext, + atoi( yytext ) ); + } - if|then|begin|end|procedure|function { - printf( "A keyword: %s\n", yytext ); - } +{DIGIT}+"."{DIGIT}* { + printf( "A float: %s (%g)\n", yytext, + atof( yytext ) ); + } - {ID} printf( "An identifier: %s\n", yytext ); +if|then|begin|end|procedure|function { + printf( "A keyword: %s\n", yytext ); + } - "+"|"-"|"*"|"/" printf( "An operator: %s\n", yytext ); +{ID} printf( "An identifier: %s\n", yytext ); - "{"[^{}\n]*"}" /* eat up one-line comments */ +"+"|"-"|"*"|"/" printf( "An operator: %s\n", yytext ); - [ \t\n]+ /* eat up whitespace */ +"{"[^{}\n]*"}" /* eat up one-line comments */ - . printf( "Unrecognized character: %s\n", yytext ); +[ \t\n]+ /* eat up whitespace */ - %% +. printf( "Unrecognized character: %s\n", yytext ); - int main( int argc, char **argv ) - { - ++argv, --argc; /* skip over program name */ - if ( argc > 0 ) - yyin = fopen( argv[0], "r" ); - else - yyin = stdin; +%% - yylex(); - } +int main( int argc, char **argv ) { + ++argv, --argc; /* skip over program name */ + if ( argc > 0 ) { + yyin = fopen( argv[0], "r" ); + } else { + yyin = stdin; + } + yylex(); +} @end verbatim @end example @@ -540,10 +573,11 @@ themselves. A @code{%top} block is similar to a @samp{%@{} ... @samp{%@}} block, except that the code in a @code{%top} block is relocated to the @emph{top} of the -generated file, before any flex definitions @footnote{Actually, +generated file, before any flex definitions @footnote{Actually, in the +C/C++ back end, @code{yyIN_HEADER} is defined before the @samp{%top} block.}. -The @code{%top} block is useful when you want certain preprocessor macros to be -defined or certain files to be included before the generated code. +The @code{%top} block is useful when you want definitions to be +evaluated or certain files to be included before the generated code. The single characters, @samp{@{} and @samp{@}} are used to delimit the @code{%top} block, as show in the example below: @@ -587,8 +621,10 @@ meaning is not well-defined and it may well cause compile-time errors Posix}, for other such features). Any @emph{indented} text or text enclosed in @samp{%@{} and @samp{%@}} -is copied verbatim to the output (with the %@{ and %@} symbols removed). -The %@{ and %@} symbols must appear unindented on lines by themselves. +is copied verbatim to the output (with the %@{ and %@} symbols +removed). The %@{ and %@} symbols must appear unindented on lines by +themselves. Because whitespace is easy to mangle without noticing, +it's good style to use the explicit %@{ and %@} delimiters. @node User Code Section, Comments in the Input, Rules Section, Format @section Format of the User Code Section @@ -606,9 +642,10 @@ The presence of this section is optional; if it is missing, the second @cindex comments, syntax of Flex supports C-style comments, that is, anything between @samp{/*} and @samp{*/} is -considered a comment. Whenever flex encounters a comment, it copies the -entire comment verbatim to the generated source code. Comments may -appear just about anywhere, but with the following exceptions: +considered a comment in the parts of the file Flex +interprets. Whenever flex encounters a comment, it copies the entire +comment verbatim to the generated source code. Comments may appear +just about anywhere, but with the following exceptions: @itemize @cindex comments, in rules section @@ -632,7 +669,7 @@ All the comments in the following example are valid: @example @verbatim %{ -/* code block */ +/* C code block - other target languages might have different comment syntax */ %} /* Definitions Section */ @@ -640,21 +677,26 @@ All the comments in the following example are valid: %% /* Rules Section */ -ruleA /* after regex */ { /* code block */ } /* after code block */ +ruleA /* after regex */ { /* C code block */ } /* after code block */ /* Rules Section (indented) */ <STATE_X>{ -ruleC ECHO; -ruleD ECHO; +ruleC yyecho(); +ruleD yyecho(); %{ -/* code block */ +/* C code block */ %} } %% -/* User Code Section */ - +/* User C Code Section */ @end verbatim @end example +If the target language is something other than C/C++, you will need to use +its normal comment syntax in actions and code blocks. Note that the +optional @{ and @} delimiters around actions a Flex syntax, not C +syntax; you will be able to use those even if, e,g., your target +language is Pascal-like and delimits blocs with begin/end. + @node Patterns, Matching, Format, Top @chapter Patterns @@ -734,7 +776,7 @@ if X is @samp{a}, @samp{b}, @samp{f}, @samp{n}, @samp{r}, @samp{t}, or @samp{v}, then the ANSI-C interpretation of @samp{\x}. Otherwise, a literal @samp{X} (used to escape operators such as @samp{*}) -@cindex NULL character in patterns, syntax of +@cindex NUL character in patterns, syntax of @item \0 a NUL character (ASCII code 0) @@ -1104,7 +1146,8 @@ a time) to its output. @cindex %array, use of @cindex %pointer, use of @vindex yytext -Note that @code{yytext} can be defined in two different ways: either as +Note that in languages with only fixed-extent arrays (like C/C+) +@code{yytext} can be defined in two different ways: either as a character @emph{pointer} or as a character @emph{array}. You can control which definition @code{flex} uses by including one of the special directives @code{%pointer} or @code{%array} in the first @@ -1114,14 +1157,14 @@ in which case @code{yytext} will be an array. The advantage of using @code{%pointer} is substantially faster scanning and no buffer overflow when matching very large tokens (unless you run out of dynamic memory). The disadvantage is that you are restricted in how your actions can -modify @code{yytext} (@pxref{Actions}), and calls to the @code{unput()} +modify @code{yytext} (@pxref{Actions}), and calls to the @code{yyunput()} function destroys the present contents of @code{yytext}, which can be a considerable porting headache when moving between different @code{lex} versions. @cindex %array, advantages of The advantage of @code{%array} is that you can then modify @code{yytext} -to your heart's content, and calls to @code{unput()} do not destroy +to your heart's content, and calls to @code{yyunput()} do not destroy @code{yytext} (@pxref{Actions}). Furthermore, existing @code{lex} programs sometimes access @code{yytext} externally using declarations of the form: @@ -1137,27 +1180,34 @@ for @code{%array}. The @code{%array} declaration defines @code{yytext} to be an array of @code{YYLMAX} characters, which defaults to a fairly large value. You -can change the size by simply #define'ing @code{YYLMAX} to a different -value in the first section of your @code{flex} input. As mentioned -above, with @code{%pointer} yytext grows dynamically to accommodate -large tokens. While this means your @code{%pointer} scanner can -accommodate very large tokens (such as matching entire blocks of -comments), bear in mind that each time the scanner must resize -@code{yytext} it also must rescan the entire token from the beginning, -so matching such tokens can prove slow. @code{yytext} presently does -@emph{not} dynamically grow if a call to @code{unput()} results in too -much text being pushed back; instead, a run-time error results. +can change the size to a different value with @code{%option yylmax += NNN}. As mentioned above, with @code{%pointer} yytext grows +dynamically to accommodate large tokens. While this means your +@code{%pointer} scanner can accommodate very large tokens (such as +matching entire blocks of comments), bear in mind that each time the +scanner must resize @code{yytext} it also must rescan the entire token +from the beginning, so matching such tokens can prove slow. +@code{yytext} presently does @emph{not} dynamically grow if a call to +@code{yyunput()} results in too much text being pushed back; instead, a +run-time error results. @cindex %array, with C++ Also note that you cannot use @code{%array} with C++ scanner classes (@pxref{Cxx}). +In target langages with automatic memory allocation and arrays, none +of this applies; you can expect @code{yytext} to dynamically resize +itself, calls to the @code{yyunput()} will not destroy the present +contents of @code{yytext}, and you will never get a run-time error +from calls to the @code{yyunput()} function except in the extremely +unlikely case that your scanner cannot allocate more memory. + @node Actions, Generated Scanner, Matching, Top @chapter Actions @cindex actions Each pattern in a rule has a corresponding @dfn{action}, which can be -any arbitrary C statement. The pattern ends at the first non-escaped +any arbitrary target-language statement. The pattern ends at the first non-escaped whitespace character; the remainder of the line is its action. If the action is empty, then when the pattern is matched the input token is simply discarded. For example, here is the specification for a program @@ -1221,22 +1271,22 @@ in any way. Actions are free to modify @code{yyleng} except they should not do so if the action also includes use of @code{yymore()} (see below). -@cindex preprocessor macros, for use in actions -There are a number of special directives which can be included within an -action: +@cindex rule hooks, for use in actions +There are a number of special hooks which can be included within an +action. @table @code -@item ECHO -@cindex ECHO +@item yyecho() +@cindex yyecho() copies yytext to the scanner's output. -@item BEGIN -@cindex BEGIN +@item yybegin() +@cindex yybegin() followed by the name of a start condition places the scanner in the corresponding start condition (see below). -@item REJECT -@cindex REJECT +@item yyreject() +@cindex yyreject() directs the scanner to proceed on to the ``second best'' rule which matched the input (or a prefix of the input). The rule is chosen as described above in @ref{Matching}, and @code{yytext} and @code{yyleng} @@ -1248,44 +1298,46 @@ whenever @samp{frob} is seen: @example @verbatim - int word_count = 0; - %% +%{ + int word_count = 0; +%} +%% - frob special(); REJECT; - [^ \t\n]+ ++word_count; +frob special(); yyreject(); +[^ \t\n]+ ++word_count; @end verbatim @end example -Without the @code{REJECT}, any occurrences of @samp{frob} in the input +Without the @code{yyreject()}, any occurrences of @samp{frob} in the input would not be counted as words, since the scanner normally executes only -one action per token. Multiple uses of @code{REJECT} are allowed, each +one action per token. Multiple uses of @code{yyreject()} are allowed, each one finding the next best choice to the currently active rule. For example, when the following scanner scans the token @samp{abcd}, it will write @samp{abcdabcaba} to the output: -@cindex REJECT, calling multiple times +@cindex yyreject(), calling multiple times @cindex |, use of @example @verbatim - %% - a | - ab | - abc | - abcd ECHO; REJECT; - .|\n /* eat up any unmatched character */ +%% +a | +ab | +abc | +abcd yyecho(); yyreject(); +.|\n /* eat up any unmatched character */ @end verbatim @end example The first three rules share the fourth's action since they use the special @samp{|} action. -@code{REJECT} is a particularly expensive feature in terms of scanner +@code{yyreject()} is a particularly expensive feature in terms of scanner performance; if it is used in @emph{any} of the scanner's actions it will slow down @emph{all} of the scanner's matching. Furthermore, -@code{REJECT} cannot be used with the @samp{-Cf} or @samp{-CF} options +@code{yyreject()} cannot be used with the @samp{-Cf} or @samp{-CF} options (@pxref{Scanner Options}). -Note also that unlike the other special actions, @code{REJECT} is a +Note also that unlike the other special actions, @code{yyreject()} is a @emph{branch}. Code immediately following it in the action will @emph{not} be executed. @@ -1301,9 +1353,9 @@ the output: @cindex yymore() to append token to previous token @example @verbatim - %% - mega- ECHO; yymore(); - kludge ECHO; +%% +mega- yyecho(); yymore(); +kludge yyecho(); @end verbatim @end example @@ -1312,7 +1364,7 @@ is matched, but the previous @samp{mega-} is still hanging around at the beginning of @code{yytext} so the -@code{ECHO} +@code{yyecho()} for the @samp{kludge} rule will actually write @samp{mega-kludge}. @end table @@ -1336,55 +1388,54 @@ following will write out @samp{foobarbar}: @cindex pushing back characters with yyless @example @verbatim - %% - foobar ECHO; yyless(3); - [a-z]+ ECHO; +%% +foobar yyecho(); yyless(3); +[a-z]+ yyecho(); @end verbatim @end example An argument of 0 to @code{yyless()} will cause the entire current input string to be scanned again. Unless you've changed how the scanner will -subsequently process its input (using @code{BEGIN}, for example), this +subsequently process its input (using @code{yybegin()}, for example), this will result in an endless loop. -Note that @code{yyless()} is a macro and can only be used in the flex -input file, not from other source files. - -@cindex unput() -@cindex pushing back characters with unput -@code{unput(c)} puts the character @code{c} back onto the input stream. +@cindex yyunput() +@cindex pushing back characters with yyunput +@code{yyunput(c)} puts the character @code{c} back onto the input stream. It will be the next character scanned. The following action will take the current token and cause it to be rescanned enclosed in parentheses. -@cindex unput(), pushing back characters -@cindex pushing back characters with unput() +@cindex yyunput(), pushing back characters +@cindex pushing back characters with yyunput() @example @verbatim - { +{ int i; - /* Copy yytext because unput() trashes yytext */ + /* Copy yytext because yyunput() trashes yytext */ char *yycopy = strdup( yytext ); - unput( ')' ); + yyunput( ')' ); for ( i = yyleng - 1; i >= 0; --i ) - unput( yycopy[i] ); - unput( '(' ); + yyunput( yycopy[i] ); + yyunput( '(' ); free( yycopy ); - } +} @end verbatim @end example -Note that since each @code{unput()} puts the given character back at the +Note that since each @code{yyunput()} puts the given character back at the @emph{beginning} of the input stream, pushing back strings must be done back-to-front. -@cindex %pointer, and unput() -@cindex unput(), and %pointer -An important potential problem when using @code{unput()} is that if you -are using @code{%pointer} (the default), a call to @code{unput()} +@cindex %pointer, and yyunput() +@cindex yyunput(), and %pointer +An important potential problem when using @code{yyunput()} in C (and, +generally, in target languages with C-like manual memory management) is +that if you +are using @code{%pointer} (the default), a call to @code{yyunput()} @emph{destroys} the contents of @code{yytext}, starting with its rightmost character and devouring one character to the left with each call. If you need the value of @code{yytext} preserved after a call to -@code{unput()} (as in the above example), you must either first copy it +@code{yyunput()} (as in the above example), you must either first copy it elsewhere, or build your scanner using @code{%array} instead (@pxref{Matching}). @@ -1393,56 +1444,45 @@ elsewhere, or build your scanner using @code{%array} instead Finally, note that you cannot put back @samp{EOF} to attempt to mark the input stream with an end-of-file. -@cindex input() -@code{input()} reads the next character from the input stream. For +@cindex yyinput() +@code{yyinput()} reads the next character from the input stream. For example, the following is one way to eat up C comments: @cindex comments, discarding @cindex discarding C comments @example @verbatim - %% - "/*" { - int c; - - for ( ; ; ) - { - while ( (c = input()) != '*' && - c != EOF ) - ; /* eat up text of comment */ - - if ( c == '*' ) - { - while ( (c = input()) == '*' ) - ; - if ( c == '/' ) - break; /* found the end */ - } - - if ( c == EOF ) - { - error( "EOF in comment" ); - break; - } - } +%% +"/*" { + int c; + + for ( ; ; ) { + while ( (c = yyinput()) != '*' && c != EOF ) + ; /* eat up text of comment */ + + if ( c == '*' ) { + while ( (c = yyinput()) == '*' ) + ; + if ( c == '/' ) + break; /* found the end */ + } + + if ( c == EOF ) { + error( "EOF in comment" ); + break; } + } + } @end verbatim @end example -@cindex input(), and C++ -@cindex yyinput() -(Note that if the scanner is compiled using @code{C++}, then -@code{input()} is instead referred to as @b{yyinput()}, in order to -avoid a name clash with the @code{C++} stream by the name of -@code{input}.) - @cindex flushing the internal buffer -@cindex YY_FLUSH_BUFFER -@code{YY_FLUSH_BUFFER;} flushes the scanner's internal buffer so that +@cindex yy_flush_current_buffer() +@code{yy_flush_current_buffer()} flushes the scanner's internal buffer so that the next time the scanner attempts to match a token, it will first -refill the buffer using @code{YY_INPUT()} (@pxref{Generated Scanner}). +refill the buffer using @code{yyread()} (@pxref{Generated Scanner}). This action is a special case of the more general -@code{yy_flush_buffer;} function, described below (@pxref{Multiple +@code{yy_flush_buffer()} function, described below (@pxref{Multiple Input Buffers}) @cindex yyterminate() @@ -1452,43 +1492,44 @@ Input Buffers}) @code{yyterminate()} can be used in lieu of a return statement in an action. It terminates the scanner and returns a 0 to the scanner's caller, indicating ``all done''. By default, @code{yyterminate()} is -also called when an end-of-file is encountered. It is a macro and may -be redefined. +also called when an end-of-file is encountered. It may be redefined. + +When the target language is C/C++, @code{yyterminate()} is a macro. +Redefining it using the C/C++ preprocessor in your definitions section +is allowed, but not recommended as doing this makes code more +difficult to port out of C/C++. In other target languages you can +swt what yyterminate() expands to with @code{%option yyterminate}. @node Generated Scanner, Start Conditions, Actions, Top @chapter The Generated Scanner @cindex yylex(), in generated scanner -The output of @code{flex} is the file @file{lex.yy.c}, which contains -the scanning routine @code{yylex()}, a number of tables used by it for -matching tokens, and a number of auxiliary routines and macros. By -default, @code{yylex()} is declared as follows: +The output of @code{flex} is a file with the name @file{lex.yy}, which +contains the scanning routine @code{yylex()}, a number of tables used +by it for matching tokens, and a number of auxiliary routines. By +default in C, @code{yylex()} is declared as follows: @example @verbatim - int yylex() - { + int yylex(void) { ... various definitions and the actions in here ... - } + } @end verbatim @end example -@cindex yylex(), overriding -(If your environment supports function prototypes, then it will be -@code{int yylex( void )}.) This definition may be changed by defining -the @code{YY_DECL} macro. For example, you could use: +@cindex yylex(), overriding, yydecl +This definition may be changed with the the @code{yydecl} option. +For example, you could put this in among your directives: @cindex yylex, overriding the prototype of @example @verbatim - #define YY_DECL float lexscan( a, b ) float a, b; +%option yydecl="float lexscan(float a, float b)" @end verbatim @end example to give the scanning routine the name @code{lexscan}, returning a float, -and taking two floats as arguments. Note that if you give arguments to -the scanning routine using a K&R-style/non-prototyped function -declaration, you must terminate the definition with a semi-colon (;). +and taking two floats as arguments. @code{flex} generates @samp{C99} function definitions by default. Flex used to have the ability to generate obsolete, er, @@ -1500,6 +1541,11 @@ traditional definitions support added extra complexity in the skeleton file. For this reason, current versions of @code{flex} generate standard C99 code only, leaving K&R-style functions to the historians. +In other languages, @code{yylex()} will be generated as a reentrant +function with a scanner context argument added. This can be enabled +in C as well, and specifying your C scanners to be reentrant is +recommended for portability. + @cindex stdin, default for yyin @cindex yyin Whenever @code{yylex()} is called, it scans tokens from the global input @@ -1511,59 +1557,64 @@ one of its actions executes a @code{return} statement. @cindex end-of-file, and yyrestart() @cindex yyrestart() If the scanner reaches an end-of-file, subsequent calls are undefined -unless either @file{yyin} is pointed at a new input file (in which case -scanning continues from that file), or @code{yyrestart()} is called. -@code{yyrestart()} takes one argument, a @code{FILE *} pointer (which -can be NULL, if you've set up @code{YY_INPUT} to scan from a source other -than @code{yyin}), and initializes @file{yyin} for scanning from that -file. Essentially there is no difference between just assigning -@file{yyin} to a new input file or using @code{yyrestart()} to do so; -the latter is available for compatibility with previous versions of -@code{flex}, and because it can be used to switch input files in the -middle of scanning. It can also be used to throw away the current input -buffer, by calling it with an argument of @file{yyin}; but it would be -better to use @code{YY_FLUSH_BUFFER} (@pxref{Actions}). Note that +unless either @file{yyin} is pointed at a new input file (in which +case scanning continues from that file), or @code{yyrestart()} is +called. @code{yyrestart()} takes one argument, an input stream, and +initializes @file{yyin} for scanning from that stream. Essentially +there is no difference between just assigning @file{yyin} to a new +input stream or using @code{yyrestart()} to do so; the latter is +available for compatibility with previous versions of @code{flex}, and +because it can be used to switch input files in the middle of +scanning. It can also be used to throw away the current input buffer, +by calling it with an argument of @code{yyin}; but it would be better +to use @code{yy_flush_current_buffer()} (@pxref{Actions}). Note that @code{yyrestart()} does @emph{not} reset the start condition to @code{INITIAL} (@pxref{Start Conditions}). +In C, an input stream is a a @code{FILE *} pointer. This pointer can +be NULL, if you've set up a @code{%yyread()} hook to scan from a +source other than @code{yyin}. + @cindex RETURN, within actions If @code{yylex()} stops scanning due to executing a @code{return} statement in one of the actions, the scanner may then be called again and it will resume scanning where it left off. -@cindex YY_INPUT -By default (and for purposes of efficiency), the scanner uses +@cindex yyread +By default (and for purposes of efficiency), scanners use block-reads rather than simple @code{getc()} calls to read characters from @file{yyin}. The nature of how it gets its input can be controlled -by defining the @code{YY_INPUT} macro. The calling sequence for -@code{YY_INPUT()} is @code{YY_INPUT(buf,result,max_size)}. Its action +by redefining the @code{yyread} function used to fill the scanner buffer. The calling sequence for +@code{yyread()} is @code{yyread(buf,max_size)}. Its action is to place up to @code{max_size} characters in the character array -@code{buf} and return in the integer variable @code{result} either the +@code{buf} and return either the number of characters read or the constant @code{YY_NULL} (0 on Unix -systems) to indicate @samp{EOF}. The default @code{YY_INPUT} reads from +systems) to indicate @samp{EOF}. The default @code{yyread()} reads from the global file-pointer @file{yyin}. -@cindex YY_INPUT, overriding -Here is a sample definition of @code{YY_INPUT} (in the definitions +@cindex yyread(), overriding +#cindex %noyyread +Here is a sample redefinition of @code{yyread()} (in the definitions section of the input file): @example @verbatim - %{ - #define YY_INPUT(buf,result,max_size) \ - { \ - int c = getchar(); \ - result = (c == EOF) ? YY_NULL : (buf[0] = c, 1); \ - } - %} +int yyread(char *buf, size_t max_size) { + int c = getchar(); + return (c == EOF) ? YY_NULL : (buf[0] = c, 1); +} @end verbatim @end example This definition will change the input processing to occur one character at a time. +When Flex sees the @code{%noyyread} option, it omits the default +definition from the boilerplate in the rest of the parser. Your +@code{yyread()} function then replaces it. + @cindex yywrap() -When the scanner receives an end-of-file indication from YY_INPUT, it +When the scanner receives an end-of-file indication from @code{yyread()}, it then checks the @code{yywrap()} function. If @code{yywrap()} returns false (zero), then it is assumed that the function has gone ahead and set up @file{yyin} to point to another input file, and scanning @@ -1574,7 +1625,7 @@ condition remains unchanged; it does @emph{not} revert to @cindex yywrap, default for @cindex noyywrap, %option -@cindex %option noyywrapp +@cindex %option noyywrap If you do not supply your own version of @code{yywrap()}, then you must either use @code{%option noyywrap} (in which case the scanner behaves as though @code{yywrap()} returned 1), or you must link with @samp{-lfl} to @@ -1583,12 +1634,12 @@ obtain the default version of the routine, which always returns 1. For scanning from in-memory buffers (e.g., scanning strings), see @ref{Scanning Strings}. @xref{Multiple Input Buffers}. -@cindex ECHO, and yyout +@cindex yyecho(), and yyout @cindex yyout @cindex stdout, as default for yyout -The scanner writes its @code{ECHO} output to the @file{yyout} global +The scanner writes its @code{yyecho()} output to the @code{yyout} global (default, @file{stdout}), which may be redefined by the user simply by -assigning it to some other @code{FILE} pointer. +assigning it to some other sream - in C/C++, a @code{FILE} pointer. @node Start Conditions, Multiple Input Buffers, Generated Scanner, Top @chapter Start Conditions @@ -1601,9 +1652,9 @@ example, @example @verbatim - <STRING>[^"]* { /* eat up the string body ... */ - ... - } +<STRING>[^"]* { /* eat up the string body ... */ + ... + } @end verbatim @end example @@ -1613,9 +1664,9 @@ condition, and @cindex start conditions, multiple @example @verbatim - <INITIAL,STRING,QUOTE>\. { /* handle an escape ... */ - ... - } +<INITIAL,STRING,QUOTE>\. { /* handle an escape ... */ + ... + } @end verbatim @end example @@ -1627,8 +1678,8 @@ Start conditions are declared in the definitions (first) section of the input using unindented lines beginning with either @samp{%s} or @samp{%x} followed by a list of names. The former declares @dfn{inclusive} start conditions, the latter @dfn{exclusive} start -conditions. A start condition is activated using the @code{BEGIN} -action. Until the next @code{BEGIN} action is executed, rules with the +conditions. A start condition is activated using the @code{yybegin()} +action. Until the next @code{yybegin()} action is executed, rules with the given start condition will be active and rules with other start conditions will be inactive. If the start condition is inclusive, then rules with no start conditions at all will also be active. If it is @@ -1647,12 +1698,12 @@ connection between the two. The set of rules: @cindex start conditions, inclusive @example @verbatim - %s example - %% +%s example +%% - <example>foo do_something(); +<example>foo do_something(); - bar something_else(); +bar something_else(); @end verbatim @end example @@ -1661,12 +1712,12 @@ is equivalent to @cindex start conditions, exclusive @example @verbatim - %x example - %% +%x example +%% - <example>foo do_something(); +<example>foo do_something(); - <INITIAL,example>bar something_else(); +<INITIAL,example>bar something_else(); @end verbatim @end example @@ -1687,51 +1738,50 @@ have been written: @cindex start conditions, use of wildcard condition (<*>) @example @verbatim - %x example - %% +%x example +%% - <example>foo do_something(); +<example>foo do_something(); - <*>bar something_else(); +<*>bar something_else(); @end verbatim @end example -The default rule (to @code{ECHO} any unmatched character) remains active +The default rule (to @code{yyecho()} any unmatched character) remains active in start conditions. It is equivalent to: @cindex start conditions, behavior of default rule @example @verbatim - <*>.|\n ECHO; +<*>.|\n yyecho(); @end verbatim @end example -@cindex BEGIN, explanation -@findex BEGIN +@cindex yybegin(), explanation +@findex yybegin() @vindex INITIAL -@code{BEGIN(0)} returns to the original state where only the rules with +@code{yybegin(0)} returns to the original state where only the rules with no start conditions are active. This state can also be referred to as -the start-condition @code{INITIAL}, so @code{BEGIN(INITIAL)} is -equivalent to @code{BEGIN(0)}. (The parentheses around the start -condition name are not required but are considered good style.) +the start-condition @code{INITIAL}, so @code{yybegin(INITIAL)} is +equivalent to @code{yybegin(0)}. -@code{BEGIN} actions can also be given as indented code at the beginning +@code{yybegin()} actions can also be given as indented code at the beginning of the rules section. For example, the following will cause the scanner to enter the @code{SPECIAL} start condition whenever @code{yylex()} is called and the global variable @code{enter_special} is true: -@cindex start conditions, using BEGIN +@cindex start conditions, using yybegin() @example @verbatim - int enter_special; + int enter_special; - %x SPECIAL - %% - if ( enter_special ) - BEGIN(SPECIAL); +%x SPECIAL +%% + if ( enter_special ) + yybegin(SPECIAL); - <SPECIAL>blahblahblah - ...more rules follow... +<SPECIAL>blahblahblah +...more rules follow... @end verbatim @end example @@ -1745,33 +1795,33 @@ treat it as a single token, the floating-point number @samp{123.456}: @cindex start conditions, for different interpretations of same input @example @verbatim - %{ - #include <math.h> - %} - %s expect +%{ +#include <math.h> +%} +%s expect - %% - expect-floats BEGIN(expect); +%% +expect-floats yybegin(expect); - <expect>[0-9]+.[0-9]+ { - printf( "found a float, = %f\n", - atof( yytext ) ); - } - <expect>\n { - /* that's the end of the line, so - * we need another "expect-number" - * before we'll recognize any more - * numbers - */ - BEGIN(INITIAL); - } +<expect>[0-9]+.[0-9]+ { + printf( "found a float, = %f\n", + atof( yytext ) ); + } +<expect>\n { + /* that's the end of the line, so + * we need another "expect-number" + * before we'll recognize any more + * numbers + */ + yybegin(INITIAL); + } - [0-9]+ { - printf( "found an integer, = %d\n", - atoi( yytext ) ); - } +[0-9]+ { + printf( "found an integer, = %d\n", + atoi( yytext ) ); + } - "." printf( "found a dot\n" ); +"." printf( "found a dot\n" ); @end verbatim @end example @@ -1782,16 +1832,16 @@ maintaining a count of the current input line. @cindex recognizing C comments @example @verbatim - %x comment - %% - int line_num = 1; +%x comment +%% + int line_num = 1; - "/*" BEGIN(comment); +"/*" yybegin(comment); - <comment>[^*\n]* /* eat anything that's not a '*' */ - <comment>"*"+[^*/\n]* /* eat up '*'s not followed by '/'s */ - <comment>\n ++line_num; - <comment>"*"+"/" BEGIN(INITIAL); +<comment>[^*\n]* /* eat anything that's not a '*' */ +<comment>"*"+[^*/\n]* /* eat up '*'s not followed by '/'s */ +<comment>\n ++line_num; +<comment>"*"+"/" yybegin(INITIAL); @end verbatim @end example @@ -1808,53 +1858,47 @@ following fashion: @cindex using integer values of start condition names @example @verbatim - %x comment foo - %% - int line_num = 1; - int comment_caller; +%x comment foo +%% + int line_num = 1; + int comment_caller; - "/*" { - comment_caller = INITIAL; - BEGIN(comment); - } +"/*" { + comment_caller = INITIAL; + yybegin(comment); + } - ... +... - <foo>"/*" { - comment_caller = foo; - BEGIN(comment); - } +<foo>"/*" { + comment_caller = foo; + yybegin(comment); + } - <comment>[^*\n]* /* eat anything that's not a '*' */ - <comment>"*"+[^*/\n]* /* eat up '*'s not followed by '/'s */ - <comment>\n ++line_num; - <comment>"*"+"/" BEGIN(comment_caller); +<comment>[^*\n]* /* eat anything that's not a '*' */ +<comment>"*"+[^*/\n]* /* eat up '*'s not followed by '/'s */ +<comment>\n ++line_num; +<comment>"*"+"/" yybegin(comment_caller); @end verbatim @end example -@cindex YY_START, example +@cindex yystart(), example Furthermore, you can access the current start condition using the -integer-valued @code{YY_START} macro. For example, the above +integer-valued @code{yystart()} rule hook. For example, the above assignments to @code{comment_caller} could instead be written -@cindex getting current start state with YY_START +@cindex getting current start state with yystart() @example @verbatim - comment_caller = YY_START; + comment_caller = yystart(); @end verbatim @end example -@vindex YY_START -Flex provides @code{YYSTATE} as an alias for @code{YY_START} (since that -is what's used by AT&T @code{lex}). - For historical reasons, start conditions do not have their own name-space within the generated scanner. The start condition names are unmodified in the generated scanner and generated header. @xref{option-header}. @xref{option-prefix}. - - Finally, here's an example of how to match C-style quoted strings using exclusive start conditions, including expanded escape sequences (but not including checking for a string that's too long): @@ -1862,60 +1906,60 @@ not including checking for a string that's too long): @cindex matching C-style double-quoted strings @example @verbatim - %x str +%x str - %% - char string_buf[MAX_STR_CONST]; - char *string_buf_ptr; +%% + char string_buf[MAX_STR_CONST]; + char *string_buf_ptr; - \" string_buf_ptr = string_buf; BEGIN(str); +\" string_buf_ptr = string_buf; yybegin(str); - <str>\" { /* saw closing quote - all done */ - BEGIN(INITIAL); - *string_buf_ptr = '\0'; - /* return string constant token type and - * value to parser - */ - } +<str>\" { /* saw closing quote - all done */ + yybegin(INITIAL); + *string_buf_ptr = '\0'; + /* return string constant token type and + * value to parser + */ + } - <str>\n { - /* error - unterminated string constant */ - /* generate error message */ - } +<str>\n { + /* error - unterminated string constant */ + /* generate error message */ + } - <str>\\[0-7]{1,3} { - /* octal escape sequence */ - int result; +<str>\\[0-7]{1,3} { + /* octal escape sequence */ + int result; - (void) sscanf( yytext + 1, "%o", &result ); + (void) sscanf( yytext + 1, "%o", &result ); - if ( result > 0xff ) - /* error, constant is out-of-bounds */ + if ( result > 0xff ) + /* error, constant is out-of-bounds */ - *string_buf_ptr++ = result; - } + *string_buf_ptr++ = result; + } - <str>\\[0-9]+ { - /* generate error - bad escape sequence; something - * like '\48' or '\0777777' - */ - } +<str>\\[0-9]+ { + /* generate error - bad escape sequence; something + * like '\48' or '\0777777' + */ + } - <str>\\n *string_buf_ptr++ = '\n'; - <str>\\t *string_buf_ptr++ = '\t'; - <str>\\r *string_buf_ptr++ = '\r'; - <str>\\b *string_buf_ptr++ = '\b'; - <str>\\f *string_buf_ptr++ = '\f'; +<str>\\n *string_buf_ptr++ = '\n'; +<str>\\t *string_buf_ptr++ = '\t'; +<str>\\r *string_buf_ptr++ = '\r'; +<str>\\b *string_buf_ptr++ = '\b'; +<str>\\f *string_buf_ptr++ = '\f'; - <str>\\(.|\n) *string_buf_ptr++ = yytext[1]; +<str>\\(.|\n) *string_buf_ptr++ = yytext[1]; - <str>[^\\\n\"]+ { - char *yptr = yytext; +<str>[^\\\n\"]+ { + char *yptr = yytext; - while ( *yptr ) - *string_buf_ptr++ = *yptr++; - } + while ( *yptr ) + *string_buf_ptr++ = *yptr++; + } @end verbatim @end example @@ -1927,7 +1971,7 @@ condition @dfn{scope}. A start condition scope is begun with: @example @verbatim - <SCs>{ +<SCs>{ @end verbatim @end example @@ -1939,12 +1983,12 @@ start condition scope, every rule automatically has the prefix @cindex extended scope of start conditions @example @verbatim - <ESC>{ - "\\n" return '\n'; - "\\r" return '\r'; - "\\f" return '\f'; - "\\0" return '\0'; - } +<ESC>{ + "\\n" return '\n'; + "\\r" return '\r'; + "\\f" return '\f'; + "\\0" return '\0'; +} @end verbatim @end example @@ -1952,10 +1996,10 @@ is equivalent to: @example @verbatim - <ESC>"\\n" return '\n'; - <ESC>"\\r" return '\r'; - <ESC>"\\f" return '\f'; - <ESC>"\\0" return '\0'; +<ESC>"\\n" return '\n'; +<ESC>"\\r" return '\r'; +<ESC>"\\f" return '\f'; +<ESC>"\\0" return '\0'; @end verbatim @end example @@ -1971,20 +2015,20 @@ pushes the current start condition onto the top of the start condition stack and switches to @code{new_state} as though you had used -@code{BEGIN new_state} +@code{yybegin(new_state)} (recall that start condition names are also integers). @end deftypefun @deftypefun void yy_pop_state () pops the top of the stack and switches to it via -@code{BEGIN}. +@code{yybegin()}. The program execution aborts, if there is no state on the stack. @end deftypefun @deftypefun int yy_top_state () returns the top of the stack without altering the stack's contents if a top state on the stack exists or the current state via -@code{YY_START} +@code{yystart()} otherwise. @end deftypefun @@ -2006,8 +2050,8 @@ stack} directive (@pxref{Scanner Options}). Some scanners (such as those which support ``include'' files) require reading from several input streams. As @code{flex} scanners do a large amount of buffering, one cannot control where the next input will be -read from by simply writing a @code{YY_INPUT()} which is sensitive to -the scanning context. @code{YY_INPUT()} is only called when the scanner +read from by simply writing a @code{yyread()} which is sensitive to +the scanning context. @code{yyread()} is only called when the scanner reaches the end of its buffer, which may be a long time after scanning a statement such as an @code{include} statement which requires switching the input source. @@ -2017,28 +2061,34 @@ for creating and switching between multiple input buffers. An input buffer is created by using: @cindex memory, allocating input buffers -@deftypefun YY_BUFFER_STATE yy_create_buffer ( FILE *file, int size ) +@deftypefun yybuffer yy_create_buffer ( FILE *file, int size ) @end deftypefun which takes a @code{FILE} pointer and a size and creates a buffer associated with the given file and large enough to hold @code{size} characters (when in doubt, use @code{YY_BUF_SIZE} for the size). It -returns a @code{YY_BUFFER_STATE} handle, which may then be passed to +returns a @code{yybuffer} handle, which may then be passed to other routines (see below). -@tindex YY_BUFFER_STATE -The @code{YY_BUFFER_STATE} type is a -pointer to an opaque @code{struct yy_buffer_state} structure, so you may -safely initialize @code{YY_BUFFER_STATE} variables to @code{((YY_BUFFER_STATE) -0)} if you wish, and also refer to the opaque structure in order to + +In target languages other than C/C++, this prototype will look +different. The input-stream type won't be @code{FILE *}, but you can +expect the same semantics expressed using the target language's native +types. + +@tindex yybuffer +The @code{yybuffer} type is a +reference to an opaque buffer state structure, so you may +safely initialize @code{yybuffer} variables to @code{((yybuffer) +NULL)} if you wish, and also refer to the opaque structure in order to correctly declare input buffers in source files other than that of your scanner. Note that the @code{FILE} pointer in the call to @code{yy_create_buffer} is only used as the value of @file{yyin} seen by -@code{YY_INPUT}. If you redefine @code{YY_INPUT()} so it no longer uses +@code{yyread()}. If you redefine @code{yyread()} so it no longer uses @file{yyin}, then you can safely pass a NULL @code{FILE} pointer to @code{yy_create_buffer}. You select a particular buffer to scan from using: -@deftypefun void yy_switch_to_buffer ( YY_BUFFER_STATE new_buffer ) +@deftypefun void yy_switch_to_buffer ( yybuffer new_buffer ) @end deftypefun The above function switches the scanner's input buffer so subsequent tokens @@ -2051,7 +2101,7 @@ instead of this function. Note also that switching input sources via either start condition. @cindex memory, deleting input buffers -@deftypefun void yy_delete_buffer ( YY_BUFFER_STATE buffer ) +@deftypefun void yy_delete_buffer ( yybuffer buffer ) @end deftypefun is used to reclaim the storage associated with a buffer. (@code{buffer} @@ -2060,7 +2110,7 @@ the current contents of a buffer using: @cindex pushing an input buffer @cindex stack, input buffer push -@deftypefun void yypush_buffer_state ( YY_BUFFER_STATE buffer ) +@deftypefun void yypush_buffer_state ( yybuffer buffer ) @end deftypefun This function pushes the new buffer state onto an internal stack. The pushed @@ -2080,24 +2130,26 @@ becomes the new current state. @cindex clearing an input buffer @cindex flushing an input buffer -@deftypefun void yy_flush_buffer ( YY_BUFFER_STATE buffer ) +@deftypefun void yy_flush_buffer ( yybuffer buffer ) @end deftypefun This function discards the buffer's contents, so the next time the scanner attempts to match a token from the buffer, it will first fill the buffer anew using -@code{YY_INPUT()}. +@code{yyread()}. -@deftypefun YY_BUFFER_STATE yy_new_buffer ( FILE *file, int size ) +@deftypefun yybuffer yy_new_buffer ( FILE *file, int size ) @end deftypefun is an alias for @code{yy_create_buffer()}, provided for compatibility with the C++ use of @code{new} and @code{delete} for creating and destroying dynamic objects. -@cindex YY_CURRENT_BUFFER, and multiple buffers Finally, the macro -@code{YY_CURRENT_BUFFER} macro returns a @code{YY_BUFFER_STATE} handle to the -current buffer. It should not be used as an lvalue. +@cindex yy_current_buffer(), and multiple buffers +Finally, @code{yy_current_buffer()} returns a +@code{yybuffer} handle to the current buffer. It should not be +used as an lvalue, because it can return NULL to indicate no buffer is +current. @cindex EOF, example using multiple input buffers Here are two examples of using these features for writing a scanner @@ -2111,36 +2163,35 @@ maintains the stack internally. @cindex handling include files with multiple input buffers @example @verbatim - /* the "incl" state is used for picking up the name - * of an include file - */ - %x incl - %% - include BEGIN(incl); +/* the "incl" state is used for picking up the name + * of an include file + */ +%x incl +%% +include yybegin(incl); - [a-z]+ ECHO; - [^a-z\n]*\n? ECHO; +[a-z]+ yyecho(); +[^a-z\n]*\n? yyecho(); - <incl>[ \t]* /* eat the whitespace */ - <incl>[^ \t\n]+ { /* got the include file name */ - yyin = fopen( yytext, "r" ); +<incl>[ \t]* /* eat the whitespace */ +<incl>[^ \t\n]+ { /* got the include file name */ + yyin = fopen( yytext, "r" ); - if ( ! yyin ) - error( ... ); + if ( ! yyin ) + error( ... ); - yypush_buffer_state(yy_create_buffer( yyin, YY_BUF_SIZE )); + yypush_buffer_state(yy_create_buffer( yyin, YY_BUF_SIZE )); - BEGIN(INITIAL); - } + yybegin(INITIAL); +} - <<EOF>> { - yypop_buffer_state(); +<<EOF>> { + yypop_buffer_state(); - if ( !YY_CURRENT_BUFFER ) - { - yyterminate(); - } - } + if ( !yy_current_buffer() ) { + yyterminate(); + } +} @end verbatim @end example @@ -2150,58 +2201,53 @@ manages its own input buffer stack manually (instead of letting flex do it). @cindex handling include files with multiple input buffers @example @verbatim - /* the "incl" state is used for picking up the name - * of an include file - */ - %x incl - - %{ - #define MAX_INCLUDE_DEPTH 10 - YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH]; - int include_stack_ptr = 0; - %} +/* the "incl" state is used for picking up the name + * of an include file + */ +%x incl - %% - include BEGIN(incl); +%{ +#define MAX_INCLUDE_DEPTH 10 +yybuffer include_stack[MAX_INCLUDE_DEPTH]; +int include_stack_ptr = 0; +%} - [a-z]+ ECHO; - [^a-z\n]*\n? ECHO; +%% +include yybegin(incl); - <incl>[ \t]* /* eat the whitespace */ - <incl>[^ \t\n]+ { /* got the include file name */ - if ( include_stack_ptr >= MAX_INCLUDE_DEPTH ) - { - fprintf( stderr, "Includes nested too deeply" ); - exit( 1 ); - } +[a-z]+ yyecho(); +[^a-z\n]*\n? yyecho(); - include_stack[include_stack_ptr++] = - YY_CURRENT_BUFFER; +<incl>[ \t]* /* eat the whitespace */ +<incl>[^ \t\n]+ { /* got the include file name */ + if ( include_stack_ptr >= MAX_INCLUDE_DEPTH ) { + fprintf( stderr, "Includes nested too deeply" ); + exit( 1 ); + } - yyin = fopen( yytext, "r" ); + include_stack[include_stack_ptr++] = + yy_current_buffer(); - if ( ! yyin ) - error( ... ); + yyin = fopen( yytext, "r" ); - yy_switch_to_buffer( - yy_create_buffer( yyin, YY_BUF_SIZE ) ); + if ( ! yyin ) + error( ... ); - BEGIN(INITIAL); - } + yy_switch_to_buffer( + yy_create_buffer( yyin, YY_BUF_SIZE ) ); - <<EOF>> { - if ( --include_stack_ptr == 0 ) - { - yyterminate(); - } + yybegin(INITIAL); +} - else - { - yy_delete_buffer( YY_CURRENT_BUFFER ); - yy_switch_to_buffer( - include_stack[include_stack_ptr] ); - } - } +<<EOF>> { + if ( --include_stack_ptr == 0 ) { + yyterminate(); + } else { + yy_delete_buffer( yy_current_buffer() ); + yy_switch_to_buffer( + include_stack[include_stack_ptr] ); + } +} @end verbatim @end example @@ -2210,18 +2256,23 @@ manages its own input buffer stack manually (instead of letting flex do it). The following routines are available for setting up input buffers for scanning in-memory strings instead of files. All of them create a new input buffer for scanning the string, and return a corresponding -@code{YY_BUFFER_STATE} handle (which you should delete with +@code{yybuffer} handle (which you should delete with @code{yy_delete_buffer()} when done with it). They also switch to the new buffer using @code{yy_switch_to_buffer()}, so the next call to @code{yylex()} will start scanning the string. -@deftypefun YY_BUFFER_STATE yy_scan_string ( const char *str ) -scans a NUL-terminated string. +@deftypefun yybuffer yy_scan_string ( char *str ) +scans a string. This declaration is correct for C/C++, in which +strings are simply character sequences terminated by a NUL. It is +expected that each target language will use the most appropriate +native string type instead. @end deftypefun -@deftypefun YY_BUFFER_STATE yy_scan_bytes ( const char *bytes, int len ) -scans @code{len} bytes (including possibly @code{NUL}s) starting at location -@code{bytes}. +@deftypefun yybuffer yy_scan_bytes ( const char *bytes, int len ) +scans @code{len} bytes (including possibly @code{NUL}s) starting at +location @code{bytes}. It is expected that each target language will +use the most appropriate native type instead of char*, such as a +reference to a byte array or slice. @end deftypefun Note that both of these functions create and scan a @emph{copy} of the @@ -2230,24 +2281,27 @@ the contents of the buffer it is scanning.) You can avoid the copy by using: @vindex YY_END_OF_BUFFER_CHAR -@deftypefun YY_BUFFER_STATE yy_scan_buffer (char *base, yy_size_t size) +@deftypefun yybuffer yy_scan_buffer (char *base, yy_size_t size) which scans in place the buffer starting at @code{base}, consisting of @code{size} bytes, the last two bytes of which @emph{must} be -@code{YY_END_OF_BUFFER_CHAR} (ASCII NUL). These last two bytes are not -scanned; thus, scanning consists of @code{base[0]} through -@code{base[size-2]}, inclusive. +@code{YY_END_OF_BUFFER_CHAR} (ASCII NUL). These last two bytes are +not scanned; thus, scanning consists of @code{base[0]} through +@code{base[size-2]}, inclusive. It is expected that each target +language will use the most appropriate native type instead of char*, +such as an owned byte array. @end deftypefun -If you fail to set up @code{base} in this manner (i.e., forget the final -two @code{YY_END_OF_BUFFER_CHAR} bytes), then @code{yy_scan_buffer()} -returns a NULL pointer instead of creating a new input buffer. +If you fail to set up @code{base} in this manner (i.e., forget the +final two @code{YY_END_OF_BUFFER_CHAR} bytes), then +@code{yy_scan_buffer()} returns a NULL pointer (and/or an error) +instead of creating a new input buffer. @deftp {Data type} yy_size_t is an integral type to which you can cast an integer expression reflecting the size of the buffer. @end deftp -@node EOF, Misc Macros, Multiple Input Buffers, Top +@node EOF, Misc Controls, Multiple Input Buffers, Top @chapter End-of-File Rules @cindex EOF, explanation @@ -2259,12 +2313,6 @@ by doing one of the following things: @itemize @item -@findex YY_NEW_FILE (now obsolete) -assigning @file{yyin} to a new input file (in previous versions of -@code{flex}, after doing the assignment you had to call the special -action @code{YY_NEW_FILE}. This is no longer necessary.) - -@item executing a @code{return} statement; @item @@ -2311,29 +2359,30 @@ example: @end verbatim @end example -@node Misc Macros, User Values, EOF, Top -@chapter Miscellaneous Macros +@node Misc Controls, User Values, EOF, Top +@chapter Miscellaneous Controls -@hkindex YY_USER_ACTION -The macro @code{YY_USER_ACTION} can be defined to provide an action +@hkindex %option pre-action +This option can be set to provide an code fragment which is always executed prior to the matched rule's action. For -example, it could be #define'd to call a routine to convert yytext to -lower-case. When @code{YY_USER_ACTION} is invoked, the variable +example, it could be set to call a routine to convert @code{yytext} to +lower-case. When the code fragment is invoked, the variable @code{yy_act} gives the number of the matched rule (rules are numbered starting with 1). Suppose you want to profile how often each of your rules is matched. The following would do the trick: -@cindex YY_USER_ACTION to track each time a rule is matched +@cindex pre-action to track each time a rule is matched @example @verbatim - #define YY_USER_ACTION ++ctr[yy_act] + %option pre-action="++ctr[yy_act]" @end verbatim @end example @vindex YY_NUM_RULES -where @code{ctr} is an array to hold the counts for the different rules. -Note that the macro @code{YY_NUM_RULES} gives the total number of rules -(including the default rule), even if you use @samp{-s)}, so a correct +where @code{ctr} is an array to hold the counts for the different +rules. Note that the public constant @code{YY_NUM_RULES} (a macro in +the default C/C++ back end) gives the total number of rules (including +the default rule), even if you use @samp{-s)}, so a correct declaration for @code{ctr} is: @example @@ -2342,51 +2391,55 @@ declaration for @code{ctr} is: @end verbatim @end example -@hkindex YY_USER_INIT -The macro @code{YY_USER_INIT} may be defined to provide an action which +@hkindex %option user-init +This option may be defined to provide an action which is always executed before the first scan (and before the scanner's internal initializations are done). For example, it could be used to call a routine to read in a data table or open a logging file. @findex yy_set_interactive -The macro @code{yy_set_interactive(is_interactive)} can be used to +The entry point @code{yy_set_interactive(is_interactive)} can be used to control whether the current buffer is considered @dfn{interactive}. An interactive buffer is processed more slowly, but must be used when the scanner's input source is indeed interactive to avoid problems due to waiting to fill buffers (see the discussion of the @samp{-I} flag in -@ref{Scanner Options}). A non-zero value in the macro invocation marks +@ref{Scanner Options}). Passing a boolean true (in C/C++, non-zero) value marks the buffer as interactive, a zero value as non-interactive. Note that -use of this macro overrides @code{%option always-interactive} or +use of this entry point overrides @code{%option always-interactive} or @code{%option never-interactive} (@pxref{Scanner Options}). @code{yy_set_interactive()} must be invoked prior to beginning to scan the buffer that is (or is not) to be considered interactive. @cindex BOL, setting it -@findex yy_set_bol -The macro @code{yy_set_bol(at_bol)} can be used to control whether the +@findex yysetbol +The rule hook @code{yysetbol(at_bol)} can be used to control whether the current buffer's scanning context for the next token match is done as -though at the beginning of a line. A non-zero macro argument makes +though at the beginning of a line. A non-zero argument makes rules anchored with @samp{^} active, while a zero argument makes @samp{^} rules inactive. @cindex BOL, checking the BOL flag -@findex YY_AT_BOL -The macro @code{YY_AT_BOL()} returns true if the next token scanned from +@cindex rule hook +@findex yyatbol +The rule hook @code{yyatbol()} returns true if the next token scanned from the current buffer will have @samp{^} rules active, false otherwise. -@cindex actions, redefining YY_BREAK -@hkindex YY_BREAK +@hkindex %option post-action In the generated scanner, the actions are all gathered in one large -switch statement and separated using @code{YY_BREAK}, which may be -redefined. By default, it is simply a @code{break}, to separate each -rule's action from the following rule's. Redefining @code{YY_BREAK} -allows, for example, C++ users to #define YY_BREAK to do nothing (while -being very careful that every rule ends with a @code{break} or a -@code{return}!) to avoid suffering from unreachable statement warnings -where because a rule's action ends with @code{return}, the -@code{YY_BREAK} is inaccessible. - -@node User Values, Yacc, Misc Macros, Top +switch statement and separated using a post-action fragment, which +may be redefined. By default, in C it is simply a @code{break}, to +separate each rule's action from the following rule's. Other target +languages may have different defaults for this action, often an empty +string. If a target language has no case statement this option will +probably be ineffective. + +Setting a post-action allows, for example, C++ users to suppress the +trailing break (while being very careful that every rule ends with a +@code{break} or a @code{return}!) to avoid suffering from unreachable +statement warnings where because a rule's action ends with +@code{return}, the break is inaccessible. + +@node User Values, Yacc, Misc Controls, Top @chapter Values Available To the User This chapter summarizes the various values available to the user in the @@ -2400,14 +2453,14 @@ lengthened (you cannot append characters to the end). @cindex yytext, default array size @cindex array, default size for yytext -@vindex YYLMAX +@vindex yylmax If the special directive @code{%array} appears in the first section of the scanner description, then @code{yytext} is instead declared -@code{char yytext[YYLMAX]}, where @code{YYLMAX} is a macro definition -that you can redefine in the first section if you don't like the default +to be an array of YYLMAX characters, where @code{YYLMAX} is a parameter +that you can redefine with a @code{%yylmax} option if you don't like the default value (generally 8KB). Using @code{%array} results in somewhat slower scanners, but the value of @code{yytext} becomes immune to calls to -@code{unput()}, which potentially destroy its value when @code{yytext} is +@code{yyunput()}, which potentially destroy its value when @code{yytext} is a character pointer. The opposite of @code{%array} is @code{%pointer}, which is the default. @@ -2421,17 +2474,21 @@ holds the length of the current token. @vindex yyin @item FILE *yyin -is the file which by default @code{flex} reads from. It may be +is the file which by default @code{flex} reads from. +In target languages other than C/C++, expect it to have +whatever natine type is associated with I/O streams. It may be redefined but doing so only makes sense before scanning begins or after an EOF has been encountered. Changing it in the midst of scanning will have unexpected results since @code{flex} buffers its input; use @code{yyrestart()} instead. Once scanning terminates because an -end-of-file has been seen, you can assign @file{yyin} at the new input +end-of-file has been seen, you can assign @file{yyin} to be the new input file and then call the scanner again to continue scanning. @findex yyrestart @item void yyrestart( FILE *new_file ) -may be called to point @file{yyin} at the new input file. The +may be called to point @file{yyin} at the new input file. +In target languages other than C/C++, expect the argument to have +whatever tyoe is associated with I/O streams. The switch-over to the new file is immediate (any previously buffered-up input is lost). Note that calling @code{yyrestart()} with @file{yyin} as an argument thus throws away the current input buffer and continues @@ -2439,17 +2496,19 @@ scanning the same input file. @vindex yyout @item FILE *yyout -is the file to which @code{ECHO} actions are done. It can be reassigned +is the output stream to which @code{yyecho()} actions are done. It can be reassigned by the user. +In target languages other than C/C++, expect it to have +whatever tyoe is associated with I/O streams. -@vindex YY_CURRENT_BUFFER -@item YY_CURRENT_BUFFER -returns a @code{YY_BUFFER_STATE} handle to the current buffer. +@vindex yy_current_buffer() +@item yy_current_buffer() +returns a @code{yybuffer} handle to the current buffer. -@vindex YY_START -@item YY_START +@vindex yystart() +@item yystart() returns an integer value corresponding to the current start condition. -You can subsequently use this value with @code{BEGIN} to return to that +You can subsequently use this value with @code{yybegin()} to return to that start condition. @end table @@ -2473,16 +2532,21 @@ is @code{TOK_NUMBER}, part of the scanner might look like: @cindex yacc interface @example @verbatim - %{ - #include "y.tab.h" - %} +%{ +#include "y.tab.h" +%} - %% +%% - [0-9]+ yylval = atoi( yytext ); return TOK_NUMBER; +[0-9]+ yylval = atoi( yytext ); return TOK_NUMBER; @end verbatim @end example +Bison is also retargetable to langages other than C. Outside the +C/C++ back end, it is likely that your Bison module will simply export +module-level constants that will be made visible to your scanner by +linkage or by explicit import statements. + @node Scanner Options, Performance, Yacc, Top @chapter Scanner Options @@ -2532,7 +2596,7 @@ The names are the same as their long-option equivalents (but without the leading @samp{--} ). @code{flex} scans your rule actions to determine whether you use the -@code{REJECT} or @code{yymore()} features. The @code{REJECT} and +@code{yyreject()} or @code{yymore()} features. The @code{yyreject()} and @code{yymore} options are available to override its decision as to whether you use the options, either by setting them (e.g., @code{%option reject)} to indicate the feature is indeed used, or unsetting them to @@ -2541,19 +2605,19 @@ indicate it actually is not used (e.g., @code{%option noyymore)}. A number of options are available for lint purists who want to suppress the appearance of unneeded routines in the generated scanner. Each of -the following, if unset (e.g., @code{%option nounput}), results in the +the following, if unset (e.g., @code{%option noyyunput}), results in the corresponding routine not appearing in the generated scanner: @example @verbatim - input, unput - yy_push_state, yy_pop_state, yy_top_state - yy_scan_buffer, yy_scan_bytes, yy_scan_string +input, yyunput +yy_push_state, yy_pop_state, yy_top_state +yy_scan_buffer, yy_scan_bytes, yy_scan_string - yyget_extra, yyset_extra, yyget_leng, yyget_text, - yyget_lineno, yyset_lineno, yyget_in, yyset_in, - yyget_out, yyset_out, yyget_lval, yyset_lval, - yyget_lloc, yyset_lloc, yyget_debug, yyset_debug +yyget_extra, yyset_extra, yyget_leng, yyget_text, +yyget_lineno, yyset_lineno, yyget_in, yyset_in, +yyget_out, yyset_out, yyget_lval, yyset_lval, +yyget_lloc, yyset_lloc, yyget_debug, yyset_debug @end verbatim @end example @@ -2571,7 +2635,7 @@ you use @code{%option stack)}. @item --header-file=FILE, @code{%option header-file="FILE"} instructs flex to write a C header to @file{FILE}. This file contains function prototypes, extern variables, and types used by the scanner. -Only the external API is exported by the header file. Many macros that +Only the external API is exported by the header file. Many rule hooks that are usable from within scanner actions are not exported to the header file. This is due to namespace problems and the goal of a clean external API. @@ -2581,7 +2645,8 @@ is substituted with the appropriate prefix. The @samp{--header-file} option is not compatible with the @samp{--c++} option, since the C++ scanner provides its own header in @file{yyFlexLexer.h}. - +This option will generally be a no-op under target languages other +than C. @anchor{option-outfile} @@ -2637,6 +2702,32 @@ the serialized tables match the in-code tables, instead of loading them. @section Options Affecting Scanner Behavior @table @samp +@anchor{option-emit} +@opindex -e +@opindex ---emit +@opindex emit +@item -e, --emit, @code{%option emit} +instructs @code{flex} to generate code using a specific back end other +than the default. The default is called ``cpp'', generates C or C++ +code, supports all legacy interfaces, and generates either +non-reentrant or re-entrant scanners. There is presently one +alternate back end, C99; it drops a lot of legacy interfaces, +generates only re-entrant scanners, and outputs modern C. The C99 +back end is intended to be a launching point for as yet unwritten back +ends generating scanners in other languages such as Go, Rust, Java, +Python, etc. + +@anchor{option-rewrite} +@item @code{%option rewrite} +Back ends other than the default C generator can't assume they have +the C preprocessor available. Therefore, Flex recognizes the +following function calls - yyecho(), yyless(), yymore(), yyinput(), +yystart(), yybegin(), yyunput(), yypanic() - and rewreites them adding a +yyscanner final argument as though they were C macros. This optiion +is set to false whrn you select or default to the C back end, and to +true otherwise. You can turn it off (or on) explicitly after your +emit option. + @anchor{option-case-insensitive} @opindex -i @opindex ---case-insensitive @@ -2649,7 +2740,6 @@ text given in @code{yytext} will have the preserved case (i.e., it will not be folded). For tricky behavior, see @ref{case and character ranges}. - @anchor{option-lex-compat} @opindex -l @opindex ---lex-compat @@ -2660,7 +2750,9 @@ implementation. Note that this does not mean @emph{full} compatibility. Use of this option costs a considerable amount of performance, and it cannot be used with the @samp{--c++}, @samp{--full}, @samp{--fast}, @samp{-Cf}, or @samp{-CF} options. For details on the compatibilities it provides, see -@ref{Lex and Posix}. This option also results in the name +@ref{Lex and Posix}. It will usuually be a no-op on back ends other +than C/C++. +This option also results in the name @code{YY_FLEX_LEX_COMPAT} being @code{#define}'d in the generated scanner. @@ -2841,7 +2933,7 @@ is performed in @code{yylex_init} at runtime. directs @code{flex} to generate a scanner that maintains the number of the current line read from its input in the global variable @code{yylineno}. This option is implied by @code{%option -lex-compat}. In a reentrant C scanner, the macro @code{yylineno} is +lex-compat}. In a reentrant C scanner, @code{yylineno} is accessible regardless of the value of @code{%option yylineno}, however, its value is not modified by @code{flex} unless @code{%option yylineno} is enabled. @@ -3037,6 +3129,13 @@ scanner, which simply calls @code{yylex()}. This option implies @code{noyywrap} (see below). +@anchor{option-yyterminate} +@opindex yyterminate +@hkindex yyterminate +@item @code{%option yyterminate} +This is a string-valuued option with which you can specify an expansion +of the yyterminate() hook. whuch normally causes the generated +scanner to return 0 as an end-of-input indication. @anchor{option-nounistd} @opindex ---nounistd @@ -3045,12 +3144,17 @@ scanner, which simply calls @code{yylex()}. This option implies suppresses inclusion of the non-ANSI header file @file{unistd.h}. This option is meant to target environments in which @file{unistd.h} does not exist. Be aware that certain options may cause flex to generate code that relies on functions -normally found in @file{unistd.h}, (e.g. @code{isatty()}, @code{read()}.) +normally found in @file{unistd.h}, (e.g. @code{isatty()}.) If you wish to use these functions, you will have to inform your compiler where to find them. -@xref{option-always-interactive}. @xref{option-read}. - +This option is obsolete, as after Flex was originally written +@file{unistd.h} became part of the Single Unix Specification and is +consequently everywhere; thuas there is no reason to use this option +on modern systems. It is included so as not to break compatibility +with old build scripts, and will have no effect on backends other than +the default C one. +@xref{option-always-interactive}. @xref{option-read}. @anchor{option-yyclass} @opindex ---yyclass @@ -3140,7 +3244,7 @@ array look-up per character scanned). @opindex ---read @opindex read @item -Cr, --read, @code{%option read} -causes the generated scanner to @emph{bypass} use of the standard I/O +causes scanner generated with the C/C++ back end to @emph{bypass} use of the standard I/O library (@code{stdio}) for input. Instead of calling @code{fread()} or @code{getc()}, the scanner will use the @code{read()} system call, resulting in a performance gain which varies from system to system, but @@ -3149,7 +3253,9 @@ or @samp{-CF}. Using @samp{-Cr} can cause strange behavior if, for example, you read from @file{yyin} using @code{stdio} prior to calling the scanner (because the scanner will miss whatever text your previous reads left in the @code{stdio} input buffer). @samp{-Cr} has no effect -if you define @code{YY_INPUT()} (@pxref{Generated Scanner}). +if you define @code{yyread()} (@pxref{Generated Scanner}). It may +be a no-op or enable different optimizations in back ends other than +the default C/C++ one. @end table The options @samp{-Cf} or @samp{-CF} and @samp{-Cm} do not make sense @@ -3284,7 +3390,7 @@ cause a serious loss of performance in the resulting scanner. If you give the flag twice, you will also get comments regarding features that lead to minor performance losses. -Note that the use of @code{REJECT}, and +Note that the use of @code{yyreject()}, and variable trailing context (@pxref{Limitations}) entails a substantial performance penalty; use of @code{yymore()}, the @samp{^} operator, and the @samp{--interactive} flag entail minor performance penalties. @@ -3346,6 +3452,14 @@ warn about certain things. In particular, if the default rule can be matched but no default rule has been given, the flex will warn you. We recommend using this option always. +@anchor{option-bufsize} +@opindex bufsize +@item @code{%option bufsize} +Forces the input buffer size, which is normally set to a good default +for your platform based on what the standard buffered-IO library in +your traget language does. This option is mainly intended for stress-testing +memory allocation in generated scanners; you probably shouldn't set it. + @end table @node Miscellaneous Options, , Debugging Options, Scanner Options @@ -3385,12 +3499,12 @@ rules. Aside from the effects on scanner speed of the table compression @samp{-C} options outlined above, there are a number of options/actions which degrade performance. These are, from most expensive to least: -@cindex REJECT, performance costs +@cindex yyreject(), performance costs @cindex yylineno, performance costs @cindex trailing context, performance costs @example @verbatim - REJECT + yyreject() arbitrary trailing context pattern sets that require backing up @@ -3406,12 +3520,12 @@ which degrade performance. These are, from most expensive to least: @end example with the first two all being quite expensive and the last two being -quite cheap. Note also that @code{unput()} is implemented as a routine -call that potentially does quite a bit of work, while @code{yyless()} is -a quite-cheap macro. So if you are just putting back some excess text +quite cheap. Note also that @code{yyunput()} is implemented as a routine +call that potentially does quite a bit of work, while @code{yyless()} +is very inexpensive. So if you are just putting back some excess text you scanned, use @code{yyless()}. -@code{REJECT} should be avoided at all costs when performance is +@code{yyreject()} should be avoided at all costs when performance is important. It is a particularly expensive option. There is one case when @code{%option yylineno} can be expensive. That is when @@ -3504,16 +3618,16 @@ The way to remove the backing up is to add ``error'' rules: @cindex backing up, eliminating by adding error rules @example @verbatim - %% - foo return TOK_KEYWORD; - foobar return TOK_KEYWORD; - - fooba | - foob | - fo { - /* false alarm, not really a keyword */ - return TOK_ID; - } +%% +foo return TOK_KEYWORD; +foobar return TOK_KEYWORD; + +fooba | +foob | +fo { + /* false alarm, not really a keyword */ + return TOK_ID; + } @end verbatim @end example @@ -3523,11 +3637,11 @@ Eliminating backing up among a list of keywords can also be done using a @cindex backing up, eliminating with catch-all rule @example @verbatim - %% - foo return TOK_KEYWORD; - foobar return TOK_KEYWORD; +%% +foo return TOK_KEYWORD; +foobar return TOK_KEYWORD; - [a-z]+ return TOK_ID; +[a-z]+ return TOK_ID; @end verbatim @end example @@ -3546,14 +3660,14 @@ Leaving just one means you gain nothing. @emph{Variable} trailing context (where both the leading and trailing parts do not have a fixed length) entails almost the same performance -loss as @code{REJECT} (i.e., substantial). So when possible a rule +loss as @code{yyreject()} (i.e., substantial). So when possible a rule like: @cindex trailing context, variable length @example @verbatim - %% - mouse|rat/(cat|dog) run(); +%% +mouse|rat/(cat|dog) run(); @end verbatim @end example @@ -3561,9 +3675,9 @@ is better written: @example @verbatim - %% - mouse/cat|dog run(); - rat/cat|dog run(); +%% +mouse/cat|dog run(); +rat/cat|dog run(); @end verbatim @end example @@ -3571,9 +3685,9 @@ or as @example @verbatim - %% - mouse|rat/cat run(); - mouse|rat/dog run(); +%% +mouse|rat/cat run(); +mouse|rat/dog run(); @end verbatim @end example @@ -3591,16 +3705,16 @@ additional work of setting up the scanning environment (e.g., @cindex performance optimization, matching longer tokens @example @verbatim - %x comment - %% - int line_num = 1; +%x comment +%% + int line_num = 1; - "/*" BEGIN(comment); +"/*" yybegin(comment); - <comment>[^*\n]* - <comment>"*"+[^*/\n]* - <comment>\n ++line_num; - <comment>"*"+"/" BEGIN(INITIAL); +<comment>[^*\n]* +<comment>"*"+[^*/\n]* +<comment>\n ++line_num; +<comment>"*"+"/" yybegin(INITIAL); @end verbatim @end example @@ -3608,17 +3722,17 @@ This could be sped up by writing it as: @example @verbatim - %x comment - %% - int line_num = 1; +%x comment +%% + int line_num = 1; - "/*" BEGIN(comment); +"/*" yybegin(comment); - <comment>[^*\n]* - <comment>[^*\n]*\n ++line_num; - <comment>"*"+[^*/\n]* - <comment>"*"+[^*/\n]*\n ++line_num; - <comment>"*"+"/" BEGIN(INITIAL); +<comment>[^*\n]* +<comment>[^*\n]*\n ++line_num; +<comment>"*"+[^*/\n]* +<comment>"*"+[^*/\n]*\n ++line_num; +<comment>"*"+"/" yybegin(INITIAL); @end verbatim @end example @@ -3640,15 +3754,15 @@ keywords. A natural first approach is: @cindex performance optimization, recognizing keywords @example @verbatim - %% - asm | - auto | - break | - ... etc ... - volatile | - while /* it's a keyword */ +%% +asm | +auto | +break | +... etc ... +volatile | +while /* it's a keyword */ - .|\n /* it's not a keyword */ +.|\n /* it's not a keyword */ @end verbatim @end example @@ -3656,16 +3770,16 @@ To eliminate the back-tracking, introduce a catch-all rule: @example @verbatim - %% - asm | - auto | - break | - ... etc ... - volatile | - while /* it's a keyword */ +%% +asm | +auto | +break | +... etc ... +volatile | +while /* it's a keyword */ - [a-z]+ | - .|\n /* it's not a keyword */ +[a-z]+ | +.|\n /* it's not a keyword */ @end verbatim @end example @@ -3675,16 +3789,16 @@ recognition of newlines with that of the other tokens: @example @verbatim - %% - asm\n | - auto\n | - break\n | - ... etc ... - volatile\n | - while\n /* it's a keyword */ +%% +asm\n | +auto\n | +break\n | +... etc ... +volatile\n | +while\n /* it's a keyword */ - [a-z]+\n | - .|\n /* it's not a keyword */ +[a-z]+\n | +.|\n /* it's not a keyword */ @end verbatim @end example @@ -3706,17 +3820,17 @@ one which doesn't include a newline: @example @verbatim - %% - asm\n | - auto\n | - break\n | - ... etc ... - volatile\n | - while\n /* it's a keyword */ +%% +asm\n | +auto\n | +break\n | +... etc ... +volatile\n | +while\n /* it's a keyword */ - [a-z]+\n | - [a-z]+ | - .|\n /* it's not a keyword */ +[a-z]+\n | +[a-z]+ | +.|\n /* it's not a keyword */ @end verbatim @end example @@ -3799,7 +3913,7 @@ returns the current setting of the debugging flag. Also provided are member functions equivalent to @code{yy_switch_to_buffer()}, @code{yy_create_buffer()} (though the first argument is an @code{istream&} object reference and not a -@code{FILE*)}, @code{yy_flush_buffer()}, @code{yy_delete_buffer()}, and +@code{FILE*)}, @code{yy_flush_current_buffer()}, @code{yy_delete_buffer()}, and @code{yyrestart()} (again, the first argument is a @code{istream&} object reference). @@ -3889,69 +4003,69 @@ Here is an example of a simple C++ scanner: @cindex C++ scanners, use of @example @verbatim - // An example of using the flex C++ scanner class. + // An example of using the flex C++ scanner class. - %{ - #include <iostream> - using namespace std; - int mylineno = 0; - %} +%{ +#include <iostream> +using namespace std; +int mylineno = 0; +%} - %option noyywrap c++ +%option noyywrap c++ - string \"[^\n"]+\" +string \"[^\n"]+\" - ws [ \t]+ +ws [ \t]+ - alpha [A-Za-z] - dig [0-9] - name ({alpha}|{dig}|\$)({alpha}|{dig}|[_.\-/$])* - num1 [-+]?{dig}+\.?([eE][-+]?{dig}+)? - num2 [-+]?{dig}*\.{dig}+([eE][-+]?{dig}+)? - number {num1}|{num2} +alpha [A-Za-z] +dig [0-9] +name ({alpha}|{dig}|\$)({alpha}|{dig}|[_.\-/$])* +num1 [-+]?{dig}+\.?([eE][-+]?{dig}+)? +num2 [-+]?{dig}*\.{dig}+([eE][-+]?{dig}+)? +number {num1}|{num2} - %% +%% - {ws} /* skip blanks and tabs */ +{ws} /* skip blanks and tabs */ - "/*" { - int c; +"/*" { + int c; - while((c = yyinput()) != 0) + while((c = yyinput()) != 0) + { + if(c == '\n') + ++mylineno; + + else if(c == '*') { - if(c == '\n') - ++mylineno; - - else if(c == '*') - { - if((c = yyinput()) == '/') - break; - else - unput(c); - } + if((c = yyinput()) == '/') + break; + else + yyunput(c); } } + } - {number} cout << "number " << YYText() << '\n'; +{number} cout << "number " << YYText() << '\n'; - \n mylineno++; +\n mylineno++; - {name} cout << "name " << YYText() << '\n'; +{name} cout << "name " << YYText() << '\n'; - {string} cout << "string " << YYText() << '\n'; +{string} cout << "string " << YYText() << '\n'; - %% +%% - // This include is required if main() is an another source file. - //#include <FlexLexer.h> + // This include is required if main() is an another source file. + //#include <FlexLexer.h> - int main( int /* argc */, char** /* argv */ ) - { - FlexLexer* lexer = new yyFlexLexer; - while(lexer->yylex() != 0) - ; - return 0; - } +int main( int /* argc */, char** /* argv */ ) +{ + FlexLexer* lexer = new yyFlexLexer; + while(lexer->yylex() != 0) + ; + return 0; +} @end verbatim @end example @@ -3991,6 +4105,9 @@ control. The most common use for reentrant scanners is from within multi-threaded applications. Any thread may create and execute a reentrant @code{flex} scanner without the need for synchronization with other threads. +All scanners generated by back ends other than the default C/C++ back +end are reentrant. + @menu * Reentrant Uses:: * Reentrant Overview:: @@ -4009,18 +4126,18 @@ the token level (i.e., instead of at the character level): @cindex reentrant scanners, multiple interleaved scanners @example @verbatim - /* Example of maintaining more than one active scanner. */ +/* Example of maintaining more than one active scanner. */ - do { - int tok1, tok2; +do { + int tok1, tok2; - tok1 = yylex( scanner_1 ); - tok2 = yylex( scanner_2 ); + tok1 = yylex( scanner_1 ); + tok2 = yylex( scanner_2 ); - if( tok1 != tok2 ) - printf("Files are different."); + if( tok1 != tok2 ) + printf("Files are different."); - } while ( tok1 && tok2 ); +} while ( tok1 && tok2 ); @end verbatim @end example @@ -4034,26 +4151,26 @@ another instance of itself. @cindex reentrant scanners, recursive invocation @example @verbatim - /* Example of recursive invocation. */ +/* Example of recursive invocation. */ - %option reentrant +%option reentrant - %% - "eval(".+")" { - yyscan_t scanner; - YY_BUFFER_STATE buf; +%% +"eval(".+")" { + yyscan_t scanner; + yybuffer buf; - yylex_init( &scanner ); - yytext[yyleng-1] = ' '; + yylex_init( &scanner ); + yytext[yyleng-1] = ' '; - buf = yy_scan_string( yytext + 5, scanner ); - yylex( scanner ); + buf = yy_scan_string( yytext + 5, scanner ); + yylex( scanner ); - yy_delete_buffer(buf,scanner); - yylex_destroy( scanner ); - } - ... - %% + yy_delete_buffer(buf,scanner); + yylex_destroy( scanner ); + } +... +%% @end verbatim @end example @@ -4071,12 +4188,15 @@ scanners. Here is a quick overview of the API: All functions take one additional argument: @code{yyscanner} @item -All global variables are replaced by their macro equivalents. -(We tell you this because it may be important to you during debugging.) +In C/C++, all global variables are replaced by their macro equivalents. +(We tell you this because it may be important to you during +debugging.) This is a historical-compatibilty hack; other back ends +probably will not emulate it. @item -@code{yylex_init} and @code{yylex_destroy} must be called before and -after @code{yylex}, respectively. +In the default C/C++ @code{yylex_init} and @code{yylex_destroy} must +be called before and after @code{yylex}, respectively. Other back ends +may or may not require this. @item Accessor methods (get/set functions) provide access to common @@ -4093,37 +4213,37 @@ First, an example of a reentrant scanner: @cindex reentrant, example of @example @verbatim - /* This scanner prints "//" comments. */ +/* This scanner prints "//" comments. */ - %option reentrant stack noyywrap - %x COMMENT +%option reentrant stack noyywrap +%x COMMENT - %% +%% - "//" yy_push_state( COMMENT, yyscanner); - .|\n +"//" yy_push_state( COMMENT, yyscanner); +.|\n - <COMMENT>\n yy_pop_state( yyscanner ); - <COMMENT>[^\n]+ fprintf( yyout, "%s\n", yytext); +<COMMENT>\n yy_pop_state( yyscanner ); +<COMMENT>[^\n]+ fprintf( yyout, "%s\n", yytext); - %% +%% - int main ( int argc, char * argv[] ) - { - yyscan_t scanner; +int main ( int argc, char * argv[] ) +{ + yyscan_t scanner; - yylex_init ( &scanner ); - yylex ( scanner ); - yylex_destroy ( scanner ); + yylex_init ( &scanner ); + yylex ( scanner ); + yylex_destroy ( scanner ); return 0; - } +} @end verbatim @end example @node Reentrant Detail, Reentrant Functions, Reentrant Example, Reentrant @section The Reentrant API in Detail -Here are the things you need to do or know to use the reentrant C API of +Here are the things you need to do or know to use the reentrant API of @code{flex}. @menu @@ -4139,21 +4259,20 @@ Here are the things you need to do or know to use the reentrant C API of @node Specify Reentrant, Extra Reentrant Argument, Reentrant Detail, Reentrant Detail @subsection Declaring a Scanner As Reentrant - %option reentrant (--reentrant) must be specified. - Notice that @code{%option reentrant} is specified in the above example -(@pxref{Reentrant Example}. Had this option not been specified, -@code{flex} would have happily generated a non-reentrant scanner without -complaining. You may explicitly specify @code{%option noreentrant}, if -you do @emph{not} want a reentrant scanner, although it is not -necessary. The default is to generate a non-reentrant scanner. +(@pxref{Reentrant Example}. Had this option not been specified with +the default C/C++ back end, @code{flex} would have happily generated a +non-reentrant scanner without complaining. You may explicitly specify +@code{%option noreentrant}, if you do @emph{not} want a reentrant +scanner, although it is not necessary - and not effective in any other +target language. @node Extra Reentrant Argument, Global Replacement, Specify Reentrant, Reentrant Detail @subsection The Extra Argument @cindex reentrant, calling functions @vindex yyscanner (reentrant only) -All functions take one additional argument: @code{yyscanner}. +All functions other than rule hooks take one additional argument: @code{yyscanner}. Notice that the calls to @code{yy_push_state} and @code{yy_pop_state} both have an argument, @code{yyscanner} , that is not present in a @@ -4169,30 +4288,43 @@ non-reentrant scanner. Here are the declarations of Notice that the argument @code{yyscanner} appears in the declaration of both functions. In fact, all @code{flex} functions in a reentrant -scanner have this additional argument. It is always the last argument +scanner have this additional argument, except for rule hooks which +get it supplied implicitly. + +It is always the last argument in the argument list, it is always of type @code{yyscan_t} (which is typedef'd to @code{void *}) and it is always named @code{yyscanner}. As you may have guessed, @code{yyscanner} is a pointer to an opaque data structure encapsulating the current state of the scanner. For a list of function declarations, -see @ref{Reentrant Functions}. Note that preprocessor macros, such as -@code{BEGIN}, @code{ECHO}, and @code{REJECT}, do not take this +see @ref{Reentrant Functions}. Rule hooks, such as +@code{yybegin()}, @code{yyecho()}, @code{yyreject()}, and @code{yystart()}, do not take this additional argument. +Rule hooks don't need to take a scanner context argument because, +under the hood, the context is supplied by the call location. + +The type name @code{yscan_t} follows C conventions. It may differ in +other target languages. + @node Global Replacement, Init and Destroy Functions, Extra Reentrant Argument, Reentrant Detail @subsection Global Variables Replaced By Macros @cindex reentrant, accessing flex variables -All global variables in traditional flex have been replaced by macro equivalents. +In the C/C++ back end global variables in traditional flex have been +replaced by macro equivalents. Be aware that this will not be true in target +languages without macros, so relying on this backward-compatibility +hack will hinder forward portability. -Note that in the above example, @code{yyout} and @code{yytext} are +Accordingly, in the above example, @code{yyout} and @code{yytext} are not plain variables. These are macros that will expand to their equivalent lvalue. All of the familiar @code{flex} globals have been replaced by their macro equivalents. In particular, @code{yytext}, @code{yyleng}, @code{yylineno}, @code{yyin}, @code{yyout}, @code{yyextra}, @code{yylval}, and @code{yylloc} are macros. You may safely use these macros in actions as if they were plain variables. We only tell you this so you don't expect to link to these variables -externally. Currently, each macro expands to a member of an internal struct, e.g., +externally. Currently, each macro expands to a member of an internal +struct, e.g., in C/C++: @example @verbatim @@ -4218,8 +4350,10 @@ to accomplish this. (See below). @findex yylex_init @findex yylex_destroy -@code{yylex_init} and @code{yylex_destroy} must be called before and -after @code{yylex}, respectively. +In the default C/C++ back end @code{yylex_init} and +@code{yylex_destroy} must be called before and after @code{yylex}, +respectively. This may not be true in other target langages, +especially those with automatic memory management. @example @verbatim @@ -4230,6 +4364,12 @@ after @code{yylex}, respectively. @end verbatim @end example +In these declarations, @code{YY_EXTRA_TYPE} is a placeholder for the type +specifier in the scanner's @code{extra-type} option. + +(The scanner type and type declaration syntax will be different in target +languages other than C/C++.) + The function @code{yylex_init} must be called before calling any other function. The argument to @code{yylex_init} is the address of an uninitialized pointer to be filled in by @code{yylex_init}, overwriting @@ -4250,8 +4390,10 @@ takes one argument, which is the value returned (via an argument) by @code{yylex_init}. Otherwise, it behaves the same as the non-reentrant version of @code{yylex}. -Both @code{yylex_init} and @code{yylex_init_extra} returns 0 (zero) on success, -or non-zero on failure, in which case errno is set to one of the following values: +With the C/C++ back end, both @code{yylex_init} and +@code{yylex_init_extra} return 0 (zero) on success or non-zero on +failure. @code{errno} is also set to one of the following values on +failure: @itemize @item ENOMEM @@ -4260,6 +4402,8 @@ Memory allocation error. @xref{memory-management}. Invalid argument. @end itemize +Other target languages may use different means of passing back an +error indication. The function @code{yylex_destroy} should be called to free resources used by the scanner. After @code{yylex_destroy} @@ -4327,7 +4471,7 @@ The above code may be called from within an action like this: @end verbatim @end example -You may find that @code{%option header-file} is particularly useful for generating +In C and C++, you may find that @code{%option header-file} is particularly useful for generating prototypes of all the accessor functions. @xref{option-header}. @node Extra Data, About yyscan_t, Accessor Methods, Reentrant Detail @@ -4352,57 +4496,62 @@ from outside the scanner, and through the shortcut macro @code{yyextra} from within the scanner itself. They are defined as follows: -@tindex YY_EXTRA_TYPE (reentrant only) +@tindex %option extra-type (reentrant only) @findex yyget_extra @findex yyset_extra @example @verbatim - #define YY_EXTRA_TYPE void* + option extra-type="void *" YY_EXTRA_TYPE yyget_extra ( yyscan_t scanner ); void yyset_extra ( YY_EXTRA_TYPE arbitrary_data , yyscan_t scanner); @end verbatim @end example +In these declarations, @code{YY_EXTRA_TYPE} is as before a placeholder for the type +specifier in the scanner's @code{extra-type} option. + In addition, an extra form of @code{yylex_init} is provided, @code{yylex_init_extra}. This function is provided so that the yyextra value can be accessed from within the very first yyalloc, used to allocate the scanner itself. -By default, @code{YY_EXTRA_TYPE} is defined as type @code{void *}. You -may redefine this type using @code{%option extra-type="your_type"} in -the scanner: +In the default C/C++ back end, the extra field in your scanner state +structure is defined as type @code{void *}; in other target languages +it might be a generic type, or no such field might be genetated at +all. In back ends which support this extra field, you may redefine +the type using @code{%option extra-type="your_type"} in the scanner: @cindex YY_EXTRA_TYPE, defining your own type @example @verbatim - /* An example of overriding YY_EXTRA_TYPE. */ - %{ - #include <sys/stat.h> - #include <unistd.h> - %} - %option reentrant - %option extra-type="struct stat *" - %% +/* An example of overriding YY_EXTRA_TYPE. */ +%{ +#include <sys/stat.h> +#include <unistd.h> +%} +%option reentrant +%option extra-type="struct stat *" +%% - __filesize__ printf( "%ld", yyextra->st_size ); - __lastmod__ printf( "%ld", yyextra->st_mtime ); - %% - void scan_file( char* filename ) - { - yyscan_t scanner; - struct stat buf; - FILE *in; +__filesize__ printf( "%ld", yyextra->st_size ); +__lastmod__ printf( "%ld", yyextra->st_mtime ); +%% +void scan_file( char* filename ) +{ + yyscan_t scanner; + struct stat buf; + FILE *in; - in = fopen( filename, "r" ); - stat( filename, &buf ); + in = fopen( filename, "r" ); + stat( filename, &buf ); - yylex_init_extra( buf, &scanner ); - yyset_in( in, scanner ); - yylex( scanner ); - yylex_destroy( scanner ); + yylex_init_extra( buf, &scanner ); + yyset_in( in, scanner ); + yylex( scanner ); + yylex_destroy( scanner ); - fclose( in ); - } + fclose( in ); +} @end verbatim @end example @@ -4411,7 +4560,7 @@ the scanner: @subsection About yyscan_t @tindex yyscan_t (reentrant only) -@code{yyscan_t} is defined as: +In C/C++, @code{yyscan_t} is defined as: @example @verbatim @@ -4425,7 +4574,7 @@ directly. In particular, you should never attempt to free it (use @code{yylex_destroy()} instead.) @node Reentrant Functions, , Reentrant Detail, Reentrant -@section Functions and Macros Available in Reentrant C Scanners +@section Functions Available in Reentrant Scanners The following Functions are available in a reentrant scanner: @@ -4462,7 +4611,7 @@ The following Functions are available in a reentrant scanner: There are no ``set'' functions for yytext and yyleng. This is intentional. -The following Macro shortcuts are available in actions in a reentrant +In the C/C++ back end, the following macro shortcuts are available in actions in a reentrant scanner: @example @@ -4522,7 +4671,7 @@ implementations do not share any code, though), with some extensions and incompatibilities, both of which are of concern to those who wish to write scanners acceptable to both implementations. @code{flex} is fully compliant with the POSIX @code{lex} specification, except that when -using @code{%pointer} (the default), a call to @code{unput()} destroys +using @code{%pointer} (the default), a call to @code{yyunput()} destroys the contents of @code{yytext}, which is counter to the POSIX specification. In this section we discuss all of the known areas of incompatibility between @code{flex}, AT&T @code{lex}, and the POSIX @@ -4546,7 +4695,8 @@ a per-scanner (single global variable) basis. @code{yylineno} is not part of the POSIX specification. @item -The @code{input()} routine is not redefinable, though it may be called +The @code{input()} routine (which has become @code{yyinput()} in modern +Flex) is not redefinable, though it may be called to read characters following whatever has been matched by a rule. If @code{input()} encounters an end-of-file the normal @code{yywrap()} processing is done. A ``real'' end-of-file is returned by @@ -4562,7 +4712,7 @@ specify any way of controlling the scanner's input other than by making an initial assignment to @file{yyin}. @item -The @code{unput()} routine is not redefinable. This restriction is in +The @code{yyunput()} routine is not redefinable. This restriction is in accordance with POSIX. @item @@ -4598,7 +4748,7 @@ reentrant, so if using C++ is an option for you, you should use them instead. @xref{Cxx}, and @ref{Reentrant} for details. @item -@code{output()} is not supported. Output from the @b{ECHO} macro is +@code{output()} is not supported. Output from the @b{yyecho()} macro is done to the file-pointer @code{yyout} (default @file{stdout)}. @item @@ -4662,7 +4812,7 @@ The @code{lex} @code{%r} (generate a Ratfor scanner) option is not supported. It is not part of the POSIX specification. @item -After a call to @code{unput()}, @emph{yytext} is undefined until the +After a call to @code{yyunput()}, @emph{yytext} is undefined until the next token is matched, unless the scanner was built using @code{%array}. This is not the case with @code{lex} or the POSIX specification. The @samp{-l} option does away with this incompatibility. @@ -4688,14 +4838,15 @@ The special table-size declarations such as @code{%a} supported by @code{lex} are not required by @code{flex} scanners.. @code{flex} ignores them. @item -The name @code{FLEX_SCANNER} is @code{#define}'d so scanners may be +In the C/C++ back end name @code{FLEX_SCANNER} is @code{#define}'d so scanners may be written for use with either @code{flex} or @code{lex}. Scanners also include @code{YY_FLEX_MAJOR_VERSION}, @code{YY_FLEX_MINOR_VERSION} and @code{YY_FLEX_SUBMINOR_VERSION} indicating which version of @code{flex} generated the scanner. For example, for the 2.5.22 release, these defines would be 2, 5 and 22 respectively. If the version of @code{flex} being used is a beta -version, then the symbol @code{FLEX_BETA} is defined. +version, then the symbol @code{FLEX_BETA} is defined (on the default C +back end only). @item The symbols @samp{[[} and @samp{]]} in the code sections of the input @@ -4727,20 +4878,20 @@ yyterminate() @item yy_set_interactive() @item -yy_set_bol() +yysetbol() @item -YY_AT_BOL() +yyatbol() <<EOF>> @item <*> @item -YY_DECL +%yydecl @item -YY_START +yystart() @item -YY_USER_ACTION +%option pre-action @item -YY_USER_INIT +%option user-init @item #line directives @item @@ -4773,7 +4924,8 @@ is (rather surprisingly) truncated to @end example @code{flex} does not truncate the action. Actions that are not enclosed -in braces are simply terminated at the end of the line. +in braces are simply terminated at the end of the line. It is good +style to use the explicit braces, though. @node Memory Management, Serialized Tables, Lex and Posix, Top @chapter Memory Management @@ -4781,7 +4933,8 @@ in braces are simply terminated at the end of the line. @cindex memory management @anchor{memory-management} This chapter describes how flex handles dynamic memory, and how you can -override the default behavior. +override the default behavior. You can safely skip it if your target +language has automatic memory management. @menu * The Default Memory Management:: @@ -4792,7 +4945,12 @@ override the default behavior. @node The Default Memory Management, Overriding The Default Memory Management, Memory Management, Memory Management @section The Default Memory Management -Flex allocates dynamic memory during initialization, and once in a while from +This section applies only to target languages with manual memory +allocation, including the default C/C++ back end. If your target +language has garbage collection you can safely ignore it. + +A Flex-generated scanner +allocates dynamic memory during initialization, and once in a while from within a call to yylex(). Initialization takes place during the first call to yylex(). Thereafter, flex may reallocate more memory if it needs to enlarge a buffer. As of version 2.5.9 Flex will clean up all memory when you call @code{yylex_destroy} @@ -4800,7 +4958,7 @@ buffer. As of version 2.5.9 Flex will clean up all memory when you call @code{yy Flex allocates dynamic memory for four purposes, listed below @footnote{The quantities given here are approximate, and may vary due to host architecture, -compiler configuration, or due to future enhancements to flex.} +compiler configuration, or due to future enhancements to flex.} @table @asis @@ -4817,7 +4975,7 @@ you must @code{#define YY_BUF_SIZE} to whatever number of bytes you want. We don to change this in the near future, but we reserve the right to do so if we ever add a more robust memory management API. -@item 64kb for the REJECT state. This will only be allocated if you use REJECT. +@item 64kb for the yyreject() state. This will only be allocated if you use yyreject(). The size is large enough to hold the same number of states as characters in the input buffer. If you override the size of the input buffer (via @code{YY_BUF_SIZE}), then you automatically override the size of this buffer as well. @@ -4830,8 +4988,8 @@ specified. You will rarely need to tune this buffer. The ideal size for this stack is the maximum depth expected. The memory for this stack is automatically destroyed when you call yylex_destroy(). @xref{option-stack}. -@item 40 bytes for each YY_BUFFER_STATE. -Flex allocates memory for each YY_BUFFER_STATE. The buffer state itself +@item 40 bytes for each yybuffer. +Flex allocates memory for each yybuffer. The buffer state itself is about 40 bytes, plus an additional large character buffer (described above.) The initial buffer state is created during initialization, and with each call to yy_create_buffer(). You can't tune the size of this, but you can tune the @@ -4855,6 +5013,8 @@ you call yylex_init(). It is destroyed when the user calls yylex_destroy(). @node Overriding The Default Memory Management, A Note About yytext And Memory, The Default Memory Management, Memory Management @section Overriding The Default Memory Management +Again, this section applies only to languages with manual memory management. + @cindex yyalloc, overriding @cindex yyrealloc, overriding @cindex yyfree, overriding @@ -4915,13 +5075,10 @@ custom allocator through @code{yyextra}. /* Suppress the default implementations. */ %option noyyalloc noyyrealloc noyyfree %option reentrant +%option extra-type="struct allocator*" /* Initialize the allocator. */ -%{ -#define YY_EXTRA_TYPE struct allocator* -#define YY_USER_INIT yyextra = allocator_create(); -%} - +%option user-init="yyextra = allocator_create();" %% .|\n ; %% @@ -4948,6 +5105,8 @@ void yyfree (void * ptr, void * yyscanner) { @cindex yytext, memory considerations +This section applies only to target languages with manual memory management. + When flex finds a match, @code{yytext} points to the first character of the match in the input buffer. The string itself is part of the input buffer, and is @emph{NOT} allocated separately. The value of yytext will be overwritten the next @@ -4982,6 +5141,8 @@ but none of them at the same time. The serialization feature allows the tables to be loaded at runtime, before scanning begins. The tables may be discarded when scanning is finished. +Note: This feature is available only when using the default C/C++ back end. + @menu * Creating Serialized Tables:: * Loading and Unloading Serialized Tables:: @@ -5148,10 +5309,10 @@ the version of flex that was used to create the serialized tables. @item th_name[] Contains the name of this table set. The default is @samp{yytables}, -and is prefixed accordingly, e.g., @samp{footables}. Must be NULL-terminated. +and is prefixed accordingly, e.g., @samp{footables}. Must be NUL-terminated. @item th_pad64[] -Zero or more NULL bytes, padding the entire header to the next 64-bit boundary +Zero or more NUL bytes, padding the entire header to the next 64-bit boundary as calculated from the beginning of the header. @end table @@ -5239,7 +5400,7 @@ The table data. This array may be a one- or two-dimensional array, of type @code{td_flags}, @code{td_hilen}, and @code{td_lolen} fields. @item td_pad64[] -Zero or more NULL bytes, padding the entire table to the next 64-bit boundary as +Zero or more NUL bytes, padding the entire table to the next 64-bit boundary as calculated from the beginning of this table. @end table @@ -5266,7 +5427,7 @@ matched because it comes after an identifier ``catch-all'' rule: @end verbatim @end example -Using @code{REJECT} in a scanner suppresses this warning. +Using @code{yyreject()} in a scanner suppresses this warning. @item @samp{warning, -s option given but default rule can be matched} means @@ -5278,7 +5439,7 @@ not intended. @item @code{reject_used_but_not_detected undefined} or @code{yymore_used_but_not_detected undefined}. These errors can occur -at compile time. They indicate that the scanner uses @code{REJECT} or +at compile time. They indicate that the scanner uses @code{yyreject()} or @code{yymore()} but that @code{flex} failed to notice the fact, meaning that @code{flex} scanned the first two sections looking for occurrences of these actions and failed to find any, but somehow you snuck some in @@ -5294,9 +5455,8 @@ its rules. This error can also occur due to internal problems. @item @samp{token too large, exceeds YYLMAX}. your scanner uses @code{%array} and one of its rules matched a string longer than the @code{YYLMAX} -constant (8K bytes by default). You can increase the value by -#define'ing @code{YYLMAX} in the definitions section of your @code{flex} -input. +constant (8K bytes by default). You can increase the value with the +@code{%yylmax} option. @item @samp{scanner requires -8 flag to use the character 'x'}. Your scanner @@ -5307,7 +5467,7 @@ See the discussion of the @samp{-7} flag, @ref{Scanner Options}, for details. @item -@samp{flex scanner push-back overflow}. you used @code{unput()} to push +@samp{flex scanner push-back overflow}. you used @code{yyunput()} to push back so much text that the scanner's buffer could not hold both the pushed-back text and the current token in @code{yytext}. Ideally the scanner should dynamically resize the buffer in this case, but at @@ -5315,9 +5475,9 @@ present it does not. @item @samp{input buffer overflow, can't enlarge buffer because scanner uses -REJECT}. the scanner was working on matching an extremely large token +yyreject()}. the scanner was working on matching an extremely large token and needed to expand the input buffer. This doesn't work with scanners -that use @code{REJECT}. +that use @code{yyreject()}. @item @samp{fatal flex scanner internal error--end of buffer missed}. This can @@ -5365,19 +5525,22 @@ in @emph{fixed} trailing context being turned into the more expensive @end verbatim @end example -Use of @code{unput()} invalidates yytext and yyleng, unless the +Some caveats are specific to the C/C++ back end: Use of +@code{yyunput()} invalidates yytext and yyleng, unless the @code{%array} directive or the @samp{-l} option has been used. Pattern-matching of @code{NUL}s is substantially slower than matching other characters. Dynamic resizing of the input buffer is slow, as it -entails rescanning all the text matched so far by the current (generally -huge) token. Due to both buffering of input and read-ahead, you cannot -intermix calls to @file{<stdio.h>} routines, such as, @b{getchar()}, -with @code{flex} rules and expect it to work. Call @code{input()} -instead. The total table entries listed by the @samp{-v} flag excludes +entails rescanning all the text matched so far by the current +(generally huge) token. Due to both buffering of input and +read-ahead, you cannot intermix calls to @file{<stdio.h>} routines +(such as @b{getchar()}) with @code{flex} rules and expect it to work. +Call @code{yyinput()} instead. + +The total table entries listed by the @samp{-v} flag excludes the number of table entries needed to determine what rule has been matched. The number of entries is equal to the number of DFA states if -the scanner does not use @code{REJECT}, and somewhat greater than the -number of states if it does. @code{REJECT} cannot be used with the +the scanner does not use @code{yyreject()}, and somewhat greater than the +number of states if it does. @code{yyreject()} cannot be used with the @samp{-f} or @samp{-F} options. The @code{flex} internal algorithms need documentation. @@ -5425,7 +5588,7 @@ publish them here. * How can I have multiple input sources feed into the same scanner at the same time?:: * Can I build nested parsers that work with the same input file?:: * How can I match text only at the end of a file?:: -* How can I make REJECT cascade across start condition boundaries?:: +* How can I make yyreject() cascade across start condition boundaries?:: * Why cant I use fast or full tables with interactive mode?:: * How much faster is -F or -f than -C?:: * If I have a simple grammar cant I just parse it with flex?:: @@ -5447,7 +5610,7 @@ publish them here. * How can I build a two-pass scanner?:: * How do I match any string not matched in the preceding rules?:: * I am trying to port code from AT&T lex that uses yysptr and yysbuf.:: -* Is there a way to make flex treat NULL like a regular character?:: +* Is there a way to make flex treat NUL like a regular character?:: * Whenever flex can not match the input it says "flex scanner jammed".:: * Why doesn't flex have non-greedy operators like perl does?:: * Memory leak - 16386 bytes allocated by malloc.:: @@ -5460,7 +5623,7 @@ publish them here. * Can I fake multi-byte character support?:: * deleteme01:: * Can you discuss some flex internals?:: -* unput() messes up yy_at_bol:: +* yyunput() messes up yyatbol:: * The | operator is not doing what I want:: * Why can't flex understand this variable trailing context pattern?:: * The ^ operator isn't working:: @@ -5514,7 +5677,6 @@ publish them here. * unnamed-faq-99:: * unnamed-faq-100:: * unnamed-faq-101:: -* What is the difference between YYLEX_PARAM and YY_DECL?:: * Why do I get "conflicting types for yylex" error?:: * How do I access the values set in a Flex action from within a Bison action?:: @end menu @@ -5602,7 +5764,7 @@ rule to match more text, then put back the extra: @example @verbatim -data_.* yyless( 5 ); BEGIN BLOCKIDSTATE; +data_.* yyless( 5 ); yybegin(BLOCKIDSTATE); @end verbatim @end example @@ -5654,7 +5816,7 @@ your scanner is free of backtracking (verified using @code{flex}'s @samp{-b} fla AND you run your scanner interactively (@samp{-I} option; default unless using special table compression options), @item -AND you feed it one character at a time by redefining @code{YY_INPUT} to do so, +AND you feed it one character at a time by redefining @code{yyread()} to do so, @end itemize then every time it matches a token, it will have exhausted its input @@ -5670,8 +5832,8 @@ piecemeal; @code{select()} could inform you that the beginning of a token is available, you call @code{yylex()} to get it, but it winds up blocking waiting for the later characters in the token. -Here's another way: Move your input multiplexing inside of @code{YY_INPUT}. That -is, whenever @code{YY_INPUT} is called, it @code{select()}'s to see where input is +Here's another way: Move your input multiplexing inside of @code{yyread()}. That +is, whenever @code{yyread()} is called, it @code{select()}'s to see where input is available. If input is available for the scanner, it reads and returns the next byte. If input is available from another source, it calls whatever function is responsible for reading from that source. (If no input is @@ -5687,7 +5849,7 @@ that @code{flex} block-buffers the input it reads from @code{yyin}. This means ``outermost'' @code{yylex()}, when called, will automatically slurp up the first 8K of input available on yyin, and subsequent calls to other @code{yylex()}'s won't see that input. You might be tempted to work around this problem by -redefining @code{YY_INPUT} to only return a small amount of text, but it turns out +redefining @code{yyread()} to only return a small amount of text, but it turns out that that approach is quite difficult. Instead, the best solution is to combine all of your scanners into one large scanner, using a different exclusive start condition for each. @@ -5698,7 +5860,7 @@ exclusive start condition for each. There is no way to write a rule which is ``match this text, but only if it comes at the end of the file''. You can fake it, though, if you happen to have a character lying around that you don't allow in your input. -Then you redefine @code{YY_INPUT} to call your own routine which, if it sees +Then you redefine @code{yyread()} to call your own routine which, if it sees an @samp{EOF}, returns the magic character first (and remembers to return a real @code{EOF} next time it's called). Then you could write: @@ -5708,8 +5870,8 @@ real @code{EOF} next time it's called). Then you could write: @end verbatim @end example -@node How can I make REJECT cascade across start condition boundaries? -@unnumberedsec How can I make REJECT cascade across start condition boundaries? +@node How can I make yyreject() cascade across start condition boundaries? +@unnumberedsec How can I make yyreject() cascade across start condition boundaries? You can do this as follows. Suppose you have a start condition @samp{A}, and after exhausting all of the possible matches in @samp{<A>}, you want to try @@ -5719,19 +5881,19 @@ matches in @samp{<INITIAL>}. Then you could use the following: @verbatim %x A %% -<A>rule_that_is_long ...; REJECT; -<A>rule ...; REJECT; /* shorter rule */ +<A>rule_that_is_long ...; yyreject(); +<A>rule ...; yyreject(); /* shorter rule */ <A>etc. ... <A>.|\n { /* Shortest and last rule in <A>, so -* cascaded REJECTs will eventually +* cascaded yyreject()s will eventually * wind up matching this rule. We want * to now switch to the initial state * and try matching from there instead. */ yyless(0); /* put back matched text */ -BEGIN(INITIAL); +yybegin(INITIAL); } @end verbatim @end example @@ -5808,10 +5970,10 @@ Here is one way which allows you to track line information: @example @verbatim <INITIAL>{ -"/*" BEGIN(IN_COMMENT); +"/*" yybegin(IN_COMMENT); } <IN_COMMENT>{ -"*/" BEGIN(INITIAL); +"*/" yybegin(INITIAL); [^*\n]+ // eat comment in chunks "*" // eat the lone star \n yylineno++; @@ -5926,20 +6088,22 @@ Just call @code{yyrestart(newfile)}. Be sure to reset the start state if you wan @node How do I execute code only during initialization (only before the first scan)? @unnumberedsec How do I execute code only during initialization (only before the first scan)? -You can specify an initial action by defining the macro @code{YY_USER_INIT} (though -note that @code{yyout} may not be available at the time this macro is executed). Or you -can add to the beginning of your rules section: +You can specify an initial action with @code{%option user-init} +(though note that @code{yyout} may not be available at the time this +option is interpreted). Or you can add to the beginning of your rules +section: @example @verbatim %% - /* Must be indented! */ - static int did_init = 0; +%{ + static bool did_init = false; - if ( ! did_init ){ -do_my_init(); - did_init = 1; + if ( ! did_init ) { + do_my_init(); + did_init = true; } +%} @end verbatim @end example @@ -5951,9 +6115,8 @@ You can specify an action for the @code{<<EOF>>} rule. @node Where else can I find help? @unnumberedsec Where else can I find help? -You can find the flex homepage on the web at -@uref{http://flex.sourceforge.net/}. See that page for details about flex -mailing lists as well. +You can find the flex repository and issue tracker at +@uref{https://github.com/westes/flex}. @node Can I include comments in the "rules" section of the file? @unnumberedsec Can I include comments in the "rules" section of the file? @@ -5989,25 +6152,24 @@ However, you can do this using multiple input buffers. @example @verbatim %% -macro/[a-z]+ { +macro/[a-z]+ { /* Saw the macro "macro" followed by extra stuff. */ -main_buffer = YY_CURRENT_BUFFER; +main_buffer = yy_current_buffer(); expansion_buffer = yy_scan_string(expand(yytext)); yy_switch_to_buffer(expansion_buffer); } -<<EOF>> { -if ( expansion_buffer ) -{ -// We were doing an expansion, return to where -// we were. -yy_switch_to_buffer(main_buffer); -yy_delete_buffer(expansion_buffer); -expansion_buffer = 0; -} -else -yyterminate(); -} +<<EOF>> { + if ( expansion_buffer ) { + // We were doing an expansion, return to where + // we were. + yy_switch_to_buffer(main_buffer); + yy_delete_buffer(expansion_buffer); + expansion_buffer = 0; + } else { + yyterminate(); + } + } @end verbatim @end example @@ -6054,14 +6216,14 @@ one to match. @unnumberedsec I am trying to port code from AT&T lex that uses yysptr and yysbuf. Those are internal variables pointing into the AT&T scanner's input buffer. I -imagine they're being manipulated in user versions of the @code{input()} and @code{unput()} +imagine they're being manipulated in user versions of the @code{input()} and @code{yyunput()} functions. If so, what you need to do is analyze those functions to figure out what they're doing, and then replace @code{input()} with an appropriate definition of -@code{YY_INPUT}. You shouldn't need to (and must not) replace -@code{flex}'s @code{unput()} function. +@code{yyread()}. You shouldn't need to (and must not) replace +@code{flex}'s @code{yyunput()} function. -@node Is there a way to make flex treat NULL like a regular character? -@unnumberedsec Is there a way to make flex treat NULL like a regular character? +@node Is there a way to make flex treat NUL like a regular character? +@unnumberedsec Is there a way to make flex treat NUL like a regular character? Yes, @samp{\0} and @samp{\x00} should both do the trick. Perhaps you have an ancient version of @code{flex}. The latest release is version @value{VERSION}. @@ -6078,7 +6240,7 @@ e.g., %% [[a bunch of rules here]] -. printf("bad input character '%s' at line %d\n", yytext, yylineno); +. printf("bad input character '%s' at line %d\n", yytext, yylineno); @end verbatim @end example @@ -6100,7 +6262,7 @@ Better is to either introduce a separate parser, or to split the scanner into multiple scanners using (exclusive) start conditions. You might have -a separate start state once you've seen the @samp{BEGIN}. In that state, you +a separate start state once you've seen the @samp{yybegin()}. In that state, you might then have a regex that will match @samp{END} (to kick you out of the state), and perhaps @samp{(.|\n)} to get a single character within the chunk ... @@ -6128,7 +6290,7 @@ you might try this: @example @verbatim /* For non-reentrant C scanner only. */ -yy_delete_buffer(YY_CURRENT_BUFFER); +yy_delete_buffer(yy_current_buffer()); yy_init = 1; @end verbatim @end example @@ -6144,18 +6306,18 @@ situation. It is possible that some other globals may need resetting as well. > We thought that it would be possible to have this number through the > evaluation of the following expression: > -> seek_position = (no_buffers)*YY_READ_BUF_SIZE + yy_c_buf_p - YY_CURRENT_BUFFER->yy_ch_buf +> seek_position = (no_buffers)*YY_READ_BUF_SIZE + yy_c_buf_p - yy_current_buffer()->yy_ch_buf @end verbatim @end example While this is the right idea, it has two problems. The first is that it's possible that @code{flex} will request less than @code{YY_READ_BUF_SIZE} during -an invocation of @code{YY_INPUT} (or that your input source will return less +an invocation of @code{yyread()} (or that your input source will return less even though @code{YY_READ_BUF_SIZE} bytes were requested). The second problem is that when refilling its internal buffer, @code{flex} keeps some characters from the previous buffer (because usually it's in the middle of a match, and needs those characters to construct @code{yytext} for the match once it's -done). Because of this, @code{yy_c_buf_p - YY_CURRENT_BUFFER->yy_ch_buf} won't +done). Because of this, @code{yy_c_buf_p - yy_current_buffer()->yy_ch_buf} won't be exactly the number of characters already read from the current buffer. An alternative solution is to count the number of characters you've matched @@ -6164,12 +6326,12 @@ example, @example @verbatim -#define YY_USER_ACTION num_chars += yyleng; +%option pre-action="num_chars += yyleng;" @end verbatim @end example (You need to be careful to update your bookkeeping if you use @code{yymore(}), -@code{yyless()}, @code{unput()}, or @code{input()}.) +@code{yyless()}, @code{yyunput()}, or @code{yyinput()}.) @node How do I use my own I/O classes in a C++ scanner? @section How do I use my own I/O classes in a C++ scanner? @@ -6205,9 +6367,9 @@ In the example below, we want to skip over characters until we see the phrase /* INCORRECT SCANNER */ %x SKIP %% -<INITIAL>startskip BEGIN(SKIP); +<INITIAL>startskip yybegin(SKIP); ... -<SKIP>"endskip" BEGIN(INITIAL); +<SKIP>"endskip" yybegin(INITIAL); <SKIP>.* ; @end verbatim @end example @@ -6217,7 +6379,7 @@ The simplest (but slow) fix is: @example @verbatim -<SKIP>"endskip" BEGIN(INITIAL); +<SKIP>"endskip" yybegin(INITIAL); <SKIP>. ; @end verbatim @end example @@ -6227,9 +6389,9 @@ making it match "endskip" plus something else. So for example: @example @verbatim -<SKIP>"endskip" BEGIN(INITIAL); +<SKIP>"endskip" yybegin(INITIAL); <SKIP>[^e]+ ; -<SKIP>. ;/* so you eat up e's, too */ +<SKIP>. ;/* so you eat up e's, too */ @end verbatim @end example @@ -6305,11 +6467,11 @@ of the '|' operator automatically makes the pattern variable length, so in this case '[Ff]oot' is preferred to '(F|f)oot'. > 4. I changed a rule that looked like this: -> <snext8>{and}{bb}/{ROMAN}[^A-Za-z] { BEGIN... +> <snext8>{and}{bb}/{ROMAN}[^A-Za-z] { yybegin... > > to the next 2 rules: -> <snext8>{and}{bb}/{ROMAN}[A-Za-z] { ECHO;} -> <snext8>{and}{bb}/{ROMAN} { BEGIN... +> <snext8>{and}{bb}/{ROMAN}[A-Za-z] { yyecho();} +> <snext8>{and}{bb}/{ROMAN} { yybegin... > > Again, I understand the using [^...] will cause a great performance loss @@ -6322,7 +6484,7 @@ regardless of how complex they are. See the "Performance Considerations" section of the man page, and also the example in MISC/fastwc/. - Vern + Vern @end verbatim @end example @@ -6365,7 +6527,7 @@ Yes. I've appended instructions on how. Before you make this change, though, you should think about whether there are ways to fundamentally simplify your scanner - those are certainly preferable! - Vern + Vern To increase the 32K limit (on a machine with 32 bit integers), you increase the magnitude of the following in flexdef.h: @@ -6403,10 +6565,10 @@ so it won't happen any time soon. In the interim, the best I can suggest (unless you want to try fixing it yourself) is to write your rules in terms of pairs of bytes, using definitions in the first section: - X \xfe\xc2 - ... - %% - foo{X}bar found_foo_fe_c2_bar(); + X \xfe\xc2 + ... + %% + foo{X}bar found_foo_fe_c2_bar(); etc. Definitely a pain - sorry about that. @@ -6414,7 +6576,7 @@ By the way, the email address you used for me is ancient, indicating you have a very old version of flex. You can get the most recent, 2.5.4, from ftp.ee.lbl.gov. - Vern + Vern @end verbatim @end example @@ -6441,7 +6603,7 @@ Fixing flex to handle wider characters is on the long-term to-do list. But since flex is a strictly spare-time project these days, this probably won't happen for quite a while, unless someone else does it first. - Vern + Vern @end verbatim @end example @@ -6497,13 +6659,13 @@ way to compress the tables. See above. - Vern + Vern @end verbatim @end example @c TODO: Evaluate this faq. -@node unput() messes up yy_at_bol -@unnumberedsec unput() messes up yy_at_bol +@node yyunput() messes up yyatbol +@unnumberedsec yyunput() messes up yyatbol @example @verbatim To: Xinying Li <xli@npac.syr.edu> @@ -6512,11 +6674,11 @@ In-reply-to: Your message of Wed, 13 Nov 1996 17:28:38 PST. Date: Wed, 13 Nov 1996 19:51:54 PST From: Vern Paxson <vern> -> "unput()" them to input flow, question occurs. If I do this after I scan -> a carriage, the variable "YY_CURRENT_BUFFER->yy_at_bol" is changed. That +> "yyunput()" them to input flow, question occurs. If I do this after I scan +> a carriage, the variable "yy_current_buffer()->yyatbol" is changed. That > means the carriage flag has gone. -You can control this by calling yy_set_bol(). It's described in the manual. +You can control this by calling yysetbol(). It's described in the manual. > And if in pre-reading it goes to the end of file, is anything done > to control the end of curren buffer and end of file? @@ -6528,7 +6690,7 @@ No, there's no way to put back an end-of-file. The latest release is 2.5.4, by the way. It fixes some bugs in 2.5.2 and 2.5.3. You can get it from ftp.ee.lbl.gov. - Vern + Vern @end verbatim @end example @@ -6552,13 +6714,13 @@ any blanks around it. If you instead want the special '|' *action* (which from your scanner appears to be the case), which is a way of giving two different rules the same action: - foo | - bar matched_foo_or_bar(); + foo | + bar matched_foo_or_bar(); then '|' *must* be separated from the first rule by whitespace and *must* be followed by a new line. You *cannot* write it as: - foo | bar matched_foo_or_bar(); + foo | bar matched_foo_or_bar(); even though you might think you could because yacc supports this syntax. The reason for this unfortunately incompatibility is historical, but it's @@ -6569,7 +6731,7 @@ from your use of '|' later confusing flex. Let me know if you still have problems. - Vern + Vern @end verbatim @end example @@ -6595,7 +6757,7 @@ parentheses. Note that you must also be building the scanner with the -l option for AT&T lex compatibility. Without this option, flex automatically encloses the definitions in parentheses. - Vern + Vern @end verbatim @end example @@ -6619,20 +6781,20 @@ From: Vern Paxson <vern> I can't get this problem to reproduce - it works fine for me. Note though that if what you have is slightly different: - COMMENT ^\*.* - %% - {COMMENT} { } + COMMENT ^\*.* + %% + {COMMENT} { } then it won't work, because flex pushes back macro definitions enclosed in ()'s, so the rule becomes - (^\*.*) { } + (^\*.*) { } and now that the '^' operator is not at the immediate beginning of the line, it's interpreted as just a regular character. You can avoid this behavior by using the "-l" lex-compatibility flag, or "%option lex-compat". - Vern + Vern @end verbatim @end example @@ -6663,7 +6825,7 @@ it should be a problem. Lex's matching is clearly wrong, and I'd hope that usually the intent remains the same as expressed with the pattern, so flex's matching will be correct. - Vern + Vern @end verbatim @end example @@ -6704,7 +6866,7 @@ use flex. The latest release is 2.5.4, by the way, available from ftp.ee.lbl.gov. - Vern + Vern @end verbatim @end example @@ -6730,7 +6892,7 @@ even compile ...) You need instead: and that should work fine - there's no restriction on what can go inside of ()'s except for the trailing context operator, '/'. - Vern + Vern @end verbatim @end example @@ -6759,13 +6921,16 @@ have been on the to-do list for a while. As flex is a purely spare-time project for me, no guarantees when this will be added (in particular, it for sure won't be for many months to come). - Vern + Vern @end verbatim @end example @c TODO: Evaluate this faq. @node ERASEME55 @unnumberedsec ERASEME55 +(Note: The @code{YY_DECL} macro is deprecated. Use the @code{%yydecl} +option instead.) + @example @verbatim To: Colin Paul Adams <colin@colina.demon.co.uk> @@ -6789,7 +6954,7 @@ What you need to do is derive a subclass from yyFlexLexer that provides the above yylex() method, squirrels away lvalp and parm into member variables, and then invokes yyFlexLexer::yylex() to do the regular scanning. - Vern + Vern @end verbatim @end example @@ -6817,7 +6982,7 @@ even considers the possibility of matching "/*". Example: - '([^']*|{ESCAPE_SEQUENCE})' + '([^']*|{ESCAPE_SEQUENCE})' will match all the text between the ''s (inclusive). So the lexer considers this as a token beginning at the first ', and doesn't even @@ -6826,7 +6991,7 @@ attempt to match other tokens inside it. I thinnk this subtlety is not worth putting in the manual, as I suspect it would confuse more people than it would enlighten. - Vern + Vern @end verbatim @end example @@ -6848,9 +7013,9 @@ From: Vern Paxson <vern> What version of flex are you using? If I feed this to 2.5.4, it complains: - "bug.l", line 5: EOF encountered inside an action - "bug.l", line 5: unrecognized rule - "bug.l", line 5: fatal parse error + "bug.l", line 5: EOF encountered inside an action + "bug.l", line 5: unrecognized rule + "bug.l", line 5: fatal parse error Not the world's greatest error message, but it manages to flag the problem. @@ -6859,7 +7024,7 @@ an action on a separate line, since it's ambiguous with an indented rule.) You can get 2.5.4 from ftp.ee.lbl.gov. - Vern + Vern @end verbatim @end example @@ -6908,9 +7073,9 @@ From: Vern Paxson <vern> > I took a quick look into the flex-sources and altered some #defines in > flexdefs.h: > -> #define INITIAL_MNS 64000 -> #define MNS_INCREMENT 1024000 -> #define MAXIMUM_MNS 64000 +> #define INITIAL_MNS 64000 +> #define MNS_INCREMENT 1024000 +> #define MAXIMUM_MNS 64000 The things to fix are to add a couple of zeroes to: @@ -6921,8 +7086,8 @@ The things to fix are to add a couple of zeroes to: and, if you get complaints about too many rules, make the following change too: - #define YY_TRAILING_MASK 0x200000 - #define YY_TRAILING_HEAD_MASK 0x400000 + #define YY_TRAILING_MASK 0x200000 + #define YY_TRAILING_HEAD_MASK 0x400000 - Vern @end verbatim @@ -6939,7 +7104,7 @@ In-reply-to: Your message of Mon, 08 Dec 1997 15:54:15 PST. Date: Mon, 15 Dec 1997 13:21:35 PST From: Vern Paxson <vern> -> stdin_handle = YY_CURRENT_BUFFER; +> stdin_handle = yy_current_buffer(); > ifstream fin( "aFile" ); > yy_switch_to_buffer( yy_create_buffer( fin, YY_BUF_SIZE ) ); > @@ -6957,7 +7122,7 @@ You need to pass &fin, to turn it into an ifstream* instead of an ifstream. Then its type will be compatible with the expected istream*, because ifstream is derived from istream. - Vern + Vern @end verbatim @end example @@ -6987,7 +7152,7 @@ No, yyrestart() doesn't imply a rewind, even though its name might sound like it does. It tells the scanner to flush its internal buffers and start reading from the given file at its present location. - Vern + Vern @end verbatim @end example @@ -7014,7 +7179,7 @@ This is a known problem with Solaris C++ (and/or Solaris yacc). I believe the fix is to explicitly insert some 'extern "C"' statements for the corresponding routines/symbols. - Vern + Vern @end verbatim @end example @@ -7045,7 +7210,7 @@ an example in the man page of how this can lead to different matching. Flex's behavior complies with the POSIX standard (or at least with the last POSIX draft I saw). - Vern + Vern @end verbatim @end example @@ -7071,7 +7236,7 @@ yytext as "extern char yytext[]" (which is what lex uses) instead of "extern char *yytext" (which is what flex uses). If it's not that, then I'm afraid I don't know what the problem might be. - Vern + Vern @end verbatim @end example @@ -7089,7 +7254,7 @@ From: Vern Paxson <vern> > The problem is that when I do this (using %option c++) start > conditions seem to not apply. -The BEGIN macro modifies the yy_start variable. For C scanners, this +The yybegin() macro modifies the yy_start variable. For C scanners, this is a static with scope visible through the whole file. For C++ scanners, it's a member variable, so it only has visible scope within a member function. Your lexbegin() routine is not a member function when you @@ -7099,7 +7264,7 @@ a declaration of yy_start in order to get your scanner to compile when using C++; instead, the correct fix is to make lexbegin() a member function (by deriving from yyFlexLexer). - Vern + Vern @end verbatim @end example @@ -7122,7 +7287,7 @@ YY_USER_ACTION to count the number of characters matched. The latest flex release, by the way, is 2.5.4, available from ftp.ee.lbl.gov. - Vern + Vern @end verbatim @end example @@ -7146,7 +7311,7 @@ you are in the file, by counting the number of characters scanned for each token (available in yyleng). It may prove convenient to do this by redefining YY_USER_ACTION, as described in the manual. - Vern + Vern @end verbatim @end example @@ -7167,7 +7332,7 @@ From: Vern Paxson <vern> One way to do this is to have the parser call a stub routine that's included in the scanner's .l file, and consequently that has access ot -BEGIN. The only ugliness is that the parser can't pass in the state +yybegin(). The only ugliness is that the parser can't pass in the state it wants, because those aren't visible - but if you don't have many such states, then using a different set of names doesn't seem like to much of a burden. @@ -7176,7 +7341,7 @@ While generating a .h file like you suggests is certainly cleaner, flex development has come to a virtual stand-still :-(, so a workaround like the above is much more pragmatic than waiting for a new feature. - Vern + Vern @end verbatim @end example @@ -7200,19 +7365,19 @@ name that is also a variable name, or something like that; flex spits out #define's for each start condition name, mapping them to a number, so you can wind up with: - %x foo - %% - ... - %% - void bar() - { - int foo = 3; - } + %x foo + %% + ... + %% + void bar() + { + int foo = 3; + } and the penultimate will turn into "int 1 = 3" after C preprocessing, since flex will put "#define foo 1" in the generated scanner. - Vern + Vern @end verbatim @end example @@ -7239,7 +7404,7 @@ on that and translate it into an RE. Sorry for the less-than-happy news ... - Vern + Vern @end verbatim @end example @@ -7268,7 +7433,7 @@ that the text will often include NUL's. So that's the first thing to look for. - Vern + Vern @end verbatim @end example @@ -7289,7 +7454,7 @@ First, to go fast, you want to match as much text as possible, which your scanners don't in the case that what they're scanning is *not* a <RN> tag. So you want a rule like: - [^<]+ + [^<]+ Second, C++ scanners are particularly slow if they're interactive, which they are by default. Using -B speeds it up by a factor of 3-4 @@ -7299,15 +7464,15 @@ Third, C++ scanners that use the istream interface are slow, because of how poorly implemented istream's are. I built two versions of the following scanner: - %% - .*\n - .* - %% + %% + .*\n + .* + %% and the C version inhales a 2.5MB file on my workstation in 0.8 seconds. The C++ istream version, using -B, takes 3.8 seconds. - Vern + Vern @end verbatim @end example @@ -7329,7 +7494,7 @@ From: Vern Paxson <vern> There shouldn't be, all it ever does with the date is ask the system for it and then print it out. - Vern + Vern @end verbatim @end example @@ -7353,7 +7518,7 @@ you use fgets instead (which you should anyway, to protect against buffer overflows), then the final \n will be preserved in the string, and you can scan that in order to find the end of the string. - Vern + Vern @end verbatim @end example @@ -7386,7 +7551,7 @@ From: Vern Paxson <vern> Derive your own subclass and make mylineno a member variable of it. - Vern + Vern @end verbatim @end example @@ -7434,7 +7599,7 @@ you have to see what problems they missed. No, definitely not. It's meant to be for those situations where you absolutely must squeeze every last cycle out of your scanner. - Vern + Vern @end verbatim @end example @@ -7466,7 +7631,7 @@ correct state and reading at the right point in the input file. I don't - but it seems like a reasonable project to undertake (unlike numerous other flex tweaks :-). - Vern + Vern @end verbatim @end example @@ -7476,11 +7641,11 @@ numerous other flex tweaks :-). @example @verbatim Received: from 131.173.17.11 (131.173.17.11 [131.173.17.11]) - by ee.lbl.gov (8.9.1/8.9.1) with ESMTP id AAA03838 - for <vern@ee.lbl.gov>; Thu, 20 Aug 1998 00:47:57 -0700 (PDT) + by ee.lbl.gov (8.9.1/8.9.1) with ESMTP id AAA03838 + for <vern@ee.lbl.gov>; Thu, 20 Aug 1998 00:47:57 -0700 (PDT) Received: from hal.cl-ki.uni-osnabrueck.de (hal.cl-ki.Uni-Osnabrueck.DE [131.173.141.2]) - by deimos.rz.uni-osnabrueck.de (8.8.7/8.8.8) with ESMTP id JAA34694 - for <vern@ee.lbl.gov>; Thu, 20 Aug 1998 09:47:55 +0200 + by deimos.rz.uni-osnabrueck.de (8.8.7/8.8.8) with ESMTP id JAA34694 + for <vern@ee.lbl.gov>; Thu, 20 Aug 1998 09:47:55 +0200 Received: (from georg@localhost) by hal.cl-ki.uni-osnabrueck.de (8.6.12/8.6.12) id JAA34834 for vern@ee.lbl.gov; Thu, 20 Aug 1998 09:47:54 +0200 From: Georg Rehm <georg@hal.cl-ki.uni-osnabrueck.de> Message-Id: <199808200747.JAA34834@hal.cl-ki.uni-osnabrueck.de> @@ -7519,7 +7684,7 @@ appeared when flexing the code. Do you have an idea what's going on here? Greetings from Germany, - Georg + Georg -- Georg Rehm georg@cl-ki.uni-osnabrueck.de Institute for Semantic Information Processing, University of Osnabrueck, FRG @@ -7549,7 +7714,7 @@ The fix is to either rethink how come you're using such a big macro and perhaps there's another/better way to do it; or to rebuild flex's own scan.c with a larger value for - #define YY_BUF_SIZE 16384 + #define YY_BUF_SIZE 16384 - Vern @end verbatim @@ -7593,12 +7758,12 @@ I agree that this is counter-intuitive for yyless(), given its functional description (it's less so for unput(), depending on whether you're unput()'ing new text or scanned text). But I don't plan to change it any time soon, as it's a pain to do so. Consequently, -you do indeed need to use yy_set_bol() and YY_AT_BOL() to tweak +you do indeed need to use yysetbol() and yyatbol() to tweak your scanner into the behavior you desire. Sorry for the less-than-completely-satisfactory answer. - Vern + Vern @end verbatim @end example @@ -7625,7 +7790,7 @@ up with that token rather than reading a fresh one. If you're using yacc, then the special "error" production can sometimes be used to consume tokens in an attempt to get the parser into a consistent state. - Vern + Vern @end verbatim @end example @@ -7655,11 +7820,11 @@ Simple, no. One approach might be to return a magic character on EWOULDBLOCK and have a rule - .*<magic-character> // put back .*, eat magic character + .*<magic-character> // put back .*, eat magic character This is off the top of my head, not sure it'll work. - Vern + Vern @end verbatim @end example @@ -7688,15 +7853,16 @@ You can't indent your rules like this - that's where the errors are coming from. Flex copies indented text to the output file, it's how you do things like - int num_lines_seen = 0; + int num_lines_seen = 0; to declare local variables. - Vern + Vern @end verbatim @end example @c TODO: Evaluate this faq. +@c Note that the file is now named cpp-flex.skl @node unnamed-faq-87 @unnumberedsec unnamed-faq-87 @example @@ -7713,7 +7879,7 @@ From: Vern Paxson <vern> It's large to optimize performance when scanning large files. You can safely make it a lot lower if needed. - Vern + Vern @end verbatim @end example @@ -7741,7 +7907,7 @@ ams */ recompile everything, and it should all work. - Vern + Vern @end verbatim @end example @@ -7777,7 +7943,7 @@ Single-rule is nice but will always have the problem of either setting restrictions on comments (like not allowing multi-line comments) and/or running the risk of consuming the entire input stream, as noted above. - Vern + Vern @end verbatim @end example @@ -7787,8 +7953,8 @@ running the risk of consuming the entire input stream, as noted above. @example @verbatim Received: from mc-qout4.whowhere.com (mc-qout4.whowhere.com [209.185.123.18]) - by ee.lbl.gov (8.9.3/8.9.3) with SMTP id IAA05100 - for <vern@ee.lbl.gov>; Tue, 15 Jun 1999 08:56:06 -0700 (PDT) + by ee.lbl.gov (8.9.3/8.9.3) with SMTP id IAA05100 + for <vern@ee.lbl.gov>; Tue, 15 Jun 1999 08:56:06 -0700 (PDT) Received: from Unknown/Local ([?.?.?.?]) by my-deja.com; Tue Jun 15 08:55:43 1999 To: vern@ee.lbl.gov Date: Tue, 15 Jun 1999 08:55:43 -0700 @@ -7866,7 +8032,7 @@ From: Vern Paxson <vern> Derive your own subclass from yyFlexLexer. - Vern + Vern @end verbatim @end example @@ -7891,7 +8057,7 @@ You need to use the parser to build a parse tree (= abstract syntax trwee), and when that's all done you recursively evaluate the tree, binding variables to values at that time. - Vern + Vern @end verbatim @end example @@ -7928,7 +8094,7 @@ Also note that for speed, you'll want to add a ".*" rule at the end, otherwise rules that don't match any of the patterns will be matched very slowly, a character at a time. - Vern + Vern @end verbatim @end example @@ -7970,7 +8136,7 @@ initscan.c to scan.c in order to build flex. Try fetching a fresh distribution from ftp.ee.lbl.gov. (Or you can first try removing ".bootstrap" and doing a make again.) - Vern + Vern @end verbatim @end example @@ -7991,14 +8157,14 @@ From: Vern Paxson <vern> Try: - cp initscan.c scan.c - touch scan.c - make scan.o + cp initscan.c scan.c + touch scan.c + make scan.o If this last tries to first build scan.c from scan.l using ./flex, then your "make" is broken, in which case compile scan.c to scan.o by hand. - Vern + Vern @end verbatim @end example @@ -8019,7 +8185,7 @@ The parser relies on calling yylex(), but you're instead using the C++ scanning class, so you need to supply a yylex() "glue" function that calls an instance scanner of the scanner (e.g., "scanner->yylex()"). - Vern + Vern @end verbatim @end example @@ -8043,7 +8209,7 @@ assuming knowledge of the AT&T lex's internal variables. For flex, you can probably do the equivalent using a switch on YYSTATE. - Vern + Vern @end verbatim @end example @@ -8072,7 +8238,7 @@ Again, for flex, no. See the file "COPYING" in the flex distribution for the legalese. - Vern + Vern @end verbatim @end example @@ -8095,7 +8261,7 @@ From: Vern Paxson <vern> You can't do this - flex is *not* a parser like yacc (which does indeed allow recursion), it is a scanner that's confined to regular expressions. - Vern + Vern @end verbatim @end example @@ -8126,31 +8292,16 @@ If this is exactly your program: then the problem is that the last rule needs to be "{whitespace}" ! - Vern + Vern @end verbatim @end example -@node What is the difference between YYLEX_PARAM and YY_DECL? -@unnumberedsec What is the difference between YYLEX_PARAM and YY_DECL? - -YYLEX_PARAM is not a flex symbol. It is for Bison. It tells Bison to pass extra -params when it calls yylex() from the parser. - -YY_DECL is the Flex declaration of yylex. The default is similar to this: - -@example -@verbatim -#define int yy_lex () -@end verbatim -@end example - - @node Why do I get "conflicting types for yylex" error? @unnumberedsec Why do I get "conflicting types for yylex" error? This is a compiler error regarding a generated Bison parser, not a Flex scanner. It means you need a prototype of yylex() in the top of the Bison file. -Be sure the prototype matches YY_DECL. +Be sure the prototype matches what you declared with @code{%option yydecl}. @node How do I access the values set in a Flex action from within a Bison action? @unnumberedsec How do I access the values set in a Flex action from within a Bison action? @@ -8166,6 +8317,8 @@ See @ref{Top, , , bison, the GNU Bison Manual}. * Bison Bridge:: * M4 Dependency:: * Common Patterns:: +* Retargeting Flex:: +* Deprecated Interfaces:: @end menu @node Makefiles and Flex, Bison Bridge, Appendices, Appendices @@ -8412,7 +8565,7 @@ removing such sequences from your code. @code{m4} is only required at the time you run @code{flex}. The generated scanner is ordinary C or C++, and does @emph{not} require @code{m4}. -@node Common Patterns, ,M4 Dependency, Appendices +@node Common Patterns,Retargeting Flex,M4 Dependency, Appendices @section Common Patterns @cindex patterns, common @@ -8508,10 +8661,10 @@ more efficient when used with automatic line number processing. @xref{option-yyl @verbatim <INITIAL>{ - "/*" BEGIN(COMMENT); + "/*" yybegin(COMMENT); } <COMMENT>{ - "*/" BEGIN(0); + "*/" yybegin(0); [^*\n]+ ; "*"[^/] ; \n ; @@ -8560,32 +8713,369 @@ to appear in a URI, including spaces and control characters. See @end table +@node Retargeting Flex, Deprecated Interfaces, Common Patterns, Appendices +@section Retargeting Flex +@cindex retargeting +@cindex language independence + +This appendix describes how to add support for a new target language +to Flex. + +@menu +* Overview:: +* Getting Started:: +* Development Steps:: +* Translation Guidelines:: +@end menu + +@node Overview, Getting Started, , Retargeting Flex +@subsection Overview + +The Flex code has been factored to isolate knowledge of the specifics +of each target language from the logic for building the lexer +state/transition tables. Code in the target language is generated via +m4 expansion of macros in a skeleton. All knowledge of the target +language is isolated in that skeleton; with only one exception, +everything Flex contributes to the output stream is m4 macro definions +that are expanded by m4 after they are introduced. + +The only assumption that is absolutely baked into the macro +definitions Flex ships is that the bodies of initializers for arrays +of integers consist of decimal numeric literals separated by commas +(and optional whitespace). + +Otherwise, knowledge of each target langage's syntax lives in a +language-specific skeleton file that is digested into a data structure +inside Flex when Flex is compiled. The skeleton files are part of the +Flex source distribution; they are not required by the Flex +executable. + +A few pieces of language-specific information that cannot conveniently +be represented in a skeleton file are supplied by a per-language +method table in the C code. All the Flex code that accesses +language-specific information goes through a global pointer named +"backend" to a method table. One method is a function to +generate an appropriate suffix for output files. + +For example: The methods for the C and C++ back end live in a source +file named @file{cpp_backend.c} (so named because both languages use the C +preprocessor), and in a skeleton file named @file{cpp-flex.skl} which +is digested into a member of the method table when Flex is built. + +@node Getting Started, Development Steps, Overview, Retargeting Flex +@subsection Getting Started + +To get started on writing a new back end, browse +@code{src/skeletons,c} to get some idea of how skeleton files are +queried, then read a skeleton file. If no sketon exists of a language +that is closer to your target, the C99 back end is specifically +intended to be cloned and used as a launch pont for new back ends. + +This file you are looking at is processed through GNU m4 during the +pre-compilation stage of Flex. At this time only macros starting with +`m4preproc_' are processed, and quoting is normal (that is, the quote +opener is a back-tick (ASCII 96) and the quote closer is a tick or +single quote (ASCII 39) @footnote{See chapter 3.2 of the m4 mamual, +``Quoting input to m4'', for more information.}. The main purpose of +this expansion phase is to set Flex version symbols. + +At Flex compilation time, the preprocessed skeleton is translated into a comma-separated +list of doublequoted strings which is stuffed into a language- +specific method block compiled into the flex binary. + +At scanner generation time, , the skeleton is generated and filtered +(again) through m4. Macros beginning with `m4_' will be processed. +The quoting is ``[[`` and ``]]'' so we don't interfere with user code. + +A line beginning with ``%#'' is a comment. These comments are omitted +from the generated scanner. + +A line beginning with ``%%'' is a stop-point, where code is inserted by +Flex. Each stop-point is numbered here and also in the code +generator. Stop points will be inserted into the generated scanner as +a comment. This is to aid those who edit skeletons. + +You'll want to start by studying the @code{M4_PROPERTY_*} macros near the +top of the skeleton file. They declare properties of the back end like +its name and normal source-code suffix. These aren't used for code +generation; they're for Flex to read and key on. + +Following those are the @code{M4_HOOK_*} macros. Rather than emit +literal language syntax, Flex ships calls to these macros which are +expected to be expanded to to the correct language syntax within the +skeleton. The other things flex ships which appear in the output code +are mostly bodies for table initializers, with associated macros for +typenames and table dimensions. The names of all such macros have the +prefix ``M4_HOOK_''; you can study them in the Flex code by grepping +for that prefix. + +Flex ships another fairly large set of macros that are guard +conditions for conditional macroexpanion. The values of these symbols +don't directly appear in the output, but they control the shape of the +generated code. The names of such macros have the prefixes +``M4_MODE_'' or ``M4_YY_''; you can study them in the Flex code by +grepping for these prefixes. Many of these symbols correspond to Flex +command-line options. + +@node Development Steps, Translation Guidelines, Getting Started, Retargeting Flex +@subsection Development Steps + +To write support for a language, you'll want to do the following +steps: + +@enumerate +@item +Clone one of the existing skeletons. If the language you are +supporting is named @var{foo}, you should create a file named +@file{foo-flex.skl}. + +@item +Modify @code{skeleton.c} to ass the digested form of your skeleton +to the @code{backends} list. + +@item +Add the name of your skeleton file to EXTRA_DIST. + +@item +Add a production to @file{src/Makefile.am} parallel to the one that +produces @file{cpp-skel.h}. Your objective is to make a string list +initializer from your skeleton file that can be linked with flex and +is pointed at by the skel nember of your language back end. + +@item +The interesting part: mutate your new back end and skeleton so they +produce code in your desired target langage. + +@item +Add a test suite for your back end. You should be able to clone +one of the existing sets of test loads to get good coverage. Note +that is highly unlikely your back end will be accepted into the +flex distribution without a test suite. Fortunately, adapting +the existing tests should not be difficult. There is some guisdance +in the @file{tests/README} file of the source distrbution. +@end enumerate + +Syntactically C-like languages such as Go, Rust, and Java should be easy +targets. Almost anything generally descended from Algol shouldn't be +much more difficult; this certainly includes the whole +Pascal/Modula/Oberon family. + +The C99 back end can be used for production, but it is really intended +as a launch point to be cloned by people writing support for +additional languages. Accordingly, it omits support for some features +that can't be practically ported out of C in order to lower the +complexity of what needs to be translated to a new target language. +These features are: the Bison bridge, header generation, and loadable +tables. + +@node Translation Guidelines, , Development Steps, Retargeting Flex +@subsection Translation Guidelines + +@itemize +@item +Don't bother supporting non-reentrant parser generation. +The interface of original Lex with all those globals hanging out +needs to be supported in C for backwards compatibility, but +there is no need to carry it forward. +@end itemize + +@itemize +@item +The ``one exception'' to target-syntax independence hinted at earlier +is some C code spliced into the skeleton when table serialization is +enabled. This option is thus available only with the C back end; you +need not bother supporting it in yours. + +@item +If your target language has an object system, you probably want your +back end to generate a class named by default FlexLexer (as the +C++ back end does) with all of the controls and query functions as +methods. As in C++, @code{%option yyclass} should modify the +class name. If your target language has a module system, the +-P option (which in C/C++ sets a common prefix on exposed entry +points) can be pressed into service to set the module name. +@end itemize + +The following assumptions in the code might trip you up and +require fixes outside a back end. + +@enumerate +@item +As previously noted, the item separator in data initializers is a +comma. Flex does not assume that a trailing comma after the last +initializer element is legal, though it is legal in C/C++. + +@item +Either case arms can be stacked as in C; that is, there is +an implicit fallthrough if the case arm has no code. Or, +there is an explicit fallthrough keyword that enables this, +as in Go. +@end enumerate + +By putting a @code{yyterminate()} call in the expansion of +@code{M4_HOOK_EOF_STATE_CASE_FALLTHROUGH} and defining an empty +@code{M4_HOOK_EOF_STATE_CASE_TERMINATE} macro, you could handle +languages like Pascal. + +@node Deprecated Interfaces, , Retargeting Flex,Appendices +@section Deprecated interfaces, , Retargeting Flex +@cindex interfaces, deprecated + +Long-time users of Flex and its predecessor, Lex, may notice that the +examples in this manual look a little different than they used to. +They have changed because of the support for targeting languages +other than C/C++. + +All the old interfaces are still in place for legacy C code to use. +But some are now deprecated and should not be used in new code. Doing +so will hinder forward portability someday. + +These changes are necessary because the documented Flex interface can +no longer rely on controlling scanner generation by defining +preprocessor macros. Most languages other than C/C++ don't have text +macros, and none of those that do emulate C #define closely enough to +preserve the behavior dependent on it. Thus interface calls in +multi-language Flex need to be functions, at least syntactically +(though many are still implemented as macros for C/C++). + +A list of deprecated interfaces and their replacements follows. +Again, all are still available in the default C/C++ back end, but not +in any other. + +@itemize +@item +BEGIN: Replaced by yybegin() + +@item +ECHO: Replaced by yyecho() + +@item +REJECT: Replaced by yyreject() + +@item +#define YY_DECL: Replaced by the @code{yydecl} option. + +@item +#define YYLMAX: Replaced by the @code{yylmax} option. + +@item +#define YY_INPUT: Replaced by the @code{noyyread} option. + +@item +YYSTART: Replaced by yystart(). + +@item +YY_AT_BOL(): Replaced by yyatbol(). + +@item +yy_set_bol(): Replaced by yysetbol() + +@item +input(): Replaced by yyinput(). This function was already yyinput() +in the C++ back end. + +@item: +unput(): Replaced by yyunput(). + +@item +#define YY_EXTRA_TYPE: Replaced by the @code{extra-type} option. + +@item +#define YY_USER_INIT: Replaced by the @code{user-init} option. + +@item +#define YY_USER_ACTION replaced by @code{pre-action} option. + +@item +#define YY_BREAK replaced by @code{post-action} option. + +@item +YYSTATE: is accepted as an alias for @code{yystart()} +(since that is what's used by AT&T @code{lex}). + +@item +YY_FLUSH_BUFFER: replaced by yy_flush_current_buffer(). + +@item +YY_CURRENT_BUFFER: replaced by yy_current_buffer(). + +@item +YY_BUFFER_STATE: replaced by yybuffer. + +@item YY_FATAL_ERROR: replaced by yypanic(). +To supply your own handler, simply use @code{%option noyypanic} +so the default handler is not generated, then write your own handler +with the same prototype. + +@item +YY_NEW_FILE: In previous versions of @code{flex}, when assigning +@file{yyin} to a new input file, after doing the assignment you had to +call the special action @code{YY_NEW_FILE}. This is no longer +necessary. + +@item +FLEX_SCANNER: Not neaningful outside of the C back end, and not defined. + +@item +FLEX_DEBUG: Outside the default C back end, this is a constant of type +bool rather rthan a preprocessor symbol. + +@item +FLEX_BETA: Its behavior can't be replicated without the +C preprocessor. Test for YY_FLEX_SUBMINOR_VERSION instead. + +@item +YY_BUF_SIZE: replaced by @code{%option bufsize}. + +@item: +yyterminate(): Backends that don't use the C preprocessor cannot +support redefinining this hook with #define. Instead, set it with +the @code{yyterminate} option. + +@end itemize + +Where entry point names have been downcased and had underscores +removed, it's because some languages in the expected target set for +Flex - notably Go and Python - have validator/linter tools that would +complain about uppercase or underscores or both. It would obviously be best +if the parts of the public API that have to change to conform to +local conventions are small and not commonly used. + +This is also why the functions @code{yy_current_buffer()}, +@code{yy_flush_current_buffer()}, and @code{yy_set_interactive()} +do not have rewrite rules (see @xref{option-rewrite}); simply +removing the underscores would have produced names that are +unncomfortably long and hard on the eyeballs. We leave +it to the API binding for each individual target language to +choose whether they should be left as is, camel-cased, or +otherwise adapted to local conventions. + @node Indices, , Appendices, Top @unnumbered Indices @menu * Concept Index:: -* Index of Functions and Macros:: +* Index of Functions:: * Index of Variables:: * Index of Data Types:: * Index of Hooks:: * Index of Scanner Options:: @end menu -@node Concept Index, Index of Functions and Macros, Indices, Indices +@node Concept Index, Index of Functions, Indices, Indices @unnumberedsec Concept Index @printindex cp -@node Index of Functions and Macros, Index of Variables, Concept Index, Indices -@unnumberedsec Index of Functions and Macros +@node Index of Functions, Index of Variables, Concept Index, Indices +@unnumberedsec Index of Functions This is an index of functions and preprocessor macros that look like functions. For macros that expand to variables or constants, see @ref{Index of Variables}. @printindex fn -@node Index of Variables, Index of Data Types, Index of Functions and Macros, Indices +@node Index of Variables, Index of Data Types, Index of Functions, Indices @unnumberedsec Index of Variables This is an index of variables, constants, and preprocessor macros @@ -8620,4 +9110,15 @@ to specific locations in the generated scanner, and may be used to insert arbitr @c endf @c nnoremap <F5> 1G/@node\s\+unnamed-faq-\d\+<cr>mfww"wy5ezt:call Faq2()<cr> +@c Remaining problem points for the multilangage interface +@c YY_NUM_RULES +@c YY_FLEX_MAJOR_VERSION +@c YY_FLEX_MINOR_VERSION +@c YY_FLEX_SUBMINOR_VERSION +@c YY_NULL +@c YY_END_OF_BUFFER_CHAR +@c YY_BUF_SIZE +@c YYLMAX + + @bye diff --git a/examples/manual/Makefile.am b/examples/manual/Makefile.am index 9ab3004..24affe1 100644 --- a/examples/manual/Makefile.am +++ b/examples/manual/Makefile.am @@ -30,6 +30,9 @@ EXTRA_DIST = \ eof_test01.txt \ eof_test02.txt \ eof_test03.txt \ + example_er.lex \ + example_r.lex \ + example_nr.lex \ expr.lex \ expr.y \ front.lex \ diff --git a/examples/manual/Makefile.examples b/examples/manual/Makefile.examples index f4d8297..472653e 100644 --- a/examples/manual/Makefile.examples +++ b/examples/manual/Makefile.examples @@ -18,10 +18,23 @@ ALLOCA = # DO NOT CHANGE ANYTHING FROM HERE ON !!!!!!!!! # ############################################################ +PATH := /usr/local/bin:${PATH} all: expr front myname eof wc replace user_act string1\ string2 yymore numbers dates cat +example_r: example_r.lex + $(LEX) example_r.lex + $(CC) lex.yy.c -o example_r + +example_nr: example_nr.lex + $(LEX) example_nr.lex + $(CC) lex.yy.c -o example_nr + +example_er: example_er.lex + $(LEX) example_er.lex + $(CC) lex.yy.c -o example_er + expr: expr.y expr.lex $(YACC) expr.y $(LEX) expr.lex diff --git a/examples/manual/README b/examples/manual/README index e11569a..5386ca8 100644 --- a/examples/manual/README +++ b/examples/manual/README @@ -10,3 +10,9 @@ To build the programs individually, type For example: make -f Makefile.examples expr + +If you ad an eample to this directory, don't frget these steps: + +* Add it to the EXTRA_DIST list in Makefile.am + +* Add a build recipe to Makefile/examples. diff --git a/examples/manual/dates.lex b/examples/manual/dates.lex index 9429e1d..f9c648d 100644 --- a/examples/manual/dates.lex +++ b/examples/manual/dates.lex @@ -54,13 +54,13 @@ day_ext (st|nd|rd|th)? /* the default is month-day-year */ <LONG>{day_of_the_week} strcpy(dow,yytext); -<LONG>{month} strcpy(month,yytext); BEGIN(DAY); +<LONG>{month} strcpy(month,yytext); yybegin(DAY); /* handle the form: day-month-year */ -<LONG>{nday}{day_ext} strcpy(day,yytext); BEGIN(DAY_FIRST); -<DAY_FIRST>{month} strcpy(month,yytext); BEGIN(LONG); -<DAY>{nday}{day_ext} strcpy(day,yytext); BEGIN(LONG); +<LONG>{nday}{day_ext} strcpy(day,yytext); yybegin(DAY_FIRST); +<DAY_FIRST>{month} strcpy(month,yytext); yybegin(LONG); +<DAY>{nday}{day_ext} strcpy(day,yytext); yybegin(LONG); <LONG>{nyear}{year_ext} { printf("Long:\n"); @@ -75,15 +75,15 @@ day_ext (st|nd|rd|th)? /* handle dates of the form: day-month-year */ -<SHORT>{nday} strcpy(day,yytext); BEGIN(YEAR_LAST); -<YEAR_LAST>{nmonth} strcpy(month,yytext);BEGIN(YLMONTH); -<YLMONTH>{nyear} strcpy(year,yytext); BEGIN(SHORT); +<SHORT>{nday} strcpy(day,yytext); yybegin(YEAR_LAST); +<YEAR_LAST>{nmonth} strcpy(month,yytext);yybegin(YLMONTH); +<YLMONTH>{nyear} strcpy(year,yytext); yybegin(SHORT); /* handle dates of the form: year-month-day */ -<SHORT>{nyear} strcpy(year,yytext); BEGIN(YEAR_FIRST); -<YEAR_FIRST>{nmonth} strcpy(month,yytext);BEGIN(YFMONTH); -<YFMONTH>{nday} strcpy(day,yytext); BEGIN(SHORT); +<SHORT>{nyear} strcpy(year,yytext); yybegin(YEAR_FIRST); +<YEAR_FIRST>{nmonth} strcpy(month,yytext);yybegin(YFMONTH); +<YFMONTH>{nday} strcpy(day,yytext); yybegin(SHORT); <SHORT>\n { @@ -96,8 +96,8 @@ day_ext (st|nd|rd|th)? strcpy(month,""); } -long\n BEGIN(LONG); -short\n BEGIN(SHORT); +long\n yybegin(LONG); +short\n yybegin(SHORT); {skip}* \n diff --git a/examples/manual/eof_rules.lex b/examples/manual/eof_rules.lex index b575f2c..14e3181 100644 --- a/examples/manual/eof_rules.lex +++ b/examples/manual/eof_rules.lex @@ -17,15 +17,15 @@ int include_count = -1; %% -^"#include"[ \t]*\" BEGIN(INCLUDE); -<INCLUDE>\" BEGIN(INITIAL); +^"#include"[ \t]*\" yybegin(INCLUDE); +<INCLUDE>\" yybegin(INITIAL); <INCLUDE>[^\"]+ { /* get the include file name */ if ( include_count >= MAX_NEST){ fprintf( stderr, "Too many include files" ); exit( 1 ); } - include_stack[++include_count] = YY_CURRENT_BUFFER; + include_stack[++include_count] = yy_current_buffer(); yyin = fopen( yytext, "r" ); if ( ! yyin ){ @@ -35,7 +35,7 @@ int include_count = -1; yy_switch_to_buffer(yy_create_buffer(yyin,YY_BUF_SIZE)); - BEGIN(INITIAL); + yybegin(INITIAL); } <INCLUDE><<EOF>> { @@ -48,7 +48,7 @@ int include_count = -1; } else { yy_delete_buffer(include_stack[include_count--] ); yy_switch_to_buffer(include_stack[include_count] ); - BEGIN(INCLUDE); + yybegin(INCLUDE); } } [a-z]+ ECHO; diff --git a/examples/manual/example_er.lex b/examples/manual/example_er.lex new file mode 100644 index 0000000..284031c --- /dev/null +++ b/examples/manual/example_er.lex @@ -0,0 +1,35 @@ +/* basic example, fully reentrant thread-safe version */ +%{ + struct stats { + int num_lines; + int num_chars; + }; +%} +%option reentrant noyywrap +%option extra-type="struct stats" +%% +\n { + struct stats ns = yyget_extra(yyscanner); + ++ns.num_lines; ++ns.num_chars; + yyset_extra(ns, yyscanner); + } +. { + struct stats ns = yyget_extra(yyscanner); + ++ns.num_chars; + yyset_extra(ns, yyscanner); + } + +%% + +int main() { + yyscan_t scanner; + struct stats ns; + + yylex_init ( &scanner ); + yylex ( scanner ); + + ns = yyget_extra(scanner); + printf( "# of lines = %d, # of chars = %d\n", + ns.num_lines, ns.num_chars); + yylex_destroy ( scanner ); +} diff --git a/examples/manual/example_nr.lex b/examples/manual/example_nr.lex new file mode 100644 index 0000000..8722b14 --- /dev/null +++ b/examples/manual/example_nr.lex @@ -0,0 +1,16 @@ +/* basic example - non-reentrant version */ +%{ + int num_lines = 0, num_chars = 0; +%} +%option noyywrap +%% +\n ++num_lines; ++num_chars; +. ++num_chars; + +%% + +int main() { + yylex(); + printf( "# of lines = %d, # of chars = %d\n", + num_lines, num_chars ); +} diff --git a/examples/manual/example_r.lex b/examples/manual/example_r.lex new file mode 100644 index 0000000..54c94d9 --- /dev/null +++ b/examples/manual/example_r.lex @@ -0,0 +1,21 @@ +/* basic example - flawed reentrant version with global */ +%{ + int num_lines = 0, num_chars = 0; +%} +%option reentrant noyywrap +%% +\n ++num_lines; ++num_chars; +. ++num_chars; + +%% + +int main() { + yyscan_t scanner; + + yylex_init ( &scanner ); + yylex ( scanner ); + yylex_destroy ( scanner ); + + printf( "# of lines = %d, # of chars = %d\n", + num_lines, num_chars ); +} diff --git a/examples/manual/front.lex b/examples/manual/front.lex index 449cb00..0737a37 100644 --- a/examples/manual/front.lex +++ b/examples/manual/front.lex @@ -3,9 +3,6 @@ #include <string.h> #include "y.tab.h" /* this comes from bison */ -#define TRUE 1 -#define FALSE 0 - #define copy_and_return(token_type) { strcpy(yylval.name,yytext); \ return(token_type); } diff --git a/examples/manual/j2t.lex b/examples/manual/j2t.lex index 08fbd21..eb09eed 100644 --- a/examples/manual/j2t.lex +++ b/examples/manual/j2t.lex @@ -158,7 +158,7 @@ void write_block_header(char *type) printf("\n\n@table @b\n"); } -"Examples:"[^\.]+ ECHO; +"Examples:"[^\.]+ yyecho(); "*"[^*\n]+"*" { /* @emph{}(emphasized) text */ yytext[yyleng-1] = '\0'; @@ -205,12 +205,12 @@ void write_block_header(char *type) yyless(loop+1); statep++; states[statep] = EXAMPLE2; - BEGIN(EXAMPLE2); + yybegin(EXAMPLE2); } <EXAMPLE,EXAMPLE2>^\n { printf("@end example\n\n"); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } /* @@ -231,7 +231,7 @@ void write_block_header(char *type) yyless(loop); statep++; states[statep] = ENUM; - BEGIN(ENUM); + yybegin(ENUM); } <ENUM>"@" printf("@@"); @@ -239,7 +239,7 @@ void write_block_header(char *type) printf(":\n\n@example\n"); statep++; states[statep] = EXAMPLE; - BEGIN(EXAMPLE); + yybegin(EXAMPLE); } @@ -250,7 +250,7 @@ void write_block_header(char *type) <ENUM>\n\n\n[ \t]+[^0-9] { printf("\n\n@end enumerate\n\n"); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } /* @@ -265,7 +265,7 @@ void write_block_header(char *type) yyless(2); statep++; states[statep] = LITEM2; - BEGIN(LITEM2); + yybegin(LITEM2); } <LITEM2>^":".+":" { (void)check_and_convert(&yytext[1]); @@ -275,9 +275,9 @@ void write_block_header(char *type) <LITEM2>\n\n\n+[^:\n] { printf("\n\n@end itemize\n\n"); - ECHO; + yyecho(); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } /* @@ -300,7 +300,7 @@ void write_block_header(char *type) yyless(loop); statep++; states[statep] = LITEM; - BEGIN(LITEM); + yybegin(LITEM); } <LITEM>^.+":" { (void)check_and_convert(yytext); @@ -318,7 +318,7 @@ void write_block_header(char *type) printf("@end itemize\n\n"); printf("%s",&buffer[loop+1]); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } /* @@ -338,27 +338,27 @@ void write_block_header(char *type) yyless((len-loop)+2); statep++; states[statep] = BITEM; - BEGIN(BITEM); + yybegin(BITEM); } <BITEM>^" "*"*" { printf("@item"); statep++; states[statep] = BITEM_ITEM; - BEGIN(BITEM_ITEM); + yybegin(BITEM_ITEM); } <BITEM>"@" printf("@@"); <BITEM>^\n { printf("@end itemize\n\n"); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } <BITEM_ITEM>[^\:]* { printf(" @b{%s}\n\n",check_and_convert(yytext)); } <BITEM_ITEM>":" { statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } /* @@ -369,13 +369,13 @@ void write_block_header(char *type) (void)check_and_convert(&yytext[1]); statep++; states[statep] = HEADING; - BEGIN(HEADING); + yybegin(HEADING); } <HEADING>:[^\n] { printf("@item @b{%s}\n",buffer); write_underline(strlen(buffer),6,'~'); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } <HEADING>:\n"*"* { if(need_closing == TRUE){ @@ -385,7 +385,7 @@ void write_block_header(char *type) printf("@chapter %s\n",buffer); write_underline(strlen(buffer),9,'*'); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } <HEADING>:\n"="* { if(need_closing == TRUE){ @@ -395,7 +395,7 @@ void write_block_header(char *type) printf("@section %s\n",buffer); write_underline(strlen(buffer),9,'='); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } <HEADING>"@" printf("@@"); <HEADING>:\n"-"* { @@ -406,7 +406,7 @@ void write_block_header(char *type) printf("@subsection %s\n",buffer); write_underline(strlen(buffer),12,'-'); statep--; - BEGIN(states[statep]); + yybegin(states[statep]); } /* @@ -417,10 +417,10 @@ void write_block_header(char *type) printf("@example\n"); statep++; states[statep] = EXAMPLE; - BEGIN(EXAMPLE); + yybegin(EXAMPLE); } <EXAMPLE>^" " -. ECHO; +. yyecho(); %% diff --git a/examples/manual/pas_include.lex b/examples/manual/pas_include.lex index 58cf590..b0a0a48 100644 --- a/examples/manual/pas_include.lex +++ b/examples/manual/pas_include.lex @@ -19,13 +19,13 @@ int include_count = -1; %% -"{" BEGIN(COMMENT); +"{" yybegin(COMMENT); -<COMMENT>"}" BEGIN(INITIAL); -<COMMENT>"$include"[ \t]*"(" BEGIN(INCLUDE); +<COMMENT>"}" yybegin(INITIAL); +<COMMENT>"$include"[ \t]*"(" yybegin(INCLUDE); <COMMENT>[ \t]* /* skip whitespace */ -<INCLUDE>")" BEGIN(COMMENT); +<INCLUDE>")" yybegin(COMMENT); <INCLUDE>[ \t]* /* skip whitespace */ <INCLUDE>[^ \t\n() ]+ { /* get the include file name */ if ( include_count >= MAX_NEST){ @@ -33,7 +33,7 @@ int include_count = -1; exit( 1 ); } - include_stack[++include_count] = YY_CURRENT_BUFFER; + include_stack[++include_count] = yy_current_buffer(); yyin = fopen( yytext, "r" ); if ( ! yyin ){ @@ -43,7 +43,7 @@ int include_count = -1; yy_switch_to_buffer(yy_create_buffer(yyin,YY_BUF_SIZE)); - BEGIN(INITIAL); + yybegin(INITIAL); } <INCLUDE><<EOF>> { @@ -61,11 +61,11 @@ int include_count = -1; } else { yy_delete_buffer(include_stack[include_count--] ); yy_switch_to_buffer(include_stack[include_count] ); - BEGIN(INCLUDE); + yybegin(INCLUDE); } } -[a-z]+ ECHO; -.|\n ECHO; +[a-z]+ yyecho(); +.|\n yyecho(); diff --git a/examples/manual/pascal.lex b/examples/manual/pascal.lex index d406bbe..df94313 100644 --- a/examples/manual/pascal.lex +++ b/examples/manual/pascal.lex @@ -31,17 +31,17 @@ bad_string \'([^'\n]|\'\')+ %% -"{" BEGIN(COMMENT1); +"{" yybegin(COMMENT1); <COMMENT1>[^}\n]+ <COMMENT1>\n ++line_number; <COMMENT1><<EOF>> yyerror("EOF in comment"); -<COMMENT1>"}" BEGIN(INITIAL); +<COMMENT1>"}" yybegin(INITIAL); -"(*" BEGIN(COMMENT2); +"(*" yybegin(COMMENT2); <COMMENT2>[^)*\n]+ <COMMENT2>\n ++line_number; <COMMENT2><<EOF>> yyerror("EOF in comment"); -<COMMENT2>"*)" BEGIN(INITIAL); +<COMMENT2>"*)" yybegin(INITIAL); <COMMENT2>[*)] /* note that FILE and BEGIN are already diff --git a/examples/manual/reject.lex b/examples/manual/reject.lex index a7b817f..9e0bbd8 100644 --- a/examples/manual/reject.lex +++ b/examples/manual/reject.lex @@ -1,12 +1,12 @@ /* - * reject.lex: An example of REJECT and unput() + * reject.lex: An example of yyreject() and yyunput() * misuse. */ %% UNIX { - unput('U'); unput('N'); unput('G'); unput('\0'); - REJECT; + yyunput('U'); yyunput('N'); yyunput('G'); yyunput('\0'); + yyreject(); } GNU printf("GNU is Not Unix!\n"); %% diff --git a/examples/manual/replace.lex b/examples/manual/replace.lex index c5c8d87..a7062bc 100644 --- a/examples/manual/replace.lex +++ b/examples/manual/replace.lex @@ -16,7 +16,7 @@ char upper_replace[1024]; "yy" printf("%s",lower_replace); "YY" printf("%s",upper_replace); -, ECHO; +, yyecho(); %% diff --git a/examples/manual/string1.lex b/examples/manual/string1.lex index b62ed88..3a3276d 100644 --- a/examples/manual/string1.lex +++ b/examples/manual/string1.lex @@ -1,5 +1,5 @@ /* - * string1.lex: Handling strings by using input() + * string1.lex: Handling strings by using yyinput() */ %{ @@ -27,13 +27,13 @@ void yyerror(char *message) buffer = malloc(ALLOC_SIZE); max_size = ALLOC_SIZE; - inch = input(); + inch = yyinput(); count = 0; while(inch != EOF && inch != '"' && inch != '\n'){ if(inch == '\\'){ - inch = input(); + inch = yyinput(); switch(inch){ - case '\n': inch = input(); break; + case '\n': inch = yyinput(); break; case 'b' : inch = '\b'; break; case 't' : inch = '\t'; break; case 'n' : inch = '\n'; break; @@ -41,10 +41,10 @@ void yyerror(char *message) case 'f' : inch = '\f'; break; case 'r' : inch = '\r'; break; case 'X' : - case 'x' : inch = input(); + case 'x' : inch = yyinput(); if(isxdigit(inch)){ temp = hextoint(toupper(inch)); - inch = input(); + inch = yyinput(); if(isxdigit(inch)){ temp = (temp << 4) + hextoint(toupper(inch)); } else { @@ -59,14 +59,14 @@ void yyerror(char *message) default: if(isodigit(inch)){ temp = inch - '0'; - inch = input(); + inch = yyinput(); if(isodigit(inch)){ temp = (temp << 3) + (inch - '0'); } else { unput(inch); goto done; } - inch = input(); + inch = yyinput(); if(isodigit(inch)){ temp = (temp << 3) + (inch - '0'); } else { @@ -82,7 +82,7 @@ void yyerror(char *message) buffer = realloc(buffer,max_size + ALLOC_SIZE); max_size += ALLOC_SIZE; } - inch = input(); + inch = yyinput(); } if(inch == EOF || inch == '\n'){ yyerror("Unterminated string."); diff --git a/examples/manual/string2.lex b/examples/manual/string2.lex index 2c9d35f..f6d0c17 100644 --- a/examples/manual/string2.lex +++ b/examples/manual/string2.lex @@ -30,17 +30,17 @@ oct [0-7]{1,3} \" { buffer = malloc(1); buffer_size = 1; strcpy(buffer,""); - BEGIN(STRING); + yybegin(STRING); } <STRING>\n { yyerror("Unterminated string"); free(buffer); - BEGIN(INITIAL); + yybegin(INITIAL); } <STRING><<EOF>> { yyerror("EOF in string"); free(buffer); - BEGIN(INITIAL); + yybegin(INITIAL); } <STRING>[^\\\n"] { buffer = realloc(buffer,buffer_size+yyleng+1); @@ -87,7 +87,7 @@ oct [0-7]{1,3} <STRING>\" { printf("string = \"%s\"",buffer); free(buffer); - BEGIN(INITIAL); + yybegin(INITIAL); } %% diff --git a/examples/manual/unput.lex b/examples/manual/unput.lex index 161471a..21b42dc 100644 --- a/examples/manual/unput.lex +++ b/examples/manual/unput.lex @@ -1,6 +1,6 @@ /* * unput.l : An example of what *not* - * to do with unput(). + * to do with yyunput(). */ @@ -24,7 +24,7 @@ void putback_yytext(void) strcpy(buffer,yytext); printf("Got: %s\n",yytext); for(i=0; i<l; i++){ - unput(buffer[i]); + yyunput(buffer[i]); } } diff --git a/examples/manual/user_act.lex b/examples/manual/user_act.lex index 156d5f9..2e1b72d 100644 --- a/examples/manual/user_act.lex +++ b/examples/manual/user_act.lex @@ -4,14 +4,13 @@ void user_action(void); -#define YY_USER_ACTION user_action(); - %} +%option pre-action = "user_action();" %% -.* ECHO; -\n ECHO; +.* yyecho(); +\n yyecho(); %% diff --git a/examples/manual/userinit.lex b/examples/manual/userinit.lex index 9a56471..26fa9e1 100644 --- a/examples/manual/userinit.lex +++ b/examples/manual/userinit.lex @@ -1,6 +1,4 @@ %{ -#define YY_USER_INIT open_input_file() - extern FILE *yyin; void open_input_file(void) @@ -27,4 +25,5 @@ void open_input_file(void) } %} +%option user-init = "open_input_file();" %% diff --git a/examples/manual/yymore.lex b/examples/manual/yymore.lex index 27d7012..e477312 100644 --- a/examples/manual/yymore.lex +++ b/examples/manual/yymore.lex @@ -16,14 +16,14 @@ void yyerror(char *message) %x STRING %% -\" BEGIN(STRING); +\" yybegin(STRING); <STRING>[^\\\n"]* yymore(); -<STRING><<EOF>> yyerror("EOF in string."); BEGIN(INITIAL); -<STRING>\n yyerror("Unterminated string."); BEGIN(INITIAL); +<STRING><<EOF>> yyerror("EOF in string."); yybegin(INITIAL); +<STRING>\n yyerror("Unterminated string."); yybegin(INITIAL); <STRING>\\\n yymore(); <STRING>\" { yytext[yyleng-1] = '\0'; - printf("string = \"%s\"",yytext); BEGIN(INITIAL); + printf("string = \"%s\"",yytext); yybegin(INITIAL); } %% diff --git a/examples/manual/yymore2.lex b/examples/manual/yymore2.lex index f49ea23..dbe4915 100644 --- a/examples/manual/yymore2.lex +++ b/examples/manual/yymore2.lex @@ -16,11 +16,11 @@ void yyerror(char *message) %x STRING %% -\" BEGIN(STRING); +\" yybegin(STRING); <STRING>[^\\\n"]* yymore(); -<STRING><<EOF>> yyerror("EOF in string."); BEGIN(INITIAL); -<STRING>\n yyerror("Unterminated string."); BEGIN(INITIAL); +<STRING><<EOF>> yyerror("EOF in string."); yybegin(INITIAL); +<STRING>\n yyerror("Unterminated string."); yybegin(INITIAL); <STRING>\\\n { bcopy(yytext,yytext+2,yyleng-2); yytext += 2; yyleng -= 2; @@ -28,6 +28,6 @@ void yyerror(char *message) } <STRING>\" { yyleng -= 1; yytext[yyleng] = '\0'; - printf("string = \"%s\"",yytext); BEGIN(INITIAL); + printf("string = \"%s\"",yytext); yybegin(INITIAL); } %% diff --git a/src/.gitignore b/src/.gitignore index 7f35139..fadd6f3 100644 --- a/src/.gitignore +++ b/src/.gitignore @@ -1,6 +1,7 @@ *.la *.lo *.o +*-flex.h config.h config.h.in flex @@ -8,6 +9,5 @@ libfl.pc parse.c parse.h scan.c -skel.c stage1scan.[cl] stage1flex diff --git a/src/FlexLexer.h b/src/FlexLexer.h index c4dad2b..ccd9eeb 100644 --- a/src/FlexLexer.h +++ b/src/FlexLexer.h @@ -95,14 +95,14 @@ public: int lineno() const { return yylineno; } - int debug() const { return yy_flex_debug; } - void set_debug( int flag ) { yy_flex_debug = flag; } + int debug() const { return yyflexdebug; } + void set_debug( int flag ) { yyflexdebug = flag; } protected: char* yytext; int yyleng; int yylineno; // only maintained if you use %option yylineno - int yy_flex_debug; // only has effect with -d or "%option debug" + int yyflexdebug; // only has effect with -d or "%option debug" }; } @@ -139,6 +139,7 @@ public: void yypush_buffer_state( yy_buffer_state* new_buffer ); void yypop_buffer_state(); + virtual int yyread(char *buf, size_t); virtual int yylex(); virtual void switch_streams( std::istream& new_in, std::ostream& new_out ); virtual void switch_streams( std::istream* new_in = 0, std::ostream* new_out = 0 ); @@ -148,8 +149,8 @@ protected: virtual int LexerInput( char* buf, int max_size ); virtual void LexerOutput( const char* buf, int size ); virtual void LexerError( const char* msg ); - - void yyunput( int c, char* buf_ptr ); + + void yyunput_r( int c, char* buf_ptr ); int yyinput(); void yy_load_buffer_state(); diff --git a/src/Makefile.am b/src/Makefile.am index e033cc8..4a4fa15 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -70,7 +70,7 @@ COMMON_SOURCES = \ scanflags.c \ scanopt.c \ scanopt.h \ - skel.c \ + skeletons.c \ sym.c \ tables.c \ tables.h \ @@ -88,16 +88,28 @@ include_HEADERS = \ FlexLexer.h EXTRA_DIST = \ - flex.skl \ + *-flex.skl \ mkskel.sh \ gettext.h CLEANFILES = stage1scan.c stage1flex$(EXEEXT) -MAINTAINERCLEANFILES = skel.c +clean-local: + rm -f *-flex.h -skel.c: flex.skl mkskel.sh flexint_shared.h tables_shared.h tables_shared.c - $(SHELL) $(srcdir)/mkskel.sh $(srcdir) $(m4) $(VERSION) > $@.tmp +cpp-flex.h: cpp-flex.skl mkskel.sh flexint_shared.h tables_shared.h tables_shared.c + $(SHELL) $(srcdir)/mkskel.sh cpp $(srcdir) $(m4) $(VERSION) > $@.tmp + $(SHELL) $(srcdir)/chkskel.sh $@.tmp + mv -f $@.tmp $@ + +c99-flex.h: c99-flex.skl mkskel.sh + $(SHELL) $(srcdir)/mkskel.sh c99 $(srcdir) $(m4) $(VERSION) > $@.tmp + $(SHELL) $(srcdir)/chkskel.sh $@.tmp + mv -f $@.tmp $@ + +go-flex.h: go-flex.skl mkskel.sh + $(SHELL) $(srcdir)/mkskel.sh go $(srcdir) $(m4) $(VERSION) > $@.tmp + $(SHELL) $(srcdir)/chkskel.sh $@.tmp mv -f $@.tmp $@ if ENABLE_BOOTSTRAP @@ -113,17 +125,19 @@ dist-hook: scan.l flex$(EXEEXT) ./flex$(EXEEXT) -o scan.c $< && \ mv -f scan.c $(distdir) -# make needs to be told to make parse.h so that parallelized runs will +# make needs to be told to make inclusions so that parallelized runs will # not fail. -stage1flex-main.$(OBJEXT): parse.h -flex-main.$(OBJEXT): parse.h +SKELINCLUDES := $(subst skl,h,$(wildcard *.skl)) + +stage1flex-main.$(OBJEXT): parse.h $(SKELINCLUDES) +flex-main.$(OBJEXT): parse.h $(SKELINCLUDES) -stage1flex-yylex.$(OBJEXT): parse.h -flex-yylex.$(OBJEXT): parse.h +stage1flex-yylex.$(OBJEXT): parse.h $(SKELINCLUDES) +flex-yylex.$(OBJEXT): parse.h $(SKELINCLUDES) -stage1flex-scan.$(OBJEXT): parse.h -flex-stage1scan.$(OBJEXT): parse.h +stage1flex-scan.$(OBJEXT): parse.h $(SKELINCLUDES) +flex-stage1scan.$(OBJEXT): parse.h $(SKELINCLUDES) # Run GNU indent on sources. Don't run this unless all the sources compile cleanly. # @@ -47,26 +47,8 @@ /* global buffers. */ struct Buf userdef_buf; /**< for user #definitions triggered by cmd-line. */ -struct Buf defs_buf; /**< for #define's autogenerated. List of strings. */ -struct Buf yydmap_buf; /**< string buffer to hold yydmap elements */ -struct Buf m4defs_buf; /**< m4 definitions. List of strings. */ struct Buf top_buf; /**< contains %top code. String buffer. */ -struct Buf *buf_print_strings(struct Buf * buf, FILE* out) -{ - int i; - - if(!buf || !out) - return buf; - - for (i=0; i < buf->nelts; i++){ - const char * s = ((char**)buf->elts)[i]; - if(s) - fprintf(out, "%s", s); - } - return buf; -} - /* Append a "%s" formatted string to a string buffer */ struct Buf *buf_prints (struct Buf *buf, const char *fmt, const char *s) { @@ -83,52 +65,6 @@ struct Buf *buf_prints (struct Buf *buf, const char *fmt, const char *s) return buf; } -/** Append a line directive to the string buffer. - * @param buf A string buffer. - * @param filename file name - * @param lineno line number - * @return buf - */ -struct Buf *buf_linedir (struct Buf *buf, const char* filename, int lineno) -{ - char *dst, *t; - const char *src; - size_t tsz; - - if (gen_line_dirs) - return buf; - - tsz = strlen("#line \"\"\n") + /* constant parts */ - 2 * strlen (filename) + /* filename with possibly all backslashes escaped */ - (size_t) (1 + ceil (log10 (abs (lineno)))) + /* line number */ - 1; /* NUL */ - t = malloc(tsz); - if (!t) - flexfatal (_("Allocation of buffer for line directive failed")); - for (dst = t + snprintf (t, tsz, "#line %d \"", lineno), src = filename; *src; *dst++ = *src++) - if (*src == '\\') /* escape backslashes */ - *dst++ = '\\'; - *dst++ = '"'; - *dst++ = '\n'; - *dst = '\0'; - buf = buf_strappend (buf, t); - free(t); - return buf; -} - - -/** Append the contents of @a src to @a dest. - * @param @a dest the destination buffer - * @param @a dest the source buffer - * @return @a dest - */ -struct Buf *buf_concat(struct Buf* dest, const struct Buf* src) -{ - buf_append(dest, src->elts, src->nelts); - return dest; -} - - /* Appends n characters in str to buf. */ struct Buf *buf_strnappend (struct Buf *buf, const char *str, int n) { @@ -146,62 +82,6 @@ struct Buf *buf_strappend (struct Buf *buf, const char *str) return buf_strnappend (buf, str, (int) strlen (str)); } -/* appends "#define str def\n" */ -struct Buf *buf_strdefine (struct Buf *buf, const char *str, const char *def) -{ - buf_strappend (buf, "#define "); - buf_strappend (buf, " "); - buf_strappend (buf, str); - buf_strappend (buf, " "); - buf_strappend (buf, def); - buf_strappend (buf, "\n"); - return buf; -} - -/** Pushes "m4_define( [[def]], [[val]])m4_dnl" to end of buffer. - * @param buf A buffer as a list of strings. - * @param def The m4 symbol to define. - * @param val The definition; may be NULL. - * @return buf - */ -struct Buf *buf_m4_define (struct Buf *buf, const char* def, const char* val) -{ - const char * fmt = "m4_define( [[%s]], [[[[%s]]]])m4_dnl\n"; - char * str; - size_t strsz; - - val = val?val:""; - strsz = strlen(fmt) + strlen(def) + strlen(val) + 2; - str = malloc(strsz); - if (!str) - flexfatal (_("Allocation of buffer for m4 def failed")); - - snprintf(str, strsz, fmt, def, val); - buf_append(buf, &str, 1); - return buf; -} - -/** Pushes "m4_undefine([[def]])m4_dnl" to end of buffer. - * @param buf A buffer as a list of strings. - * @param def The m4 symbol to undefine. - * @return buf - */ -struct Buf *buf_m4_undefine (struct Buf *buf, const char* def) -{ - const char * fmt = "m4_undefine( [[%s]])m4_dnl\n"; - char * str; - size_t strsz; - - strsz = strlen(fmt) + strlen(def) + 2; - str = malloc(strsz); - if (!str) - flexfatal (_("Allocation of buffer for m4 undef failed")); - - snprintf(str, strsz, fmt, def); - buf_append(buf, &str, 1); - return buf; -} - /* create buf with 0 elements, each of size elem_size. */ void buf_init (struct Buf *buf, size_t elem_size) { diff --git a/src/c99-flex.skl b/src/c99-flex.skl new file mode 100644 index 0000000..08b143a --- /dev/null +++ b/src/c99-flex.skl @@ -0,0 +1,2488 @@ +%# -*-C-*- vi: set ft=c: +%# +%# This is the C99 front end. While it can be used as a production +%# skeleton for generating C tokenizers it is actually intended +%# as a porting base that can be cloned and translated into non-C +%# languages. Accordingly it cuts use of the C preprocessor to +%# the bare minimum needed to keep it working in C. +%# +%# Note that using m4 rather than C for macro definitions means they +%# will not automatically be recognized and expended in user actions nor +%# in user code sections. Anything you need to be visible there +%# needs to be declared as a function or have a rewrite rule +%# that dequotes the macro. +%# +%# The m4 macros complicate reading this code enough that being +%# prescriptive about whitespace and braces is more than usually +%# important. So please set your C style to K&R, aka 1TBS with +%# tabs when editing this file. Braces around single-statement +%# if/while/for/do/switch/break bodies are mandatory. +%# + +%# Macros for preproc stage. +m4preproc_changecom + +%# Macros for runtime processing stage. +m4_changecom +m4_changequote +m4_changequote([[, ]]) + +%# Properties not used in the skeleton - meant to be read by the Flex code +m4_define([[M4_PROPERTY_BACKEND_NAME]], [[C99]]) +m4_define([[M4_PROPERTY_SOURCE_SUFFIX]], [[c]]) +m4_define([[M4_PROPERTY_TRACE_LINE_REGEXP]], [[^#line ([0-9]+) "(.*)"]]) +m4_define([[M4_PROPERTY_TRACE_LINE_TEMPLATE]], [[#line %d "%s"]]) +m4_define([[M4_PROPERTY_CONTEXT_ARG]], [[yyscanner]]) +m4_define([[M4_PROPERTY_CONTEXT_FORMAT]], [[yyscanner->%s_r]]) +m4_define([[M4_PROPERTY_BUFFERSTACK_CONTEXT_FORMAT]], [[yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->bs_%s]]) + +%# Macro hooks used by Flex code generators start here +m4_define([[M4_HOOK_INT32]], [[int32_t]]) +m4_define([[M4_HOOK_INT16]], [[int16_t]]) +m4_define([[M4_HOOK_COMMENT_OPEN]], [[/*]]) +m4_define([[M4_HOOK_COMMENT_CLOSE]], [[*/]]) +%# If this is not defined, no trace lines will be generated. +m4_define([[M4_HOOK_TRACE_LINE_FORMAT]], [[#line $1 "$2" +]]) +m4_define([[M4_HOOK_TABLE_OPENER]], [[{]]) +m4_define([[M4_HOOK_TABLE_CONTINUE]], [[},]]) +m4_define([[M4_HOOK_TABLE_CLOSER]], [[};]]) +m4_define([[M4_HOOK_RELATIVIZE]], [[$1]]) +m4_define([[M4_HOOK_STATE_ENTRY_FORMAT]], [[ &yy_transition[$1], +]]) +m4_define([[M4_HOOK_NORMAL_STATE_CASE_ARM]], [[ case $1:]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_ARM]], [[ case YY_STATE_EOF($1):]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_FALLTHROUGH]], [[ /* FALLTHROUGH */]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_TERMINATE]], [[ yyterminate(); +]]) +m4_define([[M4_HOOK_TAKE_YYTEXT]], [[yy_do_before_action(yyscanner, yy_cp, yy_bp); /* set up yytext */]]) +m4_define([[M4_HOOK_RELEASE_YYTEXT]], [[*yy_cp = yyscanner->yy_hold_char; /* undo effects of setting up yytext */]]) +m4_define([[M4_HOOK_CHAR_REWIND]], [[yyscanner->yy_c_buf_p = yy_cp -= $1;]]) +m4_define([[M4_HOOK_LINE_REWIND]], [[yy_lineno_rewind_to(yy_cp, yy_cp - $1, yyscanner);]]) +m4_define([[M4_HOOK_CHAR_FORWARD]], [[yyscanner->yy_c_buf_p = yy_cp = yy_bp + $1;]]) +m4_define([[M4_HOOK_LINE_FORWARD]], [[yy_lineno_rewind_to(yy_cp, yy_bp + $1, yyscanner);]]) +m4_define([[M4_HOOK_CONST_DEFINE_BYTE]], [[const char $1 = $2; +]]) +m4_define([[M4_HOOK_CONST_DEFINE_STATE]], [[#define $1 $2 +]]) +m4_define([[M4_HOOK_CONST_DEFINE_UINT]], [[const uint $1 = $2; +]]) +m4_define([[M4_HOOK_CONST_DEFINE_BOOL]], [[const bool $1 = $2; +]]) +m4_define([[M4_HOOK_CONST_DEFINE_UNKNOWN]], [[m4_define($1, [[$2]])]]) +m4_define([[M4_HOOK_SET_YY_DECL]], [[m4_define([[YY_DECL]], [[$1]])]]) +m4_define([[M4_HOOK_SET_OFFSET_TYPE]], [[m4_define([[YY_OFFSET_TYPE]], [[$1]])]]) +m4_define([[M4_HOOK_SET_USERINIT]], [[m4_define([[YY_USER_INIT]], [[$1]])]]) +m4_define([[M4_HOOK_SET_RULE_SETUP]], [[m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ + rule_check_bol(yyscanner); +]]) m4_ifdef([[YY_USER_ACTION]], YY_USER_ACTION) +]]) +m4_define([[M4_HOOK_SET_PREACTION]], [[m4_define([[YY_USER_ACTION]], [[$1]])]]) +m4_define([[M4_HOOK_STATE_CASE_BREAK]], [[/*LINTED*/break;]]) +m4_define([[M4_HOOK_SET_POSTACTION]], [[m4_define([[M4_HOOK_STATE_CASE_BREAK]], [[$1]])]]) +m4_define([[M4_HOOK_FATAL_ERROR]], [[yypanic($1, yyscanner);]]) +m4_define([[M4_HOOK_ECHO]], [[yyecho(yyscanner);]]) + +m4_define([[yyterminate]], m4_ifdef([[M4_MODE_YYTERMINATE]], [[M4_MODE_YYTERMINATE /* $1 */]], [[return YY_NULL /* $1 */]])) + +%# Return all but the first "n" matched characters back to the input stream. +m4_define([[yyless]], [[ + do { + /* Undo effects of setting up yytext. */ + int yyless_macro_arg = ($1); + m4_ifdef([[M4_MODE_YYLINENO]], [[yy_less_lineno(yyless_macro_arg, yyscanner);]]) + *yy_cp = yyscanner->yy_hold_char; + YY_RESTORE_YY_MORE_OFFSET + yyscanner->yy_c_buf_p = yy_cp = yy_bp + yyless_macro_arg - YY_MORE_ADJ; + yy_do_before_action(yyscanner, yy_cp, yy_bp); /* set up yytext again */ + } while ( 0 ) +]]) + +%# Can't be a function given the context-arg treatment due to the goto. +m4_define([[M4_HOOK_REJECT]], [[{ + *yy_cp = yyscanner->yy_hold_char; /* undo effects of setting up yytext */ + yy_cp = yyscanner->yy_full_match; /* restore poss. backed-over text */ +m4_ifdef( [[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + yyscanner->yy_lp = yyscanner->yy_full_lp; /* restore orig. accepting pos. */ + yyscanner->yy_state_ptr = yyscanner->yy_full_state; /* restore orig. state */ + yy_current_state = *yyscanner->yy_state_ptr; /* restore curr. state */ +]]) + ++yyscanner->yy_lp; + goto find_rule; +} +]]) + +%% [0.0] Make hook macros available to Flex + +const int YY_FLEX_MAJOR_VERSION = FLEX_MAJOR_VERSION; +const int YY_FLEX_MINOR_VERSION = FLEX_MINOR_VERSION; +const int YY_FLEX_SUBMINOR_VERSION = FLEX_SUBMINOR_VERSION; + +%# STARTS prefix machinery +%# +%# The complexity here is necessary so that m4 preserves +%# the argument lists to each C function. +%# +%# If your target language has OO you can generate a class definition, +%# use option yyclass to pass in the class name, and drop prefix support. +%# In languages like Go with a coincept of package names the prefix option +%# should be used for setting that. + +m4_ifdef( [[M4_MODE_PREFIX]],, [[m4_define([[M4_MODE_PREFIX]], [[yy]])]]) + +m4preproc_define(`M4_GEN_PREFIX',`` +[[#ifdef yy$1 +#define ]]M4_MODE_PREFIX[[$1_ALREADY_DEFINED +#else +#define yy$1 ]]M4_MODE_PREFIX[[$1 +#endif]] +'') + +m4_ifelse(M4_MODE_PREFIX,yy,, + M4_GEN_PREFIX(`_create_buffer') + M4_GEN_PREFIX(`_delete_buffer') + M4_GEN_PREFIX(`_scan_buffer') + M4_GEN_PREFIX(`_scan_string') + M4_GEN_PREFIX(`_scan_bytes') + M4_GEN_PREFIX(`_init_buffer') + M4_GEN_PREFIX(`_flush_buffer') + M4_GEN_PREFIX(`_load_buffer_state') + M4_GEN_PREFIX(`_switch_to_buffer') + M4_GEN_PREFIX(`push_buffer_state') + M4_GEN_PREFIX(`pop_buffer_state') + M4_GEN_PREFIX(`ensure_buffer_stack') + M4_GEN_PREFIX(`lex') + M4_GEN_PREFIX(`restart') + M4_GEN_PREFIX(`lex_init') + M4_GEN_PREFIX(`lex_init_extra') + M4_GEN_PREFIX(`lex_destroy') + M4_GEN_PREFIX(`get_debug') + M4_GEN_PREFIX(`set_debug') + M4_GEN_PREFIX(`get_extra') + M4_GEN_PREFIX(`set_extra') + M4_GEN_PREFIX(`get_in') + M4_GEN_PREFIX(`set_in') + M4_GEN_PREFIX(`get_out') + M4_GEN_PREFIX(`set_out') + M4_GEN_PREFIX(`get_leng') + M4_GEN_PREFIX(`get_text') + M4_GEN_PREFIX(`get_lineno') + M4_GEN_PREFIX(`set_lineno') + M4_GEN_PREFIX(`get_column') + M4_GEN_PREFIX(`set_column') + M4_GEN_PREFIX(`wrap') +) + +m4_ifdef( [[M4_YY_BISON_LVAL]], +[[ + M4_GEN_PREFIX(`get_lval') + M4_GEN_PREFIX(`set_lval') +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ + M4_GEN_PREFIX(`get_lloc') + M4_GEN_PREFIX(`set_lloc') +]]) + + +m4_ifelse(M4_MODE_PREFIX,yy,, + M4_GEN_PREFIX(`alloc') + M4_GEN_PREFIX(`realloc') + M4_GEN_PREFIX(`free') +) + +%# ENDS prefix machinery + +/* STARTS platform-specific and compiler-specific definitions. */ + +m4_ifdef( [[M4_YY_ALWAYS_INTERACTIVE]], , +[[m4_ifdef( [[M4_YY_NEVER_INTERACTIVE]], , +[[/* Feature test macros. Flex uses functions that require a minimum set of + * macros defined. As defining some macros may hide function declarations that + * user code might use, be conservative and respect user's definitions as much + * as possible. In glibc, feature test macros may not be all set up until one + * of the libc header (that includes <features.h>) is included. This creates + * a circular dependency when we check the macros. <assert.h> is the safest + * header we can include and does not declare too many functions we don't need. + */ +#if !defined(__GNU_LIBRARY__) && defined(__STDC__) +#include <assert.h> +#endif +#if !(defined(_POSIX_C_SOURCE) || defined(_XOPEN_SOURCE) || \ + defined(_POSIX_SOURCE)) +# define _POSIX_C_SOURCE 1 /* Required for fileno() */ +# define _POSIX_SOURCE 1 +#endif]])]]) + +#include <stdio.h> +#include <string.h> +#include <errno.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <unistd.h> /* requred for isatty() */ + +#if defined(__GNUC__) && __GNUC__ >= 3 +#define yynoreturn __attribute__((__noreturn__)) +#else +#define yynoreturn +#endif + +/* + * Anywhere other than C this won't be a thing, + * because strings will have an associated length field. + */ +const int YY_END_OF_BUFFER_CHAR = 0; + +/* ENDS platform-specific and compiler-specific definitions. */ + +/* + * Amount of stuff to slurp up with each read. + * We assume the stdio library has already + * chosen a fit size foe whatever platform + * we're running on. + */ +const int YY_READ_BUF_SIZE = BUFSIZ; + +/* Size of default input buffer. We want to be able to fit two + * OS-level reads, but efficiency gains as the buffer size + * increases fall off after that + */ +const int YY_BUF_SIZE = m4_ifdef([[M4_MODE_YY_BUFSIZE]], [[M4_MODE_YY_BUFSIZE]], [[2 * YY_READ_BUF_SIZE]]); + +/* Returned upon end-of-file. */ +const int YY_NULL = 0; + +/* Promotes a possibly negative, possibly signed char to an + * integer in range [0..255] for use as an array index. + */ +m4_define([[YY_SC_TO_UI]], [[((YY_CHAR)($1))]]) + +/* In the ancestal C back end this was a void pointer, meant to be + * opaque pointer. That indirection has been removed here as it + * wouldn't easily port to other langages. + */ +typedef struct yyguts_t *yyscan_t; + +/* Action number for EOF rule of a given start state. */ +m4_define([[YY_STATE_EOF]], [[YY_END_OF_BUFFER + $1 + 1]]) + +/* The state buf must be large enough to hold one state per character in the main buffer. + */ +m4_define([[YY_STATE_BUF_SIZE]], [[((YY_BUF_SIZE + 2) * sizeof(yy_state_type))]]) + +const bool FLEX_DEBUG = m4_ifdef([[M4_MODE_DEBUG]], [[true]], [[false]]); + +typedef uint8_t YY_CHAR; + +m4_ifdef( [[M4_MODE_FULLSPD]], [[m4_dnl +typedef const struct yy_trans_info *yy_state_type; +]], [[ +typedef int yy_state_type; +]]) + +%% [1.0] DFA + +struct yy_trans_info { + /* We require that yy_verify and yy_nxt must be of the same size int. */ +m4_ifdef([[M4_MODE_REAL_FULLSPD]], [[ + YY_OFFSET_TYPE yy_verify; + + /* In cases where its sister yy_verify *is* a "yes, there is + * a transition", yy_nxt is the offset (in records) to the + * next state. In most cases where there is no transition, + * the value of yy_nxt is irrelevant. If yy_nxt is the -1th + * record of a state, though, then yy_nxt is the action number + * for that state. + */ + YY_OFFSET_TYPE yy_nxt; +]]) +m4_ifdef([[M4_MODE_NO_REAL_FULLSPD]], [[ + /* We generate a bogus 'struct yy_trans_info' data type + * so we can guarantee that it is always declared in the skel. + * This is so we can compile "sizeof(struct yy_trans_info)" + * in any scanner. + */ + int32_t yy_verify; + int32_t yy_nxt; +]]) +}; + +%% [2.0] payload macros for the DFA data tables are inserted here + +m4_ifdef( [[M4_HOOK_NXT_ROWS]],[[m4_dnl +static const M4_HOOK_NXT_TYPE yy_nxt[][M4_HOOK_NXT_ROWS] = +M4_HOOK_NXT_BODY +]]) + +m4_ifdef( [[M4_MODE_YYLINENO]],[[m4_dnl +/* Table of booleans, true if rule could match eol. */ +static const M4_HOOK_EOLTABLE_TYPE yy_rule_can_match_eol[M4_HOOK_EOLTABLE_SIZE] = { 0, +M4_HOOK_EOLTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_NEED_ACCEPT]],[[m4_dnl +static const M4_HOOK_ACCEPT_TYPE yy_accept[M4_HOOK_ACCEPT_SIZE] = { 0, +M4_HOOK_ACCEPT_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_MODE_USEECS]],[[m4_dnl +/* Character equivalence-class mapping */ +static const YY_CHAR yy_ec[M4_HOOK_ECSTABLE_SIZE] = { 0, +M4_HOOK_ECSTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_MODE_USEMECS]],[[m4_dnl +/* Character meta-equivalence-class mappings */ +static const YY_CHAR yy_meta[M4_HOOK_MECSTABLE_SIZE] = { 0, +M4_HOOK_MECSTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_TRANSTABLE_SIZE]],[[m4_dnl +/* The transition table */ +static const struct yy_trans_info yy_transition[M4_HOOK_TRANSTABLE_SIZE] = { +M4_HOOK_TRANSTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_STARTTABLE_SIZE]],[[m4_dnl +/* Table of pointers to start states. */ +static const struct yy_trans_info *yy_start_state_list[M4_HOOK_STARTTABLE_SIZE] = { +M4_HOOK_STARTTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_ACCLIST_TYPE]],[[m4_dnl +static const M4_HOOK_ACCLIST_TYPE yy_acclist[M4_HOOK_ACCLIST_SIZE] = { 0, +M4_HOOK_ACCLIST_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_BASE_TYPE]],[[m4_dnl +static const M4_HOOK_BASE_TYPE yy_base[M4_HOOK_BASE_SIZE] = { 0, +M4_HOOK_BASE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_DEF_TYPE]],[[m4_dnl +static const M4_HOOK_DEF_TYPE yy_def[M4_HOOK_DEF_SIZE] = { 0, +M4_HOOK_DEF_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_YYNXT_TYPE]],[[m4_dnl +static const M4_HOOK_YYNXT_TYPE yy_nxt[M4_HOOK_YYNXT_SIZE] = { 0, +M4_HOOK_YYNXT_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_CHK_TYPE]],[[m4_dnl +static const M4_HOOK_CHK_TYPE yy_chk[M4_HOOK_CHK_SIZE] = { 0, +M4_HOOK_CHK_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_HOOK_NULTRANS_SIZE]],[[m4_dnl +static const yy_state_type yy_NUL_trans[M4_HOOK_NULTRANS_SIZE] = { 0, +M4_HOOK_NULTRANS_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef( [[M4_MODE_DEBUG]],[[m4_dnl +/* Rule to line-number mapping */ +static const M4_HOOK_DEBUGTABLE_TYPE yy_rule_linenum[M4_HOOK_DEBUGTABLE_SIZE] = { 0, +M4_HOOK_DEBUGTABLE_BODY[[]]m4_dnl +}; +]]) + +%% [3.0] static declarations conditional on mode switches go here + +M4_YY_SC_DEFS + +typedef struct yy_buffer_state *yybuffer; + +%# These are not part of the exported interface and can safely be renamed +/* These must be #defines, not const variables, because they're used as case + * arm values. GCC will allow case arm expressions to include references to + * const variables, but this is not standard-conformant and other compilers + * may not. + */ +#define EOB_ACT_CONTINUE_SCAN 0 +#define EOB_ACT_END_OF_FILE 1 +#define EOB_ACT_LAST_MATCH 2 +#define YY_BUFFER_NEW 0 +#define YY_BUFFER_NORMAL 1 + /* When an EOF's been seen but there's still some text to process + * then we mark the buffer as YY_EOF_PENDING, to indicate that we + * shouldn't try reading from the input source any more. We might + * still have a bunch of tokens to match, though, because of + * possible backing-up. + * + * When we actually see the EOF, we change the status to "new" + * (via yyrestart()), so that the user can continue scanning by + * just pointing yyin at a new input file. + */ +#define YY_BUFFER_EOF_PENDING 2 + +struct yy_buffer_state { + FILE *yy_input_file; + char *yy_ch_buf; /* input buffer */ + char *yy_buf_pos; /* current position in input buffer */ + + /* Size of input buffer in bytes, not including room for EOB + * characters. + */ + int yy_buf_size; + + /* Number of characters read into yy_ch_buf, not including EOB + * characters. + */ + int yy_n_chars; + + /* Whether we "own" the buffer - i.e., we know we created it, + * and can realloc() it to grow it, and should free() it to + * delete it. + */ + bool yy_is_our_buffer; + + /* Whether this is an "interactive" input source; if so, and + * if we're using stdio for input, then we want to use char-by-char + * rather than a buffered read, to make sure we stop fetching input after + * each newline. + */ + bool yy_is_interactive; + + /* Whether we're considered to be at the beginning of a line. + * If so, '^' rules will be active on the next match, otherwise + * not. + */ + bool yyatbol_flag; + + int bs_yylineno; /**< The line count. */ + int bs_yycolumn; /**< The column count. */ + + /* Whether to try to fill the input buffer when we reach the + * end of it. + */ + bool yy_fill_buffer; + + int yy_buffer_status; +}; + +/* Watch out: yytext_ptr is a variable when yytext is an array, + * but it's a macro when yytext is a pointer. + */ +m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +m4_define([[yytext_ptr]], [[yytext_r]]) +]]) + +m4_ifdef( [[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[m4_dnl +%# These must match the values in the file flexdef.h +%# of the flex source code, otherwise havoc will ensue. +const int YY_TRAILING_MASK = 0x2000; +const int YY_TRAILING_HEAD_MASK = 0x4000; +]]) +/* Holds the entire state of the reentrant scanner. */ +struct yyguts_t { + /* User-defined. Not touched by flex. */ + m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[M4_MODE_EXTRA_TYPE yyextra_r;]]) + + /* The rest are the same as the globals declared in the non-reentrant scanner. */ + FILE *yyin_r, *yyout_r; + size_t yy_buffer_stack_top; /**< index of top of stack. */ + size_t yy_buffer_stack_max; /**< capacity of stack. */ + yybuffer * yy_buffer_stack; /**< Stack as an array. */ + char yy_hold_char; + int yy_n_chars; + int yyleng_r; + char *yy_c_buf_p; + bool yy_init; + int yy_start; + bool yy_did_buffer_switch_on_eof; + int yy_start_stack_ptr; + int yy_start_stack_depth; + int *yy_start_stack; + yy_state_type yy_last_accepting_state; + char* yy_last_accepting_cpos; + + int yylineno_r; + int yyflexdebug_r; + +m4_ifdef( [[M4_MODE_USES_REJECT]], [[ + yy_state_type *yy_state_buf; + yy_state_type *yy_state_ptr; + char *yy_full_match; + int yy_lp; + +m4_ifdef( [[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[m4_dnl + /* These are only needed for trailing context rules */ + int yy_looking_for_trail_begin; + int yy_full_lp; + int *yy_full_state; +]]) +]]) +m4_ifdef( [[M4_MODE_YYTEXT_IS_ARRAY]], [[ + char yytext_r[YYLMAX]; + char *yytext_ptr; + int yy_more_offset; + int yy_prev_more_offset; +]], [[ + char *yytext_r; + bool yy_more_flag; + int yy_more_len; +]]) +m4_ifdef( [[M4_YY_BISON_LVAL]], [[ + YYSTYPE * yylval_r; +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], [[ + YYLTYPE * yylloc_r; +]]) +}; /* end struct yyguts_t */ + +m4_ifdef( [[M4_YY_BISON_LVAL]], +[[ +/* This must go here because YYSTYPE and YYLTYPE are included + * from bison output in section 1.*/ +m4_define([[yylval]], [[yyscanner->yylval_r]]) +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ +m4_define([[yylloc]], [[yyscanner->yylloc_r]]) +]]) + +m4_ifdef( [[M4_YY_NO_FLEX_ALLOC]],, +[[ +void *yyalloc(size_t size, yyscan_t yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + return malloc(size); +} +]]) + +m4_ifdef( [[M4_YY_NO_FLEX_REALLOC]],, +[[ +void *yyrealloc(void * ptr, size_t size, yyscan_t yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + return realloc(ptr, size); +} +]]) + +m4_ifdef( [[M4_YY_NO_FLEX_FREE]],, +[[ +void yyfree(void * ptr, yyscan_t yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + /* The cast to (char *) in the following accommodates both + * implementations that use char* generic pointers, and those + * that use void* generic pointers. It works with the latter + * because both ANSI C and C++ allow castless assignment from + * any pointer type to void*, and deal with argument conversions + * as though doing an assignment. + */ + free( (char *) ptr ); +} +]]) + +m4_ifdef( [[M4_MODE_NO_YYWRAP]], [[ +int M4_MODE_PREFIX[[wrap]](yyscan_t yyscanner) { + return /*CONSTCOND*/1; +} +]]) + +/* Helpers for special functions, also part of public API */ + +/* Returns the top of the stack, or NULL. */ +yybuffer yy_current_buffer(yyscan_t yyscanner) { + return ( yyscanner->yy_buffer_stack \ + ? yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] \ + : NULL); +} + +static void yy_load_buffer_state (yyscan_t yyscanner) +{ + yyscanner->yy_n_chars = yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_n_chars; + yyscanner->yytext_ptr = yyscanner->yy_c_buf_p = yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_pos; + yyscanner->yyin_r = yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_input_file; + yyscanner->yy_hold_char = *yyscanner->yy_c_buf_p; +} + +/** Discard all buffered characters. On the next scan, yyread() will be called. + * @param b the buffer state to be flushed, usually @c yy_current_buffer(yyscanner). + * @param yyscanner The scanner object. + */ +void yy_flush_buffer(yybuffer b, yyscan_t yyscanner) +{ + if ( b == NULL ) { + return; + } + b->yy_n_chars = 0; + + /* We always need two end-of-buffer characters. The first causes + * a transition to the end-of-buffer state. The second causes + * a jam in that state. + */ + b->yy_ch_buf[0] = YY_END_OF_BUFFER_CHAR; + b->yy_ch_buf[1] = YY_END_OF_BUFFER_CHAR; + + b->yy_buf_pos = &b->yy_ch_buf[0]; + + b->yyatbol_flag = true; + b->yy_buffer_status = YY_BUFFER_NEW; + + if ( b == yy_current_buffer(yyscanner) ) { + yy_load_buffer_state( yyscanner ); + } +} + +void yy_flush_current_buffer(yyscan_t yyscanner) { + yy_flush_buffer( yy_current_buffer(yyscanner), yyscanner); +} + +const int YY_EXIT_FAILURE = 2; + +m4_ifdef( [[M4_YY_NO_YYPANIC]],, [[ +/* This function has a magic rewrite rule */ +static void yynoreturn yypanic(const char* msg, yyscan_t yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + fprintf( stderr, "%s\n", msg ); + exit( YY_EXIT_FAILURE ); +} +]]) + +/* Allocates the stack if it does not exist. + * Guarantees space for at least one push. + */ +static void yyensure_buffer_stack (yyscan_t yyscanner) +{ + size_t num_to_alloc; + + if (yyscanner->yy_buffer_stack == NULL) { + /* First allocation is just for 2 elements, since we don't know if this + * scanner will even need a stack. We use 2 instead of 1 to avoid an + * immediate realloc on the next call. + */ + num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */ + yyscanner->yy_buffer_stack = (struct yy_buffer_state**)yyalloc + (num_to_alloc * sizeof(struct yy_buffer_state*), yyscanner); + if ( yyscanner->yy_buffer_stack == NULL ) { + yypanic( "out of dynamic memory in yyensure_buffer_stack()", yyscanner ); + } + + memset(yyscanner->yy_buffer_stack, 0, num_to_alloc * sizeof(struct yy_buffer_state*)); + + yyscanner->yy_buffer_stack_max = num_to_alloc; + yyscanner->yy_buffer_stack_top = 0; + return; + } + + if (yyscanner->yy_buffer_stack_top >= (yyscanner->yy_buffer_stack_max) - 1) { + /* Increase the buffer to prepare for a possible push. */ + size_t grow_size = 8 /* arbitrary grow size */; + + num_to_alloc = yyscanner->yy_buffer_stack_max + grow_size; + yyscanner->yy_buffer_stack = (struct yy_buffer_state**)yyrealloc + (yyscanner->yy_buffer_stack, + num_to_alloc * sizeof(struct yy_buffer_state*), + yyscanner); + if (yyscanner->yy_buffer_stack == NULL) { + yypanic( "out of dynamic memory in yyensure_buffer_stack()", yyscanner ); + } + /* zero only the new slots.*/ + memset(yyscanner->yy_buffer_stack + yyscanner->yy_buffer_stack_max, 0, grow_size * sizeof(struct yy_buffer_state*)); + yyscanner->yy_buffer_stack_max = num_to_alloc; + } +} + +/* Initializes or reinitializes a buffer. + * This function is sometimes called more than once on the same buffer, + * such as during a yyrestart() or at EOF. + */ +static void yy_init_buffer(yybuffer b, FILE * file, yyscan_t yyscanner) +{ + int oerrno = errno; + + yy_flush_buffer( b, yyscanner); + + b->yy_input_file = file; + b->yy_fill_buffer = true; + + /* If b is the current buffer, then yy_init_buffer was _probably_ + * called from yyrestart() or through yy_get_next_buffer. + * In that case, we don't want to reset the lineno or column. + */ + if (b != yy_current_buffer(yyscanner)) { + b->bs_yylineno = 1; + b->bs_yycolumn = 0; + } + +m4_ifdef( [[M4_YY_ALWAYS_INTERACTIVE]], +[[ + b->yy_is_interactive = true; +]], +[[ + m4_ifdef( [[M4_YY_NEVER_INTERACTIVE]], + [[ + b->yy_is_interactive = false; + ]], + [[ + b->yy_is_interactive = (file != NULL) && (isatty( fileno(file) ) > 0); + ]]) +]]) + errno = oerrno; +} + +/** Allocate and initialize an input buffer state. + * @param file A readable stream. + * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. + * @param yyscanner The scanner object. + * @return the allocated buffer state. + */ +yybuffer yy_create_buffer(FILE * file, int size, yyscan_t yyscanner) +{ + yybuffer b; + + b = (yybuffer) yyalloc( sizeof( struct yy_buffer_state ), yyscanner ); + if ( b == NULL ) { + yypanic( "out of dynamic memory in yy_create_buffer()", yyscanner ); + } + b->yy_buf_size = size; + + /* yy_ch_buf has to be 2 characters longer than the size given because + * we need to put in 2 end-of-buffer characters. + */ + b->yy_ch_buf = (char *) yyalloc( (size_t) (b->yy_buf_size + 2), yyscanner ); + if ( b->yy_ch_buf == NULL ) { + yypanic( "out of dynamic memory in yy_create_buffer()", yyscanner ); + } + b->yy_is_our_buffer = true; + + yy_init_buffer( b, file, yyscanner); + + return b; +} + +/** Destroy the buffer. + * @param b a buffer created with yy_create_buffer() + * @param yyscanner The scanner object. + */ +void yy_delete_buffer(yybuffer b, yyscan_t yyscanner) +{ + + if ( b == NULL ) { + return; + } + if ( b == yy_current_buffer(yyscanner) ) { /* Not sure if we should pop here. */ + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = (yybuffer) 0; + } + if ( b->yy_is_our_buffer ) { + yyfree( (void *) b->yy_ch_buf, yyscanner ); + } + yyfree( (void *) b, yyscanner ); +} + + +/** Pushes the new state onto the stack. The new state becomes + * the current state. This function will allocate the stack + * if necessary. + * @param new_buffer The new state. + * @param yyscanner The scanner object. + */ +void yypush_buffer_state(yybuffer new_buffer, yyscan_t yyscanner) +{ + if (new_buffer == NULL) { + return; + } + yyensure_buffer_stack(yyscanner); + + /* This block is copied from yy_switch_to_buffer. */ + if ( yy_current_buffer(yyscanner) != NULL ) { + /* Flush out information for old buffer. */ + *yyscanner->yy_c_buf_p = yyscanner->yy_hold_char; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_pos = yyscanner->yy_c_buf_p; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_n_chars = yyscanner->yy_n_chars; + } + + /* Only push if top exists. Otherwise, replace top. */ + if (yy_current_buffer(yyscanner)) { + yyscanner->yy_buffer_stack_top++; + } + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = new_buffer; + + /* copied from yy_switch_to_buffer. */ + yy_load_buffer_state( yyscanner ); + yyscanner->yy_did_buffer_switch_on_eof = true; +} + + +/** Removes and deletes the top of the stack, if present. + * The next element becomes the new top. + * @param yyscanner The scanner object. + */ +void yypop_buffer_state (yyscan_t yyscanner) +{ + if (yy_current_buffer(yyscanner) == NULL) { + return; + } + yy_delete_buffer(yy_current_buffer(yyscanner), yyscanner); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = NULL; + if (yyscanner->yy_buffer_stack_top > 0) { + --yyscanner->yy_buffer_stack_top; + } + if (yy_current_buffer(yyscanner) != NULL) { + yy_load_buffer_state( yyscanner ); + yyscanner->yy_did_buffer_switch_on_eof = true; + } +} + + +/** Immediately switch to a different input stream. + * @param input_file A readable stream. + * @param yyscanner The scanner object. + * @note This function does not reset the start condition to @c INITIAL . + */ +void yyrestart(FILE * input_file, yyscan_t yyscanner) +{ + + if ( yy_current_buffer(yyscanner) == NULL ) { + yyensure_buffer_stack (yyscanner); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = + yy_create_buffer( yyscanner->yyin_r, YY_BUF_SIZE, yyscanner); + } + + yy_init_buffer( yy_current_buffer(yyscanner), input_file, yyscanner); + yy_load_buffer_state( yyscanner ); +} + +static void yybumpline( yyscan_t yyscanner) { + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->bs_yylineno++; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->bs_yycolumn=0; +} + +/* START special functions + * + * Flex's scanner knows these are special and inserts the yyscanner argument + * at the end of the argument list. TODO: support OO languages with a + * property that causes it to be prepended with a dot instead. + */ + +/* Enter a start condition. */ +void yybegin(int s, yyscan_t yyscanner) { + yyscanner->yy_start = 1 + 2 * (s); +} + +/* Translate the current start state into a value that can be later handed + * to yybegin() to return to the state. + */ +int yystart(yyscan_t yyscanner) { + return ((yyscanner->yy_start - 1) / 2); +} + +/* This used to be an fputs(), but since the string might contain NULs, + * we now use fwrite(). + */ +void yyecho(yyscan_t yyscanner) { + fwrite(yyscanner->yytext_r, (size_t) yyscanner->yyleng_r, 1, yyscanner->yyout_r); +} + +m4_ifdef( [[M4_YY_NO_YYUNPUT]],, [[ +void yyunput(char c, yyscan_t yyscanner) +{ + char *yy_cp; + + yy_cp = yyscanner->yy_c_buf_p; + + /* undo effects of setting up yytext */ + *yy_cp = yyscanner->yy_hold_char; + + if ( yy_cp < yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf + 2 ) { + /* need to shift things up to make room */ + /* +2 for EOB chars. */ + int number_to_move = yyscanner->yy_n_chars + 2; + char *dest = &yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[ + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_size + 2]; + char *source = + &yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[number_to_move]; + + while ( source > yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf ) { + *--dest = *--source; + } + yy_cp += (int) (dest - source); + yyscanner->yytext_ptr += (int) (dest - source); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_n_chars = + yyscanner->yy_n_chars = (int) yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_size; + + if ( yy_cp < yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf + 2 ) { + yypanic( "flex scanner push-back overflow", yyscanner); + } + } + + *--yy_cp = c; + +m4_ifdef( [[M4_MODE_YYLINENO]], +[[ + if ( c == '\n' ){ + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->bs_yylineno--; + } +]]) + + yyscanner->yy_hold_char = *yy_cp; + yyscanner->yy_c_buf_p = yy_cp; +} +]]) + +%# yymore has a magic rewute rule. It's declared here, rather than with the other +%# magic functions, so yy_get_next_buffer() won't need a forward declaration. +m4_ifdef([[M4_MODE_YYMORE_USED]], [[ +m4_ifdef( [[M4_MODE_YYTEXT_IS_ARRAY]], [[ +void yymore(yyscan_t yyscanner) {yyscanner->yy_more_offset = strlen(yyscanner->yytext_r);} +m4_define([[YY_MORE_ADJ]], [[0]]) +m4_define([[YY_RESTORE_YY_MORE_OFFSET]], [[{ +yyscanner->yy_more_offset = yyscanner->yy_prev_more_offset; +yyscanner->yyleng_r -= yyscanner->yy_more_offset; +} +]]) +]]) +m4_ifdef( [[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +void yymore(yyscan_t yyscanner) {yyscanner->yy_more_flag = true;} +m4_define([[YY_MORE_ADJ]], [[yyscanner->yy_more_len]]) +m4_define([[YY_RESTORE_YY_MORE_OFFSET]], [[]]) +]]) +]]) + +m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[ +m4_define([[YY_MORE_ADJ]], [[0]]) +m4_define([[YY_RESTORE_YY_MORE_OFFSET]], [[]]) +]]) + +m4_ifdef( [[M4_MODE_USER_YYREAD]],, [[ +/* Gets input and stuffs it into "buf". number of characters read, or YY_NULL, + * is returned in "result". + */ +static int yyread(char *buf, size_t max_size, yyscan_t yyscanner) { + int result; +m4_ifdef( [[M4_MODE_CPP_USE_READ]], [[ + errno=0; + while ( (result = (int) read( fileno(yyscanner->yyin_r), buf, (size_t) max_size )) < 0 ) { + if( errno != EINTR) { + yypanic( "input in flex scanner failed", yyscanner); + break; + } + errno=0; + clearerr(yyscanner->yyin_r); + } +]]) +m4_ifdef( [[M4_MODE_NO_CPP_USE_READ]], [[ + if ( yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_is_interactive ) { + int c = '*'; + int n; + for ( n = 0; n < max_size && + (c = getc( yyscanner->yyin_r )) != EOF && c != '\n'; ++n ) { + buf[n] = (char) c; + } + if ( c == '\n' ) { + buf[n++] = (char) c; + } + if ( c == EOF && ferror( yyscanner->yyin_r ) ) { + yypanic( "input in flex scanner failed", yyscanner); + } + result = n; + } else { + errno=0; + while ( (result = (int) fread(buf, 1, (size_t) max_size, yyscanner->yyin_r)) == 0 && ferror(yyscanner->yyin_r)) { + if( errno != EINTR) { + yypanic( "input in flex scanner failed", yyscanner); + break; + } + errno=0; + clearerr(yyscanner->yyin_r); + } + } +]]) + return result; +} +]]) + +/* yy_get_next_buffer - try to read in a new buffer + * + * Returns a code representing an action: + * EOB_ACT_LAST_MATCH - + * EOB_ACT_CONTINUE_SCAN - continue scanning from current position + * EOB_ACT_END_OF_FILE - end of file + */ +static int yy_get_next_buffer (yyscan_t yyscanner) +{ + char *dest = yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf; + char *source = yyscanner->yytext_ptr; + int number_to_move, i; + int ret_val; + + if ( yyscanner->yy_c_buf_p > &yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[yyscanner->yy_n_chars + 1] ) { + yypanic( "fatal flex scanner internal error--end of buffer missed", yyscanner); + } + if ( !yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_fill_buffer ) { + /* Don't try to fill the buffer, so this is an EOF. */ + if ( yyscanner->yy_c_buf_p - yyscanner->yytext_ptr - YY_MORE_ADJ == 1 ) { + /* We matched a single character, the EOB, so + * treat this as a final EOF. + */ + return EOB_ACT_END_OF_FILE; + } else { + /* We matched some text prior to the EOB, first + * process it. + */ + return EOB_ACT_LAST_MATCH; + } + } + + /* Try to read more data. */ + + /* First move last chars to start of buffer. */ + number_to_move = (int) (yyscanner->yy_c_buf_p - yyscanner->yytext_ptr - 1); + + for ( i = 0; i < number_to_move; ++i ) { + *(dest++) = *(source++); + } + if ( yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buffer_status == YY_BUFFER_EOF_PENDING ) { + /* don't do the read, it's not guaranteed to return an EOF, + * just force an EOF + */ + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_n_chars = yyscanner->yy_n_chars = 0; + } else { + int num_to_read = + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_size - number_to_move - 1; + + while ( num_to_read <= 0 ) { /* Not enough room in the buffer - grow it. */ +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + yypanic( + "input buffer overflow, can't enlarge buffer because scanner uses reject", yyscanner ); +]], +[[ + /* just a shorter name for the current buffer */ + yybuffer b = yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]; + + int yy_c_buf_p_offset = + (int) (yyscanner->yy_c_buf_p - b->yy_ch_buf); + + if ( b->yy_is_our_buffer ) { + int new_size = b->yy_buf_size * 2; + + if ( new_size <= 0 ) { + b->yy_buf_size += b->yy_buf_size / 8; + } else { + b->yy_buf_size *= 2; + } + b->yy_ch_buf = (char *) + /* Include room in for 2 EOB chars. */ + yyrealloc( (void *) b->yy_ch_buf, + (size_t) (b->yy_buf_size + 2), yyscanner ); + } else { + /* Can't grow it, we don't own it. */ + b->yy_ch_buf = NULL; + } + if ( b->yy_ch_buf == NULL ) { + yypanic("fatal error - scanner input buffer overflow", yyscanner); + } + yyscanner->yy_c_buf_p = &b->yy_ch_buf[yy_c_buf_p_offset]; + + num_to_read = yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_size - + number_to_move - 1; +]]) + } + + if ( num_to_read > YY_READ_BUF_SIZE ) { + num_to_read = YY_READ_BUF_SIZE; + } + /* Read in more data. */ + yyscanner->yy_n_chars = yyread(&yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[number_to_move], num_to_read, yyscanner); + + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_n_chars = yyscanner->yy_n_chars; + } + + if ( yyscanner->yy_n_chars == 0 ) { + if ( number_to_move == YY_MORE_ADJ ) { + ret_val = EOB_ACT_END_OF_FILE; + yyrestart( yyscanner->yyin_r, yyscanner); + } else { + ret_val = EOB_ACT_LAST_MATCH; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buffer_status = + YY_BUFFER_EOF_PENDING; + } + } else { + ret_val = EOB_ACT_CONTINUE_SCAN; + } + if ((yyscanner->yy_n_chars + number_to_move) > yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_size) { + /* Extend the array by 50%, plus the number we really need. */ + int new_size = yyscanner->yy_n_chars + number_to_move + (yyscanner->yy_n_chars >> 1); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf = (char *) yyrealloc( + (void *) yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf, (size_t) new_size, yyscanner ); + if ( yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf == NULL ) { + yypanic( "out of dynamic memory in yy_get_next_buffer()", yyscanner); + } + /* "- 2" to take care of EOB's */ + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_size = (int) (new_size - 2); + } + + yyscanner->yy_n_chars += number_to_move; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[yyscanner->yy_n_chars] = YY_END_OF_BUFFER_CHAR; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[yyscanner->yy_n_chars + 1] = YY_END_OF_BUFFER_CHAR; + + yyscanner->yytext_ptr = &yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[0]; + + return ret_val; +} + +int yyinput (yyscan_t yyscanner) +{ + int c; + + *yyscanner->yy_c_buf_p = yyscanner->yy_hold_char; + + if ( *yyscanner->yy_c_buf_p == YY_END_OF_BUFFER_CHAR ) { + /* yy_c_buf_p now points to the character we want to return. + * If this occurs *before* the EOB characters, then it's a + * valid NUL; if not, then we've hit the end of the buffer. + */ + if ( yyscanner->yy_c_buf_p < &yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[yyscanner->yy_n_chars] ) { + /* This was really a NUL. */ + *yyscanner->yy_c_buf_p = '\0'; + } else { + /* need more input */ + int offset = (int) (yyscanner->yy_c_buf_p - yyscanner->yytext_ptr); + ++yyscanner->yy_c_buf_p; + + switch ( yy_get_next_buffer( yyscanner ) ) { + case EOB_ACT_LAST_MATCH: + /* This happens because yy_g_n_b() + * sees that we've accumulated a + * token and flags that we need to + * try matching the token before + * proceeding. But for input(), + * there's no matching to consider. + * So convert the EOB_ACT_LAST_MATCH + * to EOB_ACT_END_OF_FILE. + */ + + /* Reset buffer status. */ + yyrestart( yyscanner->yyin_r, yyscanner); + + /*FALLTHROUGH*/ + + case EOB_ACT_END_OF_FILE: + if ( yywrap( yyscanner ) ) { + return 0; + } + if ( ! yyscanner->yy_did_buffer_switch_on_eof ) { + yyrestart( yyscanner->yyin_r, yyscanner ); + } + return yyinput(yyscanner); + + case EOB_ACT_CONTINUE_SCAN: + yyscanner->yy_c_buf_p = yyscanner->yytext_ptr + offset; + break; + } + } + } + + c = *(unsigned char *) yyscanner->yy_c_buf_p; /* cast for 8-bit char's */ + *yyscanner->yy_c_buf_p = '\0'; /* preserve yytext */ + yyscanner->yy_hold_char = *++yyscanner->yy_c_buf_p; + +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yyatbol_flag = (c == '\n'); +m4_ifdef([[M4_MODE_YYLINENO]], [[ + if ( yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yyatbol_flag ) { + yybumpline( yyscanner ); + } +]]) +]]) +m4_ifdef([[M4_MODE_NO_BOL_NEEDED]], [[ +m4_ifdef([[M4_MODE_YYLINENO]], [[ + if ( c == '\n' ) { + yybumpline( yyscanner ); + } + ]]) +]]) + + return c; +} + +/* ENDS special functions + +/* Macros after this point can all be overridden by user definitions in + * section 1. + */ + +m4_ifdef( [[M4_MODE_NO_YYWRAP]],, [[ +extern int yywrap ( yyscan_t yyscanner ); +]]) + +m4_ifdef( [[M4_YY_STACK_USED]], +[[ + m4_ifdef( [[M4_YY_NO_PUSH_STATE]],, + [[ + static void yy_push_state ( int _new_state, yyscan_t yyscanner); + ]]) + m4_ifdef( [[M4_YY_NO_POP_STATE]],, + [[ + static void yy_pop_state ( yyscan_t yyscanner ); + ]]) + m4_ifdef( [[M4_YY_NO_TOP_STATE]],, + [[ + static int yy_top_state ( yyscan_t yyscanner ); + ]]) +]], +[[ +m4_define( [[M4_YY_NO_PUSH_STATE]]) +m4_define( [[M4_YY_NO_POP_STATE]]) +m4_define( [[M4_YY_NO_TOP_STATE]]) +]]) + +/* STARTS Accessor methods to globals. + These are made visible to non-reentrant scanners for convenience. */ + +m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[m4_dnl +m4_ifdef([[M4_YY_NO_GET_EXTRA]],, [[m4_dnl +/** Get the user-defined data for this scanner. + * @param yyscanner The scanner object. + */ +M4_MODE_EXTRA_TYPE yyget_extra (yyscan_t yyscanner) { + return yyscanner->yyextra_r; +} +]]) +]]) + +m4_ifdef( [[M4_YY_NO_GET_LINENO]],, +[[ +/** Get the current line number. + * @param yyscanner The scanner object. + */ +int yyget_lineno (yyscan_t yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + if (cb == NULL) { + return 0; + } + return cb->bs_yylineno; +} +]]) + +m4_ifdef( [[M4_YY_NO_GET_COLUMN]],, +[[ +/** Get the current column number. + * @param yyscanner The scanner object. + */ +int yyget_column (yyscan_t yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + if (cb == NULL) { + return 0; + } + return cb->bs_yycolumn; +} +]]) + +m4_ifdef( [[M4_YY_NO_GET_IN]],, +[[ +/** Get the input stream. + * @param yyscanner The scanner object. + */ +FILE *yyget_in (yyscan_t yyscanner) { + return yyscanner->yyin_r; +} +]]) + +m4_ifdef( [[M4_YY_NO_GET_OUT]],, +[[ +/** Get the output stream. + * @param yyscanner The scanner object. + */ +FILE *yyget_out (yyscan_t yyscanner) { + return yyscanner->yyout_r; +} +]]) + +m4_ifdef( [[M4_YY_NO_GET_LENG]],, +[[ +/** Get the length of the current token. + * @param yyscanner The scanner object. + */ +int yyget_leng (yyscan_t yyscanner) { + return yyscanner->yyleng_r; +} +]]) + +/** Get the current token. + * @param yyscanner The scanner object. + */ +m4_ifdef( [[M4_YY_NO_GET_TEXT]],, +[[ +char *yyget_text (yyscan_t yyscanner) { + return yyscanner->yytext_r; +} +]]) + +m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[m4_dnl +m4_ifdef( [[M4_YY_NO_SET_EXTRA]],, +[[ +/** Set the user-defined data. This data is never touched by the scanner. + * @param user_defined The data to be associated with this scanner. + * @param yyscanner The scanner object. + */ +void yyset_extra(M4_MODE_EXTRA_TYPE user_defined, yyscan_t yyscanner) { + yyscanner->yyextra_r = user_defined ; +} +]]) +]]) + +m4_ifdef( [[M4_YY_NO_SET_LINENO]],, +[[ +/** Set the current line number. + * @param _line_number line number + * @param yyscanner The scanner object. + */ +void yyset_lineno(int _line_number, yyscan_t yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + /* lineno is only valid if an input buffer exists. */ + if (cb == NULL ) { + yypanic( "yyset_lineno called with no buffer", yyscanner ); + } + cb->bs_yylineno = _line_number; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_COLUMN]],, +[[ +/** Set the current column. + * @param _column_no column number + * @param yyscanner The scanner object. + */ +void yyset_column(int _column_no, yyscan_t yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + /* column is only valid if an input buffer exists. */ + if (cb == NULL ) { + yypanic( "yyset_column called with no buffer", yyscanner ); + } + cb->bs_yycolumn = _column_no; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_IN]],, +[[ +/** Set the input stream. This does not discard the current + * input buffer. + * @param _in_str A readable stream. + * @param yyscanner The scanner object. + * @see yy_switch_to_buffer + */ +void yyset_in(FILE * _in_str, yyscan_t yyscanner) { + yyscanner->yyin_r = _in_str ; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_OUT]],, +[[ +void yyset_out( FILE * _out_str, yyscan_t yyscanner) { + yyscanner->yyout_r = _out_str ; +} +]]) + + +m4_ifdef( [[M4_YY_NO_GET_DEBUG]],, +[[ +int yyget_debug (yyscan_t yyscanner) { + return yyscanner->yyflexdebug_r; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_DEBUG]],, +[[ +void yyset_debug(int _bdebug, yyscan_t yyscanner) { + yyscanner->yyflexdebug_r = _bdebug ; +} +]]) + +m4_ifdef([[M4_YY_BISON_LVAL]], [[ +m4_ifdef( [[M4_YY_NO_GET_LVAL]],, +[[ +YYSTYPE * yyget_lval (yyscan_t yyscanner) { + return yylval; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_LVAL]],, +[[ +void yyset_lval(YYSTYPE *yylval_param, yyscan_t yyscanner) { + yylval = yylval_param; +} +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ +m4_ifdef( [[M4_YY_NO_GET_LLOC]],, +[[ +YYLTYPE *yyget_lloc (yyscan_t yyscanner) { + return yylloc; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_LLOC]],, +[[ +void yyset_lloc(YYLTYPE *yylloc_param, yyscan_t yyscanner) { + yylloc = yylloc_param; +} +]]) +]]) + +]]) + +/* ENDS public accessor functions */ + +/* Number of entries by which start-condition stack grows. */ +const int YY_START_STACK_INCR = 25; + +/* Default declaration of generated scanner - a define so the user can + * easily add parameters. + */ +m4_ifdef([[YY_DECL]],, [[m4_dnl +m4_define( [[M4_YY_LEX_PROTO]], [[(yyscan_t yyscanner)]]) +m4_define( [[M4_YY_LEX_DECLARATION]], [[(yyscan_t yyscanner)]]) + +m4_ifdef( [[M4_YY_BISON_LVAL]], +[[ + m4_dnl The bison pure parser is used. Redefine yylex to + m4_dnl accept the lval parameter. + + m4_define( [[M4_YY_LEX_PROTO]], [[\]] + [[(YYSTYPE * yylval_param, yyscan_t yyscanner)]]) + m4_define( [[M4_YY_LEX_DECLARATION]], [[\]] + [[(YYSTYPE * yylval_param, yyscan_t yyscanner)]]) +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ + m4_dnl Locations are used. yylex should also accept the ylloc parameter. + + m4_define( [[M4_YY_LEX_PROTO]], [[\]] + [[(YYSTYPE * yylval_param, YYLTYPE * yylloc_param, yyscan_t yyscanner)]]) + m4_define( [[M4_YY_LEX_DECLARATION]], [[\]] + [[(YYSTYPE *yylval_param, YYLTYPE *yylloc_param, yyscan_t yyscanner)]]) +]]) + +extern int yylex M4_YY_LEX_PROTO; + +m4_define([[YY_DECL]], [[int yylex M4_YY_LEX_DECLARATION]]) +]]) + +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ +static void rule_check_bol(yyscan_t yyscanner) { + if ( yyscanner->yyleng_r > 0 ) { \ + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yyatbol_flag = (yyscanner->yytext_r[yyscanner->yyleng_r - 1] == '\n'); + } +} +]]) + +/* Done after the current pattern has been matched and before the + * corresponding action - sets up yytext. yy_cp and yy_bp are the + * end abd start pointers for the input buffer segment that is + * claimed as yytext. + */ +void yy_do_before_action(yyscan_t yyscanner, char *yy_cp, char *yy_bp) { + yyscanner->yytext_ptr = yy_bp; \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[yyscanner->yytext_ptr -= yyscanner->yy_more_len; \ + yyscanner->yyleng_r = (int) (yy_cp - yyscanner->yytext_ptr);]])]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[yyscanner->yyleng_r = (int) (yy_cp - yy_bp);]]) \ + m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[yyscanner->yyleng_r = (int) (yy_cp - yy_bp);]]) \ + yyscanner->yy_hold_char = *yy_cp; \ + *yy_cp = '\0'; \ +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[ \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[if ( yyscanner->yyleng_r + yyscanner->yy_more_offset >= YYLMAX ) \ + yypanic( "token too large, exceeds YYLMAX", yyscanner);]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[if ( yyscanner->yyleng_r >= YYLMAX ) \ + yypanic( "token too large, exceeds YYLMAX", yyscanner);]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[strncpy( &yyscanner->yytext_r[yyscanner->yy_more_offset], yyscanner->yytext_ptr, yyscanner->yyleng_r + 1);]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yyscanner->yyleng_r += yyscanner->yy_more_offset;]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yyscanner->yy_prev_more_offset = yyscanner->yy_more_offset;]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yyscanner->yy_more_offset = 0;]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[strncpy( yyscanner->yytext_r, yyscanner->yytext_ptr, yyscanner->yyleng_r + 1);]]) \ +]]) \ + yyscanner->yy_c_buf_p = yy_cp; +} + +m4_ifdef( [[M4_MODE_YYLINENO]], [[ +/* FIXME: gate on yy_rule_can_match_eol, this is no longer a macro + * and we can get at yy_act */ +static void yy_less_lineno(int n, yyscan_t yyscanner) { + int yyl; + for ( yyl = n; yyl < yyscanner->yyleng_r; ++yyl ) { + if ( yyscanner->yytext_r[yyl] == '\n' ) { + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->bs_yylineno--; + } + } +} + +static void yy_lineno_rewind_to(char *yy_cp, char *dst, yyscan_t yyscanner) { + const char *p; + for ( p = yy_cp-1; p >= dst; --p) { + if ( *p == '\n' ) { + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->bs_yylineno--; + } + } +} +]]) + +void yy_set_interactive(bool is_interactive, yyscan_t yyscanner) { + if ( yy_current_buffer(yyscanner) == NULL ) { + yyensure_buffer_stack (yyscanner); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = + yy_create_buffer( yyscanner->yyin_r, YY_BUF_SIZE, yyscanner); + } + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_is_interactive = is_interactive; +} + + +void yysetbol(bool at_bol, yyscan_t yyscanner) { + if ( yy_current_buffer(yyscanner) == NULL ) { + yyensure_buffer_stack (yyscanner); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = + yy_create_buffer( yyscanner->yyin_r, YY_BUF_SIZE, yyscanner); + } + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yyatbol_flag = at_bol; +} + +bool yyatbol(yyscan_t yyscanner) { + return (yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yyatbol_flag); +} + +/** Switch to a different input buffer. + * @param new_buffer The new input buffer. + * @param yyscanner The scanner object. + */ +void yy_switch_to_buffer(yybuffer new_buffer, yyscan_t yyscanner) +{ + + /* TODO. We should be able to replace this entire function body + * with + * yypop_buffer_state(); + * yypush_buffer_state(new_buffer); + */ + yyensure_buffer_stack (yyscanner); + if ( yy_current_buffer(yyscanner) == new_buffer ) { + return; + } + if ( yy_current_buffer(yyscanner) ) { + /* Flush out information for old buffer. */ + *yyscanner->yy_c_buf_p = yyscanner->yy_hold_char; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buf_pos = yyscanner->yy_c_buf_p; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_n_chars = yyscanner->yy_n_chars; + } + + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = new_buffer; + yy_load_buffer_state( yyscanner ); + + /* We don't actually know whether we did this switch during + * EOF (yywrap()) processing, but the only time this flag + * is looked at is after yywrap() is called, so it's safe + * to go ahead and always set it. + */ + yyscanner->yy_did_buffer_switch_on_eof = true; +} + + +m4_ifdef( [[M4_YY_NO_SCAN_BUFFER]],, +[[ +/** Setup the input buffer state to scan directly from a user-specified character buffer. + * @param base the character buffer + * @param size the size in bytes of the character buffer + * @param yyscanner The scanner object. + * @return the newly allocated buffer state object. + */ +yybuffer yy_scan_buffer(char * base, size_t size, yyscan_t yyscanner) +{ + yybuffer b; + + if ( size < 2 || + base[size-2] != YY_END_OF_BUFFER_CHAR || + base[size-1] != YY_END_OF_BUFFER_CHAR ) { + /* They forgot to leave room for the EOB's. */ + return NULL; + } + b = (yybuffer) yyalloc( sizeof( struct yy_buffer_state ), yyscanner ); + if ( b == NULL ) { + yypanic( "out of dynamic memory in yy_scan_buffer()", yyscanner ); + } + b->yy_buf_size = (int) (size - 2); /* "- 2" to take care of EOB's */ + b->yy_buf_pos = b->yy_ch_buf = base; + b->yy_is_our_buffer = false; + b->yy_input_file = NULL; + b->yy_n_chars = b->yy_buf_size; + b->yy_is_interactive = false; + b->yyatbol_flag = true; + b->yy_fill_buffer = false; + b->yy_buffer_status = YY_BUFFER_NEW; + + yy_switch_to_buffer( b, yyscanner ); + + return b; +} +]]) + +m4_ifdef( [[M4_YY_NO_SCAN_BYTES]],, +[[ +/** Setup the input buffer state to scan the given bytes. The next call to yylex() will + * scan from a @e copy of @a bytes. + * @param yybytes the byte buffer to scan + * @param _yybytes_len the number of bytes in the buffer pointed to by @a bytes. + * @param yyscanner The scanner object. + * @return the newly allocated buffer state object. + */ +yybuffer yy_scan_bytes(const char * yybytes, int _yybytes_len, yyscan_t yyscanner) { + yybuffer b; + char *buf; + size_t n; + int i; + + /* Get memory for full buffer, including space for trailing EOB's. */ + n = (size_t) (_yybytes_len + 2); + buf = (char *) yyalloc( n, yyscanner ); + if ( buf == 0 ) { + yypanic( "out of dynamic memory in yy_scan_bytes()", yyscanner ); + } + for ( i = 0; i < _yybytes_len; ++i ) { + buf[i] = yybytes[i]; + } + buf[_yybytes_len] = buf[_yybytes_len+1] = YY_END_OF_BUFFER_CHAR; + + b = yy_scan_buffer( buf, n, yyscanner); + if ( b == NULL ) { + yypanic( "bad buffer in yy_scan_bytes()", yyscanner ); + } + /* It's okay to grow etc. this buffer, and we should throw it + * away when we're done. + */ + b->yy_is_our_buffer = true; + + return b; +} +]]) + +m4_ifdef( [[M4_YY_NO_SCAN_STRING]],, +[[ +/** Setup the input buffer state to scan a string. The next call to yylex() will + * scan from a @e copy of @a str. + * @param yystr a NUL-terminated string to scan + * @param yyscanner The scanner object. + * @return the newly allocated buffer state object. + * @note If you want to scan bytes that may contain NUL values, then use + * yy_scan_bytes() instead. + */ +yybuffer yy_scan_string(const char * yystr, yyscan_t yyscanner) +{ + return yy_scan_bytes( yystr, (int) strlen(yystr), yyscanner); +} +]]) + +m4_ifdef( [[M4_YY_NO_PUSH_STATE]],, +[[ +static void yy_push_state(int _new_state, yyscan_t yyscanner) +{ + if ( yyscanner->yy_start_stack_ptr >= yyscanner->yy_start_stack_depth ) { + size_t new_size; + + yyscanner->yy_start_stack_depth += YY_START_STACK_INCR; + new_size = (size_t) yyscanner->yy_start_stack_depth * sizeof( int ); + + if ( yyscanner->yy_start_stack == NULL ) { + yyscanner->yy_start_stack = (int *) yyalloc( new_size, yyscanner ); + + } else { + yyscanner->yy_start_stack = (int *) yyrealloc( + (void *) yyscanner->yy_start_stack, new_size, yyscanner ); + } + if ( yyscanner->yy_start_stack == NULL ) { + yypanic( "out of memory expanding start-condition stack", yyscanner); + } + } + yyscanner->yy_start_stack[yyscanner->yy_start_stack_ptr++] = yystart(yyscanner); + + yybegin(_new_state, yyscanner); +} +]]) + + +m4_ifdef( [[M4_YY_NO_POP_STATE]],, +[[ +static void yy_pop_state (yyscan_t yyscanner) +{ + if ( --yyscanner->yy_start_stack_ptr < 0 ) { + yypanic( "start-condition stack underflow", yyscanner ); + } + yybegin(yyscanner->yy_start_stack[yyscanner->yy_start_stack_ptr], yyscanner); +} +]]) + + +m4_ifdef( [[M4_YY_NO_TOP_STATE]],, +[[ +static int yy_top_state (yyscan_t yyscanner) +{ + return yyscanner->yy_start_stack_ptr > 0 ? yyscanner->yy_start_stack[yyscanner->yy_start_stack_ptr - 1] : yystart(yyscanner); +} +]]) + +%# Code snippets used in various cases of code generation in the main scanner. + +m4_define([[M4_GEN_BACKING_UP]], [[ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_HAS_BACKING_UP]], [[ + /* Generate code to keep backing-up information. */ +m4_ifdef([[M4_MODE_FULLSPD]], [[ + if ( yy_current_state[-1].yy_nxt ) { +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD]], [[ + if ( yy_accept[yy_current_state] ) { +]]) + yyscanner->yy_last_accepting_state = yy_current_state; + yyscanner->yy_last_accepting_cpos = yy_cp; + } +]]) +]]) +]]) + +%# yy_c was formerly YY_CHAR, changed to int because table can now +%# have up to 0x101 entries, since we no longer generate a separate +%# NUL table. +%# +%# Note: on x86-64 architecture with gcc -O2, we save an instruction +%# in the main loop, since the character can now be zero-extended in +%# the process of retrieving it from the input stream or the yy_ec[] +%# or yy_meta[] arrays, whereas previously it was zero-extended by a +%# register-to-register move just prior to the yy_chk[] table lookup +m4_define([[M4_GEN_NEXT_COMPRESSED_STATE]], [[ + int yy_c = $1; + /* Save the backing-up info \before/ computing the next state + * because we always compute one more state than needed - we + * always proceed until we reach a jam state + */ + M4_GEN_BACKING_UP + + while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { + yy_current_state = (int) yy_def[yy_current_state]; + +m4_ifdef([[M4_MODE_USEMECS]], [[ + /* We've arranged it so that templates are never chained + * to one another. This means we can afford to make a + * very simple test to see if we need to convert to + * yy_c's meta-equivalence class without worrying + * about erroneously looking up the meta-equivalence + * class twice + */ + + /* lastdfa + 2 == YY_JAMSTATE + 1 is the beginning of the templates */ + if (yy_current_state >= YY_JAMSTATE + 1) { + yy_c = yy_meta[yy_c]; + } +]]) + } + yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; +]]) + +m4_define([[M4_GEN_START_STATE]], [[ + /* Generate the code to find the start state. */ +m4_ifdef([[M4_MODE_FULLSPD]], [[ +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[yy_current_state = yy_start_state_list[yyscanner->yy_start + yyatbol()];]]) +m4_ifdef([[M4_MODE_NO_BOL_NEEDED]], [[yy_current_state = yy_start_state_list[yyscanner->yy_start];]]) +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD]], [[ + yy_current_state = yyscanner->yy_start; +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[yy_current_state += yyatbol(yyscanner);]]) + /* Set up for storing up states. */ + m4_ifdef( [[M4_MODE_USES_REJECT]], [[ + yyscanner->yy_state_ptr = yyscanner->yy_state_buf; + *yyscanner->yy_state_ptr++ = yy_current_state; +]]) +]]) +]]) + +m4_define([[M4_GEN_NEXT_MATCH_FULLSPD]], [[ + { + const struct yy_trans_info *yy_trans_info; + YY_CHAR yy_c; + + for ( yy_c = $1; + (yy_trans_info = &yy_current_state[yy_c])->yy_verify == yy_c; + yy_c = $2 ) + { + yy_current_state += yy_trans_info->yy_nxt; + + M4_GEN_BACKING_UP + } + } +]]) + +/* + * Helpers for yylex() + */ + +%# Conditional indirection through an equivalence map +m4_ifdef([[M4_MODE_USEECS]], m4_define([[M4_EC]], [[*(yy_ec+$1)]])) +m4_ifdef([[M4_MODE_NO_USEECS]], [[m4_define([[M4_EC]], [[$1]])]]) + +/* yy_get_previous_state - get the state just before the EOB char was reached */ + +static yy_state_type yy_get_previous_state (yyscan_t yyscanner) { + yy_state_type yy_current_state; + char *yy_cp; + + M4_GEN_START_STATE + for ( yy_cp = yyscanner->yytext_ptr + YY_MORE_ADJ; yy_cp < yyscanner->yy_c_buf_p; ++yy_cp ) { + /* Generate the code to find the next state. */ + m4_ifdef([[M4_MODE_NO_NULTRANS]], [[m4_define([[CHAR_MAP_3]], [[(*yy_cp ? M4_EC(YY_SC_TO_UI(*yy_cp)) : YY_NUL_EC)]])]]) + m4_ifdef([[M4_MODE_NULTRANS]], [[m4_define([[CHAR_MAP_3]], [[M4_EC(YY_SC_TO_UI(*yy_cp))]])]]) + + m4_ifdef([[M4_MODE_NULTRANS]], [[ + /* Compressed tables back up *before* they match. */ + m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[M4_GEN_BACKING_UP]]) + if ( *yy_cp ) { + ]]) + + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[ + m4_ifdef([[M4_MODE_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state][CHAR_MAP_3];]]) + m4_ifdef([[M4_MODE_NO_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + CHAR_MAP_3];]]) + ]]) + + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_current_state += yy_current_state[CHAR_MAP_3].yy_nxt;]]) + m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[M4_GEN_NEXT_COMPRESSED_STATE(CHAR_MAP_3)]]) + +m4_ifdef([[M4_MODE_NULTRANS]], [[ + } else { + yy_current_state = yy_NUL_trans[yy_current_state]; + } +]]) + + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[M4_GEN_BACKING_UP]]) + m4_ifdef([[M4_MODE_FULLSPD]], [[M4_GEN_BACKING_UP]]) + m4_ifdef([[M4_MODE_USES_REJECT]], [[*yyscanner->yy_state_ptr++ = yy_current_state;]]) + } + + return yy_current_state; +} + +/* yy_try_NUL_trans - try to make a transition on the NUL character + * + * synopsis + * next_state = yy_try_NUL_trans( current_state ); + */ +static yy_state_type yy_try_NUL_trans(yy_state_type yy_current_state, yyscan_t yyscanner) +{ + bool yy_is_jam; + /* Generate code for handling NUL's, if needed. */ + + /* First, deal with backing up and setting up yy_cp if the scanner + * finds that it should JAM on the NUL. + * + * Only generate a definition for "yy_cp" if we'll generate code + * that uses it. Otherwise lint and the like complain. + */ + m4_ifdef([[M4_MODE_NEED_YY_CP]], [[char *yy_cp = yyscanner->yy_c_buf_p;]]) + +%# Note that this statement block and the following three are +%# not executed serially but are an if-then-else cascade +%# for different table modes. +m4_ifdef([[M4_MODE_NULTRANS]], [[ + yy_current_state = yy_NUL_trans[yy_current_state]; + yy_is_jam = (yy_current_state == 0); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS]], [[ +m4_ifdef([[M4_MODE_NULTRANS_FULLTBL]], [[ +m4_ifdef([[M4_MODE_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state][YY_NUL_EC];]]) +m4_ifdef([[M4_MODE_NO_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + YY_NUL_EC];]]) + yy_is_jam = (yy_current_state <= 0); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS_FULLTBL]], [[ +m4_ifdef([[M4_MODE_NULTRANS_FULLSPD]], [[ + int yy_c = YY_NUL_EC; + + const struct yy_trans_info *yy_trans_info; + + yy_trans_info = &yy_current_state[(unsigned int) yy_c]; + yy_current_state += yy_trans_info->yy_nxt; + yy_is_jam = (yy_trans_info->yy_verify != yy_c); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS_FULLSPD]], [[ +M4_GEN_NEXT_COMPRESSED_STATE(YY_NUL_EC) +yy_is_jam = (yy_current_state == YY_JAMSTATE); +m4_ifdef([[M4_MODE_USES_REJECT]], [[ + /* Only stack this state if it's a transition we + * actually make. If we stack it on a jam, then + * the state stack and yy_c_buf_p get out of sync. + */ + if ( ! yy_is_jam ) { + *yyscanner->yy_state_ptr++ = yy_current_state; + } + ]]) +]]) +]]) +]]) +%# End of if-else cascade + +m4_ifdef([[M4_MODE_NULTRANS_WRAP]], [[ + /* If we've entered an accepting state, back up; note that + * compressed tables have *already* done such backing up, so + * we needn't bother with it again. + */ + if ( ! yy_is_jam ) { + M4_GEN_BACKING_UP + } +]]) + + (void)yyscanner; /* forestall unused-argument warning */ + return yy_is_jam ? 0 : yy_current_state; +} + +/** The main scanner function which does all the work. + */ +YY_DECL { + yy_state_type yy_current_state; + char *yy_cp, *yy_bp; + int yy_act; + +m4_ifdef( [[M4_YY_BISON_LVAL]], +[[ + yylval = yylval_param; +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ + yylloc = yylloc_param; +]]) + + if ( !yyscanner->yy_init ) { + yyscanner->yy_init = true; + + m4_ifdef([[YY_USER_INIT]], [[YY_USER_INIT]]) + +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + /* Create the reject buffer large enough to save one state per allowed character. */ + if ( yyscanner->yy_state_buf == NULL ) { + yyscanner->yy_state_buf = (yy_state_type *)yyalloc(YY_STATE_BUF_SIZE, yyscanner); + } + if ( yyscanner->yy_state_buf == NULL) { + yypanic( "out of dynamic memory in yylex()", yyscanner); + } +]]) + + if ( yyscanner->yy_start == 0 ) { + yyscanner->yy_start = 1; /* first start state */ + } + if ( yyscanner->yyin_r == NULL ) { + yyscanner->yyin_r = stdin; + } + if ( yyscanner->yyout_r == NULL ) { + yyscanner->yyout_r = stdout; + } + if ( yy_current_buffer(yyscanner) == NULL ) { + yyensure_buffer_stack (yyscanner); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = + yy_create_buffer( yyscanner->yyin_r, YY_BUF_SIZE, yyscanner); + } + + yy_load_buffer_state( yyscanner ); + } + + /* open scope of user declarationns */ + { +%% [4.0] user's declarations go here + + while ( /*CONSTCOND*/1 ) { /* loops until end-of-file is reached */ +m4_ifdef( [[M4_MODE_YYMORE_USED]], [[ +m4_ifdef( [[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ + yyscanner->yy_more_len = 0; + if ( yyscanner->yy_more_flag ) { + yyscanner->yy_more_len = (int) (yyscanner->yy_c_buf_p - yyscanner->yytext_ptr); + yyscanner->yy_more_flag = false; + } +]]) +]]) + yy_cp = yyscanner->yy_c_buf_p; + + /* Support of yytext. */ + *yy_cp = yyscanner->yy_hold_char; + + /* yy_bp points to the position in yy_ch_buf of the start of + * the current run. + */ + yy_bp = yy_cp; + +M4_GEN_START_STATE + + yy_match: + /* Generate the code to find the next match. */ +m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[m4_dnl +m4_ifdef([[M4_MODE_GENTABLES]], [[m4_dnl + while ((yy_current_state = yy_nxt[yy_current_state][ M4_EC(YY_SC_TO_UI(*yy_cp)) ]) > 0) { +]]) +m4_ifdef([[M4_MODE_NO_GENTABLES]], [[ + while ((yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + M4_EC(YY_SC_TO_UI(*yy_cp)) ]) > 0) { +]]) +M4_GEN_BACKING_UP + yy_cp++; + } + yy_current_state = -yy_current_state; +]]) +m4_ifdef([[M4_MODE_FULLSPD]], [[ + M4_GEN_NEXT_MATCH_FULLSPD(M4_EC(YY_SC_TO_UI(*yy_cp)), M4_EC(YY_SC_TO_UI(*++yy_cp))) +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[ + do { + M4_GEN_NEXT_COMPRESSED_STATE(M4_EC(YY_SC_TO_UI(*yy_cp))) + + m4_ifdef([[M4_MODE_USES_REJECT]], [[*yyscanner->yy_state_ptr++ = yy_current_state;]]) + ++yy_cp; + + } + m4_ifdef([[M4_MODE_INTERACTIVE]], [[while ( yy_base[yy_current_state] != YY_JAMBASE );]]) + m4_ifdef([[M4_MODE_NO_INTERACTIVE]], [[while ( yy_current_state != YY_JAMSTATE );]]) + +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_NO_INTERACTIVE]], [[ + /* Do the guaranteed-needed backing up to figure out + * the match. + */ + yy_cp = yyscanner->yy_last_accepting_cpos; + yy_current_state = yyscanner->yy_last_accepting_state; +]]) +]]) +]]) + + yy_find_action: + /* code to find the action number goes here */ + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_act = yy_current_state[-1].yy_nxt;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yy_act = yy_accept[yy_current_state];]]) +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT]], [[ + yy_current_state = *--yyscanner->yy_state_ptr; + yyscanner->yy_lp = yy_accept[yy_current_state]; +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT_REALLY_USED]], [[find_rule: /* we branch to this label when backing up */]]) + for ( ; ; ) { /* loop until we find out what rule we matched */ + if (yyscanner->yy_lp && yyscanner->yy_lp < yy_accept[yy_current_state + 1]) { + yy_act = yy_acclist[yyscanner->yy_lp]; +m4_ifdef([[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + if ((yy_act & YY_TRAILING_HEAD_MASK) != 0 || yyscanner->yy_looking_for_trail_begin) { + if (yy_act == yyscanner->yy_looking_for_trail_begin) { + yyscanner->yy_looking_for_trail_begin = 0; + yy_act &= ~YY_TRAILING_HEAD_MASK; + break; + } + } else if (( yy_act & YY_TRAILING_MASK) != 0) { + yyscanner->yy_looking_for_trail_begin = yy_act & ~YY_TRAILING_MASK; + yyscanner->yy_looking_for_trail_begin |= YY_TRAILING_HEAD_MASK; +m4_ifdef([[M4_MODE_REAL_REJECT]], [[ + /* Remember matched text in case we back up + * due to REJECT. + */ + yyscanner->yy_full_match = yy_cp; + yyscanner->yy_full_state = yyscanner->yy_state_ptr; + yyscanner->yy_full_lp = yyscanner->yy_lp; +]]) + } else { + yyscanner->yy_full_match = yy_cp; + yyscanner->yy_full_state = yyscanner->yy_state_ptr; + yyscanner->yy_full_lp = yyscanner->yy_lp; + break; + } + ++yyscanner->yy_lp; + goto find_rule; +]]) +m4_ifdef([[M4_MODE_NO_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + /* Remember matched text in case we back up due to + * trailing context plus REJECT. + */ + yyscanner->yy_full_match = yy_cp; + break; +]]) + } + + --yy_cp; + + /* We could consolidate the following two lines with those at + * the beginning, but at the cost of complaints that we're + * branching inside a loop. + */ + yy_current_state = *--yyscanner->yy_state_ptr; + yyscanner->yy_lp = yy_accept[yy_current_state]; + } /* close for */ +]]) +m4_ifdef([[M4_MODE_FIND_ACTION_COMPRESSED]], [[ yy_act = yy_accept[yy_current_state]; + if ( yy_act == 0 ) { /* have to back up */ + yy_cp = yyscanner->yy_last_accepting_cpos; + yy_current_state = yyscanner->yy_last_accepting_state; + yy_act = yy_accept[yy_current_state]; + } +]]) + + yy_do_before_action(yyscanner, yy_cp, yy_bp); + +m4_ifdef( [[M4_MODE_YYLINENO]],[[ +m4_define([[M4_YYL_BASE]], [[m4_ifdef([[M4_MODE_YYMORE_USED]], + [[m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], + [[yyscanner->yy_prev_more_offset]], [[yyscanner->yy_more_len]])]], [[0]])]]) + if ( yy_act != YY_END_OF_BUFFER && yy_rule_can_match_eol[yy_act] ) { + int yyl; + for ( yyl = M4_YYL_BASE; yyl < yyscanner->yyleng_r; ++yyl ) { + if ( yyscanner->yytext_r[yyl] == '\n' ) { + yybumpline( yyscanner ); + + } + } + } +]]) + + do_action: /* This label is used only to access EOF actions. */ + +m4_ifdef([[M4_MODE_DEBUG]], [[ + if ( yyscanner->yyflexdebug_r ) { + if ( yy_act == 0 ) { + fprintf( stderr, "--scanner backing up\n" ); + } else if ( yy_act < YY_NUM_RULES ) { + fprintf( stderr, "--accepting rule at line %ld (\"%s\")\n", + (long)yy_rule_linenum[yy_act], yyscanner->yytext_r ); + } else if ( yy_act == YY_NUM_RULES ) { + fprintf( stderr, "--accepting default rule (\"%s\")\n", + yyscanner->yytext_r ); + } else if ( yy_act == YY_NUM_RULES + 1 ) { + fprintf( stderr, "--(end of buffer or a NUL)\n" ); + } else { + fprintf( stderr, "--EOF (start condition %d)\n", yystart(yyscanner) ); + } + } +]]) + + switch ( yy_act ) { /* beginning of action switch */ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_HAS_BACKING_UP]], [[ + case 0: /* must back up */ + /* undo the effects of yy_do_before_action() */ + *yy_cp = yyscanner->yy_hold_char; + + /* Backing-up info for compressed tables is taken \after/ */ + /* yy_cp has been incremented for the next state. */ + yy_cp = yyscanner->yy_last_accepting_cpos; + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_cp++;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yy_cp++;]]) + + yy_current_state = yyscanner->yy_last_accepting_state; + goto yy_find_action; +]]) +]]) +%% [5.0] user actions get inserted here + + case YY_END_OF_BUFFER: + { + /* Amount of text matched not including the EOB char. */ + int yy_amount_of_matched_text = (int) (yy_cp - yyscanner->yytext_ptr) - 1; + + /* Undo the effects of yy_do_before_action(). */ + *yy_cp = yyscanner->yy_hold_char; + YY_RESTORE_YY_MORE_OFFSET + + if ( yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buffer_status == YY_BUFFER_NEW ) { + /* We're scanning a new file or input source. It's + * possible that this happened because the user + * just pointed yyin at a new source and called + * yylex(). If so, then we have to assure + * consistency between yy_current_buffer(yyscanner) and our + * globals. Here is the right place to do so, because + * this is the first action (other than possibly a + * back-up) that will match for the new input source. + */ + yyscanner->yy_n_chars = yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_n_chars; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_input_file = yyscanner->yyin_r; + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_buffer_status = YY_BUFFER_NORMAL; + } + + /* Note that here we test for yy_c_buf_p "<=" to the position + * of the first EOB in the buffer, since yy_c_buf_p will + * already have been incremented past the NUL character + * (since all states make transitions on EOB to the + * end-of-buffer state). Contrast this with the test + * in input(). + */ + if ( yyscanner->yy_c_buf_p <= &yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[yyscanner->yy_n_chars] ) { /* This was really a NUL. */ + yy_state_type yy_next_state; + + yyscanner->yy_c_buf_p = yyscanner->yytext_ptr + yy_amount_of_matched_text; + + yy_current_state = yy_get_previous_state( yyscanner ); + + /* Okay, we're now positioned to make the NUL + * transition. We couldn't have + * yy_get_previous_state() go ahead and do it + * for us because it doesn't know how to deal + * with the possibility of jamming (and we don't + * want to build jamming into it because then it + * will run more slowly). + */ + + yy_next_state = yy_try_NUL_trans( yy_current_state, yyscanner); + + yy_bp = yyscanner->yytext_ptr + YY_MORE_ADJ; + + if ( yy_next_state ) { + /* Consume the NUL. */ + yy_cp = ++yyscanner->yy_c_buf_p; + yy_current_state = yy_next_state; + goto yy_match; + } else { +%# Disguised case statement on table modes + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_cp = yyscanner->yy_c_buf_p;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yy_cp = yyscanner->yy_c_buf_p;]]) +m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_NOT_MODE_INTERACTIVE]], [[ + /* Do the guaranteed-needed backing up to figure + * out the match. + */ + yy_cp = yyscanner->yy_last_accepting_cpos; + yy_current_state = yyscanner->yy_last_accepting_state; +]]) +]]) +%# Disguised case statement on table modes ends +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT_OR_INTERACTIVE]], [[ + /* Still need to initialize yy_cp, though + * yy_current_state was set up by + * yy_get_previous_state(). + */ + yy_cp = yyscanner->yy_c_buf_p; +]]) +]]) + goto yy_find_action; + } + } else { /* not a NUL */ + switch ( yy_get_next_buffer( yyscanner ) ) { + case EOB_ACT_END_OF_FILE: + yyscanner->yy_did_buffer_switch_on_eof = false; + + if ( yywrap( yyscanner ) ) { + /* Note: because we've taken care in + * yy_get_next_buffer() to have set up + * yytext, we can now set up + * yy_c_buf_p so that if some total + * hoser (like flex itself) wants to + * call the scanner after we return the + * YY_NULL, it'll still work - another + * YY_NULL will get returned. + */ + yyscanner->yy_c_buf_p = yyscanner->yytext_ptr + YY_MORE_ADJ; + + yy_act = YY_STATE_EOF(yystart(yyscanner)); + goto do_action; + } else { + if ( ! yyscanner->yy_did_buffer_switch_on_eof ) { + yyrestart( yyscanner->yyin_r, yyscanner ); + } + } + break; + case EOB_ACT_CONTINUE_SCAN: + yyscanner->yy_c_buf_p = + yyscanner->yytext_ptr + yy_amount_of_matched_text; + + yy_current_state = yy_get_previous_state( yyscanner ); + + yy_cp = yyscanner->yy_c_buf_p; + yy_bp = yyscanner->yytext_ptr + YY_MORE_ADJ; + goto yy_match; + + case EOB_ACT_LAST_MATCH: + yyscanner->yy_c_buf_p = + &yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top]->yy_ch_buf[yyscanner->yy_n_chars]; + + yy_current_state = yy_get_previous_state( yyscanner ); + + yy_cp = yyscanner->yy_c_buf_p; + yy_bp = yyscanner->yytext_ptr + YY_MORE_ADJ; + goto yy_find_action; + } /* end EOB inner switch */ + } /* end if */ + break; + } /* case YY_END_OF_BUFFER */ + default: + yypanic("fatal flex scanner internal error--no action found", yyscanner); + } /* end of action switch */ + } /* end of scanning one token */ + } /* end of user's declarations */ +} /* end of yylex */ + +m4_undefine([[yyless]]) + +/* Redefine yyless() so it works in section 3 code. */ + +void yyless(int n, yyscan_t yyscanner) { + /* Undo effects of setting up yytext. */ + m4_ifdef([[M4_MODE_YYLINENO]], [[yy_less_lineno(n, yyscanner);]]) + yyscanner->yytext_r[yyscanner->yyleng_r] = yyscanner->yy_hold_char; + yyscanner->yy_c_buf_p = yyscanner->yytext_r + n; + yyscanner->yy_hold_char = *yyscanner->yy_c_buf_p; + *yyscanner->yy_c_buf_p = '\0'; + yyscanner->yyleng_r = n; +} + +static int yy_init_globals (yyscan_t yyscanner) { + /* + * This function is called from yylex_destroy(), so don't allocate here. + */ + yyscanner->yy_buffer_stack = NULL; + yyscanner->yy_buffer_stack_top = 0; + yyscanner->yy_buffer_stack_max = 0; + yyscanner->yy_c_buf_p = NULL; + yyscanner->yy_init = false; + yyscanner->yy_start = 0; + yyscanner->yy_start_stack_ptr = 0; + yyscanner->yy_start_stack_depth = 0; + yyscanner->yy_start_stack = NULL; + +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + yyscanner->yy_state_buf = 0; + yyscanner->yy_state_ptr = 0; + yyscanner->yy_full_match = 0; + yyscanner->yy_lp = 0; +]]) + +m4_ifdef( [[M4_MODE_YYTEXT_IS_ARRAY]], +[[ + yyscanner->yytext_ptr = 0; + yyscanner->yy_more_offset = 0; + yyscanner->yy_prev_more_offset = 0; +]]) + + yyscanner->yyin_r = NULL; + yyscanner->yyout_r = NULL; + + /* For future reference: Set errno on error, since we are called by + * yylex_init() + */ + return 0; +} + +/* User-visible API */ + +/* yylex_init is special because it creates the scanner itself, so it is + * the ONLY reentrant function that doesn't take the scanner as the last argument. + * That's why we explicitly handle the declaration, instead of using our macros. + */ +int yylex_init(yyscan_t* ptr_yy_globals) { + if (ptr_yy_globals == NULL) { + errno = EINVAL; + return 1; + } + + *ptr_yy_globals = (yyscan_t) yyalloc ( sizeof( struct yyguts_t ), NULL ); + + if (*ptr_yy_globals == NULL) { + errno = ENOMEM; + return 1; + } + + /* By setting to 0xAA, we expose bugs in yy_init_globals. Leave at 0x00 for releases. */ + memset(*ptr_yy_globals,0x00,sizeof(struct yyguts_t)); + + return yy_init_globals ( *ptr_yy_globals ); +} + +m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[m4_dnl +/* yylex_init_extra has the same functionality as yylex_init, but follows the + * convention of taking the scanner as the last argument. Note however, that + * this is a *pointer* to a scanner, as it will be allocated by this call (and + * is the reason, too, why this function also must handle its own declaration). + * The user defined value in the first argument will be available to yyalloc in + * the yyextra field. + */ +int yylex_init_extra( M4_MODE_EXTRA_TYPE yy_user_defined, yyscan_t* ptr_yy_globals ) { + struct yyguts_t dummy_yyguts; + + yyset_extra (yy_user_defined, &dummy_yyguts); + + if (ptr_yy_globals == NULL) { + errno = EINVAL; + return 1; + } + + *ptr_yy_globals = (yyscan_t) yyalloc ( sizeof( struct yyguts_t ), &dummy_yyguts ); + + if (*ptr_yy_globals == NULL) { + errno = ENOMEM; + return 1; + } + + /* By setting to 0xAA, we expose bugs in + yy_init_globals. Leave at 0x00 for releases. */ + memset(*ptr_yy_globals,0x00,sizeof(struct yyguts_t)); + + yyset_extra (yy_user_defined, *ptr_yy_globals); + + return yy_init_globals ( *ptr_yy_globals ); +} +]]) + +/* yylex_destroy is for both reentrant and non-reentrant scanners. */ +int yylex_destroy (yyscan_t yyscanner) { + + /* Pop the buffer stack, destroying each element. */ + while(yy_current_buffer(yyscanner)) { + yy_delete_buffer( yy_current_buffer(yyscanner), yyscanner ); + yyscanner->yy_buffer_stack[yyscanner->yy_buffer_stack_top] = NULL; + yypop_buffer_state(yyscanner); + } + + /* Destroy the stack itself. */ + yyfree(yyscanner->yy_buffer_stack, yyscanner); + yyscanner->yy_buffer_stack = NULL; + + /* Destroy the start condition stack. */ + yyfree( yyscanner->yy_start_stack, yyscanner ); + yyscanner->yy_start_stack = NULL; + +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + yyfree ( yyscanner->yy_state_buf, yyscanner); + yyscanner->yy_state_buf = NULL; +]]) + + /* Reset the globals. This is important in a non-reentrant scanner so the next time + * yylex() is called, initialization will occur. */ + yy_init_globals(yyscanner); + + /* Destroy the main struct (reentrant only). */ + yyfree ( yyscanner, yyscanner ); + yyscanner = NULL; + return 0; +} + +m4_ifdef([[M4_YY_MAIN]], [[ +int main () { + yyscan_t lexer; + yylex_init(&lexer); + yylex( lexer ); + yylex_destroy( lexer); + yylex(); + + return 0; +} +]]) + +%# Local Variables: +%# mode:c +%# c-file-style:"k&r" +%# c-basic-offset:8 +%# End: @@ -101,7 +101,7 @@ static void dump_cclp (FILE* file, int cclp) putc ('[', file); - for (i = 0; i < csize; ++i) { + for (i = 0; i < ctrl.csize; ++i) { if (ccl_contains(cclp, i)){ int start_char = i; @@ -109,7 +109,7 @@ static void dump_cclp (FILE* file, int cclp) fputs (readable_form (i), file); - while (++i < csize && ccl_contains(cclp,i)) ; + while (++i < ctrl.csize && ccl_contains(cclp,i)) ; if (i - 1 > start_char) /* this was a run */ @@ -138,7 +138,7 @@ ccl_set_diff (int a, int b) * addding each char in a that is not in b. * (This could be O(n^2), but n is small and bounded.) */ - for ( ch = 0; ch < csize; ++ch ) + for ( ch = 0; ch < ctrl.csize; ++ch ) if (ccl_contains (a, ch) && !ccl_contains(b, ch)) ccladd (d, ch); @@ -250,7 +250,7 @@ void list_character_set (FILE *file, int cset[]) putc ('[', file); - for (i = 0; i < csize; ++i) { + for (i = 0; i < ctrl.csize; ++i) { if (cset[i]) { int start_char = i; @@ -258,7 +258,7 @@ void list_character_set (FILE *file, int cset[]) fputs (readable_form (i), file); - while (++i < csize && cset[i]) ; + while (++i < ctrl.csize && cset[i]) ; if (i - 1 > start_char) /* this was a run */ diff --git a/src/chkskel.sh b/src/chkskel.sh new file mode 100644 index 0000000..9603ff8 --- /dev/null +++ b/src/chkskel.sh @@ -0,0 +1,33 @@ +#! /bin/sh + +# This file is part of flex. + +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: + +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. + +# Neither the name of the University nor the names of its contributors +# may be used to endorse or promote products derived from this software +# without specific prior written permission. + +# THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR +# IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED +# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +# PURPOSE. + +if test ! $# = 1; then + echo 'Usage: chkskel.sh file' >&2 + exit 1 +fi +file=$1 +lines=$(grep -c '^ "%%' "${file}") +if [ ! "${lines}" -eq 6 ]; then + echo 'ERROR: skeleton does not have the right number of %% section lines' + exit 2 +fi diff --git a/src/cpp-flex.skl b/src/cpp-flex.skl new file mode 100644 index 0000000..e33265d --- /dev/null +++ b/src/cpp-flex.skl @@ -0,0 +1,4305 @@ +%# -*-C-*- vi: set ft=c: +%# +%# All generated macros for the m4 stage contain the text "m4" or "M4" +%# in them. This is to distinguish them from CPP macros. +%# The exception to this rule is YY_G, which is an m4 macro, +%# but it needs to be remain short because it is used everywhere. +%# +%# The m4 macros complicate reading this code enough that being +%# prescriptive about whitespace and braces is more than usually +%# important. So please set your C style to K&R, aka 1TBS with +%# tabs when editing this file. Braces around single-statement +%# if/while/for/do/switch/break bodies are mandatory. +%# + +%# Macros for preproc stage. +m4preproc_changecom + +%# Macros for runtime processing stage. +m4_changecom +m4_changequote +m4_changequote([[, ]]) + +%# Properties not used in the skeleton - meant to be read by the Flex code +m4_define([[M4_PROPERTY_BACKEND_NAME]], [[C/C++]]) +%#m4_define([[M4_PROPERTY_SOURCE_SUFFIX]], [[]]) +m4_define([[M4_PROPERTY_TRACE_LINE_REGEXP]], [[^#line ([0-9]+) "(.*)"]]) +m4_define([[M4_PROPERTY_TRACE_LINE_TEMPLATE]], [[#line %d "%s"]]) + +%# Macro hooks used by Flex code generators start here +m4_define([[M4_HOOK_INT32]], [[flex_int32_t]]) +m4_define([[M4_HOOK_INT16]], [[flex_int16_t]]) +m4_define([[M4_HOOK_COMMENT_OPEN]], [[/*]]) +m4_define([[M4_HOOK_COMMENT_CLOSE]], [[*/]]) +%# If this is not defined, no trace lines will be generated. +m4_define([[M4_HOOK_TRACE_LINE_FORMAT]], [[#line $1 "$2" +]]) +m4_define([[M4_HOOK_TABLE_OPENER]], [[{]]) +m4_define([[M4_HOOK_TABLE_CONTINUE]], [[},]]) +m4_define([[M4_HOOK_TABLE_CLOSER]], [[};]]) +m4_define([[M4_HOOK_RELATIVIZE]], [[$1]]) +m4_define([[M4_HOOK_STATE_ENTRY_FORMAT]], [[ &yy_transition[$1], +]]) +m4_define([[M4_HOOK_NORMAL_STATE_CASE_ARM]], [[ case $1:]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_ARM]], [[ case YY_STATE_EOF($1):]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_FALLTHROUGH]], [[ /* FALLTHROUGH */]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_TERMINATE]], [[ yyterminate(); +]]) +m4_define([[M4_HOOK_TAKE_YYTEXT]], [[YY_DO_BEFORE_ACTION; /* set up yytext */]]) +m4_define([[M4_HOOK_RELEASE_YYTEXT]], [[*yy_cp = YY_G(yy_hold_char); /* undo effects of setting up yytext */]]) +m4_define([[M4_HOOK_CHAR_REWIND]], [[YY_G(yy_c_buf_p) = yy_cp -= $1;]]) +m4_define([[M4_HOOK_LINE_REWIND]], [[YY_LINENO_REWIND_TO(yy_cp - $1);]]) +m4_define([[M4_HOOK_CHAR_FORWARD]], [[YY_G(yy_c_buf_p) = yy_cp = yy_bp + $1;]]) +m4_define([[M4_HOOK_LINE_FORWARD]], [[YY_LINENO_REWIND_TO(yy_bp + $1);]]) +m4_define([[M4_HOOK_CONST_DEFINE_BYTE]], [[#define $1 $2 +]]) +m4_define([[M4_HOOK_CONST_DEFINE_STATE]], [[#define $1 $2 +]]) +m4_define([[M4_HOOK_CONST_DEFINE_UINT]], [[#define $1 $2 +]]) +m4_define([[M4_HOOK_CONST_DEFINE_BOOL]], [[#define $1 $2 +]]) +m4_define([[M4_HOOK_CONST_DEFINE_UNKNOWN]], [[#define $1 $2 +]]) +m4_define([[M4_HOOK_SET_OFFSET_TYPE]], [[#define YY_OFFSET_TYPE $1 +]]) +m4_define([[M4_HOOK_SET_YY_DECL]], [[#define YY_DECL $1 +]]) +m4_define([[M4_HOOK_SET_USERINIT]], [[#define YY_USER_INIT $1 +]]) +m4_define([[M4_HOOK_SET_RULE_SETUP]], [[YY_RULE_SETUP +]]) +m4_define([[M4_HOOK_SET_PREACTION]], [[#define YY_USER_ACTION $1 +]]) +m4_define([[M4_HOOK_STATE_CASE_BREAK]], [[/*LINTED*/break;]]) +m4_define([[M4_HOOK_SET_POSTACTION]], [[m4_define([[M4_HOOK_STATE_CASE_BREAK]], [[$1]])]]) +m4_define([[M4_HOOK_FATAL_ERROR]], [[yypanic($1 M4_YY_CALL_LAST_ARG);]]) +m4_define([[M4_HOOK_ECHO]], [[yyecho();]]) + +m4_ifdef([[M4_MODE_YYTERMINATE]], [[m4_define([[yyterminate]], [[M4_MODE_YYTERMINATE]])]]) + +m4_define([[M4_HOOK_REJECT]], [[yyreject()]]) + +%% [0.0] Make hook macros available to Flex + +m4_ifdef( [[M4_YY_IN_HEADER]], [[m4_dnl +#ifndef M4_MODE_PREFIX[[HEADER_H]] +#define M4_MODE_PREFIX[[HEADER_H]] +#define M4_MODE_PREFIX[[IN_HEADER]] 1 + +]]) +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +m4_ifelse(M4_MODE_PREFIX,yy,, +#define yy_create_buffer M4_MODE_PREFIX[[_create_buffer]] +#define yy_delete_buffer M4_MODE_PREFIX[[_delete_buffer]] +#define yy_scan_buffer M4_MODE_PREFIX[[_scan_buffer]] +#define yy_scan_string M4_MODE_PREFIX[[_scan_string]] +#define yy_scan_bytes M4_MODE_PREFIX[[_scan_bytes]] +#define yy_init_buffer M4_MODE_PREFIX[[_init_buffer]] +#define yy_flush_buffer M4_MODE_PREFIX[[_flush_buffer]] +#define yy_load_buffer_state M4_MODE_PREFIX[[_load_buffer_state]] +#define yy_switch_to_buffer M4_MODE_PREFIX[[_switch_to_buffer]] +#define yypush_buffer_state M4_MODE_PREFIX[[push_buffer_state]] +#define yypop_buffer_state M4_MODE_PREFIX[[pop_buffer_state]] +#define yyensure_buffer_stack M4_MODE_PREFIX[[ensure_buffer_stack]] +#define yyflexdebug M4_MODE_PREFIX[[flexdebug]] +#define yyin M4_MODE_PREFIX[[in]] +#define yyleng M4_MODE_PREFIX[[leng]] +#define yylex M4_MODE_PREFIX[[lex]] +#define yylineno M4_MODE_PREFIX[[lineno]] +#define yyout M4_MODE_PREFIX[[out]] +#define yyrestart M4_MODE_PREFIX[[restart]] +#define yytext M4_MODE_PREFIX[[text]] +#define yywrap M4_MODE_PREFIX[[wrap]] +#define yyalloc M4_MODE_PREFIX[[alloc]] +#define yyrealloc M4_MODE_PREFIX[[realloc]] +#define yyfree M4_MODE_PREFIX[[free]] +#define yyread M4_MODE_PREFIX[[read]] +) +]]) +]]) +]]) +#define FLEX_SCANNER +#define YY_FLEX_MAJOR_VERSION FLEX_MAJOR_VERSION +#define YY_FLEX_MINOR_VERSION FLEX_MINOR_VERSION +#define YY_FLEX_SUBMINOR_VERSION FLEX_SUBMINOR_VERSION +#if YY_FLEX_SUBMINOR_VERSION > 0 +#define FLEX_BETA +#endif + +%# Some negated symbols +m4_ifdef( [[M4_YY_IN_HEADER]], , [[m4_define([[M4_YY_NOT_IN_HEADER]], [[]])]]) +m4_ifdef( [[M4_YY_REENTRANT]], , [[m4_define([[M4_YY_NOT_REENTRANT]], [[]])]]) + +%# This is the m4 way to say "(stack_used || is_reentrant)" +m4_ifdef( [[M4_YY_STACK_USED]], [[m4_define([[M4_YY_HAS_START_STACK_VARS]])]]) +m4_ifdef( [[M4_YY_REENTRANT]], [[m4_define([[M4_YY_HAS_START_STACK_VARS]])]]) + +%# Prefixes. +%# The complexity here is necessary so that m4 preserves +%# the argument lists to each C function. + + +m4_ifdef( [[M4_MODE_PREFIX]],, [[m4_define([[M4_MODE_PREFIX]], [[yy]])]]) + +m4preproc_define(`M4_GEN_PREFIX',`` +[[#ifdef yy$1 +#define ]]M4_MODE_PREFIX[[$1_ALREADY_DEFINED +#else +#define yy$1 ]]M4_MODE_PREFIX[[$1 +#endif]] +'m4preproc_divert(1)` +[[#ifndef ]]M4_MODE_PREFIX[[$1_ALREADY_DEFINED +#undef yy$1 +#endif]]'m4preproc_divert(0)') + +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + /* The c++ scanner is a mess. The FlexLexer.h header file relies on the + * following macro. This is required in order to pass the c++-multiple-scanners + * test in the regression suite. We get reports that it breaks inheritance. + * We will address this in a future release of flex, or omit the C++ scanner + * altogether. + */ + #define yyFlexLexer M4_MODE_PREFIX[[FlexLexer]] +]]) + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifelse(M4_MODE_PREFIX,yy,, + M4_GEN_PREFIX(`_create_buffer') + M4_GEN_PREFIX(`_delete_buffer') + M4_GEN_PREFIX(`_scan_buffer') + M4_GEN_PREFIX(`_scan_string') + M4_GEN_PREFIX(`_scan_bytes') + M4_GEN_PREFIX(`_init_buffer') + M4_GEN_PREFIX(`_flush_buffer') + M4_GEN_PREFIX(`_load_buffer_state') + M4_GEN_PREFIX(`_switch_to_buffer') + M4_GEN_PREFIX(`push_buffer_state') + M4_GEN_PREFIX(`pop_buffer_state') + M4_GEN_PREFIX(`ensure_buffer_stack') + M4_GEN_PREFIX(`lex') + M4_GEN_PREFIX(`restart') + M4_GEN_PREFIX(`lex_init') + M4_GEN_PREFIX(`lex_init_extra') + M4_GEN_PREFIX(`lex_destroy') + M4_GEN_PREFIX(`get_debug') + M4_GEN_PREFIX(`set_debug') + M4_GEN_PREFIX(`get_extra') + M4_GEN_PREFIX(`set_extra') + M4_GEN_PREFIX(`get_in') + M4_GEN_PREFIX(`set_in') + M4_GEN_PREFIX(`get_out') + M4_GEN_PREFIX(`set_out') + M4_GEN_PREFIX(`get_leng') + M4_GEN_PREFIX(`get_text') + M4_GEN_PREFIX(`get_lineno') + M4_GEN_PREFIX(`set_lineno') + m4_ifdef( [[M4_YY_REENTRANT]], + [[ + M4_GEN_PREFIX(`get_column') + M4_GEN_PREFIX(`set_column') + ]]) + M4_GEN_PREFIX(`wrap') +) +]]) + +m4_ifdef( [[M4_YY_BISON_LVAL]], +[[ + M4_GEN_PREFIX(`get_lval') + M4_GEN_PREFIX(`set_lval') +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ + M4_GEN_PREFIX(`get_lloc') + M4_GEN_PREFIX(`set_lloc') +]]) + + +m4_ifelse(M4_MODE_PREFIX,yy,, + M4_GEN_PREFIX(`alloc') + M4_GEN_PREFIX(`realloc') + M4_GEN_PREFIX(`free') +) + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifelse(M4_MODE_PREFIX,yy,, +m4_ifdef( [[M4_YY_NOT_REENTRANT]], +[[ + M4_GEN_PREFIX(`text') + M4_GEN_PREFIX(`leng') + M4_GEN_PREFIX(`in') + M4_GEN_PREFIX(`out') + M4_GEN_PREFIX(`flexdebug') + M4_GEN_PREFIX(`lineno') +]]) +) +]]) + + +m4_ifdef( [[M4_MODE_TABLESEXT]], +[[ + M4_GEN_PREFIX(`tables_fload') + M4_GEN_PREFIX(`tables_destroy') + M4_GEN_PREFIX(`TABLES_NAME') +]]) + +/* First, we deal with platform-specific or compiler-specific issues. */ + +/* begin standard C headers. */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_YY_ALWAYS_INTERACTIVE]], , +[[m4_ifdef( [[M4_YY_NEVER_INTERACTIVE]], , +[[/* Feature test macros. Flex uses functions that require a minimum set of + * macros defined. As defining some macros may hide function declarations that + * user code might use, be conservative and respect user's definitions as much + * as possible. In glibc, feature test macros may not be all set up until one + * of the libc header (that includes <features.h>) is included. This creates + * a circular dependency when we check the macros. <assert.h> is the safest + * header we can include and does not declare too many functions we don't need. + */ +#if !defined(__GNU_LIBRARY__) && defined(__STDC__) +#include <assert.h> +#endif +#if !(defined(_POSIX_C_SOURCE) || defined(_XOPEN_SOURCE) || \ + defined(_POSIX_SOURCE)) +# define _POSIX_C_SOURCE 1 /* Required for fileno() */ +# define _POSIX_SOURCE 1 +#endif]])]]) +#include <stdio.h> +#include <string.h> +#include <errno.h> +#include <stdlib.h> +]]) + +m4_ifdef([[M4_MODE_TABLESEXT]], [[ +#include <sys/types.h> +#include <netinet/in.h> +]]) +/* end standard C headers. */ + +/* begin standard C++ headers. */ +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +#include <iostream> +#include <errno.h> +#include <cstdlib> +#include <cstdio> +#include <cstring> +/* end standard C++ headers. */ +]]) + +m4preproc_include(`flexint_shared.h') + +/* TODO: this is always defined, so inline it */ +#define yyconst const + +#if defined(__GNUC__) && __GNUC__ >= 3 +#define yynoreturn __attribute__((__noreturn__)) +#else +#define yynoreturn +#endif + +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +/* Returned upon end-of-file. */ +#define YY_NULL 0 +]]) + +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +/* Promotes a possibly negative, possibly signed char to an + * integer in range [0..255] for use as an array index. + */ +#define YY_SC_TO_UI(c) ((YY_CHAR) (c)) +]]) + + + +m4_ifdef([[M4_YY_REENTRANT]], [[ + +/* An opaque pointer. */ +#ifndef YY_TYPEDEF_YY_SCANNER_T +#define YY_TYPEDEF_YY_SCANNER_T +typedef void* yyscan_t; +#endif + +%# Declare yyguts variable +m4_define( [[M4_YY_DECL_GUTS_VAR]], [[struct yyguts_t * yyg = (struct yyguts_t*)yyscanner]]) +%# Perform a noop access on yyguts to prevent unused variable complains +m4_define( [[M4_YY_NOOP_GUTS_VAR]], [[(void)yyg]]) +%# For use wherever a Global is accessed or assigned. +m4_define( [[YY_G]], [[yyg->$1]]) + +%# For use in function prototypes to append the additional argument. +m4_define( [[M4_YY_PROTO_LAST_ARG]], [[, yyscan_t yyscanner]]) +m4_define( [[M4_YY_PROTO_ONLY_ARG]], [[yyscan_t yyscanner]]) + +m4_define( [[M4_YY_DEF_LAST_ARG]], [[, yyscan_t yyscanner]]) +m4_define( [[M4_YY_DEF_ONLY_ARG]], [[yyscan_t yyscanner]]) +m4_define( [[M4_YY_DECL_LAST_ARG]], [[yyscan_t yyscanner;]]) + +%# For use in function calls to pass the additional argument. +m4_define( [[M4_YY_CALL_LAST_ARG]], [[, yyscanner]]) +m4_define( [[M4_YY_CALL_ONLY_ARG]], [[yyscanner]]) + +%# For use in function documentation to adjust for additional argument. +m4_define( [[M4_YY_DOC_PARAM]], [[@param yyscanner The scanner object.]]) + +/* For convenience, these vars (plus the bison vars far below) + are macros in the reentrant scanner. */ +#define yyin YY_G(yyin_r) +#define yyout YY_G(yyout_r) +#define yyextra YY_G(yyextra_r) +#define yyleng YY_G(yyleng_r) +#define yytext YY_G(yytext_r) +#define yylineno (YY_CURRENT_BUFFER_LVALUE->yy_bs_lineno) +#define yycolumn (YY_CURRENT_BUFFER_LVALUE->yy_bs_column) +#define yyflexdebug YY_G(yyflexdebug_r) + +m4_define( [[M4_YY_INCR_LINENO]], +[[ + do{ yylineno++; + yycolumn=0; + }while(0) +]]) + +]]) + + + +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ + +m4_define( [[M4_YY_INCR_LINENO]], +[[ + yylineno++; +]]) + +%# Define these macros to be no-ops. +m4_define( [[M4_YY_DECL_GUTS_VAR]], [[m4_dnl]]) +m4_define( [[M4_YY_NOOP_GUTS_VAR]], [[m4_dnl]]) +m4_define( [[YY_G]], [[($1)]]) +m4_define( [[M4_YY_PROTO_LAST_ARG]]) +m4_define( [[M4_YY_PROTO_ONLY_ARG]], [[void]]) +m4_define( [[M4_YY_DEF_LAST_ARG]]) + +m4_define( [[M4_YY_DEF_ONLY_ARG]], [[void]]) +m4_define([[M4_YY_DECL_LAST_ARG]]) +m4_define([[M4_YY_CALL_LAST_ARG]]) +m4_define([[M4_YY_CALL_ONLY_ARG]]) +m4_define( [[M4_YY_DOC_PARAM]], ) + +]]) + + +%# Generate C99 function defs. +m4_define( [[YYFARGS1]], [[($1 $2 M4_YY_DEF_LAST_ARG)]]) +m4_define( [[YYFARGS2]], [[($1 $2, $3 $4 M4_YY_DEF_LAST_ARG)]]) +m4_define( [[YYFARGS3]], [[($1 $2, $3 $4, $5 $6 M4_YY_DEF_LAST_ARG)]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* Enter a start condition. This macro really ought to take a parameter, + * but we do it the disgusting crufty way forced on us by the ()-less + * definition of BEGIN. + */ +#define yybegin(s) YY_G(yy_start) = 1 + 2 * (s) +/* Legacy interface */ +#define BEGIN YY_G(yy_start) = 1 + 2 * +/* Translate the current start state into a value that can be later handed + * to BEGIN to return to the state. The YYSTATE alias is for lex + * compatibility. + */ +#define yystart() ((YY_G(yy_start) - 1) / 2) +/* Legacy interfaces */ +#define YY_START ((YY_G(yy_start) - 1) / 2) +#define YYSTATE YY_START +/* Action number for EOF rule of a given start state. */ +#define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1) +/* Special action meaning "start processing a new file". */ +#define YY_NEW_FILE yyrestart( yyin M4_YY_CALL_LAST_ARG ) +#define YY_END_OF_BUFFER_CHAR 0 +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* The state buf must be large enough to hold one state per character in the main buffer. + */ +#define YY_STATE_BUF_SIZE ((YY_BUF_SIZE + 2) * sizeof(yy_state_type)) +]]) + + +#ifndef YY_TYPEDEF_YY_BUFFER_STATE +#define YY_TYPEDEF_YY_BUFFER_STATE +typedef struct yy_buffer_state *yybuffer; +/* Legacy interface */ +typedef struct yy_buffer_state *YY_BUFFER_STATE; +#endif + +#ifndef YY_TYPEDEF_YY_SIZE_T +#define YY_TYPEDEF_YY_SIZE_T +typedef size_t yy_size_t; +#endif + +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +extern int yyleng; +]]) + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +extern FILE *yyin, *yyout; +]]) +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +#define EOB_ACT_CONTINUE_SCAN 0 +#define EOB_ACT_END_OF_FILE 1 +#define EOB_ACT_LAST_MATCH 2 + m4_ifdef( [[M4_MODE_YYLINENO]], + [[ + /* Note: We specifically omit the test for yy_rule_can_match_eol because it requires + * access to the local variable yy_act. Since yyless() is a macro, it would break + * existing scanners that call yyless() from OUTSIDE yylex. + * One obvious solution it to make yy_act a global. I tried that, and saw + * a 5% performance hit in a non-yylineno scanner, because yy_act is + * normally declared as a register variable-- so it is not worth it. + */ + #define YY_LESS_LINENO(n) \ + do { \ + int yyl;\ + for ( yyl = n; yyl < yyleng; ++yyl ) { \ + if ( yytext[yyl] == '\n' ) { \ + --yylineno;\ + } \ + } \ + }while(0) + #define YY_LINENO_REWIND_TO(dst) \ + do {\ + const char *p;\ + for ( p = yy_cp-1; p >= (dst); --p) { \ + if ( *p == '\n' ) { \ + --yylineno;\ + } \ + } \ + }while(0) + ]], + [[ + #define YY_LESS_LINENO(n) + #define YY_LINENO_REWIND_TO(ptr) + ]]) +/* Return all but the first "n" matched characters back to the input stream. */ +#define yyless(n) \ + do \ + { \ + /* Undo effects of setting up yytext. */ \ + int yyless_macro_arg = (n); \ + YY_LESS_LINENO(yyless_macro_arg);\ + *yy_cp = YY_G(yy_hold_char); \ + YY_RESTORE_YY_MORE_OFFSET \ + YY_G(yy_c_buf_p) = yy_cp = yy_bp + yyless_macro_arg - YY_MORE_ADJ; \ + YY_DO_BEFORE_ACTION; /* set up yytext again */ \ + } \ + while ( 0 ) +#define yyunput(c) yyunput_r( c, YY_G(yytext_ptr) M4_YY_CALL_LAST_ARG ) +/* Legacy interface */ +#define unput(c) yyunput_r( c, YY_G(yytext_ptr) M4_YY_CALL_LAST_ARG ) +]]) + +#ifndef YY_STRUCT_YY_BUFFER_STATE +#define YY_STRUCT_YY_BUFFER_STATE +struct yy_buffer_state + { +m4_ifdef([[M4_MODE_C_ONLY]], [[ + FILE *yy_input_file; +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + std::streambuf* yy_input_file; +]]) + char *yy_ch_buf; /* input buffer */ + char *yy_buf_pos; /* current position in input buffer */ + + /* Size of input buffer in bytes, not including room for EOB + * characters. + */ + int yy_buf_size; + + /* Number of characters read into yy_ch_buf, not including EOB + * characters. + */ + int yy_n_chars; + + /* Whether we "own" the buffer - i.e., we know we created it, + * and can realloc() it to grow it, and should free() it to + * delete it. + */ + int yy_is_our_buffer; + + /* Whether this is an "interactive" input source; if so, and + * if we're using stdio for input, then we want to use getc() + * instead of fread(), to make sure we stop fetching input after + * each newline. + */ + int yy_is_interactive; + + /* Whether we're considered to be at the beginning of a line. + * If so, '^' rules will be active on the next match, otherwise + * not. + */ + int yyatbol; + + int yy_bs_lineno; /**< The line count. */ + int yy_bs_column; /**< The column count. */ + + /* Whether to try to fill the input buffer when we reach the + * end of it. + */ + int yy_fill_buffer; + + int yy_buffer_status; +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +#define YY_BUFFER_NEW 0 +#define YY_BUFFER_NORMAL 1 + /* When an EOF's been seen but there's still some text to process + * then we mark the buffer as YY_EOF_PENDING, to indicate that we + * shouldn't try reading from the input source any more. We might + * still have a bunch of tokens to match, though, because of + * possible backing-up. + * + * When we actually see the EOF, we change the status to "new" + * (via yyrestart()), so that the user can continue scanning by + * just pointing yyin at a new input file. + */ +#define YY_BUFFER_EOF_PENDING 2 +]]) + }; +#endif /* !YY_STRUCT_YY_BUFFER_STATE */ + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# Standard (non-C++) definition +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ + +/* Stack of input buffers. */ +static size_t yy_buffer_stack_top = 0; /**< index of top of stack. */ +static size_t yy_buffer_stack_max = 0; /**< capacity of stack. */ +static yybuffer * yy_buffer_stack = NULL; /**< Stack as an array. */ +]]) +]]) +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* We provide macros for accessing buffer states in case in the + * future we want to put the buffer states in a more general + * "scanner state". + * + * Returns the top of the stack, or NULL. + */ +#define yy_current_buffer() ( YY_G(yy_buffer_stack) \ + ? YY_G(yy_buffer_stack)[YY_G(yy_buffer_stack_top)] \ + : NULL) +/* Legacy interface */ +#define YY_CURRENT_BUFFER yy_current_buffer() +/* Same as previous macro, but useful when we know that the buffer stack is not + * NULL or when we need an lvalue. For internal use only. + */ +#define YY_CURRENT_BUFFER_LVALUE YY_G(yy_buffer_stack)[YY_G(yy_buffer_stack_top)] +]]) + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# Standard (non-C++) definition + +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +/* yy_hold_char holds the character lost when yytext is formed. */ +static char yy_hold_char; +static int yy_n_chars; /* number of characters read into yy_ch_buf */ +int yyleng; + +/* Points to current character in buffer. */ +static char *yy_c_buf_p = NULL; +static int yy_init = 0; /* whether we need to initialize */ +static int yy_start = 0; /* start state number */ + +/* Flag which is used to allow yywrap()'s to do buffer switches + * instead of setting up a fresh yyin. A bit of a hack ... + */ +static int yy_did_buffer_switch_on_eof; +]]) +]]) + +void yyrestart ( FILE *input_file M4_YY_PROTO_LAST_ARG ); +void yy_switch_to_buffer ( yybuffer new_buffer M4_YY_PROTO_LAST_ARG ); +yybuffer yy_create_buffer ( FILE *file, int size M4_YY_PROTO_LAST_ARG ); +void yy_delete_buffer ( yybuffer b M4_YY_PROTO_LAST_ARG ); +void yy_flush_buffer ( yybuffer b M4_YY_PROTO_LAST_ARG ); +void yypush_buffer_state ( yybuffer new_buffer M4_YY_PROTO_LAST_ARG ); +void yypop_buffer_state ( M4_YY_PROTO_ONLY_ARG ); + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +static void yyensure_buffer_stack ( M4_YY_PROTO_ONLY_ARG ); +static void yy_load_buffer_state ( M4_YY_PROTO_ONLY_ARG ); +static void yy_init_buffer ( yybuffer b, FILE *file M4_YY_PROTO_LAST_ARG ); +#define yy_flush_current_buffer() yy_flush_buffer( yy_current_buffer() M4_YY_CALL_LAST_ARG) +#define YY_FLUSH_BUFFER yy_flush_current_buffer() +]]) + +yybuffer yy_scan_buffer ( char *base, yy_size_t size M4_YY_PROTO_LAST_ARG ); +yybuffer yy_scan_string ( const char *yy_str M4_YY_PROTO_LAST_ARG ); +yybuffer yy_scan_bytes ( const char *bytes, int len M4_YY_PROTO_LAST_ARG ); + +]]) + +void *yyalloc ( yy_size_t M4_YY_PROTO_LAST_ARG ); +void *yyrealloc ( void *, yy_size_t M4_YY_PROTO_LAST_ARG ); +void yyfree ( void * M4_YY_PROTO_LAST_ARG ); + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +#define yy_new_buffer yy_create_buffer +#define yy_set_interactive(is_interactive) { \ + if ( yy_current_buffer() == NULL ) { \ + yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); \ + YY_CURRENT_BUFFER_LVALUE = \ + yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); \ + } \ + YY_CURRENT_BUFFER_LVALUE->yy_is_interactive = is_interactive; \ +} +#define yysetbol(at_bol) \ + { \ + if ( yy_current_buffer() == NULL ) { \ + yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); \ + YY_CURRENT_BUFFER_LVALUE = \ + yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); \ + } \ + YY_CURRENT_BUFFER_LVALUE->yyatbol = at_bol; \ +} +#define yyatbol() (YY_CURRENT_BUFFER_LVALUE->yyatbol) +/* Legacy interface */ +#define YY_AT_BOL() (YY_CURRENT_BUFFER_LVALUE->yyatbol) +#define yy_set_bol(at_bol) \ + { \ + if ( yy_current_buffer() == NULL ) { \ + yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); \ + YY_CURRENT_BUFFER_LVALUE = \ + yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); \ + } \ + YY_CURRENT_BUFFER_LVALUE->yyatbol = at_bol; \ +} +]]) + +/* Begin user sect3 */ + +m4_ifdef( [[M4_MODE_NO_YYWRAP]], [[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef([[M4_YY_REENTRANT]], [[ +#define M4_MODE_PREFIX[[wrap]](yyscanner) (/*CONSTCOND*/1) +]]) +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +#define M4_MODE_PREFIX[[wrap]]() (/*CONSTCOND*/1) +]])m4_dnl +]]) +#define YY_SKIP_YYWRAP +]]) +m4_ifdef( [[M4_MODE_DEBUG]], [[ +#define FLEX_DEBUG +]]) +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], [[m4_dnl +typedef flex_uint8_t YY_CHAR; +]])m4_dnl +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_define([[M4_TMP_STDINIT]], [[FILE *yyin = stdin, *yyout = stdout;]]) +m4_define([[M4_TMP_NO_STDINIT]], [[FILE *yyin = NULL, *yyout = NULL;]]) +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], [[ +m4_ifdef( [[M4_MODE_DO_STDINIT]], [[m4_dnl +m4_ifdef([[M4_YY_REENTRANT]], [[ +#ifdef VMS +#ifdef __VMS_POSIX +#define YY_STDINIT +#endif +#else +#define YY_STDINIT +#endif +]]) +#ifdef VMS"); +#ifndef __VMS_POSIX +M4_TMP_NO_STDINIT +#else +M4_TMP_STDINIT +#endif +#else +M4_TMP_STDINIT +#endif +]]) +m4_ifdef( [[M4_MODE_NO_DO_STDINIT]], [[ +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +M4_TMP_NO_STDINIT +]]) +]]) +]]) +m4_undefine([[M4_TMP_STDINIT]]) +m4_undefine([[M4_TMP_NO_STDINIT]]) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +#define yytext_ptr yytext +m4_ifdef( [[M4_MODE_INTERACTIVE]], [[#define YY_INTERACTIVE]]) +]]) +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], [[m4_dnl +m4_ifdef( [[M4_MODE_FULLSPD]], [[m4_dnl +typedef const struct yy_trans_info *yy_state_type; +]], [[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +typedef int yy_state_type; +]]) +]]) +]]) +m4_ifdef([[M4_MODE_LEX_COMPAT]], [[#define YY_FLEX_LEX_COMPAT]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +extern int yylineno; +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +int yylineno = 1; +]]) +]]) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + +#include <FlexLexer.h> +m4_ifdef([[M4_MODE_NO_YYWRAP]], [[ +int yyFlexLexer::yywrap() { return 1;} +]]) +m4_ifdef([[M4_MODE_YYCLASS]], [[ +int yyFlexLexer::yylex() + { + LexerError( "yyFlexLexer::yylex invoked but %option yyclass used" ); + return 0; + } + +#define YY_DECL int M4_YY_CLASS_NAME::yylex() +]]) +]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ +/* Watch out: yytext_ptr is a variable when yytext is an array, + * but it's a macro when yytext is a pointer. + */ +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[ +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[extern char yytext[];]]) +]]) +m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +m4_ifdef([[M4_YY_REENTRANT]], [[#define yytext_ptr yytext_r]],[[ +extern char *yytext; + +#ifdef yytext_ptr +#undef yytext_ptr +#endif +#define yytext_ptr yytext +]]) +]]) +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +%% [1.5] DFA +]]) + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# Standard (non-C++) definition + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +static yy_state_type yy_get_previous_state ( M4_YY_PROTO_ONLY_ARG ); +static yy_state_type yy_try_NUL_trans ( yy_state_type current_state M4_YY_PROTO_LAST_ARG); +static int yy_get_next_buffer ( M4_YY_PROTO_ONLY_ARG ); +static void yynoreturn yypanic ( const char* msg M4_YY_PROTO_LAST_ARG ); +]]) + +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], [[ +struct yy_trans_info + { + /* We require that yy_verify and yy_nxt must be of the same size int. */ +m4_ifdef([[M4_MODE_REAL_FULLSPD]], [[ + YY_OFFSET_TYPE yy_verify; + + /* In cases where its sister yy_verify *is* a "yes, there is + * a transition", yy_nxt is the offset (in records) to the + * next state. In most cases where there is no transition, + * the value of yy_nxt is irrelevant. If yy_nxt is the -1th + * record of a state, though, then yy_nxt is the action number + * for that state. + */ + YY_OFFSET_TYPE yy_nxt; +]]) +m4_ifdef([[M4_MODE_NO_REAL_FULLSPD]], [[ + /* We generate a bogus 'struct yy_trans_info' data type + * so we can guarantee that it is always declared in the skel. + * This is so we can compile "sizeof(struct yy_trans_info)" + * in any scanner. + */ + flex_int32_t yy_verify; + flex_int32_t yy_nxt; +]]) + }; +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* Done after the current pattern has been matched and before the + * corresponding action - sets up yytext. + */ +#define YY_DO_BEFORE_ACTION \ + YY_G(yytext_ptr) = yy_bp; \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[YY_G(yytext_ptr) -= YY_G(yy_more_len); \ + yyleng = (int) (yy_cp - YY_G(yytext_ptr));]])]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[yyleng = (int) (yy_cp - yy_bp);]]) \ + m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[yyleng = (int) (yy_cp - yy_bp);]]) \ + YY_G(yy_hold_char) = *yy_cp; \ + *yy_cp = '\0'; \ +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[ \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[if ( yyleng + YY_G(yy_more_offset) >= YYLMAX ) \ + YY_FATAL_ERROR( "token too large, exceeds YYLMAX" );]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[if ( yyleng >= YYLMAX ) \ + YY_FATAL_ERROR( "token too large, exceeds YYLMAX" );]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yy_flex_strncpy( &yytext[YY_G(yy_more_offset)], YY_G(yytext_ptr), yyleng + 1 M4_YY_CALL_LAST_ARG);]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yyleng += YY_G(yy_more_offset);]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[YY_G(yy_prev_more_offset) = YY_G(yy_more_offset);]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[YY_G(yy_more_offset) = 0;]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[yy_flex_strncpy( yytext, YY_G(yytext_ptr), yyleng + 1 M4_YY_CALL_LAST_ARG);]]) \ +]]) \ + YY_G(yy_c_buf_p) = yy_cp; + +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ +m4_ifdef( [[M4_MODE_C_ONLY]], +[[ +extern int yyflexdebug; +int yyflexdebug = m4_ifdef([[M4_MODE_DEBUG]],[[1]],[[0]]); +/* Legacy interface */ +#ifndef yy_flex_debug +#define yy_flex_debug yyflexdebug +#endif +]]) +]]) + +%% [2.0] data tables for the DFA are inserted here +m4_ifdef( [[M4_HOOK_NXT_ROWS]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_NXT_TYPE yy_nxt[][M4_HOOK_NXT_ROWS] = +M4_HOOK_NXT_BODY +]], [[ +#undef YY_NXT_LOLEN +#define YY_NXT_LOLEN ([[]]M4_HOOK_NXT_ROWS[[]]) +static const M4_HOOK_NXT_TYPE *yy_nxt =0; +]]) + +]]) + +m4_ifdef( [[M4_MODE_YYLINENO]],[[m4_dnl +/* Table of booleans, true if rule could match eol. */ +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_EOLTABLE_TYPE yy_rule_can_match_eol[M4_HOOK_EOLTABLE_SIZE] = { 0, +M4_HOOK_EOLTABLE_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_EOLTABLE_TYPE * yy_rule_can_match_eol = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_NEED_ACCEPT]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_ACCEPT_TYPE yy_accept[M4_HOOK_ACCEPT_SIZE] = { 0, +M4_HOOK_ACCEPT_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_ACCEPT_TYPE * yy_accept = 0; +]]) + +]]) + +m4_ifdef( [[M4_MODE_USEECS]],[[m4_dnl +/* Character equivalence-class mapping */ +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const YY_CHAR yy_ec[M4_HOOK_ECSTABLE_SIZE] = { 0, +M4_HOOK_ECSTABLE_BODY[[]]m4_dnl +}; +]], [[ +static const YY_CHAR * yy_ec = 0; +]]) + +]]) + +m4_ifdef( [[M4_MODE_USEMECS]],[[m4_dnl +/* Character meta-equivalence-class mappings */ +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const YY_CHAR yy_meta[M4_HOOK_MECSTABLE_SIZE] = { 0, +M4_HOOK_MECSTABLE_BODY[[]]m4_dnl +}; +]], [[ +static const YY_CHAR * yy_meta = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_TRANSTABLE_SIZE]],[[m4_dnl +/* The transition table */ +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const struct yy_trans_info yy_transition[M4_HOOK_TRANSTABLE_SIZE] = { +M4_HOOK_TRANSTABLE_BODY[[]]m4_dnl +}; +]], [[ +static const struct yy_trans_info *yy_transition = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_STARTTABLE_SIZE]],[[m4_dnl +/* Table of pointers to start states. */ +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const struct yy_trans_info *yy_start_state_list[M4_HOOK_STARTTABLE_SIZE] = { +M4_HOOK_STARTTABLE_BODY[[]]m4_dnl +}; +]], [[ +static const struct yy_trans_info **yy_start_state_list =0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_ACCLIST_TYPE]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_ACCLIST_TYPE yy_acclist[M4_HOOK_ACCLIST_SIZE] = { 0, +M4_HOOK_ACCLIST_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_ACCLIST_TYPE * yy_acclist = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_BASE_TYPE]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_BASE_TYPE yy_base[M4_HOOK_BASE_SIZE] = { 0, +M4_HOOK_BASE_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_BASE_TYPE * yy_base = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_DEF_TYPE]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_DEF_TYPE yy_def[M4_HOOK_DEF_SIZE] = { 0, +M4_HOOK_DEF_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_DEF_TYPE * yy_def = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_YYNXT_TYPE]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_YYNXT_TYPE yy_nxt[M4_HOOK_YYNXT_SIZE] = { 0, +M4_HOOK_YYNXT_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_YYNXT_TYPE * yy_nxt = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_CHK_TYPE]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_CHK_TYPE yy_chk[M4_HOOK_CHK_SIZE] = { 0, +M4_HOOK_CHK_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_CHK_TYPE * yy_chk = 0; +]]) + +]]) + +m4_ifdef( [[M4_HOOK_NULTRANS_SIZE]],[[m4_dnl +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const yy_state_type yy_NUL_trans[M4_HOOK_NULTRANS_SIZE] = { 0, +M4_HOOK_NULTRANS_BODY[[]]m4_dnl +}; +]], [[ +static const yy_state_type * yy_NUL_trans = 0; +]]) + +]]) + +m4_ifdef( [[M4_MODE_DEBUG]],[[m4_dnl +/* Rule to line-number mapping */ +m4_ifdef( [[M4_MODE_GENTABLES]],[[m4_dnl +static const M4_HOOK_DEBUGTABLE_TYPE yy_rule_linenum[M4_HOOK_DEBUGTABLE_SIZE] = { 0, +M4_HOOK_DEBUGTABLE_BODY[[]]m4_dnl +}; +]], [[ +static const M4_HOOK_DEBUGTABLE_TYPE * yy_rule_linenum = 0; +]]) + +]]) + +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +m4_ifdef( [[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_MODE_NO_USES_REJECT]],[[ +m4_ifdef( [[M4_YY_NOT_REENTRANT]], [[ +/* Definitions for backing up. We don't need them if yyreject() + * is being used because then we use an alternative backing-up + * technique instead. + */ +static yy_state_type yy_last_accepting_state; +static char *yy_last_accepting_cpos; + +]]) +]]) +]]) + +m4_ifdef( [[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[m4_dnl +%# These must match the values in the file flexdef.h +%# of the flex source code, otherwise havoc will ensue. +#define YY_TRAILING_MASK 0x2000 +#define YY_TRAILING_HEAD_MASK 0x4000 +]]) +m4_ifdef( [[M4_MODE_USES_REJECT]],[[ +m4_ifdef( [[M4_YY_NOT_REENTRANT]], [[ +m4_ifdef( [[M4_MODE_C_ONLY]], [[ +/* Declare state buffer variables. */ +static yy_state_type *yy_state_buf=0, *yy_state_ptr=0; +static char *yy_full_match; +static int yy_lp; +m4_ifdef( [[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[ +static int yy_looking_for_trail_begin = 0; +static int yy_full_lp; +static int *yy_full_state; +]]) +]]) +]]) +#define yyreject() \ +{ \ +*yy_cp = YY_G(yy_hold_char); /* undo effects of setting up yytext */ \ +yy_cp = YY_G(yy_full_match); /* restore poss. backed-over text */ \ +m4_ifdef( [[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[ \ +YY_G(yy_lp) = YY_G(yy_full_lp); /* restore orig. accepting pos. */ \ +YY_G(yy_state_ptr) = YY_G(yy_full_state); /* restore orig. state */ \ +yy_current_state = *YY_G(yy_state_ptr); /* restore curr. state */ \ +]]) \ +++YY_G(yy_lp); \ +goto find_rule; \ +} +#define REJECT yyreject() +]]) +m4_ifdef( [[M4_MODE_NO_USES_REJECT]],[[ +/* The intent behind this definition is that it'll catch + * any uses of yyreject() which flex missed. + */ +#define yyreject() reject_used_but_not_detected +#define REJECT reject_used_but_not_detected +]]) + +m4_ifdef([[M4_MODE_YYMORE_USED]], [[ +m4_ifdef( [[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_MODE_YYTEXT_IS_ARRAY]], [[ +m4_ifdef( [[M4_YY_NOT_REENTRANT]], [[ +static int yy_more_offset = 0; +static int yy_prev_more_offset = 0; +]]) +]]) +m4_ifdef( [[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +m4_ifdef( [[M4_YY_NOT_REENTRANT]], [[ +static int yy_more_flag = 0; +static int yy_more_len = 0; +]]) +]]) +]]) +m4_ifdef( [[M4_MODE_YYTEXT_IS_ARRAY]], [[ +#define yymore() (YY_G(yy_more_offset) = yy_flex_strlen( yytext M4_YY_CALL_LAST_ARG)) +#define YY_NEED_STRLEN +#define YY_MORE_ADJ 0 +#define YY_RESTORE_YY_MORE_OFFSET \ +{ \ +YY_G(yy_more_offset) = YY_G(yy_prev_more_offset); \ +yyleng -= YY_G(yy_more_offset); \ +} +]]) +m4_ifdef( [[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +#define yymore() (YY_G(yy_more_flag) = 1) +#define YY_MORE_ADJ YY_G(yy_more_len) +#define YY_RESTORE_YY_MORE_OFFSET +]]) +]]) + +m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[ +#define yymore() yymore_used_but_not_detected +#define YY_MORE_ADJ 0 +#define YY_RESTORE_YY_MORE_OFFSET +]]) + +m4_ifdef( [[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_MODE_YYTEXT_IS_ARRAY]], [[ +m4_ifdef( [[M4_YY_NOT_REENTRANT]], [[ +char yytext[YYLMAX]; +char *yytext_ptr; +]]) +]]) +m4_ifdef( [[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +m4_ifdef( [[M4_YY_NOT_REENTRANT]], [[ +char *yytext; +]]) +]]) +]]) + +%% [3.0] static declarations conditional on mode switches go here +]]) + +m4_ifdef( [[M4_YY_IN_HEADER]], [[#ifdef YY_HEADER_EXPORT_START_CONDITIONS]]) +M4_YY_SC_DEFS +m4_ifdef( [[M4_YY_IN_HEADER]], [[#endif]]) + +m4_ifdef( [[M4_YY_NO_UNISTD_H]],, +[[ +#ifndef YY_NO_UNISTD_H +/* Special case for "unistd.h", since it is non-ANSI. We include it way + * down here because we want the user's section 1 to have been scanned first. + * The user has a chance to override it with an option. + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +#include <unistd.h> +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +#include <unistd.h> +]]) +#endif +]]) + +m4_ifdef( [[M4_EXTRA_TYPE_DEFS]], +[[ +#define YY_EXTRA_TYPE M4_EXTRA_TYPE_DEFS +]], +[[ +#ifndef YY_EXTRA_TYPE +#define YY_EXTRA_TYPE void * +#endif +]] +) + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# Reentrant structure and macros (non-C++). +m4_ifdef([[M4_YY_REENTRANT]], [[ + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* Holds the entire state of the reentrant scanner. */ +struct yyguts_t { + /* User-defined. Not touched by flex. */ + YY_EXTRA_TYPE yyextra_r; + + /* The rest are the same as the globals declared in the non-reentrant scanner. */ + FILE *yyin_r, *yyout_r; + size_t yy_buffer_stack_top; /**< index of top of stack. */ + size_t yy_buffer_stack_max; /**< capacity of stack. */ + yybuffer * yy_buffer_stack; /**< Stack as an array. */ + char yy_hold_char; + int yy_n_chars; + int yyleng_r; + char *yy_c_buf_p; + int yy_init; + int yy_start; + int yy_did_buffer_switch_on_eof; + int yy_start_stack_ptr; + int yy_start_stack_depth; + int *yy_start_stack; + yy_state_type yy_last_accepting_state; + char* yy_last_accepting_cpos; + + int yylineno_r; + int yyflexdebug_r; + +m4_ifdef( [[M4_MODE_USES_REJECT]], [[ + yy_state_type *yy_state_buf; + yy_state_type *yy_state_ptr; + char *yy_full_match; + int yy_lp; + +m4_ifdef( [[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[m4_dnl + /* These are only needed for trailing context rules */ + int yy_looking_for_trail_begin; + int yy_full_lp; + int *yy_full_state; +]]) +]]) +m4_ifdef( [[M4_MODE_REENTRANT_TEXT_IS_ARRAY]], [[ + char yytext_r[YYLMAX]; + char *yytext_ptr; + int yy_more_offset; + int yy_prev_more_offset; +]], [[ + char *yytext_r; + int yy_more_flag; + int yy_more_len; +]]) +m4_ifdef( [[M4_YY_BISON_LVAL]], [[ + YYSTYPE * yylval_r; +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], [[ + YYLTYPE * yylloc_r; +]]) +}; /* end struct yyguts_t */ +]]) + + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +static int yy_init_globals ( M4_YY_PROTO_ONLY_ARG ); +]]) +]]) + +m4_ifdef([[M4_YY_REENTRANT]], [[ + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ + m4_ifdef( [[M4_YY_BISON_LVAL]], + [[ + /* This must go here because YYSTYPE and YYLTYPE are included + * from bison output in section 1.*/ + # define yylval YY_G(yylval_r) + ]]) + + m4_ifdef( [[<M4_YY_BISON_LLOC>]], + [[ + # define yylloc YY_G(yylloc_r) + ]]) +]]) + +int yylex_init (yyscan_t* scanner); + +int yylex_init_extra ( YY_EXTRA_TYPE user_defined, yyscan_t* scanner); + +]]) + +]]) + +/* Accessor methods to globals. + These are made visible to non-reentrant scanners for convenience. */ + +m4_ifdef( [[M4_YY_NO_DESTROY]],, +[[ +int yylex_destroy ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_GET_DEBUG]],, +[[ +int yyget_debug ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_SET_DEBUG]],, +[[ +void yyset_debug ( int debug_flag M4_YY_PROTO_LAST_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_GET_EXTRA]],, +[[ +YY_EXTRA_TYPE yyget_extra ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_SET_EXTRA]],, +[[ +void yyset_extra ( YY_EXTRA_TYPE user_defined M4_YY_PROTO_LAST_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_GET_IN]],, +[[ +FILE *yyget_in ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_SET_IN]],, +[[ +void yyset_in ( FILE * _in_str M4_YY_PROTO_LAST_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_GET_OUT]],, +[[ +FILE *yyget_out ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_SET_OUT]],, +[[ +void yyset_out ( FILE * _out_str M4_YY_PROTO_LAST_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_GET_LENG]],, +[[ + int yyget_leng ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_GET_TEXT]],, +[[ +char *yyget_text ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_GET_LINENO]],, +[[ +int yyget_lineno ( M4_YY_PROTO_ONLY_ARG ); +]]) + +m4_ifdef( [[M4_YY_NO_SET_LINENO]],, +[[ +void yyset_lineno ( int _line_number M4_YY_PROTO_LAST_ARG ); +]]) + +m4_ifdef( [[M4_YY_REENTRANT]], +[[ +m4_ifdef( [[M4_YY_NO_GET_COLUMN]],, +[[ +int yyget_column ( M4_YY_PROTO_ONLY_ARG ); +]]) +]]) + +m4_ifdef( [[M4_YY_REENTRANT]], +[[ +m4_ifdef( [[M4_YY_NO_SET_COLUMN]],, +[[ +void yyset_column ( int _column_no M4_YY_PROTO_LAST_ARG ); +]]) +]]) + +m4_ifdef([[M4_YY_BISON_LVAL]], [[ +m4_ifdef( [[M4_YY_NO_GET_LVAL]],, +[[ +YYSTYPE * yyget_lval ( M4_YY_PROTO_ONLY_ARG ); +]]) + +void yyset_lval ( YYSTYPE * yylval_param M4_YY_PROTO_LAST_ARG ); + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ + m4_ifdef( [[M4_YY_NO_GET_LLOC]],, + [[ + YYLTYPE *yyget_lloc ( M4_YY_PROTO_ONLY_ARG ); + ]]) + + m4_ifdef( [[M4_YY_NO_SET_LLOC]],, + [[ + void yyset_lloc ( YYLTYPE * yylloc_param M4_YY_PROTO_LAST_ARG ); + ]]) +]]) +]]) + +/* Macros after this point can all be overridden by user definitions in + * section 1. + */ + +#ifndef YY_SKIP_YYWRAP +#ifdef __cplusplus +extern "C" int yywrap ( M4_YY_PROTO_ONLY_ARG ); +#else +extern int yywrap ( M4_YY_PROTO_ONLY_ARG ); +#endif +#endif + +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +#ifndef YY_NO_YYUNPUT + m4_ifdef( [[M4_YY_NO_YYUNPUT]],, + [[ + static void yyunput_r ( int c, char *buf_ptr M4_YY_PROTO_LAST_ARG); + ]]) +#endif +]]) +]]) + +#ifndef yytext_ptr +static void yy_flex_strncpy ( char *, const char *, int M4_YY_PROTO_LAST_ARG); +#endif + +#ifdef YY_NEED_STRLEN +static int yy_flex_strlen ( const char * M4_YY_PROTO_LAST_ARG); +#endif + +m4_ifdef([[M4_MODE_NO_YYINPUT]], [[#define YY_NO_YYINPUT 1]]) + +#ifndef YY_NO_YYINPUT +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +static int yyinput ( M4_YY_PROTO_ONLY_ARG ); +#ifndef __cplusplus +#define input yyinput +#endif +]]) +]]) +#endif + + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# TODO: This is messy. +m4_ifdef( [[M4_YY_STACK_USED]], +[[ + +m4_ifdef( [[M4_YY_NOT_REENTRANT]], +[[ + m4_ifdef( [[M4_YY_NOT_IN_HEADER]], + [[ + static int yy_start_stack_ptr = 0; + static int yy_start_stack_depth = 0; + static int *yy_start_stack = NULL; + ]]) +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ + m4_ifdef( [[M4_YY_NO_PUSH_STATE]],, + [[ + static void yy_push_state ( int _new_state M4_YY_PROTO_LAST_ARG); + ]]) + m4_ifdef( [[M4_YY_NO_POP_STATE]],, + [[ + static void yy_pop_state ( M4_YY_PROTO_ONLY_ARG ); + ]]) + m4_ifdef( [[M4_YY_NO_TOP_STATE]],, + [[ + static int yy_top_state ( M4_YY_PROTO_ONLY_ARG ); + ]]) +]]) + +]], +[[ +m4_define( [[M4_YY_NO_PUSH_STATE]]) +m4_define( [[M4_YY_NO_POP_STATE]]) +m4_define( [[M4_YY_NO_TOP_STATE]]) +]]) +]]) + +/* + * Amount of stuff to slurp up with each read. + * We assume the stdio library has already + * chosen a fit size foe whatever platform + * we're running on. + */ +#define YY_READ_BUF_SIZE BUFSIZ + +/* Size of default input buffer. We want to be able to fit two + * OS-level reads, but efficiency gains as the buffer size + * increases fall off after that + */ +#ifndef YY_BUF_SIZE +#define YY_BUF_SIZE (m4_ifdef([[M4_MODE_YY_BUFSIZE]], [[M4_MODE_YY_BUFSIZE]], [[2 * YY_READ_BUF_SIZE]])) +#endif + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* Copy whatever the last rule matched to the standard output. */ +#ifndef yyecho +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# Standard (non-C++) definition +/* This used to be an fputs(), but since the string might contain NUL's, + * we now use fwrite(). + */ +#define yyecho() do { if (fwrite( yytext, (size_t) yyleng, 1, yyout )) {} } while (0) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +%# C++ definition +#define yyecho() LexerOutput( yytext, yyleng ) +]]) +#endif +/* Legacy interface */ +#define ECHO yyecho() +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +#ifndef YY_EXIT_FAILURE +#define YY_EXIT_FAILURE 2 +#endif + +m4_ifdef([[M4_MODE_YY_NO_YYPANIC]],,[[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static void yynoreturn yypanic YYFARGS1(const char*, msg) { + M4_YY_DECL_GUTS_VAR(); + M4_YY_NOOP_GUTS_VAR(); + fprintf( stderr, "%s\n", msg ); + exit( YY_EXIT_FAILURE ); +} +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::LexerError( const char* msg ) { + M4_YY_DECL_GUTS_VAR(); + std::cerr << msg << std::endl; + exit( YY_EXIT_FAILURE ); +} +]]) +]]) + +/* Report a fatal error. Legacy interface. */ +#ifndef YY_FATAL_ERROR +m4_ifdef([[M4_MODE_C_ONLY]], [[ +#define YY_FATAL_ERROR(msg) yypanic( msg M4_YY_CALL_LAST_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +#define YY_FATAL_ERROR(msg) LexerError( msg ) +]]) +#endif + +/* Legacy interface */ +#ifndef YY_INPUT +#define YY_INPUT(buf,result,max_size) do {result = yyread(buf, max_size M4_YY_CALL_LAST_ARG);} while (0) + +m4_ifdef( [[M4_MODE_USER_YYREAD]], , [[ +/* Gets input and stuffs it into "buf". number of characters read, or YY_NULL, + * is returned in "result". + */ +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +int yyFlexLexer::yyread(char *buf, size_t max_size) { +]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static int yyread(char *buf, size_t max_size M4_YY_PROTO_LAST_ARG) { +]]) + int result; + M4_YY_DECL_GUTS_VAR(); +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_MODE_CPP_USE_READ]], [[ + errno=0; + while ( (result = (int) read( fileno(yyin), buf, (yy_size_t) max_size )) < 0 ) { + if( errno != EINTR) { + YY_FATAL_ERROR( "input in flex scanner failed" ); + break; + } + errno=0; + clearerr(yyin); + } +]]) +m4_ifdef( [[M4_MODE_NO_CPP_USE_READ]], [[ + if ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) { + int c = '*'; + int n; + for ( n = 0; n < max_size && + (c = getc( yyin )) != EOF && c != '\n'; ++n ) { + buf[n] = (char) c; + } + if ( c == '\n' ) { + buf[n++] = (char) c; + } + if ( c == EOF && ferror( yyin ) ) { + YY_FATAL_ERROR( "input in flex scanner failed" ); + } + result = n; + } else { + errno=0; + while ( (result = (int) fread(buf, 1, (yy_size_t) max_size, yyin)) == 0 && ferror(yyin)) { + if( errno != EINTR) { + YY_FATAL_ERROR( "input in flex scanner failed" ); + break; + } + errno=0; + clearerr(yyin); + } + } +]]) +]]) + +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +%# C++ definition + if ( (int)(result = LexerInput( (char *) buf, max_size )) < 0 ) { + YY_FATAL_ERROR( "input in flex scanner failed" ); + } +]]) + return result; +} +#endif +]]) +]]) + +m4_ifdef( [[M4_MODE_YYTERMINATE]], , [[ +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* No semi-colon after return; correct usage is to write "yyterminate();" - + * we don't want an extra ';' after the "return" because that will cause + * some compilers to complain about unreachable statements. + */ +#ifndef yyterminate +#define yyterminate() return YY_NULL +#endif +]]) +]]) + +/* Number of entries by which start-condition stack grows. */ +#ifndef YY_START_STACK_INCR +#define YY_START_STACK_INCR 25 +#endif + +m4_ifdef([[M4_MODE_TABLESEXT]], [[ +%# structures and prototypes +m4preproc_include(`tables_shared.h') + +/* Load the DFA tables from the given stream. */ +int yytables_fload (FILE * fp M4_YY_PROTO_LAST_ARG); + +/* Unload the tables from memory. */ +int yytables_destroy (M4_YY_PROTO_ONLY_ARG); +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl + +/** Describes a mapping from a serialized table id to its deserialized state in + * this scanner. This is the bridge between our "generic" deserialization code + * and the specifics of this scanner. + */ +struct yytbl_dmap { + enum yytbl_id dm_id;/**< table identifier */ + void **dm_arr; /**< address of pointer to store the deserialized table. */ + size_t dm_sz; /**< local sizeof() each element in table. */ +}; + +/** A {0,0,0}-terminated list of structs, forming the map */ +static struct yytbl_dmap yydmap[] = { +m4_ifdef([[M4_HOOK_NXT_TYPE]], [[ {YYTD_ID_NXT, (void**)&yy_nxt, sizeof(M4_HOOK_NXT_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_YYNXT_TYPE]], [[ {YYTD_ID_NXT, (void**)&yy_nxt, sizeof(M4_HOOK_YYNXT_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_MODE_FULLSPD]], [[ {YYTD_ID_START_STATE_LIST, (void**)&yy_start_state_list, sizeof(struct yy_trans_info*)},]],[[m4_dnl]]) +m4_ifdef([[M4_MODE_YYLINENO]], [[ {YYTD_ID_RULE_CAN_MATCH_EOL, (void**)&yy_rule_can_match_eol, sizeof(M4_HOOK_EOLTABLE_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_MODE_USEECS]], [[ {YYTD_ID_EC, (void**)&yy_ec, sizeof(YY_CHAR)},]],[[m4_dnl]]) +m4_ifdef([[M4_MODE_USEMECS]], [[ {YYTD_ID_META, (void**)&yy_meta, sizeof(YY_CHAR)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_ACCLIST_TYPE]], [[ {YYTD_ID_ACCLIST, (void**)&yy_acclist, sizeof(M4_HOOK_ACCLIST_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_MKCTBL_TYPE]], [[ {YYTD_ID_TRANSITION, (void**)&yy_transition, sizeof(M4_HOOK_MKCTBL_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_MKFTBL_TYPE]], [[ {YYTD_ID_ACCEPT, (void**)&yy_accept, sizeof(M4_HOOK_MKFTBL_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_ACCEPT_TYPE]], [[ {YYTD_ID_ACCEPT, (void**)&yy_accept, sizeof(M4_HOOK_ACCEPT_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_BASE_TYPE]], [[ {YYTD_ID_BASE, (void**)&yy_base, sizeof(M4_HOOK_BASE_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_DEF_TYPE]], [[ {YYTD_ID_DEF, (void**)&yy_def, sizeof(M4_HOOK_DEF_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_CHK_TYPE]], [[ {YYTD_ID_CHK, (void**)&yy_chk, sizeof(M4_HOOK_CHK_TYPE)},]],[[m4_dnl]]) +m4_ifdef([[M4_HOOK_NULTRANS_TYPE]], [[ {YYTD_ID_NUL_TRANS, (void**)&yy_NUL_trans, sizeof(M4_HOOK_NULTRANS_TYPE)},]],[[m4_dnl]]) + {0,0,0} +}; + +/** A tables-reader object to maintain some state in the read. */ +struct yytbl_reader { + FILE * fp; /**< input stream */ + flex_uint32_t bread; /**< bytes read since beginning of current tableset */ +}; + +]]) +/* end tables serialization structures and prototypes */ + +]]) + +/* Default declaration of generated scanner - a define so the user can + * easily add parameters. + */ +#ifndef YY_DECL +#define YY_DECL_IS_OURS 1 +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# Standard (non-C++) definition + + +m4_define( [[M4_YY_LEX_PROTO]], [[(M4_YY_PROTO_ONLY_ARG)]]) +m4_define( [[M4_YY_LEX_DECLARATION]], [[(M4_YY_DEF_ONLY_ARG)]]) + +m4_ifdef( [[M4_YY_BISON_LVAL]], +[[ + m4_dnl The bison pure parser is used. Redefine yylex to + m4_dnl accept the lval parameter. + + m4_define( [[M4_YY_LEX_PROTO]], [[\]] + [[(YYSTYPE * yylval_param M4_YY_PROTO_LAST_ARG)]]) + m4_define( [[M4_YY_LEX_DECLARATION]], [[\]] + [[YYFARGS1(YYSTYPE *,yylval_param)]]) +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ + m4_dnl Locations are used. yylex should also accept the ylloc parameter. + + m4_define( [[M4_YY_LEX_PROTO]], [[\]] + [[(YYSTYPE * yylval_param, YYLTYPE * yylloc_param M4_YY_PROTO_LAST_ARG)]]) + m4_define( [[M4_YY_LEX_DECLARATION]], [[\]] + [[YYFARGS2(YYSTYPE *,yylval_param, YYLTYPE *,yylloc_param)]]) +]]) + +extern int yylex M4_YY_LEX_PROTO; + +#define YY_DECL int yylex M4_YY_LEX_DECLARATION +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +%# C++ definition +#define YY_DECL int yyFlexLexer::yylex() +]]) +#endif /* !YY_DECL */ + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* Code executed at the beginning of each rule, after yytext and yyleng + * have been set up. + */ +#ifndef YY_USER_ACTION +#define YY_USER_ACTION +#endif +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +#define YY_RULE_SETUP \ + m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ if ( yyleng > 0 ) { \ + YY_CURRENT_BUFFER_LVALUE->yyatbol = (yytext[yyleng - 1] == '\n'); \ + } \ +]]) YY_USER_ACTION +]]) + +%# Code snippets used in various cases of code generation in the main scanner. + +m4_define([[M4_GEN_BACKING_UP]], [[ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_HAS_BACKING_UP]], [[ + /* Generate code to keep backing-up information. */ +m4_ifdef([[M4_MODE_FULLSPD]], [[ + if ( yy_current_state[-1].yy_nxt ) { +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD]], [[ + if ( yy_accept[yy_current_state] ) { +]]) + YY_G(yy_last_accepting_state) = yy_current_state; + YY_G(yy_last_accepting_cpos) = yy_cp; + } +]]) +]]) +]]) + +%# yy_c was formerly YY_CHAR, changed to int because table can now +%# have up to 0x101 entries, since we no longer generate a separate +%# NUL table. +%# +%# Note: on x86-64 architecture with gcc -O2, we save an instruction +%# in the main loop, since the character can now be zero-extended in +%# the process of retrieving it from the input stream or the yy_ec[] +%# or yy_meta[] arrays, whereas previously it was zero-extended by a +%# register-to-register move just prior to the yy_chk[] table lookup +m4_define([[M4_GEN_NEXT_COMPRESSED_STATE]], [[ + int yy_c = $1; + /* Save the backing-up info \before/ computing the next state + * because we always compute one more state than needed - we + * always proceed until we reach a jam state + */ + M4_GEN_BACKING_UP + + while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { + yy_current_state = (int) yy_def[yy_current_state]; + +m4_ifdef([[M4_MODE_USEMECS]], [[ + /* We've arranged it so that templates are never chained + * to one another. This means we can afford to make a + * very simple test to see if we need to convert to + * yy_c's meta-equivalence class without worrying + * about erroneously looking up the meta-equivalence + * class twice + */ + + /* lastdfa + 2 == YY_JAMSTATE + 1 is the beginning of the templates */ + if (yy_current_state >= YY_JAMSTATE + 1) { + yy_c = yy_meta[yy_c]; + } +]]) + } + yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c]; +]]) + +m4_define([[M4_GEN_START_STATE]], [[ + /* Generate the code to find the start state. */ +m4_ifdef([[M4_MODE_FULLSPD]], [[ +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[yy_current_state", "yy_start_state_list[YY_G(yy_start) + yyatbol()];]]) +m4_ifdef([[M4_MODE_NO_BOL_NEEDED]], [[yy_current_state = yy_start_state_list[YY_G(yy_start)];]]) +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD]], [[ + yy_current_state = YY_G(yy_start); +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[yy_current_state += yyatbol();]]) + /* Set up for storing up states. */ + m4_ifdef( [[M4_MODE_USES_REJECT]], [[ + YY_G(yy_state_ptr) = YY_G(yy_state_buf); + *YY_G(yy_state_ptr)++ = yy_current_state; +]]) +]]) +]]) + +m4_define([[M4_GEN_NEXT_MATCH_FULLSPD]], [[ + { + const struct yy_trans_info *yy_trans_info; + YY_CHAR yy_c; + + for ( yy_c = $1; + (yy_trans_info = &yy_current_state[yy_c])->yy_verify == yy_c; + yy_c = $2 ) + { + yy_current_state += yy_trans_info->yy_nxt; + + M4_GEN_BACKING_UP + } + } +]]) + +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +/** The main scanner function which does all the work. + */ +YY_DECL { + yy_state_type yy_current_state; + char *yy_cp, *yy_bp; + int yy_act; + M4_YY_DECL_GUTS_VAR(); + +m4_ifdef( [[M4_YY_NOT_REENTRANT]], +[[ + m4_ifdef( [[M4_YY_BISON_LVAL]], + [[ + YYSTYPE * yylval; + ]]) + m4_ifdef( [[<M4_YY_BISON_LLOC>]], + [[ + YYLTYPE * yylloc; + ]]) +]]) + +m4_ifdef( [[M4_YY_BISON_LVAL]], +[[ + yylval = yylval_param; +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ + yylloc = yylloc_param; +]]) + + if ( !YY_G(yy_init) ) { + YY_G(yy_init) = 1; + +#ifdef YY_USER_INIT + YY_USER_INIT; +#endif + +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + /* Create the reject buffer large enough to save one state per allowed character. */ + if ( ! YY_G(yy_state_buf) ) { + YY_G(yy_state_buf) = (yy_state_type *)yyalloc(YY_STATE_BUF_SIZE M4_YY_CALL_LAST_ARG); + } + if ( ! YY_G(yy_state_buf) ) { + YY_FATAL_ERROR( "out of dynamic memory in yylex()" ); + } +]]) + + if ( ! YY_G(yy_start) ) { + YY_G(yy_start) = 1; /* first start state */ + } + if ( ! yyin ) { +m4_ifdef([[M4_MODE_C_ONLY]], [[ + yyin = stdin; +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + yyin.rdbuf(std::cin.rdbuf()); +]]) + } + if ( ! yyout ) { +m4_ifdef([[M4_MODE_C_ONLY]], [[ + yyout = stdout; +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + yyout.rdbuf(std::cout.rdbuf()); +]]) + } + if ( yy_current_buffer() == NULL ) { + yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); + YY_CURRENT_BUFFER_LVALUE = + yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); + } + + yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); + } + + /* open scope of user declarationns */ + { +%% [4.0] user's declarations go here + + while ( /*CONSTCOND*/1 ) { /* loops until end-of-file is reached */ +m4_ifdef( [[M4_MODE_YYMORE_USED]], [[ +m4_ifdef( [[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ + YY_G(yy_more_len) = 0; + if ( YY_G(yy_more_flag) ) { + YY_G(yy_more_len) = (int) (YY_G(yy_c_buf_p) - YY_G(yytext_ptr)); + YY_G(yy_more_flag) = 0; + } +]]) +]]) + yy_cp = YY_G(yy_c_buf_p); + + /* Support of yytext. */ + *yy_cp = YY_G(yy_hold_char); + + /* yy_bp points to the position in yy_ch_buf of the start of + * the current run. + */ + yy_bp = yy_cp; + +M4_GEN_START_STATE + + yy_match: + /* Generate the code to find the next match. */ +%# Conditional indirection through an equivalence map +m4_ifdef([[M4_MODE_USEECS]], m4_define([[M4_EC]], [[*(yy_ec+$1)]])) +m4_ifdef([[M4_MODE_NO_USEECS]], [[m4_define([[M4_EC]], [[$1]])]]) + +m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[m4_dnl +m4_ifdef([[M4_MODE_GENTABLES]], [[m4_dnl + while ((yy_current_state = yy_nxt[yy_current_state][ M4_EC(YY_SC_TO_UI(*yy_cp)) ]) > 0) { +]]) +m4_ifdef([[M4_MODE_NO_GENTABLES]], [[ + while ((yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + M4_EC(YY_SC_TO_UI(*yy_cp)) ]) > 0) { +]]) +M4_GEN_BACKING_UP + yy_cp++; + } + yy_current_state = -yy_current_state; +]]) +m4_ifdef([[M4_MODE_FULLSPD]], [[ + M4_GEN_NEXT_MATCH_FULLSPD(M4_EC(YY_SC_TO_UI(*yy_cp)), M4_EC(YY_SC_TO_UI(*++yy_cp))) +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[ + do { + M4_GEN_NEXT_COMPRESSED_STATE(M4_EC(YY_SC_TO_UI(*yy_cp))) + + m4_ifdef([[M4_MODE_USES_REJECT]], [[*YY_G(yy_state_ptr)++ = yy_current_state;]]) + ++yy_cp; + + } + m4_ifdef([[M4_MODE_INTERACTIVE]], [[while ( yy_base[yy_current_state] != YY_JAMBASE );]]) + m4_ifdef([[M4_MODE_NO_INTERACTIVE]], [[while ( yy_current_state != YY_JAMSTATE );]]) + +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_NO_INTERACTIVE]], [[ + /* Do the guaranteed-needed backing up to figure out + * the match. + */ + yy_cp = YY_G(yy_last_accepting_cpos); + yy_current_state = YY_G(yy_last_accepting_state); +]]) +]]) +]]) + + yy_find_action: + /* code to find the action number goes here */ + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_act = yy_current_state[-1].yy_nxt;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yy_act = yy_accept[yy_current_state];]]) +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT]], [[ + yy_current_state = *--YY_G(yy_state_ptr); + YY_G(yy_lp) = yy_accept[yy_current_state]; +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT_REALLY_USED]], [[find_rule: /* we branch to this label when backing up */]]) + for ( ; ; ) { /* loop until we find out what rule we matched */ + if (YY_G(yy_lp) && YY_G(yy_lp) < yy_accept[yy_current_state + 1]) { + yy_act = yy_acclist[YY_G(yy_lp)]; +m4_ifdef([[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + if ((yy_act & YY_TRAILING_HEAD_MASK) != 0 || YY_G(yy_looking_for_trail_begin)) { + if (yy_act == YY_G(yy_looking_for_trail_begin)) { + YY_G(yy_looking_for_trail_begin) = 0; + yy_act &= ~YY_TRAILING_HEAD_MASK; + break; + } + } else if (( yy_act & YY_TRAILING_MASK) != 0) { + YY_G(yy_looking_for_trail_begin) = yy_act & ~YY_TRAILING_MASK; + YY_G(yy_looking_for_trail_begin) |= YY_TRAILING_HEAD_MASK; +m4_ifdef([[M4_MODE_REAL_REJECT]], [[ + /* Remember matched text in case we back up + * due to REJECT. + */ + YY_G(yy_full_match) = yy_cp; + YY_G(yy_full_state) = YY_G(yy_state_ptr); + YY_G(yy_full_lp) = YY_G(yy_lp); +]]) + } else { + YY_G(yy_full_match) = yy_cp; + YY_G(yy_full_state) = YY_G(yy_state_ptr); + YY_G(yy_full_lp) = YY_G(yy_lp); + break; + } + ++YY_G(yy_lp); + goto find_rule; +]]) +m4_ifdef([[M4_MODE_NO_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + /* Remember matched text in case we back up due to + * trailing context plus REJECT. + */ + YY_G(yy_full_match) = yy_cp; + break; +]]) + } + + --yy_cp; + + /* We could consolidate the following two lines with those at + * the beginning, but at the cost of complaints that we're + * branching inside a loop. + */ + yy_current_state = *--YY_G(yy_state_ptr); + YY_G(yy_lp) = yy_accept[yy_current_state]; + } /* close for */ +]]) +m4_ifdef([[M4_MODE_FIND_ACTION_COMPRESSED]], [[ yy_act = yy_accept[yy_current_state]; + if ( yy_act == 0 ) { /* have to back up */ + yy_cp = YY_G(yy_last_accepting_cpos); + yy_current_state = YY_G(yy_last_accepting_state); + yy_act = yy_accept[yy_current_state]; + } +]]) + + YY_DO_BEFORE_ACTION; + +m4_ifdef( [[M4_MODE_YYLINENO]],[[ +m4_define([[M4_YYL_BASE]], [[m4_ifdef([[M4_MODE_YYMORE_USED]], + [[m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], + [[YY_G(yy_prev_more_offset)]], [[YY_G(yy_more_len)]])]], [[0]])]]) + if ( yy_act != YY_END_OF_BUFFER && yy_rule_can_match_eol[yy_act] ) { + int yyl; + for ( yyl = M4_YYL_BASE; yyl < yyleng; ++yyl ) { + if ( yytext[yyl] == '\n' ) { + M4_YY_INCR_LINENO(); + + } + } + } +]]) + + do_action: /* This label is used only to access EOF actions. */ + +m4_ifdef([[M4_MODE_DEBUG]], [[ + if ( yyflexdebug ) { + if ( yy_act == 0 ) { +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + std::cerr << "--scanner backing up\n"; +]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ + fprintf( stderr, "--scanner backing up\n" ); +]]) + } else if ( yy_act < YY_NUM_RULES ) { +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + std::cerr << "--accepting rule at line " << yy_rule_linenum[yy_act] << + "(\"" << yytext << "\")\n"; +]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ + fprintf( stderr, "--accepting rule at line %ld (\"%s\")\n", + (long)yy_rule_linenum[yy_act], yytext ); +]]) + } else if ( yy_act == YY_NUM_RULES ) { +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + std::cerr << "--accepting default rule (\"" << yytext << "\")\n"; +]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ + fprintf( stderr, "--accepting default rule (\"%s\")\n", + yytext ); +]]) + } else if ( yy_act == YY_NUM_RULES + 1 ) { + +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + std::cerr << "--(end of buffer or a NUL)\n"; +]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ + fprintf( stderr, "--(end of buffer or a NUL)\n" ); +]]) + } else { +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + std::cerr << "--EOF (start condition " << yystart() << ")\n"; +]]) +m4_ifdef([[M4_MODE_C_ONLY]], [[ + fprintf( stderr, "--EOF (start condition %d)\n", yystart() ); +]]) + } + } +]]) + + switch ( yy_act ) { /* beginning of action switch */ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_HAS_BACKING_UP]], [[ + case 0: /* must back up */ + /* undo the effects of YY_DO_BEFORE_ACTION */ + *yy_cp = YY_G(yy_hold_char); + + /* Backing-up info for compressed tables is taken \after/ */ + /* yy_cp has been incremented for the next state. */ + yy_cp = YY_G(yy_last_accepting_cpos); + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_cp++;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yy_cp++;]]) + + yy_current_state = YY_G(yy_last_accepting_state); + goto yy_find_action; +]]) +]]) +%% [5.0] user actions get inserted here + + case YY_END_OF_BUFFER: + { + /* Amount of text matched not including the EOB char. */ + int yy_amount_of_matched_text = (int) (yy_cp - YY_G(yytext_ptr)) - 1; + + /* Undo the effects of YY_DO_BEFORE_ACTION. */ + *yy_cp = YY_G(yy_hold_char); + YY_RESTORE_YY_MORE_OFFSET + + if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_NEW ) { + /* We're scanning a new file or input source. It's + * possible that this happened because the user + * just pointed yyin at a new source and called + * yylex(). If so, then we have to assure + * consistency between yy_current_buffer() and our + * globals. Here is the right place to do so, because + * this is the first action (other than possibly a + * back-up) that will match for the new input source. + */ + YY_G(yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; +m4_ifdef([[M4_MODE_C_ONLY]], [[ + YY_CURRENT_BUFFER_LVALUE->yy_input_file = yyin; +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + YY_CURRENT_BUFFER_LVALUE->yy_input_file = yyin.rdbuf(); +]]) + YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = YY_BUFFER_NORMAL; + } + + /* Note that here we test for yy_c_buf_p "<=" to the position + * of the first EOB in the buffer, since yy_c_buf_p will + * already have been incremented past the NUL character + * (since all states make transitions on EOB to the + * end-of-buffer state). Contrast this with the test + * in input(). + */ + if ( YY_G(yy_c_buf_p) <= &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)] ) { /* This was really a NUL. */ + yy_state_type yy_next_state; + + YY_G(yy_c_buf_p) = YY_G(yytext_ptr) + yy_amount_of_matched_text; + + yy_current_state = yy_get_previous_state( M4_YY_CALL_ONLY_ARG ); + + /* Okay, we're now positioned to make the NUL + * transition. We couldn't have + * yy_get_previous_state() go ahead and do it + * for us because it doesn't know how to deal + * with the possibility of jamming (and we don't + * want to build jamming into it because then it + * will run more slowly). + */ + + yy_next_state = yy_try_NUL_trans( yy_current_state M4_YY_CALL_LAST_ARG); + + yy_bp = YY_G(yytext_ptr) + YY_MORE_ADJ; + + if ( yy_next_state ) { + /* Consume the NUL. */ + yy_cp = ++YY_G(yy_c_buf_p); + yy_current_state = yy_next_state; + goto yy_match; + } else { +%# Disguised case statement on table modes + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_cp = YY_G(yy_c_buf_p);]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yy_cp = YY_G(yy_c_buf_p);]]) +m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_NOT_MODE_INTERACTIVE]], [[ + /* Do the guaranteed-needed backing up to figure + * out the match. + */ + yy_cp = YY_G(yy_last_accepting_cpos); + yy_current_state = YY_G(yy_last_accepting_state); +]]) +]]) +%# Disguised case statement on table modes ends +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT_OR_INTERACTIVE]], [[ + /* Still need to initialize yy_cp, though + * yy_current_state was set up by + * yy_get_previous_state(). + */ + yy_cp = YY_G(yy_c_buf_p); +]]) +]]) + goto yy_find_action; + } + } else { /* not a NUL */ + switch ( yy_get_next_buffer( M4_YY_CALL_ONLY_ARG ) ) { + case EOB_ACT_END_OF_FILE: + YY_G(yy_did_buffer_switch_on_eof) = 0; + + if ( yywrap( M4_YY_CALL_ONLY_ARG ) ) { + /* Note: because we've taken care in + * yy_get_next_buffer() to have set up + * yytext, we can now set up + * yy_c_buf_p so that if some total + * hoser (like flex itself) wants to + * call the scanner after we return the + * YY_NULL, it'll still work - another + * YY_NULL will get returned. + */ + YY_G(yy_c_buf_p) = YY_G(yytext_ptr) + YY_MORE_ADJ; + + yy_act = YY_STATE_EOF(yystart()); + goto do_action; + } else { + if ( ! YY_G(yy_did_buffer_switch_on_eof) ) { + YY_NEW_FILE; + } + } + break; + case EOB_ACT_CONTINUE_SCAN: + YY_G(yy_c_buf_p) = + YY_G(yytext_ptr) + yy_amount_of_matched_text; + + yy_current_state = yy_get_previous_state( M4_YY_CALL_ONLY_ARG ); + + yy_cp = YY_G(yy_c_buf_p); + yy_bp = YY_G(yytext_ptr) + YY_MORE_ADJ; + goto yy_match; + + case EOB_ACT_LAST_MATCH: + YY_G(yy_c_buf_p) = + &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)]; + + yy_current_state = yy_get_previous_state( M4_YY_CALL_ONLY_ARG ); + + yy_cp = YY_G(yy_c_buf_p); + yy_bp = YY_G(yytext_ptr) + YY_MORE_ADJ; + goto yy_find_action; + } /* end EOB inner switch */ + } /* end if */ + break; + } /* case YY_END_OF_BUFFER */ + default: + YY_FATAL_ERROR("fatal flex scanner internal error--no action found" ); + } /* end of action switch */ + } /* end of scanning one token */ + } /* end of user's declarations */ +} /* end of yylex */ +]]) + +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +/* The contents of this function are C++ specific, so the YY_G macro is not used. + * This constructor simply maintains backward compatibility. + * DEPRECATED + */ +yyFlexLexer::yyFlexLexer( std::istream* arg_yyin, std::ostream* arg_yyout ): + yyin(arg_yyin ? arg_yyin->rdbuf() : std::cin.rdbuf()), + yyout(arg_yyout ? arg_yyout->rdbuf() : std::cout.rdbuf()) +{ + ctor_common(); +} + +/* The contents of this function are C++ specific, so the YY_G macro is not used. + */ +yyFlexLexer::yyFlexLexer( std::istream& arg_yyin, std::ostream& arg_yyout ): + yyin(arg_yyin.rdbuf()), + yyout(arg_yyout.rdbuf()) +{ + ctor_common(); +} + +/* The contents of this function are C++ specific, so the YY_G macro is not used. + */ +void yyFlexLexer::ctor_common() { + yy_c_buf_p = 0; + yy_init = 0; + yy_start = 0; + yyflexdebug = 0; + yylineno = 1; // this will only get updated if %option yylineno + + yy_did_buffer_switch_on_eof = 0; + + yy_looking_for_trail_begin = 0; + yy_more_flag = 0; + yy_more_len = 0; + yy_more_offset = yy_prev_more_offset = 0; + + yy_start_stack_ptr = yy_start_stack_depth = 0; + yy_start_stack = NULL; + + yy_buffer_stack = NULL; + yy_buffer_stack_top = 0; + yy_buffer_stack_max = 0; + + +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + yy_state_buf = new yy_state_type[YY_STATE_BUF_SIZE]; +]], +[[ + yy_state_buf = 0; +]]) +} + +/* The contents of this function are C++ specific, so the YY_G macro is not used. + */ +yyFlexLexer::~yyFlexLexer() { + delete [] yy_state_buf; + yyfree( yy_start_stack M4_YY_CALL_LAST_ARG ); + yy_delete_buffer( yy_current_buffer() M4_YY_CALL_LAST_ARG); + yyfree( yy_buffer_stack M4_YY_CALL_LAST_ARG ); +} + +/* The contents of this function are C++ specific, so the YY_G macro is not used. + */ +void yyFlexLexer::switch_streams( std::istream& new_in, std::ostream& new_out ) { + // was if( new_in ) + yy_delete_buffer( YY_CURRENT_BUFFER M4_YY_CALL_LAST_ARG); + yy_switch_to_buffer( yy_create_buffer( new_in, YY_BUF_SIZE M4_YY_CALL_LAST_ARG) M4_YY_CALL_LAST_ARG); + + // was if( new_out ) + yyout.rdbuf(new_out.rdbuf()); +} + +/* The contents of this function are C++ specific, so the YY_G macro is not used. + */ +void yyFlexLexer::switch_streams( std::istream* new_in, std::ostream* new_out ) +{ + if( ! new_in ) { + new_in = &yyin; + } + + if ( ! new_out ) { + new_out = &yyout; + } + + switch_streams(*new_in, *new_out); +} + +#ifdef YY_INTERACTIVE +int yyFlexLexer::LexerInput( char* buf, int /* max_size */ ) +#else +int yyFlexLexer::LexerInput( char* buf, int max_size ) +#endif +{ + if ( yyin.eof() || yyin.fail() ) { + return 0; + } +#ifdef YY_INTERACTIVE + yyin.get( buf[0] ); + + if ( yyin.eof() ) { + return 0; + } + if ( yyin.bad() ) { + return -1; + } + return 1; + +#else + (void) yyin.read( buf, max_size ); + + if ( yyin.bad() ) { + return -1; + } else { + return yyin.gcount(); + } +#endif +} + +void yyFlexLexer::LexerOutput( const char* buf, int size ) { + (void) yyout.write( buf, size ); +} +]]) +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* yy_get_next_buffer - try to read in a new buffer + * + * Returns a code representing an action: + * EOB_ACT_LAST_MATCH - + * EOB_ACT_CONTINUE_SCAN - continue scanning from current position + * EOB_ACT_END_OF_FILE - end of file + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static int yy_get_next_buffer (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +int yyFlexLexer::yy_get_next_buffer() +]]) +{ + M4_YY_DECL_GUTS_VAR(); + char *dest = YY_CURRENT_BUFFER_LVALUE->yy_ch_buf; + char *source = YY_G(yytext_ptr); + int number_to_move, i; + int ret_val; + + if ( YY_G(yy_c_buf_p) > &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars) + 1] ) { + YY_FATAL_ERROR( "fatal flex scanner internal error--end of buffer missed" ); + } + if ( YY_CURRENT_BUFFER_LVALUE->yy_fill_buffer == 0 ) { + /* Don't try to fill the buffer, so this is an EOF. */ + if ( YY_G(yy_c_buf_p) - YY_G(yytext_ptr) - YY_MORE_ADJ == 1 ) { + /* We matched a single character, the EOB, so + * treat this as a final EOF. + */ + return EOB_ACT_END_OF_FILE; + } else { + /* We matched some text prior to the EOB, first + * process it. + */ + return EOB_ACT_LAST_MATCH; + } + } + + /* Try to read more data. */ + + /* First move last chars to start of buffer. */ + number_to_move = (int) (YY_G(yy_c_buf_p) - YY_G(yytext_ptr) - 1); + + for ( i = 0; i < number_to_move; ++i ) { + *(dest++) = *(source++); + } + if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_EOF_PENDING ) { + /* don't do the read, it's not guaranteed to return an EOF, + * just force an EOF + */ + YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars) = 0; + } else { + int num_to_read = + YY_CURRENT_BUFFER_LVALUE->yy_buf_size - number_to_move - 1; + + while ( num_to_read <= 0 ) { /* Not enough room in the buffer - grow it. */ +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + YY_FATAL_ERROR( +"input buffer overflow, can't enlarge buffer because scanner uses yyreject()" ); +]], +[[ + /* just a shorter name for the current buffer */ + yybuffer b = YY_CURRENT_BUFFER_LVALUE; + + int yy_c_buf_p_offset = + (int) (YY_G(yy_c_buf_p) - b->yy_ch_buf); + + if ( b->yy_is_our_buffer ) { + int new_size = b->yy_buf_size * 2; + + if ( new_size <= 0 ) { + b->yy_buf_size += b->yy_buf_size / 8; + } else { + b->yy_buf_size *= 2; + } + b->yy_ch_buf = (char *) + /* Include room in for 2 EOB chars. */ + yyrealloc( (void *) b->yy_ch_buf, + (yy_size_t) (b->yy_buf_size + 2) M4_YY_CALL_LAST_ARG ); + } else { + /* Can't grow it, we don't own it. */ + b->yy_ch_buf = NULL; + } + if ( ! b->yy_ch_buf ) { + YY_FATAL_ERROR( + "fatal error - scanner input buffer overflow" ); + } + YY_G(yy_c_buf_p) = &b->yy_ch_buf[yy_c_buf_p_offset]; + + num_to_read = YY_CURRENT_BUFFER_LVALUE->yy_buf_size - + number_to_move - 1; +]]) + } + + if ( num_to_read > YY_READ_BUF_SIZE ) { + num_to_read = YY_READ_BUF_SIZE; + } + /* Read in more data. */ + YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]), + YY_G(yy_n_chars), num_to_read ); + + YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars); + } + + if ( YY_G(yy_n_chars) == 0 ) { + if ( number_to_move == YY_MORE_ADJ ) { + ret_val = EOB_ACT_END_OF_FILE; + yyrestart( yyin M4_YY_CALL_LAST_ARG); + } else { + ret_val = EOB_ACT_LAST_MATCH; + YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = + YY_BUFFER_EOF_PENDING; + } + } else { + ret_val = EOB_ACT_CONTINUE_SCAN; + } + if ((YY_G(yy_n_chars) + number_to_move) > YY_CURRENT_BUFFER_LVALUE->yy_buf_size) { + /* Extend the array by 50%, plus the number we really need. */ + int new_size = YY_G(yy_n_chars) + number_to_move + (YY_G(yy_n_chars) >> 1); + YY_CURRENT_BUFFER_LVALUE->yy_ch_buf = (char *) yyrealloc( + (void *) YY_CURRENT_BUFFER_LVALUE->yy_ch_buf, (yy_size_t) new_size M4_YY_CALL_LAST_ARG ); + if ( ! YY_CURRENT_BUFFER_LVALUE->yy_ch_buf ) { + YY_FATAL_ERROR( "out of dynamic memory in yy_get_next_buffer()" ); + } + /* "- 2" to take care of EOB's */ + YY_CURRENT_BUFFER_LVALUE->yy_buf_size = (int) (new_size - 2); + } + + YY_G(yy_n_chars) += number_to_move; + YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)] = YY_END_OF_BUFFER_CHAR; + YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars) + 1] = YY_END_OF_BUFFER_CHAR; + + YY_G(yytext_ptr) = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[0]; + + return ret_val; +} +]]) + +/* yy_get_previous_state - get the state just before the EOB char was reached */ + +m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static yy_state_type yy_get_previous_state (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +yy_state_type yyFlexLexer::yy_get_previous_state() +]]) +{ + yy_state_type yy_current_state; + char *yy_cp; + M4_YY_DECL_GUTS_VAR(); + + M4_GEN_START_STATE + for ( yy_cp = YY_G(yytext_ptr) + YY_MORE_ADJ; yy_cp < YY_G(yy_c_buf_p); ++yy_cp ) { + /* Generate the code to find the next state. */ + m4_ifdef([[M4_MODE_NO_NULTRANS]], [[m4_define([[CHAR_MAP_3]], [[(*yy_cp ? M4_EC(YY_SC_TO_UI(*yy_cp)) : YY_NUL_EC)]])]]) + m4_ifdef([[M4_MODE_NULTRANS]], [[m4_define([[CHAR_MAP_3]], [[M4_EC(YY_SC_TO_UI(*yy_cp))]])]]) + + m4_ifdef([[M4_MODE_NULTRANS]], [[ + /* Compressed tables back up *before* they match. */ + m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[M4_GEN_BACKING_UP]]) + if ( *yy_cp ) { + ]]) + + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[ + m4_ifdef([[M4_MODE_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state][CHAR_MAP_3];]]) + m4_ifdef([[M4_MODE_NO_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + CHAR_MAP_3];]]) + ]]) + + m4_ifdef([[M4_MODE_FULLSPD]], [[yy_current_state += yy_current_state[CHAR_MAP_3].yy_nxt;]]) + m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[M4_GEN_NEXT_COMPRESSED_STATE(CHAR_MAP_3)]]) + +m4_ifdef([[M4_MODE_NULTRANS]], [[ + } else { + yy_current_state = yy_NUL_trans[yy_current_state]; + } +]]) + + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[M4_GEN_BACKING_UP]]) + m4_ifdef([[M4_MODE_FULLSPD]], [[M4_GEN_BACKING_UP]]) + m4_ifdef([[M4_MODE_USES_REJECT]], [[*YY_G(yy_state_ptr)++ = yy_current_state;]]) + } + + return yy_current_state; +} + + +/* yy_try_NUL_trans - try to make a transition on the NUL character + * + * synopsis + * next_state = yy_try_NUL_trans( current_state ); + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static yy_state_type yy_try_NUL_trans YYFARGS1( yy_state_type, yy_current_state) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +yy_state_type yyFlexLexer::yy_try_NUL_trans( yy_state_type yy_current_state ) +]]) +{ + int yy_is_jam; + M4_YY_DECL_GUTS_VAR(); /* This var may be unused depending upon options. */ + /* Generate code for handling NUL's, if needed. */ + + /* First, deal with backing up and setting up yy_cp if the scanner + * finds that it should JAM on the NUL. + * + * Only generate a definition for "yy_cp" if we'll generate code + * that uses it. Otherwise lint and the like complain. + */ + m4_ifdef([[M4_MODE_NEED_YY_CP]], [[char *yy_cp = YY_G(yy_c_buf_p);]]) + +%# Note that this statement block and the following three are +%# not executed serially but are an if-then-else cascade +%# for different table modes. +m4_ifdef([[M4_MODE_NULTRANS]], [[ + yy_current_state = yy_NUL_trans[yy_current_state]; + yy_is_jam = (yy_current_state == 0); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS]], [[ +m4_ifdef([[M4_MODE_NULTRANS_FULLTBL]], [[ +m4_ifdef([[M4_MODE_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state][YY_NUL_EC];]]) +m4_ifdef([[M4_MODE_NO_GENTABLES]], [[yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + YY_NUL_EC];]]) + yy_is_jam = (yy_current_state <= 0); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS_FULLTBL]], [[ +m4_ifdef([[M4_MODE_NULTRANS_FULLSPD]], [[ + int yy_c = YY_NUL_EC; + + const struct yy_trans_info *yy_trans_info; + + yy_trans_info = &yy_current_state[(unsigned int) yy_c]; + yy_current_state += yy_trans_info->yy_nxt; + yy_is_jam = (yy_trans_info->yy_verify != yy_c); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS_FULLSPD]], [[ +M4_GEN_NEXT_COMPRESSED_STATE(YY_NUL_EC) +yy_is_jam = (yy_current_state == YY_JAMSTATE); +m4_ifdef([[M4_MODE_USES_REJECT]], [[ + /* Only stack this state if it's a transition we + * actually make. If we stack it on a jam, then + * the state stack and yy_c_buf_p get out of sync. + */ + if ( ! yy_is_jam ) { + *YY_G(yy_state_ptr)++ = yy_current_state; + } + ]]) +]]) +]]) +]]) +%# End of if-else cascade + +m4_ifdef([[M4_MODE_NULTRANS_WRAP]], [[ + /* If we've entered an accepting state, back up; note that + * compressed tables have *already* done such backing up, so + * we needn't bother with it again. + */ + if ( ! yy_is_jam ) { + M4_GEN_BACKING_UP + } +]]) + + M4_YY_NOOP_GUTS_VAR(); + return yy_is_jam ? 0 : yy_current_state; +} + +m4_ifdef([[M4_MODE_CXX_ONLY]], [[m4_undefine([[M4_YY_NO_YYUNPUT]])]]) +m4_ifdef( [[M4_YY_NO_YYUNPUT]], , [[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static void yyunput_r YYFARGS2( int,c, char *,yy_bp) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yyunput_r( int c, char* yy_bp) +]]) +{ + char *yy_cp; + M4_YY_DECL_GUTS_VAR(); + + yy_cp = YY_G(yy_c_buf_p); + + /* undo effects of setting up yytext */ + *yy_cp = YY_G(yy_hold_char); + + if ( yy_cp < YY_CURRENT_BUFFER_LVALUE->yy_ch_buf + 2 ) { + /* need to shift things up to make room */ + /* +2 for EOB chars. */ + int number_to_move = YY_G(yy_n_chars) + 2; + char *dest = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[ + YY_CURRENT_BUFFER_LVALUE->yy_buf_size + 2]; + char *source = + &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]; + + while ( source > YY_CURRENT_BUFFER_LVALUE->yy_ch_buf ) { + *--dest = *--source; + } + yy_cp += (int) (dest - source); + yy_bp += (int) (dest - source); + YY_CURRENT_BUFFER_LVALUE->yy_n_chars = + YY_G(yy_n_chars) = (int) YY_CURRENT_BUFFER_LVALUE->yy_buf_size; + + if ( yy_cp < YY_CURRENT_BUFFER_LVALUE->yy_ch_buf + 2 ) { + YY_FATAL_ERROR( "flex scanner push-back overflow" ); + } + } + + *--yy_cp = (char) c; + +m4_ifdef( [[M4_MODE_YYLINENO]], +[[ + if ( c == '\n' ){ + --yylineno; + } +]]) + + YY_G(yytext_ptr) = yy_bp; + YY_G(yy_hold_char) = *yy_cp; + YY_G(yy_c_buf_p) = yy_cp; +} +]]) + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +#ifndef YY_NO_YYINPUT +int yyinput (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +int yyFlexLexer::yyinput() +]]) +{ + int c; + M4_YY_DECL_GUTS_VAR(); + + *YY_G(yy_c_buf_p) = YY_G(yy_hold_char); + + if ( *YY_G(yy_c_buf_p) == YY_END_OF_BUFFER_CHAR ) { + /* yy_c_buf_p now points to the character we want to return. + * If this occurs *before* the EOB characters, then it's a + * valid NUL; if not, then we've hit the end of the buffer. + */ + if ( YY_G(yy_c_buf_p) < &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)] ) { + /* This was really a NUL. */ + *YY_G(yy_c_buf_p) = '\0'; + } else { + /* need more input */ + int offset = (int) (YY_G(yy_c_buf_p) - YY_G(yytext_ptr)); + ++YY_G(yy_c_buf_p); + + switch ( yy_get_next_buffer( M4_YY_CALL_ONLY_ARG ) ) { + case EOB_ACT_LAST_MATCH: + /* This happens because yy_g_n_b() + * sees that we've accumulated a + * token and flags that we need to + * try matching the token before + * proceeding. But for input(), + * there's no matching to consider. + * So convert the EOB_ACT_LAST_MATCH + * to EOB_ACT_END_OF_FILE. + */ + + /* Reset buffer status. */ + yyrestart( yyin M4_YY_CALL_LAST_ARG); + + /*FALLTHROUGH*/ + + case EOB_ACT_END_OF_FILE: + if ( yywrap( M4_YY_CALL_ONLY_ARG ) ) { + return 0; + } + if ( ! YY_G(yy_did_buffer_switch_on_eof) ) { + YY_NEW_FILE; + } + return yyinput(M4_YY_CALL_ONLY_ARG); + + case EOB_ACT_CONTINUE_SCAN: + YY_G(yy_c_buf_p) = YY_G(yytext_ptr) + offset; + break; + } + } + } + + c = *(unsigned char *) YY_G(yy_c_buf_p); /* cast for 8-bit char's */ + *YY_G(yy_c_buf_p) = '\0'; /* preserve yytext */ + YY_G(yy_hold_char) = *++YY_G(yy_c_buf_p); + +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ + YY_CURRENT_BUFFER_LVALUE->yyatbol = (c == '\n'); +m4_ifdef([[M4_MODE_YYLINENO]], [[ + if ( YY_CURRENT_BUFFER_LVALUE->yyatbol ) { + M4_YY_INCR_LINENO(); + } +]]) +]]) +m4_ifdef([[M4_MODE_NO_BOL_NEEDED]], [[ +m4_ifdef([[M4_MODE_YYLINENO]], [[ + if ( c == '\n' ) { + M4_YY_INCR_LINENO(); + } + ]]) +]]) + + return c; +} +m4_ifdef([[M4_MODE_C_ONLY]], [[ +#endif /* ifndef YY_NO_YYINPUT */ +]]) + +/** Immediately switch to a different input stream. + * @param input_file A readable stream. + * M4_YY_DOC_PARAM + * @note This function does not reset the start condition to @c INITIAL . + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +void yyrestart YYFARGS1( FILE *,input_file) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yyrestart( std::istream& input_file ) +]]) +{ + M4_YY_DECL_GUTS_VAR(); + + if ( yy_current_buffer() == NULL ) { + yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); + YY_CURRENT_BUFFER_LVALUE = + yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); + } + + yy_init_buffer( yy_current_buffer(), input_file M4_YY_CALL_LAST_ARG); + yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); +} + +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +/** Delegate to the new version that takes an istream reference. + * @param input_file A readable stream. + * M4_YY_DOC_PARAM + * @note This function does not reset the start condition to @c INITIAL . + */ +void yyFlexLexer::yyrestart( std::istream* input_file ) +{ + if( ! input_file ) { + input_file = &yyin; + } + yyrestart( *input_file ); +} +]]) + +/** Switch to a different input buffer. + * @param new_buffer The new input buffer. + * M4_YY_DOC_PARAM + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +void yy_switch_to_buffer YYFARGS1( yybuffer ,new_buffer) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yy_switch_to_buffer( yybuffer new_buffer ) +]]) +{ + M4_YY_DECL_GUTS_VAR(); + + /* TODO. We should be able to replace this entire function body + * with + * yypop_buffer_state(); + * yypush_buffer_state(new_buffer); + */ + yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); + if ( yy_current_buffer() == new_buffer ) { + return; + } + if ( yy_current_buffer() ) { + /* Flush out information for old buffer. */ + *YY_G(yy_c_buf_p) = YY_G(yy_hold_char); + YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = YY_G(yy_c_buf_p); + YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars); + } + + YY_CURRENT_BUFFER_LVALUE = new_buffer; + yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); + + /* We don't actually know whether we did this switch during + * EOF (yywrap()) processing, but the only time this flag + * is looked at is after yywrap() is called, so it's safe + * to go ahead and always set it. + */ + YY_G(yy_did_buffer_switch_on_eof) = 1; +} + + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static void yy_load_buffer_state (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yy_load_buffer_state() +]]) +{ + M4_YY_DECL_GUTS_VAR(); + YY_G(yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; + YY_G(yytext_ptr) = YY_G(yy_c_buf_p) = YY_CURRENT_BUFFER_LVALUE->yy_buf_pos; +m4_ifdef([[M4_MODE_C_ONLY]], [[ + yyin = YY_CURRENT_BUFFER_LVALUE->yy_input_file; +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + yyin.rdbuf(YY_CURRENT_BUFFER_LVALUE->yy_input_file); +]]) + YY_G(yy_hold_char) = *YY_G(yy_c_buf_p); +} + +/** Allocate and initialize an input buffer state. + * @param file A readable stream. + * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. + * M4_YY_DOC_PARAM + * @return the allocated buffer state. + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +yybuffer yy_create_buffer YYFARGS2( FILE *,file, int ,size) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +yybuffer yyFlexLexer::yy_create_buffer( std::istream& file, int size ) +]]) +{ + yybuffer b; + M4_YY_DECL_GUTS_VAR(); + + b = (yybuffer) yyalloc( sizeof( struct yy_buffer_state ) M4_YY_CALL_LAST_ARG ); + if ( b == NULL ) { + YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); + } + b->yy_buf_size = size; + + /* yy_ch_buf has to be 2 characters longer than the size given because + * we need to put in 2 end-of-buffer characters. + */ + b->yy_ch_buf = (char *) yyalloc( (yy_size_t) (b->yy_buf_size + 2) M4_YY_CALL_LAST_ARG ); + if ( b->yy_ch_buf == NULL ) { + YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); + } + b->yy_is_our_buffer = 1; + + yy_init_buffer( b, file M4_YY_CALL_LAST_ARG); + + return b; +} + +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +/** Delegate creation of buffers to the new version that takes an istream reference. + * @param file A readable stream. + * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. + * M4_YY_DOC_PARAM + * @return the allocated buffer state. + */ +yybuffer yyFlexLexer::yy_create_buffer( std::istream* file, int size ) +{ + return yy_create_buffer( *file, size ); +} +]]) + +/** Destroy the buffer. + * @param b a buffer created with yy_create_buffer() + * M4_YY_DOC_PARAM + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +void yy_delete_buffer YYFARGS1( yybuffer ,b) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yy_delete_buffer( yybuffer b ) +]]) +{ + M4_YY_DECL_GUTS_VAR(); + + if ( b == NULL ) { + return; + } + if ( b == yy_current_buffer() ) { /* Not sure if we should pop here. */ + YY_CURRENT_BUFFER_LVALUE = (yybuffer) 0; + } + if ( b->yy_is_our_buffer ) { + yyfree( (void *) b->yy_ch_buf M4_YY_CALL_LAST_ARG ); + } + yyfree( (void *) b M4_YY_CALL_LAST_ARG ); +} + + +/* Initializes or reinitializes a buffer. + * This function is sometimes called more than once on the same buffer, + * such as during a yyrestart() or at EOF. + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static void yy_init_buffer YYFARGS2( yybuffer ,b, FILE *,file) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yy_init_buffer( yybuffer b, std::istream& file ) +]]) +{ + int oerrno = errno; + M4_YY_DECL_GUTS_VAR(); + + yy_flush_buffer( b M4_YY_CALL_LAST_ARG); + +m4_ifdef([[M4_MODE_C_ONLY]], [[ + b->yy_input_file = file; +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + b->yy_input_file = file.rdbuf(); +]]) + b->yy_fill_buffer = 1; + + /* If b is the current buffer, then yy_init_buffer was _probably_ + * called from yyrestart() or through yy_get_next_buffer. + * In that case, we don't want to reset the lineno or column. + */ + if (b != yy_current_buffer()) { + b->yy_bs_lineno = 1; + b->yy_bs_column = 0; + } + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef( [[M4_YY_ALWAYS_INTERACTIVE]], +[[ + b->yy_is_interactive = 1; +]], +[[ + m4_ifdef( [[M4_YY_NEVER_INTERACTIVE]], + [[ + b->yy_is_interactive = 0; + ]], + [[ + b->yy_is_interactive = file ? (isatty( fileno(file) ) > 0) : 0; + ]]) +]]) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ + b->yy_is_interactive = 0; +]]) + errno = oerrno; +} + +/** Discard all buffered characters. On the next scan, YY_INPUT will be called. + * @param b the buffer state to be flushed, usually @c yy_current_buffer(). + * M4_YY_DOC_PARAM + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +void yy_flush_buffer YYFARGS1( yybuffer ,b) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yy_flush_buffer( yybuffer b ) +]]) +{ + M4_YY_DECL_GUTS_VAR(); + if ( b == NULL ) { + return; + } + b->yy_n_chars = 0; + + /* We always need two end-of-buffer characters. The first causes + * a transition to the end-of-buffer state. The second causes + * a jam in that state. + */ + b->yy_ch_buf[0] = YY_END_OF_BUFFER_CHAR; + b->yy_ch_buf[1] = YY_END_OF_BUFFER_CHAR; + + b->yy_buf_pos = &b->yy_ch_buf[0]; + + b->yyatbol = 1; + b->yy_buffer_status = YY_BUFFER_NEW; + + if ( b == yy_current_buffer() ) { + yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); + } +} + +/** Pushes the new state onto the stack. The new state becomes + * the current state. This function will allocate the stack + * if necessary. + * @param new_buffer The new state. + * M4_YY_DOC_PARAM + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +void yypush_buffer_state YYFARGS1(yybuffer,new_buffer) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yypush_buffer_state (yybuffer new_buffer) +]]) +{ + M4_YY_DECL_GUTS_VAR(); + if (new_buffer == NULL) { + return; + } + yyensure_buffer_stack(M4_YY_CALL_ONLY_ARG); + + /* This block is copied from yy_switch_to_buffer. */ + if ( yy_current_buffer() != NULL ) { + /* Flush out information for old buffer. */ + *YY_G(yy_c_buf_p) = YY_G(yy_hold_char); + YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = YY_G(yy_c_buf_p); + YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars); + } + + /* Only push if top exists. Otherwise, replace top. */ + if (yy_current_buffer()) { + YY_G(yy_buffer_stack_top)++; + } + YY_CURRENT_BUFFER_LVALUE = new_buffer; + + /* copied from yy_switch_to_buffer. */ + yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); + YY_G(yy_did_buffer_switch_on_eof) = 1; +} + + +/** Removes and deletes the top of the stack, if present. + * The next element becomes the new top. + * M4_YY_DOC_PARAM + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +void yypop_buffer_state (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yypop_buffer_state (void) +]]) +{ + M4_YY_DECL_GUTS_VAR(); + if (yy_current_buffer() == NULL) { + return; + } + yy_delete_buffer(yy_current_buffer() M4_YY_CALL_LAST_ARG); + YY_CURRENT_BUFFER_LVALUE = NULL; + if (YY_G(yy_buffer_stack_top) > 0) { + --YY_G(yy_buffer_stack_top); + } + if (yy_current_buffer() != NULL) { + yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); + YY_G(yy_did_buffer_switch_on_eof) = 1; + } +} + + +/* Allocates the stack if it does not exist. + * Guarantees space for at least one push. + */ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static void yyensure_buffer_stack (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yyensure_buffer_stack(void) +]]) +{ + yy_size_t num_to_alloc; + M4_YY_DECL_GUTS_VAR(); + + if (YY_G(yy_buffer_stack) == NULL) { + /* First allocation is just for 2 elements, since we don't know if this + * scanner will even need a stack. We use 2 instead of 1 to avoid an + * immediate realloc on the next call. + */ + num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */ + YY_G(yy_buffer_stack) = (struct yy_buffer_state**)yyalloc + (num_to_alloc * sizeof(struct yy_buffer_state*) + M4_YY_CALL_LAST_ARG); + if ( YY_G(yy_buffer_stack == NULL) ) { + YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); + } + + memset(YY_G(yy_buffer_stack), 0, num_to_alloc * sizeof(struct yy_buffer_state*)); + + YY_G(yy_buffer_stack_max) = num_to_alloc; + YY_G(yy_buffer_stack_top) = 0; + return; + } + + if (YY_G(yy_buffer_stack_top) >= (YY_G(yy_buffer_stack_max)) - 1) { + /* Increase the buffer to prepare for a possible push. */ + yy_size_t grow_size = 8 /* arbitrary grow size */; + + num_to_alloc = YY_G(yy_buffer_stack_max) + grow_size; + YY_G(yy_buffer_stack) = (struct yy_buffer_state**)yyrealloc + (YY_G(yy_buffer_stack), + num_to_alloc * sizeof(struct yy_buffer_state*) + M4_YY_CALL_LAST_ARG); + if (YY_G(yy_buffer_stack) == NULL) { + YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); + } + /* zero only the new slots.*/ + memset(YY_G(yy_buffer_stack) + YY_G(yy_buffer_stack_max), 0, grow_size * sizeof(struct yy_buffer_state*)); + YY_G(yy_buffer_stack_max) = num_to_alloc; + } +} + + + + +m4_ifdef( [[M4_YY_NO_SCAN_BUFFER]],, +[[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +/** Setup the input buffer state to scan directly from a user-specified character buffer. + * @param base the character buffer + * @param size the size in bytes of the character buffer + * M4_YY_DOC_PARAM + * @return the newly allocated buffer state object. + */ +yybuffer yy_scan_buffer YYFARGS2( char *,base, yy_size_t ,size) +{ + yybuffer b; + m4_dnl M4_YY_DECL_GUTS_VAR(); + + if ( size < 2 || + base[size-2] != YY_END_OF_BUFFER_CHAR || + base[size-1] != YY_END_OF_BUFFER_CHAR ) { + /* They forgot to leave room for the EOB's. */ + return NULL; + } + b = (yybuffer) yyalloc( sizeof( struct yy_buffer_state ) M4_YY_CALL_LAST_ARG ); + if ( b == NULL ) { + YY_FATAL_ERROR( "out of dynamic memory in yy_scan_buffer()" ); + } + b->yy_buf_size = (int) (size - 2); /* "- 2" to take care of EOB's */ + b->yy_buf_pos = b->yy_ch_buf = base; + b->yy_is_our_buffer = 0; + b->yy_input_file = NULL; + b->yy_n_chars = b->yy_buf_size; + b->yy_is_interactive = 0; + b->yyatbol = 1; + b->yy_fill_buffer = 0; + b->yy_buffer_status = YY_BUFFER_NEW; + + yy_switch_to_buffer( b M4_YY_CALL_LAST_ARG ); + + return b; +} +]]) +]]) + + +m4_ifdef( [[M4_YY_NO_SCAN_STRING]],, +[[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +/** Setup the input buffer state to scan a string. The next call to yylex() will + * scan from a @e copy of @a str. + * @param yystr a NUL-terminated string to scan + * M4_YY_DOC_PARAM + * @return the newly allocated buffer state object. + * @note If you want to scan bytes that may contain NUL values, then use + * yy_scan_bytes() instead. + */ +yybuffer yy_scan_string YYFARGS1( const char *, yystr) +{ + m4_dnl M4_YY_DECL_GUTS_VAR(); + + return yy_scan_bytes( yystr, (int) strlen(yystr) M4_YY_CALL_LAST_ARG); +} +]]) +]]) + + +m4_ifdef( [[M4_YY_NO_SCAN_BYTES]],, +[[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +/** Setup the input buffer state to scan the given bytes. The next call to yylex() will + * scan from a @e copy of @a bytes. + * @param yybytes the byte buffer to scan + * @param _yybytes_len the number of bytes in the buffer pointed to by @a bytes. + * M4_YY_DOC_PARAM + * @return the newly allocated buffer state object. + */ +yybuffer yy_scan_bytes YYFARGS2( const char *,yybytes, int ,_yybytes_len) { + yybuffer b; + char *buf; + yy_size_t n; + int i; + m4_dnl M4_YY_DECL_GUTS_VAR(); + + /* Get memory for full buffer, including space for trailing EOB's. */ + n = (yy_size_t) (_yybytes_len + 2); + buf = (char *) yyalloc( n M4_YY_CALL_LAST_ARG ); + if ( buf == 0 ) { + YY_FATAL_ERROR( "out of dynamic memory in yy_scan_bytes()" ); + } + for ( i = 0; i < _yybytes_len; ++i ) { + buf[i] = yybytes[i]; + } + buf[_yybytes_len] = buf[_yybytes_len+1] = YY_END_OF_BUFFER_CHAR; + + b = yy_scan_buffer( buf, n M4_YY_CALL_LAST_ARG); + if ( b == NULL ) { + YY_FATAL_ERROR( "bad buffer in yy_scan_bytes()" ); + } + /* It's okay to grow etc. this buffer, and we should throw it + * away when we're done. + */ + b->yy_is_our_buffer = 1; + + return b; +} +]]) +]]) + + +m4_ifdef( [[M4_YY_NO_PUSH_STATE]],, +[[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static void yy_push_state YYFARGS1( int ,_new_state) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yy_push_state( int _new_state ) +]]) +{ + M4_YY_DECL_GUTS_VAR(); + if ( YY_G(yy_start_stack_ptr) >= YY_G(yy_start_stack_depth) ) { + yy_size_t new_size; + + YY_G(yy_start_stack_depth) += YY_START_STACK_INCR; + new_size = (yy_size_t) YY_G(yy_start_stack_depth) * sizeof( int ); + + if ( ! YY_G(yy_start_stack) ) { + YY_G(yy_start_stack) = (int *) yyalloc( new_size M4_YY_CALL_LAST_ARG ); + + } else { + YY_G(yy_start_stack) = (int *) yyrealloc( + (void *) YY_G(yy_start_stack), new_size M4_YY_CALL_LAST_ARG ); + } + if ( ! YY_G(yy_start_stack) ) { + YY_FATAL_ERROR( "out of memory expanding start-condition stack" ); + } + } + YY_G(yy_start_stack)[YY_G(yy_start_stack_ptr)++] = yystart(); + + yybegin(_new_state); +} +]]) + + +m4_ifdef( [[M4_YY_NO_POP_STATE]],, +[[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static void yy_pop_state (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +void yyFlexLexer::yy_pop_state() +]]) +{ + M4_YY_DECL_GUTS_VAR(); + if ( --YY_G(yy_start_stack_ptr) < 0 ) { + YY_FATAL_ERROR( "start-condition stack underflow" ); + } + yybegin(YY_G(yy_start_stack)[YY_G(yy_start_stack_ptr)]); +} +]]) + + +m4_ifdef( [[M4_YY_NO_TOP_STATE]],, +[[ +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static int yy_top_state (M4_YY_DEF_ONLY_ARG) +]]) +m4_ifdef([[M4_MODE_CXX_ONLY]], [[ +int yyFlexLexer::yy_top_state() +]]) +{ + M4_YY_DECL_GUTS_VAR(); + return YY_G(yy_start_stack_ptr) > 0 ? YY_G(yy_start_stack)[YY_G(yy_start_stack_ptr) - 1] : yystart(); +} +]]) + +/* Redefine yyless() so it works in section 3 code. */ + +#undef yyless +#define yyless(n) \ + do { \ + /* Undo effects of setting up yytext. */ \ + int yyless_macro_arg = (n); \ + YY_LESS_LINENO(yyless_macro_arg);\ + yytext[yyleng] = YY_G(yy_hold_char); \ + YY_G(yy_c_buf_p) = yytext + yyless_macro_arg; \ + YY_G(yy_hold_char) = *YY_G(yy_c_buf_p); \ + *YY_G(yy_c_buf_p) = '\0'; \ + yyleng = yyless_macro_arg; \ + } while ( 0 ) + + + +/* Accessor methods (get/set functions) to struct members. */ + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +m4_ifdef([[M4_YY_REENTRANT]], [[ +m4_ifdef( [[M4_YY_NO_GET_EXTRA]],, +[[ +/** Get the user-defined data for this scanner. + * M4_YY_DOC_PARAM + */ +YY_EXTRA_TYPE yyget_extra (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yyextra; +} +]]) +]]) + +m4_ifdef( [[M4_YY_NO_GET_LINENO]],, +[[ +/** Get the current line number. + * M4_YY_DOC_PARAM + */ +int yyget_lineno (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + + m4_ifdef( [[M4_YY_REENTRANT]], + [[ + if (yy_current_buffer() == NULL) { + return 0; + } + ]]) + return yylineno; +} +]]) + +m4_ifdef( [[M4_YY_REENTRANT]], +[[ +m4_ifdef( [[M4_YY_NO_GET_COLUMN]],, +[[ +/** Get the current column number. + * M4_YY_DOC_PARAM + */ +int yyget_column (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + +m4_ifdef( [[M4_YY_REENTRANT]], [[ + if (yy_current_buffer() == NULL) { + return 0; + } +]]) + return yycolumn; +} +]]) +]]) + +m4_ifdef( [[M4_YY_NO_GET_IN]],, +[[ +/** Get the input stream. + * M4_YY_DOC_PARAM + */ +FILE *yyget_in (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yyin; +} +]]) + +m4_ifdef( [[M4_YY_NO_GET_OUT]],, +[[ +/** Get the output stream. + * M4_YY_DOC_PARAM + */ +FILE *yyget_out (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yyout; +} +]]) + +m4_ifdef( [[M4_YY_NO_GET_LENG]],, +[[ +/** Get the length of the current token. + * M4_YY_DOC_PARAM + */ +int yyget_leng (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yyleng; +} +]]) + +/** Get the current token. + * M4_YY_DOC_PARAM + */ +m4_ifdef( [[M4_YY_NO_GET_TEXT]],, +[[ +char *yyget_text (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yytext; +} +]]) + +m4_ifdef([[M4_YY_REENTRANT]], [[ +m4_ifdef( [[M4_YY_NO_SET_EXTRA]],, +[[ +/** Set the user-defined data. This data is never touched by the scanner. + * @param user_defined The data to be associated with this scanner. + * M4_YY_DOC_PARAM + */ +void yyset_extra YYFARGS1( YY_EXTRA_TYPE ,user_defined) { + M4_YY_DECL_GUTS_VAR(); + yyextra = user_defined ; +} +]]) +]]) + +m4_ifdef( [[M4_YY_NO_SET_LINENO]],, +[[ +/** Set the current line number. + * @param _line_number line number + * M4_YY_DOC_PARAM + */ +void yyset_lineno YYFARGS1( int ,_line_number) { + M4_YY_DECL_GUTS_VAR(); + + m4_ifdef( [[M4_YY_REENTRANT]], + [[ + /* lineno is only valid if an input buffer exists. */ + if (yy_current_buffer() == NULL ) { + YY_FATAL_ERROR( "yyset_lineno called with no buffer" ); + } + ]]) + yylineno = _line_number; +} +]]) + +m4_ifdef( [[M4_YY_REENTRANT]], +[[ +m4_ifdef( [[M4_YY_NO_SET_COLUMN]],, +[[ +/** Set the current column. + * @param _column_no column number + * M4_YY_DOC_PARAM + */ +void yyset_column YYFARGS1( int , _column_no) { + M4_YY_DECL_GUTS_VAR(); + +m4_ifdef( [[M4_YY_REENTRANT]], [[ + /* column is only valid if an input buffer exists. */ + if (yy_current_buffer() == NULL ) { + YY_FATAL_ERROR( "yyset_column called with no buffer" ); + } +]]) + yycolumn = _column_no; +} +]]) +]]) + + +m4_ifdef( [[M4_YY_NO_SET_IN]],, +[[ +/** Set the input stream. This does not discard the current + * input buffer. + * @param _in_str A readable stream. + * M4_YY_DOC_PARAM + * @see yy_switch_to_buffer + */ +void yyset_in YYFARGS1( FILE * ,_in_str) { + M4_YY_DECL_GUTS_VAR(); + yyin = _in_str ; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_OUT]],, +[[ +void yyset_out YYFARGS1( FILE * ,_out_str) { + M4_YY_DECL_GUTS_VAR(); + yyout = _out_str ; +} +]]) + + +m4_ifdef( [[M4_YY_NO_GET_DEBUG]],, +[[ +int yyget_debug (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yyflexdebug; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_DEBUG]],, +[[ +void yyset_debug YYFARGS1( int ,_bdebug) { + M4_YY_DECL_GUTS_VAR(); + yyflexdebug = _bdebug ; +} +]]) +]]) + +m4_ifdef([[M4_YY_REENTRANT]], [[ +/* Accessor methods for yylval and yylloc */ + +m4_ifdef([[M4_YY_BISON_LVAL]], [[ +m4_ifdef( [[M4_YY_NO_GET_LVAL]],, +[[ +YYSTYPE * yyget_lval (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yylval; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_LVAL]],, +[[ +void yyset_lval YYFARGS1( YYSTYPE * ,yylval_param) { + M4_YY_DECL_GUTS_VAR(); + yylval = yylval_param; +} +]]) + +m4_ifdef( [[<M4_YY_BISON_LLOC>]], +[[ +m4_ifdef( [[M4_YY_NO_GET_LLOC]],, +[[ +YYLTYPE *yyget_lloc (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + return yylloc; +} +]]) + +m4_ifdef( [[M4_YY_NO_SET_LLOC]],, +[[ +void yyset_lloc YYFARGS1( YYLTYPE * ,yylloc_param) { + M4_YY_DECL_GUTS_VAR(); + yylloc = yylloc_param; +} +]]) +]]) + +]]) + + +/* User-visible API */ + +/* yylex_init is special because it creates the scanner itself, so it is + * the ONLY reentrant function that doesn't take the scanner as the last argument. + * That's why we explicitly handle the declaration, instead of using our macros. + */ +int yylex_init(yyscan_t* ptr_yy_globals) { + if (ptr_yy_globals == NULL) { + errno = EINVAL; + return 1; + } + + *ptr_yy_globals = (yyscan_t) yyalloc ( sizeof( struct yyguts_t ), NULL ); + + if (*ptr_yy_globals == NULL) { + errno = ENOMEM; + return 1; + } + + /* By setting to 0xAA, we expose bugs in yy_init_globals. Leave at 0x00 for releases. */ + memset(*ptr_yy_globals,0x00,sizeof(struct yyguts_t)); + + return yy_init_globals ( *ptr_yy_globals ); +} + + +/* yylex_init_extra has the same functionality as yylex_init, but follows the + * convention of taking the scanner as the last argument. Note however, that + * this is a *pointer* to a scanner, as it will be allocated by this call (and + * is the reason, too, why this function also must handle its own declaration). + * The user defined value in the first argument will be available to yyalloc in + * the yyextra field. + */ +int yylex_init_extra( YY_EXTRA_TYPE yy_user_defined, yyscan_t* ptr_yy_globals ) { + struct yyguts_t dummy_yyguts; + + yyset_extra (yy_user_defined, &dummy_yyguts); + + if (ptr_yy_globals == NULL) { + errno = EINVAL; + return 1; + } + + *ptr_yy_globals = (yyscan_t) yyalloc ( sizeof( struct yyguts_t ), &dummy_yyguts ); + + if (*ptr_yy_globals == NULL) { + errno = ENOMEM; + return 1; + } + + /* By setting to 0xAA, we expose bugs in + yy_init_globals. Leave at 0x00 for releases. */ + memset(*ptr_yy_globals,0x00,sizeof(struct yyguts_t)); + + yyset_extra (yy_user_defined, *ptr_yy_globals); + + return yy_init_globals ( *ptr_yy_globals ); +} + +]]) +%# Actually, that ended an if-rentrant section + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +static int yy_init_globals (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + /* Initialization is the same as for the non-reentrant scanner. + * This function is called from yylex_destroy(), so don't allocate here. + */ + +m4_ifdef( [[M4_MODE_YYLINENO]], +[[ + m4_ifdef( [[M4_YY_NOT_REENTRANT]], + [[ + /* We do not touch yylineno unless the option is enabled. */ + yylineno = 1; + ]]) +]]) + YY_G(yy_buffer_stack) = NULL; + YY_G(yy_buffer_stack_top) = 0; + YY_G(yy_buffer_stack_max) = 0; + YY_G(yy_c_buf_p) = NULL; + YY_G(yy_init) = 0; + YY_G(yy_start) = 0; + +m4_ifdef( [[M4_YY_HAS_START_STACK_VARS]], +[[ + YY_G(yy_start_stack_ptr) = 0; + YY_G(yy_start_stack_depth) = 0; + YY_G(yy_start_stack) = NULL; +]]) + +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + YY_G(yy_state_buf) = 0; + YY_G(yy_state_ptr) = 0; + YY_G(yy_full_match) = 0; + YY_G(yy_lp) = 0; +]]) + +m4_ifdef( [[M4_MODE_REENTRANT_TEXT_IS_ARRAY]], +[[ + YY_G(yytext_ptr) = 0; + YY_G(yy_more_offset) = 0; + YY_G(yy_prev_more_offset) = 0; +]]) + +/* Defined in main.c */ +#ifdef YY_STDINIT + yyin = stdin; + yyout = stdout; +#else + yyin = NULL; + yyout = NULL; +#endif + + /* For future reference: Set errno on error, since we are called by + * yylex_init() + */ + return 0; +} +]]) + + +m4_ifdef([[M4_MODE_C_ONLY]], [[ +%# SNIP! this currently causes conflicts with the c++ scanner +/* yylex_destroy is for both reentrant and non-reentrant scanners. */ +int yylex_destroy (M4_YY_DEF_ONLY_ARG) { + M4_YY_DECL_GUTS_VAR(); + + /* Pop the buffer stack, destroying each element. */ + while(yy_current_buffer()) { + yy_delete_buffer( yy_current_buffer() M4_YY_CALL_LAST_ARG ); + YY_CURRENT_BUFFER_LVALUE = NULL; + yypop_buffer_state(M4_YY_CALL_ONLY_ARG); + } + + /* Destroy the stack itself. */ + yyfree(YY_G(yy_buffer_stack) M4_YY_CALL_LAST_ARG); + YY_G(yy_buffer_stack) = NULL; + +m4_ifdef( [[M4_YY_HAS_START_STACK_VARS]], +[[ + /* Destroy the start condition stack. */ + yyfree( YY_G(yy_start_stack) M4_YY_CALL_LAST_ARG ); + YY_G(yy_start_stack) = NULL; +]]) + +m4_ifdef( [[M4_MODE_USES_REJECT]], +[[ + yyfree ( YY_G(yy_state_buf) M4_YY_CALL_LAST_ARG); + YY_G(yy_state_buf) = NULL; +]]) + + /* Reset the globals. This is important in a non-reentrant scanner so the next time + * yylex() is called, initialization will occur. */ + yy_init_globals( M4_YY_CALL_ONLY_ARG); + +m4_ifdef([[M4_YY_REENTRANT]], [[ + /* Destroy the main struct (reentrant only). */ + yyfree ( yyscanner M4_YY_CALL_LAST_ARG ); + yyscanner = NULL; +]]) + return 0; +} +]]) + + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +/* + * Internal utility routines. + */ +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +#ifndef yytext_ptr +static void yy_flex_strncpy YYFARGS3( char*,s1, const char *,s2, int,n) { + M4_YY_DECL_GUTS_VAR(); + M4_YY_NOOP_GUTS_VAR(); + + int i; + for ( i = 0; i < n; ++i ) { + s1[i] = s2[i]; + } +} +#endif +]]) + +m4_ifdef( [[M4_YY_NOT_IN_HEADER]], +[[ +#ifdef YY_NEED_STRLEN +static int yy_flex_strlen YYFARGS1( const char *,s) +{ + int n; + for ( n = 0; s[n]; ++n ) + ; + + return n; +} +#endif +]]) + +m4_ifdef( [[M4_YY_NO_FLEX_ALLOC]],, +[[ +void *yyalloc YYFARGS1( yy_size_t ,size) { + M4_YY_DECL_GUTS_VAR(); + M4_YY_NOOP_GUTS_VAR(); + return malloc(size); +} +]]) + +m4_ifdef( [[M4_YY_NO_FLEX_REALLOC]],, +[[ +void *yyrealloc YYFARGS2( void *,ptr, yy_size_t ,size) { + M4_YY_DECL_GUTS_VAR(); + M4_YY_NOOP_GUTS_VAR(); + + /* The cast to (char *) in the following accommodates both + * implementations that use char* generic pointers, and those + * that use void* generic pointers. It works with the latter + * because both ANSI C and C++ allow castless assignment from + * any pointer type to void*, and deal with argument conversions + * as though doing an assignment. + */ + return realloc(ptr, size); +} +]]) + +m4_ifdef( [[M4_YY_NO_FLEX_FREE]],, +[[ +void yyfree YYFARGS1( void *,ptr) { + M4_YY_DECL_GUTS_VAR(); + M4_YY_NOOP_GUTS_VAR(); + free( (char *) ptr ); /* see yyrealloc() for (char *) cast */ +} +]]) + +m4_ifdef([[M4_MODE_TABLESEXT]], [[ +%# definitions +m4preproc_include(`tables_shared.c') + +static int yytbl_read8 (void *v, struct yytbl_reader * rd) { + errno = 0; + if (fread (v, sizeof (flex_uint8_t), 1, rd->fp) != 1) { + errno = EIO; + return -1; + } + rd->bread += (flex_uint32_t) sizeof(flex_uint8_t); + return 0; +} + +static int yytbl_read16 (void *v, struct yytbl_reader * rd) { + errno = 0; + if (fread (v, sizeof (flex_uint16_t), 1, rd->fp) != 1) { + errno = EIO; + return -1; + } + *((flex_uint16_t *) v) = ntohs (*((flex_uint16_t *) v)); + rd->bread += (flex_uint32_t) sizeof(flex_uint16_t); + return 0; +} + +static int yytbl_read32 (void *v, struct yytbl_reader * rd) { + errno = 0; + if (fread (v, sizeof (flex_uint32_t), 1, rd->fp) != 1) { + errno = EIO; + return -1; + } + *((flex_uint32_t *) v) = ntohl (*((flex_uint32_t *) v)); + rd->bread += (flex_uint32_t) sizeof(flex_uint32_t); + return 0; +} + +/** Read the header */ +static int yytbl_hdr_read YYFARGS2(struct yytbl_hdr *, th, struct yytbl_reader *, rd) { + size_t bytes; + memset (th, 0, sizeof (struct yytbl_hdr)); + + if (yytbl_read32 (&(th->th_magic), rd) != 0) { + return -1; + } + if (th->th_magic != YYTBL_MAGIC) { + YY_FATAL_ERROR( "bad magic number" ); /* TODO: not fatal. */ + return -1; + } + + if (yytbl_read32 (&(th->th_hsize), rd) != 0 + || yytbl_read32 (&(th->th_ssize), rd) != 0 + || yytbl_read16 (&(th->th_flags), rd) != 0) { + return -1; + } + /* Sanity check on header size. Greater than 1k suggests some funny business. */ + if (th->th_hsize < 16 || th->th_hsize > 1024) { + YY_FATAL_ERROR( "insane header size detected" ); /* TODO: not fatal. */ + return -1; + } + + /* Allocate enough space for the version and name fields */ + bytes = th->th_hsize - 14; + th->th_version = (char *) yyalloc (bytes M4_YY_CALL_LAST_ARG); + if ( ! th->th_version ) { + YY_FATAL_ERROR( "out of dynamic memory in yytbl_hdr_read()" ); + } + /* we read it all into th_version, and point th_name into that data */ + if (fread (th->th_version, 1, bytes, rd->fp) != bytes) { + errno = EIO; + yyfree(th->th_version M4_YY_CALL_LAST_ARG); + th->th_version = NULL; + return -1; + } else { + rd->bread += (flex_uint32_t) bytes; + } + th->th_name = th->th_version + strlen (th->th_version) + 1; + return 0; +} + +/** lookup id in the dmap list. + * @param dmap pointer to first element in list + * @return NULL if not found. + */ +static struct yytbl_dmap *yytbl_dmap_lookup YYFARGS2(struct yytbl_dmap *, dmap, int, id) { + M4_YY_DECL_GUTS_VAR(); + M4_YY_NOOP_GUTS_VAR(); + + while (dmap->dm_id) { + if ((int)(dmap->dm_id) == id) { + return dmap; + } else { + dmap++; + } + } + return NULL; +} + +/** Read a table while mapping its contents to the local array. + * @param dmap used to performing mapping + * @return 0 on success + */ +static int yytbl_data_load YYFARGS2(struct yytbl_dmap *, dmap, struct yytbl_reader*, rd) { + struct yytbl_data td; + struct yytbl_dmap *transdmap=0; + int len, i, rv, inner_loop_count; + void *p=0; + + memset (&td, 0, sizeof (struct yytbl_data)); + + if (yytbl_read16 (&td.td_id, rd) != 0 + || yytbl_read16 (&td.td_flags, rd) != 0 + || yytbl_read32 (&td.td_hilen, rd) != 0 + || yytbl_read32 (&td.td_lolen, rd) != 0) { + return -1; + } + /* Lookup the map for the transition table so we have it in case we need it + * inside the loop below. This scanner might not even have a transition + * table, which is ok. + */ + transdmap = yytbl_dmap_lookup (dmap, YYTD_ID_TRANSITION M4_YY_CALL_LAST_ARG); + + if ((dmap = yytbl_dmap_lookup (dmap, td.td_id M4_YY_CALL_LAST_ARG)) == NULL) { + YY_FATAL_ERROR( "table id not found in map." ); /* TODO: not fatal. */ + return -1; + } + + /* Allocate space for table. + * The --full yy_transition table is a special case, since we + * need the dmap.dm_sz entry to tell us the sizeof the individual + * struct members. + */ + { + size_t bytes; + + if ((td.td_flags & YYTD_STRUCT)) { + bytes = sizeof(struct yy_trans_info) * td.td_lolen * (td.td_hilen ? td.td_hilen : 1); + } else { + bytes = td.td_lolen * (td.td_hilen ? td.td_hilen : 1) * dmap->dm_sz; + } + if (M4_YY_TABLES_VERIFY) { + /* We point to the array itself */ + p = dmap->dm_arr; + } else { + /* We point to the address of a pointer. */ + *dmap->dm_arr = p = (void *) yyalloc (bytes M4_YY_CALL_LAST_ARG); + } + if ( ! p ) { + YY_FATAL_ERROR( "out of dynamic memory in yytbl_data_load()" ); + } + } + + /* If it's a struct, we read 2 integers to get one element */ + if ((td.td_flags & YYTD_STRUCT) != 0) { + inner_loop_count = 2; + } else { + inner_loop_count = 1; + } + /* read and map each element. + * This loop iterates once for each element of the td_data array. + * Notice that we increment 'i' in the inner loop. + */ + len = yytbl_calc_total_len (&td); + for (i = 0; i < len; ) { + int j; + + /* This loop really executes exactly 1 or 2 times. + * The second time is to handle the second member of the + * YYTD_STRUCT for the yy_transition array. + */ + for (j = 0; j < inner_loop_count; j++, i++) { + flex_int32_t t32; + + /* read into t32 no matter what the real size is. */ + { + flex_int16_t t16; + flex_int8_t t8; + + switch (YYTDFLAGS2BYTES (td.td_flags)) { + case sizeof (flex_int32_t): + rv = yytbl_read32 (&t32, rd); + break; + case sizeof (flex_int16_t): + rv = yytbl_read16 (&t16, rd); + t32 = t16; + break; + case sizeof (flex_int8_t): + rv = yytbl_read8 (&t8, rd); + t32 = t8; + break; + default: + YY_FATAL_ERROR( "invalid td_flags" ); /* TODO: not fatal. */ + return -1; + } + } + if (rv != 0) { + return -1; + } + /* copy into the deserialized array... */ + + if ((td.td_flags & YYTD_STRUCT)) { + /* t32 is the j'th member of a two-element struct. */ + void *v; + + v = j == 0 ? &(((struct yy_trans_info *) p)->yy_verify) + : &(((struct yy_trans_info *) p)->yy_nxt); + + switch (dmap->dm_sz) { + case sizeof (flex_int32_t): + if (M4_YY_TABLES_VERIFY) { + if( ((flex_int32_t *) v)[0] != (flex_int32_t) t32) + YY_FATAL_ERROR( "tables verification failed at YYTD_STRUCT flex_int32_t" ); + } else { + ((flex_int32_t *) v)[0] = (flex_int32_t) t32; + } + break; + case sizeof (flex_int16_t): + if (M4_YY_TABLES_VERIFY ) { + if(((flex_int16_t *) v)[0] != (flex_int16_t) t32) + YY_FATAL_ERROR( "tables verification failed at YYTD_STRUCT flex_int16_t" ); + } else { + ((flex_int16_t *) v)[0] = (flex_int16_t) t32; + } + break; + case sizeof(flex_int8_t): + if (M4_YY_TABLES_VERIFY ) { + if( ((flex_int8_t *) v)[0] != (flex_int8_t) t32) + YY_FATAL_ERROR( "tables verification failed at YYTD_STRUCT flex_int8_t" ); + } else { + ((flex_int8_t *) v)[0] = (flex_int8_t) t32; + } + break; + default: + YY_FATAL_ERROR( "invalid dmap->dm_sz for struct" ); /* TODO: not fatal. */ + return -1; + } + + /* if we're done with j, increment p */ + if (j == 1) { + p = (struct yy_trans_info *) p + 1; + } + } + else if ((td.td_flags & YYTD_PTRANS)) { + /* t32 is an index into the transition array. */ + struct yy_trans_info *v; + + if (!transdmap) { + YY_FATAL_ERROR( "transition table not found" ); /* TODO: not fatal. */ + return -1; + } + + if( M4_YY_TABLES_VERIFY) { + v = &(((struct yy_trans_info *) (transdmap->dm_arr))[t32]); + } else { + v = &((*((struct yy_trans_info **) (transdmap->dm_arr)))[t32]); + } + if(M4_YY_TABLES_VERIFY ) { + if( ((struct yy_trans_info **) p)[0] != v) + YY_FATAL_ERROR( "tables verification failed at YYTD_PTRANS" ); + } else { + ((struct yy_trans_info **) p)[0] = v; + } + /* increment p */ + p = (struct yy_trans_info **) p + 1; + } + else { + /* t32 is a plain int. copy data, then incrememnt p. */ + switch (dmap->dm_sz) { + case sizeof (flex_int32_t): + if(M4_YY_TABLES_VERIFY ) { + if( ((flex_int32_t *) p)[0] != (flex_int32_t) t32) { + YY_FATAL_ERROR( "tables verification failed at flex_int32_t" ); + } + } else { + ((flex_int32_t *) p)[0] = (flex_int32_t) t32; + } + p = ((flex_int32_t *) p) + 1; + break; + case sizeof (flex_int16_t): + if(M4_YY_TABLES_VERIFY ) { + if( ((flex_int16_t *) p)[0] != (flex_int16_t) t32) { + YY_FATAL_ERROR( "tables verification failed at flex_int16_t" ); + } + } else { + ((flex_int16_t *) p)[0] = (flex_int16_t) t32; + } + p = ((flex_int16_t *) p) + 1; + break; + case sizeof (flex_int8_t): + if(M4_YY_TABLES_VERIFY ){ + if( ((flex_int8_t *) p)[0] != (flex_int8_t) t32) { + YY_FATAL_ERROR( "tables verification failed at flex_int8_t" ); + } + } else { + ((flex_int8_t *) p)[0] = (flex_int8_t) t32; + } + p = ((flex_int8_t *) p) + 1; + break; + default: + YY_FATAL_ERROR( "invalid dmap->dm_sz for plain int" ); /* TODO: not fatal. */ + return -1; + } + } + } + + } + + /* Now eat padding. */ + { + while (rd->bread % (8 * sizeof(flex_uint8_t)) > 0) { + flex_int8_t t8; + if(yytbl_read8(&t8,rd) != 0) + return -1; + } + } + + return 0; +} + +/* The name for this specific scanner's tables. */ +#define YYTABLES_NAME "m4_ifdef([[M4_MODE_PREFIX]], M4_MODE_PREFIX, [[yy]])tables" + +/* Find the key and load the DFA tables from the given stream. */ +static int yytbl_fload YYFARGS2(FILE *, fp, const char *, key) { + int rv=0; + struct yytbl_hdr th; + struct yytbl_reader rd; + + rd.fp = fp; + th.th_version = NULL; + + /* Keep trying until we find the right set of tables or end of file. */ + while (!feof(rd.fp)) { + rd.bread = 0; + if (yytbl_hdr_read (&th, &rd M4_YY_CALL_LAST_ARG) != 0) { + rv = -1; + goto return_rv; + } + + /* A NULL key means choose the first set of tables. */ + if (key == NULL) { + break; + } + + if (strcmp(th.th_name,key) != 0) { + /* Skip ahead to next set */ + fseek(rd.fp, th.th_ssize - th.th_hsize, SEEK_CUR); + yyfree(th.th_version M4_YY_CALL_LAST_ARG); + th.th_version = NULL; + } + else { + break; + } + } + + while (rd.bread < th.th_ssize) { + /* Load the data tables */ + if(yytbl_data_load (yydmap,&rd M4_YY_CALL_LAST_ARG) != 0){ + rv = -1; + goto return_rv; + } + } + +return_rv: + if(th.th_version) { + yyfree(th.th_version M4_YY_CALL_LAST_ARG); + th.th_version = NULL; + } + + return rv; +} + +/** Load the DFA tables for this scanner from the given stream. */ +int yytables_fload YYFARGS1(FILE *, fp) { + if( yytbl_fload(fp, YYTABLES_NAME M4_YY_CALL_LAST_ARG) != 0) { + return -1; + } + return 0; +} + +/** Destroy the loaded tables, freeing memory, etc.. */ +int yytables_destroy (M4_YY_DEF_ONLY_ARG) { + struct yytbl_dmap *dmap=0; + + if(!M4_YY_TABLES_VERIFY){ + /* Walk the dmap, freeing the pointers */ + for(dmap=yydmap; dmap->dm_id; dmap++) { + void * v; + v = dmap->dm_arr; + if(v && *(char**)v){ + yyfree(*(char**)v M4_YY_CALL_LAST_ARG); + *(char**)v = NULL; + } + } + } + + return 0; +} + +/* end table serialization code definitions */ +]]) + + +m4_ifdef([[M4_YY_MAIN]], [[ +int main (void); + +int main () { +m4_ifdef([[M4_YY_REENTRANT]], [[ + yyscan_t lexer; + yylex_init(&lexer); + yylex( lexer ); + yylex_destroy( lexer); + +]]) +m4_ifdef([[M4_YY_NOT_REENTRANT]], [[ + yylex(); +]]) + + return 0; +} +]]) + +]]) +m4_ifdef( [[M4_YY_IN_HEADER]], +[[ +#undef YY_NEW_FILE +#undef YY_FLUSH_BUFFER +#undef yysetbol +#undef yy_new_buffer +#undef yy_set_interactive +#undef YY_DO_BEFORE_ACTION + +#ifdef YY_DECL_IS_OURS +#undef YY_DECL_IS_OURS +#undef YY_DECL +#endif +m4preproc_undivert(1) +#undef M4_MODE_PREFIX[[IN_HEADER]] +#endif /* M4_MODE_PREFIX[[HEADER_H]] */ +m4_undefine([[M4_YY_IN_HEADER]])m4_dnl +]]) + +%# Local Variables: +%# mode:c +%# c-file-style:"k&r" +%# c-basic-offset:8 +%# End: @@ -54,19 +54,19 @@ void check_for_backing_up (int ds, int state[]) if ((reject && !dfaacc[ds].dfaacc_set) || (!reject && !dfaacc[ds].dfaacc_state)) { /* state is non-accepting */ ++num_backing_up; - if (backing_up_report) { - fprintf (backing_up_file, + if (env.backing_up_report) { + fprintf (ctrl.backing_up_file, _("State #%d is non-accepting -\n"), ds); /* identify the state */ - dump_associated_rules (backing_up_file, ds); + dump_associated_rules (ctrl.backing_up_file, ds); /* Now identify it further using the out- and * jam-transitions. */ - dump_transitions (backing_up_file, state); + dump_transitions (ctrl.backing_up_file, state); - putc ('\n', backing_up_file); + putc ('\n', ctrl.backing_up_file); } } } @@ -185,7 +185,7 @@ void dump_transitions (FILE *file, int state[]) int i, ec; int out_char_set[CSIZE]; - for (i = 0; i < csize; ++i) { + for (i = 0; i < ctrl.csize; ++i) { ec = ABS (ecgroup[i]); out_char_set[i] = state[ec]; } @@ -195,7 +195,7 @@ void dump_transitions (FILE *file, int state[]) list_character_set (file, out_char_set); /* now invert the members of the set to get the jam transitions */ - for (i = 0; i < csize; ++i) + for (i = 0; i < ctrl.csize; ++i) out_char_set[i] = !out_char_set[i]; fprintf (file, _("\n jam-transitions: EOF ")); @@ -366,9 +366,12 @@ void increase_max_dfas (void) * * Creates the dfa corresponding to the ndfa we've constructed. The * dfa starts out in state #1. + * + * Return the amount of space, in bytes, allocated for the next table. + * In some modes this can be zero. */ -void ntod (void) +size_t ntod (void) { int *accset, ds, nacc, newds; int sym, hashval, numstates, dsize; @@ -406,7 +409,7 @@ void ntod (void) */ todo_head = todo_next = 0; - for (i = 0; i <= csize; ++i) { + for (i = 0; i <= ctrl.csize; ++i) { duplist[i] = NIL; symlist[i] = false; } @@ -414,7 +417,7 @@ void ntod (void) for (i = 0; i <= num_rules; ++i) accset[i] = NIL; - if (trace) { + if (env.trace) { dumpnfa (scset[1]); fputs (_("\n\nDFA Dump:\n\n"), stderr); } @@ -452,33 +455,14 @@ void ntod (void) /* Note that the test for ecgroup[0] == numecs below accomplishes * both (1) and (2) above + * + * New way: we will only use NUL table for fulltbl, because the + * scanner will use an integer instead of YY_CHAR as noted above */ - if (!fullspd && ecgroup[0] == numecs) { - /* NUL is alone in its equivalence class, which is the - * last one. - */ - int use_NUL_table = (numecs == csize); - - if (fulltbl && !use_NUL_table) { - /* We still may want to use the table if numecs - * is a power of 2. - */ - if (numecs <= csize && is_power_of_2(numecs)) { - use_NUL_table = true; - } - } - - if (use_NUL_table) - nultrans = - allocate_integer_array (current_max_dfas); - - /* From now on, nultrans != nil indicates that we're - * saving null transitions for later, separate encoding. - */ - } + if (ctrl.fulltbl && ecgroup[0] == numecs && is_power_of_2(numecs)) + nultrans = allocate_integer_array (current_max_dfas); - - if (fullspd) { + if (ctrl.fullspd) { for (i = 0; i <= numecs; ++i) state[i] = 0; @@ -486,7 +470,7 @@ void ntod (void) dfaacc[0].dfaacc_state = 0; } - else if (fulltbl) { + else if (ctrl.fulltbl) { if (nultrans) /* We won't be including NUL's transitions in the * table, so build it for entries from 0 .. numecs - 1. @@ -512,32 +496,19 @@ void ntod (void) yynxt_tbl->td_hilen = 1; yynxt_tbl->td_lolen = (flex_uint32_t) num_full_table_rows; yynxt_tbl->td_data = yynxt_data = - calloc(yynxt_tbl->td_lolen * - yynxt_tbl->td_hilen, - sizeof (flex_int32_t)); + calloc(yynxt_tbl->td_lolen * + yynxt_tbl->td_hilen, + sizeof (flex_int32_t)); yynxt_curr = 0; - buf_prints (&yydmap_buf, - "\t{YYTD_ID_NXT, (void**)&yy_nxt, sizeof(%s)},\n", - long_align ? "flex_int32_t" : "flex_int16_t"); - - /* Unless -Ca, declare it "short" because it's a real - * long-shot that that won't be large enough. - */ + struct packtype_t *ptype = optimize_pack(0); + /* Note: Used when ctrl.fulltbl is on. Alternately defined elsewhere */ + out_str ("m4_define([[M4_HOOK_NXT_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_NXT_ROWS]], [[%d]])", num_full_table_rows); + outn ("m4_define([[M4_HOOK_NXT_BODY]], [[m4_dnl"); + outn ("M4_HOOK_TABLE_OPENER"); if (gentables) - out_str_dec - ("static const %s yy_nxt[][%d] =\n {\n", - long_align ? "flex_int32_t" : "flex_int16_t", - num_full_table_rows); - else { - out_dec ("#undef YY_NXT_LOLEN\n#define YY_NXT_LOLEN (%d)\n", num_full_table_rows); - out_str ("static const %s *yy_nxt =0;\n", - long_align ? "flex_int32_t" : "flex_int16_t"); - } - - - if (gentables) - outn (" {"); + outn ("M4_HOOK_TABLE_OPENER"); /* Generate 0 entries for state #0. */ for (i = 0; i < num_full_table_rows; ++i) { @@ -547,7 +518,7 @@ void ntod (void) dataflush (); if (gentables) - outn (" },\n"); + outn ("M4_HOOK_TABLE_CONTINUE"); } /* Create the first states. */ @@ -581,7 +552,7 @@ void ntod (void) } } - if (!fullspd) { + if (!ctrl.fullspd) { if (!snstods (nset, 0, accset, 0, 0, &end_of_buffer_state)) flexfatal (_ ("could not create unique end-of-buffer state")); @@ -604,7 +575,7 @@ void ntod (void) dset = dss[ds]; dsize = dfasiz[ds]; - if (trace) + if (env.trace) fprintf (stderr, _("state # %d:\n"), ds); sympartition (dset, dsize, symlist, duplist); @@ -641,7 +612,7 @@ void ntod (void) state[sym] = newds; - if (trace) + if (env.trace) fprintf (stderr, "\t%d\t%d\n", sym, newds); @@ -659,7 +630,7 @@ void ntod (void) targ = state[duplist[sym]]; state[sym] = targ; - if (trace) + if (env.trace) fprintf (stderr, "\t%d\t%d\n", sym, targ); @@ -691,7 +662,7 @@ void ntod (void) state[NUL_ec] = 0; /* remove transition */ } - if (fulltbl) { + if (ctrl.fulltbl) { /* Each time we hit here, it's another td_hilen, so we realloc. */ yynxt_tbl->td_hilen++; @@ -700,10 +671,8 @@ void ntod (void) yynxt_tbl->td_hilen * yynxt_tbl->td_lolen * sizeof (flex_int32_t)); - - if (gentables) - outn (" {"); + outn ("M4_HOOK_TABLE_OPENER"); /* Supply array's 0-element. */ if (ds == end_of_buffer_state) { @@ -728,10 +697,10 @@ void ntod (void) dataflush (); if (gentables) - outn (" },\n"); + outn ("M4_HOOK_TABLE_CONTINUE"); } - else if (fullspd) + else if (ctrl.fullspd) place_state (state, ds, totaltrans); else if (ds == end_of_buffer_state) @@ -759,8 +728,9 @@ void ntod (void) } } - if (fulltbl) { - dataend (); + if (ctrl.fulltbl) { + dataend ("M4_HOOK_TABLE_CLOSER"); + outn("/* body */]])"); if (tablesext) { yytbl_data_compress (yynxt_tbl); if (yytbl_data_fwrite (&tableswr, yynxt_tbl) < 0) @@ -773,7 +743,7 @@ void ntod (void) } } - else if (!fullspd) { + else if (!ctrl.fullspd) { cmptmps (); /* create compressed template entries */ /* Create tables for all the states with only one @@ -788,8 +758,11 @@ void ntod (void) mkdeftbl (); } + free(accset); free(nset); + + return (yynxt_tbl != NULL) ? (yynxt_tbl->td_hilen * sizeof(int32_t)) : 0; } @@ -1023,7 +996,7 @@ void sympartition (int ds[], int numstates, int symlist[], int duplist[]) tch = transchar[ns]; if (tch != SYM_EPSILON) { - if (tch < -lastccl || tch >= csize) { + if (tch < -lastccl || tch >= ctrl.csize) { flexfatal (_ ("bad transition character detected in sympartition()")); } diff --git a/src/filter.c b/src/filter.c index 9a5e777..33e1119 100644 --- a/src/filter.c +++ b/src/filter.c @@ -160,13 +160,13 @@ bool filter_apply_chain (struct filter * chain) * to sync the stream. This is a Hail Mary situation. It seems to work. */ close (pipes[1]); -clearerr(stdin); + clearerr(stdin); if (dup2 (pipes[0], fileno (stdin)) == -1) flexfatal (_("dup2(pipes[0],0)")); close (pipes[0]); - fseek (stdin, 0, SEEK_CUR); - ungetc(' ', stdin); /* still an evil hack, but one that works better */ - (void)fgetc(stdin); /* on NetBSD than the fseek attempt does */ + fseek (stdin, 0, SEEK_CUR); + ungetc(' ', stdin); /* still an evil hack, but one that works better */ + (void)fgetc(stdin); /* on NetBSD than the fseek attempt does */ /* run as a filter, either internally or by exec */ if (chain->filter_func) { @@ -179,8 +179,8 @@ clearerr(stdin); else { execvp (chain->argv[0], (char **const) (chain->argv)); - lerr_fatal ( _("exec of %s failed"), - chain->argv[0]); + lerr_fatal ( _("exec of %s failed"), + chain->argv[0]); } FLEX_EXIT (1); @@ -191,7 +191,7 @@ clearerr(stdin); if (dup2 (pipes[1], fileno (stdout)) == -1) flexfatal (_("dup2(pipes[1],1)")); close (pipes[1]); - fseek (stdout, 0, SEEK_CUR); + fseek (stdout, 0, SEEK_CUR); return true; } @@ -256,29 +256,24 @@ int filter_tee_header (struct filter *chain) */ if (write_header) { - fputs (check_4_gnu_m4, to_h); + fputs (check_4_gnu_m4, to_h); fputs ("m4_changecom`'m4_dnl\n", to_h); fputs ("m4_changequote`'m4_dnl\n", to_h); - fputs ("m4_changequote([[,]])[[]]m4_dnl\n", to_h); - fputs ("m4_define([[M4_YY_NOOP]])[[]]m4_dnl\n", to_h); - fputs ("m4_define( [[M4_YY_IN_HEADER]],[[]])m4_dnl\n", - to_h); - fprintf (to_h, "#ifndef %sHEADER_H\n", prefix); - fprintf (to_h, "#define %sHEADER_H 1\n", prefix); - fprintf (to_h, "#define %sIN_HEADER 1\n\n", prefix); + fputs ("m4_changequote([[,]])[[]]m4_dnl\n", to_h); + fputs ("m4_define([[M4_YY_NOOP]])[[]]m4_dnl\n", to_h); + fputs ("m4_define([[M4_YY_IN_HEADER]],[[]])m4_dnl\n", to_h); fprintf (to_h, "m4_define( [[M4_YY_OUTFILE_NAME]],[[%s]])m4_dnl\n", - headerfilename ? headerfilename : "<stdout>"); - + env.headerfilename != NULL ? env.headerfilename : "<stdout>"); } - fputs (check_4_gnu_m4, to_c); + fputs (check_4_gnu_m4, to_c); fputs ("m4_changecom`'m4_dnl\n", to_c); fputs ("m4_changequote`'m4_dnl\n", to_c); fputs ("m4_changequote([[,]])[[]]m4_dnl\n", to_c); fputs ("m4_define([[M4_YY_NOOP]])[[]]m4_dnl\n", to_c); fprintf (to_c, "m4_define( [[M4_YY_OUTFILE_NAME]],[[%s]])m4_dnl\n", - outfilename ? outfilename : "<stdout>"); + env.outfilename != NULL ? env.outfilename : "<stdout>"); while (fgets (buf, sizeof buf, stdin)) { fputs (buf, to_c); @@ -290,13 +285,8 @@ int filter_tee_header (struct filter *chain) fprintf (to_h, "\n"); /* write a fake line number. It will get fixed by the linedir filter. */ - if (gen_line_dirs) - fprintf (to_h, "#line 4000 \"M4_YY_OUTFILE_NAME\"\n"); - - fprintf (to_h, "#undef %sIN_HEADER\n", prefix); - fprintf (to_h, "#endif /* %sHEADER_H */\n", prefix); - fputs ("m4_undefine( [[M4_YY_IN_HEADER]])m4_dnl\n", to_h); - + if (ctrl.gen_line_dirs) + line_directive_out (to_h, NULL, 4000); fflush (to_h); if (ferror (to_h)) lerr (_("error writing output file %s"), @@ -310,11 +300,11 @@ int filter_tee_header (struct filter *chain) fflush (to_c); if (ferror (to_c)) lerr (_("error writing output file %s"), - outfilename ? outfilename : "<stdout>"); + env.outfilename != NULL ? env.outfilename : "<stdout>"); else if (fclose (to_c)) lerr (_("error closing output file %s"), - outfilename ? outfilename : "<stdout>"); + env.outfilename != NULL ? env.outfilename : "<stdout>"); while (wait (0) > 0) ; @@ -338,6 +328,7 @@ static bool is_blank_line (const char *str) int filter_fix_linedirs (struct filter *chain) { char buf[4096]; + const char *traceline_template, *cp; const size_t readsz = sizeof buf; int lineno = 1; bool in_gen = true; /* in generated code */ @@ -350,9 +341,12 @@ int filter_fix_linedirs (struct filter *chain) regmatch_t m[10]; - /* Check for #line directive. */ - if (buf[0] == '#' - && regexec (®ex_linedir, buf, 3, m, 0) == 0) { + /* Check for directive. Note wired-in assumption: + * field reference 1 is line number, 2 is filename. + */ + if (ctrl.traceline_re != NULL && + ctrl.traceline_template != NULL && + regexec (®ex_linedir, buf, 3, m, 0) == 0) { char *fname; @@ -360,10 +354,10 @@ int filter_fix_linedirs (struct filter *chain) fname = regmatch_dup (&m[2], buf); if (strcmp (fname, - outfilename ? outfilename : "<stdout>") + env.outfilename != NULL ? env.outfilename : "<stdout>") == 0 || strcmp (fname, - headerfilename ? headerfilename : "<stdout>") + env.headerfilename != NULL ? env.headerfilename : "<stdout>") == 0) { char *s1, *s2; @@ -387,8 +381,9 @@ int filter_fix_linedirs (struct filter *chain) /* Adjust the line directives. */ in_gen = true; - snprintf (buf, readsz, "#line %d \"%s\"\n", + snprintf (buf, readsz, traceline_template, lineno + 1, filename); + strncat(buf, "\n", sizeof(buf)-1); } else { /* it's a #line directive for code we didn't write */ @@ -418,11 +413,11 @@ int filter_fix_linedirs (struct filter *chain) fflush (stdout); if (ferror (stdout)) lerr (_("error writing output file %s"), - outfilename ? outfilename : "<stdout>"); + env.outfilename != NULL ? env.outfilename : "<stdout>"); else if (fclose (stdout)) lerr (_("error closing output file %s"), - outfilename ? outfilename : "<stdout>"); + env.outfilename != NULL ? env.outfilename : "<stdout>"); return 0; } diff --git a/src/flex.skl b/src/flex.skl deleted file mode 100644 index 9a69520..0000000 --- a/src/flex.skl +++ /dev/null @@ -1,3443 +0,0 @@ -%# -*-C-*- vi: set ft=c: -%# This file is processed in several stages. -%# Here are the stages, as best as I can describe: -%# -%# 1. flex.skl is processed through GNU m4 during the -%# pre-compilation stage of flex. Only macros starting -%# with `m4preproc_' are processed, and quoting is normal. -%# -%# 2. The preprocessed skeleton is translated into a C array, saved -%# as "skel.c" and compiled into the flex binary. The %# comment -%# lines are removed. -%# -%# 3. At runtime, the skeleton is generated and filtered (again) -%# through m4. Macros beginning with `m4_' will be processed. -%# The quoting is "[[" and "]]" so we don't interfere with -%# user code. -%# -%# All generated macros for the m4 stage contain the text "m4" or "M4" -%# in them. This is to distinguish them from CPP macros. -%# The exception to this rule is YY_G, which is an m4 macro, -%# but it needs to be remain short because it is used everywhere. -%# -/* A lexical scanner generated by flex */ - -%# Macros for preproc stage. -m4preproc_changecom - -%# Macros for runtime processing stage. -m4_changecom -m4_changequote -m4_changequote([[, ]]) - -%# -%# Lines in this skeleton starting with a "%" character are "control lines" -%# and affect the generation of the scanner. The possible control codes are -%# listed and processed in misc.c. -%# -%# %# - A comment. The current line is omitted from the generated scanner. -%# %if-c++-only - The following lines are printed for C++ scanners ONLY. -%# %if-c-only - The following lines are NOT printed for C++ scanners. -%# %if-c-or-c++ - The following lines are printed in BOTH C and C++ scanners. -%# %if-reentrant - Print for reentrant scanners.(push) -%# %if-not-reentrant - Print for non-reentrant scanners. (push) -%# %if-bison-bridge - Print for bison-bridge. (push) -%# %if-not-bison-bridge - Print for non-bison-bridge. (push) -%# %endif - pop from the previous if code. -%# %% - A stop-point, where code is inserted by flex. -%# Each stop-point is numbered here and also in the code generator. -%# (See gen.c, etc. for details.) -%# %not-for-header - Begin code that should NOT appear in a ".h" file. -%# %ok-for-header - %c and %e are used for building a header file. -%# %if-tables-serialization -%# -%# All control-lines EXCEPT comment lines ("%#") will be inserted into -%# the generated scanner as a C-style comment. This is to aid those who -%# edit the skeleton. -%# - -%not-for-header -%if-c-only -%if-not-reentrant -m4_ifelse(M4_YY_PREFIX,yy,, -#define yy_create_buffer M4_YY_PREFIX[[_create_buffer]] -#define yy_delete_buffer M4_YY_PREFIX[[_delete_buffer]] -#define yy_scan_buffer M4_YY_PREFIX[[_scan_buffer]] -#define yy_scan_string M4_YY_PREFIX[[_scan_string]] -#define yy_scan_bytes M4_YY_PREFIX[[_scan_bytes]] -#define yy_init_buffer M4_YY_PREFIX[[_init_buffer]] -#define yy_flush_buffer M4_YY_PREFIX[[_flush_buffer]] -#define yy_load_buffer_state M4_YY_PREFIX[[_load_buffer_state]] -#define yy_switch_to_buffer M4_YY_PREFIX[[_switch_to_buffer]] -#define yypush_buffer_state M4_YY_PREFIX[[push_buffer_state]] -#define yypop_buffer_state M4_YY_PREFIX[[pop_buffer_state]] -#define yyensure_buffer_stack M4_YY_PREFIX[[ensure_buffer_stack]] -#define yy_flex_debug M4_YY_PREFIX[[_flex_debug]] -#define yyin M4_YY_PREFIX[[in]] -#define yyleng M4_YY_PREFIX[[leng]] -#define yylex M4_YY_PREFIX[[lex]] -#define yylineno M4_YY_PREFIX[[lineno]] -#define yyout M4_YY_PREFIX[[out]] -#define yyrestart M4_YY_PREFIX[[restart]] -#define yytext M4_YY_PREFIX[[text]] -#define yywrap M4_YY_PREFIX[[wrap]] -#define yyalloc M4_YY_PREFIX[[alloc]] -#define yyrealloc M4_YY_PREFIX[[realloc]] -#define yyfree M4_YY_PREFIX[[free]] -) -%endif -%endif -%ok-for-header - -#define FLEX_SCANNER -#define YY_FLEX_MAJOR_VERSION FLEX_MAJOR_VERSION -#define YY_FLEX_MINOR_VERSION FLEX_MINOR_VERSION -#define YY_FLEX_SUBMINOR_VERSION FLEX_SUBMINOR_VERSION -#if YY_FLEX_SUBMINOR_VERSION > 0 -#define FLEX_BETA -#endif - -%# Some negated symbols -m4_ifdef( [[M4_YY_IN_HEADER]], , [[m4_define([[M4_YY_NOT_IN_HEADER]], [[]])]]) -m4_ifdef( [[M4_YY_REENTRANT]], , [[m4_define([[M4_YY_NOT_REENTRANT]], [[]])]]) - -%# This is the m4 way to say "(stack_used || is_reentrant) -m4_ifdef( [[M4_YY_STACK_USED]], [[m4_define([[M4_YY_HAS_START_STACK_VARS]])]]) -m4_ifdef( [[M4_YY_REENTRANT]], [[m4_define([[M4_YY_HAS_START_STACK_VARS]])]]) - -%# Prefixes. -%# The complexity here is necessary so that m4 preserves -%# the argument lists to each C function. - - -m4_ifdef( [[M4_YY_PREFIX]],, [[m4_define([[M4_YY_PREFIX]], [[yy]])]]) - -m4preproc_define(`M4_GEN_PREFIX',`` -[[#ifdef yy$1 -#define ]]M4_YY_PREFIX[[$1_ALREADY_DEFINED -#else -#define yy$1 ]]M4_YY_PREFIX[[$1 -#endif]] -'m4preproc_divert(1)` -[[#ifndef ]]M4_YY_PREFIX[[$1_ALREADY_DEFINED -#undef yy$1 -#endif]]'m4preproc_divert(0)') - -%if-c++-only - /* The c++ scanner is a mess. The FlexLexer.h header file relies on the - * following macro. This is required in order to pass the c++-multiple-scanners - * test in the regression suite. We get reports that it breaks inheritance. - * We will address this in a future release of flex, or omit the C++ scanner - * altogether. - */ - #define yyFlexLexer M4_YY_PREFIX[[FlexLexer]] -%endif - -%if-c-only -m4_ifelse(M4_YY_PREFIX,yy,, - M4_GEN_PREFIX(`_create_buffer') - M4_GEN_PREFIX(`_delete_buffer') - M4_GEN_PREFIX(`_scan_buffer') - M4_GEN_PREFIX(`_scan_string') - M4_GEN_PREFIX(`_scan_bytes') - M4_GEN_PREFIX(`_init_buffer') - M4_GEN_PREFIX(`_flush_buffer') - M4_GEN_PREFIX(`_load_buffer_state') - M4_GEN_PREFIX(`_switch_to_buffer') - M4_GEN_PREFIX(`push_buffer_state') - M4_GEN_PREFIX(`pop_buffer_state') - M4_GEN_PREFIX(`ensure_buffer_stack') - M4_GEN_PREFIX(`lex') - M4_GEN_PREFIX(`restart') - M4_GEN_PREFIX(`lex_init') - M4_GEN_PREFIX(`lex_init_extra') - M4_GEN_PREFIX(`lex_destroy') - M4_GEN_PREFIX(`get_debug') - M4_GEN_PREFIX(`set_debug') - M4_GEN_PREFIX(`get_extra') - M4_GEN_PREFIX(`set_extra') - M4_GEN_PREFIX(`get_in') - M4_GEN_PREFIX(`set_in') - M4_GEN_PREFIX(`get_out') - M4_GEN_PREFIX(`set_out') - M4_GEN_PREFIX(`get_leng') - M4_GEN_PREFIX(`get_text') - M4_GEN_PREFIX(`get_lineno') - M4_GEN_PREFIX(`set_lineno') - m4_ifdef( [[M4_YY_REENTRANT]], - [[ - M4_GEN_PREFIX(`get_column') - M4_GEN_PREFIX(`set_column') - ]]) - M4_GEN_PREFIX(`wrap') -) -%endif - -m4_ifdef( [[M4_YY_BISON_LVAL]], -[[ - M4_GEN_PREFIX(`get_lval') - M4_GEN_PREFIX(`set_lval') -]]) - -m4_ifdef( [[<M4_YY_BISON_LLOC>]], -[[ - M4_GEN_PREFIX(`get_lloc') - M4_GEN_PREFIX(`set_lloc') -]]) - - -m4_ifelse(M4_YY_PREFIX,yy,, - M4_GEN_PREFIX(`alloc') - M4_GEN_PREFIX(`realloc') - M4_GEN_PREFIX(`free') -) - -%if-c-only -m4_ifelse(M4_YY_PREFIX,yy,, -m4_ifdef( [[M4_YY_NOT_REENTRANT]], -[[ - M4_GEN_PREFIX(`text') - M4_GEN_PREFIX(`leng') - M4_GEN_PREFIX(`in') - M4_GEN_PREFIX(`out') - M4_GEN_PREFIX(`_flex_debug') - M4_GEN_PREFIX(`lineno') -]]) -) -%endif - - -m4_ifdef( [[M4_YY_TABLES_EXTERNAL]], -[[ - M4_GEN_PREFIX(`tables_fload') - M4_GEN_PREFIX(`tables_destroy') - M4_GEN_PREFIX(`TABLES_NAME') -]]) - -/* First, we deal with platform-specific or compiler-specific issues. */ - -/* begin standard C headers. */ -%if-c-only -m4_ifdef( [[M4_YY_ALWAYS_INTERACTIVE]], , -[[m4_ifdef( [[M4_YY_NEVER_INTERACTIVE]], , -[[/* Feature test macros. Flex uses functions that require a minimum set of - * macros defined. As defining some macros may hide function declarations that - * user code might use, be conservative and respect user's definitions as much - * as possible. In glibc, feature test macros may not be all set up until one - * of the libc header (that includes <features.h>) is included. This creates - * a circular dependency when we check the macros. <assert.h> is the safest - * header we can include and does not declare too many functions we don't need. - */ -#if !defined(__GNU_LIBRARY__) && defined(__STDC__) -#include <assert.h> -#endif -#if !(defined(_POSIX_C_SOURCE) || defined(_XOPEN_SOURCE) || \ - defined(_POSIX_SOURCE)) -# define _POSIX_C_SOURCE 1 /* Required for fileno() */ -# define _POSIX_SOURCE 1 -#endif]])]]) -#include <stdio.h> -#include <string.h> -#include <errno.h> -#include <stdlib.h> -%endif - -%if-tables-serialization -#include <sys/types.h> -#include <netinet/in.h> -%endif -/* end standard C headers. */ - -/* begin standard C++ headers. */ -%if-c++-only -#include <iostream> -#include <errno.h> -#include <cstdlib> -#include <cstdio> -#include <cstring> -/* end standard C++ headers. */ -%endif - -%if-c-or-c++ -m4preproc_include(`flexint_shared.h') -%endif - -/* TODO: this is always defined, so inline it */ -#define yyconst const - -#if defined(__GNUC__) && __GNUC__ >= 3 -#define yynoreturn __attribute__((__noreturn__)) -#else -#define yynoreturn -#endif - -%not-for-header -/* Returned upon end-of-file. */ -#define YY_NULL 0 -%ok-for-header - -%not-for-header -/* Promotes a possibly negative, possibly signed char to an - * integer in range [0..255] for use as an array index. - */ -#define YY_SC_TO_UI(c) ((YY_CHAR) (c)) -%ok-for-header - - - -%if-reentrant - -/* An opaque pointer. */ -#ifndef YY_TYPEDEF_YY_SCANNER_T -#define YY_TYPEDEF_YY_SCANNER_T -typedef void* yyscan_t; -#endif - -%# Declare yyguts variable -m4_define( [[M4_YY_DECL_GUTS_VAR]], [[struct yyguts_t * yyg = (struct yyguts_t*)yyscanner]]) -%# Perform a noop access on yyguts to prevent unused variable complains -m4_define( [[M4_YY_NOOP_GUTS_VAR]], [[(void)yyg]]) -%# For use wherever a Global is accessed or assigned. -m4_define( [[YY_G]], [[yyg->$1]]) - -%# For use in function prototypes to append the additional argument. -m4_define( [[M4_YY_PROTO_LAST_ARG]], [[, yyscan_t yyscanner]]) -m4_define( [[M4_YY_PROTO_ONLY_ARG]], [[yyscan_t yyscanner]]) - -m4_define( [[M4_YY_DEF_LAST_ARG]], [[, yyscan_t yyscanner]]) -m4_define( [[M4_YY_DEF_ONLY_ARG]], [[yyscan_t yyscanner]]) -m4_define( [[M4_YY_DECL_LAST_ARG]], [[yyscan_t yyscanner;]]) - -%# For use in function calls to pass the additional argument. -m4_define( [[M4_YY_CALL_LAST_ARG]], [[, yyscanner]]) -m4_define( [[M4_YY_CALL_ONLY_ARG]], [[yyscanner]]) - -%# For use in function documentation to adjust for additional argument. -m4_define( [[M4_YY_DOC_PARAM]], [[@param yyscanner The scanner object.]]) - -/* For convenience, these vars (plus the bison vars far below) - are macros in the reentrant scanner. */ -#define yyin YY_G(yyin_r) -#define yyout YY_G(yyout_r) -#define yyextra YY_G(yyextra_r) -#define yyleng YY_G(yyleng_r) -#define yytext YY_G(yytext_r) -#define yylineno (YY_CURRENT_BUFFER_LVALUE->yy_bs_lineno) -#define yycolumn (YY_CURRENT_BUFFER_LVALUE->yy_bs_column) -#define yy_flex_debug YY_G(yy_flex_debug_r) - -m4_define( [[M4_YY_INCR_LINENO]], -[[ - do{ yylineno++; - yycolumn=0; - }while(0) -]]) - -%endif - - - -%if-not-reentrant - -m4_define( [[M4_YY_INCR_LINENO]], -[[ - yylineno++; -]]) - -%# Define these macros to be no-ops. -m4_define( [[M4_YY_DECL_GUTS_VAR]], [[m4_dnl]]) -m4_define( [[M4_YY_NOOP_GUTS_VAR]], [[m4_dnl]]) -m4_define( [[YY_G]], [[($1)]]) -m4_define( [[M4_YY_PROTO_LAST_ARG]]) -m4_define( [[M4_YY_PROTO_ONLY_ARG]], [[void]]) -m4_define( [[M4_YY_DEF_LAST_ARG]]) - -m4_define( [[M4_YY_DEF_ONLY_ARG]], [[void]]) -m4_define([[M4_YY_DECL_LAST_ARG]]) -m4_define([[M4_YY_CALL_LAST_ARG]]) -m4_define([[M4_YY_CALL_ONLY_ARG]]) -m4_define( [[M4_YY_DOC_PARAM]], ) - -%endif - - -%# Generate C99 function defs. -m4_define( [[YYFARGS1]], [[($1 $2 M4_YY_DEF_LAST_ARG)]]) -m4_define( [[YYFARGS2]], [[($1 $2, $3 $4 M4_YY_DEF_LAST_ARG)]]) -m4_define( [[YYFARGS3]], [[($1 $2, $3 $4, $5 $6 M4_YY_DEF_LAST_ARG)]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Enter a start condition. This macro really ought to take a parameter, - * but we do it the disgusting crufty way forced on us by the ()-less - * definition of BEGIN. - */ -#define BEGIN YY_G(yy_start) = 1 + 2 * -/* Translate the current start state into a value that can be later handed - * to BEGIN to return to the state. The YYSTATE alias is for lex - * compatibility. - */ -#define YY_START ((YY_G(yy_start) - 1) / 2) -#define YYSTATE YY_START -/* Action number for EOF rule of a given start state. */ -#define YY_STATE_EOF(state) (YY_END_OF_BUFFER + state + 1) -/* Special action meaning "start processing a new file". */ -#define YY_NEW_FILE yyrestart( yyin M4_YY_CALL_LAST_ARG ) -#define YY_END_OF_BUFFER_CHAR 0 -]]) - -/* Size of default input buffer. */ -#ifndef YY_BUF_SIZE -#ifdef __ia64__ -/* On IA-64, the buffer size is 16k, not 8k. - * Moreover, YY_BUF_SIZE is 2*YY_READ_BUF_SIZE in the general case. - * Ditto for the __ia64__ case accordingly. - */ -#define YY_BUF_SIZE 32768 -#else -#define YY_BUF_SIZE 16384 -#endif /* __ia64__ */ -#endif - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* The state buf must be large enough to hold one state per character in the main buffer. - */ -#define YY_STATE_BUF_SIZE ((YY_BUF_SIZE + 2) * sizeof(yy_state_type)) -]]) - - -#ifndef YY_TYPEDEF_YY_BUFFER_STATE -#define YY_TYPEDEF_YY_BUFFER_STATE -typedef struct yy_buffer_state *YY_BUFFER_STATE; -#endif - -#ifndef YY_TYPEDEF_YY_SIZE_T -#define YY_TYPEDEF_YY_SIZE_T -typedef size_t yy_size_t; -#endif - -%if-not-reentrant -extern int yyleng; -%endif - -%if-c-only -%if-not-reentrant -extern FILE *yyin, *yyout; -%endif -%endif - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -#define EOB_ACT_CONTINUE_SCAN 0 -#define EOB_ACT_END_OF_FILE 1 -#define EOB_ACT_LAST_MATCH 2 - m4_ifdef( [[M4_YY_USE_LINENO]], - [[ - /* Note: We specifically omit the test for yy_rule_can_match_eol because it requires - * access to the local variable yy_act. Since yyless() is a macro, it would break - * existing scanners that call yyless() from OUTSIDE yylex. - * One obvious solution it to make yy_act a global. I tried that, and saw - * a 5% performance hit in a non-yylineno scanner, because yy_act is - * normally declared as a register variable-- so it is not worth it. - */ - #define YY_LESS_LINENO(n) \ - do { \ - int yyl;\ - for ( yyl = n; yyl < yyleng; ++yyl )\ - if ( yytext[yyl] == '\n' )\ - --yylineno;\ - }while(0) - #define YY_LINENO_REWIND_TO(dst) \ - do {\ - const char *p;\ - for ( p = yy_cp-1; p >= (dst); --p)\ - if ( *p == '\n' )\ - --yylineno;\ - }while(0) - ]], - [[ - #define YY_LESS_LINENO(n) - #define YY_LINENO_REWIND_TO(ptr) - ]]) -/* Return all but the first "n" matched characters back to the input stream. */ -#define yyless(n) \ - do \ - { \ - /* Undo effects of setting up yytext. */ \ - int yyless_macro_arg = (n); \ - YY_LESS_LINENO(yyless_macro_arg);\ - *yy_cp = YY_G(yy_hold_char); \ - YY_RESTORE_YY_MORE_OFFSET \ - YY_G(yy_c_buf_p) = yy_cp = yy_bp + yyless_macro_arg - YY_MORE_ADJ; \ - YY_DO_BEFORE_ACTION; /* set up yytext again */ \ - } \ - while ( 0 ) -#define unput(c) yyunput( c, YY_G(yytext_ptr) M4_YY_CALL_LAST_ARG ) -]]) - -#ifndef YY_STRUCT_YY_BUFFER_STATE -#define YY_STRUCT_YY_BUFFER_STATE -struct yy_buffer_state - { -%if-c-only - FILE *yy_input_file; -%endif - -%if-c++-only - std::streambuf* yy_input_file; -%endif - - - char *yy_ch_buf; /* input buffer */ - char *yy_buf_pos; /* current position in input buffer */ - - /* Size of input buffer in bytes, not including room for EOB - * characters. - */ - int yy_buf_size; - - /* Number of characters read into yy_ch_buf, not including EOB - * characters. - */ - int yy_n_chars; - - /* Whether we "own" the buffer - i.e., we know we created it, - * and can realloc() it to grow it, and should free() it to - * delete it. - */ - int yy_is_our_buffer; - - /* Whether this is an "interactive" input source; if so, and - * if we're using stdio for input, then we want to use getc() - * instead of fread(), to make sure we stop fetching input after - * each newline. - */ - int yy_is_interactive; - - /* Whether we're considered to be at the beginning of a line. - * If so, '^' rules will be active on the next match, otherwise - * not. - */ - int yy_at_bol; - - int yy_bs_lineno; /**< The line count. */ - int yy_bs_column; /**< The column count. */ - - - /* Whether to try to fill the input buffer when we reach the - * end of it. - */ - int yy_fill_buffer; - - int yy_buffer_status; -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -#define YY_BUFFER_NEW 0 -#define YY_BUFFER_NORMAL 1 - /* When an EOF's been seen but there's still some text to process - * then we mark the buffer as YY_EOF_PENDING, to indicate that we - * shouldn't try reading from the input source any more. We might - * still have a bunch of tokens to match, though, because of - * possible backing-up. - * - * When we actually see the EOF, we change the status to "new" - * (via yyrestart()), so that the user can continue scanning by - * just pointing yyin at a new input file. - */ -#define YY_BUFFER_EOF_PENDING 2 -]]) - }; -#endif /* !YY_STRUCT_YY_BUFFER_STATE */ - -%if-c-only Standard (non-C++) definition -%not-for-header -%if-not-reentrant - -/* Stack of input buffers. */ -static size_t yy_buffer_stack_top = 0; /**< index of top of stack. */ -static size_t yy_buffer_stack_max = 0; /**< capacity of stack. */ -static YY_BUFFER_STATE * yy_buffer_stack = NULL; /**< Stack as an array. */ -%endif -%ok-for-header -%endif - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* We provide macros for accessing buffer states in case in the - * future we want to put the buffer states in a more general - * "scanner state". - * - * Returns the top of the stack, or NULL. - */ -#define YY_CURRENT_BUFFER ( YY_G(yy_buffer_stack) \ - ? YY_G(yy_buffer_stack)[YY_G(yy_buffer_stack_top)] \ - : NULL) -/* Same as previous macro, but useful when we know that the buffer stack is not - * NULL or when we need an lvalue. For internal use only. - */ -#define YY_CURRENT_BUFFER_LVALUE YY_G(yy_buffer_stack)[YY_G(yy_buffer_stack_top)] -]]) - -%if-c-only Standard (non-C++) definition - -%if-not-reentrant -%not-for-header -/* yy_hold_char holds the character lost when yytext is formed. */ -static char yy_hold_char; -static int yy_n_chars; /* number of characters read into yy_ch_buf */ -int yyleng; - -/* Points to current character in buffer. */ -static char *yy_c_buf_p = NULL; -static int yy_init = 0; /* whether we need to initialize */ -static int yy_start = 0; /* start state number */ - -/* Flag which is used to allow yywrap()'s to do buffer switches - * instead of setting up a fresh yyin. A bit of a hack ... - */ -static int yy_did_buffer_switch_on_eof; -%ok-for-header -%endif - -void yyrestart ( FILE *input_file M4_YY_PROTO_LAST_ARG ); -void yy_switch_to_buffer ( YY_BUFFER_STATE new_buffer M4_YY_PROTO_LAST_ARG ); -YY_BUFFER_STATE yy_create_buffer ( FILE *file, int size M4_YY_PROTO_LAST_ARG ); -void yy_delete_buffer ( YY_BUFFER_STATE b M4_YY_PROTO_LAST_ARG ); -void yy_flush_buffer ( YY_BUFFER_STATE b M4_YY_PROTO_LAST_ARG ); -void yypush_buffer_state ( YY_BUFFER_STATE new_buffer M4_YY_PROTO_LAST_ARG ); -void yypop_buffer_state ( M4_YY_PROTO_ONLY_ARG ); - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -static void yyensure_buffer_stack ( M4_YY_PROTO_ONLY_ARG ); -static void yy_load_buffer_state ( M4_YY_PROTO_ONLY_ARG ); -static void yy_init_buffer ( YY_BUFFER_STATE b, FILE *file M4_YY_PROTO_LAST_ARG ); -#define YY_FLUSH_BUFFER yy_flush_buffer( YY_CURRENT_BUFFER M4_YY_CALL_LAST_ARG) -]]) - -YY_BUFFER_STATE yy_scan_buffer ( char *base, yy_size_t size M4_YY_PROTO_LAST_ARG ); -YY_BUFFER_STATE yy_scan_string ( const char *yy_str M4_YY_PROTO_LAST_ARG ); -YY_BUFFER_STATE yy_scan_bytes ( const char *bytes, int len M4_YY_PROTO_LAST_ARG ); - -%endif - -void *yyalloc ( yy_size_t M4_YY_PROTO_LAST_ARG ); -void *yyrealloc ( void *, yy_size_t M4_YY_PROTO_LAST_ARG ); -void yyfree ( void * M4_YY_PROTO_LAST_ARG ); - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -#define yy_new_buffer yy_create_buffer -#define yy_set_interactive(is_interactive) \ - { \ - if ( ! YY_CURRENT_BUFFER ){ \ - yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); \ - YY_CURRENT_BUFFER_LVALUE = \ - yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); \ - } \ - YY_CURRENT_BUFFER_LVALUE->yy_is_interactive = is_interactive; \ - } -#define yy_set_bol(at_bol) \ - { \ - if ( ! YY_CURRENT_BUFFER ){\ - yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); \ - YY_CURRENT_BUFFER_LVALUE = \ - yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); \ - } \ - YY_CURRENT_BUFFER_LVALUE->yy_at_bol = at_bol; \ - } -#define YY_AT_BOL() (YY_CURRENT_BUFFER_LVALUE->yy_at_bol) -]]) - -%% [1.0] yytext/yyin/yyout/yy_state_type/yylineno etc. def's & init go here - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -%% [1.5] DFA -]]) - -%if-c-only Standard (non-C++) definition - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -static yy_state_type yy_get_previous_state ( M4_YY_PROTO_ONLY_ARG ); -static yy_state_type yy_try_NUL_trans ( yy_state_type current_state M4_YY_PROTO_LAST_ARG); -static int yy_get_next_buffer ( M4_YY_PROTO_ONLY_ARG ); -static void yynoreturn yy_fatal_error ( const char* msg M4_YY_PROTO_LAST_ARG ); -]]) - -%endif - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Done after the current pattern has been matched and before the - * corresponding action - sets up yytext. - */ -#define YY_DO_BEFORE_ACTION \ - YY_G(yytext_ptr) = yy_bp; \ -%% [2.0] code to fiddle yytext and yyleng for yymore() goes here \ - YY_G(yy_hold_char) = *yy_cp; \ - *yy_cp = '\0'; \ -%% [3.0] code to copy yytext_ptr to yytext[] goes here, if %array \ - YY_G(yy_c_buf_p) = yy_cp; -%% [4.0] data tables for the DFA and the user's section 1 definitions go here -]]) - -m4_ifdef( [[M4_YY_IN_HEADER]], [[#ifdef YY_HEADER_EXPORT_START_CONDITIONS]]) -M4_YY_SC_DEFS -m4_ifdef( [[M4_YY_IN_HEADER]], [[#endif]]) - -m4_ifdef( [[M4_YY_NO_UNISTD_H]],, -[[ -#ifndef YY_NO_UNISTD_H -/* Special case for "unistd.h", since it is non-ANSI. We include it way - * down here because we want the user's section 1 to have been scanned first. - * The user has a chance to override it with an option. - */ -%if-c-only -#include <unistd.h> -%endif -%if-c++-only -#include <unistd.h> -%endif -#endif -]]) - -m4_ifdef( [[M4_EXTRA_TYPE_DEFS]], -[[ -#define YY_EXTRA_TYPE M4_EXTRA_TYPE_DEFS -]], -[[ -#ifndef YY_EXTRA_TYPE -#define YY_EXTRA_TYPE void * -#endif -]] -) - -%if-c-only Reentrant structure and macros (non-C++). -%if-reentrant - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Holds the entire state of the reentrant scanner. */ -struct yyguts_t - { - - /* User-defined. Not touched by flex. */ - YY_EXTRA_TYPE yyextra_r; - - /* The rest are the same as the globals declared in the non-reentrant scanner. */ - FILE *yyin_r, *yyout_r; - size_t yy_buffer_stack_top; /**< index of top of stack. */ - size_t yy_buffer_stack_max; /**< capacity of stack. */ - YY_BUFFER_STATE * yy_buffer_stack; /**< Stack as an array. */ - char yy_hold_char; - int yy_n_chars; - int yyleng_r; - char *yy_c_buf_p; - int yy_init; - int yy_start; - int yy_did_buffer_switch_on_eof; - int yy_start_stack_ptr; - int yy_start_stack_depth; - int *yy_start_stack; - yy_state_type yy_last_accepting_state; - char* yy_last_accepting_cpos; - - int yylineno_r; - int yy_flex_debug_r; - -m4_ifdef( [[M4_YY_USES_REJECT]], -[[ - yy_state_type *yy_state_buf; - yy_state_type *yy_state_ptr; - char *yy_full_match; - int yy_lp; - - /* These are only needed for trailing context rules, - * but there's no conditional variable for that yet. */ - int yy_looking_for_trail_begin; - int yy_full_lp; - int *yy_full_state; -]]) - -m4_ifdef( [[M4_YY_TEXT_IS_ARRAY]], -[[ - char yytext_r[YYLMAX]; - char *yytext_ptr; - int yy_more_offset; - int yy_prev_more_offset; -]], -[[ - char *yytext_r; - int yy_more_flag; - int yy_more_len; -]]) - -m4_ifdef( [[M4_YY_BISON_LVAL]], -[[ - YYSTYPE * yylval_r; -]]) - -m4_ifdef( [[<M4_YY_BISON_LLOC>]], -[[ - YYLTYPE * yylloc_r; -]]) - - }; /* end struct yyguts_t */ -]]) - - -%if-c-only -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -static int yy_init_globals ( M4_YY_PROTO_ONLY_ARG ); -]]) -%endif - -%if-reentrant - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ - m4_ifdef( [[M4_YY_BISON_LVAL]], - [[ - /* This must go here because YYSTYPE and YYLTYPE are included - * from bison output in section 1.*/ - # define yylval YY_G(yylval_r) - ]]) - - m4_ifdef( [[<M4_YY_BISON_LLOC>]], - [[ - # define yylloc YY_G(yylloc_r) - ]]) -]]) - -int yylex_init (yyscan_t* scanner); - -int yylex_init_extra ( YY_EXTRA_TYPE user_defined, yyscan_t* scanner); - -%endif - -%endif End reentrant structures and macros. - -/* Accessor methods to globals. - These are made visible to non-reentrant scanners for convenience. */ - -m4_ifdef( [[M4_YY_NO_DESTROY]],, -[[ -int yylex_destroy ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_GET_DEBUG]],, -[[ -int yyget_debug ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_SET_DEBUG]],, -[[ -void yyset_debug ( int debug_flag M4_YY_PROTO_LAST_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_GET_EXTRA]],, -[[ -YY_EXTRA_TYPE yyget_extra ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_SET_EXTRA]],, -[[ -void yyset_extra ( YY_EXTRA_TYPE user_defined M4_YY_PROTO_LAST_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_GET_IN]],, -[[ -FILE *yyget_in ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_SET_IN]],, -[[ -void yyset_in ( FILE * _in_str M4_YY_PROTO_LAST_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_GET_OUT]],, -[[ -FILE *yyget_out ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_SET_OUT]],, -[[ -void yyset_out ( FILE * _out_str M4_YY_PROTO_LAST_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_GET_LENG]],, -[[ - int yyget_leng ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_GET_TEXT]],, -[[ -char *yyget_text ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_GET_LINENO]],, -[[ -int yyget_lineno ( M4_YY_PROTO_ONLY_ARG ); -]]) - -m4_ifdef( [[M4_YY_NO_SET_LINENO]],, -[[ -void yyset_lineno ( int _line_number M4_YY_PROTO_LAST_ARG ); -]]) - -m4_ifdef( [[M4_YY_REENTRANT]], -[[ -m4_ifdef( [[M4_YY_NO_GET_COLUMN]],, -[[ -int yyget_column ( M4_YY_PROTO_ONLY_ARG ); -]]) -]]) - -m4_ifdef( [[M4_YY_REENTRANT]], -[[ -m4_ifdef( [[M4_YY_NO_SET_COLUMN]],, -[[ -void yyset_column ( int _column_no M4_YY_PROTO_LAST_ARG ); -]]) -]]) - -%if-bison-bridge -m4_ifdef( [[M4_YY_NO_GET_LVAL]],, -[[ -YYSTYPE * yyget_lval ( M4_YY_PROTO_ONLY_ARG ); -]]) - -void yyset_lval ( YYSTYPE * yylval_param M4_YY_PROTO_LAST_ARG ); - -m4_ifdef( [[<M4_YY_BISON_LLOC>]], -[[ - m4_ifdef( [[M4_YY_NO_GET_LLOC]],, - [[ - YYLTYPE *yyget_lloc ( M4_YY_PROTO_ONLY_ARG ); - ]]) - - m4_ifdef( [[M4_YY_NO_SET_LLOC]],, - [[ - void yyset_lloc ( YYLTYPE * yylloc_param M4_YY_PROTO_LAST_ARG ); - ]]) -]]) -%endif - -/* Macros after this point can all be overridden by user definitions in - * section 1. - */ - -#ifndef YY_SKIP_YYWRAP -#ifdef __cplusplus -extern "C" int yywrap ( M4_YY_PROTO_ONLY_ARG ); -#else -extern int yywrap ( M4_YY_PROTO_ONLY_ARG ); -#endif -#endif - -%not-for-header -#ifndef YY_NO_UNPUT - m4_ifdef( [[M4_YY_NO_UNPUT]],, - [[ - static void yyunput ( int c, char *buf_ptr M4_YY_PROTO_LAST_ARG); - ]]) -#endif -%ok-for-header -%endif - -#ifndef yytext_ptr -static void yy_flex_strncpy ( char *, const char *, int M4_YY_PROTO_LAST_ARG); -#endif - -#ifdef YY_NEED_STRLEN -static int yy_flex_strlen ( const char * M4_YY_PROTO_LAST_ARG); -#endif - -#ifndef YY_NO_INPUT -%if-c-only Standard (non-C++) definition -%not-for-header -#ifdef __cplusplus -static int yyinput ( M4_YY_PROTO_ONLY_ARG ); -#else -static int input ( M4_YY_PROTO_ONLY_ARG ); -#endif -%ok-for-header -%endif -#endif - - -%if-c-only -%# TODO: This is messy. -m4_ifdef( [[M4_YY_STACK_USED]], -[[ - -m4_ifdef( [[M4_YY_NOT_REENTRANT]], -[[ - m4_ifdef( [[M4_YY_NOT_IN_HEADER]], - [[ - static int yy_start_stack_ptr = 0; - static int yy_start_stack_depth = 0; - static int *yy_start_stack = NULL; - ]]) -]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ - m4_ifdef( [[M4_YY_NO_PUSH_STATE]],, - [[ - static void yy_push_state ( int _new_state M4_YY_PROTO_LAST_ARG); - ]]) - m4_ifdef( [[M4_YY_NO_POP_STATE]],, - [[ - static void yy_pop_state ( M4_YY_PROTO_ONLY_ARG ); - ]]) - m4_ifdef( [[M4_YY_NO_TOP_STATE]],, - [[ - static int yy_top_state ( M4_YY_PROTO_ONLY_ARG ); - ]]) -]]) - -]], -[[ -m4_define( [[M4_YY_NO_PUSH_STATE]]) -m4_define( [[M4_YY_NO_POP_STATE]]) -m4_define( [[M4_YY_NO_TOP_STATE]]) -]]) -%endif - -/* Amount of stuff to slurp up with each read. */ -#ifndef YY_READ_BUF_SIZE -#ifdef __ia64__ -/* On IA-64, the buffer size is 16k, not 8k */ -#define YY_READ_BUF_SIZE 16384 -#else -#define YY_READ_BUF_SIZE 8192 -#endif /* __ia64__ */ -#endif - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Copy whatever the last rule matched to the standard output. */ -#ifndef ECHO -%if-c-only Standard (non-C++) definition -/* This used to be an fputs(), but since the string might contain NUL's, - * we now use fwrite(). - */ -#define ECHO do { if (fwrite( yytext, (size_t) yyleng, 1, yyout )) {} } while (0) -%endif -%if-c++-only C++ definition -#define ECHO LexerOutput( yytext, yyleng ) -%endif -#endif -]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Gets input and stuffs it into "buf". number of characters read, or YY_NULL, - * is returned in "result". - */ -#ifndef YY_INPUT -#define YY_INPUT(buf,result,max_size) \ -%% [5.0] fread()/read() definition of YY_INPUT goes here unless we're doing C++ \ -\ -%if-c++-only C++ definition \ - if ( (int)(result = LexerInput( (char *) buf, max_size )) < 0 ) \ - YY_FATAL_ERROR( "input in flex scanner failed" ); -%endif - -#endif -]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* No semi-colon after return; correct usage is to write "yyterminate();" - - * we don't want an extra ';' after the "return" because that will cause - * some compilers to complain about unreachable statements. - */ -#ifndef yyterminate -#define yyterminate() return YY_NULL -#endif -]]) - -/* Number of entries by which start-condition stack grows. */ -#ifndef YY_START_STACK_INCR -#define YY_START_STACK_INCR 25 -#endif - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Report a fatal error. */ -#ifndef YY_FATAL_ERROR -%if-c-only -#define YY_FATAL_ERROR(msg) yy_fatal_error( msg M4_YY_CALL_LAST_ARG) -%endif -%if-c++-only -#define YY_FATAL_ERROR(msg) LexerError( msg ) -%endif -#endif -]]) - -%if-tables-serialization structures and prototypes -m4preproc_include(`tables_shared.h') - -/* Load the DFA tables from the given stream. */ -int yytables_fload (FILE * fp M4_YY_PROTO_LAST_ARG); - -/* Unload the tables from memory. */ -int yytables_destroy (M4_YY_PROTO_ONLY_ARG); -%not-for-header - -/** Describes a mapping from a serialized table id to its deserialized state in - * this scanner. This is the bridge between our "generic" deserialization code - * and the specifics of this scanner. - */ -struct yytbl_dmap { - enum yytbl_id dm_id;/**< table identifier */ - void **dm_arr; /**< address of pointer to store the deserialized table. */ - size_t dm_sz; /**< local sizeof() each element in table. */ -}; - -/** A {0,0,0}-terminated list of structs, forming the map */ -static struct yytbl_dmap yydmap[] = -{ -%tables-yydmap generated elements - {0,0,0} -}; - -/** A tables-reader object to maintain some state in the read. */ -struct yytbl_reader { - FILE * fp; /**< input stream */ - flex_uint32_t bread; /**< bytes read since beginning of current tableset */ -}; - -%endif -/* end tables serialization structures and prototypes */ - -%ok-for-header - -/* Default declaration of generated scanner - a define so the user can - * easily add parameters. - */ -#ifndef YY_DECL -#define YY_DECL_IS_OURS 1 -%if-c-only Standard (non-C++) definition - - -m4_define( [[M4_YY_LEX_PROTO]], [[(M4_YY_PROTO_ONLY_ARG)]]) -m4_define( [[M4_YY_LEX_DECLARATION]], [[(M4_YY_DEF_ONLY_ARG)]]) - -m4_ifdef( [[M4_YY_BISON_LVAL]], -[[ - m4_dnl The bison pure parser is used. Redefine yylex to - m4_dnl accept the lval parameter. - - m4_define( [[M4_YY_LEX_PROTO]], [[\]] - [[(YYSTYPE * yylval_param M4_YY_PROTO_LAST_ARG)]]) - m4_define( [[M4_YY_LEX_DECLARATION]], [[\]] - [[YYFARGS1(YYSTYPE *,yylval_param)]]) -]]) - -m4_ifdef( [[<M4_YY_BISON_LLOC>]], -[[ - m4_dnl Locations are used. yylex should also accept the ylloc parameter. - - m4_define( [[M4_YY_LEX_PROTO]], [[\]] - [[(YYSTYPE * yylval_param, YYLTYPE * yylloc_param M4_YY_PROTO_LAST_ARG)]]) - m4_define( [[M4_YY_LEX_DECLARATION]], [[\]] - [[YYFARGS2(YYSTYPE *,yylval_param, YYLTYPE *,yylloc_param)]]) -]]) - -extern int yylex M4_YY_LEX_PROTO; - -#define YY_DECL int yylex M4_YY_LEX_DECLARATION -%endif -%if-c++-only C++ definition -#define YY_DECL int yyFlexLexer::yylex() -%endif -#endif /* !YY_DECL */ - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Code executed at the beginning of each rule, after yytext and yyleng - * have been set up. - */ -#ifndef YY_USER_ACTION -#define YY_USER_ACTION -#endif -]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* Code executed at the end of each rule. */ -#ifndef YY_BREAK -#define YY_BREAK /*LINTED*/break; -#endif -]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -%% [6.0] YY_RULE_SETUP definition goes here -]]) - -%not-for-header -/** The main scanner function which does all the work. - */ -YY_DECL -{ - yy_state_type yy_current_state; - char *yy_cp, *yy_bp; - int yy_act; - M4_YY_DECL_GUTS_VAR(); - -m4_ifdef( [[M4_YY_NOT_REENTRANT]], -[[ - m4_ifdef( [[M4_YY_BISON_LVAL]], - [[ - YYSTYPE * yylval; - ]]) - m4_ifdef( [[<M4_YY_BISON_LLOC>]], - [[ - YYLTYPE * yylloc; - ]]) -]]) - -m4_ifdef( [[M4_YY_BISON_LVAL]], -[[ - yylval = yylval_param; -]]) - -m4_ifdef( [[<M4_YY_BISON_LLOC>]], -[[ - yylloc = yylloc_param; -]]) - - if ( !YY_G(yy_init) ) - { - YY_G(yy_init) = 1; - -#ifdef YY_USER_INIT - YY_USER_INIT; -#endif - -m4_ifdef( [[M4_YY_USES_REJECT]], -[[ - /* Create the reject buffer large enough to save one state per allowed character. */ - if ( ! YY_G(yy_state_buf) ) - YY_G(yy_state_buf) = (yy_state_type *)yyalloc(YY_STATE_BUF_SIZE M4_YY_CALL_LAST_ARG); - if ( ! YY_G(yy_state_buf) ) - YY_FATAL_ERROR( "out of dynamic memory in yylex()" ); -]]) - - if ( ! YY_G(yy_start) ) - YY_G(yy_start) = 1; /* first start state */ - - if ( ! yyin ) -%if-c-only - yyin = stdin; -%endif -%if-c++-only - yyin.rdbuf(std::cin.rdbuf()); -%endif - - if ( ! yyout ) -%if-c-only - yyout = stdout; -%endif -%if-c++-only - yyout.rdbuf(std::cout.rdbuf()); -%endif - - if ( ! YY_CURRENT_BUFFER ) { - yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); - YY_CURRENT_BUFFER_LVALUE = - yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); - } - - yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); - } - - { -%% [7.0] user's declarations go here - - while ( /*CONSTCOND*/1 ) /* loops until end-of-file is reached */ - { -%% [8.0] yymore()-related code goes here - yy_cp = YY_G(yy_c_buf_p); - - /* Support of yytext. */ - *yy_cp = YY_G(yy_hold_char); - - /* yy_bp points to the position in yy_ch_buf of the start of - * the current run. - */ - yy_bp = yy_cp; - -%% [9.0] code to set up and find next match goes here - -yy_find_action: -%% [10.0] code to find the action number goes here - - YY_DO_BEFORE_ACTION; - -%% [11.0] code for yylineno update goes here - -do_action: /* This label is used only to access EOF actions. */ - -%% [12.0] debug code goes here - - switch ( yy_act ) - { /* beginning of action switch */ -%% [13.0] actions go here - - case YY_END_OF_BUFFER: - { - /* Amount of text matched not including the EOB char. */ - int yy_amount_of_matched_text = (int) (yy_cp - YY_G(yytext_ptr)) - 1; - - /* Undo the effects of YY_DO_BEFORE_ACTION. */ - *yy_cp = YY_G(yy_hold_char); - YY_RESTORE_YY_MORE_OFFSET - - if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_NEW ) - { - /* We're scanning a new file or input source. It's - * possible that this happened because the user - * just pointed yyin at a new source and called - * yylex(). If so, then we have to assure - * consistency between YY_CURRENT_BUFFER and our - * globals. Here is the right place to do so, because - * this is the first action (other than possibly a - * back-up) that will match for the new input source. - */ - YY_G(yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; -%if-c-only - YY_CURRENT_BUFFER_LVALUE->yy_input_file = yyin; -%endif -%if-c++-only - YY_CURRENT_BUFFER_LVALUE->yy_input_file = yyin.rdbuf(); -%endif - YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = YY_BUFFER_NORMAL; - } - - /* Note that here we test for yy_c_buf_p "<=" to the position - * of the first EOB in the buffer, since yy_c_buf_p will - * already have been incremented past the NUL character - * (since all states make transitions on EOB to the - * end-of-buffer state). Contrast this with the test - * in input(). - */ - if ( YY_G(yy_c_buf_p) <= &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)] ) - { /* This was really a NUL. */ - yy_state_type yy_next_state; - - YY_G(yy_c_buf_p) = YY_G(yytext_ptr) + yy_amount_of_matched_text; - - yy_current_state = yy_get_previous_state( M4_YY_CALL_ONLY_ARG ); - - /* Okay, we're now positioned to make the NUL - * transition. We couldn't have - * yy_get_previous_state() go ahead and do it - * for us because it doesn't know how to deal - * with the possibility of jamming (and we don't - * want to build jamming into it because then it - * will run more slowly). - */ - - yy_next_state = yy_try_NUL_trans( yy_current_state M4_YY_CALL_LAST_ARG); - - yy_bp = YY_G(yytext_ptr) + YY_MORE_ADJ; - - if ( yy_next_state ) - { - /* Consume the NUL. */ - yy_cp = ++YY_G(yy_c_buf_p); - yy_current_state = yy_next_state; - goto yy_match; - } - - else - { -%% [14.0] code to do back-up for compressed tables and set up yy_cp goes here - goto yy_find_action; - } - } - - else switch ( yy_get_next_buffer( M4_YY_CALL_ONLY_ARG ) ) - { - case EOB_ACT_END_OF_FILE: - { - YY_G(yy_did_buffer_switch_on_eof) = 0; - - if ( yywrap( M4_YY_CALL_ONLY_ARG ) ) - { - /* Note: because we've taken care in - * yy_get_next_buffer() to have set up - * yytext, we can now set up - * yy_c_buf_p so that if some total - * hoser (like flex itself) wants to - * call the scanner after we return the - * YY_NULL, it'll still work - another - * YY_NULL will get returned. - */ - YY_G(yy_c_buf_p) = YY_G(yytext_ptr) + YY_MORE_ADJ; - - yy_act = YY_STATE_EOF(YY_START); - goto do_action; - } - - else - { - if ( ! YY_G(yy_did_buffer_switch_on_eof) ) - YY_NEW_FILE; - } - break; - } - - case EOB_ACT_CONTINUE_SCAN: - YY_G(yy_c_buf_p) = - YY_G(yytext_ptr) + yy_amount_of_matched_text; - - yy_current_state = yy_get_previous_state( M4_YY_CALL_ONLY_ARG ); - - yy_cp = YY_G(yy_c_buf_p); - yy_bp = YY_G(yytext_ptr) + YY_MORE_ADJ; - goto yy_match; - - case EOB_ACT_LAST_MATCH: - YY_G(yy_c_buf_p) = - &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)]; - - yy_current_state = yy_get_previous_state( M4_YY_CALL_ONLY_ARG ); - - yy_cp = YY_G(yy_c_buf_p); - yy_bp = YY_G(yytext_ptr) + YY_MORE_ADJ; - goto yy_find_action; - } - break; - } - - default: - YY_FATAL_ERROR( - "fatal flex scanner internal error--no action found" ); - } /* end of action switch */ - } /* end of scanning one token */ - } /* end of user's declarations */ -} /* end of yylex */ -%ok-for-header - -%if-c++-only -%not-for-header -/* The contents of this function are C++ specific, so the YY_G macro is not used. - * This constructor simply maintains backward compatibility. - * DEPRECATED - */ -yyFlexLexer::yyFlexLexer( std::istream* arg_yyin, std::ostream* arg_yyout ): - yyin(arg_yyin ? arg_yyin->rdbuf() : std::cin.rdbuf()), - yyout(arg_yyout ? arg_yyout->rdbuf() : std::cout.rdbuf()) -{ - ctor_common(); -} - -/* The contents of this function are C++ specific, so the YY_G macro is not used. - */ -yyFlexLexer::yyFlexLexer( std::istream& arg_yyin, std::ostream& arg_yyout ): - yyin(arg_yyin.rdbuf()), - yyout(arg_yyout.rdbuf()) -{ - ctor_common(); -} - -/* The contents of this function are C++ specific, so the YY_G macro is not used. - */ -void yyFlexLexer::ctor_common() -{ - yy_c_buf_p = 0; - yy_init = 0; - yy_start = 0; - yy_flex_debug = 0; - yylineno = 1; // this will only get updated if %option yylineno - - yy_did_buffer_switch_on_eof = 0; - - yy_looking_for_trail_begin = 0; - yy_more_flag = 0; - yy_more_len = 0; - yy_more_offset = yy_prev_more_offset = 0; - - yy_start_stack_ptr = yy_start_stack_depth = 0; - yy_start_stack = NULL; - - yy_buffer_stack = NULL; - yy_buffer_stack_top = 0; - yy_buffer_stack_max = 0; - - -m4_ifdef( [[M4_YY_USES_REJECT]], -[[ - yy_state_buf = new yy_state_type[YY_STATE_BUF_SIZE]; -]], -[[ - yy_state_buf = 0; -]]) -} - -/* The contents of this function are C++ specific, so the YY_G macro is not used. - */ -yyFlexLexer::~yyFlexLexer() -{ - delete [] yy_state_buf; - yyfree( yy_start_stack M4_YY_CALL_LAST_ARG ); - yy_delete_buffer( YY_CURRENT_BUFFER M4_YY_CALL_LAST_ARG); - yyfree( yy_buffer_stack M4_YY_CALL_LAST_ARG ); -} - -/* The contents of this function are C++ specific, so the YY_G macro is not used. - */ -void yyFlexLexer::switch_streams( std::istream& new_in, std::ostream& new_out ) -{ - // was if( new_in ) - yy_delete_buffer( YY_CURRENT_BUFFER M4_YY_CALL_LAST_ARG); - yy_switch_to_buffer( yy_create_buffer( new_in, YY_BUF_SIZE M4_YY_CALL_LAST_ARG) M4_YY_CALL_LAST_ARG); - - // was if( new_out ) - yyout.rdbuf(new_out.rdbuf()); -} - -/* The contents of this function are C++ specific, so the YY_G macro is not used. - */ -void yyFlexLexer::switch_streams( std::istream* new_in, std::ostream* new_out ) -{ - if( ! new_in ) { - new_in = &yyin; - } - - if ( ! new_out ) { - new_out = &yyout; - } - - switch_streams(*new_in, *new_out); -} - -#ifdef YY_INTERACTIVE -int yyFlexLexer::LexerInput( char* buf, int /* max_size */ ) -#else -int yyFlexLexer::LexerInput( char* buf, int max_size ) -#endif -{ - if ( yyin.eof() || yyin.fail() ) - return 0; - -#ifdef YY_INTERACTIVE - yyin.get( buf[0] ); - - if ( yyin.eof() ) - return 0; - - if ( yyin.bad() ) - return -1; - - return 1; - -#else - (void) yyin.read( buf, max_size ); - - if ( yyin.bad() ) - return -1; - else - return yyin.gcount(); -#endif -} - -void yyFlexLexer::LexerOutput( const char* buf, int size ) -{ - (void) yyout.write( buf, size ); -} -%ok-for-header -%endif - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* yy_get_next_buffer - try to read in a new buffer - * - * Returns a code representing an action: - * EOB_ACT_LAST_MATCH - - * EOB_ACT_CONTINUE_SCAN - continue scanning from current position - * EOB_ACT_END_OF_FILE - end of file - */ -%if-c-only -static int yy_get_next_buffer (M4_YY_DEF_ONLY_ARG) -%endif -%if-c++-only -int yyFlexLexer::yy_get_next_buffer() -%endif -{ - M4_YY_DECL_GUTS_VAR(); - char *dest = YY_CURRENT_BUFFER_LVALUE->yy_ch_buf; - char *source = YY_G(yytext_ptr); - int number_to_move, i; - int ret_val; - - if ( YY_G(yy_c_buf_p) > &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars) + 1] ) - YY_FATAL_ERROR( - "fatal flex scanner internal error--end of buffer missed" ); - - if ( YY_CURRENT_BUFFER_LVALUE->yy_fill_buffer == 0 ) - { /* Don't try to fill the buffer, so this is an EOF. */ - if ( YY_G(yy_c_buf_p) - YY_G(yytext_ptr) - YY_MORE_ADJ == 1 ) - { - /* We matched a single character, the EOB, so - * treat this as a final EOF. - */ - return EOB_ACT_END_OF_FILE; - } - - else - { - /* We matched some text prior to the EOB, first - * process it. - */ - return EOB_ACT_LAST_MATCH; - } - } - - /* Try to read more data. */ - - /* First move last chars to start of buffer. */ - number_to_move = (int) (YY_G(yy_c_buf_p) - YY_G(yytext_ptr) - 1); - - for ( i = 0; i < number_to_move; ++i ) - *(dest++) = *(source++); - - if ( YY_CURRENT_BUFFER_LVALUE->yy_buffer_status == YY_BUFFER_EOF_PENDING ) - /* don't do the read, it's not guaranteed to return an EOF, - * just force an EOF - */ - YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars) = 0; - - else - { - int num_to_read = - YY_CURRENT_BUFFER_LVALUE->yy_buf_size - number_to_move - 1; - - while ( num_to_read <= 0 ) - { /* Not enough room in the buffer - grow it. */ -m4_ifdef( [[M4_YY_USES_REJECT]], -[[ - YY_FATAL_ERROR( -"input buffer overflow, can't enlarge buffer because scanner uses REJECT" ); -]], -[[ - /* just a shorter name for the current buffer */ - YY_BUFFER_STATE b = YY_CURRENT_BUFFER_LVALUE; - - int yy_c_buf_p_offset = - (int) (YY_G(yy_c_buf_p) - b->yy_ch_buf); - - if ( b->yy_is_our_buffer ) - { - int new_size = b->yy_buf_size * 2; - - if ( new_size <= 0 ) - b->yy_buf_size += b->yy_buf_size / 8; - else - b->yy_buf_size *= 2; - - b->yy_ch_buf = (char *) - /* Include room in for 2 EOB chars. */ - yyrealloc( (void *) b->yy_ch_buf, - (yy_size_t) (b->yy_buf_size + 2) M4_YY_CALL_LAST_ARG ); - } - else - /* Can't grow it, we don't own it. */ - b->yy_ch_buf = NULL; - - if ( ! b->yy_ch_buf ) - YY_FATAL_ERROR( - "fatal error - scanner input buffer overflow" ); - - YY_G(yy_c_buf_p) = &b->yy_ch_buf[yy_c_buf_p_offset]; - - num_to_read = YY_CURRENT_BUFFER_LVALUE->yy_buf_size - - number_to_move - 1; -]]) - } - - if ( num_to_read > YY_READ_BUF_SIZE ) - num_to_read = YY_READ_BUF_SIZE; - - /* Read in more data. */ - YY_INPUT( (&YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]), - YY_G(yy_n_chars), num_to_read ); - - YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars); - } - - if ( YY_G(yy_n_chars) == 0 ) - { - if ( number_to_move == YY_MORE_ADJ ) - { - ret_val = EOB_ACT_END_OF_FILE; - yyrestart( yyin M4_YY_CALL_LAST_ARG); - } - - else - { - ret_val = EOB_ACT_LAST_MATCH; - YY_CURRENT_BUFFER_LVALUE->yy_buffer_status = - YY_BUFFER_EOF_PENDING; - } - } - - else - ret_val = EOB_ACT_CONTINUE_SCAN; - - if ((YY_G(yy_n_chars) + number_to_move) > YY_CURRENT_BUFFER_LVALUE->yy_buf_size) { - /* Extend the array by 50%, plus the number we really need. */ - int new_size = YY_G(yy_n_chars) + number_to_move + (YY_G(yy_n_chars) >> 1); - YY_CURRENT_BUFFER_LVALUE->yy_ch_buf = (char *) yyrealloc( - (void *) YY_CURRENT_BUFFER_LVALUE->yy_ch_buf, (yy_size_t) new_size M4_YY_CALL_LAST_ARG ); - if ( ! YY_CURRENT_BUFFER_LVALUE->yy_ch_buf ) - YY_FATAL_ERROR( "out of dynamic memory in yy_get_next_buffer()" ); - /* "- 2" to take care of EOB's */ - YY_CURRENT_BUFFER_LVALUE->yy_buf_size = (int) (new_size - 2); - } - - YY_G(yy_n_chars) += number_to_move; - YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)] = YY_END_OF_BUFFER_CHAR; - YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars) + 1] = YY_END_OF_BUFFER_CHAR; - - YY_G(yytext_ptr) = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[0]; - - return ret_val; -} -]]) - -/* yy_get_previous_state - get the state just before the EOB char was reached */ - -%if-c-only -%not-for-header - static yy_state_type yy_get_previous_state (M4_YY_DEF_ONLY_ARG) -%endif -%if-c++-only - yy_state_type yyFlexLexer::yy_get_previous_state() -%endif -{ - yy_state_type yy_current_state; - char *yy_cp; - M4_YY_DECL_GUTS_VAR(); - -%% [15.0] code to get the start state into yy_current_state goes here - - for ( yy_cp = YY_G(yytext_ptr) + YY_MORE_ADJ; yy_cp < YY_G(yy_c_buf_p); ++yy_cp ) - { -%% [16.0] code to find the next state goes here - } - - return yy_current_state; -} - - -/* yy_try_NUL_trans - try to make a transition on the NUL character - * - * synopsis - * next_state = yy_try_NUL_trans( current_state ); - */ -%if-c-only - static yy_state_type yy_try_NUL_trans YYFARGS1( yy_state_type, yy_current_state) -%endif -%if-c++-only - yy_state_type yyFlexLexer::yy_try_NUL_trans( yy_state_type yy_current_state ) -%endif -{ - int yy_is_jam; - M4_YY_DECL_GUTS_VAR(); /* This var may be unused depending upon options. */ -%% [17.0] code to find the next state, and perhaps do backing up, goes here - - M4_YY_NOOP_GUTS_VAR(); -return yy_is_jam ? 0 : yy_current_state; -} - - -#ifndef YY_NO_UNPUT -%if-c-only -m4_ifdef( [[M4_YY_NO_UNPUT]],, -[[ - static void yyunput YYFARGS2( int,c, char *,yy_bp) -%endif -%if-c++-only - void yyFlexLexer::yyunput( int c, char* yy_bp) -%endif -{ - char *yy_cp; - M4_YY_DECL_GUTS_VAR(); - - yy_cp = YY_G(yy_c_buf_p); - - /* undo effects of setting up yytext */ - *yy_cp = YY_G(yy_hold_char); - - if ( yy_cp < YY_CURRENT_BUFFER_LVALUE->yy_ch_buf + 2 ) - { /* need to shift things up to make room */ - /* +2 for EOB chars. */ - int number_to_move = YY_G(yy_n_chars) + 2; - char *dest = &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[ - YY_CURRENT_BUFFER_LVALUE->yy_buf_size + 2]; - char *source = - &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[number_to_move]; - - while ( source > YY_CURRENT_BUFFER_LVALUE->yy_ch_buf ) - *--dest = *--source; - - yy_cp += (int) (dest - source); - yy_bp += (int) (dest - source); - YY_CURRENT_BUFFER_LVALUE->yy_n_chars = - YY_G(yy_n_chars) = (int) YY_CURRENT_BUFFER_LVALUE->yy_buf_size; - - if ( yy_cp < YY_CURRENT_BUFFER_LVALUE->yy_ch_buf + 2 ) - YY_FATAL_ERROR( "flex scanner push-back overflow" ); - } - - *--yy_cp = (char) c; - -%% [18.0] update yylineno here -m4_ifdef( [[M4_YY_USE_LINENO]], -[[ - if ( c == '\n' ){ - --yylineno; - } -]]) - - YY_G(yytext_ptr) = yy_bp; - YY_G(yy_hold_char) = *yy_cp; - YY_G(yy_c_buf_p) = yy_cp; -} -%if-c-only -]]) -%endif -#endif - -%if-c-only -#ifndef YY_NO_INPUT -#ifdef __cplusplus - static int yyinput (M4_YY_DEF_ONLY_ARG) -#else - static int input (M4_YY_DEF_ONLY_ARG) -#endif - -%endif -%if-c++-only - int yyFlexLexer::yyinput() -%endif -{ - int c; - M4_YY_DECL_GUTS_VAR(); - - *YY_G(yy_c_buf_p) = YY_G(yy_hold_char); - - if ( *YY_G(yy_c_buf_p) == YY_END_OF_BUFFER_CHAR ) - { - /* yy_c_buf_p now points to the character we want to return. - * If this occurs *before* the EOB characters, then it's a - * valid NUL; if not, then we've hit the end of the buffer. - */ - if ( YY_G(yy_c_buf_p) < &YY_CURRENT_BUFFER_LVALUE->yy_ch_buf[YY_G(yy_n_chars)] ) - /* This was really a NUL. */ - *YY_G(yy_c_buf_p) = '\0'; - - else - { /* need more input */ - int offset = (int) (YY_G(yy_c_buf_p) - YY_G(yytext_ptr)); - ++YY_G(yy_c_buf_p); - - switch ( yy_get_next_buffer( M4_YY_CALL_ONLY_ARG ) ) - { - case EOB_ACT_LAST_MATCH: - /* This happens because yy_g_n_b() - * sees that we've accumulated a - * token and flags that we need to - * try matching the token before - * proceeding. But for input(), - * there's no matching to consider. - * So convert the EOB_ACT_LAST_MATCH - * to EOB_ACT_END_OF_FILE. - */ - - /* Reset buffer status. */ - yyrestart( yyin M4_YY_CALL_LAST_ARG); - - /*FALLTHROUGH*/ - - case EOB_ACT_END_OF_FILE: - { - if ( yywrap( M4_YY_CALL_ONLY_ARG ) ) - return 0; - - if ( ! YY_G(yy_did_buffer_switch_on_eof) ) - YY_NEW_FILE; -#ifdef __cplusplus - return yyinput(M4_YY_CALL_ONLY_ARG); -#else - return input(M4_YY_CALL_ONLY_ARG); -#endif - } - - case EOB_ACT_CONTINUE_SCAN: - YY_G(yy_c_buf_p) = YY_G(yytext_ptr) + offset; - break; - } - } - } - - c = *(unsigned char *) YY_G(yy_c_buf_p); /* cast for 8-bit char's */ - *YY_G(yy_c_buf_p) = '\0'; /* preserve yytext */ - YY_G(yy_hold_char) = *++YY_G(yy_c_buf_p); - -%% [19.0] update BOL and yylineno - - return c; -} -%if-c-only -#endif /* ifndef YY_NO_INPUT */ -%endif - -/** Immediately switch to a different input stream. - * @param input_file A readable stream. - * M4_YY_DOC_PARAM - * @note This function does not reset the start condition to @c INITIAL . - */ -%if-c-only - void yyrestart YYFARGS1( FILE *,input_file) -%endif -%if-c++-only - void yyFlexLexer::yyrestart( std::istream& input_file ) -%endif -{ - M4_YY_DECL_GUTS_VAR(); - - if ( ! YY_CURRENT_BUFFER ){ - yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); - YY_CURRENT_BUFFER_LVALUE = - yy_create_buffer( yyin, YY_BUF_SIZE M4_YY_CALL_LAST_ARG); - } - - yy_init_buffer( YY_CURRENT_BUFFER, input_file M4_YY_CALL_LAST_ARG); - yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); -} - -%if-c++-only -/** Delegate to the new version that takes an istream reference. - * @param input_file A readable stream. - * M4_YY_DOC_PARAM - * @note This function does not reset the start condition to @c INITIAL . - */ -void yyFlexLexer::yyrestart( std::istream* input_file ) -{ - if( ! input_file ) { - input_file = &yyin; - } - yyrestart( *input_file ); -} -%endif - -/** Switch to a different input buffer. - * @param new_buffer The new input buffer. - * M4_YY_DOC_PARAM - */ -%if-c-only - void yy_switch_to_buffer YYFARGS1( YY_BUFFER_STATE ,new_buffer) -%endif -%if-c++-only - void yyFlexLexer::yy_switch_to_buffer( YY_BUFFER_STATE new_buffer ) -%endif -{ - M4_YY_DECL_GUTS_VAR(); - - /* TODO. We should be able to replace this entire function body - * with - * yypop_buffer_state(); - * yypush_buffer_state(new_buffer); - */ - yyensure_buffer_stack (M4_YY_CALL_ONLY_ARG); - if ( YY_CURRENT_BUFFER == new_buffer ) - return; - - if ( YY_CURRENT_BUFFER ) - { - /* Flush out information for old buffer. */ - *YY_G(yy_c_buf_p) = YY_G(yy_hold_char); - YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = YY_G(yy_c_buf_p); - YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars); - } - - YY_CURRENT_BUFFER_LVALUE = new_buffer; - yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); - - /* We don't actually know whether we did this switch during - * EOF (yywrap()) processing, but the only time this flag - * is looked at is after yywrap() is called, so it's safe - * to go ahead and always set it. - */ - YY_G(yy_did_buffer_switch_on_eof) = 1; -} - - -%if-c-only -static void yy_load_buffer_state (M4_YY_DEF_ONLY_ARG) -%endif -%if-c++-only - void yyFlexLexer::yy_load_buffer_state() -%endif -{ - M4_YY_DECL_GUTS_VAR(); - YY_G(yy_n_chars) = YY_CURRENT_BUFFER_LVALUE->yy_n_chars; - YY_G(yytext_ptr) = YY_G(yy_c_buf_p) = YY_CURRENT_BUFFER_LVALUE->yy_buf_pos; -%if-c-only - yyin = YY_CURRENT_BUFFER_LVALUE->yy_input_file; -%endif -%if-c++-only - yyin.rdbuf(YY_CURRENT_BUFFER_LVALUE->yy_input_file); -%endif - YY_G(yy_hold_char) = *YY_G(yy_c_buf_p); -} - -/** Allocate and initialize an input buffer state. - * @param file A readable stream. - * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. - * M4_YY_DOC_PARAM - * @return the allocated buffer state. - */ -%if-c-only - YY_BUFFER_STATE yy_create_buffer YYFARGS2( FILE *,file, int ,size) -%endif -%if-c++-only - YY_BUFFER_STATE yyFlexLexer::yy_create_buffer( std::istream& file, int size ) -%endif -{ - YY_BUFFER_STATE b; - m4_dnl M4_YY_DECL_GUTS_VAR(); - - b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) M4_YY_CALL_LAST_ARG ); - if ( ! b ) - YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); - - b->yy_buf_size = size; - - /* yy_ch_buf has to be 2 characters longer than the size given because - * we need to put in 2 end-of-buffer characters. - */ - b->yy_ch_buf = (char *) yyalloc( (yy_size_t) (b->yy_buf_size + 2) M4_YY_CALL_LAST_ARG ); - if ( ! b->yy_ch_buf ) - YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" ); - - b->yy_is_our_buffer = 1; - - yy_init_buffer( b, file M4_YY_CALL_LAST_ARG); - - return b; -} - -%if-c++-only -/** Delegate creation of buffers to the new version that takes an istream reference. - * @param file A readable stream. - * @param size The character buffer size in bytes. When in doubt, use @c YY_BUF_SIZE. - * M4_YY_DOC_PARAM - * @return the allocated buffer state. - */ - YY_BUFFER_STATE yyFlexLexer::yy_create_buffer( std::istream* file, int size ) -{ - return yy_create_buffer( *file, size ); -} -%endif - -/** Destroy the buffer. - * @param b a buffer created with yy_create_buffer() - * M4_YY_DOC_PARAM - */ -%if-c-only - void yy_delete_buffer YYFARGS1( YY_BUFFER_STATE ,b) -%endif -%if-c++-only - void yyFlexLexer::yy_delete_buffer( YY_BUFFER_STATE b ) -%endif -{ - M4_YY_DECL_GUTS_VAR(); - - if ( ! b ) - return; - - if ( b == YY_CURRENT_BUFFER ) /* Not sure if we should pop here. */ - YY_CURRENT_BUFFER_LVALUE = (YY_BUFFER_STATE) 0; - - if ( b->yy_is_our_buffer ) - yyfree( (void *) b->yy_ch_buf M4_YY_CALL_LAST_ARG ); - - yyfree( (void *) b M4_YY_CALL_LAST_ARG ); -} - - -/* Initializes or reinitializes a buffer. - * This function is sometimes called more than once on the same buffer, - * such as during a yyrestart() or at EOF. - */ -%if-c-only - static void yy_init_buffer YYFARGS2( YY_BUFFER_STATE ,b, FILE *,file) -%endif -%if-c++-only - void yyFlexLexer::yy_init_buffer( YY_BUFFER_STATE b, std::istream& file ) -%endif - -{ - int oerrno = errno; - M4_YY_DECL_GUTS_VAR(); - - yy_flush_buffer( b M4_YY_CALL_LAST_ARG); - -%if-c-only - b->yy_input_file = file; -%endif -%if-c++-only - b->yy_input_file = file.rdbuf(); -%endif - b->yy_fill_buffer = 1; - - /* If b is the current buffer, then yy_init_buffer was _probably_ - * called from yyrestart() or through yy_get_next_buffer. - * In that case, we don't want to reset the lineno or column. - */ - if (b != YY_CURRENT_BUFFER){ - b->yy_bs_lineno = 1; - b->yy_bs_column = 0; - } - -%if-c-only -m4_ifdef( [[M4_YY_ALWAYS_INTERACTIVE]], -[[ - b->yy_is_interactive = 1; -]], -[[ - m4_ifdef( [[M4_YY_NEVER_INTERACTIVE]], - [[ - b->yy_is_interactive = 0; - ]], - [[ - b->yy_is_interactive = file ? (isatty( fileno(file) ) > 0) : 0; - ]]) -]]) -%endif -%if-c++-only - b->yy_is_interactive = 0; -%endif - errno = oerrno; -} - -/** Discard all buffered characters. On the next scan, YY_INPUT will be called. - * @param b the buffer state to be flushed, usually @c YY_CURRENT_BUFFER. - * M4_YY_DOC_PARAM - */ -%if-c-only - void yy_flush_buffer YYFARGS1( YY_BUFFER_STATE ,b) -%endif -%if-c++-only - void yyFlexLexer::yy_flush_buffer( YY_BUFFER_STATE b ) -%endif -{ - M4_YY_DECL_GUTS_VAR(); - if ( ! b ) - return; - - b->yy_n_chars = 0; - - /* We always need two end-of-buffer characters. The first causes - * a transition to the end-of-buffer state. The second causes - * a jam in that state. - */ - b->yy_ch_buf[0] = YY_END_OF_BUFFER_CHAR; - b->yy_ch_buf[1] = YY_END_OF_BUFFER_CHAR; - - b->yy_buf_pos = &b->yy_ch_buf[0]; - - b->yy_at_bol = 1; - b->yy_buffer_status = YY_BUFFER_NEW; - - if ( b == YY_CURRENT_BUFFER ) - yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); -} - -%if-c-or-c++ -/** Pushes the new state onto the stack. The new state becomes - * the current state. This function will allocate the stack - * if necessary. - * @param new_buffer The new state. - * M4_YY_DOC_PARAM - */ -%if-c-only -void yypush_buffer_state YYFARGS1(YY_BUFFER_STATE,new_buffer) -%endif -%if-c++-only -void yyFlexLexer::yypush_buffer_state (YY_BUFFER_STATE new_buffer) -%endif -{ - M4_YY_DECL_GUTS_VAR(); - if (new_buffer == NULL) - return; - - yyensure_buffer_stack(M4_YY_CALL_ONLY_ARG); - - /* This block is copied from yy_switch_to_buffer. */ - if ( YY_CURRENT_BUFFER ) - { - /* Flush out information for old buffer. */ - *YY_G(yy_c_buf_p) = YY_G(yy_hold_char); - YY_CURRENT_BUFFER_LVALUE->yy_buf_pos = YY_G(yy_c_buf_p); - YY_CURRENT_BUFFER_LVALUE->yy_n_chars = YY_G(yy_n_chars); - } - - /* Only push if top exists. Otherwise, replace top. */ - if (YY_CURRENT_BUFFER) - YY_G(yy_buffer_stack_top)++; - YY_CURRENT_BUFFER_LVALUE = new_buffer; - - /* copied from yy_switch_to_buffer. */ - yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); - YY_G(yy_did_buffer_switch_on_eof) = 1; -} -%endif - - -%if-c-or-c++ -/** Removes and deletes the top of the stack, if present. - * The next element becomes the new top. - * M4_YY_DOC_PARAM - */ -%if-c-only -void yypop_buffer_state (M4_YY_DEF_ONLY_ARG) -%endif -%if-c++-only -void yyFlexLexer::yypop_buffer_state (void) -%endif -{ - M4_YY_DECL_GUTS_VAR(); - if (!YY_CURRENT_BUFFER) - return; - - yy_delete_buffer(YY_CURRENT_BUFFER M4_YY_CALL_LAST_ARG); - YY_CURRENT_BUFFER_LVALUE = NULL; - if (YY_G(yy_buffer_stack_top) > 0) - --YY_G(yy_buffer_stack_top); - - if (YY_CURRENT_BUFFER) { - yy_load_buffer_state( M4_YY_CALL_ONLY_ARG ); - YY_G(yy_did_buffer_switch_on_eof) = 1; - } -} -%endif - - -%if-c-or-c++ -/* Allocates the stack if it does not exist. - * Guarantees space for at least one push. - */ -%if-c-only -static void yyensure_buffer_stack (M4_YY_DEF_ONLY_ARG) -%endif -%if-c++-only -void yyFlexLexer::yyensure_buffer_stack(void) -%endif -{ - yy_size_t num_to_alloc; - M4_YY_DECL_GUTS_VAR(); - - if (!YY_G(yy_buffer_stack)) { - - /* First allocation is just for 2 elements, since we don't know if this - * scanner will even need a stack. We use 2 instead of 1 to avoid an - * immediate realloc on the next call. - */ - num_to_alloc = 1; /* After all that talk, this was set to 1 anyways... */ - YY_G(yy_buffer_stack) = (struct yy_buffer_state**)yyalloc - (num_to_alloc * sizeof(struct yy_buffer_state*) - M4_YY_CALL_LAST_ARG); - if ( ! YY_G(yy_buffer_stack) ) - YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); - - - memset(YY_G(yy_buffer_stack), 0, num_to_alloc * sizeof(struct yy_buffer_state*)); - - YY_G(yy_buffer_stack_max) = num_to_alloc; - YY_G(yy_buffer_stack_top) = 0; - return; - } - - if (YY_G(yy_buffer_stack_top) >= (YY_G(yy_buffer_stack_max)) - 1){ - - /* Increase the buffer to prepare for a possible push. */ - yy_size_t grow_size = 8 /* arbitrary grow size */; - - num_to_alloc = YY_G(yy_buffer_stack_max) + grow_size; - YY_G(yy_buffer_stack) = (struct yy_buffer_state**)yyrealloc - (YY_G(yy_buffer_stack), - num_to_alloc * sizeof(struct yy_buffer_state*) - M4_YY_CALL_LAST_ARG); - if ( ! YY_G(yy_buffer_stack) ) - YY_FATAL_ERROR( "out of dynamic memory in yyensure_buffer_stack()" ); - - /* zero only the new slots.*/ - memset(YY_G(yy_buffer_stack) + YY_G(yy_buffer_stack_max), 0, grow_size * sizeof(struct yy_buffer_state*)); - YY_G(yy_buffer_stack_max) = num_to_alloc; - } -} -%endif - - - - -m4_ifdef( [[M4_YY_NO_SCAN_BUFFER]],, -[[ -%if-c-only -/** Setup the input buffer state to scan directly from a user-specified character buffer. - * @param base the character buffer - * @param size the size in bytes of the character buffer - * M4_YY_DOC_PARAM - * @return the newly allocated buffer state object. - */ -YY_BUFFER_STATE yy_scan_buffer YYFARGS2( char *,base, yy_size_t ,size) -{ - YY_BUFFER_STATE b; - m4_dnl M4_YY_DECL_GUTS_VAR(); - - if ( size < 2 || - base[size-2] != YY_END_OF_BUFFER_CHAR || - base[size-1] != YY_END_OF_BUFFER_CHAR ) - /* They forgot to leave room for the EOB's. */ - return NULL; - - b = (YY_BUFFER_STATE) yyalloc( sizeof( struct yy_buffer_state ) M4_YY_CALL_LAST_ARG ); - if ( ! b ) - YY_FATAL_ERROR( "out of dynamic memory in yy_scan_buffer()" ); - - b->yy_buf_size = (int) (size - 2); /* "- 2" to take care of EOB's */ - b->yy_buf_pos = b->yy_ch_buf = base; - b->yy_is_our_buffer = 0; - b->yy_input_file = NULL; - b->yy_n_chars = b->yy_buf_size; - b->yy_is_interactive = 0; - b->yy_at_bol = 1; - b->yy_fill_buffer = 0; - b->yy_buffer_status = YY_BUFFER_NEW; - - yy_switch_to_buffer( b M4_YY_CALL_LAST_ARG ); - - return b; -} -%endif -]]) - - -m4_ifdef( [[M4_YY_NO_SCAN_STRING]],, -[[ -%if-c-only -/** Setup the input buffer state to scan a string. The next call to yylex() will - * scan from a @e copy of @a str. - * @param yystr a NUL-terminated string to scan - * M4_YY_DOC_PARAM - * @return the newly allocated buffer state object. - * @note If you want to scan bytes that may contain NUL values, then use - * yy_scan_bytes() instead. - */ -YY_BUFFER_STATE yy_scan_string YYFARGS1( const char *, yystr) -{ - m4_dnl M4_YY_DECL_GUTS_VAR(); - - return yy_scan_bytes( yystr, (int) strlen(yystr) M4_YY_CALL_LAST_ARG); -} -%endif -]]) - - -m4_ifdef( [[M4_YY_NO_SCAN_BYTES]],, -[[ -%if-c-only -/** Setup the input buffer state to scan the given bytes. The next call to yylex() will - * scan from a @e copy of @a bytes. - * @param yybytes the byte buffer to scan - * @param _yybytes_len the number of bytes in the buffer pointed to by @a bytes. - * M4_YY_DOC_PARAM - * @return the newly allocated buffer state object. - */ -YY_BUFFER_STATE yy_scan_bytes YYFARGS2( const char *,yybytes, int ,_yybytes_len) -{ - YY_BUFFER_STATE b; - char *buf; - yy_size_t n; - int i; - m4_dnl M4_YY_DECL_GUTS_VAR(); - - /* Get memory for full buffer, including space for trailing EOB's. */ - n = (yy_size_t) (_yybytes_len + 2); - buf = (char *) yyalloc( n M4_YY_CALL_LAST_ARG ); - if ( ! buf ) - YY_FATAL_ERROR( "out of dynamic memory in yy_scan_bytes()" ); - - for ( i = 0; i < _yybytes_len; ++i ) - buf[i] = yybytes[i]; - - buf[_yybytes_len] = buf[_yybytes_len+1] = YY_END_OF_BUFFER_CHAR; - - b = yy_scan_buffer( buf, n M4_YY_CALL_LAST_ARG); - if ( ! b ) - YY_FATAL_ERROR( "bad buffer in yy_scan_bytes()" ); - - /* It's okay to grow etc. this buffer, and we should throw it - * away when we're done. - */ - b->yy_is_our_buffer = 1; - - return b; -} -%endif -]]) - - -m4_ifdef( [[M4_YY_NO_PUSH_STATE]],, -[[ -%if-c-only - static void yy_push_state YYFARGS1( int ,_new_state) -%endif -%if-c++-only - void yyFlexLexer::yy_push_state( int _new_state ) -%endif -{ - M4_YY_DECL_GUTS_VAR(); - if ( YY_G(yy_start_stack_ptr) >= YY_G(yy_start_stack_depth) ) - { - yy_size_t new_size; - - YY_G(yy_start_stack_depth) += YY_START_STACK_INCR; - new_size = (yy_size_t) YY_G(yy_start_stack_depth) * sizeof( int ); - - if ( ! YY_G(yy_start_stack) ) - YY_G(yy_start_stack) = (int *) yyalloc( new_size M4_YY_CALL_LAST_ARG ); - - else - YY_G(yy_start_stack) = (int *) yyrealloc( - (void *) YY_G(yy_start_stack), new_size M4_YY_CALL_LAST_ARG ); - - if ( ! YY_G(yy_start_stack) ) - YY_FATAL_ERROR( "out of memory expanding start-condition stack" ); - } - - YY_G(yy_start_stack)[YY_G(yy_start_stack_ptr)++] = YY_START; - - BEGIN(_new_state); -} -]]) - - -m4_ifdef( [[M4_YY_NO_POP_STATE]],, -[[ -%if-c-only - static void yy_pop_state (M4_YY_DEF_ONLY_ARG) -%endif -%if-c++-only - void yyFlexLexer::yy_pop_state() -%endif -{ - M4_YY_DECL_GUTS_VAR(); - if ( --YY_G(yy_start_stack_ptr) < 0 ) - YY_FATAL_ERROR( "start-condition stack underflow" ); - - BEGIN(YY_G(yy_start_stack)[YY_G(yy_start_stack_ptr)]); -} -]]) - - -m4_ifdef( [[M4_YY_NO_TOP_STATE]],, -[[ -%if-c-only - static int yy_top_state (M4_YY_DEF_ONLY_ARG) -%endif -%if-c++-only - int yyFlexLexer::yy_top_state() -%endif -{ - M4_YY_DECL_GUTS_VAR(); - return YY_G(yy_start_stack_ptr) > 0 ? YY_G(yy_start_stack)[YY_G(yy_start_stack_ptr) - 1] : YY_START; -} -]]) - -#ifndef YY_EXIT_FAILURE -#define YY_EXIT_FAILURE 2 -#endif - -%if-c-only -static void yynoreturn yy_fatal_error YYFARGS1(const char*, msg) -{ - M4_YY_DECL_GUTS_VAR(); - M4_YY_NOOP_GUTS_VAR(); - fprintf( stderr, "%s\n", msg ); - exit( YY_EXIT_FAILURE ); -} -%endif -%if-c++-only -void yyFlexLexer::LexerError( const char* msg ) -{ - M4_YY_DECL_GUTS_VAR(); - std::cerr << msg << std::endl; - exit( YY_EXIT_FAILURE ); -} -%endif - -/* Redefine yyless() so it works in section 3 code. */ - -#undef yyless -#define yyless(n) \ - do \ - { \ - /* Undo effects of setting up yytext. */ \ - int yyless_macro_arg = (n); \ - YY_LESS_LINENO(yyless_macro_arg);\ - yytext[yyleng] = YY_G(yy_hold_char); \ - YY_G(yy_c_buf_p) = yytext + yyless_macro_arg; \ - YY_G(yy_hold_char) = *YY_G(yy_c_buf_p); \ - *YY_G(yy_c_buf_p) = '\0'; \ - yyleng = yyless_macro_arg; \ - } \ - while ( 0 ) - - - -/* Accessor methods (get/set functions) to struct members. */ - -%if-c-only -%if-reentrant -m4_ifdef( [[M4_YY_NO_GET_EXTRA]],, -[[ -/** Get the user-defined data for this scanner. - * M4_YY_DOC_PARAM - */ -YY_EXTRA_TYPE yyget_extra (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yyextra; -} -]]) -%endif - -m4_ifdef( [[M4_YY_NO_GET_LINENO]],, -[[ -/** Get the current line number. - * M4_YY_DOC_PARAM - */ -int yyget_lineno (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - - m4_ifdef( [[M4_YY_REENTRANT]], - [[ - if (! YY_CURRENT_BUFFER) - return 0; - ]]) - return yylineno; -} -]]) - -m4_ifdef( [[M4_YY_REENTRANT]], -[[ -m4_ifdef( [[M4_YY_NO_GET_COLUMN]],, -[[ -/** Get the current column number. - * M4_YY_DOC_PARAM - */ -int yyget_column (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - - m4_ifdef( [[M4_YY_REENTRANT]], - [[ - if (! YY_CURRENT_BUFFER) - return 0; - ]]) - return yycolumn; -} -]]) -]]) - -m4_ifdef( [[M4_YY_NO_GET_IN]],, -[[ -/** Get the input stream. - * M4_YY_DOC_PARAM - */ -FILE *yyget_in (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yyin; -} -]]) - -m4_ifdef( [[M4_YY_NO_GET_OUT]],, -[[ -/** Get the output stream. - * M4_YY_DOC_PARAM - */ -FILE *yyget_out (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yyout; -} -]]) - -m4_ifdef( [[M4_YY_NO_GET_LENG]],, -[[ -/** Get the length of the current token. - * M4_YY_DOC_PARAM - */ -int yyget_leng (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yyleng; -} -]]) - -/** Get the current token. - * M4_YY_DOC_PARAM - */ -m4_ifdef( [[M4_YY_NO_GET_TEXT]],, -[[ -char *yyget_text (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yytext; -} -]]) - -%if-reentrant -m4_ifdef( [[M4_YY_NO_SET_EXTRA]],, -[[ -/** Set the user-defined data. This data is never touched by the scanner. - * @param user_defined The data to be associated with this scanner. - * M4_YY_DOC_PARAM - */ -void yyset_extra YYFARGS1( YY_EXTRA_TYPE ,user_defined) -{ - M4_YY_DECL_GUTS_VAR(); - yyextra = user_defined ; -} -]]) -%endif - -m4_ifdef( [[M4_YY_NO_SET_LINENO]],, -[[ -/** Set the current line number. - * @param _line_number line number - * M4_YY_DOC_PARAM - */ -void yyset_lineno YYFARGS1( int ,_line_number) -{ - M4_YY_DECL_GUTS_VAR(); - - m4_ifdef( [[M4_YY_REENTRANT]], - [[ - /* lineno is only valid if an input buffer exists. */ - if (! YY_CURRENT_BUFFER ) - YY_FATAL_ERROR( "yyset_lineno called with no buffer" ); - ]]) - yylineno = _line_number; -} -]]) - -m4_ifdef( [[M4_YY_REENTRANT]], -[[ -m4_ifdef( [[M4_YY_NO_SET_COLUMN]],, -[[ -/** Set the current column. - * @param _column_no column number - * M4_YY_DOC_PARAM - */ -void yyset_column YYFARGS1( int , _column_no) -{ - M4_YY_DECL_GUTS_VAR(); - - m4_ifdef( [[M4_YY_REENTRANT]], - [[ - /* column is only valid if an input buffer exists. */ - if (! YY_CURRENT_BUFFER ) - YY_FATAL_ERROR( "yyset_column called with no buffer" ); - ]]) - yycolumn = _column_no; -} -]]) -]]) - - -m4_ifdef( [[M4_YY_NO_SET_IN]],, -[[ -/** Set the input stream. This does not discard the current - * input buffer. - * @param _in_str A readable stream. - * M4_YY_DOC_PARAM - * @see yy_switch_to_buffer - */ -void yyset_in YYFARGS1( FILE * ,_in_str) -{ - M4_YY_DECL_GUTS_VAR(); - yyin = _in_str ; -} -]]) - -m4_ifdef( [[M4_YY_NO_SET_OUT]],, -[[ -void yyset_out YYFARGS1( FILE * ,_out_str) -{ - M4_YY_DECL_GUTS_VAR(); - yyout = _out_str ; -} -]]) - - -m4_ifdef( [[M4_YY_NO_GET_DEBUG]],, -[[ -int yyget_debug (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yy_flex_debug; -} -]]) - -m4_ifdef( [[M4_YY_NO_SET_DEBUG]],, -[[ -void yyset_debug YYFARGS1( int ,_bdebug) -{ - M4_YY_DECL_GUTS_VAR(); - yy_flex_debug = _bdebug ; -} -]]) -%endif - -%if-reentrant -/* Accessor methods for yylval and yylloc */ - -%if-bison-bridge -m4_ifdef( [[M4_YY_NO_GET_LVAL]],, -[[ -YYSTYPE * yyget_lval (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yylval; -} -]]) - -m4_ifdef( [[M4_YY_NO_SET_LVAL]],, -[[ -void yyset_lval YYFARGS1( YYSTYPE * ,yylval_param) -{ - M4_YY_DECL_GUTS_VAR(); - yylval = yylval_param; -} -]]) - -m4_ifdef( [[<M4_YY_BISON_LLOC>]], -[[ - m4_ifdef( [[M4_YY_NO_GET_LLOC]],, - [[ -YYLTYPE *yyget_lloc (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - return yylloc; -} - ]]) - - m4_ifdef( [[M4_YY_NO_SET_LLOC]],, - [[ -void yyset_lloc YYFARGS1( YYLTYPE * ,yylloc_param) -{ - M4_YY_DECL_GUTS_VAR(); - yylloc = yylloc_param; -} - ]]) -]]) - -%endif - - -/* User-visible API */ - -/* yylex_init is special because it creates the scanner itself, so it is - * the ONLY reentrant function that doesn't take the scanner as the last argument. - * That's why we explicitly handle the declaration, instead of using our macros. - */ -int yylex_init(yyscan_t* ptr_yy_globals) -{ - if (ptr_yy_globals == NULL){ - errno = EINVAL; - return 1; - } - - *ptr_yy_globals = (yyscan_t) yyalloc ( sizeof( struct yyguts_t ), NULL ); - - if (*ptr_yy_globals == NULL){ - errno = ENOMEM; - return 1; - } - - /* By setting to 0xAA, we expose bugs in yy_init_globals. Leave at 0x00 for releases. */ - memset(*ptr_yy_globals,0x00,sizeof(struct yyguts_t)); - - return yy_init_globals ( *ptr_yy_globals ); -} - - -/* yylex_init_extra has the same functionality as yylex_init, but follows the - * convention of taking the scanner as the last argument. Note however, that - * this is a *pointer* to a scanner, as it will be allocated by this call (and - * is the reason, too, why this function also must handle its own declaration). - * The user defined value in the first argument will be available to yyalloc in - * the yyextra field. - */ -int yylex_init_extra( YY_EXTRA_TYPE yy_user_defined, yyscan_t* ptr_yy_globals ) -{ - struct yyguts_t dummy_yyguts; - - yyset_extra (yy_user_defined, &dummy_yyguts); - - if (ptr_yy_globals == NULL){ - errno = EINVAL; - return 1; - } - - *ptr_yy_globals = (yyscan_t) yyalloc ( sizeof( struct yyguts_t ), &dummy_yyguts ); - - if (*ptr_yy_globals == NULL){ - errno = ENOMEM; - return 1; - } - - /* By setting to 0xAA, we expose bugs in - yy_init_globals. Leave at 0x00 for releases. */ - memset(*ptr_yy_globals,0x00,sizeof(struct yyguts_t)); - - yyset_extra (yy_user_defined, *ptr_yy_globals); - - return yy_init_globals ( *ptr_yy_globals ); -} - -%endif if-c-only -%# Actually, that ended an if-rentrant section - -%if-c-only -static int yy_init_globals (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - /* Initialization is the same as for the non-reentrant scanner. - * This function is called from yylex_destroy(), so don't allocate here. - */ - -m4_ifdef( [[M4_YY_USE_LINENO]], -[[ - m4_ifdef( [[M4_YY_NOT_REENTRANT]], - [[ - /* We do not touch yylineno unless the option is enabled. */ - yylineno = 1; - ]]) -]]) - YY_G(yy_buffer_stack) = NULL; - YY_G(yy_buffer_stack_top) = 0; - YY_G(yy_buffer_stack_max) = 0; - YY_G(yy_c_buf_p) = NULL; - YY_G(yy_init) = 0; - YY_G(yy_start) = 0; - -m4_ifdef( [[M4_YY_HAS_START_STACK_VARS]], -[[ - YY_G(yy_start_stack_ptr) = 0; - YY_G(yy_start_stack_depth) = 0; - YY_G(yy_start_stack) = NULL; -]]) - -m4_ifdef( [[M4_YY_USES_REJECT]], -[[ - YY_G(yy_state_buf) = 0; - YY_G(yy_state_ptr) = 0; - YY_G(yy_full_match) = 0; - YY_G(yy_lp) = 0; -]]) - -m4_ifdef( [[M4_YY_TEXT_IS_ARRAY]], -[[ - YY_G(yytext_ptr) = 0; - YY_G(yy_more_offset) = 0; - YY_G(yy_prev_more_offset) = 0; -]]) - -/* Defined in main.c */ -#ifdef YY_STDINIT - yyin = stdin; - yyout = stdout; -#else - yyin = NULL; - yyout = NULL; -#endif - - /* For future reference: Set errno on error, since we are called by - * yylex_init() - */ - return 0; -} -%endif - - -%if-c-only SNIP! this currently causes conflicts with the c++ scanner -/* yylex_destroy is for both reentrant and non-reentrant scanners. */ -int yylex_destroy (M4_YY_DEF_ONLY_ARG) -{ - M4_YY_DECL_GUTS_VAR(); - - /* Pop the buffer stack, destroying each element. */ - while(YY_CURRENT_BUFFER){ - yy_delete_buffer( YY_CURRENT_BUFFER M4_YY_CALL_LAST_ARG ); - YY_CURRENT_BUFFER_LVALUE = NULL; - yypop_buffer_state(M4_YY_CALL_ONLY_ARG); - } - - /* Destroy the stack itself. */ - yyfree(YY_G(yy_buffer_stack) M4_YY_CALL_LAST_ARG); - YY_G(yy_buffer_stack) = NULL; - -m4_ifdef( [[M4_YY_HAS_START_STACK_VARS]], -[[ - /* Destroy the start condition stack. */ - yyfree( YY_G(yy_start_stack) M4_YY_CALL_LAST_ARG ); - YY_G(yy_start_stack) = NULL; -]]) - -m4_ifdef( [[M4_YY_USES_REJECT]], -[[ - yyfree ( YY_G(yy_state_buf) M4_YY_CALL_LAST_ARG); - YY_G(yy_state_buf) = NULL; -]]) - - /* Reset the globals. This is important in a non-reentrant scanner so the next time - * yylex() is called, initialization will occur. */ - yy_init_globals( M4_YY_CALL_ONLY_ARG); - -%if-reentrant - /* Destroy the main struct (reentrant only). */ - yyfree ( yyscanner M4_YY_CALL_LAST_ARG ); - yyscanner = NULL; -%endif - return 0; -} -%endif - - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -/* - * Internal utility routines. - */ -]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -#ifndef yytext_ptr -static void yy_flex_strncpy YYFARGS3( char*,s1, const char *,s2, int,n) -{ - M4_YY_DECL_GUTS_VAR(); - M4_YY_NOOP_GUTS_VAR(); - - int i; - for ( i = 0; i < n; ++i ) - s1[i] = s2[i]; -} -#endif -]]) - -m4_ifdef( [[M4_YY_NOT_IN_HEADER]], -[[ -#ifdef YY_NEED_STRLEN -static int yy_flex_strlen YYFARGS1( const char *,s) -{ - int n; - for ( n = 0; s[n]; ++n ) - ; - - return n; -} -#endif -]]) - -m4_ifdef( [[M4_YY_NO_FLEX_ALLOC]],, -[[ -void *yyalloc YYFARGS1( yy_size_t ,size) -{ - M4_YY_DECL_GUTS_VAR(); - M4_YY_NOOP_GUTS_VAR(); - return malloc(size); -} -]]) - -m4_ifdef( [[M4_YY_NO_FLEX_REALLOC]],, -[[ -void *yyrealloc YYFARGS2( void *,ptr, yy_size_t ,size) -{ - M4_YY_DECL_GUTS_VAR(); - M4_YY_NOOP_GUTS_VAR(); - - /* The cast to (char *) in the following accommodates both - * implementations that use char* generic pointers, and those - * that use void* generic pointers. It works with the latter - * because both ANSI C and C++ allow castless assignment from - * any pointer type to void*, and deal with argument conversions - * as though doing an assignment. - */ - return realloc(ptr, size); -} -]]) - -m4_ifdef( [[M4_YY_NO_FLEX_FREE]],, -[[ -void yyfree YYFARGS1( void *,ptr) -{ - M4_YY_DECL_GUTS_VAR(); - M4_YY_NOOP_GUTS_VAR(); - free( (char *) ptr ); /* see yyrealloc() for (char *) cast */ -} -]]) - -%if-tables-serialization definitions -m4preproc_include(`tables_shared.c') - -static int yytbl_read8 (void *v, struct yytbl_reader * rd) -{ - errno = 0; - if (fread (v, sizeof (flex_uint8_t), 1, rd->fp) != 1){ - errno = EIO; - return -1; - } - rd->bread += (flex_uint32_t) sizeof(flex_uint8_t); - return 0; -} - -static int yytbl_read16 (void *v, struct yytbl_reader * rd) -{ - errno = 0; - if (fread (v, sizeof (flex_uint16_t), 1, rd->fp) != 1){ - errno = EIO; - return -1; - } - *((flex_uint16_t *) v) = ntohs (*((flex_uint16_t *) v)); - rd->bread += (flex_uint32_t) sizeof(flex_uint16_t); - return 0; -} - -static int yytbl_read32 (void *v, struct yytbl_reader * rd) -{ - errno = 0; - if (fread (v, sizeof (flex_uint32_t), 1, rd->fp) != 1){ - errno = EIO; - return -1; - } - *((flex_uint32_t *) v) = ntohl (*((flex_uint32_t *) v)); - rd->bread += (flex_uint32_t) sizeof(flex_uint32_t); - return 0; -} - -/** Read the header */ -static int yytbl_hdr_read YYFARGS2(struct yytbl_hdr *, th, struct yytbl_reader *, rd) -{ - size_t bytes; - memset (th, 0, sizeof (struct yytbl_hdr)); - - if (yytbl_read32 (&(th->th_magic), rd) != 0) - return -1; - - if (th->th_magic != YYTBL_MAGIC){ - YY_FATAL_ERROR( "bad magic number" ); /* TODO: not fatal. */ - return -1; - } - - if (yytbl_read32 (&(th->th_hsize), rd) != 0 - || yytbl_read32 (&(th->th_ssize), rd) != 0 - || yytbl_read16 (&(th->th_flags), rd) != 0) - return -1; - - /* Sanity check on header size. Greater than 1k suggests some funny business. */ - if (th->th_hsize < 16 || th->th_hsize > 1024){ - YY_FATAL_ERROR( "insane header size detected" ); /* TODO: not fatal. */ - return -1; - } - - /* Allocate enough space for the version and name fields */ - bytes = th->th_hsize - 14; - th->th_version = (char *) yyalloc (bytes M4_YY_CALL_LAST_ARG); - if ( ! th->th_version ) - YY_FATAL_ERROR( "out of dynamic memory in yytbl_hdr_read()" ); - - /* we read it all into th_version, and point th_name into that data */ - if (fread (th->th_version, 1, bytes, rd->fp) != bytes){ - errno = EIO; - yyfree(th->th_version M4_YY_CALL_LAST_ARG); - th->th_version = NULL; - return -1; - } - else - rd->bread += (flex_uint32_t) bytes; - - th->th_name = th->th_version + strlen (th->th_version) + 1; - return 0; -} - -/** lookup id in the dmap list. - * @param dmap pointer to first element in list - * @return NULL if not found. - */ -static struct yytbl_dmap *yytbl_dmap_lookup YYFARGS2(struct yytbl_dmap *, dmap, - int, id) -{ - M4_YY_DECL_GUTS_VAR(); - M4_YY_NOOP_GUTS_VAR(); - - while (dmap->dm_id) - if ((int)(dmap->dm_id) == id) - return dmap; - else - dmap++; - return NULL; -} - -/** Read a table while mapping its contents to the local array. - * @param dmap used to performing mapping - * @return 0 on success - */ -static int yytbl_data_load YYFARGS2(struct yytbl_dmap *, dmap, struct yytbl_reader*, rd) -{ - struct yytbl_data td; - struct yytbl_dmap *transdmap=0; - int len, i, rv, inner_loop_count; - void *p=0; - - memset (&td, 0, sizeof (struct yytbl_data)); - - if (yytbl_read16 (&td.td_id, rd) != 0 - || yytbl_read16 (&td.td_flags, rd) != 0 - || yytbl_read32 (&td.td_hilen, rd) != 0 - || yytbl_read32 (&td.td_lolen, rd) != 0) - return -1; - - /* Lookup the map for the transition table so we have it in case we need it - * inside the loop below. This scanner might not even have a transition - * table, which is ok. - */ - transdmap = yytbl_dmap_lookup (dmap, YYTD_ID_TRANSITION M4_YY_CALL_LAST_ARG); - - if ((dmap = yytbl_dmap_lookup (dmap, td.td_id M4_YY_CALL_LAST_ARG)) == NULL){ - YY_FATAL_ERROR( "table id not found in map." ); /* TODO: not fatal. */ - return -1; - } - - /* Allocate space for table. - * The --full yy_transition table is a special case, since we - * need the dmap.dm_sz entry to tell us the sizeof the individual - * struct members. - */ - { - size_t bytes; - - if ((td.td_flags & YYTD_STRUCT)) - bytes = sizeof(struct yy_trans_info) * td.td_lolen * (td.td_hilen ? td.td_hilen : 1); - else - bytes = td.td_lolen * (td.td_hilen ? td.td_hilen : 1) * dmap->dm_sz; - - if(M4_YY_TABLES_VERIFY) - /* We point to the array itself */ - p = dmap->dm_arr; - else - /* We point to the address of a pointer. */ - *dmap->dm_arr = p = (void *) yyalloc (bytes M4_YY_CALL_LAST_ARG); - if ( ! p ) - YY_FATAL_ERROR( "out of dynamic memory in yytbl_data_load()" ); - } - - /* If it's a struct, we read 2 integers to get one element */ - if ((td.td_flags & YYTD_STRUCT) != 0) - inner_loop_count = 2; - else - inner_loop_count = 1; - - /* read and map each element. - * This loop iterates once for each element of the td_data array. - * Notice that we increment 'i' in the inner loop. - */ - len = yytbl_calc_total_len (&td); - for (i = 0; i < len; ){ - int j; - - - /* This loop really executes exactly 1 or 2 times. - * The second time is to handle the second member of the - * YYTD_STRUCT for the yy_transition array. - */ - for (j = 0; j < inner_loop_count; j++, i++) { - flex_int32_t t32; - - /* read into t32 no matter what the real size is. */ - { - flex_int16_t t16; - flex_int8_t t8; - - switch (YYTDFLAGS2BYTES (td.td_flags)) { - case sizeof (flex_int32_t): - rv = yytbl_read32 (&t32, rd); - break; - case sizeof (flex_int16_t): - rv = yytbl_read16 (&t16, rd); - t32 = t16; - break; - case sizeof (flex_int8_t): - rv = yytbl_read8 (&t8, rd); - t32 = t8; - break; - default: - YY_FATAL_ERROR( "invalid td_flags" ); /* TODO: not fatal. */ - return -1; - } - } - if (rv != 0) - return -1; - - /* copy into the deserialized array... */ - - if ((td.td_flags & YYTD_STRUCT)) { - /* t32 is the j'th member of a two-element struct. */ - void *v; - - v = j == 0 ? &(((struct yy_trans_info *) p)->yy_verify) - : &(((struct yy_trans_info *) p)->yy_nxt); - - switch (dmap->dm_sz) { - case sizeof (flex_int32_t): - if (M4_YY_TABLES_VERIFY){ - if( ((flex_int32_t *) v)[0] != (flex_int32_t) t32) - YY_FATAL_ERROR( "tables verification failed at YYTD_STRUCT flex_int32_t" ); - }else - ((flex_int32_t *) v)[0] = (flex_int32_t) t32; - break; - case sizeof (flex_int16_t): - if (M4_YY_TABLES_VERIFY ){ - if(((flex_int16_t *) v)[0] != (flex_int16_t) t32) - YY_FATAL_ERROR( "tables verification failed at YYTD_STRUCT flex_int16_t" ); - }else - ((flex_int16_t *) v)[0] = (flex_int16_t) t32; - break; - case sizeof(flex_int8_t): - if (M4_YY_TABLES_VERIFY ){ - if( ((flex_int8_t *) v)[0] != (flex_int8_t) t32) - YY_FATAL_ERROR( "tables verification failed at YYTD_STRUCT flex_int8_t" ); - }else - ((flex_int8_t *) v)[0] = (flex_int8_t) t32; - break; - default: - YY_FATAL_ERROR( "invalid dmap->dm_sz for struct" ); /* TODO: not fatal. */ - return -1; - } - - /* if we're done with j, increment p */ - if (j == 1) - p = (struct yy_trans_info *) p + 1; - } - else if ((td.td_flags & YYTD_PTRANS)) { - /* t32 is an index into the transition array. */ - struct yy_trans_info *v; - - - if (!transdmap){ - YY_FATAL_ERROR( "transition table not found" ); /* TODO: not fatal. */ - return -1; - } - - if( M4_YY_TABLES_VERIFY) - v = &(((struct yy_trans_info *) (transdmap->dm_arr))[t32]); - else - v = &((*((struct yy_trans_info **) (transdmap->dm_arr)))[t32]); - - if(M4_YY_TABLES_VERIFY ){ - if( ((struct yy_trans_info **) p)[0] != v) - YY_FATAL_ERROR( "tables verification failed at YYTD_PTRANS" ); - }else - ((struct yy_trans_info **) p)[0] = v; - - /* increment p */ - p = (struct yy_trans_info **) p + 1; - } - else { - /* t32 is a plain int. copy data, then incrememnt p. */ - switch (dmap->dm_sz) { - case sizeof (flex_int32_t): - if(M4_YY_TABLES_VERIFY ){ - if( ((flex_int32_t *) p)[0] != (flex_int32_t) t32) - YY_FATAL_ERROR( "tables verification failed at flex_int32_t" ); - }else - ((flex_int32_t *) p)[0] = (flex_int32_t) t32; - p = ((flex_int32_t *) p) + 1; - break; - case sizeof (flex_int16_t): - if(M4_YY_TABLES_VERIFY ){ - if( ((flex_int16_t *) p)[0] != (flex_int16_t) t32) - YY_FATAL_ERROR( "tables verification failed at flex_int16_t" ); - }else - ((flex_int16_t *) p)[0] = (flex_int16_t) t32; - p = ((flex_int16_t *) p) + 1; - break; - case sizeof (flex_int8_t): - if(M4_YY_TABLES_VERIFY ){ - if( ((flex_int8_t *) p)[0] != (flex_int8_t) t32) - YY_FATAL_ERROR( "tables verification failed at flex_int8_t" ); - }else - ((flex_int8_t *) p)[0] = (flex_int8_t) t32; - p = ((flex_int8_t *) p) + 1; - break; - default: - YY_FATAL_ERROR( "invalid dmap->dm_sz for plain int" ); /* TODO: not fatal. */ - return -1; - } - } - } - - } - - /* Now eat padding. */ - { - while (rd->bread % (8 * sizeof(flex_uint8_t)) > 0) { - flex_int8_t t8; - if(yytbl_read8(&t8,rd) != 0) - return -1; - } - } - - return 0; -} - -%define-yytables The name for this specific scanner's tables. - -/* Find the key and load the DFA tables from the given stream. */ -static int yytbl_fload YYFARGS2(FILE *, fp, const char *, key) -{ - int rv=0; - struct yytbl_hdr th; - struct yytbl_reader rd; - - rd.fp = fp; - th.th_version = NULL; - - /* Keep trying until we find the right set of tables or end of file. */ - while (!feof(rd.fp)) { - rd.bread = 0; - if (yytbl_hdr_read (&th, &rd M4_YY_CALL_LAST_ARG) != 0){ - rv = -1; - goto return_rv; - } - - /* A NULL key means choose the first set of tables. */ - if (key == NULL) - break; - - if (strcmp(th.th_name,key) != 0){ - /* Skip ahead to next set */ - fseek(rd.fp, th.th_ssize - th.th_hsize, SEEK_CUR); - yyfree(th.th_version M4_YY_CALL_LAST_ARG); - th.th_version = NULL; - } - else - break; - } - - while (rd.bread < th.th_ssize){ - /* Load the data tables */ - if(yytbl_data_load (yydmap,&rd M4_YY_CALL_LAST_ARG) != 0){ - rv = -1; - goto return_rv; - } - } - -return_rv: - if(th.th_version){ - yyfree(th.th_version M4_YY_CALL_LAST_ARG); - th.th_version = NULL; - } - - return rv; -} - -/** Load the DFA tables for this scanner from the given stream. */ -int yytables_fload YYFARGS1(FILE *, fp) -{ - - if( yytbl_fload(fp, YYTABLES_NAME M4_YY_CALL_LAST_ARG) != 0) - return -1; - return 0; -} - -/** Destroy the loaded tables, freeing memory, etc.. */ -int yytables_destroy (M4_YY_DEF_ONLY_ARG) -{ - struct yytbl_dmap *dmap=0; - - if(!M4_YY_TABLES_VERIFY){ - /* Walk the dmap, freeing the pointers */ - for(dmap=yydmap; dmap->dm_id; dmap++) { - void * v; - v = dmap->dm_arr; - if(v && *(char**)v){ - yyfree(*(char**)v M4_YY_CALL_LAST_ARG); - *(char**)v = NULL; - } - } - } - - return 0; -} - -/* end table serialization code definitions */ -%endif - - -m4_ifdef([[M4_YY_MAIN]], [[ -int main (void); - -int main () -{ - -%if-reentrant - yyscan_t lexer; - yylex_init(&lexer); - yylex( lexer ); - yylex_destroy( lexer); - -%endif -%if-not-reentrant - yylex(); -%endif - - return 0; -} -]]) - -%ok-for-header -m4_ifdef( [[M4_YY_IN_HEADER]], -[[ -#undef YY_NEW_FILE -#undef YY_FLUSH_BUFFER -#undef yy_set_bol -#undef yy_new_buffer -#undef yy_set_interactive -#undef YY_DO_BEFORE_ACTION - -#ifdef YY_DECL_IS_OURS -#undef YY_DECL_IS_OURS -#undef YY_DECL -#endif -m4preproc_undivert(1) -]]) diff --git a/src/flexdef.h b/src/flexdef.h index a48c64d..74d65b3 100644 --- a/src/flexdef.h +++ b/src/flexdef.h @@ -113,8 +113,6 @@ /* Whether an integer is a power of two */ #define is_power_of_2(n) ((n) > 0 && ((n) & ((n) - 1)) == 0) -#define unspecified -1 - /* Special chk[] values marking the slots taking by end-of-buffer and action * numbers. */ @@ -298,51 +296,140 @@ */ #define BAD_SUBSCRIPT -32767 -/* Absolute value of largest number that can be stored in a short, with a - * bit of slop thrown in for general paranoia. +typedef enum trit_t { + trit_unspecified = -1, + trit_false = 0, + trit_true = 1, +} trit; + +/* Control variables. These are in a struct to avoid having to replicate definitions + * twice for each option, instead a single struct can be declared and externed. + * If it's in this structure, it has a corresponding m4 symbol. */ -#define MAX_SHORT 32700 +struct ctrl_bundle_t { + bool always_interactive;// always use cheacter-by-character input + FILE *backing_up_file; // file to summarize backing-up states to + bool bison_bridge_lval; // (--bison-bridge), bison pure calling convention. + bool bison_bridge_lloc; // (--bison-locations), bison yylloc. + size_t bufsize; // input buffer size + bool C_plus_plus; // (-+ flag) generate a C++ scanner class + int csize; // size of character set for the scanner + // 128 for 7-bit chars and 256 for 8-bit + bool ddebug; // (-d) make a "debug" scanner + trit do_main; // generate main part to make lexer standalone. + bool do_stdinit; // whether to initialize yyin/yyout to stdin/stdout + bool do_yylineno; // if true, generate code to maintain yylineno + bool do_yywrap; // do yywrap() processing on EOF. + // If false, EOF treated as "no more files" + bool fullspd; // (-F flag) use Jacobson method of table representation + bool fulltbl; // (-Cf flag) don't compress the DFA state table + bool gen_line_dirs; // (no -L flag) generate #line directives + trit interactive; // (-I) generate an interactive scanner + bool never_interactive; // always use buffered input, don't check for tty. + bool lex_compat; // (-l), maximize compatibility with AT&T lex + bool long_align; // (-Ca flag), favor long-word alignment for speed + bool no_yyinput; // suppress use of yyinput() + bool no_unistd; // suppress inclusion of unistd.h + bool posix_compat; // (-X) maximize compatibility with POSIX lex + char *prefix; // prefix for externally visible names, default "yy" + trit reject_really_used;// Force generation of support code for reject operation + bool reentrant; // if true (-R), generate a reentrant C scanner + bool rewrite; // Appl;y magic rewre rles to special fumctions + bool stack_used; // Enable use of start-condition stacks + bool no_section3_escape;// True if the undocumented option --unsafe-no-m4-sect3-escape was passed + bool spprdflt; // (-s) suppress the default rule + bool useecs; // (-Ce flag) use equivalence classes + bool usemecs; // (-Cm flag), use meta-equivalence classes + bool use_read; // (-f, -F, or -Cr) use read() for scanner input + // otherwise, use fread(). + char *yyclass; // yyFlexLexer subclass to use for YY_DECL + char *yydecl; // user-specfied prototype for yylex. + int yylmax; // Maximum buffer length if %array + trit yymore_really_used;// Force geberation of support code for yymore + bool yytext_is_array; // if true (i.e., %array directive), then declare + // yytext as array instead of a character pointer. + // Nice and inefficient. + bool noyyread; // User supplied a yyread function, don't generate default + char *userinit; // Code fragment to be inserted before scanning + char *preaction; // Code fragment to be inserted before each action + char *postaction; // Code fragment to be inserted after each action + char *emit; // Specify target language to emit. + char *yyterminate; // Set a non-default termination hook. + bool no_yypanic; // if true, no not generate default yypanic function + // flags corresponding to the huge mass of --no-yy options + bool no_yy_push_state; + bool no_yy_pop_state; + bool no_yy_top_state; + bool no_yyunput; + bool no_yy_scan_buffer; + bool no_yy_scan_bytes; + bool no_yy_scan_string; + bool no_yyget_extra; + bool no_yyset_extra; + bool no_yyget_leng; + bool no_yyget_text; + bool no_yyget_lineno; + bool no_yyset_lineno; + bool no_yyget_column; + bool no_yyset_column; + bool no_yyget_in; + bool no_yyset_in; + bool no_yyget_out; + bool no_yyset_out; + bool no_yyget_lval; + bool no_yyset_lval; + bool no_yyget_lloc; + bool no_yyset_lloc; + bool no_flex_alloc; + bool no_flex_realloc; + bool no_flex_free; + bool no_get_debug; + bool no_set_debug; + // Properties read from the skeleton + const char *backend_name; // What the back end tells you its name is + const char *traceline_re; // Regular expression for recognizing tracelines */ + const char *traceline_template; // templare for emitting trace lines */ + bool have_state_entry_format; // Do we know how to make a state entry address? +}; + +/* Environment variables. These control the lexer operation, but do + * not have corresponding m4 symbols and do not affect the behavior of + * the generated parser. + */ +struct env_bundle_t { + bool backing_up_report; // (-b flag), generate "lex.backup" file + // listing backing-up states + bool did_outfilename; // whether outfilename was explicitly set + char *headerfilename; // name of the .h file to generate + bool nowarn; // (-w) do not generate warnings + int performance_hint; // if > 0 (i.e., -p flag), generate a report + // relating to scanner performance; + // if > 1 (-p -p), report + // on minor performance problems, too. + char *outfilename; // output file name + bool printstats; // (-v) dump statistics + char *skelname; // name of skeleton for code generation + FILE *skelfile; // the skeleton file'd descriptor + bool trace; // (-T) env.trace processing + bool trace_hex; // use hex in trace/debug outputs not octal + bool use_stdout; // the -t flag +}; + +/* Name and byte-width information on a type for code-generation purposes. */ +struct packtype_t { + char *name; + size_t width; +}; +extern struct ctrl_bundle_t ctrl; +extern struct env_bundle_t env; /* Declarations for global variables. */ /* Variables for flags: - * printstats - if true (-v), dump statistics * syntaxerror - true if a syntax error has been found * eofseen - true if we've seen an eof in the input file - * ddebug - if true (-d), make a "debug" scanner - * trace - if true (-T), trace processing - * nowarn - if true (-w), do not generate warnings - * spprdflt - if true (-s), suppress the default rule - * interactive - if true (-I), generate an interactive scanner - * lex_compat - if true (-l), maximize compatibility with AT&T lex - * posix_compat - if true (-X), maximize compatibility with POSIX lex - * do_yylineno - if true, generate code to maintain yylineno - * useecs - if true (-Ce flag), use equivalence classes - * fulltbl - if true (-Cf flag), don't compress the DFA state table - * usemecs - if true (-Cm flag), use meta-equivalence classes - * fullspd - if true (-F flag), use Jacobson method of table representation - * gen_line_dirs - if true (i.e., no -L flag), generate #line directives - * performance_report - if > 0 (i.e., -p flag), generate a report relating - * to scanner performance; if > 1 (-p -p), report on minor performance - * problems, too - * backing_up_report - if true (i.e., -b flag), generate "lex.backup" file - * listing backing-up states - * C_plus_plus - if true (i.e., -+ flag), generate a C++ scanner class; - * otherwise, a standard C scanner - * reentrant - if true (-R), generate a reentrant C scanner. - * bison_bridge_lval - if true (--bison-bridge), bison pure calling convention. - * bison_bridge_lloc - if true (--bison-locations), bison yylloc. - * long_align - if true (-Ca flag), favor long-word alignment. - * use_read - if true (-f, -F, or -Cr) then use read() for scanner input; - * otherwise, use fread(). - * yytext_is_array - if true (i.e., %array directive), then declare - * yytext as a array instead of a character pointer. Nice and inefficient. - * do_yywrap - do yywrap() processing on EOF. If false, EOF treated as - * "no more files". - * csize - size of character set for the scanner we're generating; - * 128 for 7-bit chars and 256 for 8-bit * yymore_used - if true, yymore() is used in input rules * reject - if true, generate back-up tables for REJECT macro * real_reject - if true, scanner really uses REJECT (as opposed to just @@ -353,40 +440,20 @@ * yymore_really_used - whether to treat yymore() as really used, regardless * of what we think based on references to it in the user's actions. * reject_really_used - same for REJECT - * trace_hex - use hexadecimal numbers in trace/debug outputs instead of octals - */ + */ -extern int printstats, syntaxerror, eofseen, ddebug, trace, nowarn, - spprdflt; -extern int interactive, lex_compat, posix_compat, do_yylineno; -extern int useecs, fulltbl, usemecs, fullspd; -extern int gen_line_dirs, performance_report, backing_up_report; -extern int reentrant, bison_bridge_lval, bison_bridge_lloc; -extern int C_plus_plus, long_align, use_read, yytext_is_array, do_yywrap; -extern int csize; +extern int syntaxerror, eofseen; extern int yymore_used, reject, real_reject, continued_action, in_rule; -extern int yymore_really_used, reject_really_used; -extern int trace_hex; - /* Variables used in the flex input routines: * datapos - characters on current output line * dataline - number of contiguous lines of data in current data * statement. Used to generate readable -f output * linenum - current input line number - * skelfile - the skeleton file - * skel - compiled-in skeleton array * skel_ind - index into "skel" array, if skelfile is nil * yyin - input file - * backing_up_file - file to summarize backing-up states to * infilename - name of input file * outfilename - name of output file - * headerfilename - name of the .h file to generate - * did_outfilename - whether outfilename was explicitly set - * prefix - the prefix used for externally visible names ("yy" by default) - * yyclass - yyFlexLexer subclass to use for YY_DECL - * do_stdinit - whether to initialize yyin/yyout to stdin/stdout - * use_stdout - the -t flag * input_files - array holding names of input files * num_input_files - size of input_files array * program_name - name with which program was invoked @@ -402,13 +469,9 @@ extern int trace_hex; */ extern int datapos, dataline, linenum; -extern FILE *skelfile, *backing_up_file; -extern const char *skel[]; extern int skel_ind; -extern char *infilename, *outfilename, *headerfilename; -extern int did_outfilename; -extern char *prefix, *yyclass, *extra_type; -extern int do_stdinit, use_stdout; +extern char *infilename; +extern char *extra_type; extern char **input_files; extern int num_input_files; extern char *program_name; @@ -461,6 +524,7 @@ extern int onenext[ONE_STACK_SIZE], onedef[ONE_STACK_SIZE], onesp; * rule_has_nl - true if rule could possibly match a newline * ccl_has_nl - true if current ccl could match a newline * nlch - default eol char + * footprint - total size of tables, in bytes. */ extern int maximum_mns, current_mns, current_max_rules; @@ -470,6 +534,8 @@ extern int *accptnum, *assoc_rule, *state_type; extern int *rule_type, *rule_linenum, *rule_useful; extern bool *rule_has_nl, *ccl_has_nl; extern int nlch; +extern size_t footprint; + /* Different types of states; values are useful as masks, as well, for * routines like check_trailing_context(). @@ -609,6 +675,7 @@ extern unsigned char *ccltbl; /* Variables for miscellaneous information: * nmstr - last NAME scanned by the scanner + * nmval - last numeric scanned by the scanner * sectnum - section number currently being parsed * nummt - number of empty nxt/chk table entries * hshcol - number of hash collisions detected by snstods @@ -628,7 +695,7 @@ extern unsigned char *ccltbl; */ extern char nmstr[MAXLINE]; -extern int sectnum, nummt, hshcol, dfaeql, numeps, eps2, num_reallocs; +extern int sectnum, nummt, hshcol, dfaeql, numeps, eps2, num_reallocs, nmval; extern int tmpuses, totnst, peakpairs, numuniq, numdup, hshsave; extern int num_backing_up, bol_needed; @@ -707,7 +774,7 @@ extern int *epsclosure(int *, int *, int[], int *, int *); /* Increase the maximum number of dfas. */ extern void increase_max_dfas(void); -extern void ntod(void); /* convert a ndfa to a dfa */ +extern size_t ntod(void); /* convert a ndfa to a dfa */ /* Converts a set of ndfa states into a dfa state. */ extern int snstods(int[], int, int[], int, int, int *); @@ -732,34 +799,20 @@ extern void mkechar(int, int[], int[]); extern void do_indent(void); /* indent to the current level */ -/* Generate the code to keep backing-up information. */ -extern void gen_backing_up(void); +/* Set a conditional amd make it visible in generated code */ +extern void visible_define (const char *); -/* Generate the code to perform the backing up. */ -extern void gen_bu_action(void); +/* And again, with an explicit value part. */ +extern void visible_define_str (const char *, const char *); + +/* This time the value part is an int */ +extern void visible_define_int (const char *, const int); /* Generate full speed compressed transition table. */ extern void genctbl(void); -/* Generate the code to find the action number. */ -extern void gen_find_action(void); - -extern void genftbl(void); /* generate full transition table */ - -/* Generate the code to find the next compressed-table state. */ -extern void gen_next_compressed_state(char *); - -/* Generate the code to find the next match. */ -extern void gen_next_match(void); - -/* Generate the code to find the next state. */ -extern void gen_next_state(int); - -/* Generate the code to make a NUL transition. */ -extern void gen_NUL_trans(void); - -/* Generate the code to find the start state. */ -extern void gen_start_state(void); +/* generate full transition table */ +extern void genftbl(void); /* Generate data statements for the transition tables. */ extern void gentabs(void); @@ -770,8 +823,11 @@ extern void indent_put2s(const char *, const char *); /* Write out a string + newline at the current indentation level. */ extern void indent_puts(const char *); -extern void make_tables(void); /* generate transition tables */ +/* generate transition tables */ +extern void make_tables(void); +/* Select a type for optimal packing */ +struct packtype_t *optimize_pack(size_t); /* from file main.c */ @@ -782,9 +838,6 @@ extern void usage(void); /* from file misc.c */ -/* Add a #define to the action file. */ -extern void action_define(const char *defname, int value); - /* Add the given text to the stored actions. */ extern void add_action(const char *new_text); @@ -810,7 +863,7 @@ extern char *xstrdup(const char *); extern int cclcmp(const void *, const void *); /* Finish up a block of data declarations. */ -extern void dataend(void); +extern void dataend(const char *); /* Flush generated data statements. */ extern void dataflush(void); @@ -857,7 +910,7 @@ extern void lerr_fatal(const char *, ...) ; /* Spit out a "#line" statement. */ -extern void line_directive_out(FILE *, int); +extern void line_directive_out(FILE *, char *, int); /* Mark the current position in the action array as the end of the section 1 * user defs. @@ -884,7 +937,6 @@ extern void out_dec(const char *, int); extern void out_dec2(const char *, int, int); extern void out_hex(const char *, unsigned int); extern void out_str(const char *, const char *); -extern void out_str3(const char *, const char *, const char *, const char *); extern void out_str_dec(const char *, const char *, int); extern void outc(int); extern void outn(const char *); @@ -895,9 +947,6 @@ extern void out_m4_define(const char* def, const char* val); */ extern char *readable_form(int); -/* Write out one section of the skeleton file. */ -extern void skelout(void); - /* Output a yy_trans_info structure. */ extern void transition_struct_out(int, int); @@ -972,6 +1021,8 @@ extern void lwarn(const char *); /* report a warning */ extern void yyerror(const char *); /* report a parse error */ extern int yyparse(void); /* the YACC parser */ +/* Ship a comment to the generated output */ +extern void comment(const char *); /* from file scan.l */ @@ -981,6 +1032,22 @@ extern int flexscan(void); /* Open the given file (if NULL, stdin) for scanning. */ extern void set_input_file(char *); +/* from file skeletons.c */ + +/* return the correct file suffix for the selrcted back end */ +const char *suffix (void); + +/* Mine a text-valued property out of the skeleton file */ +extern const char *skel_property(const char *); + +/* Is the default back end selected?*/ +extern bool is_default_backend(void); + +/* Select a backend by name */ +extern void backend_by_name(const char *); + +/* Write out one section of the skeleton file. */ +extern void skelout(bool); /* from file sym.c */ @@ -1000,6 +1067,8 @@ extern void scinstal(const char *, int); /* make a start condition */ /* Lookup the number associated with a start condition. */ extern int sclookup(const char *); +/* Supply context argument for a function if required */ +extern void context_call(char *); /* from file tblcmp.c */ @@ -1043,22 +1112,12 @@ struct Buf { extern void buf_init(struct Buf * buf, size_t elem_size); extern void buf_destroy(struct Buf * buf); extern struct Buf *buf_append(struct Buf * buf, const void *ptr, int n_elem); -extern struct Buf *buf_concat(struct Buf* dest, const struct Buf* src); extern struct Buf *buf_strappend(struct Buf *, const char *str); extern struct Buf *buf_strnappend(struct Buf *, const char *str, int nchars); -extern struct Buf *buf_strdefine(struct Buf * buf, const char *str, const char *def); extern struct Buf *buf_prints(struct Buf *buf, const char *fmt, const char* s); -extern struct Buf *buf_m4_define(struct Buf *buf, const char* def, const char* val); -extern struct Buf *buf_m4_undefine(struct Buf *buf, const char* def); -extern struct Buf *buf_print_strings(struct Buf * buf, FILE* out); -extern struct Buf *buf_linedir(struct Buf *buf, const char* filename, int lineno); extern struct Buf userdef_buf; /* a string buffer for #define's generated by user-options on cmd line. */ -extern struct Buf defs_buf; /* a char* buffer to save #define'd some symbols generated by flex. */ -extern struct Buf yydmap_buf; /* a string buffer to hold yydmap elements */ -extern struct Buf m4defs_buf; /* Holds m4 definitions. */ extern struct Buf top_buf; /* contains %top code. String buffer. */ -extern bool no_section3_escape; /* True if the undocumented option --unsafe-no-m4-sect3-escape was passed */ /* For blocking out code from the header file. */ #define OUT_BEGIN_CODE() outn("m4_ifdef( [[M4_YY_IN_HEADER]],,[[m4_dnl") @@ -1129,7 +1188,7 @@ extern int filter_fix_linedirs(struct filter *chain); */ extern regex_t regex_linedir; -bool flex_init_regex(void); +bool flex_init_regex(const char *); void flex_regcomp(regex_t *preg, const char *regex, int cflags); char *regmatch_dup (regmatch_t * m, const char *src); char *regmatch_cpy (regmatch_t * m, char *dest, const char *src); @@ -34,69 +34,31 @@ #include "flexdef.h" #include "tables.h" +/* These typedefs are only used for computing footprint sizes, + * You need to make sure they match reality in the skeleton file to + * get accurate numbers, but they don't otherwise matter. + */ +typedef char YY_CHAR; +struct yy_trans_info {int32_t yy_verify; int32_t yy_nxt;}; /* declare functions that have forward references */ void genecs(void); - -static int indent_level = 0; /* each level is 8 spaces */ - -#define set_indent(indent_val) indent_level = indent_val - -/* Almost everything is done in terms of arrays starting at 1, so provide - * a null entry for the zero element of all C arrays. (The exception - * to this is that the fast table representation generally uses the - * 0 elements of its arrays, too.) - */ - -static const char *get_int16_decl (void) -{ - return (gentables) - ? "static const flex_int16_t %s[%d] =\n { 0,\n" - : "static const flex_int16_t * %s = 0;\n"; -} - - -static const char *get_int32_decl (void) -{ - return (gentables) - ? "static const flex_int32_t %s[%d] =\n { 0,\n" - : "static const flex_int32_t * %s = 0;\n"; -} - -static const char *get_state_decl (void) -{ - return (gentables) - ? "static const yy_state_type %s[%d] =\n { 0,\n" - : "static const yy_state_type * %s = 0;\n"; -} - -static const char *get_yy_char_decl (void) +struct packtype_t *optimize_pack(size_t sz) { - return (gentables) - ? "static const YY_CHAR %s[%d] =\n { 0,\n" - : "static const YY_CHAR * %s = 0;\n"; + /* FIXME: There's a 32-bit assumption lurking here */ + static struct packtype_t out; + if (sz == 0) { + out.name = ctrl.long_align ? "M4_HOOK_INT32" : "M4_HOOK_INT16"; + out.width = ctrl.long_align ? 32 : 16; + } else { + out.name = (ctrl.long_align || sz >= INT16_MAX) ? "M4_HOOK_INT32" : "M4_HOOK_INT16"; + out.width = (ctrl.long_align || sz >= INT16_MAX) ? 32 : 16; + } + return &out; } -/* Indent to the current level. */ - -void do_indent (void) -{ - int i = indent_level * 8; - - while (i >= 8) { - outc ('\t'); - i -= 8; - } - - while (i > 0) { - outc (' '); - --i; - } -} - - /** Make the table for possible eol matches. * @return the newly allocated rule_can_match_eol table */ @@ -116,9 +78,6 @@ static struct yytbl_data *mkeoltbl (void) for (i = 1; i <= num_rules; i++) tdata[i] = rule_has_nl[i] ? 1 : 0; - buf_prints (&yydmap_buf, - "\t{YYTD_ID_RULE_CAN_MATCH_EOL, (void**)&yy_rule_can_match_eol, sizeof(%s)},\n", - "flex_int32_t"); return tbl; } @@ -126,11 +85,12 @@ static struct yytbl_data *mkeoltbl (void) static void geneoltbl (void) { int i; + struct packtype_t *ptype = optimize_pack(num_rules); - outn ("m4_ifdef( [[M4_YY_USE_LINENO]],[["); - outn ("/* Table of booleans, true if rule could match eol. */"); - out_str_dec (get_int32_decl (), "yy_rule_can_match_eol", - num_rules + 1); + outn ("m4_ifdef( [[M4_MODE_YYLINENO]],[["); + out_str ("m4_define([[M4_HOOK_EOLTABLE_TYPE]], [[%s]])\n", ptype->name); + out_dec ("m4_define([[M4_HOOK_EOLTABLE_SIZE]], [[%d]])", num_rules + 1); + outn ("m4_define([[M4_HOOK_EOLTABLE_BODY]], [[m4_dnl"); if (gentables) { for (i = 1; i <= num_rules; i++) { @@ -139,61 +99,13 @@ static void geneoltbl (void) if ((i % 20) == 19) out ("\n "); } - out (" };\n"); } + footprint += num_rules * ptype->width; + outn ("]])"); outn ("]])"); } -/* Generate the code to keep backing-up information. */ - -void gen_backing_up (void) -{ - if (reject || num_backing_up == 0) - return; - - if (fullspd) - indent_puts ("if ( yy_current_state[-1].yy_nxt )"); - else - indent_puts ("if ( yy_accept[yy_current_state] )"); - - ++indent_level; - indent_puts ("{"); - indent_puts ("YY_G(yy_last_accepting_state) = yy_current_state;"); - indent_puts ("YY_G(yy_last_accepting_cpos) = yy_cp;"); - indent_puts ("}"); - --indent_level; -} - - -/* Generate the code to perform the backing up. */ - -void gen_bu_action (void) -{ - if (reject || num_backing_up == 0) - return; - - set_indent (3); - - indent_puts ("case 0: /* must back up */"); - indent_puts ("/* undo the effects of YY_DO_BEFORE_ACTION */"); - indent_puts ("*yy_cp = YY_G(yy_hold_char);"); - - if (fullspd || fulltbl) - indent_puts ("yy_cp = YY_G(yy_last_accepting_cpos) + 1;"); - else - /* Backing-up info for compressed tables is taken \after/ - * yy_cp has been incremented for the next state. - */ - indent_puts ("yy_cp = YY_G(yy_last_accepting_cpos);"); - - indent_puts ("yy_current_state = YY_G(yy_last_accepting_state);"); - indent_puts ("goto yy_find_action;"); - outc ('\n'); - - set_indent (0); -} - /** mkctbl - make full speed compressed transition table * This is an array of structs; each struct a pair of integers. * You should call mkssltbl() immediately after this. @@ -208,10 +120,8 @@ static struct yytbl_data *mkctbl (void) flex_int32_t *tdata = 0, curr = 0; int end_of_buffer_action = num_rules + 1; - buf_prints (&yydmap_buf, - "\t{YYTD_ID_TRANSITION, (void**)&yy_transition, sizeof(%s)},\n", - ((tblend + numecs + 1) >= INT16_MAX - || long_align) ? "flex_int32_t" : "flex_int16_t"); + struct packtype_t *ptype = optimize_pack(tblend + numecs + 1); + out_str ("m4_define([[M4_HOOK_MKCTBL_TYPE]], [[%s]])", ptype->name); tbl = calloc(1, sizeof (struct yytbl_data)); yytbl_data_init (tbl, YYTD_ID_TRANSITION); @@ -288,7 +198,6 @@ static struct yytbl_data *mkctbl (void) } } - /* Here's the final, end-of-buffer state. */ tdata[curr++] = chk[tblend + 1]; tdata[curr++] = nxt[tblend + 1]; @@ -321,10 +230,6 @@ static struct yytbl_data *mkssltbl (void) for (i = 0; i <= lastsc * 2; ++i) tdata[i] = base[i]; - buf_prints (&yydmap_buf, - "\t{YYTD_ID_START_STATE_LIST, (void**)&yy_start_state_list, sizeof(%s)},\n", - "struct yy_trans_info*"); - return tbl; } @@ -338,10 +243,8 @@ void genctbl (void) int end_of_buffer_action = num_rules + 1; /* Table of verify for transition and offset to next state. */ - if (gentables) - out_dec ("static const struct yy_trans_info yy_transition[%d] =\n {\n", tblend + numecs + 1); - else - outn ("static const struct yy_trans_info *yy_transition = 0;"); + out_dec ("m4_define([[M4_HOOK_TRANSTABLE_SIZE]], [[%d]])", tblend + numecs + 1); + outn ("m4_define([[M4_HOOK_TRANSTABLE_BODY]], [[m4_dnl"); /* We want the transition to be represented as the offset to the * next state, not the actual state number, which is what it currently @@ -408,25 +311,22 @@ void genctbl (void) transition_struct_out (chk[tblend + 1], nxt[tblend + 1]); transition_struct_out (chk[tblend + 2], nxt[tblend + 2]); - if (gentables) - outn (" };\n"); + outn ("]])"); + footprint += sizeof(struct yy_trans_info) * (tblend + numecs + 1); - /* Table of pointers to start states. */ - if (gentables) - out_dec ("static const struct yy_trans_info *yy_start_state_list[%d] =\n", lastsc * 2 + 1); - else - outn ("static const struct yy_trans_info **yy_start_state_list =0;"); + out_dec ("m4_define([[M4_HOOK_STARTTABLE_SIZE]], [[%d]])", lastsc * 2 + 1); if (gentables) { - outn (" {"); - + outn ("m4_define([[M4_HOOK_STARTTABLE_BODY]], [[m4_dnl"); for (i = 0; i <= lastsc * 2; ++i) - out_dec (" &yy_transition[%d],\n", base[i]); + out_dec ("M4_HOOK_STATE_ENTRY_FORMAT(%d)", base[i]); - dataend (); + dataend (NULL); + outn("]])"); + footprint += sizeof(struct yy_trans_info *) * (lastsc * 2 + 1); } - if (useecs) + if (ctrl.useecs) genecs (); } @@ -443,20 +343,16 @@ static struct yytbl_data *mkecstbl (void) yytbl_data_init (tbl, YYTD_ID_EC); tbl->td_flags |= YYTD_DATA32; tbl->td_hilen = 0; - tbl->td_lolen = (flex_uint32_t) csize; + tbl->td_lolen = (flex_uint32_t) ctrl.csize; tbl->td_data = tdata = calloc(tbl->td_lolen, sizeof (flex_int32_t)); - for (i = 1; i < csize; ++i) { + for (i = 1; i < ctrl.csize; ++i) { ecgroup[i] = ABS (ecgroup[i]); tdata[i] = ecgroup[i]; } - buf_prints (&yydmap_buf, - "\t{YYTD_ID_EC, (void**)&yy_ec, sizeof(%s)},\n", - "YY_CHAR"); - return tbl; } @@ -467,23 +363,26 @@ void genecs (void) int ch, row; int numrows; - out_str_dec (get_yy_char_decl (), "yy_ec", csize); + out_dec ("m4_define([[M4_HOOK_ECSTABLE_SIZE]], [[%d]])", ctrl.csize); + outn ("m4_define([[M4_HOOK_ECSTABLE_BODY]], [[m4_dnl"); - for (ch = 1; ch < csize; ++ch) { + for (ch = 1; ch < ctrl.csize; ++ch) { ecgroup[ch] = ABS (ecgroup[ch]); mkdata (ecgroup[ch]); } - dataend (); + dataend (NULL); + outn("]])"); + footprint += sizeof(YY_CHAR) * ctrl.csize; - if (trace) { + if (env.trace) { fputs (_("\n\nEquivalence Classes:\n\n"), stderr); /* Print in 8 columns */ - numrows = csize / 8; + numrows = ctrl.csize / 8; for (row = 0; row < numrows; ++row) { - for (ch = row; ch < csize; ch += numrows) { + for (ch = row; ch < ctrl.csize; ch += numrows) { fprintf (stderr, "%4s = %-2d", readable_form (ch), ecgroup[ch]); @@ -495,150 +394,6 @@ void genecs (void) } } - -/* Generate the code to find the action number. */ - -void gen_find_action (void) -{ - if (fullspd) - indent_puts ("yy_act = yy_current_state[-1].yy_nxt;"); - - else if (fulltbl) - indent_puts ("yy_act = yy_accept[yy_current_state];"); - - else if (reject) { - indent_puts ("yy_current_state = *--YY_G(yy_state_ptr);"); - indent_puts ("YY_G(yy_lp) = yy_accept[yy_current_state];"); - - if (!variable_trailing_context_rules) - outn ("m4_ifdef( [[M4_YY_USES_REJECT]],\n[["); - if(reject_really_used) - outn ("find_rule: /* we branch to this label when backing up */"); - if (!variable_trailing_context_rules) - outn ("]])\n"); - - indent_puts - ("for ( ; ; ) /* until we find what rule we matched */"); - - ++indent_level; - - indent_puts ("{"); - - indent_puts - ("if ( YY_G(yy_lp) && YY_G(yy_lp) < yy_accept[yy_current_state + 1] )"); - ++indent_level; - indent_puts ("{"); - indent_puts ("yy_act = yy_acclist[YY_G(yy_lp)];"); - - if (variable_trailing_context_rules) { - indent_puts - ("if ( yy_act & YY_TRAILING_HEAD_MASK ||"); - indent_puts (" YY_G(yy_looking_for_trail_begin) )"); - ++indent_level; - indent_puts ("{"); - - indent_puts - ("if ( yy_act == YY_G(yy_looking_for_trail_begin) )"); - ++indent_level; - indent_puts ("{"); - indent_puts ("YY_G(yy_looking_for_trail_begin) = 0;"); - indent_puts ("yy_act &= ~YY_TRAILING_HEAD_MASK;"); - indent_puts ("break;"); - indent_puts ("}"); - --indent_level; - - indent_puts ("}"); - --indent_level; - - indent_puts - ("else if ( yy_act & YY_TRAILING_MASK )"); - ++indent_level; - indent_puts ("{"); - indent_puts - ("YY_G(yy_looking_for_trail_begin) = yy_act & ~YY_TRAILING_MASK;"); - indent_puts - ("YY_G(yy_looking_for_trail_begin) |= YY_TRAILING_HEAD_MASK;"); - - if (real_reject) { - /* Remember matched text in case we back up - * due to REJECT. - */ - indent_puts - ("YY_G(yy_full_match) = yy_cp;"); - indent_puts - ("YY_G(yy_full_state) = YY_G(yy_state_ptr);"); - indent_puts ("YY_G(yy_full_lp) = YY_G(yy_lp);"); - } - - indent_puts ("}"); - --indent_level; - - indent_puts ("else"); - ++indent_level; - indent_puts ("{"); - indent_puts ("YY_G(yy_full_match) = yy_cp;"); - indent_puts - ("YY_G(yy_full_state) = YY_G(yy_state_ptr);"); - indent_puts ("YY_G(yy_full_lp) = YY_G(yy_lp);"); - indent_puts ("break;"); - indent_puts ("}"); - --indent_level; - - indent_puts ("++YY_G(yy_lp);"); - indent_puts ("goto find_rule;"); - } - - else { - /* Remember matched text in case we back up due to - * trailing context plus REJECT. - */ - ++indent_level; - indent_puts ("{"); - indent_puts ("YY_G(yy_full_match) = yy_cp;"); - indent_puts ("break;"); - indent_puts ("}"); - --indent_level; - } - - indent_puts ("}"); - --indent_level; - - indent_puts ("--yy_cp;"); - - /* We could consolidate the following two lines with those at - * the beginning, but at the cost of complaints that we're - * branching inside a loop. - */ - indent_puts ("yy_current_state = *--YY_G(yy_state_ptr);"); - indent_puts ("YY_G(yy_lp) = yy_accept[yy_current_state];"); - - indent_puts ("}"); - - --indent_level; - } - - else { /* compressed */ - indent_puts ("yy_act = yy_accept[yy_current_state];"); - - if (interactive && !reject) { - /* Do the guaranteed-needed backing up to figure out - * the match. - */ - indent_puts ("if ( yy_act == 0 )"); - ++indent_level; - indent_puts ("{ /* have to back up */"); - indent_puts - ("yy_cp = YY_G(yy_last_accepting_cpos);"); - indent_puts - ("yy_current_state = YY_G(yy_last_accepting_state);"); - indent_puts - ("yy_act = yy_accept[yy_current_state];"); - indent_puts ("}"); - --indent_level; - } - } -} - /* mkftbl - make the full table and return the struct . * you should call mkecstbl() after this. */ @@ -666,14 +421,11 @@ struct yytbl_data *mkftbl (void) tdata[i] = anum; - if (trace && anum) + if (env.trace && anum) fprintf (stderr, _("state # %d accepts: [%d]\n"), i, anum); } - buf_prints (&yydmap_buf, - "\t{YYTD_ID_ACCEPT, (void**)&yy_accept, sizeof(%s)},\n", - long_align ? "flex_int32_t" : "flex_int16_t"); return tbl; } @@ -684,25 +436,30 @@ void genftbl (void) { int i; int end_of_buffer_action = num_rules + 1; - - out_str_dec (long_align ? get_int32_decl () : get_int16_decl (), - "yy_accept", lastdfa + 1); + struct packtype_t *ptype = optimize_pack(num_rules + 1); dfaacc[end_of_buffer_state].dfaacc_state = end_of_buffer_action; + outn ("m4_define([[M4_HOOK_NEED_ACCEPT]], 1)"); + out_str ("m4_define([[M4_HOOK_ACCEPT_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_ACCEPT_SIZE]], [[%d]])", lastdfa + 1); + outn ("m4_define([[M4_HOOK_ACCEPT_BODY]], [[m4_dnl"); + for (i = 1; i <= lastdfa; ++i) { int anum = dfaacc[i].dfaacc_state; mkdata (anum); - if (trace && anum) + if (env.trace && anum) fprintf (stderr, _("state # %d accepts: [%d]\n"), i, anum); } - dataend (); + dataend (NULL); + outn("]])"); + footprint += (lastdfa + 1) * ptype->width; - if (useecs) + if (ctrl.useecs) genecs (); /* Don't have to dump the actual full table entries - they were @@ -710,356 +467,18 @@ void genftbl (void) */ } - -/* Generate the code to find the next compressed-table state. */ - -void gen_next_compressed_state (char *char_map) -{ - indent_put2s ("YY_CHAR yy_c = %s;", char_map); - - /* Save the backing-up info \before/ computing the next state - * because we always compute one more state than needed - we - * always proceed until we reach a jam state - */ - gen_backing_up (); - - indent_puts - ("while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state )"); - ++indent_level; - indent_puts ("{"); - indent_puts ("yy_current_state = (int) yy_def[yy_current_state];"); - - if (usemecs) { - /* We've arrange it so that templates are never chained - * to one another. This means we can afford to make a - * very simple test to see if we need to convert to - * yy_c's meta-equivalence class without worrying - * about erroneously looking up the meta-equivalence - * class twice - */ - do_indent (); - - /* lastdfa + 2 is the beginning of the templates */ - out_dec ("if ( yy_current_state >= %d )\n", lastdfa + 2); - - ++indent_level; - indent_puts ("yy_c = yy_meta[yy_c];"); - --indent_level; - } - - indent_puts ("}"); - --indent_level; - - indent_puts - ("yy_current_state = yy_nxt[yy_base[yy_current_state] + yy_c];"); -} - - -/* Generate the code to find the next match. */ - -void gen_next_match (void) -{ - /* NOTE - changes in here should be reflected in gen_next_state() and - * gen_NUL_trans(). - */ - char *char_map = useecs ? - "yy_ec[YY_SC_TO_UI(*yy_cp)] " : "YY_SC_TO_UI(*yy_cp)"; - - char *char_map_2 = useecs ? - "yy_ec[YY_SC_TO_UI(*++yy_cp)] " : "YY_SC_TO_UI(*++yy_cp)"; - - if (fulltbl) { - if (gentables) - indent_put2s - ("while ( (yy_current_state = yy_nxt[yy_current_state][ %s ]) > 0 )", - char_map); - else - indent_put2s - ("while ( (yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + %s ]) > 0 )", - char_map); - - ++indent_level; - - if (num_backing_up > 0) { - indent_puts ("{"); - gen_backing_up (); - outc ('\n'); - } - - indent_puts ("++yy_cp;"); - - if (num_backing_up > 0) - - indent_puts ("}"); - - --indent_level; - - outc ('\n'); - indent_puts ("yy_current_state = -yy_current_state;"); - } - - else if (fullspd) { - indent_puts ("{"); - indent_puts - ("const struct yy_trans_info *yy_trans_info;\n"); - indent_puts ("YY_CHAR yy_c;\n"); - indent_put2s ("for ( yy_c = %s;", char_map); - indent_puts - (" (yy_trans_info = &yy_current_state[yy_c])->"); - indent_puts ("yy_verify == yy_c;"); - indent_put2s (" yy_c = %s )", char_map_2); - - ++indent_level; - - if (num_backing_up > 0) - indent_puts ("{"); - - indent_puts ("yy_current_state += yy_trans_info->yy_nxt;"); - - if (num_backing_up > 0) { - outc ('\n'); - gen_backing_up (); - indent_puts ("}"); - } - - --indent_level; - indent_puts ("}"); - } - - else { /* compressed */ - indent_puts ("do"); - - ++indent_level; - indent_puts ("{"); - - gen_next_state (false); - - indent_puts ("++yy_cp;"); - - - indent_puts ("}"); - --indent_level; - - do_indent (); - - if (interactive) - out_dec ("while ( yy_base[yy_current_state] != %d );\n", jambase); - else - out_dec ("while ( yy_current_state != %d );\n", - jamstate); - - if (!reject && !interactive) { - /* Do the guaranteed-needed backing up to figure out - * the match. - */ - indent_puts - ("yy_cp = YY_G(yy_last_accepting_cpos);"); - indent_puts - ("yy_current_state = YY_G(yy_last_accepting_state);"); - } - } -} - - -/* Generate the code to find the next state. */ - -void gen_next_state (int worry_about_NULs) -{ /* NOTE - changes in here should be reflected in gen_next_match() */ - char char_map[256]; - - if (worry_about_NULs && !nultrans) { - if (useecs) - snprintf (char_map, sizeof(char_map), - "(*yy_cp ? yy_ec[YY_SC_TO_UI(*yy_cp)] : %d)", - NUL_ec); - else - snprintf (char_map, sizeof(char_map), - "(*yy_cp ? YY_SC_TO_UI(*yy_cp) : %d)", - NUL_ec); - } - - else - strcpy (char_map, useecs ? - "yy_ec[YY_SC_TO_UI(*yy_cp)] " : - "YY_SC_TO_UI(*yy_cp)"); - - if (worry_about_NULs && nultrans) { - if (!fulltbl && !fullspd) - /* Compressed tables back up *before* they match. */ - gen_backing_up (); - - indent_puts ("if ( *yy_cp )"); - ++indent_level; - indent_puts ("{"); - } - - if (fulltbl) { - if (gentables) - indent_put2s - ("yy_current_state = yy_nxt[yy_current_state][%s];", - char_map); - else - indent_put2s - ("yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + %s];", - char_map); - } - - else if (fullspd) - indent_put2s - ("yy_current_state += yy_current_state[%s].yy_nxt;", - char_map); - - else - gen_next_compressed_state (char_map); - - if (worry_about_NULs && nultrans) { - - indent_puts ("}"); - --indent_level; - indent_puts ("else"); - ++indent_level; - indent_puts - ("yy_current_state = yy_NUL_trans[yy_current_state];"); - --indent_level; - } - - if (fullspd || fulltbl) - gen_backing_up (); - - if (reject) - indent_puts ("*YY_G(yy_state_ptr)++ = yy_current_state;"); -} - - -/* Generate the code to make a NUL transition. */ - -void gen_NUL_trans (void) -{ /* NOTE - changes in here should be reflected in gen_next_match() */ - /* Only generate a definition for "yy_cp" if we'll generate code - * that uses it. Otherwise lint and the like complain. - */ - int need_backing_up = (num_backing_up > 0 && !reject); - - if (need_backing_up && (!nultrans || fullspd || fulltbl)) - /* We're going to need yy_cp lying around for the call - * below to gen_backing_up(). - */ - indent_puts ("char *yy_cp = YY_G(yy_c_buf_p);"); - - outc ('\n'); - - if (nultrans) { - indent_puts - ("yy_current_state = yy_NUL_trans[yy_current_state];"); - indent_puts ("yy_is_jam = (yy_current_state == 0);"); - } - - else if (fulltbl) { - do_indent (); - if (gentables) - out_dec ("yy_current_state = yy_nxt[yy_current_state][%d];\n", NUL_ec); - else - out_dec ("yy_current_state = yy_nxt[yy_current_state*YY_NXT_LOLEN + %d];\n", NUL_ec); - indent_puts ("yy_is_jam = (yy_current_state <= 0);"); - } - - else if (fullspd) { - do_indent (); - out_dec ("int yy_c = %d;\n", NUL_ec); - - indent_puts - ("const struct yy_trans_info *yy_trans_info;\n"); - indent_puts - ("yy_trans_info = &yy_current_state[(unsigned int) yy_c];"); - indent_puts ("yy_current_state += yy_trans_info->yy_nxt;"); - - indent_puts - ("yy_is_jam = (yy_trans_info->yy_verify != yy_c);"); - } - - else { - char NUL_ec_str[20]; - - snprintf (NUL_ec_str, sizeof(NUL_ec_str), "%d", NUL_ec); - gen_next_compressed_state (NUL_ec_str); - - do_indent (); - out_dec ("yy_is_jam = (yy_current_state == %d);\n", - jamstate); - - if (reject) { - /* Only stack this state if it's a transition we - * actually make. If we stack it on a jam, then - * the state stack and yy_c_buf_p get out of sync. - */ - indent_puts ("if ( ! yy_is_jam )"); - ++indent_level; - indent_puts - ("*YY_G(yy_state_ptr)++ = yy_current_state;"); - --indent_level; - } - } - - /* If we've entered an accepting state, back up; note that - * compressed tables have *already* done such backing up, so - * we needn't bother with it again. - */ - if (need_backing_up && (fullspd || fulltbl)) { - outc ('\n'); - indent_puts ("if ( ! yy_is_jam )"); - ++indent_level; - indent_puts ("{"); - gen_backing_up (); - indent_puts ("}"); - --indent_level; - } -} - - -/* Generate the code to find the start state. */ - -void gen_start_state (void) -{ - if (fullspd) { - if (bol_needed) { - indent_puts - ("yy_current_state = yy_start_state_list[YY_G(yy_start) + YY_AT_BOL()];"); - } - else - indent_puts - ("yy_current_state = yy_start_state_list[YY_G(yy_start)];"); - } - - else { - indent_puts ("yy_current_state = YY_G(yy_start);"); - - if (bol_needed) - indent_puts ("yy_current_state += YY_AT_BOL();"); - - if (reject) { - /* Set up for storing up states. */ - outn ("m4_ifdef( [[M4_YY_USES_REJECT]],\n[["); - indent_puts - ("YY_G(yy_state_ptr) = YY_G(yy_state_buf);"); - indent_puts - ("*YY_G(yy_state_ptr)++ = yy_current_state;"); - outn ("]])"); - } - } -} - - /* gentabs - generate data statements for the transition tables */ void gentabs (void) { - int i, j, k, *accset, nacc, *acc_array, total_states; + int sz, i, j, k, *accset, nacc, *acc_array, total_states; int end_of_buffer_action = num_rules + 1; struct yytbl_data *yyacc_tbl = 0, *yymeta_tbl = 0, *yybase_tbl = 0, - *yydef_tbl = 0, *yynxt_tbl = 0, *yychk_tbl = 0, *yyacclist_tbl=0; + *yydef_tbl = 0, *yynxt_tbl = 0, *yychk_tbl = 0, *yyacclist_tbl=0; flex_int32_t *yyacc_data = 0, *yybase_data = 0, *yydef_data = 0, - *yynxt_data = 0, *yychk_data = 0, *yyacclist_data=0; + *yynxt_data = 0, *yychk_data = 0, *yyacclist_data=0; flex_int32_t yybase_curr = 0, yyacclist_curr=0,yyacc_curr=0; + struct packtype_t *ptype; acc_array = allocate_integer_array (current_max_dfas); nummt = 0; @@ -1085,22 +504,20 @@ void gentabs (void) EOB_accepting_list[1] = end_of_buffer_action; accsiz[end_of_buffer_state] = 1; dfaacc[end_of_buffer_state].dfaacc_set = - EOB_accepting_list; - - out_str_dec (long_align ? get_int32_decl () : - get_int16_decl (), "yy_acclist", MAX (numas, - 1) + 1); - - buf_prints (&yydmap_buf, - "\t{YYTD_ID_ACCLIST, (void**)&yy_acclist, sizeof(%s)},\n", - long_align ? "flex_int32_t" : "flex_int16_t"); - - yyacclist_tbl = calloc(1,sizeof(struct yytbl_data)); - yytbl_data_init (yyacclist_tbl, YYTD_ID_ACCLIST); - yyacclist_tbl->td_lolen = (flex_uint32_t) (MAX(numas,1) + 1); - yyacclist_tbl->td_data = yyacclist_data = - calloc(yyacclist_tbl->td_lolen, sizeof (flex_int32_t)); - yyacclist_curr = 1; + EOB_accepting_list; + + sz = MAX (numas, 1) + 1; + ptype = optimize_pack(sz); + out_str ("m4_define([[M4_HOOK_ACCLIST_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_ACCLIST_SIZE]], [[%d]])", sz); + outn ("m4_define([[M4_HOOK_ACCLIST_BODY]], [[m4_dnl"); + + yyacclist_tbl = calloc(1,sizeof(struct yytbl_data)); + yytbl_data_init (yyacclist_tbl, YYTD_ID_ACCLIST); + yyacclist_tbl->td_lolen = (flex_uint32_t) (MAX(numas,1) + 1); + yyacclist_tbl->td_data = yyacclist_data = + calloc(yyacclist_tbl->td_lolen, sizeof (flex_int32_t)); + yyacclist_curr = 1; j = 1; /* index into "yy_acclist" array */ @@ -1111,7 +528,7 @@ void gentabs (void) accset = dfaacc[i].dfaacc_set; nacc = accsiz[i]; - if (trace) + if (env.trace) fprintf (stderr, _("state # %d accepts: "), i); @@ -1136,9 +553,9 @@ void gentabs (void) } mkdata (accnum); - yyacclist_data[yyacclist_curr++] = accnum; + yyacclist_data[yyacclist_curr++] = accnum; - if (trace) { + if (env.trace) { fprintf (stderr, "[%d]", accset[k]); @@ -1156,19 +573,21 @@ void gentabs (void) /* add accepting number for the "jam" state */ acc_array[i] = j; - dataend (); - if (tablesext) { - yytbl_data_compress (yyacclist_tbl); - if (yytbl_data_fwrite (&tableswr, yyacclist_tbl) < 0) - flexerror (_("Could not write yyacclist_tbl")); - yytbl_data_destroy (yyacclist_tbl); - yyacclist_tbl = NULL; - } + dataend (NULL); + outn("]])"); + footprint += sz * ptype->width; + if (tablesext) { + yytbl_data_compress (yyacclist_tbl); + if (yytbl_data_fwrite (&tableswr, yyacclist_tbl) < 0) + flexerror (_("Could not write yyacclist_tbl")); + yytbl_data_destroy (yyacclist_tbl); + yyacclist_tbl = NULL; + } } else { dfaacc[end_of_buffer_state].dfaacc_state = - end_of_buffer_action; + end_of_buffer_action; for (i = 1; i <= lastdfa; ++i) acc_array[i] = dfaacc[i].dfaacc_state; @@ -1187,7 +606,7 @@ void gentabs (void) /* "lastdfa + 2" is the size of "yy_accept"; includes room for C arrays * beginning at 0 and for "jam" state. */ - k = lastdfa + 2; + sz = lastdfa + 2; if (reject) /* We put a "cap" on the table associating lists of accepting @@ -1195,27 +614,27 @@ void gentabs (void) * where the end of an accepting list is by looking at where * the list for the next state starts. */ - ++k; + ++sz; - out_str_dec (long_align ? get_int32_decl () : get_int16_decl (), - "yy_accept", k); - - buf_prints (&yydmap_buf, - "\t{YYTD_ID_ACCEPT, (void**)&yy_accept, sizeof(%s)},\n", - long_align ? "flex_int32_t" : "flex_int16_t"); + /* Note that this table is alternately defined if ctrl.fulltbl */ + ptype = optimize_pack(sz); + outn ("m4_define([[M4_HOOK_NEED_ACCEPT]], 1)"); + out_str ("m4_define([[M4_HOOK_ACCEPT_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_ACCEPT_SIZE]], [[%d]])", sz); + outn ("m4_define([[M4_HOOK_ACCEPT_BODY]], [[m4_dnl"); yyacc_tbl = calloc(1, sizeof (struct yytbl_data)); yytbl_data_init (yyacc_tbl, YYTD_ID_ACCEPT); - yyacc_tbl->td_lolen = (flex_uint32_t) k; + yyacc_tbl->td_lolen = (flex_uint32_t) sz; yyacc_tbl->td_data = yyacc_data = - calloc(yyacc_tbl->td_lolen, sizeof (flex_int32_t)); - yyacc_curr=1; + calloc(yyacc_tbl->td_lolen, sizeof (flex_int32_t)); + yyacc_curr=1; for (i = 1; i <= lastdfa; ++i) { mkdata (acc_array[i]); yyacc_data[yyacc_curr++] = acc_array[i]; - if (!reject && trace && acc_array[i]) + if (!reject && env.trace && acc_array[i]) fprintf (stderr, _("state # %d accepts: [%d]\n"), i, acc_array[i]); } @@ -1230,7 +649,10 @@ void gentabs (void) yyacc_data[yyacc_curr++] = acc_array[i]; } - dataend (); + dataend (NULL); + outn ("]])"); + footprint += sz * ptype->width; + if (tablesext) { yytbl_data_compress (yyacc_tbl); if (yytbl_data_fwrite (&tableswr, yyacc_tbl) < 0) @@ -1240,7 +662,7 @@ void gentabs (void) yyacc_tbl = NULL; /* End generating yy_accept */ - if (useecs) { + if (ctrl.useecs) { genecs (); if (tablesext) { @@ -1255,7 +677,7 @@ void gentabs (void) } } - if (usemecs) { + if (ctrl.usemecs) { /* Begin generating yy_meta */ /* Write out meta-equivalence classes (used to index * templates with). @@ -1265,20 +687,17 @@ void gentabs (void) yytbl_data_init (yymeta_tbl, YYTD_ID_META); yymeta_tbl->td_lolen = (flex_uint32_t) (numecs + 1); yymeta_tbl->td_data = yymecs_data = - calloc(yymeta_tbl->td_lolen, - sizeof (flex_int32_t)); + calloc(yymeta_tbl->td_lolen, + sizeof (flex_int32_t)); - if (trace) + if (env.trace) fputs (_("\n\nMeta-Equivalence Classes:\n"), stderr); - - out_str_dec (get_yy_char_decl (), "yy_meta", numecs + 1); - buf_prints (&yydmap_buf, - "\t{YYTD_ID_META, (void**)&yy_meta, sizeof(%s)},\n", - "YY_CHAR"); - + out_dec ("m4_define([[M4_HOOK_MECSTABLE_SIZE]], [[%d]])", numecs+1); + outn ("m4_define([[M4_HOOK_MECSTABLE_BODY]], [[m4_dnl"); + for (i = 1; i <= numecs; ++i) { - if (trace) + if (env.trace) fprintf (stderr, "%d = %d\n", i, ABS (tecbck[i])); @@ -1286,7 +705,9 @@ void gentabs (void) yymecs_data[i] = ABS (tecbck[i]); } - dataend (); + dataend (NULL); + outn ("]])"); + footprint += sizeof(YY_CHAR) * (numecs + 1); if (tablesext) { yytbl_data_compress (yymeta_tbl); if (yytbl_data_fwrite (&tableswr, yymeta_tbl) < 0) @@ -1300,20 +721,18 @@ void gentabs (void) total_states = lastdfa + numtemps; /* Begin generating yy_base */ - out_str_dec ((tblend >= INT16_MAX || long_align) ? - get_int32_decl () : get_int16_decl (), - "yy_base", total_states + 1); - - buf_prints (&yydmap_buf, - "\t{YYTD_ID_BASE, (void**)&yy_base, sizeof(%s)},\n", - (tblend >= INT16_MAX - || long_align) ? "flex_int32_t" : "flex_int16_t"); + sz = total_states + 1; + ptype = optimize_pack(sz); + out_str ("m4_define([[M4_HOOK_BASE_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_BASE_SIZE]], [[%d]])", sz); + outn ("m4_define([[M4_HOOK_BASE_BODY]], [[m4_dnl"); + yybase_tbl = calloc (1, sizeof (struct yytbl_data)); yytbl_data_init (yybase_tbl, YYTD_ID_BASE); yybase_tbl->td_lolen = (flex_uint32_t) (total_states + 1); yybase_tbl->td_data = yybase_data = - calloc(yybase_tbl->td_lolen, - sizeof (flex_int32_t)); + calloc(yybase_tbl->td_lolen, + sizeof (flex_int32_t)); yybase_curr = 1; for (i = 1; i <= lastdfa; ++i) { @@ -1345,7 +764,10 @@ void gentabs (void) def[i] = jamstate; } - dataend (); + dataend (NULL); + outn ("]])"); + footprint += sz * ptype->width; + if (tablesext) { yytbl_data_compress (yybase_tbl); if (yytbl_data_fwrite (&tableswr, yybase_tbl) < 0) @@ -1357,27 +779,26 @@ void gentabs (void) /* Begin generating yy_def */ - out_str_dec ((total_states >= INT16_MAX || long_align) ? - get_int32_decl () : get_int16_decl (), - "yy_def", total_states + 1); - - buf_prints (&yydmap_buf, - "\t{YYTD_ID_DEF, (void**)&yy_def, sizeof(%s)},\n", - (total_states >= INT16_MAX - || long_align) ? "flex_int32_t" : "flex_int16_t"); + ptype = optimize_pack(total_states + 1); + out_str ("m4_define([[M4_HOOK_DEF_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_DEF_SIZE]], [[%d]])", total_states + 1); + outn ("m4_define([[M4_HOOK_DEF_BODY]], [[m4_dnl"); yydef_tbl = calloc(1, sizeof (struct yytbl_data)); yytbl_data_init (yydef_tbl, YYTD_ID_DEF); - yydef_tbl->td_lolen = (flex_uint32_t) (total_states + 1); + yydef_tbl->td_lolen = (flex_uint32_t)(total_states + 1); yydef_tbl->td_data = yydef_data = - calloc(yydef_tbl->td_lolen, sizeof (flex_int32_t)); + calloc(yydef_tbl->td_lolen, sizeof (flex_int32_t)); for (i = 1; i <= total_states; ++i) { mkdata (def[i]); yydef_data[i] = def[i]; } - dataend (); + dataend (NULL); + outn ("]])"); + footprint += (total_states + 1) * ptype->width; + if (tablesext) { yytbl_data_compress (yydef_tbl); if (yytbl_data_fwrite (&tableswr, yydef_tbl) < 0) @@ -1388,21 +809,19 @@ void gentabs (void) /* End generating yy_def */ - /* Begin generating yy_nxt */ - out_str_dec ((total_states >= INT16_MAX || long_align) ? - get_int32_decl () : get_int16_decl (), "yy_nxt", - tblend + 1); - - buf_prints (&yydmap_buf, - "\t{YYTD_ID_NXT, (void**)&yy_nxt, sizeof(%s)},\n", - (total_states >= INT16_MAX - || long_align) ? "flex_int32_t" : "flex_int16_t"); + ptype = optimize_pack(tblend + 1); + /* Note: Used when !ctrl.fulltbl && !ctrl.fullspd). + * (Alternately defined when ctrl.fullspd) + */ + out_str ("m4_define([[M4_HOOK_YYNXT_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_YYNXT_SIZE]], [[%d]])", tblend + 1); + outn ("m4_define([[M4_HOOK_YYNXT_BODY]], [[m4_dnl"); yynxt_tbl = calloc (1, sizeof (struct yytbl_data)); yytbl_data_init (yynxt_tbl, YYTD_ID_NXT); yynxt_tbl->td_lolen = (flex_uint32_t) (tblend + 1); yynxt_tbl->td_data = yynxt_data = - calloc (yynxt_tbl->td_lolen, sizeof (flex_int32_t)); + calloc (yynxt_tbl->td_lolen, sizeof (flex_int32_t)); for (i = 1; i <= tblend; ++i) { /* Note, the order of the following test is important. @@ -1415,7 +834,10 @@ void gentabs (void) yynxt_data[i] = nxt[i]; } - dataend (); + dataend (NULL); + outn("]])"); + footprint += ptype->width * (tblend + 1); + if (tablesext) { yytbl_data_compress (yynxt_tbl); if (yytbl_data_fwrite (&tableswr, yynxt_tbl) < 0) @@ -1426,20 +848,16 @@ void gentabs (void) /* End generating yy_nxt */ /* Begin generating yy_chk */ - out_str_dec ((total_states >= INT16_MAX || long_align) ? - get_int32_decl () : get_int16_decl (), "yy_chk", - tblend + 1); - - buf_prints (&yydmap_buf, - "\t{YYTD_ID_CHK, (void**)&yy_chk, sizeof(%s)},\n", - (total_states >= INT16_MAX - || long_align) ? "flex_int32_t" : "flex_int16_t"); - + ptype = optimize_pack(tblend + 1); + out_str ("m4_define([[M4_HOOK_CHK_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_CHK_SIZE]], [[%d]])", tblend + 1); + outn ("m4_define([[M4_HOOK_CHK_BODY]], [[m4_dnl"); + yychk_tbl = calloc (1, sizeof (struct yytbl_data)); yytbl_data_init (yychk_tbl, YYTD_ID_CHK); yychk_tbl->td_lolen = (flex_uint32_t) (tblend + 1); yychk_tbl->td_data = yychk_data = - calloc(yychk_tbl->td_lolen, sizeof (flex_int32_t)); + calloc(yychk_tbl->td_lolen, sizeof (flex_int32_t)); for (i = 1; i <= tblend; ++i) { if (chk[i] == 0) @@ -1449,7 +867,10 @@ void gentabs (void) yychk_data[i] = chk[i]; } - dataend (); + dataend (NULL); + outn ("]])"); + footprint += ptype->width * (tblend + 1); + if (tablesext) { yytbl_data_compress (yychk_tbl); if (yytbl_data_fwrite (&tableswr, yychk_tbl) < 0) @@ -1463,141 +884,44 @@ void gentabs (void) } -/* Write out a formatted string (with a secondary string argument) at the - * current indentation level, adding a final newline. - */ - -void indent_put2s (const char *fmt, const char *arg) +void visible_define (const char *symname) { - do_indent (); - out_str (fmt, arg); - outn (""); + out_m4_define(symname, NULL); + comment(symname); + outc ('\n'); } - -/* Write out a string at the current indentation level, adding a final - * newline. - */ - -void indent_puts (const char *str) +void visible_define_str (const char *symname, const char *val) { - do_indent (); - outn (str); + char buf[128]; + out_m4_define(symname, val); + snprintf(buf, sizeof(buf), "%s = %s", symname, val); + comment(buf); + outc ('\n'); } +void visible_define_int (const char *symname, const int val) +{ + char nbuf[24], buf[128]; + snprintf(nbuf, sizeof(nbuf), "%d", val); + out_m4_define(symname, nbuf); + snprintf(buf, sizeof(buf), "%s = %d", symname, val); + comment(buf); + outc ('\n'); +} -/* make_tables - generate transition tables and finishes generating output file +/* make_tables - generate transition tables */ void make_tables (void) { + char buf[128]; int i; - int did_eof_rule = false; struct yytbl_data *yynultrans_tbl = NULL; - - skelout (); /* %% [2.0] - break point in skel */ - - /* First, take care of YY_DO_BEFORE_ACTION depending on yymore - * being used. - */ - set_indent (1); - - if (yymore_used && !yytext_is_array) { - indent_puts ("YY_G(yytext_ptr) -= YY_G(yy_more_len); \\"); - indent_puts - ("yyleng = (int) (yy_cp - YY_G(yytext_ptr)); \\"); - } - - else - indent_puts ("yyleng = (int) (yy_cp - yy_bp); \\"); - - /* Now also deal with copying yytext_ptr to yytext if needed. */ - skelout (); /* %% [3.0] - break point in skel */ - if (yytext_is_array) { - if (yymore_used) - indent_puts - ("if ( yyleng + YY_G(yy_more_offset) >= YYLMAX ) \\"); - else - indent_puts ("if ( yyleng >= YYLMAX ) \\"); - - ++indent_level; - indent_puts - ("YY_FATAL_ERROR( \"token too large, exceeds YYLMAX\" ); \\"); - --indent_level; - - if (yymore_used) { - indent_puts - ("yy_flex_strncpy( &yytext[YY_G(yy_more_offset)], YY_G(yytext_ptr), yyleng + 1 M4_YY_CALL_LAST_ARG); \\"); - indent_puts ("yyleng += YY_G(yy_more_offset); \\"); - indent_puts - ("YY_G(yy_prev_more_offset) = YY_G(yy_more_offset); \\"); - indent_puts ("YY_G(yy_more_offset) = 0; \\"); - } - else { - indent_puts - ("yy_flex_strncpy( yytext, YY_G(yytext_ptr), yyleng + 1 M4_YY_CALL_LAST_ARG); \\"); - } - } - - set_indent (0); - - skelout (); /* %% [4.0] - break point in skel */ - - /* This is where we REALLY begin generating the tables. */ - out_dec ("#define YY_NUM_RULES %d\n", num_rules); - out_dec ("#define YY_END_OF_BUFFER %d\n", num_rules + 1); - - if (fullspd) { - /* Need to define the transet type as a size large - * enough to hold the biggest offset. - */ - int total_table_size = tblend + numecs + 1; - char *trans_offset_type = - (total_table_size >= INT16_MAX || long_align) ? - "flex_int32_t" : "flex_int16_t"; - - set_indent (0); - indent_puts ("struct yy_trans_info"); - ++indent_level; - indent_puts ("{"); - - /* We require that yy_verify and yy_nxt must be of the same size int. */ - indent_put2s ("%s yy_verify;", trans_offset_type); - - /* In cases where its sister yy_verify *is* a "yes, there is - * a transition", yy_nxt is the offset (in records) to the - * next state. In most cases where there is no transition, - * the value of yy_nxt is irrelevant. If yy_nxt is the -1th - * record of a state, though, then yy_nxt is the action number - * for that state. - */ - - indent_put2s ("%s yy_nxt;", trans_offset_type); - indent_puts ("};"); - --indent_level; - } - else { - /* We generate a bogus 'struct yy_trans_info' data type - * so we can guarantee that it is always declared in the skel. - * This is so we can compile "sizeof(struct yy_trans_info)" - * in any scanner. - */ - indent_puts - ("/* This struct is not used in this scanner,"); - indent_puts (" but its presence is necessary. */"); - indent_puts ("struct yy_trans_info"); - ++indent_level; - indent_puts ("{"); - indent_puts ("flex_int32_t yy_verify;"); - indent_puts ("flex_int32_t yy_nxt;"); - indent_puts ("};"); - --indent_level; - } - - if (fullspd) { + if (ctrl.fullspd) { genctbl (); if (tablesext) { struct yytbl_data *tbl; @@ -1615,7 +939,7 @@ void make_tables (void) yytbl_data_destroy (tbl); tbl = 0; - if (useecs) { + if (ctrl.useecs) { tbl = mkecstbl (); yytbl_data_compress (tbl); if (yytbl_data_fwrite (&tableswr, tbl) < 0) @@ -1626,19 +950,23 @@ void make_tables (void) } } } - else if (fulltbl) { + else if (ctrl.fulltbl) { genftbl (); if (tablesext) { struct yytbl_data *tbl; + /* Alternately defined if !ctrl.ffullspd && !ctrl.fulltbl */ + struct packtype_t *ptype; tbl = mkftbl (); yytbl_data_compress (tbl); + ptype = optimize_pack(tbl->td_lolen); + out_str ("m4_define([[M4_HOOK_ACCEPT_TYPE]], [[%s]])", ptype->name); if (yytbl_data_fwrite (&tableswr, tbl) < 0) flexerror (_("Could not write ftbl")); yytbl_data_destroy (tbl); tbl = 0; - if (useecs) { + if (ctrl.useecs) { tbl = mkecstbl (); yytbl_data_compress (tbl); if (yytbl_data_fwrite (&tableswr, tbl) < 0) @@ -1652,7 +980,28 @@ void make_tables (void) else gentabs (); - if (do_yylineno) { + snprintf(buf, sizeof(buf), "footprint: %ld bytes\n", footprint); + comment(buf); + outc ('\n'); + + // Only at this point do we know if the automaton has backups. + // Some m4 conditionals require this information. + + comment("m4 controls begin\n"); + + if (num_backing_up > 0) + visible_define ( "M4_MODE_HAS_BACKING_UP"); + + // These are used for NUL transitions + if ((num_backing_up > 0 && !reject) && (!nultrans || ctrl.fullspd || ctrl.fulltbl)) + visible_define ( "M4_MODE_NEED_YY_CP"); + if ((num_backing_up > 0 && !reject) && (ctrl.fullspd || ctrl.fulltbl)) + visible_define ( "M4_MODE_NULTRANS_WRAP"); + + comment("m4 controls end\n"); + out ("\n"); + + if (ctrl.do_yylineno) { geneoltbl (); @@ -1668,52 +1017,42 @@ void make_tables (void) } } - /* Definitions for backing up. We don't need them if REJECT - * is being used because then we use an alternative backin-up - * technique instead. - */ - if (num_backing_up > 0 && !reject) { - if (!C_plus_plus && !reentrant) { - indent_puts - ("static yy_state_type yy_last_accepting_state;"); - indent_puts - ("static char *yy_last_accepting_cpos;\n"); - } - } - if (nultrans) { flex_int32_t *yynultrans_data = 0; /* Begin generating yy_NUL_trans */ - out_str_dec (get_state_decl (), "yy_NUL_trans", - lastdfa + 1); - buf_prints (&yydmap_buf, - "\t{YYTD_ID_NUL_TRANS, (void**)&yy_NUL_trans, sizeof(%s)},\n", - (fullspd) ? "struct yy_trans_info*" : - "flex_int32_t"); + out_str ("m4_define([[M4_HOOK_NULTRANS_TYPE]], [[%s]])", (ctrl.fullspd) ? "struct yy_trans_info*" : "M4_HOOK_INT32"); + out_dec ("m4_define([[M4_HOOK_NULTRANS_SIZE]], [[%d]])", lastdfa + 1); + outn ("m4_define([[M4_HOOK_NULTRANS_BODY]], [[m4_dnl"); yynultrans_tbl = calloc(1, sizeof (struct yytbl_data)); yytbl_data_init (yynultrans_tbl, YYTD_ID_NUL_TRANS); - if (fullspd) + // Performance kludge for C. Gives a small improvement + // in table loading time. + if (ctrl.fullspd && ctrl.have_state_entry_format) yynultrans_tbl->td_flags |= YYTD_PTRANS; yynultrans_tbl->td_lolen = (flex_uint32_t) (lastdfa + 1); yynultrans_tbl->td_data = yynultrans_data = - calloc(yynultrans_tbl->td_lolen, - sizeof (flex_int32_t)); + calloc(yynultrans_tbl->td_lolen, + sizeof (flex_int32_t)); for (i = 1; i <= lastdfa; ++i) { - if (fullspd) { + if ((yynultrans_tbl->td_flags & YYTD_PTRANS) != 0) { + // Only works in very C-like languages out_dec (" &yy_transition[%d],\n", base[i]); yynultrans_data[i] = base[i]; } else { + // This will work anywhere mkdata (nultrans[i]); yynultrans_data[i] = nultrans[i]; } } - dataend (); + dataend (NULL); + outn("]])"); + footprint += (lastdfa + 1) * (ctrl.fullspd ? sizeof(struct yy_trans_info *) : sizeof(int32_t)); if (tablesext) { yytbl_data_compress (yynultrans_tbl); if (yytbl_data_fwrite (&tableswr, yynultrans_tbl) < @@ -1725,440 +1064,23 @@ void make_tables (void) if (yynultrans_tbl != NULL) { yytbl_data_destroy (yynultrans_tbl); yynultrans_tbl = NULL; - } + } /* End generating yy_NUL_trans */ } - if (!C_plus_plus && !reentrant) { - indent_puts ("extern int yy_flex_debug;"); - indent_put2s ("int yy_flex_debug = %s;\n", - ddebug ? "1" : "0"); - } + if (ctrl.ddebug) { /* Spit out table mapping rules to line numbers. */ + /* Policy choice: we don't include this space + * in the table metering. + */ + struct packtype_t *ptype = optimize_pack(num_rules); + out_str ("m4_define([[M4_HOOK_DEBUGTABLE_TYPE]], [[%s]])", ptype->name); + out_dec ("m4_define([[M4_HOOK_DEBUGTABLE_SIZE]], [[%d]])", num_rules); + outn ("m4_define([[M4_HOOK_DEBUGTABLE_BODY]], [[m4_dnl"); - if (ddebug) { /* Spit out table mapping rules to line numbers. */ - out_str_dec (long_align ? get_int32_decl () : - get_int16_decl (), "yy_rule_linenum", - num_rules); for (i = 1; i < num_rules; ++i) mkdata (rule_linenum[i]); - dataend (); - } - - if (reject) { - outn ("m4_ifdef( [[M4_YY_USES_REJECT]],\n[["); - /* Declare state buffer variables. */ - if (!C_plus_plus && !reentrant) { - outn ("static yy_state_type *yy_state_buf=0, *yy_state_ptr=0;"); - outn ("static char *yy_full_match;"); - outn ("static int yy_lp;"); - } - - if (variable_trailing_context_rules) { - if (!C_plus_plus && !reentrant) { - outn ("static int yy_looking_for_trail_begin = 0;"); - outn ("static int yy_full_lp;"); - outn ("static int *yy_full_state;"); - } - - out_hex ("#define YY_TRAILING_MASK 0x%x\n", - (unsigned int) YY_TRAILING_MASK); - out_hex ("#define YY_TRAILING_HEAD_MASK 0x%x\n", - (unsigned int) YY_TRAILING_HEAD_MASK); - } - - outn ("#define REJECT \\"); - outn ("{ \\"); - outn ("*yy_cp = YY_G(yy_hold_char); /* undo effects of setting up yytext */ \\"); - outn ("yy_cp = YY_G(yy_full_match); /* restore poss. backed-over text */ \\"); - - if (variable_trailing_context_rules) { - outn ("YY_G(yy_lp) = YY_G(yy_full_lp); /* restore orig. accepting pos. */ \\"); - outn ("YY_G(yy_state_ptr) = YY_G(yy_full_state); /* restore orig. state */ \\"); - outn ("yy_current_state = *YY_G(yy_state_ptr); /* restore curr. state */ \\"); - } - - outn ("++YY_G(yy_lp); \\"); - outn ("goto find_rule; \\"); - - outn ("}"); - outn ("]])\n"); - } - - else { - outn ("/* The intent behind this definition is that it'll catch"); - outn (" * any uses of REJECT which flex missed."); - outn (" */"); - outn ("#define REJECT reject_used_but_not_detected"); - } - - if (yymore_used) { - if (!C_plus_plus) { - if (yytext_is_array) { - if (!reentrant){ - indent_puts ("static int yy_more_offset = 0;"); - indent_puts ("static int yy_prev_more_offset = 0;"); - } - } - else if (!reentrant) { - indent_puts - ("static int yy_more_flag = 0;"); - indent_puts - ("static int yy_more_len = 0;"); - } - } - - if (yytext_is_array) { - indent_puts - ("#define yymore() (YY_G(yy_more_offset) = yy_flex_strlen( yytext M4_YY_CALL_LAST_ARG))"); - indent_puts ("#define YY_NEED_STRLEN"); - indent_puts ("#define YY_MORE_ADJ 0"); - indent_puts - ("#define YY_RESTORE_YY_MORE_OFFSET \\"); - ++indent_level; - indent_puts ("{ \\"); - indent_puts - ("YY_G(yy_more_offset) = YY_G(yy_prev_more_offset); \\"); - indent_puts ("yyleng -= YY_G(yy_more_offset); \\"); - indent_puts ("}"); - --indent_level; - } - else { - indent_puts - ("#define yymore() (YY_G(yy_more_flag) = 1)"); - indent_puts - ("#define YY_MORE_ADJ YY_G(yy_more_len)"); - indent_puts ("#define YY_RESTORE_YY_MORE_OFFSET"); - } - } - - else { - indent_puts - ("#define yymore() yymore_used_but_not_detected"); - indent_puts ("#define YY_MORE_ADJ 0"); - indent_puts ("#define YY_RESTORE_YY_MORE_OFFSET"); - } - - if (!C_plus_plus) { - if (yytext_is_array) { - outn ("#ifndef YYLMAX"); - outn ("#define YYLMAX 8192"); - outn ("#endif\n"); - if (!reentrant){ - outn ("char yytext[YYLMAX];"); - outn ("char *yytext_ptr;"); - } - } - - else { - if(! reentrant) - outn ("char *yytext;"); - } - } - - out (&action_array[defs1_offset]); - - line_directive_out (stdout, 0); - - skelout (); /* %% [5.0] - break point in skel */ - - if (!C_plus_plus) { - if (use_read) { - outn ("\terrno=0; \\"); - outn ("\twhile ( (result = (int) read( fileno(yyin), buf, (yy_size_t) max_size )) < 0 ) \\"); - outn ("\t{ \\"); - outn ("\t\tif( errno != EINTR) \\"); - outn ("\t\t{ \\"); - outn ("\t\t\tYY_FATAL_ERROR( \"input in flex scanner failed\" ); \\"); - outn ("\t\t\tbreak; \\"); - outn ("\t\t} \\"); - outn ("\t\terrno=0; \\"); - outn ("\t\tclearerr(yyin); \\"); - outn ("\t}\\"); - } - - else { - outn ("\tif ( YY_CURRENT_BUFFER_LVALUE->yy_is_interactive ) \\"); - outn ("\t\t{ \\"); - outn ("\t\tint c = '*'; \\"); - outn ("\t\tint n; \\"); - outn ("\t\tfor ( n = 0; n < max_size && \\"); - outn ("\t\t\t (c = getc( yyin )) != EOF && c != '\\n'; ++n ) \\"); - outn ("\t\t\tbuf[n] = (char) c; \\"); - outn ("\t\tif ( c == '\\n' ) \\"); - outn ("\t\t\tbuf[n++] = (char) c; \\"); - outn ("\t\tif ( c == EOF && ferror( yyin ) ) \\"); - outn ("\t\t\tYY_FATAL_ERROR( \"input in flex scanner failed\" ); \\"); - outn ("\t\tresult = n; \\"); - outn ("\t\t} \\"); - outn ("\telse \\"); - outn ("\t\t{ \\"); - outn ("\t\terrno=0; \\"); - outn ("\t\twhile ( (result = (int) fread(buf, 1, (yy_size_t) max_size, yyin)) == 0 && ferror(yyin)) \\"); - outn ("\t\t\t{ \\"); - outn ("\t\t\tif( errno != EINTR) \\"); - outn ("\t\t\t\t{ \\"); - outn ("\t\t\t\tYY_FATAL_ERROR( \"input in flex scanner failed\" ); \\"); - outn ("\t\t\t\tbreak; \\"); - outn ("\t\t\t\t} \\"); - outn ("\t\t\terrno=0; \\"); - outn ("\t\t\tclearerr(yyin); \\"); - outn ("\t\t\t} \\"); - outn ("\t\t}\\"); - } - } - - skelout (); /* %% [6.0] - break point in skel */ - - indent_puts ("#define YY_RULE_SETUP \\"); - ++indent_level; - if (bol_needed) { - indent_puts ("if ( yyleng > 0 ) \\"); - ++indent_level; - indent_puts ("YY_CURRENT_BUFFER_LVALUE->yy_at_bol = \\"); - indent_puts ("\t\t(yytext[yyleng - 1] == '\\n'); \\"); - --indent_level; - } - indent_puts ("YY_USER_ACTION"); - --indent_level; - - skelout (); /* %% [7.0] - break point in skel */ - - /* Copy prolog to output file. */ - out (&action_array[prolog_offset]); - - line_directive_out (stdout, 0); - - skelout (); /* %% [8.0] - break point in skel */ - - set_indent (2); - - if (yymore_used && !yytext_is_array) { - indent_puts ("YY_G(yy_more_len) = 0;"); - indent_puts ("if ( YY_G(yy_more_flag) )"); - ++indent_level; - indent_puts ("{"); - indent_puts - ("YY_G(yy_more_len) = (int) (YY_G(yy_c_buf_p) - YY_G(yytext_ptr));"); - indent_puts ("YY_G(yy_more_flag) = 0;"); - indent_puts ("}"); - --indent_level; - } - - skelout (); /* %% [9.0] - break point in skel */ - - gen_start_state (); - - /* Note, don't use any indentation. */ - outn ("yy_match:"); - gen_next_match (); - - skelout (); /* %% [10.0] - break point in skel */ - set_indent (2); - gen_find_action (); - - skelout (); /* %% [11.0] - break point in skel */ - outn ("m4_ifdef( [[M4_YY_USE_LINENO]],[["); - indent_puts - ("if ( yy_act != YY_END_OF_BUFFER && yy_rule_can_match_eol[yy_act] )"); - ++indent_level; - indent_puts ("{"); - indent_puts ("int yyl;"); - do_indent (); - out_str ("for ( yyl = %s; yyl < yyleng; ++yyl )\n", - yymore_used ? (yytext_is_array ? "YY_G(yy_prev_more_offset)" : - "YY_G(yy_more_len)") : "0"); - ++indent_level; - indent_puts ("if ( yytext[yyl] == '\\n' )"); - ++indent_level; - indent_puts ("M4_YY_INCR_LINENO();"); - --indent_level; - --indent_level; - indent_puts ("}"); - --indent_level; - outn ("]])"); - - skelout (); /* %% [12.0] - break point in skel */ - if (ddebug) { - indent_puts ("if ( yy_flex_debug )"); - ++indent_level; - - indent_puts ("{"); - indent_puts ("if ( yy_act == 0 )"); - ++indent_level; - indent_puts (C_plus_plus ? - "std::cerr << \"--scanner backing up\\n\";" : - "fprintf( stderr, \"--scanner backing up\\n\" );"); - --indent_level; - - do_indent (); - out_dec ("else if ( yy_act < %d )\n", num_rules); - ++indent_level; - - if (C_plus_plus) { - indent_puts - ("std::cerr << \"--accepting rule at line \" << yy_rule_linenum[yy_act] <<"); - indent_puts - (" \"(\\\"\" << yytext << \"\\\")\\n\";"); - } - else { - indent_puts - ("fprintf( stderr, \"--accepting rule at line %ld (\\\"%s\\\")\\n\","); - - indent_puts - (" (long)yy_rule_linenum[yy_act], yytext );"); - } - - --indent_level; - - do_indent (); - out_dec ("else if ( yy_act == %d )\n", num_rules); - ++indent_level; - - if (C_plus_plus) { - indent_puts - ("std::cerr << \"--accepting default rule (\\\"\" << yytext << \"\\\")\\n\";"); - } - else { - indent_puts - ("fprintf( stderr, \"--accepting default rule (\\\"%s\\\")\\n\","); - indent_puts (" yytext );"); - } - - --indent_level; - - do_indent (); - out_dec ("else if ( yy_act == %d )\n", num_rules + 1); - ++indent_level; - - indent_puts (C_plus_plus ? - "std::cerr << \"--(end of buffer or a NUL)\\n\";" : - "fprintf( stderr, \"--(end of buffer or a NUL)\\n\" );"); - - --indent_level; - - do_indent (); - outn ("else"); - ++indent_level; - - if (C_plus_plus) { - indent_puts - ("std::cerr << \"--EOF (start condition \" << YY_START << \")\\n\";"); - } - else { - indent_puts - ("fprintf( stderr, \"--EOF (start condition %d)\\n\", YY_START );"); - } - - --indent_level; - - indent_puts ("}"); - --indent_level; - } - - /* Copy actions to output file. */ - skelout (); /* %% [13.0] - break point in skel */ - ++indent_level; - gen_bu_action (); - out (&action_array[action_offset]); - - line_directive_out (stdout, 0); - - /* generate cases for any missing EOF rules */ - for (i = 1; i <= lastsc; ++i) - if (!sceof[i]) { - do_indent (); - out_str ("case YY_STATE_EOF(%s):\n", scname[i]); - did_eof_rule = true; - } - - if (did_eof_rule) { - ++indent_level; - indent_puts ("yyterminate();"); - --indent_level; - } - - - /* Generate code for handling NUL's, if needed. */ - - /* First, deal with backing up and setting up yy_cp if the scanner - * finds that it should JAM on the NUL. - */ - skelout (); /* %% [14.0] - break point in skel */ - set_indent (4); - - if (fullspd || fulltbl) - indent_puts ("yy_cp = YY_G(yy_c_buf_p);"); - - else { /* compressed table */ - if (!reject && !interactive) { - /* Do the guaranteed-needed backing up to figure - * out the match. - */ - indent_puts - ("yy_cp = YY_G(yy_last_accepting_cpos);"); - indent_puts - ("yy_current_state = YY_G(yy_last_accepting_state);"); - } - - else - /* Still need to initialize yy_cp, though - * yy_current_state was set up by - * yy_get_previous_state(). - */ - indent_puts ("yy_cp = YY_G(yy_c_buf_p);"); - } - - - /* Generate code for yy_get_previous_state(). */ - set_indent (1); - skelout (); /* %% [15.0] - break point in skel */ - - gen_start_state (); - - set_indent (2); - skelout (); /* %% [16.0] - break point in skel */ - gen_next_state (true); - - set_indent (1); - skelout (); /* %% [17.0] - break point in skel */ - gen_NUL_trans (); - - skelout (); /* %% [18.0] - break point in skel */ - skelout (); /* %% [19.0] - break point in skel */ - /* Update BOL and yylineno inside of input(). */ - if (bol_needed) { - indent_puts - ("YY_CURRENT_BUFFER_LVALUE->yy_at_bol = (c == '\\n');"); - if (do_yylineno) { - indent_puts - ("if ( YY_CURRENT_BUFFER_LVALUE->yy_at_bol )"); - ++indent_level; - indent_puts ("M4_YY_INCR_LINENO();"); - --indent_level; - } - } - - else if (do_yylineno) { - indent_puts ("if ( c == '\\n' )"); - ++indent_level; - indent_puts ("M4_YY_INCR_LINENO();"); - --indent_level; - } - - skelout (); - - /* Copy remainder of input to output. */ - - line_directive_out (stdout, 1); - - if (sectnum == 3) { - OUT_BEGIN_CODE (); - if (!no_section3_escape) - fputs("[[", stdout); - (void) flexscan (); /* copy remainder of input to output */ - if (!no_section3_escape) - fputs("]]", stdout); - OUT_END_CODE (); + dataend (NULL); + outn("]])"); } } diff --git a/src/go-flex.skl b/src/go-flex.skl new file mode 100644 index 0000000..39b3cd1 --- /dev/null +++ b/src/go-flex.skl @@ -0,0 +1,2334 @@ +%# -*-C-*- vi: set ft=c: +%# +%# This is a fake Go back end. It clones C99, giving output files +%# a .go extension but generating C code. Its purpose is to verify build and test be +%# before actually trying to generate Go. +%# +%# Differences from C: +%# +%# 1. The prefix property doesn't set a prefix for function and variable names. +%# Instead it sets the Go package name. +%# +%# 2. The (normally hidden) type of the yyguts_t structure becomes FlexLexer, +%# as in the pre-existing C++ support. The obfuscation of this type by +%# a void* i s removed. +%# +%# 3. Because of where expansion of magic names like yytext is done, the _r suffix +%# on some public member names is not required to keep them from being clobbered +%# and has been removed. +%# +%# 4. The following names change: +%# +%# YY_END_OF_BUFFER_CHAR -> flexBufferSentinel +%# YY_READ_BUF_SIZE -> flexReadBufferSize +%# YY_BUF_SIZE -> flexInputBufferSize +%# YY_NUL -> flexEOF + +%# Macros for preproc stage. +m4preproc_changecom + +%# Macros for runtime processing stage. +m4_changecom +m4_changequote +m4_changequote([[, ]]) + +%# Properties not used in the skeleton - meant to be read by the Flex code +m4_define([[M4_PROPERTY_BACKEND_NAME]], [[Go]]) +m4_define([[M4_PROPERTY_SOURCE_SUFFIX]], [[go]]) +m4_define([[M4_PROPERTY_TRACE_LINE_REGEXP]], [[^#line ([0-9]+) "(.*)"]]) +m4_define([[M4_PROPERTY_TRACE_LINE_TEMPLATE]], [[#line %d "%s"]]) +m4_define([[M4_PROPERTY_CONTEXT_ARG]], [[yyscanner]]) +m4_define([[M4_PROPERTY_CONTEXT_FORMAT]], [[yyscanner->%s]]) +m4_define([[M4_PROPERTY_BUFFERSTACK_CONTEXT_FORMAT]], [[yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->bs_%s]]) +m4_define([[M4_PROPERTY_PREFIX]], [main]]) + +%# Macro hooks used by Flex code generators start here +m4_define([[M4_HOOK_INT32]], [[int32_t]]) +m4_define([[M4_HOOK_INT16]], [[int16_t]]) +m4_define([[M4_HOOK_COMMENT_OPEN]], [[/*]]) +m4_define([[M4_HOOK_COMMENT_CLOSE]], [[*/]]) +%# If this is not defined, no trace lines will be generated. +m4_define([[M4_HOOK_TRACE_LINE_FORMAT]], [[#line $1 "$2" +]]) +m4_define([[M4_HOOK_TABLE_OPENER]], [[{]]) +m4_define([[M4_HOOK_TABLE_CONTINUE]], [[},]]) +m4_define([[M4_HOOK_TABLE_CLOSER]], [[};]]) +m4_define([[M4_HOOK_RELATIVIZE]], [[$1]]) +m4_define([[M4_HOOK_STATE_ENTRY_FORMAT]], [[ &yyTransition[$1], +]]) +m4_define([[M4_HOOK_NORMAL_STATE_CASE_ARM]], [[ case $1:]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_ARM]], [[ case YY_STATE_EOF($1):]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_FALLTHROUGH]], [[ /* FALLTHROUGH */]]) +m4_define([[M4_HOOK_EOF_STATE_CASE_TERMINATE]], [[ yyterminate(); +]]) +m4_define([[M4_HOOK_TAKE_YYTEXT]], [[yyDoBeforeAction(yyscanner, yyCp, yyBp); /* set up yytext */]]) +m4_define([[M4_HOOK_RELEASE_YYTEXT]], [[*yyCp = yyscanner->yyHoldChar; /* undo effects of setting up yytext */]]) +m4_define([[M4_HOOK_CHAR_REWIND]], [[yyscanner->yyCBufP = yyCp -= $1;]]) +m4_define([[M4_HOOK_LINE_REWIND]], [[yyLinenoRewindTo(yyCp, yyCp - $1, yyscanner);]]) +m4_define([[M4_HOOK_CHAR_FORWARD]], [[yyscanner->yyCBufP = yyCp = yyBp + $1;]]) +m4_define([[M4_HOOK_LINE_FORWARD]], [[yyLinenoRewindTo(yyCp, yyBp + $1, yyscanner);]]) +m4_define([[M4_HOOK_CONST_DEFINE_BYTE]], [[const char $1 = $2; +]]) +m4_define([[M4_HOOK_CONST_DEFINE_STATE]], [[#define $1 $2 +]]) +m4_define([[M4_HOOK_CONST_DEFINE_UINT]], [[const uint $1 = $2; +]]) +m4_define([[M4_HOOK_CONST_DEFINE_BOOL]], [[const bool $1 = $2; +]]) +m4_define([[M4_HOOK_CONST_DEFINE_UNKNOWN]], [[m4_define($1, [[$2]])]]) +m4_define([[M4_HOOK_SET_YY_DECL]], [[m4_define([[YY_DECL]], [[$1]])]]) +m4_define([[M4_HOOK_SET_OFFSET_TYPE]], [[m4_define([[YY_OFFSET_TYPE]], [[$1]])]]) +m4_define([[M4_HOOK_SET_USERINIT]], [[m4_define([[YY_USER_INIT]], [[$1]])]]) +m4_define([[M4_HOOK_SET_RULE_SETUP]], [[m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ + rule_check_bol(yyscanner); +]]) m4_ifdef([[YY_USER_ACTION]], YY_USER_ACTION) +]]) +m4_define([[M4_HOOK_SET_PREACTION]], [[m4_define([[YY_USER_ACTION]], [[$1]])]]) +m4_define([[M4_HOOK_STATE_CASE_BREAK]], [[/*LINTED*/break;]]) +m4_define([[M4_HOOK_SET_POSTACTION]], [[m4_define([[M4_HOOK_STATE_CASE_BREAK]], [[$1]])]]) +m4_define([[M4_HOOK_FATAL_ERROR]], [[yypanic($1, yyscanner);]]) +m4_define([[M4_HOOK_ECHO]], [[yyecho(yyscanner);]]) + +m4_define([[yyterminate]], m4_ifdef([[M4_MODE_YYTERMINATE]], [[M4_MODE_YYTERMINATE /* $1 */]], [[return flexEOF /* $1 */]])) + +%# Return all but the first "n" matched characters back to the input stream. +m4_define([[yyless]], [[ + do { + /* Undo effects of setting up yytext. */ + int yylessMacroArg = ($1); + m4_ifdef([[M4_MODE_YYLINENO]], [[yyLessLineno(yylessMacroArg, yyscanner);]]) + *yyCp = yyscanner->yyHoldChar; + YY_RESTORE_YY_MORE_OFFSET + yyscanner->yyCBufP = yyCp = yyBp + yylessMacroArg - YY_MORE_ADJ; + yyDoBeforeAction(yyscanner, yyCp, yyBp); /* set up yytext again */ + } while (0) +]]) + +%# Can't be a function given the context-arg treatment due to the goto. +m4_define([[M4_HOOK_REJECT]], [[{ + *yyCp = yyscanner->yyHoldChar; /* undo effects of setting up yytext */ + yyCp = yyscanner->yyFullMatch; /* restore poss. backed-over text */ +m4_ifdef([[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + yyscanner->yyLp = yyscanner->yyFullLp; /* restore orig. accepting pos. */ + yyscanner->yyStatePtr = yyscanner->yyFullState; /* restore orig. state */ + yyCurrentState = *yyscanner->yyStatePtr; /* restore curr. state */ +]]) + ++yyscanner->yyLp; + goto findRuleLabel; +} +]]) + +%% [0.0] Make hook macros available to Flex + +// package M4_MODE_PREFIX + +const int YY_FLEX_MAJOR_VERSION = FLEX_MAJOR_VERSION; +const int YY_FLEX_MINOR_VERSION = FLEX_MINOR_VERSION; +const int YY_FLEX_SUBMINOR_VERSION = FLEX_SUBMINOR_VERSION; + +/* STARTS platform-specific and compiler-specific definitions. */ + +m4_ifdef([[M4_YY_ALWAYS_INTERACTIVE]], , +[[m4_ifdef([[M4_YY_NEVER_INTERACTIVE]], , +[[/* Feature test macros. Flex uses functions that require a minimum set of + * macros defined. As defining some macros may hide function declarations that + * user code might use, be conservative and respect user's definitions as much + * as possible. In glibc, feature test macros may not be all set up until one + * of the libc header (that includes <features.h>) is included. This creates + * a circular dependency when we check the macros. <assert.h> is the safest + * header we can include and does not declare too many functions we don't need. + */ +#if !defined(__GNU_LIBRARY__) && defined(__STDC__) +#include <assert.h> +#endif +#if !(defined(_POSIX_C_SOURCE) || defined(_XOPEN_SOURCE) || \ + defined(_POSIX_SOURCE)) +# define _POSIX_C_SOURCE 1 /* Required for fileno() */ +# define _POSIX_SOURCE 1 +#endif]])]]) + +#include <stdio.h> +#include <string.h> +#include <errno.h> +#include <stdlib.h> +#include <stdbool.h> +#include <stdint.h> +#include <unistd.h> /* requred for isatty() */ + +#if defined(__GNUC__) && __GNUC__ >= 3 +#define yynoreturn __attribute__((__noreturn__)) +#else +#define yynoreturn +#endif + +/* + * Anywhere other than C this won't be a thing, + * because strings will have an associated length field. + */ +const int flexBufferSentinel = 0; + +/* ENDS platform-specific and compiler-specific definitions. */ + +/* + * Amount of stuff to slurp up with each read. + * We assume the stdio library has already + * chosen a fit size foe whatever platform + * we're running on. + */ +const int flexReadBufferSize = BUFSIZ; + +/* Size of default input buffer. We want to be able to fit two + * OS-level reads, but efficiency gains as the buffer size + * increases fall off after that + */ +const int flexInputBufferSize = m4_ifdef([[M4_MODE_YY_BUFSIZE]], [[M4_MODE_YY_BUFSIZE]], [[2 * flexReadBufferSize]]); + +/* Returned upon end-of-file. */ +const int flexEOF = 0; + +/* Promotes a possibly negative, possibly signed char to an + * integer in range [0..255] for use as an array index. + */ +m4_define([[YY_SC_TO_UI]], [[((YY_CHAR)($1))]]) + +/* Action number for EOF rule of a given start state. */ +m4_define([[YY_STATE_EOF]], [[YY_END_OF_BUFFER + $1 + 1]]) + +/* The state buf must be large enough to hold one state per character in the main buffer. + */ +m4_define([[YY_STATE_BUF_SIZE]], [[((flexInputBufferSize + 2) * sizeof(yyStateType))]]) + +const bool FLEX_DEBUG = m4_ifdef([[M4_MODE_DEBUG]], [[true]], [[false]]); + +typedef uint8_t YY_CHAR; + +m4_ifdef([[M4_MODE_FULLSPD]], [[m4_dnl +typedef const struct yy_trans_info *yyStateType; +]], [[ +typedef int yyStateType; +]]) + +%% [1.0] DFA + +struct yyTransInfo { + /* We require that yyVerify and yyNxt must be of the same size int. */ +m4_ifdef([[M4_MODE_REAL_FULLSPD]], [[ + YY_OFFSET_TYPE yyVerify; + + /* In cases where its sister yyVerify *is* a "yes, there is + * a transition", yyNxt is the offset (in records) to the + * next state. In most cases where there is no transition, + * the value of yyNxt is irrelevant. If yyNxt is the -1th + * record of a state, though, then yyNxt is the action number + * for that state. + */ + YY_OFFSET_TYPE yyNxt; +]]) +m4_ifdef([[M4_MODE_NO_REAL_FULLSPD]], [[ + /* We generate a bogus 'struct yyTransInfo' data type + * so we can guarantee that it is always declared in the skel. + * This is so we can compile "sizeof(struct yyTransInfo)" + * in any scanner. + */ + int32_t yyVerify; + int32_t yyNxt; +]]) +}; + +%% [2.0] payload macros for the for the DFA data tables are inserted here + +m4_ifdef([[M4_HOOK_NXT_ROWS]],[[m4_dnl +static const M4_HOOK_NXT_TYPE yyNxt[][M4_HOOK_NXT_ROWS] = +M4_HOOK_NXT_BODY +]]) + +m4_ifdef([[M4_MODE_YYLINENO]],[[m4_dnl +/* Table of booleans, true if rule could match eol. */ +static const M4_HOOK_EOLTABLE_TYPE yyRuleCanMatchEOL[M4_HOOK_EOLTABLE_SIZE] = { 0, +M4_HOOK_EOLTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_NEED_ACCEPT]],[[m4_dnl +static const M4_HOOK_ACCEPT_TYPE yyAccept[M4_HOOK_ACCEPT_SIZE] = { 0, +M4_HOOK_ACCEPT_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_MODE_USEECS]],[[m4_dnl +/* Character equivalence-class mapping */ +static const YY_CHAR yyEC[M4_HOOK_ECSTABLE_SIZE] = { 0, +M4_HOOK_ECSTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_MODE_USEMECS]],[[m4_dnl +/* Character meta-equivalence-class mappings */ +static const YY_CHAR yyMeta[M4_HOOK_MECSTABLE_SIZE] = { 0, +M4_HOOK_MECSTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_TRANSTABLE_SIZE]],[[m4_dnl +/* The transition table */ +static const struct yyTransInfo yyTransition[M4_HOOK_TRANSTABLE_SIZE] = { +M4_HOOK_TRANSTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_STARTTABLE_SIZE]],[[m4_dnl +/* Table of pointers to start states. */ +static const struct yyTransInfo *yyStartStateList[M4_HOOK_STARTTABLE_SIZE] = { +M4_HOOK_STARTTABLE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_ACCLIST_TYPE]],[[m4_dnl +static const M4_HOOK_ACCLIST_TYPE yyAcclist[M4_HOOK_ACCLIST_SIZE] = { 0, +M4_HOOK_ACCLIST_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_BASE_TYPE]],[[m4_dnl +static const M4_HOOK_BASE_TYPE yyBase[M4_HOOK_BASE_SIZE] = { 0, +M4_HOOK_BASE_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_DEF_TYPE]],[[m4_dnl +static const M4_HOOK_DEF_TYPE yyDef[M4_HOOK_DEF_SIZE] = { 0, +M4_HOOK_DEF_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_YYNXT_TYPE]],[[m4_dnl +static const M4_HOOK_YYNXT_TYPE yyNxt[M4_HOOK_YYNXT_SIZE] = { 0, +M4_HOOK_YYNXT_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_CHK_TYPE]],[[m4_dnl +static const M4_HOOK_CHK_TYPE yyChk[M4_HOOK_CHK_SIZE] = { 0, +M4_HOOK_CHK_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_HOOK_NULTRANS_SIZE]],[[m4_dnl +static const yyStateType yyNULtrans[M4_HOOK_NULTRANS_SIZE] = { 0, +M4_HOOK_NULTRANS_BODY[[]]m4_dnl +}; +]]) + +m4_ifdef([[M4_MODE_DEBUG]],[[m4_dnl +/* Rule to line-number mapping */ +static const M4_HOOK_DEBUGTABLE_TYPE yy_rule_linenum[M4_HOOK_DEBUGTABLE_SIZE] = { 0, +M4_HOOK_DEBUGTABLE_BODY[[]]m4_dnl +}; +]]) + +%% [3.0] static declarations conditional on mode switches go here + +M4_YY_SC_DEFS + +typedef struct yy_buffer_state *yybuffer; + +%# These are not part of the exported interface and can safely be renamed. +/* These must be #defines, not const variables, because they're used as case + * arm values. GCC will allow case arm expressions to include references to + * const variables, but this is not standard-conformant and other compilers + * may not. + */ +#define EOB_ACT_CONTINUE_SCAN 0 +#define EOB_ACT_END_OF_FILE 1 +#define EOB_ACT_LAST_MATCH 2 +#define YY_BUFFER_NEW 0 +#define YY_BUFFER_NORMAL 1 + /* When an EOF's been seen but there's still some text to process + * then we mark the buffer as YY_EOF_PENDING, to indicate that we + * shouldn't try reading from the input source any more. We might + * still have a bunch of tokens to match, though, because of + * possible backing-up. + * + * When we actually see the EOF, we change the status to "new" + * (via yyrestart()), so that the user can continue scanning by + * just pointing yyin at a new input file. + */ +#define YY_BUFFER_EOF_PENDING 2 + +struct yy_buffer_state { + FILE *yyInputFile; + char *yyChBuf; /* input buffer */ + char *yyBufPos; /* current position in input buffer */ + + /* Size of input buffer in bytes, not including room for EOB + * characters. + */ + int yyInputBufSize; + + /* Number of characters read into yyChBuf, not including EOB + * characters. + */ + int yyNChars; + + /* Whether we "own" the buffer - i.e., we know we created it, + * and can realloc() it to grow it, and should free() it to + * delete it. + */ + bool yyIsOurBuffer; + + /* Whether this is an "interactive" input source; if so, and + * if we're using stdio for input, then we want to use char-by-char + * rather than a buffered read, to make sure we stop fetching input after + * each newline. + */ + bool yyIsInteractive; + + /* Whether we're considered to be at the beginning of a line. + * If so, '^' rules will be active on the next match, otherwise + * not. + */ + bool yyatbolFlag; + + int bs_yylineno; /**< The line count. */ + int bs_yycolumn; /**< The column count. */ + + /* Whether to try to fill the input buffer when we reach the + * end of it. + */ + bool yyFillBuffer; + + int yyBufferStatus; +}; + +/* Watch out: yytext_ptr is a variable when yytext is an array, + * but it's a macro when yytext is a pointer. + */ +m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +m4_define([[yytext_ptr]], [[yytext]]) +]]) + +m4_ifdef([[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[m4_dnl +%# These must match the values in the file flexdef.h +%# of the flex source code, otherwise havoc will ensue. +const int YY_TRAILING_MASK = 0x2000; +const int YY_TRAILING_HEAD_MASK = 0x4000; +]]) +/* Holds the entire state of the reentrant scanner. */ +typedef struct yyguts_t { + /* Public interface */ + FILE *yyin, *yyout; + int yyleng; + int yylineno; + int yyflexdebug; +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[ + char yytext[YYLMAX]; +]], [[ + char *yytext; +]]) + + /* User-defined. Not touched by flex. */ + m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[M4_MODE_EXTRA_TYPE yyextra;]]) + + size_t yyBufferStackTop; /**< index of top of stack. */ + size_t yyBufferStackMax; /**< capacity of stack. */ + yybuffer * yyBufferStack; /**< Stack as an array. */ + char yyHoldChar; + int yyNChars; + char *yyCBufP; + bool yyInit; + int yyStart; + bool yyDidBufferSwitchOnEof; + int yyStartStackOffset; + int yyStartStackDepth; + int *yyStartStack; + yyStateType yyLastAcceptingState; + char* yyLastAcceptingCharPos; + +m4_ifdef([[M4_MODE_USES_REJECT]], [[ + yyStateType *yyStateBuf; + yyStateType *yyStatePtr; + char *yyFullMatch; + int yyLp; +m4_ifdef([[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[m4_dnl + /* These are only needed for trailing context rules */ + int yyLookingForTrailBegin; + int yyFullLp; + int *yyFullState; +]]) +]]) +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[ + char *yytext_ptr; + int yyMoreOffset; + int yyPrevMoreOffset; +]], [[ + bool yyMoreFlag; + int yyMoreLen; +]]) +} FlexLexer; /* end struct yyguts_t */ + +m4_ifdef([[M4_YY_NO_FLEX_ALLOC]],, +[[ +void *yyalloc(size_t size, FlexLexer *yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + return malloc(size); +} +]]) + +m4_ifdef([[M4_YY_NO_FLEX_REALLOC]],, +[[ +void *yyrealloc(void *ptr, size_t size, FlexLexer *yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + return realloc(ptr, size); +} +]]) + +m4_ifdef([[M4_YY_NO_FLEX_FREE]],, +[[ +void yyfree(void * ptr, FlexLexer *yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + /* The cast to (char *) in the following accommodates both + * implementations that use char* generic pointers, and those + * that use void* generic pointers. It works with the latter + * because both ANSI C and C++ allow castless assignment from + * any pointer type to void*, and deal with argument conversions + * as though doing an assignment. + */ + free((char *) ptr); +} +]]) + +m4_ifdef([[M4_MODE_NO_YYWRAP]], [[ +int yywrap(FlexLexer *yyscanner) { + return /*CONSTCOND*/1; +} +]]) + +/* Helpers for special functions, also part of public API */ + +/* Returns the top of the stack, or NULL. */ +yybuffer yy_current_buffer(FlexLexer *yyscanner) { + return (yyscanner->yyBufferStack \ + ? yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] \ + : NULL); +} + +static void yy_load_buffer_state (FlexLexer *yyscanner) +{ + yyscanner->yyNChars = yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyNChars; + yyscanner->yytext_ptr = yyscanner->yyCBufP = yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyBufPos; + yyscanner->yyin = yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputFile; + yyscanner->yyHoldChar = *yyscanner->yyCBufP; +} + +/** Discard all buffered characters. On the next scan, yyread() will be called. + * @param b the buffer state to be flushed, usually @c yy_current_buffer(yyscanner). + * @param yyscanner The scanner object. + */ +void yy_flush_buffer(yybuffer b, FlexLexer *yyscanner) +{ + if (b == NULL) { + return; + } + b->yyNChars = 0; + + /* We always need two end-of-buffer characters. The first causes + * a transition to the end-of-buffer state. The second causes + * a jam in that state. + */ + b->yyChBuf[0] = flexBufferSentinel; + b->yyChBuf[1] = flexBufferSentinel; + + b->yyBufPos = &b->yyChBuf[0]; + + b->yyatbolFlag = true; + b->yyBufferStatus = YY_BUFFER_NEW; + + if (b == yy_current_buffer(yyscanner)) { + yy_load_buffer_state(yyscanner); + } +} + +void yy_flush_current_buffer(FlexLexer *yyscanner) { + yy_flush_buffer(yy_current_buffer(yyscanner), yyscanner); +} + +const int YY_EXIT_FAILURE = 2; + +m4_ifdef([[M4_YY_NO_YYPANIC]],, [[ +/* This function has a magic rewrite rule */ +static void yynoreturn yypanic(const char* msg, FlexLexer *yyscanner) { + (void)yyscanner; /* forestall unused-argument warning */ + fprintf(stderr, "%s\n", msg); + exit(YY_EXIT_FAILURE); +} +]]) + +/* Allocates the stack if it does not exist. + * Guarantees space for at least one push. + */ +static void yyensure_buffer_stack (FlexLexer *yyscanner) +{ + size_t numToAlloc; + + if (yyscanner->yyBufferStack == NULL) { + /* First allocation is just for 2 elements, since we don't know if this + * scanner will even need a stack. We use 2 instead of 1 to avoid an + * immediate realloc on the next call. + */ + numToAlloc = 1; /* After all that talk, this was set to 1 anyways... */ + yyscanner->yyBufferStack = (struct yy_buffer_state**)yyalloc + (numToAlloc * sizeof(struct yy_buffer_state*), yyscanner); + if (yyscanner->yyBufferStack == NULL) { + yypanic("out of dynamic memory in yyensure_buffer_stack()", yyscanner); + } + + memset(yyscanner->yyBufferStack, 0, numToAlloc * sizeof(struct yy_buffer_state*)); + + yyscanner->yyBufferStackMax = numToAlloc; + yyscanner->yyBufferStackTop = 0; + return; + } + + if (yyscanner->yyBufferStackTop >= (yyscanner->yyBufferStackMax) - 1) { + /* Increase the buffer to prepare for a possible push. */ + size_t grow_size = 8 /* arbitrary grow size */; + + numToAlloc = yyscanner->yyBufferStackMax + grow_size; + yyscanner->yyBufferStack = (struct yy_buffer_state**)yyrealloc + (yyscanner->yyBufferStack, + numToAlloc * sizeof(struct yy_buffer_state*), + yyscanner); + if (yyscanner->yyBufferStack == NULL) { + yypanic("out of dynamic memory in yyensure_buffer_stack()", yyscanner); + } + /* zero only the new slots.*/ + memset(yyscanner->yyBufferStack + yyscanner->yyBufferStackMax, 0, grow_size * sizeof(struct yy_buffer_state*)); + yyscanner->yyBufferStackMax = numToAlloc; + } +} + +/* Initializes or reinitializes a buffer. + * This function is sometimes called more than once on the same buffer, + * such as during a yyrestart() or at EOF. + */ +static void yy_init_buffer(yybuffer b, FILE * file, FlexLexer *yyscanner) +{ + int oerrno = errno; + + yy_flush_buffer(b, yyscanner); + + b->yyInputFile = file; + b->yyFillBuffer = true; + + /* If b is the current buffer, then yy_init_buffer was _probably_ + * called from yyrestart() or through yy_get_next_buffer. + * In that case, we don't want to reset the lineno or column. + */ + if (b != yy_current_buffer(yyscanner)) { + b->bs_yylineno = 1; + b->bs_yycolumn = 0; + } + +m4_ifdef([[M4_YY_ALWAYS_INTERACTIVE]], +[[ + b->yyIsInteractive = true; +]], +[[ + m4_ifdef([[M4_YY_NEVER_INTERACTIVE]], + [[ + b->yyIsInteractive = false; + ]], + [[ + b->yyIsInteractive = (file != NULL) && (isatty(fileno(file)) > 0); + ]]) +]]) + errno = oerrno; +} + +/** Allocate and initialize an input buffer state. + * @param file A readable stream. + * @param size The character buffer size in bytes. When in doubt, use @c flexInputBufferSize. + * @param yyscanner The scanner object. + * @return the allocated buffer state. + */ +yybuffer yy_create_buffer(FILE *file, int size, FlexLexer *yyscanner) +{ + yybuffer b; + + b = (yybuffer) yyalloc(sizeof(struct yy_buffer_state), yyscanner); + if ( b == NULL) { + yypanic("out of dynamic memory in yy_create_buffer()", yyscanner); + } + b->yyInputBufSize = size; + + /* yyChBuf has to be 2 characters longer than the size given because + * we need to put in 2 end-of-buffer characters. + */ + b->yyChBuf = (char *) yyalloc((size_t) (b->yyInputBufSize + 2), yyscanner); + if (b->yyChBuf == NULL) { + yypanic("out of dynamic memory in yy_create_buffer()", yyscanner); + } + b->yyIsOurBuffer = true; + + yy_init_buffer(b, file, yyscanner); + + return b; +} + +/** Destroy the buffer. + * @param b a buffer created with yy_create_buffer() + * @param yyscanner The scanner object. + */ +void yy_delete_buffer(yybuffer b, FlexLexer *yyscanner) +{ + + if (b == NULL) { + return; + } + if (b == yy_current_buffer(yyscanner)) { /* Not sure if we should pop here. */ + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = (yybuffer)NULL; + } + if (b->yyIsOurBuffer) { + yyfree((void *) b->yyChBuf, yyscanner); + } + yyfree((void *) b, yyscanner); +} + + +/** Pushes the new state onto the stack. The new state becomes + * the current state. This function will allocate the stack + * if necessary. + * @param new_buffer The new state. + * @param yyscanner The scanner object. + */ +void yypush_buffer_state(yybuffer new_buffer, FlexLexer *yyscanner) +{ + if (new_buffer == NULL) { + return; + } + yyensure_buffer_stack(yyscanner); + + /* This block is copied from yy_switch_to_buffer. */ + if (yy_current_buffer(yyscanner) != NULL) { + /* Flush out information for old buffer. */ + *yyscanner->yyCBufP = yyscanner->yyHoldChar; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyBufPos = yyscanner->yyCBufP; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyNChars = yyscanner->yyNChars; + } + + /* Only push if top exists. Otherwise, replace top. */ + if (yy_current_buffer(yyscanner)) { + yyscanner->yyBufferStackTop++; + } + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = new_buffer; + + /* copied from yy_switch_to_buffer. */ + yy_load_buffer_state(yyscanner); + yyscanner->yyDidBufferSwitchOnEof = true; +} + + +/** Removes and deletes the top of the stack, if present. + * The next element becomes the new top. + * @param yyscanner The scanner object. + */ +void yypop_buffer_state (FlexLexer *yyscanner) +{ + if (yy_current_buffer(yyscanner) == NULL) { + return; + } + yy_delete_buffer(yy_current_buffer(yyscanner), yyscanner); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = NULL; + if (yyscanner->yyBufferStackTop > 0) { + --yyscanner->yyBufferStackTop; + } + if (yy_current_buffer(yyscanner) != NULL) { + yy_load_buffer_state(yyscanner); + yyscanner->yyDidBufferSwitchOnEof = true; + } +} + + +/** Immediately switch to a different input stream. + * @param input_file A readable stream. + * @param yyscanner The scanner object. + * @note This function does not reset the start condition to @c INITIAL . + */ +void yyrestart(FILE * input_file, FlexLexer *yyscanner) +{ + + if (yy_current_buffer(yyscanner) == NULL) { + yyensure_buffer_stack (yyscanner); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = + yy_create_buffer(yyscanner->yyin, flexInputBufferSize, yyscanner); + } + + yy_init_buffer(yy_current_buffer(yyscanner), input_file, yyscanner); + yy_load_buffer_state(yyscanner); +} + +static void yybumpline(FlexLexer *yyscanner) { + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->bs_yylineno++; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->bs_yycolumn=0; +} + +/* START special functions + * + * Flex's scanner knows these are special and inserts the yyscanner argument + * at the end of the argument list. TODO: support OO languages with a + * property that causes it to be prepended with a dot instead. + */ + +/* Enter a start condition. */ +void yybegin(int s, FlexLexer *yyscanner) { + yyscanner->yyStart = 1 + 2 * (s); +} + +/* Translate the current start state into a value that can be later handed + * to yybegin() to return to the state. + */ +int yystart(FlexLexer *yyscanner) { + return ((yyscanner->yyStart - 1) / 2); +} + +/* This used to be an fputs(), but since the string might contain NULs, + * we now use fwrite(). + */ +void yyecho(FlexLexer *yyscanner) { + fwrite(yyscanner->yytext, (size_t) yyscanner->yyleng, 1, yyscanner->yyout); +} + +m4_ifdef([[M4_YY_NO_YYUNPUT]],, [[ +void yyunput(char c, FlexLexer *yyscanner) +{ + char *yyCp; + + yyCp = yyscanner->yyCBufP; + + /* undo effects of setting up yytext */ + *yyCp = yyscanner->yyHoldChar; + + if (yyCp < yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf + 2) { + /* need to shift things up to make room */ + /* +2 for EOB chars. */ + int numberToMove = yyscanner->yyNChars + 2; + char *dest = &yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[ + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputBufSize + 2]; + char *source = &yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[numberToMove]; + + while (source > yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf) { + *--dest = *--source; + } + yyCp += (int) (dest - source); + yyscanner->yytext_ptr += (int) (dest - source); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyNChars = + yyscanner->yyNChars = (int) yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputBufSize; + + if (yyCp < yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf + 2) { + yypanic("flex scanner push-back overflow", yyscanner); + } + } + + *--yyCp = c; + +m4_ifdef([[M4_MODE_YYLINENO]], +[[ + if (c == '\n') { + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->bs_yylineno--; + } +]]) + + yyscanner->yyHoldChar = *yyCp; + yyscanner->yyCBufP = yyCp; +} +]]) + +m4_ifdef([[M4_MODE_USER_YYREAD]],, [[ +/* Gets input and stuffs it into "buf". Number of characters read, or flexEOF, + * is returned in "result". + */ +static int yyread(char *buf, size_t maxSize, FlexLexer *yyscanner) { + int result; +m4_ifdef([[M4_MODE_CPP_USE_READ]], [[ + errno=0; + while ((result = (int) read(fileno(yyscanner->yyin), buf, (size_t) maxSize)) < 0) { + if(errno != EINTR) { + yypanic("input in flex scanner failed", yyscanner); + break; + } + errno=0; + clearerr(yyscanner->yyin); + } +]]) +m4_ifdef([[M4_MODE_NO_CPP_USE_READ]], [[ + if (yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyIsInteractive) { + int c = '*'; + int n; + for (n = 0; n < maxSize && + (c = getc(yyscanner->yyin)) != EOF && c != '\n'; ++n) { + buf[n] = (char) c; + } + if (c == '\n') { + buf[n++] = (char) c; + } + if (c == EOF && ferror(yyscanner->yyin)) { + yypanic("input in flex scanner failed", yyscanner); + } + result = n; + } else { + errno=0; + while ((result = (int) fread(buf, 1, (size_t) maxSize, yyscanner->yyin)) == 0 && ferror(yyscanner->yyin)) { + if(errno != EINTR) { + yypanic("input in flex scanner failed", yyscanner); + break; + } + errno=0; + clearerr(yyscanner->yyin); + } + } +]]) + return result; +} +]]) + +%# yymore has a magic rewute rule. It's declared here, rather than with the other +%# magic functions, so yy_get_next_buffer() won't need a forward declaration. +m4_ifdef([[M4_MODE_YYMORE_USED]], [[ +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[ +void yymore(FlexLexer *yyscanner) {yyscanner->yyMoreOffset = strlen(yyscanner->yytext);} +m4_define([[YY_MORE_ADJ]], [[0]]) +m4_define([[YY_RESTORE_YY_MORE_OFFSET]], [[{ +yyscanner->yyMoreOffset = yyscanner->yyPrevMoreOffset; +yyscanner->yyleng -= yyscanner->yyMoreOffset; +} +]]) +]]) +m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ +void yymore(FlexLexer *yyscanner) {yyscanner->yyMoreFlag = true;} +m4_define([[YY_MORE_ADJ]], [[yyscanner->yyMoreLen]]) +m4_define([[YY_RESTORE_YY_MORE_OFFSET]], [[]]) +]]) +]]) + +m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[ +m4_define([[YY_MORE_ADJ]], [[0]]) +m4_define([[YY_RESTORE_YY_MORE_OFFSET]], [[]]) +]]) + +/* yy_get_next_buffer - try to read in a new buffer + * + * Returns a code representing an action: + * EOB_ACT_LAST_MATCH - + * EOB_ACT_CONTINUE_SCAN - continue scanning from current position + * EOB_ACT_END_OF_FILE - end of file + */ +static int yy_get_next_buffer (FlexLexer *yyscanner) +{ + char *dest = yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf; + char *source = yyscanner->yytext_ptr; + int numberToMove, i; + int retVal; + + if (yyscanner->yyCBufP > &yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[yyscanner->yyNChars + 1]) { + yypanic("fatal flex scanner internal error--end of buffer missed", yyscanner); + } + if (!yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyFillBuffer) { + /* Don't try to fill the buffer, so this is an EOF. */ + if (yyscanner->yyCBufP - yyscanner->yytext_ptr - YY_MORE_ADJ == 1) { + /* We matched a single character, the EOB, so + * treat this as a final EOF. + */ + return EOB_ACT_END_OF_FILE; + } else { + /* We matched some text prior to the EOB, first + * process it. + */ + return EOB_ACT_LAST_MATCH; + } + } + + /* Try to read more data. */ + + /* First move last chars to start of buffer. */ + numberToMove = (int) (yyscanner->yyCBufP - yyscanner->yytext_ptr - 1); + + for (i = 0; i < numberToMove; ++i) { + *(dest++) = *(source++); + } + if (yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyBufferStatus == YY_BUFFER_EOF_PENDING) { + /* don't do the read, it's not guaranteed to return an EOF, + * just force an EOF + */ + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyNChars = yyscanner->yyNChars = 0; + } else { + int numToRead = + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputBufSize - numberToMove - 1; + + while (numToRead <= 0) { /* Not enough room in the buffer - grow it. */ +m4_ifdef([[M4_MODE_USES_REJECT]], +[[ + yypanic( + "input buffer overflow, can't enlarge buffer because scanner uses reject", yyscanner); +]], +[[ + /* just a shorter name for the current buffer */ + yybuffer b = yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]; + + int yyCBufP_offset = + (int) (yyscanner->yyCBufP - b->yyChBuf); + + if (b->yyIsOurBuffer) { + int newSize = b->yyInputBufSize * 2; + + if (newSize <= 0) { + b->yyInputBufSize += b->yyInputBufSize / 8; + } else { + b->yyInputBufSize *= 2; + } + b->yyChBuf = (char *) + /* Include room in for 2 EOB chars. */ + yyrealloc((void *) b->yyChBuf, + (size_t) (b->yyInputBufSize + 2), yyscanner); + } else { + /* Can't grow it, we don't own it. */ + b->yyChBuf = NULL; + } + if (b->yyChBuf == NULL) { + yypanic("fatal error - scanner input buffer overflow", yyscanner); + } + yyscanner->yyCBufP = &b->yyChBuf[yyCBufP_offset]; + + numToRead = yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputBufSize - + numberToMove - 1; +]]) + } + + if (numToRead > flexReadBufferSize) { + numToRead = flexReadBufferSize; + } + /* Read in more data. */ + yyscanner->yyNChars = yyread(&yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[numberToMove], numToRead, yyscanner); + + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyNChars = yyscanner->yyNChars; + } + + if (yyscanner->yyNChars == 0) { + if (numberToMove == YY_MORE_ADJ) { + retVal = EOB_ACT_END_OF_FILE; + yyrestart(yyscanner->yyin, yyscanner); + } else { + retVal = EOB_ACT_LAST_MATCH; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyBufferStatus = + YY_BUFFER_EOF_PENDING; + } + } else { + retVal = EOB_ACT_CONTINUE_SCAN; + } + if ((yyscanner->yyNChars + numberToMove) > yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputBufSize) { + /* Extend the array by 50%, plus the number we really need. */ + int newSize = yyscanner->yyNChars + numberToMove + (yyscanner->yyNChars >> 1); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf = (char *) yyrealloc( + (void *) yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf, (size_t) newSize, yyscanner); + if (yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf == NULL) { + yypanic("out of dynamic memory in yy_get_next_buffer()", yyscanner); + } + /* "- 2" to take care of EOB's */ + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputBufSize = (int) (newSize - 2); + } + + yyscanner->yyNChars += numberToMove; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[yyscanner->yyNChars] = flexBufferSentinel; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[yyscanner->yyNChars + 1] = flexBufferSentinel; + + yyscanner->yytext_ptr = &yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[0]; + + return retVal; +} + +int yyinput(FlexLexer *yyscanner) +{ + int c; + + *yyscanner->yyCBufP = yyscanner->yyHoldChar; + + if (*yyscanner->yyCBufP == flexBufferSentinel) { + /* yyCBufP now points to the character we want to return. + * If this occurs *before* the EOB characters, then it's a + * valid NUL; if not, then we've hit the end of the buffer. + */ + if (yyscanner->yyCBufP < &yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[yyscanner->yyNChars]) { + /* This was really a NUL. */ + *yyscanner->yyCBufP = '\0'; + } else { + /* need more input */ + int offset = (int) (yyscanner->yyCBufP - yyscanner->yytext_ptr); + ++yyscanner->yyCBufP; + + switch (yy_get_next_buffer(yyscanner)) { + case EOB_ACT_LAST_MATCH: + /* This happens because yy_g_n_b() + * sees that we've accumulated a + * token and flags that we need to + * try matching the token before + * proceeding. But for input(), + * there's no matching to consider. + * So convert the EOB_ACT_LAST_MATCH + * to EOB_ACT_END_OF_FILE. + */ + + /* Reset buffer status. */ + yyrestart(yyscanner->yyin, yyscanner); + + /*FALLTHROUGH*/ + + case EOB_ACT_END_OF_FILE: + if (yywrap(yyscanner)) { + return 0; + } + if (! yyscanner->yyDidBufferSwitchOnEof) { + yyrestart(yyscanner->yyin, yyscanner); + } + return yyinput(yyscanner); + + case EOB_ACT_CONTINUE_SCAN: + yyscanner->yyCBufP = yyscanner->yytext_ptr + offset; + break; + } + } + } + + c = *(unsigned char *) yyscanner->yyCBufP; /* cast for 8-bit char's */ + *yyscanner->yyCBufP = '\0'; /* preserve yytext */ + yyscanner->yyHoldChar = *++yyscanner->yyCBufP; + +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyatbolFlag = (c == '\n'); +m4_ifdef([[M4_MODE_YYLINENO]], [[ + if (yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyatbolFlag) { + yybumpline(yyscanner); + } +]]) +]]) +m4_ifdef([[M4_MODE_NO_BOL_NEEDED]], [[ +m4_ifdef([[M4_MODE_YYLINENO]], [[ + if (c == '\n') { + yybumpline(yyscanner); + } + ]]) +]]) + + return c; +} + +/* ENDS special functions + +/* Macros after this point can all be overridden by user definitions in + * section 1. + */ + +m4_ifdef([[M4_MODE_NO_YYWRAP]],, [[ +extern int yywrap (FlexLexer *yyscanner); +]]) + +m4_ifdef([[M4_YY_STACK_USED]], +[[ + m4_ifdef([[M4_YY_NO_PUSH_STATE]],, + [[ + static void yy_push_state (int newState, FlexLexer *yyscanner); + ]]) + m4_ifdef([[M4_YY_NO_POP_STATE]],, + [[ + static void yy_pop_state (FlexLexer *yyscanner); + ]]) + m4_ifdef([[M4_YY_NO_TOP_STATE]],, + [[ + static int yy_top_state (FlexLexer *yyscanner); + ]]) +]], +[[ +m4_define([[M4_YY_NO_PUSH_STATE]]) +m4_define([[M4_YY_NO_POP_STATE]]) +m4_define([[M4_YY_NO_TOP_STATE]]) +]]) + +/* STARTS Accessor methods to globals. + These are made visible to non-reentrant scanners for convenience. */ + +m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[m4_dnl +m4_ifdef([[M4_YY_NO_GET_EXTRA]],, [[m4_dnl +/** Get the user-defined data for this scanner. + * @param yyscanner The scanner object. + */ +M4_MODE_EXTRA_TYPE yyget_extra(FlexLexer *yyscanner) { + return yyscanner->yyextra; +} +]]) +]]) + +m4_ifdef([[M4_YY_NO_GET_LINENO]],, +[[ +/** Get the current line number. + * @param yyscanner The scanner object. + */ +int yyget_lineno(FlexLexer *yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + if (cb == NULL) { + return 0; + } + return cb->bs_yylineno; +} +]]) + +m4_ifdef([[M4_YY_NO_GET_COLUMN]],, +[[ +/** Get the current column number. + * @param yyscanner The scanner object. + */ +int yyget_column (FlexLexer *yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + if (cb == NULL) { + return 0; + } + return cb->bs_yycolumn; +} +]]) + +m4_ifdef([[M4_YY_NO_GET_IN]],, +[[ +/** Get the input stream. + * @param yyscanner The scanner object. + */ +FILE *yyget_in (FlexLexer *yyscanner) { + return yyscanner->yyin; +} +]]) + +m4_ifdef([[M4_YY_NO_GET_OUT]],, +[[ +/** Get the output stream. + * @param yyscanner The scanner object. + */ +FILE *yyget_out (FlexLexer *yyscanner) { + return yyscanner->yyout; +} +]]) + +m4_ifdef([[M4_YY_NO_GET_LENG]],, +[[ +/** Get the length of the current token. + * @param yyscanner The scanner object. + */ +int yyget_leng (FlexLexer *yyscanner) { + return yyscanner->yyleng; +} +]]) + +/** Get the current token. + * @param yyscanner The scanner object. + */ +m4_ifdef([[M4_YY_NO_GET_TEXT]],, +[[ +char *yyget_text (FlexLexer *yyscanner) { + return yyscanner->yytext; +} +]]) + +m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[m4_dnl +m4_ifdef([[M4_YY_NO_SET_EXTRA]],, +[[ +/** Set the user-defined data. This data is never touched by the scanner. + * @param userDefined The data to be associated with this scanner. + * @param yyscanner The scanner object. + */ +void yyset_extra(M4_MODE_EXTRA_TYPE userDefined, FlexLexer *yyscanner) { + yyscanner->yyextra = userDefined; +} +]]) +]]) + +m4_ifdef([[M4_YY_NO_SET_LINENO]],, +[[ +/** Set the current line number. + * @param lineNumber line number + * @param yyscanner The scanner object. + */ +void yyset_lineno(int lineNumber, FlexLexer *yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + /* lineno is only valid if an input buffer exists. */ + if (cb == NULL) { + yypanic("yyset_lineno called with no buffer", yyscanner); + } + cb->bs_yylineno = lineNumber; +} +]]) + +m4_ifdef([[M4_YY_NO_SET_COLUMN]],, +[[ +/** Set the current column. + * @param columnNo column number + * @param yyscanner The scanner object. + */ +void yyset_column(int columnNo, FlexLexer *yyscanner) { + yybuffer cb = yy_current_buffer(yyscanner); + + /* column is only valid if an input buffer exists. */ + if (cb == NULL) { + yypanic("yyset_column called with no buffer", yyscanner); + } + cb->bs_yycolumn = columnNo; +} +]]) + +m4_ifdef([[M4_YY_NO_SET_IN]],, +[[ +/** Set the input stream. This does not discard the current + * input buffer. + * @param inStream A readable stream. + * @param yyscanner The scanner object. + * @see yy_switch_to_buffer + */ +void yyset_in(FILE *inStream, FlexLexer *yyscanner) { + yyscanner->yyin = inStream ; +} +]]) + +m4_ifdef([[M4_YY_NO_SET_OUT]],, +[[ +void yyset_out(FILE *outStream, FlexLexer *yyscanner) { + yyscanner->yyout = outStream ; +} +]]) + + +m4_ifdef([[M4_YY_NO_GET_DEBUG]],, +[[ +int yyget_debug(FlexLexer *yyscanner) { + return yyscanner->yyflexdebug; +} +]]) + +m4_ifdef([[M4_YY_NO_SET_DEBUG]],, +[[ +void yyset_debug(int bdebug, FlexLexer *yyscanner) { + yyscanner->yyflexdebug = bdebug ; +} +]]) + +/* ENDS public accessor functions */ + +/* Number of entries by which start-condition stack grows. */ +const int YY_START_STACK_INCR = 25; + +/* Default declaration of generated scanner - a define so the user can + * easily add parameters. + */ +m4_ifdef([[YY_DECL]],, [[m4_dnl +m4_define([[M4_YY_LEX_PROTO]], [[(FlexLexer *yyscanner)]]) +m4_define([[M4_YY_LEX_DECLARATION]], [[(FlexLexer *yyscanner)]]) +m4_define([[YY_DECL]], [[int yylex M4_YY_LEX_DECLARATION]]) +]]) + +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[ +static void rule_check_bol(FlexLexer *yyscanner) { + if (yyscanner->yyleng > 0) { \ + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyatbolFlag = (yyscanner->yytext[yyscanner->yyleng - 1] == '\n'); + } +} +]]) + +/* Done after the current pattern has been matched and before the + * corresponding action - sets up yytext. yyCp and yyBp are the + * end abd start pointers for the input buffer segment that is + * claimed as yytext. + */ +void yyDoBeforeAction(FlexLexer *yyscanner, char *yyCp, char *yyBp) { + yyscanner->yytext_ptr = yyBp; \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[yyscanner->yytext_ptr -= yyscanner->yyMoreLen; \ + yyscanner->yyleng = (int) (yyCp - yyscanner->yytext_ptr);]])]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[yyscanner->yyleng = (int) (yyCp - yyBp);]]) \ + m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[yyscanner->yyleng = (int) (yyCp - yyBp);]]) \ + yyscanner->yyHoldChar = *yyCp; \ + *yyCp = '\0'; \ +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], [[ \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[if (yyscanner->yyleng + yyscanner->yyMoreOffset >= YYLMAX) \ + yypanic("token too large, exceeds YYLMAX", yyscanner);]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[if (yyscanner->yyleng >= YYLMAX) \ + yypanic("token too large, exceeds YYLMAX", yyscanner);]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[strncpy(&yyscanner->yytext[yyscanner->yyMoreOffset], yyscanner->yytext_ptr, yyscanner->yyleng + 1);]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yyscanner->yyleng += yyscanner->yyMoreOffset;]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yyscanner->yyPrevMoreOffset = yyscanner->yyMoreOffset;]]) \ + m4_ifdef([[M4_MODE_YYMORE_USED]], [[yyscanner->yyMoreOffset = 0;]]) \ + m4_ifdef([[M4_MODE_NO_YYMORE_USED]], [[strncpy(yyscanner->yytext, yyscanner->yytext_ptr, yyscanner->yyleng + 1);]]) \ +]]) \ + yyscanner->yyCBufP = yyCp; +} + +m4_ifdef([[M4_MODE_YYLINENO]], [[ +/* FIXME: gate on yyRuleCanMatchEOL, this is no longer a macro + * and we can get at yyAct */ +static void yyLessLineno(int n, FlexLexer *yyscanner) { + int yyl; + for (yyl = n; yyl < yyscanner->yyleng; ++yyl) { + if (yyscanner->yytext[yyl] == '\n') { + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->bs_yylineno--; + } + } +} + +static void yyLinenoRewindTo(char *yyCp, char *dst, FlexLexer *yyscanner) { + const char *p; + for (p = yyCp-1; p >= dst; --p) { + if (*p == '\n') { + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->bs_yylineno--; + } + } +} +]]) + +void yy_set_interactive(bool is_interactive, FlexLexer *yyscanner) { + if (yy_current_buffer(yyscanner) == NULL) { + yyensure_buffer_stack (yyscanner); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = + yy_create_buffer(yyscanner->yyin, flexInputBufferSize, yyscanner); + } + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyIsInteractive = is_interactive; +} + + +void yysetbol(bool atBOL, FlexLexer *yyscanner) { + if (yy_current_buffer(yyscanner) == NULL) { + yyensure_buffer_stack (yyscanner); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = + yy_create_buffer(yyscanner->yyin, flexInputBufferSize, yyscanner); + } + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyatbolFlag = atBOL; +} + +bool yyatbol(FlexLexer *yyscanner) { + return (yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyatbolFlag); +} + +/** Switch to a different input buffer. + * @param newBuffer The new input buffer. + * @param yyscanner The scanner object. + */ +void yy_switch_to_buffer(yybuffer newBuffer, FlexLexer *yyscanner) +{ + + /* TODO. We should be able to replace this entire function body + * with + * yypop_buffer_state(); + * yypush_buffer_state(newBuffer); + */ + yyensure_buffer_stack (yyscanner); + if (yy_current_buffer(yyscanner) == newBuffer) { + return; + } + if (yy_current_buffer(yyscanner)) { + /* Flush out information for old buffer. */ + *yyscanner->yyCBufP = yyscanner->yyHoldChar; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyBufPos = yyscanner->yyCBufP; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyNChars = yyscanner->yyNChars; + } + + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = newBuffer; + yy_load_buffer_state(yyscanner); + + /* We don't actually know whether we did this switch during + * EOF (yywrap()) processing, but the only time this flag + * is looked at is after yywrap() is called, so it's safe + * to go ahead and always set it. + */ + yyscanner->yyDidBufferSwitchOnEof = true; +} + + +m4_ifdef([[M4_YY_NO_SCAN_BUFFER]],, +[[ +/** Setup the input buffer state to scan directly from a user-specified character buffer. + * @param base the character buffer + * @param size the size in bytes of the character buffer + * @param yyscanner The scanner object. + * @return the newly allocated buffer state object. + */ +yybuffer yy_scan_buffer(char *base, size_t size, FlexLexer *yyscanner) +{ + yybuffer b; + + if (size < 2 || + base[size-2] != flexBufferSentinel || + base[size-1] != flexBufferSentinel) { + /* They forgot to leave room for the EOB's. */ + return NULL; + } + b = (yybuffer) yyalloc(sizeof(struct yy_buffer_state), yyscanner); + if (b == NULL) { + yypanic("out of dynamic memory in yy_scan_buffer()", yyscanner); + } + b->yyInputBufSize = (int) (size - 2); /* "- 2" to take care of EOB's */ + b->yyBufPos = b->yyChBuf = base; + b->yyIsOurBuffer = false; + b->yyInputFile = NULL; + b->yyNChars = b->yyInputBufSize; + b->yyIsInteractive = false; + b->yyatbolFlag = true; + b->yyFillBuffer = false; + b->yyBufferStatus = YY_BUFFER_NEW; + + yy_switch_to_buffer(b, yyscanner); + + return b; +} +]]) + +m4_ifdef([[M4_YY_NO_SCAN_BYTES]],, +[[ +/** Setup the input buffer state to scan the given bytes. The next call to yylex() will + * scan from a @e copy of @a bytes. + * @param yybytes the byte buffer to scan + * @param _yybytes_len the number of bytes in the buffer pointed to by @a bytes. + * @param yyscanner The scanner object. + * @return the newly allocated buffer state object. + */ +yybuffer yy_scan_bytes(const char * yybytes, int _yybytes_len, FlexLexer *yyscanner) { + yybuffer b; + char *buf; + size_t n; + int i; + + /* Get memory for full buffer, including space for trailing EOB's. */ + n = (size_t) (_yybytes_len + 2); + buf = (char *) yyalloc(n, yyscanner); + if (buf == 0) { + yypanic("out of dynamic memory in yy_scan_bytes()", yyscanner); + } + for (i = 0; i < _yybytes_len; ++i) { + buf[i] = yybytes[i]; + } + buf[_yybytes_len] = buf[_yybytes_len+1] = flexBufferSentinel; + + b = yy_scan_buffer(buf, n, yyscanner); + if (b == NULL) { + yypanic("bad buffer in yy_scan_bytes()", yyscanner); + } + /* It's okay to grow etc. this buffer, and we should throw it + * away when we're done. + */ + b->yyIsOurBuffer = true; + + return b; +} +]]) + +m4_ifdef([[M4_YY_NO_SCAN_STRING]],, +[[ +/** Setup the input buffer state to scan a string. The next call to yylex() will + * scan from a @e copy of @a str. + * @param yystr a NUL-terminated string to scan + * @param yyscanner The scanner object. + * @return the newly allocated buffer state object. + * @note If you want to scan bytes that may contain NUL values, then use + * yy_scan_bytes() instead. + */ +yybuffer yy_scan_string(const char * yystr, FlexLexer *yyscanner) +{ + return yy_scan_bytes(yystr, (int) strlen(yystr), yyscanner); +} +]]) + +m4_ifdef([[M4_YY_NO_PUSH_STATE]],, +[[ +static void yy_push_state(int newState, FlexLexer *yyscanner) +{ + if (yyscanner->yyStartStackOffset >= yyscanner->yyStartStackDepth) { + size_t newSize; + + yyscanner->yyStartStackDepth += YY_START_STACK_INCR; + newSize = (size_t) yyscanner->yyStartStackDepth * sizeof(int); + + if (yyscanner->yyStartStack == NULL) { + yyscanner->yyStartStack = (int *) yyalloc(newSize, yyscanner); + + } else { + yyscanner->yyStartStack = (int *) yyrealloc( + (void *) yyscanner->yyStartStack, newSize, yyscanner); + } + if (yyscanner->yyStartStack == NULL) { + yypanic("out of memory expanding start-condition stack", yyscanner); + } + } + yyscanner->yyStartStack[yyscanner->yyStartStackOffset++] = yystart(yyscanner); + + yybegin(newState, yyscanner); +} +]]) + + +m4_ifdef([[M4_YY_NO_POP_STATE]],, +[[ +static void yy_pop_state(FlexLexer *yyscanner) +{ + if (--yyscanner->yyStartStackOffset < 0) { + yypanic("start-condition stack underflow", yyscanner); + } + yybegin(yyscanner->yyStartStack[yyscanner->yyStartStackOffset], yyscanner); +} +]]) + + +m4_ifdef([[M4_YY_NO_TOP_STATE]],, +[[ +static int yy_top_state(FlexLexer *yyscanner) +{ + return yyscanner->yyStartStackOffset > 0 ? yyscanner->yyStartStack[yyscanner->yyStartStackOffset - 1] : yystart(yyscanner); +} +]]) + +%# Code snippets used in various cases of code generation in the main scanner. + +m4_define([[M4_GEN_BACKING_UP]], [[ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_HAS_BACKING_UP]], [[ + /* Generate code to keep backing-up information. */ +m4_ifdef([[M4_MODE_FULLSPD]], [[ + if (yyCurrentState[-1].yyNxt) { +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD]], [[ + if (yyAccept[yyCurrentState]) { +]]) + yyscanner->yyLastAcceptingState = yyCurrentState; + yyscanner->yyLastAcceptingCharPos = yyCp; + } +]]) +]]) +]]) + +%# yyChar was formerly YY_CHAR, changed to int because table can now +%# have up to 0x101 entries, since we no longer generate a separate +%# NUL table. +%# +%# Note: on x86-64 architecture with gcc -O2, we save an instruction +%# in the main loop, since the character can now be zero-extended in +%# the process of retrieving it from the input stream or the yyEC[] +%# or yyMeta[] arrays, whereas previously it was zero-extended by a +%# register-to-register move just prior to the yyChk[] table lookup +m4_define([[M4_GEN_NEXT_COMPRESSED_STATE]], [[ + int yyChar = $1; + /* Save the backing-up info \before/ computing the next state + * because we always compute one more state than needed - we + * always proceed until we reach a jam state + */ + M4_GEN_BACKING_UP + + while (yyChk[yyBase[yyCurrentState] + yyChar] != yyCurrentState) { + yyCurrentState = (int) yyDef[yyCurrentState]; + +m4_ifdef([[M4_MODE_USEMECS]], [[ + /* We've arranged it so that templates are never chained + * to one another. This means we can afford to make a + * very simple test to see if we need to convert to + * yyChar's meta-equivalence class without worrying + * about erroneously looking up the meta-equivalence + * class twice + */ + + /* lastdfa + 2 == YY_JAMSTATE + 1 is the beginning of the templates */ + if (yyCurrentState >= YY_JAMSTATE + 1) { + yyChar = yyMeta[yyChar]; + } +]]) + } + yyCurrentState = yyNxt[yyBase[yyCurrentState] + yyChar]; +]]) + +m4_define([[M4_GEN_START_STATE]], [[ + /* Generate the code to find the start state. */ +m4_ifdef([[M4_MODE_FULLSPD]], [[ +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[yyCurrentState = yyStartStateList[yyscanner->yyStart + yyatbol()];]]) +m4_ifdef([[M4_MODE_NO_BOL_NEEDED]], [[yyCurrentState = yyStartStateList[yyscanner->yyStart];]]) +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD]], [[ + yyCurrentState = yyscanner->yyStart; +m4_ifdef([[M4_MODE_BOL_NEEDED]], [[yyCurrentState += yyatbol(yyscanner);]]) + /* Set up for storing up states. */ + m4_ifdef([[M4_MODE_USES_REJECT]], [[ + yyscanner->yyStatePtr = yyscanner->yyStateBuf; + *yyscanner->yyStatePtr++ = yyCurrentState; +]]) +]]) +]]) + +m4_define([[M4_GEN_NEXT_MATCH_FULLSPD]], [[ + { + const struct yyTransInfo *yyTransInfo; + YY_CHAR yyChar; + + for (yyChar = $1; + (yyTransInfo = &yyCurrentState[yyChar])->yyVerify == yyChar; + yyChar = $2) + { + yyCurrentState += yyTransInfo->yyNxt; + + M4_GEN_BACKING_UP + } + } +]]) + +/* + * Helpers for yylex() + */ + +%# Conditional indirection through an equivalence map +m4_ifdef([[M4_MODE_USEECS]], m4_define([[M4_EC]], [[*(yyEC+$1)]])) +m4_ifdef([[M4_MODE_NO_USEECS]], [[m4_define([[M4_EC]], [[$1]])]]) + +/* yyGetPreviousState - get the state just before the EOB char was reached */ + +static yyStateType yyGetPreviousState(FlexLexer *yyscanner) { + yyStateType yyCurrentState; + char *yyCp; + + M4_GEN_START_STATE + for (yyCp = yyscanner->yytext_ptr + YY_MORE_ADJ; yyCp < yyscanner->yyCBufP; ++yyCp) { + /* Generate the code to find the next state. */ + m4_ifdef([[M4_MODE_NO_NULTRANS]], [[m4_define([[CHAR_MAP_3]], [[(*yyCp ? M4_EC(YY_SC_TO_UI(*yyCp)) : YY_NUL_EC)]])]]) + m4_ifdef([[M4_MODE_NULTRANS]], [[m4_define([[CHAR_MAP_3]], [[M4_EC(YY_SC_TO_UI(*yyCp))]])]]) + + m4_ifdef([[M4_MODE_NULTRANS]], [[ + /* Compressed tables back up *before* they match. */ + m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[M4_GEN_BACKING_UP]]) + if (*yyCp) { + ]]) + + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[ + m4_ifdef([[M4_MODE_GENTABLES]], [[yyCurrentState = yyNxt[yyCurrentState][CHAR_MAP_3];]]) + m4_ifdef([[M4_MODE_NO_GENTABLES]], [[yyCurrentState = yyNxt[yyCurrentState*YYNXT_LOLEN + CHAR_MAP_3];]]) + ]]) + + m4_ifdef([[M4_MODE_FULLSPD]], [[yyCurrentState += yyCurrentState[CHAR_MAP_3].yyNxt;]]) + m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[M4_GEN_NEXT_COMPRESSED_STATE(CHAR_MAP_3)]]) + +m4_ifdef([[M4_MODE_NULTRANS]], [[ + } else { + yyCurrentState = yyNULtrans[yyCurrentState]; + } +]]) + + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[M4_GEN_BACKING_UP]]) + m4_ifdef([[M4_MODE_FULLSPD]], [[M4_GEN_BACKING_UP]]) + m4_ifdef([[M4_MODE_USES_REJECT]], [[*yyscanner->yyStatePtr++ = yyCurrentState;]]) + } + + return yyCurrentState; +} + +/* yyTryNULtrans - try to make a transition on the NUL character + * + * synopsis + * next_state = yyTryNULtrans(current_state); + */ +static yyStateType yyTryNULtrans(yyStateType yyCurrentState, FlexLexer *yyscanner) +{ + bool yyIsJam; + /* Generate code for handling NULs, if needed. */ + + /* First, deal with backing up and setting up yyCp if the scanner + * finds that it should JAM on the NUL. + * + * Only generate a definition for "yyCp" if we'll generate code + * that uses it. Otherwise validators complain. + */ + m4_ifdef([[M4_MODE_NEED_YY_CP]], [[char *yyCp = yyscanner->yyCBufP;]]) + +%# Note that this statement block and the following three are +%# not executed serially but are an if-then-else cascade +%# for different table modes. +m4_ifdef([[M4_MODE_NULTRANS]], [[ + yyCurrentState = yyNULtrans[yyCurrentState]; + yyIsJam = (yyCurrentState == 0); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS]], [[ +m4_ifdef([[M4_MODE_NULTRANS_FULLTBL]], [[ +m4_ifdef([[M4_MODE_GENTABLES]], [[yyCurrentState = yyNxt[yyCurrentState][YY_NUL_EC];]]) +m4_ifdef([[M4_MODE_NO_GENTABLES]], [[yyCurrentState = yyNxt[yyCurrentState*YYNXT_LOLEN + YY_NUL_EC];]]) + yyIsJam = (yyCurrentState <= 0); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS_FULLTBL]], [[ +m4_ifdef([[M4_MODE_NULTRANS_FULLSPD]], [[ + int yyChar = YY_NUL_EC; + + const struct yyTransInfo *yyTransInfo; + + yyTransInfo = &yyCurrentState[(unsigned int) yyChar]; + yyCurrentState += yyTransInfo->yyNxt; + yyIsJam = (yyTransInfo->yyVerify != yyChar); +]]) + +m4_ifdef([[M4_MODE_NO_NULTRANS_FULLSPD]], [[ +M4_GEN_NEXT_COMPRESSED_STATE(YY_NUL_EC) +yyIsJam = (yyCurrentState == YY_JAMSTATE); +m4_ifdef([[M4_MODE_USES_REJECT]], [[ + /* Only stack this state if it's a transition we + * actually make. If we stack it on a jam, then + * the state stack and yyCBufP get out of sync. + */ + if (! yyIsJam) { + *yyscanner->yyStatePtr++ = yyCurrentState; + } + ]]) +]]) +]]) +]]) +%# End of if-else cascade + +m4_ifdef([[M4_MODE_NULTRANS_WRAP]], [[ + /* If we've entered an accepting state, back up; note that + * compressed tables have *already* done such backing up, so + * we needn't bother with it again. + */ + if (!yyIsJam) { + M4_GEN_BACKING_UP + } +]]) + + (void)yyscanner; /* forestall unused-argument warning */ + return yyIsJam ? 0 : yyCurrentState; +} + +/** The main scanner function which does all the work. + */ +YY_DECL { + yyStateType yyCurrentState; + char *yyCp, *yyBp; + int yyAct; + +m4_ifdef([[M4_YY_BISON_LVAL]], +[[ + yylval = yylval_param; +]]) + +m4_ifdef([[<M4_YY_BISON_LLOC>]], +[[ + yylloc = yylloc_param; +]]) + + if (!yyscanner->yyInit) { + yyscanner->yyInit = true; + + m4_ifdef([[YY_USER_INIT]], [[YY_USER_INIT]]) + +m4_ifdef([[M4_MODE_USES_REJECT]], +[[ + /* Create the reject buffer large enough to save one state per allowed character. */ + if (yyscanner->yyStateBuf == NULL) { + yyscanner->yyStateBuf = (yyStateType *)yyalloc(YY_STATE_BUF_SIZE, yyscanner); + } + if (yyscanner->yyStateBuf == NULL) { + yypanic("out of dynamic memory in yylex()", yyscanner); + } +]]) + + if (yyscanner->yyStart == 0) { + yyscanner->yyStart = 1; /* first start state */ + } + if (yyscanner->yyin == NULL) { + yyscanner->yyin = stdin; + } + if (yyscanner->yyout == NULL) { + yyscanner->yyout = stdout; + } + if (yy_current_buffer(yyscanner) == NULL) { + yyensure_buffer_stack (yyscanner); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = + yy_create_buffer(yyscanner->yyin, flexInputBufferSize, yyscanner); + } + + yy_load_buffer_state(yyscanner); + } + + /* open scope of user declarationns */ + { +%% [4.0] user's declarations go here + + while (/*CONSTCOND*/1) { /* loops until end-of-file is reached */ +m4_ifdef([[M4_MODE_YYMORE_USED]], [[ +m4_ifdef([[M4_MODE_NO_YYTEXT_IS_ARRAY]], [[ + yyscanner->yyMoreLen = 0; + if (yyscanner->yyMoreFlag) { + yyscanner->yyMoreLen = (int) (yyscanner->yyCBufP - yyscanner->yytext_ptr); + yyscanner->yyMoreFlag = false; + } +]]) +]]) + yyCp = yyscanner->yyCBufP; + + /* Support of yytext. */ + *yyCp = yyscanner->yyHoldChar; + + /* yyBp points to the position in yyChBuf of the start of + * the current run. + */ + yyBp = yyCp; + +M4_GEN_START_STATE + + yyMatchLabel: + /* Generate the code to find the next match. */ +m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[m4_dnl +m4_ifdef([[M4_MODE_GENTABLES]], [[m4_dnl + while ((yyCurrentState = yyNxt[yyCurrentState][ M4_EC(YY_SC_TO_UI(*yyCp)) ]) > 0) { +]]) +m4_ifdef([[M4_MODE_NO_GENTABLES]], [[ + while ((yyCurrentState = yyNxt[yyCurrentState*YYNXT_LOLEN + M4_EC(YY_SC_TO_UI(*yyCp)) ]) > 0) { +]]) +M4_GEN_BACKING_UP + yyCp++; + } + yyCurrentState = -yyCurrentState; +]]) +m4_ifdef([[M4_MODE_FULLSPD]], [[ + M4_GEN_NEXT_MATCH_FULLSPD(M4_EC(YY_SC_TO_UI(*yyCp)), M4_EC(YY_SC_TO_UI(*++yyCp))) +]]) +m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[ + do { + M4_GEN_NEXT_COMPRESSED_STATE(M4_EC(YY_SC_TO_UI(*yyCp))) + + m4_ifdef([[M4_MODE_USES_REJECT]], [[*yyscanner->yyStatePtr++ = yyCurrentState;]]) + ++yyCp; + + } + m4_ifdef([[M4_MODE_INTERACTIVE]], [[while (yyBase[yyCurrentState] != YY_JAMBASE);]]) + m4_ifdef([[M4_MODE_NO_INTERACTIVE]], [[while (yyCurrentState != YY_JAMSTATE);]]) + +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_NO_INTERACTIVE]], [[ + /* Do the guaranteed-needed backing up to figure out + * the match. + */ + yyCp = yyscanner->yyLastAcceptingCharPos; + yyCurrentState = yyscanner->yyLastAcceptingState; +]]) +]]) +]]) + + yyFindActionLabel: + /* code to find the action number goes here */ + m4_ifdef([[M4_MODE_FULLSPD]], [[yyAct = yyCurrentState[-1].yyNxt;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yyAct = yyAccept[yyCurrentState];]]) +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT]], [[ + yyCurrentState = *--yyscanner->yyStatePtr; + yyscanner->yyLp = yyAccept[yyCurrentState]; +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT_REALLY_USED]], [[findRuleLabel: /* we branch to this label when backing up */]]) + for (; ;) { /* loop until we find out what rule we matched */ + if (yyscanner->yyLp && yyscanner->yyLp < yyAccept[yyCurrentState + 1]) { + yyAct = yyAcclist[yyscanner->yyLp]; +m4_ifdef([[M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + if ((yyAct & YY_TRAILING_HEAD_MASK) != 0 || yyscanner->yyLookingForTrailBegin) { + if (yyAct == yyscanner->yyLookingForTrailBegin) { + yyscanner->yyLookingForTrailBegin = 0; + yyAct &= ~YY_TRAILING_HEAD_MASK; + break; + } + } else if ((yyAct & YY_TRAILING_MASK) != 0) { + yyscanner->yyLookingForTrailBegin = yyAct & ~YY_TRAILING_MASK; + yyscanner->yyLookingForTrailBegin |= YY_TRAILING_HEAD_MASK; +m4_ifdef([[M4_MODE_REAL_REJECT]], [[ + /* Remember matched text in case we back up + * due to REJECT. + */ + yyscanner->yyFullMatch = yyCp; + yyscanner->yyFullState = yyscanner->yyStatePtr; + yyscanner->yyFullLp = yyscanner->yyLp; +]]) + } else { + yyscanner->yyFullMatch = yyCp; + yyscanner->yyFullState = yyscanner->yyStatePtr; + yyscanner->yyFullLp = yyscanner->yyLp; + break; + } + ++yyscanner->yyLp; + goto findRuleLabel; +]]) +m4_ifdef([[M4_MODE_NO_VARIABLE_TRAILING_CONTEXT_RULES]], [[ + /* Remember matched text in case we back up due to + * trailing context plus REJECT. + */ + yyscanner->yyFullMatch = yyCp; + break; +]]) + } + + --yyCp; + + /* We could consolidate the following two lines with those at + * the beginning, but at the cost of complaints that we're + * branching inside a loop. + */ + yyCurrentState = *--yyscanner->yyStatePtr; + yyscanner->yyLp = yyAccept[yyCurrentState]; + } /* close for */ +]]) +m4_ifdef([[M4_MODE_FIND_ACTION_COMPRESSED]], [[ yyAct = yyAccept[yyCurrentState]; + if (yyAct == 0) { /* have to back up */ + yyCp = yyscanner->yyLastAcceptingCharPos; + yyCurrentState = yyscanner->yyLastAcceptingState; + yyAct = yyAccept[yyCurrentState]; + } +]]) + + yyDoBeforeAction(yyscanner, yyCp, yyBp); + +m4_ifdef([[M4_MODE_YYLINENO]],[[ +m4_define([[M4_YYL_BASE]], [[m4_ifdef([[M4_MODE_YYMORE_USED]], + [[m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], + [[yyscanner->yyPrevMoreOffset]], [[yyscanner->yyMoreLen]])]], [[0]])]]) + if (yyAct != YY_END_OF_BUFFER && yyRuleCanMatchEOL[yyAct]) { + int yyl; + for (yyl = M4_YYL_BASE; yyl < yyscanner->yyleng; ++yyl) { + if (yyscanner->yytext[yyl] == '\n') { + yybumpline(yyscanner); + + } + } + } +]]) + + doActionLabel: /* This label is used only to access EOF actions. */ + +m4_ifdef([[M4_MODE_DEBUG]], [[ + if (yyscanner->yyflexdebug) { + if (yyAct == 0) { + fprintf(stderr, "--scanner backing up\n"); + } else if (yyAct < YY_NUM_RULES) { + fprintf(stderr, "--accepting rule at line %ld (\"%s\")\n", + (long)yy_rule_linenum[yyAct], yyscanner->yytext); + } else if (yyAct == YY_NUM_RULES) { + fprintf(stderr, "--accepting default rule (\"%s\")\n", + yyscanner->yytext); + } else if (yyAct == YY_NUM_RULES + 1) { + fprintf(stderr, "--(end of buffer or a NUL)\n"); + } else { + fprintf(stderr, "--EOF (start condition %d)\n", yystart(yyscanner)); + } + } +]]) + + switch (yyAct) { /* beginning of action switch */ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_MODE_HAS_BACKING_UP]], [[ + case 0: /* must back up */ + /* undo the effects of yyDoBeforeAction() */ + *yyCp = yyscanner->yyHoldChar; + + /* Backing-up info for compressed tables is taken *after* */ + /* yyCp has been incremented for the next state. */ + yyCp = yyscanner->yyLastAcceptingCharPos; + m4_ifdef([[M4_MODE_FULLSPD]], [[yyCp++;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yyCp++;]]) + + yyCurrentState = yyscanner->yyLastAcceptingState; + goto yyFindActionLabel; +]]) +]]) +%% [5.0] user actions get inserted here + + case YY_END_OF_BUFFER: + { + /* Amount of text matched not including the EOB char. */ + int yyAmountOfMatchedText = (int) (yyCp - yyscanner->yytext_ptr) - 1; + + /* Undo the effects of yyDoBeforeAction(). */ + *yyCp = yyscanner->yyHoldChar; + YY_RESTORE_YY_MORE_OFFSET + + if (yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyBufferStatus == YY_BUFFER_NEW) { + /* We're scanning a new file or input source. It's + * possible that this happened because the user + * just pointed yyin at a new source and called + * yylex(). If so, then we have to assure + * consistency between yy_current_buffer(yyscanner) and our + * globals. Here is the right place to do so, because + * this is the first action (other than possibly a + * back-up) that will match for the new input source. + */ + yyscanner->yyNChars = yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyNChars; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyInputFile = yyscanner->yyin; + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyBufferStatus = YY_BUFFER_NORMAL; + } + + /* Note that here we test for yyCBufP "<=" to the position + * of the first EOB in the buffer, since yyCBufP will + * already have been incremented past the NUL character + * (since all states make transitions on EOB to the + * end-of-buffer state). Contrast this with the test + * in input(). + */ + if (yyscanner->yyCBufP <= &yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[yyscanner->yyNChars]) { /* This was really a NUL. */ + yyStateType yyNextState; + + yyscanner->yyCBufP = yyscanner->yytext_ptr + yyAmountOfMatchedText; + + yyCurrentState = yyGetPreviousState(yyscanner); + + /* Okay, we're now positioned to make the NUL + * transition. We couldn't have + * yyGetPreviousState() go ahead and do it + * for us because it doesn't know how to deal + * with the possibility of jamming (and we don't + * want to build jamming into it because then it + * will run more slowly). + */ + + yyNextState = yyTryNULtrans(yyCurrentState, yyscanner); + + yyBp = yyscanner->yytext_ptr + YY_MORE_ADJ; + + if (yyNextState) { + /* Consume the NUL. */ + yyCp = ++yyscanner->yyCBufP; + yyCurrentState = yyNextState; + goto yyMatchLabel; + } else { +%# Disguised case statement on table modes + m4_ifdef([[M4_MODE_FULLSPD]], [[yyCp = yyscanner->yyCBufP;]]) + m4_ifdef([[M4_MODE_FIND_ACTION_FULLTBL]], [[yyCp = yyscanner->yyCBufP;]]) +m4_ifdef([[M4_MODE_NO_FULLSPD_OR_FULLTBL]], [[ +m4_ifdef([[M4_MODE_NO_USES_REJECT]], [[ +m4_ifdef([[M4_NOT_MODE_INTERACTIVE]], [[ + /* Do the guaranteed-needed backing up to figure + * out the match. + */ + yyCp = yyscanner->yyLastAcceptingCharPos; + yyCurrentState = yyscanner->yyLastAcceptingState; +]]) +]]) +%# Disguised case statement on table modes ends +m4_ifdef([[M4_MODE_FIND_ACTION_REJECT_OR_INTERACTIVE]], [[ + /* Still need to initialize yyCp, though + * yyCurrentState was set up by + * yyGetPreviousState(). + */ + yyCp = yyscanner->yyCBufP; +]]) +]]) + goto yyFindActionLabel; + } + } else { /* not a NUL */ + switch (yy_get_next_buffer(yyscanner)) { + case EOB_ACT_END_OF_FILE: + yyscanner->yyDidBufferSwitchOnEof = false; + + if (yywrap(yyscanner)) { + /* Note: because we've taken care in + * yy_get_next_buffer() to have set up + * yytext, we can now set up + * yyCBufP so that if some total + * hoser (like flex itself) wants to + * call the scanner after we return the + * flexEOF, it'll still work - another + * flexEOF will get returned. + */ + yyscanner->yyCBufP = yyscanner->yytext_ptr + YY_MORE_ADJ; + + yyAct = YY_STATE_EOF(yystart(yyscanner)); + goto doActionLabel; + } else { + if (! yyscanner->yyDidBufferSwitchOnEof) { + yyrestart(yyscanner->yyin, yyscanner); + } + } + break; + case EOB_ACT_CONTINUE_SCAN: + yyscanner->yyCBufP = yyscanner->yytext_ptr + yyAmountOfMatchedText; + + yyCurrentState = yyGetPreviousState(yyscanner); + + yyCp = yyscanner->yyCBufP; + yyBp = yyscanner->yytext_ptr + YY_MORE_ADJ; + goto yyMatchLabel; + + case EOB_ACT_LAST_MATCH: + yyscanner->yyCBufP = + &yyscanner->yyBufferStack[yyscanner->yyBufferStackTop]->yyChBuf[yyscanner->yyNChars]; + + yyCurrentState = yyGetPreviousState(yyscanner); + + yyCp = yyscanner->yyCBufP; + yyBp = yyscanner->yytext_ptr + YY_MORE_ADJ; + goto yyFindActionLabel; + } /* end EOB inner switch */ + } /* end if */ + break; + } /* case YY_END_OF_BUFFER */ + default: + yypanic("fatal flex scanner internal error--no action found", yyscanner); + } /* end of action switch */ + } /* end of scanning one token */ + } /* end of user's declarations */ +} /* end of yylex */ + +m4_undefine([[yyless]]) + +/* Redefine yyless() so it works in section 3 code. */ + +void yyless(int n, FlexLexer *yyscanner) { + /* Undo effects of setting up yytext. */ + m4_ifdef([[M4_MODE_YYLINENO]], [[yyLessLineno(n, yyscanner);]]) + yyscanner->yytext[yyscanner->yyleng] = yyscanner->yyHoldChar; + yyscanner->yyCBufP = yyscanner->yytext + n; + yyscanner->yyHoldChar = *yyscanner->yyCBufP; + *yyscanner->yyCBufP = '\0'; + yyscanner->yyleng = n; +} + +static int yy_init_globals (FlexLexer *yyscanner) { + /* + * This function is called from yylex_destroy(), so don't allocate here. + */ + yyscanner->yyBufferStack = NULL; + yyscanner->yyBufferStackTop = 0; + yyscanner->yyBufferStackMax = 0; + yyscanner->yyCBufP = NULL; + yyscanner->yyInit = false; + yyscanner->yyStart = 0; + yyscanner->yyStartStackOffset = 0; + yyscanner->yyStartStackDepth = 0; + yyscanner->yyStartStack = NULL; + +m4_ifdef([[M4_MODE_USES_REJECT]], +[[ + yyscanner->yyStateBuf = NULL; + yyscanner->yyStatePtr = NULL; + yyscanner->yyFullMatch = NULL; + yyscanner->yyLp = 0; +]]) + +m4_ifdef([[M4_MODE_YYTEXT_IS_ARRAY]], +[[ + yyscanner->yytext_ptr = 0; + yyscanner->yyMoreOffset = 0; + yyscanner->yyPrevMoreOffset = 0; +]]) + + yyscanner->yyin = NULL; + yyscanner->yyout = NULL; + + return 0; +} + +/* User-visible API */ + +/* yylex_init is special because it creates the scanner itself, so it is + * the ONLY reentrant function that doesn't take the scanner as the last argument. + * That's why we explicitly handle the declaration, instead of using our macros. + */ +int yylex_init(FlexLexer **globalsPtr) { + if (globalsPtr == NULL) { + errno = EINVAL; + return 1; + } + + *globalsPtr = (FlexLexer *) yyalloc (sizeof(FlexLexer), NULL); + + if (*globalsPtr == NULL) { + errno = ENOMEM; + return 1; + } + + /* By setting to 0xAA, we expose bugs in yy_init_globals. Leave at 0x00 for releases. */ + memset(*globalsPtr,0x00,sizeof(FlexLexer)); + + return yy_init_globals (*globalsPtr); +} + +m4_ifdef([[M4_MODE_EXTRA_TYPE]], [[m4_dnl +/* yylex_init_extra has the same functionality as yylex_init, but follows the + * convention of taking the scanner as the last argument. Note however, that + * this is a *pointer* to a scanner, as it will be allocated by this call (and + * is the reason, too, why this function also must handle its own declaration). + * The user defined value in the first argument will be available to yyalloc in + * the yyextra field. + */ +int yylex_init_extra(M4_MODE_EXTRA_TYPE yyUserDefined, FlexLexer **globalsPtr) { + FlexLexer dummyYyguts; + + yyset_extra(yyUserDefined, &dummy_yyguts); + + if (globalsPtr == NULL) { + errno = EINVAL; + return 1; + } + + *globalsPtr = (FlexLexer *) yyalloc(sizeof(FlexLexer), &dummyYyguts); + + if (*globalsPtr == NULL) { + errno = ENOMEM; + return 1; + } + + /* By setting to 0xAA, we expose bugs in + yy_init_globals. Leave at 0x00 for releases. */ + memset(*globalsPtr,0x00,sizeof(FlexLexer)); + + yyset_extra(yyUserDefined, *globalsPtr); + + return yy_init_globals(*globalsPtr); +} +]]) + +/* yylex_destroy is for both reentrant and non-reentrant scanners. */ +int yylex_destroy(FlexLexer *yyscanner) { + + /* Pop the buffer stack, destroying each element. */ + while(yy_current_buffer(yyscanner)) { + yy_delete_buffer(yy_current_buffer(yyscanner), yyscanner); + yyscanner->yyBufferStack[yyscanner->yyBufferStackTop] = NULL; + yypop_buffer_state(yyscanner); + } + + /* Destroy the stack itself. */ + yyfree(yyscanner->yyBufferStack, yyscanner); + yyscanner->yyBufferStack = NULL; + + /* Destroy the start condition stack. */ + yyfree(yyscanner->yyStartStack, yyscanner); + yyscanner->yyStartStack = NULL; + +m4_ifdef([[M4_MODE_USES_REJECT]], +[[ + yyfree(yyscanner->yyStateBuf, yyscanner); + yyscanner->yyStateBuf = NULL; +]]) + + /* Reset the globals. This is important in a non-reentrant scanner so the next time + * yylex() is called, initialization will occur. */ + yy_init_globals(yyscanner); + + /* Destroy the main struct (reentrant only). */ + yyfree (yyscanner, yyscanner); + yyscanner = NULL; + return 0; +} + +m4_ifdef([[M4_YY_MAIN]], [[ +int main () { + FlexLexer *lexer; + yylex_init(&lexer); + yylex(lexer); + yylex_destroy(lexer); + yylex(); + + return 0; +} +]]) + +%# Local Variables: +%# mode:c +%# c-file-style:"k&r" +%# c-basic-offset:8 +%# End: @@ -46,28 +46,17 @@ void flexinit(int, char **); void readin(void); void set_up_initial_allocations(void); - /* these globals are all defined and commented in flexdef.h */ -int printstats, syntaxerror, eofseen, ddebug, trace, nowarn, spprdflt; -int interactive, lex_compat, posix_compat, do_yylineno, - useecs, fulltbl, usemecs; -int fullspd, gen_line_dirs, performance_report, backing_up_report; -int C_plus_plus, long_align, use_read, yytext_is_array, do_yywrap, - csize; -int reentrant, bison_bridge_lval, bison_bridge_lloc; +int syntaxerror, eofseen; int yymore_used, reject, real_reject, continued_action, in_rule; -int yymore_really_used, reject_really_used; -int trace_hex = 0; int datapos, dataline, linenum; FILE *skelfile = NULL; int skel_ind = 0; char *action_array; int action_size, defs1_offset, prolog_offset, action_offset, action_index; -char *infilename = NULL, *outfilename = NULL, *headerfilename = NULL; -int did_outfilename; -char *prefix, *yyclass, *extra_type = NULL; -int do_stdinit, use_stdout; +char *infilename = NULL; +char *extra_type = NULL; int onestate[ONE_STACK_SIZE], onesym[ONE_STACK_SIZE]; int onenext[ONE_STACK_SIZE], onedef[ONE_STACK_SIZE], onesp; int maximum_mns, current_mns, current_max_rules; @@ -96,10 +85,9 @@ int lastccl, *cclmap, *ccllen, *cclng, cclreuse; int current_maxccls, current_max_ccl_tbl_size; unsigned char *ccltbl; char nmstr[MAXLINE]; -int sectnum, nummt, hshcol, dfaeql, numeps, eps2, num_reallocs; +int sectnum, nummt, hshcol, dfaeql, numeps, eps2, num_reallocs, nmval; int tmpuses, totnst, peakpairs, numuniq, numdup, hshsave; int num_backing_up, bol_needed; -FILE *backing_up_file; int end_of_buffer_state; char **input_files; int num_input_files; @@ -110,6 +98,10 @@ int nlch = '\n'; bool tablesext, tablesverify, gentables; char *tablesfilename=0,*tablesname=0; struct yytbl_writer tableswr; +size_t footprint; + +struct ctrl_bundle_t ctrl; +struct env_bundle_t env; /* Make sure program_name is initialized so we don't crash if writing * out an error message before getting the program name from argv[0]. @@ -125,7 +117,6 @@ extern FILE* yyout; static char outfile_path[MAXLINE]; static int outfile_created = 0; -static char *skelname = NULL; static int _stdout_closed = 0; /* flag to prevent double-fclose() on stdout. */ const char *escaped_qstart = "]]M4_YY_NOOP[M4_YY_NOOP[M4_YY_NOOP[["; const char *escaped_qend = "]]M4_YY_NOOP]M4_YY_NOOP]M4_YY_NOOP[["; @@ -138,6 +129,7 @@ int flex_main (int argc, char *argv[]); int flex_main (int argc, char *argv[]) { int i, exit_status, child_status; + int did_eof_rule = false; /* Set a longjmp target. Yes, I know it's a hack, but it gets worse: The * return value of setjmp, if non-zero, is the desired exit code PLUS ONE. @@ -148,44 +140,128 @@ int flex_main (int argc, char *argv[]) */ exit_status = setjmp (flex_main_jmp_buf); if (exit_status){ - if (stdout && !_stdout_closed && !ferror(stdout)){ - fflush(stdout); - fclose(stdout); - } - while (wait(&child_status) > 0){ - if (!WIFEXITED (child_status) - || WEXITSTATUS (child_status) != 0){ - /* report an error of a child - */ - if( exit_status <= 1 ) - exit_status = 2; - - } - } - return exit_status - 1; - } + if (stdout && !_stdout_closed && !ferror(stdout)){ + fflush(stdout); + fclose(stdout); + } + while (wait(&child_status) > 0){ + if (!WIFEXITED (child_status) + || WEXITSTATUS (child_status) != 0){ + /* report an error of a child + */ + if( exit_status <= 1 ) + exit_status = 2; + + } + } + return exit_status - 1; + } flexinit (argc, argv); readin (); - skelout (); - /* %% [1.5] DFA */ - ntod (); + skelout (true); /* %% [1.0] DFA */ + footprint += ntod (); for (i = 1; i <= num_rules; ++i) if (!rule_useful[i] && i != default_rule) line_warning (_("rule cannot be matched"), rule_linenum[i]); - if (spprdflt && !reject && rule_useful[default_rule]) + if (ctrl.spprdflt && !reject && rule_useful[default_rule]) line_warning (_ ("-s option given but default rule can be matched"), rule_linenum[default_rule]); + comment("START of m4 controls\n"); + + // mode switches for yy_trans_info specification + // nultrans + if (nultrans) + visible_define ( "M4_MODE_NULTRANS"); + else { + visible_define ( "M4_MODE_NO_NULTRANS"); + if (ctrl.fulltbl) + visible_define ( "M4_MODE_NULTRANS_FULLTBL"); + else + visible_define ( "M4_MODE_NO_NULTRANS_FULLTBL"); + if (ctrl.fullspd) + visible_define ( "M4_MODE_NULTRANS_FULLSPD"); + else + visible_define ( "M4_MODE_NO_NULTRANS_FULLSPD"); + } + + comment("END of m4 controls\n"); + out ("\n"); + + comment("START of Flex-generated definitions\n"); + out_str_dec ("M4_HOOK_CONST_DEFINE_UINT(%s, %d)", "YY_NUM_RULES", num_rules); + out_str_dec ("M4_HOOK_CONST_DEFINE_STATE(%s, %d)", "YY_END_OF_BUFFER", num_rules + 1); + out_str_dec ("M4_HOOK_CONST_DEFINE_STATE(%s, %d)", "YY_JAMBASE", jambase); + out_str_dec ("M4_HOOK_CONST_DEFINE_STATE(%s, %d)", "YY_JAMSTATE", jamstate); + out_str_dec ("M4_HOOK_CONST_DEFINE_BYTE(%s, %d)", "YY_NUL_EC", NUL_ec); + /* Need to define the transet type as a size large + * enough to hold the biggest offset. + */ + out_str ("M4_HOOK_SET_OFFSET_TYPE(%s)", optimize_pack(tblend + numecs + 1)->name); + comment("END of Flex-generated definitions\n"); + + skelout (true); /* %% [2.0] - tables get dumped here */ + /* Generate the C state transition tables from the DFA. */ make_tables (); + skelout (true); /* %% [3.0] - mode-dependent static declarations get dumped here */ + + out (&action_array[defs1_offset]); + + line_directive_out (stdout, NULL, linenum); + + skelout (true); /* %% [4.0] - various random yylex internals get dumped here */ + + /* Copy prolog to output file. */ + out (&action_array[prolog_offset]); + + line_directive_out (stdout, NULL, linenum); + + skelout (true); /* %% [5.0] - main loop of matching-engine code gets dumped here */ + + /* Copy actions to output file. */ + out (&action_array[action_offset]); + + line_directive_out (stdout, NULL, linenum); + + /* generate cases for any missing EOF rules */ + for (i = 1; i <= lastsc; ++i) + if (!sceof[i]) { + out_str ("M4_HOOK_EOF_STATE_CASE_ARM(%s)", scname[i]); + outc('\n'); + out ("M4_HOOK_EOF_STATE_CASE_FALLTHROUGH"); + outc('\n'); + did_eof_rule = true; + } + + if (did_eof_rule) { + out ("M4_HOOK_EOF_STATE_CASE_TERMINATE"); + } + + skelout (true); + + /* Copy remainder of input to output. */ + + line_directive_out (stdout, infilename, linenum); + + if (sectnum == 3) { + OUT_BEGIN_CODE (); + if (!ctrl.no_section3_escape) + fputs("[[", stdout); + (void) flexscan (); /* copy remainder of input to output */ + if (!ctrl.no_section3_escape) + fputs("]]", stdout); + OUT_END_CODE (); + } + /* Note, flexend does not return. It exits with its argument * as status. */ @@ -209,297 +285,180 @@ int main (int argc, char *argv[]) return flex_main (argc, argv); } +/* Set up the output filter chain. */ + +void initialize_output_filters(void) +{ + const char * m4 = NULL; + + output_chain = filter_create_int(NULL, filter_tee_header, env.headerfilename); + if ( !(m4 = getenv("M4"))) { + char *slash; + m4 = M4; + if ((slash = strrchr(M4, '/')) != NULL) { + m4 = slash+1; + /* break up $PATH */ + const char *path = getenv("PATH"); + if (!path) { + m4 = M4; + } else { + int m4_length = strlen(m4); + do { + size_t length = strlen(path); + struct stat sbuf; + + const char *endOfDir = strchr(path, ':'); + if (!endOfDir) + endOfDir = path+length; + + { + char *m4_path = calloc(endOfDir-path + 1 + m4_length + 1, 1); + + memcpy(m4_path, path, endOfDir-path); + m4_path[endOfDir-path] = '/'; + memcpy(m4_path + (endOfDir-path) + 1, m4, m4_length + 1); + if (stat(m4_path, &sbuf) == 0 && + (S_ISREG(sbuf.st_mode)) && sbuf.st_mode & S_IXUSR) { + m4 = m4_path; + break; + } + free(m4_path); + } + path = endOfDir+1; + } while (path[0]); + if (!path[0]) + m4 = M4; + } + } + } + filter_create_ext(output_chain, m4, "-P", 0); + filter_create_int(output_chain, filter_fix_linedirs, NULL); + + /* For debugging, only run the requested number of filters. */ + if (preproc_level > 0) { + filter_truncate(output_chain, preproc_level); + filter_apply_chain(output_chain); + } +} + + /* check_options - check user-specified options */ void check_options (void) { int i; - const char * m4 = NULL; - if (lex_compat) { - if (C_plus_plus) + if (ctrl.lex_compat) { + if (ctrl.C_plus_plus) flexerror (_("Can't use -+ with -l option")); - if (fulltbl || fullspd) + if (ctrl.fulltbl || ctrl.fullspd) flexerror (_("Can't use -f or -F with -l option")); - if (reentrant || bison_bridge_lval) + if (ctrl.reentrant || ctrl.bison_bridge_lval) flexerror (_ - ("Can't use --reentrant or --bison-bridge with -l option")); + ("Can't use --ctrl.reentrant or --bison-bridge with -l option")); - yytext_is_array = true; - do_yylineno = true; - use_read = false; + ctrl.yytext_is_array = true; + ctrl.do_yylineno = true; + ctrl.use_read = false; } #if 0 /* This makes no sense whatsoever. I'm removing it. */ - if (do_yylineno) + if (ctrl.do_yylineno) /* This should really be "maintain_backup_tables = true" */ - reject_really_used = true; + ctrl.reject_really_used = true; #endif - if (csize == unspecified) { - if ((fulltbl || fullspd) && !useecs) - csize = DEFAULT_CSIZE; + if (ctrl.csize == trit_unspecified) { + if ((ctrl.fulltbl || ctrl.fullspd) && !ctrl.useecs) + ctrl.csize = DEFAULT_CSIZE; else - csize = CSIZE; + ctrl.csize = CSIZE; } - if (interactive == unspecified) { - if (fulltbl || fullspd) - interactive = false; + if (ctrl.interactive == trit_unspecified) { + if (ctrl.fulltbl || ctrl.fullspd) + ctrl.interactive = trit_false; else - interactive = true; + ctrl.interactive = trit_true; } - if (fulltbl || fullspd) { - if (usemecs) + if (ctrl.fulltbl || ctrl.fullspd) { + if (ctrl.usemecs) flexerror (_ ("-Cf/-CF and -Cm don't make sense together")); - if (interactive) + if (ctrl.interactive != trit_false) flexerror (_("-Cf/-CF and -I are incompatible")); - if (lex_compat) + if (ctrl.lex_compat) flexerror (_ ("-Cf/-CF are incompatible with lex-compatibility mode")); - if (fulltbl && fullspd) + if (ctrl.fulltbl && ctrl.fullspd) flexerror (_ ("-Cf and -CF are mutually exclusive")); } - if (C_plus_plus && fullspd) + if (ctrl.C_plus_plus && ctrl.fullspd) flexerror (_("Can't use -+ with -CF option")); - if (C_plus_plus && yytext_is_array) { + if (ctrl.C_plus_plus && ctrl.yytext_is_array) { lwarn (_("%array incompatible with -+ option")); - yytext_is_array = false; + ctrl.yytext_is_array = false; } - if (C_plus_plus && (reentrant)) + if (ctrl.C_plus_plus && (ctrl.reentrant)) flexerror (_("Options -+ and --reentrant are mutually exclusive.")); - if (C_plus_plus && bison_bridge_lval) + if (ctrl.C_plus_plus && ctrl.bison_bridge_lval) flexerror (_("bison bridge not supported for the C++ scanner.")); - if (useecs) { /* Set up doubly-linked equivalence classes. */ + if (ctrl.useecs) { /* Set up doubly-linked equivalence classes. */ - /* We loop all the way up to csize, since ecgroup[csize] is + /* We loop all the way up to ctrl.csize, since ecgroup[ctrl.csize] is * the position used for NUL characters. */ ecgroup[1] = NIL; - for (i = 2; i <= csize; ++i) { + for (i = 2; i <= ctrl.csize; ++i) { ecgroup[i] = i - 1; nextecm[i - 1] = i; } - nextecm[csize] = NIL; + nextecm[ctrl.csize] = NIL; } else { /* Put everything in its own equivalence class. */ - for (i = 1; i <= csize; ++i) { + for (i = 1; i <= ctrl.csize; ++i) { ecgroup[i] = i; nextecm[i] = BAD_SUBSCRIPT; /* to catch errors */ } } - if (extra_type) - buf_m4_define( &m4defs_buf, "M4_EXTRA_TYPE_DEFS", extra_type); - - if (!use_stdout) { + if (!env.use_stdout) { FILE *prev_stdout; - if (!did_outfilename) { - char *suffix; - - if (C_plus_plus) - suffix = "cc"; - else - suffix = "c"; - + if (!env.did_outfilename) { snprintf (outfile_path, sizeof(outfile_path), outfile_template, - prefix, suffix); + ctrl.prefix, suffix()); - outfilename = outfile_path; + env.outfilename = outfile_path; } - prev_stdout = freopen (outfilename, "w+", stdout); + prev_stdout = freopen (env.outfilename, "w+", stdout); if (prev_stdout == NULL) - lerr (_("could not create %s"), outfilename); + lerr (_("could not create %s"), env.outfilename); outfile_created = 1; } - - - /* Setup the filter chain. */ - output_chain = filter_create_int(NULL, filter_tee_header, headerfilename); - if ( !(m4 = getenv("M4"))) { - char *slash; - m4 = M4; - if ((slash = strrchr(M4, '/')) != NULL) { - m4 = slash+1; - /* break up $PATH */ - const char *path = getenv("PATH"); - if (!path) { - m4 = M4; - } else { - int m4_length = strlen(m4); - do { - size_t length = strlen(path); - struct stat sbuf; - - const char *endOfDir = strchr(path, ':'); - if (!endOfDir) - endOfDir = path+length; - - { - char *m4_path = calloc(endOfDir-path + 1 + m4_length + 1, 1); - - memcpy(m4_path, path, endOfDir-path); - m4_path[endOfDir-path] = '/'; - memcpy(m4_path + (endOfDir-path) + 1, m4, m4_length + 1); - if (stat(m4_path, &sbuf) == 0 && - (S_ISREG(sbuf.st_mode)) && sbuf.st_mode & S_IXUSR) { - m4 = m4_path; - break; - } - free(m4_path); - } - path = endOfDir+1; - } while (path[0]); - if (!path[0]) - m4 = M4; - } - } - } - filter_create_ext(output_chain, m4, "-P", 0); - filter_create_int(output_chain, filter_fix_linedirs, NULL); - - /* For debugging, only run the requested number of filters. */ - if (preproc_level > 0) { - filter_truncate(output_chain, preproc_level); - filter_apply_chain(output_chain); - } - yyout = stdout; - - - /* always generate the tablesverify flag. */ - buf_m4_define (&m4defs_buf, "M4_YY_TABLES_VERIFY", tablesverify ? "1" : "0"); - if (tablesext) - gentables = false; - - if (tablesverify) - /* force generation of C tables. */ - gentables = true; - - - if (tablesext) { - FILE *tablesout; - struct yytbl_hdr hdr; - char *pname = 0; - size_t nbytes = 0; - - buf_m4_define (&m4defs_buf, "M4_YY_TABLES_EXTERNAL", NULL); - - if (!tablesfilename) { - nbytes = strlen (prefix) + strlen (tablesfile_template) + 2; - tablesfilename = pname = calloc(nbytes, 1); - snprintf (pname, nbytes, tablesfile_template, prefix); - } - - if ((tablesout = fopen (tablesfilename, "w")) == NULL) - lerr (_("could not create %s"), tablesfilename); - free(pname); - tablesfilename = 0; - - yytbl_writer_init (&tableswr, tablesout); - - nbytes = strlen (prefix) + strlen ("tables") + 2; - tablesname = calloc(nbytes, 1); - snprintf (tablesname, nbytes, "%stables", prefix); - yytbl_hdr_init (&hdr, flex_version, tablesname); - - if (yytbl_hdr_fwrite (&tableswr, &hdr) <= 0) - flexerror (_("could not write tables header")); - } - - if (skelname && (skelfile = fopen (skelname, "r")) == NULL) - lerr (_("can't open skeleton file %s"), skelname); - - if (reentrant) { - buf_m4_define (&m4defs_buf, "M4_YY_REENTRANT", NULL); - if (yytext_is_array) - buf_m4_define (&m4defs_buf, "M4_YY_TEXT_IS_ARRAY", NULL); - } - - if ( bison_bridge_lval) - buf_m4_define (&m4defs_buf, "M4_YY_BISON_LVAL", NULL); - - if ( bison_bridge_lloc) - buf_m4_define (&m4defs_buf, "<M4_YY_BISON_LLOC>", NULL); - - if (strchr(prefix, '[') || strchr(prefix, ']')) - flexerror(_("Prefix cannot include '[' or ']'")); - buf_m4_define(&m4defs_buf, "M4_YY_PREFIX", prefix); - - if (did_outfilename) - line_directive_out (stdout, 0); - - if (do_yylineno) - buf_m4_define (&m4defs_buf, "M4_YY_USE_LINENO", NULL); - - /* Create the alignment type. */ - buf_strdefine (&userdef_buf, "YY_INT_ALIGNED", - long_align ? "long int" : "short int"); - - /* Define the start condition macros. */ - { - struct Buf tmpbuf; - buf_init(&tmpbuf, sizeof(char)); - for (i = 1; i <= lastsc; i++) { - char *str, *fmt = "#define %s %d\n"; - size_t strsz; - - strsz = strlen(fmt) + strlen(scname[i]) + (size_t)(1 + ceil (log10(i))) + 2; - str = malloc(strsz); - if (!str) - flexfatal(_("allocation of macro definition failed")); - snprintf(str, strsz, fmt, scname[i], i - 1); - buf_strappend(&tmpbuf, str); - free(str); - } - buf_m4_define(&m4defs_buf, "M4_YY_SC_DEFS", tmpbuf.elts); - buf_destroy(&tmpbuf); - } - - /* This is where we begin writing to the file. */ - - /* Dump the %top code. */ - if( top_buf.elts) - outn((char*) top_buf.elts); - - /* Dump the m4 definitions. */ - buf_print_strings(&m4defs_buf, stdout); - m4defs_buf.nelts = 0; /* memory leak here. */ - - /* Place a bogus line directive, it will be fixed in the filter. */ - if (gen_line_dirs) - outn("#line 0 \"M4_YY_OUTFILE_NAME\"\n"); - - /* Dump the user defined preproc directives. */ - if (userdef_buf.elts) - outn ((char *) (userdef_buf.elts)); - - skelout (); - /* %% [1.0] */ } /* flexend - terminate flex @@ -516,320 +475,135 @@ void flexend (int exit_status) if (++called_before) FLEX_EXIT (exit_status); + if (ctrl.yyclass != NULL && !ctrl.C_plus_plus) + flexerror (_("%option yyclass only meaningful for C++ scanners")); + if (skelfile != NULL) { if (ferror (skelfile)) lerr (_("input error reading skeleton file %s"), - skelname); + env.skelname); else if (fclose (skelfile)) lerr (_("error closing skeleton file %s"), - skelname); + env.skelname); } -#if 0 - fprintf (header_out, - "#ifdef YY_HEADER_EXPORT_START_CONDITIONS\n"); - fprintf (header_out, - "/* Beware! Start conditions are not prefixed. */\n"); - - /* Special case for "INITIAL" */ - fprintf (header_out, - "#undef INITIAL\n#define INITIAL 0\n"); - for (i = 2; i <= lastsc; i++) - fprintf (header_out, "#define %s %d\n", scname[i], i - 1); - fprintf (header_out, - "#endif /* YY_HEADER_EXPORT_START_CONDITIONS */\n\n"); - - /* Kill ALL flex-related macros. This is so the user - * can #include more than one generated header file. */ - fprintf (header_out, "#ifndef YY_HEADER_NO_UNDEFS\n"); - fprintf (header_out, - "/* Undefine all internal macros, etc., that do no belong in the header. */\n\n"); - - { - const char * undef_list[] = { - - "BEGIN", - "ECHO", - "EOB_ACT_CONTINUE_SCAN", - "EOB_ACT_END_OF_FILE", - "EOB_ACT_LAST_MATCH", - "FLEX_SCANNER", - "REJECT", - "YYFARGS0", - "YYFARGS1", - "YYFARGS2", - "YYFARGS3", - "YYLMAX", - "YYSTATE", - "YY_AT_BOL", - "YY_BREAK", - "YY_BUFFER_EOF_PENDING", - "YY_BUFFER_NEW", - "YY_BUFFER_NORMAL", - "YY_BUF_SIZE", - "M4_YY_CALL_LAST_ARG", - "M4_YY_CALL_ONLY_ARG", - "YY_CURRENT_BUFFER", - "YY_DECL", - "M4_YY_DECL_LAST_ARG", - "M4_YY_DEF_LAST_ARG", - "M4_YY_DEF_ONLY_ARG", - "YY_DO_BEFORE_ACTION", - "YY_END_OF_BUFFER", - "YY_END_OF_BUFFER_CHAR", - "YY_EXIT_FAILURE", - "YY_EXTRA_TYPE", - "YY_FATAL_ERROR", - "YY_FLEX_DEFINED_ECHO", - "YY_FLEX_LEX_COMPAT", - "YY_FLEX_MAJOR_VERSION", - "YY_FLEX_MINOR_VERSION", - "YY_FLEX_SUBMINOR_VERSION", - "YY_FLUSH_BUFFER", - "YY_G", - "YY_INPUT", - "YY_INTERACTIVE", - "YY_INT_ALIGNED", - "YY_LAST_ARG", - "YY_LESS_LINENO", - "YY_LEX_ARGS", - "YY_LEX_DECLARATION", - "YY_LEX_PROTO", - "YY_MAIN", - "YY_MORE_ADJ", - "YY_NEED_STRLEN", - "YY_NEW_FILE", - "YY_NULL", - "YY_NUM_RULES", - "YY_ONLY_ARG", - "YY_PARAMS", - "YY_PROTO", - "M4_YY_PROTO_LAST_ARG", - "M4_YY_PROTO_ONLY_ARG void", - "YY_READ_BUF_SIZE", - "YY_REENTRANT", - "YY_RESTORE_YY_MORE_OFFSET", - "YY_RULE_SETUP", - "YY_SC_TO_UI", - "YY_SKIP_YYWRAP", - "YY_START", - "YY_START_STACK_INCR", - "YY_STATE_EOF", - "YY_STDINIT", - "YY_TRAILING_HEAD_MASK", - "YY_TRAILING_MASK", - "YY_USER_ACTION", - "YY_USE_CONST", - "YY_USE_PROTOS", - "unput", - "yyTABLES_NAME", - "yy_create_buffer", - "yy_delete_buffer", - "yy_flex_debug", - "yy_flush_buffer", - "yy_init_buffer", - "yy_load_buffer_state", - "yy_new_buffer", - "yy_scan_buffer", - "yy_scan_bytes", - "yy_scan_string", - "yy_set_bol", - "yy_set_interactive", - "yy_switch_to_buffer", - "yypush_buffer_state", - "yypop_buffer_state", - "yyensure_buffer_stack", - "yyalloc", - "const", - "yyextra", - "yyfree", - "yyget_debug", - "yyget_extra", - "yyget_in", - "yyget_leng", - "yyget_column", - "yyget_lineno", - "yyget_lloc", - "yyget_lval", - "yyget_out", - "yyget_text", - "yyin", - "yyleng", - "yyless", - "yylex", - "yylex_destroy", - "yylex_init", - "yylex_init_extra", - "yylineno", - "yylloc", - "yylval", - "yymore", - "yyout", - "yyrealloc", - "yyrestart", - "yyset_debug", - "yyset_extra", - "yyset_in", - "yyset_column", - "yyset_lineno", - "yyset_lloc", - "yyset_lval", - "yyset_out", - "yytables_destroy", - "yytables_fload", - "yyterminate", - "yytext", - "yytext_ptr", - "yywrap", - - /* must be null-terminated */ - NULL}; - - - for (i=0; undef_list[i] != NULL; i++) - fprintf (header_out, "#undef %s\n", undef_list[i]); - } - - /* undef any of the auto-generated symbols. */ - for (i = 0; i < defs_buf.nelts; i++) { - - /* don't undef start conditions */ - if (sclookup (((char **) defs_buf.elts)[i]) > 0) - continue; - fprintf (header_out, "#undef %s\n", - ((char **) defs_buf.elts)[i]); - } - - fprintf (header_out, - "#endif /* !YY_HEADER_NO_UNDEFS */\n"); - fprintf (header_out, "\n"); - fprintf (header_out, "#undef %sIN_HEADER\n", prefix); - fprintf (header_out, "#endif /* %sHEADER_H */\n", prefix); - - if (ferror (header_out)) - lerr (_("error creating header file %s"), - headerfilename); - fflush (header_out); - fclose (header_out); -#endif - if (exit_status != 0 && outfile_created) { if (ferror (stdout)) lerr (_("error writing output file %s"), - outfilename); + env.outfilename); else if ((_stdout_closed = 1) && fclose (stdout)) lerr (_("error closing output file %s"), - outfilename); + env.outfilename); - else if (unlink (outfilename)) + else if (unlink (env.outfilename)) lerr (_("error deleting output file %s"), - outfilename); + env.outfilename); } - if (backing_up_report && backing_up_file) { + if (env.backing_up_report && ctrl.backing_up_file) { if (num_backing_up == 0) - fprintf (backing_up_file, _("No backing up.\n")); - else if (fullspd || fulltbl) - fprintf (backing_up_file, + fprintf (ctrl.backing_up_file, _("No backing up.\n")); + else if (ctrl.fullspd || ctrl.fulltbl) + fprintf (ctrl.backing_up_file, _ ("%d backing up (non-accepting) states.\n"), num_backing_up); else - fprintf (backing_up_file, + fprintf (ctrl.backing_up_file, _("Compressed tables always back up.\n")); - if (ferror (backing_up_file)) + if (ferror (ctrl.backing_up_file)) lerr (_("error writing backup file %s"), backing_name); - else if (fclose (backing_up_file)) + else if (fclose (ctrl.backing_up_file)) lerr (_("error closing backup file %s"), backing_name); } - if (printstats) { + if (env.printstats) { fprintf (stderr, _("%s version %s usage statistics:\n"), program_name, flex_version); fprintf (stderr, _(" scanner options: -")); - if (C_plus_plus) + if (ctrl.C_plus_plus) putc ('+', stderr); - if (backing_up_report) + if (env.backing_up_report) putc ('b', stderr); - if (ddebug) + if (ctrl.ddebug) putc ('d', stderr); if (sf_case_ins()) putc ('i', stderr); - if (lex_compat) + if (ctrl.lex_compat) putc ('l', stderr); - if (posix_compat) + if (ctrl.posix_compat) putc ('X', stderr); - if (performance_report > 0) + if (env.performance_hint > 0) putc ('p', stderr); - if (performance_report > 1) + if (env.performance_hint > 1) putc ('p', stderr); - if (spprdflt) + if (ctrl.spprdflt) putc ('s', stderr); - if (reentrant) + if (ctrl.reentrant) fputs ("--reentrant", stderr); - if (bison_bridge_lval) + if (ctrl.bison_bridge_lval) fputs ("--bison-bridge", stderr); - if (bison_bridge_lloc) + if (ctrl.bison_bridge_lloc) fputs ("--bison-locations", stderr); - if (use_stdout) + if (env.use_stdout) putc ('t', stderr); - if (printstats) + if (env.printstats) putc ('v', stderr); /* always true! */ - if (nowarn) + if (env.nowarn) putc ('w', stderr); - if (interactive == false) + if (ctrl.interactive == trit_false) putc ('B', stderr); - if (interactive == true) + if (ctrl.interactive == trit_true) putc ('I', stderr); - if (!gen_line_dirs) + if (!ctrl.gen_line_dirs) putc ('L', stderr); - if (trace) + if (env.trace) putc ('T', stderr); - if (csize == unspecified) - /* We encountered an error fairly early on, so csize + if (ctrl.csize == trit_unspecified) + /* We encountered an error fairly early on, so ctrl.csize * never got specified. Define it now, to prevent * bogus table sizes being written out below. */ - csize = 256; + ctrl.csize = 256; - if (csize == 128) + if (ctrl.csize == 128) putc ('7', stderr); else putc ('8', stderr); fprintf (stderr, " -C"); - if (long_align) + if (ctrl.long_align) putc ('a', stderr); - if (fulltbl) + if (ctrl.fulltbl) putc ('f', stderr); - if (fullspd) + if (ctrl.fullspd) putc ('F', stderr); - if (useecs) + if (ctrl.useecs) putc ('e', stderr); - if (usemecs) + if (ctrl.usemecs) putc ('m', stderr); - if (use_read) + if (ctrl.use_read) putc ('r', stderr); - if (did_outfilename) - fprintf (stderr, " -o%s", outfilename); + if (env.did_outfilename) + fprintf (stderr, " -o%s", env.outfilename); - if (skelname) - fprintf (stderr, " -S%s", skelname); + if (env.skelname != NULL) + fprintf (stderr, " -S%s", env.skelname); - if (strcmp (prefix, "yy")) - fprintf (stderr, " -P%s", prefix); + if (strcmp (ctrl.prefix, "yy")) + fprintf (stderr, " -P%s", ctrl.prefix); putc ('\n', stderr); @@ -843,7 +617,7 @@ void flexend (int exit_status) if (num_backing_up == 0) fprintf (stderr, _(" No backing up\n")); - else if (fullspd || fulltbl) + else if (ctrl.fullspd || ctrl.fulltbl) fprintf (stderr, _ (" %d backing-up (non-accepting) states\n"), @@ -880,7 +654,7 @@ void flexend (int exit_status) _(" %d/%d unique/duplicate transitions\n"), numuniq, numdup); - if (fulltbl) { + if (ctrl.fulltbl) { tblsiz = lastdfa * numecs; fprintf (stderr, _(" %d table entries\n"), tblsiz); @@ -911,20 +685,20 @@ void flexend (int exit_status) numtemps, tmpuses); } - if (useecs) { - tblsiz = tblsiz + csize; + if (ctrl.useecs) { + tblsiz = tblsiz + ctrl.csize; fprintf (stderr, _ (" %d/%d equivalence classes created\n"), - numecs, csize); + numecs, ctrl.csize); } - if (usemecs) { + if (ctrl.usemecs) { tblsiz = tblsiz + numecs; fprintf (stderr, _ (" %d/%d meta-equivalence classes created\n"), - nummecs, csize); + nummecs, ctrl.csize); } fprintf (stderr, @@ -949,28 +723,27 @@ void flexinit (int argc, char **argv) char *arg; scanopt_t sopt; - printstats = syntaxerror = trace = spprdflt = false; - lex_compat = posix_compat = C_plus_plus = backing_up_report = - ddebug = fulltbl = false; - fullspd = long_align = nowarn = yymore_used = continued_action = - false; - do_yylineno = yytext_is_array = in_rule = reject = do_stdinit = - false; - yymore_really_used = reject_really_used = unspecified; - interactive = csize = unspecified; - do_yywrap = gen_line_dirs = usemecs = useecs = true; - reentrant = bison_bridge_lval = bison_bridge_lloc = false; - performance_report = 0; - did_outfilename = 0; - prefix = "yy"; - yyclass = 0; - use_read = use_stdout = false; + memset(&ctrl, '\0', sizeof(ctrl)); + syntaxerror = false; + yymore_used = continued_action = false; + in_rule = reject = false; + ctrl.yymore_really_used = ctrl.reject_really_used = trit_unspecified; + + ctrl.do_main = trit_unspecified; + ctrl.interactive = ctrl.csize = trit_unspecified; + ctrl.do_yywrap = ctrl.gen_line_dirs = ctrl.usemecs = ctrl.useecs = true; + ctrl.reentrant = ctrl.bison_bridge_lval = ctrl.bison_bridge_lloc = false; + env.performance_hint = 0; + ctrl.prefix = "yy"; + ctrl.rewrite = false; + ctrl.yylmax = BUFSIZ; + tablesext = tablesverify = false; gentables = true; tablesfilename = tablesname = NULL; sawcmpflag = false; - + /* Initialize dynamic array for holding the rule actions. */ action_size = 2048; /* default size of action array in bytes */ action_array = allocate_character_array (action_size); @@ -979,28 +752,16 @@ void flexinit (int argc, char **argv) /* Initialize any buffers. */ buf_init (&userdef_buf, sizeof (char)); /* one long string */ - buf_init (&defs_buf, sizeof (char *)); /* list of strings */ - buf_init (&yydmap_buf, sizeof (char)); /* one long string */ buf_init (&top_buf, sizeof (char)); /* one long string */ - { - const char * m4defs_init_str[] = {"m4_changequote\n", - "m4_changequote([[, ]])\n"}; - buf_init (&m4defs_buf, sizeof (char *)); - buf_append (&m4defs_buf, &m4defs_init_str, 2); - } - - sf_init (); - - /* initialize regex lib */ - flex_init_regex(); + sf_init (); /* Enable C++ if program name ends with '+'. */ program_name = argv[0]; if (program_name != NULL && program_name[strlen (program_name) - 1] == '+') - C_plus_plus = true; + ctrl.C_plus_plus = true; /* read flags */ sopt = scanopt_init (flexopts, argc, argv, 0); @@ -1023,429 +784,401 @@ void flexinit (int argc, char **argv) } switch ((enum flexopt_flag_t) rv) { - case OPT_CPLUSPLUS: - C_plus_plus = true; + case OPT_CPLUSPLUS: + ctrl.C_plus_plus = true; break; - case OPT_BATCH: - interactive = false; + case OPT_BATCH: + ctrl.interactive = trit_false; break; - case OPT_BACKUP: - backing_up_report = true; + case OPT_BACKUP: + env.backing_up_report = trit_true; break; - case OPT_BACKUP_FILE: - backing_up_report = true; + case OPT_BACKUP_FILE: + env.backing_up_report = true; backing_name = arg; break; - case OPT_DONOTHING: + case OPT_DONOTHING: break; - case OPT_COMPRESSION: + case OPT_COMPRESSION: if (!sawcmpflag) { - useecs = false; - usemecs = false; - fulltbl = false; + ctrl.useecs = false; + ctrl.usemecs = false; + ctrl.fulltbl = false; sawcmpflag = true; } for (i = 0; arg && arg[i] != '\0'; i++) switch (arg[i]) { - case 'a': - long_align = true; + case 'a': + ctrl.long_align = true; break; - case 'e': - useecs = true; + case 'e': + ctrl.useecs = true; break; - case 'F': - fullspd = true; + case 'F': + ctrl.fullspd = true; break; - case 'f': - fulltbl = true; + case 'f': + ctrl.fulltbl = true; break; - case 'm': - usemecs = true; + case 'm': + ctrl.usemecs = true; break; - case 'r': - use_read = true; + case 'r': + ctrl.use_read = true; break; - default: + default: lerr (_ - ("unknown -C option '%c'"), - arg[i]); + ("unknown -C option '%c'"), + arg[i]); break; } break; - case OPT_DEBUG: - ddebug = true; + case OPT_DEBUG: + ctrl.ddebug = true; break; - case OPT_NO_DEBUG: - ddebug = false; + case OPT_NO_DEBUG: + ctrl.ddebug = false; break; - case OPT_FULL: - useecs = usemecs = false; - use_read = fulltbl = true; + case OPT_FULL: + ctrl.useecs = ctrl.usemecs = false; + ctrl.use_read = ctrl.fulltbl = true; break; - case OPT_FAST: - useecs = usemecs = false; - use_read = fullspd = true; + case OPT_FAST: + ctrl.useecs = ctrl.usemecs = false; + ctrl.use_read = ctrl.fullspd = true; break; - case OPT_HELP: + case OPT_HELP: usage (); FLEX_EXIT (0); - case OPT_INTERACTIVE: - interactive = true; + case OPT_INTERACTIVE: + ctrl.interactive = true; break; - case OPT_CASE_INSENSITIVE: + case OPT_CASE_INSENSITIVE: sf_set_case_ins(true); break; - case OPT_LEX_COMPAT: - lex_compat = true; + case OPT_LEX_COMPAT: + ctrl.lex_compat = true; break; - case OPT_POSIX_COMPAT: - posix_compat = true; + case OPT_POSIX_COMPAT: + ctrl.posix_compat = true; break; - case OPT_PREPROC_LEVEL: - preproc_level = (int) strtol(arg,NULL,0); - break; + case OPT_PREPROC_LEVEL: + preproc_level = (int) strtol(arg,NULL,0); + break; - case OPT_MAIN: - buf_strdefine (&userdef_buf, "YY_MAIN", "1"); - do_yywrap = false; + case OPT_MAIN: + ctrl.do_yywrap = false; + ctrl.do_main = trit_true; break; - case OPT_NO_MAIN: - buf_strdefine (&userdef_buf, "YY_MAIN", "0"); + case OPT_NO_MAIN: + ctrl.do_main = trit_false; break; - case OPT_NO_LINE: - gen_line_dirs = false; + case OPT_NO_LINE: + ctrl.gen_line_dirs = false; break; - case OPT_OUTFILE: - outfilename = arg; - did_outfilename = 1; + case OPT_OUTFILE: + env.outfilename = arg; + env.did_outfilename = 1; break; - case OPT_PREFIX: - prefix = arg; + case OPT_PREFIX: + ctrl.prefix = arg; break; - case OPT_PERF_REPORT: - ++performance_report; + case OPT_PERF_REPORT: + ++env.performance_hint; break; - case OPT_BISON_BRIDGE: - bison_bridge_lval = true; + case OPT_BISON_BRIDGE: + ctrl.bison_bridge_lval = true; break; - case OPT_BISON_BRIDGE_LOCATIONS: - bison_bridge_lval = bison_bridge_lloc = true; + case OPT_BISON_BRIDGE_LOCATIONS: + ctrl.bison_bridge_lval = ctrl.bison_bridge_lloc = true; break; - case OPT_REENTRANT: - reentrant = true; + case OPT_REENTRANT: + ctrl.reentrant = true; break; - case OPT_NO_REENTRANT: - reentrant = false; + case OPT_NO_REENTRANT: + ctrl.reentrant = false; break; - case OPT_SKEL: - skelname = arg; + case OPT_SKEL: + env.skelname = arg; break; - case OPT_DEFAULT: - spprdflt = false; + case OPT_DEFAULT: + ctrl.spprdflt = false; break; - case OPT_NO_DEFAULT: - spprdflt = true; + case OPT_NO_DEFAULT: + ctrl.spprdflt = true; break; - case OPT_STDOUT: - use_stdout = true; + case OPT_STDOUT: + env.use_stdout = true; break; - case OPT_NO_UNISTD_H: - //buf_strdefine (&userdef_buf, "YY_NO_UNISTD_H", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_UNISTD_H",0); + case OPT_NO_UNISTD_H: + ctrl.no_unistd = true; break; - case OPT_TABLES_FILE: + case OPT_TABLES_FILE: tablesext = true; tablesfilename = arg; break; - case OPT_TABLES_VERIFY: + case OPT_TABLES_VERIFY: tablesverify = true; break; - case OPT_TRACE: - trace = true; + case OPT_TRACE: + env.trace = true; break; - case OPT_VERBOSE: - printstats = true; + case OPT_VERBOSE: + env.printstats = true; break; - case OPT_VERSION: - printf ("%s %s\n", (C_plus_plus ? "flex++" : "flex"), flex_version); + case OPT_VERSION: + printf ("%s %s\n", (ctrl.C_plus_plus ? "flex++" : "flex"), flex_version); FLEX_EXIT (0); - case OPT_WARN: - nowarn = false; + case OPT_WARN: + env.nowarn = false; break; - case OPT_NO_WARN: - nowarn = true; + case OPT_NO_WARN: + env.nowarn = true; break; - case OPT_7BIT: - csize = 128; + case OPT_7BIT: + ctrl.csize = 128; break; - case OPT_8BIT: - csize = CSIZE; + case OPT_8BIT: + ctrl.csize = CSIZE; break; - case OPT_ALIGN: - long_align = true; + case OPT_ALIGN: + ctrl.long_align = true; break; - case OPT_NO_ALIGN: - long_align = false; + case OPT_NO_ALIGN: + ctrl.long_align = false; break; - case OPT_ALWAYS_INTERACTIVE: - buf_m4_define (&m4defs_buf, "M4_YY_ALWAYS_INTERACTIVE", 0); + case OPT_ALWAYS_INTERACTIVE: + ctrl.always_interactive = true; break; - case OPT_NEVER_INTERACTIVE: - buf_m4_define( &m4defs_buf, "M4_YY_NEVER_INTERACTIVE", 0); + case OPT_NEVER_INTERACTIVE: + ctrl.never_interactive = true; break; - case OPT_ARRAY: - yytext_is_array = true; + case OPT_ARRAY: + ctrl.yytext_is_array = true; break; - case OPT_POINTER: - yytext_is_array = false; + case OPT_POINTER: + ctrl.yytext_is_array = false; break; - case OPT_ECS: - useecs = true; + case OPT_ECS: + ctrl.useecs = true; break; - case OPT_NO_ECS: - useecs = false; + case OPT_NO_ECS: + ctrl.useecs = false; break; - case OPT_HEADER_FILE: - headerfilename = arg; + case OPT_EMIT: + ctrl.emit = arg; break; - case OPT_META_ECS: - usemecs = true; + case OPT_HEADER_FILE: + env.headerfilename = arg; break; - case OPT_NO_META_ECS: - usemecs = false; + case OPT_META_ECS: + ctrl.usemecs = true; break; - case OPT_PREPROCDEFINE: - { - /* arg is "symbol" or "symbol=definition". */ - char *def; + case OPT_NO_META_ECS: + ctrl.usemecs = false; + break; - for (def = arg; - *def != '\0' && *def != '='; ++def) ; + case OPT_PREPROCDEFINE: + { + /* arg is "symbol" or "symbol=definition". */ + char *def; + char buf2[4096]; - buf_strappend (&userdef_buf, "#define "); - if (*def == '\0') { - buf_strappend (&userdef_buf, arg); - buf_strappend (&userdef_buf, - " 1\n"); - } - else { - buf_strnappend (&userdef_buf, arg, - (int) (def - arg)); - buf_strappend (&userdef_buf, " "); - buf_strappend (&userdef_buf, - def + 1); - buf_strappend (&userdef_buf, "\n"); - } - } - break; + for (def = arg; + *def != '\0' && *def != '='; ++def) + continue; + if (*def == '\0') + def = "1"; + + snprintf(buf2, sizeof(buf2), "M4_HOOK_CONST_DEFINE_UNKNOWN(%s, %s)", arg, def); + buf_strappend (&userdef_buf, buf2); + } + break; - case OPT_READ: - use_read = true; + case OPT_READ: + ctrl.use_read = true; break; - case OPT_STACK: - //buf_strdefine (&userdef_buf, "YY_STACK_USED", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_STACK_USED",0); + case OPT_STACK: + ctrl.stack_used = true; break; - case OPT_STDINIT: - do_stdinit = true; + case OPT_STDINIT: + ctrl.do_stdinit = true; break; - case OPT_NO_STDINIT: - do_stdinit = false; + case OPT_NO_STDINIT: + ctrl.do_stdinit = false; break; - case OPT_YYCLASS: - yyclass = arg; + case OPT_YYCLASS: + ctrl.yyclass = arg; break; - case OPT_YYLINENO: - do_yylineno = true; + case OPT_YYLINENO: + ctrl.do_yylineno = true; break; - case OPT_NO_YYLINENO: - do_yylineno = false; + case OPT_NO_YYLINENO: + ctrl.do_yylineno = false; break; - case OPT_YYWRAP: - do_yywrap = true; + case OPT_YYWRAP: + ctrl.do_yywrap = true; break; - case OPT_NO_YYWRAP: - do_yywrap = false; + case OPT_NO_YYWRAP: + ctrl.do_yywrap = false; break; - case OPT_YYMORE: - yymore_really_used = true; + case OPT_YYMORE: + ctrl.yymore_really_used = true; break; - case OPT_NO_YYMORE: - yymore_really_used = false; + case OPT_NO_YYMORE: + ctrl.yymore_really_used = false; break; - case OPT_REJECT: - reject_really_used = true; + case OPT_REJECT: + ctrl.reject_really_used = true; break; - case OPT_NO_REJECT: - reject_really_used = false; + case OPT_NO_REJECT: + ctrl.reject_really_used = false; break; - case OPT_NO_YY_PUSH_STATE: - //buf_strdefine (&userdef_buf, "YY_NO_PUSH_STATE", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_PUSH_STATE",0); + case OPT_NO_YY_PUSH_STATE: + ctrl.no_yy_push_state = true; break; - case OPT_NO_YY_POP_STATE: - //buf_strdefine (&userdef_buf, "YY_NO_POP_STATE", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_POP_STATE",0); + case OPT_NO_YY_POP_STATE: + ctrl.no_yy_pop_state = true; break; - case OPT_NO_YY_TOP_STATE: - //buf_strdefine (&userdef_buf, "YY_NO_TOP_STATE", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_TOP_STATE",0); + case OPT_NO_YY_TOP_STATE: + ctrl.no_yy_top_state = true; break; - case OPT_NO_UNPUT: - //buf_strdefine (&userdef_buf, "YY_NO_UNPUT", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_UNPUT",0); + case OPT_NO_YYUNPUT: + ctrl.no_yyunput = true; break; - case OPT_NO_YY_SCAN_BUFFER: - //buf_strdefine (&userdef_buf, "YY_NO_SCAN_BUFFER", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SCAN_BUFFER",0); + case OPT_NO_YY_SCAN_BUFFER: + ctrl.no_yy_scan_buffer = true; break; - case OPT_NO_YY_SCAN_BYTES: - //buf_strdefine (&userdef_buf, "YY_NO_SCAN_BYTES", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SCAN_BYTES",0); + case OPT_NO_YY_SCAN_BYTES: + ctrl.no_yy_scan_bytes = true; break; - case OPT_NO_YY_SCAN_STRING: - //buf_strdefine (&userdef_buf, "YY_NO_SCAN_STRING", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SCAN_STRING",0); + case OPT_NO_YY_SCAN_STRING: + ctrl.no_yy_scan_string = true; break; - case OPT_NO_YYGET_EXTRA: - //buf_strdefine (&userdef_buf, "YY_NO_GET_EXTRA", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_EXTRA",0); + case OPT_NO_YYGET_EXTRA: + ctrl.no_yyget_extra = true; break; - case OPT_NO_YYSET_EXTRA: - //buf_strdefine (&userdef_buf, "YY_NO_SET_EXTRA", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SET_EXTRA",0); + case OPT_NO_YYSET_EXTRA: + ctrl.no_yyset_extra = true; break; - case OPT_NO_YYGET_LENG: - //buf_strdefine (&userdef_buf, "YY_NO_GET_LENG", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_LENG",0); + case OPT_NO_YYGET_LENG: + ctrl.no_yyget_leng = true; break; - case OPT_NO_YYGET_TEXT: - //buf_strdefine (&userdef_buf, "YY_NO_GET_TEXT", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_TEXT",0); + case OPT_NO_YYGET_TEXT: + ctrl.no_yyget_text = true; break; - case OPT_NO_YYGET_LINENO: - //buf_strdefine (&userdef_buf, "YY_NO_GET_LINENO", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_LINENO",0); + case OPT_NO_YYGET_LINENO: + ctrl.no_yyget_lineno = true; break; - case OPT_NO_YYSET_LINENO: - //buf_strdefine (&userdef_buf, "YY_NO_SET_LINENO", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SET_LINENO",0); + case OPT_NO_YYSET_LINENO: + ctrl.no_yyset_lineno = true; break; - case OPT_NO_YYGET_COLUMN: - //buf_strdefine (&userdef_buf, "YY_NO_GET_COLUMN", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_COLUMN",0); + case OPT_NO_YYGET_COLUMN: + ctrl.no_yyget_column = true; break; - case OPT_NO_YYSET_COLUMN: - //buf_strdefine (&userdef_buf, "YY_NO_SET_COLUMN", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SET_COLUMN",0); + case OPT_NO_YYSET_COLUMN: + ctrl.no_yyset_column = true; break; - case OPT_NO_YYGET_IN: - //buf_strdefine (&userdef_buf, "YY_NO_GET_IN", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_IN",0); + case OPT_NO_YYGET_IN: + ctrl.no_yyget_in = true; break; - case OPT_NO_YYSET_IN: - //buf_strdefine (&userdef_buf, "YY_NO_SET_IN", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SET_IN",0); + case OPT_NO_YYSET_IN: + ctrl.no_yyset_in = true; break; - case OPT_NO_YYGET_OUT: - //buf_strdefine (&userdef_buf, "YY_NO_GET_OUT", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_OUT",0); + case OPT_NO_YYGET_OUT: + ctrl.no_yyget_out = true; break; - case OPT_NO_YYSET_OUT: - //buf_strdefine (&userdef_buf, "YY_NO_SET_OUT", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SET_OUT",0); + case OPT_NO_YYSET_OUT: + ctrl.no_yyset_out = true; break; - case OPT_NO_YYGET_LVAL: - //buf_strdefine (&userdef_buf, "YY_NO_GET_LVAL", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_LVAL",0); + case OPT_NO_YYGET_LVAL: + ctrl.no_yyget_lval = true; break; - case OPT_NO_YYSET_LVAL: - //buf_strdefine (&userdef_buf, "YY_NO_SET_LVAL", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SET_LVAL",0); + case OPT_NO_YYSET_LVAL: + ctrl.no_yyset_lval = true; break; - case OPT_NO_YYGET_LLOC: - //buf_strdefine (&userdef_buf, "YY_NO_GET_LLOC", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_GET_LLOC",0); + case OPT_NO_YYGET_LLOC: + ctrl.no_yyget_lloc = true; break; - case OPT_NO_YYSET_LLOC: - //buf_strdefine (&userdef_buf, "YY_NO_SET_LLOC", "1"); - buf_m4_define( &m4defs_buf, "M4_YY_NO_SET_LLOC",0); + case OPT_NO_YYSET_LLOC: + ctrl.no_yyset_lloc = true; break; - case OPT_HEX: - trace_hex = 1; + + case OPT_HEX: + env.trace_hex = true; break; - case OPT_NO_SECT3_ESCAPE: - no_section3_escape = true; + case OPT_NO_SECT3_ESCAPE: + ctrl.no_section3_escape = true; break; } /* switch */ } /* while scanopt() */ @@ -1460,7 +1193,7 @@ void flexinit (int argc, char **argv) num_rules = num_eof_rules = default_rule = 0; numas = numsnpairs = tmpuses = 0; numecs = numeps = eps2 = num_reallocs = hshcol = dfaeql = totnst = - 0; + 0; numuniq = numdup = hshsave = eofseen = datapos = dataline = 0; num_backing_up = onesp = numprots = 0; variable_trailing_context_rules = bol_needed = false; @@ -1481,11 +1214,9 @@ void flexinit (int argc, char **argv) void readin (void) { - static char yy_stdinit[] = "FILE *yyin = stdin, *yyout = stdout;"; - static char yy_nostdinit[] = - "FILE *yyin = NULL, *yyout = NULL;"; + char buf[256]; - line_directive_out(NULL, 1); + line_directive_out(NULL, infilename, linenum); if (yyparse ()) { pinpoint_message (_("fatal parse error")); @@ -1495,6 +1226,101 @@ void readin (void) if (syntaxerror) flexend (1); + /* On --emit, -e, or change backends This is where backend + * properties are collected, which means they can't be set + * from a custom skelfile. Note: might have been called sooner + * when %option emit was evaluated; this catches command-line + * optiins and the default case. + */ + backend_by_name(ctrl.emit); + + initialize_output_filters(); + + yyout = stdout; + + if (tablesext) + gentables = false; + + if (tablesverify) + /* force generation of C tables. */ + gentables = true; + + + if (tablesext) { + FILE *tablesout; + struct yytbl_hdr hdr; + char *pname = 0; + size_t nbytes = 0; + + if (!tablesfilename) { + nbytes = strlen (ctrl.prefix) + strlen (tablesfile_template) + 2; + tablesfilename = pname = calloc(nbytes, 1); + snprintf (pname, nbytes, tablesfile_template, ctrl.prefix); + } + + if ((tablesout = fopen (tablesfilename, "w")) == NULL) + lerr (_("could not create %s"), tablesfilename); + free(pname); + tablesfilename = 0; + + yytbl_writer_init (&tableswr, tablesout); + + nbytes = strlen (ctrl.prefix) + strlen ("tables") + 2; + tablesname = calloc(nbytes, 1); + snprintf (tablesname, nbytes, "%stables", ctrl.prefix); + yytbl_hdr_init (&hdr, flex_version, tablesname); + + if (yytbl_hdr_fwrite (&tableswr, &hdr) <= 0) + flexerror (_("could not write tables header")); + } + + if (env.skelname && (skelfile = fopen (env.skelname, "r")) == NULL) + lerr (_("can't open skeleton file %s"), env.skelname); + + if (strchr(ctrl.prefix, '[') || strchr(ctrl.prefix, ']')) + flexerror(_("Prefix cannot include '[' or ']'")); + + if (env.did_outfilename) + line_directive_out (stdout, NULL, linenum); + + /* This is where we begin writing to the file. */ + + skelout(false); /* [0.0] Make hook macros available, silently */ + + comment("A lexical scanner generated by flex\n"); + + /* Dump the %top code. */ + if( top_buf.elts) + outn((char*) top_buf.elts); + + /* Place a bogus line directive, it will be fixed in the filter. */ + line_directive_out(NULL, NULL, 0); + + /* User may want to set the scanner prototype */ + if (ctrl.yydecl != NULL) { + out_str ("M4_HOOK_SET_YY_DECL(%s)\n", ctrl.yydecl); + } + + if (ctrl.userinit != NULL) { + out_str ("M4_HOOK_SET_USERINIT(%s)\n", ctrl.userinit); + } + if (ctrl.preaction != NULL) { + out_str ("M4_HOOK_SET_PREACTION(%s)\n", ctrl.preaction); + } + if (ctrl.postaction != NULL) { + out_str ("M4_HOOK_SET_POSTACTION(%s)\n", ctrl.postaction); + } + + /* This has to be a stright textual substitution rather + * than a constant declaration because in C a const is + * not const enough to be a static array bound. + */ + out_dec ("m4_define([[YYLMAX]], [[%d]])\n", ctrl.yylmax); + + /* Dump the user defined preproc directives. */ + if (userdef_buf.elts) + outn ((char *) (userdef_buf.elts)); + /* If the user explicitly requested posix compatibility by specifing the * posix-compat option, then we check for conflicting options. However, if * the POSIXLY_CORRECT variable is set, then we quietly make flex as @@ -1504,41 +1330,41 @@ void readin (void) * Note: The posix option was added to flex to provide the posix behavior * of the repeat operator in regular expressions, e.g., `ab{3}' */ - if (posix_compat) { + if (ctrl.posix_compat) { /* TODO: This is where we try to make flex behave according to - * posiz, AND check for conflicting options. How far should we go + * POSIX, *and* check for conflicting ctrl. How far should we go * with this? Should we disable all the neat-o flex features? */ /* Update: Estes says no, since other flex features don't violate posix. */ } if (getenv ("POSIXLY_CORRECT")) { - posix_compat = true; + ctrl.posix_compat = true; } - if (backing_up_report) { - backing_up_file = fopen (backing_name, "w"); - if (backing_up_file == NULL) + if (env.backing_up_report) { + ctrl.backing_up_file = fopen (backing_name, "w"); + if (ctrl.backing_up_file == NULL) lerr (_ ("could not create backing-up info file %s"), backing_name); } else - backing_up_file = NULL; + ctrl.backing_up_file = NULL; - if (yymore_really_used == true) + if (ctrl.yymore_really_used == true) yymore_used = true; - else if (yymore_really_used == false) + else if (ctrl.yymore_really_used == false) yymore_used = false; - if (reject_really_used == true) + if (ctrl.reject_really_used == true) reject = true; - else if (reject_really_used == false) + else if (ctrl.reject_really_used == false) reject = false; - if (performance_report > 0) { - if (lex_compat) { + if (env.performance_hint > 0) { + if (ctrl.lex_compat) { fprintf (stderr, _ ("-l AT&T lex compatibility option entails a large performance penalty\n")); @@ -1547,14 +1373,14 @@ void readin (void) (" and may be the actual source of other reported performance penalties\n")); } - else if (do_yylineno) { + else if (ctrl.do_yylineno) { fprintf (stderr, _ ("%%option yylineno entails a performance penalty ONLY on rules that can match newline characters\n")); } - if (performance_report > 1) { - if (interactive) + if (env.performance_hint > 1) { + if (ctrl.interactive == trit_true) fprintf (stderr, _ ("-I (interactive) entails a minor performance penalty\n")); @@ -1582,11 +1408,11 @@ void readin (void) if (variable_trailing_context_rules) reject = true; - if ((fulltbl || fullspd) && reject) { + if ((ctrl.fulltbl || ctrl.fullspd) && reject) { if (real_reject) flexerror (_ ("REJECT cannot be used with -f or -F")); - else if (do_yylineno) + else if (ctrl.do_yylineno) flexerror (_ ("%option yylineno cannot be used with REJECT")); else @@ -1594,150 +1420,290 @@ void readin (void) ("variable trailing context rules cannot be used with -f or -F")); } - if (reject){ - out_m4_define( "M4_YY_USES_REJECT", NULL); - //outn ("\n#define YY_USES_REJECT"); - } - - if (!do_yywrap) { - if (!C_plus_plus) { - if (reentrant) - out_str ("\n#define %swrap(yyscanner) (/*CONSTCOND*/1)\n", prefix); - else - out_str ("\n#define %swrap() (/*CONSTCOND*/1)\n", prefix); + if (ctrl.useecs) + numecs = cre8ecs (nextecm, ecgroup, ctrl.csize); + else + numecs = ctrl.csize; + + /* Now map the equivalence class for NUL to its expected place. */ + ecgroup[0] = ecgroup[ctrl.csize]; + NUL_ec = ABS (ecgroup[0]); + + if (ctrl.useecs) + ccl2ecl (); + + // These are used to conditionalize code in the lex skeleton + // that historically used to be generated by C code in flex + // itself; by shoving all this stuff out to the skeleton file + // we make it easier to retarget the code generation. + snprintf(buf, sizeof(buf), "Target: %s\n", ctrl.backend_name); + comment(buf); + comment("START of m4 controls\n"); + + /* Define the start condition macros. */ + { + struct Buf tmpbuf; + int i; + buf_init(&tmpbuf, sizeof(char)); + for (i = 1; i <= lastsc; i++) { + char *str, *fmt = "M4_HOOK_CONST_DEFINE_STATE(%s, %d)"; + size_t strsz; + + strsz = strlen(fmt) + strlen(scname[i]) + (size_t)(1 + ceil (log10(i))) + 2; + str = malloc(strsz); + if (!str) + flexfatal(_("allocation of macro definition failed")); + snprintf(str, strsz, fmt, scname[i], i - 1); + buf_strappend(&tmpbuf, str); + free(str); } - outn ("#define YY_SKIP_YYWRAP"); + // FIXME: Not dumped visibly because we plan to do away with the indirection + out_m4_define("M4_YY_SC_DEFS", tmpbuf.elts); + buf_destroy(&tmpbuf); } - if (ddebug) - outn ("\n#define FLEX_DEBUG"); + if (ctrl.bison_bridge_lval) + visible_define("M4_YY_BISON_LVAL"); - OUT_BEGIN_CODE (); - outn ("typedef flex_uint8_t YY_CHAR;"); - OUT_END_CODE (); + if (ctrl.bison_bridge_lloc) + visible_define("<M4_YY_BISON_LLOC>"); - if (C_plus_plus) { - outn ("#define yytext_ptr yytext"); + if (extra_type != NULL) + visible_define_str ("M4_MODE_EXTRA_TYPE", extra_type); - if (interactive) - outn ("#define YY_INTERACTIVE"); + /* always generate the tablesverify flag. */ + visible_define_str ("M4_YY_TABLES_VERIFY", tablesverify ? "1" : "0"); + + if (ctrl.reentrant) { + visible_define ("M4_YY_REENTRANT"); + if (ctrl.yytext_is_array) + visible_define ("M4_MODE_REENTRANT_TEXT_IS_ARRAY"); } - else { - OUT_BEGIN_CODE (); - /* In reentrant scanner, stdinit is handled in flex.skl. */ - if (do_stdinit) { - if (reentrant){ - outn ("#ifdef VMS"); - outn ("#ifdef __VMS_POSIX"); - outn ("#define YY_STDINIT"); - outn ("#endif"); - outn ("#else"); - outn ("#define YY_STDINIT"); - outn ("#endif"); - } - - outn ("#ifdef VMS"); - outn ("#ifndef __VMS_POSIX"); - outn (yy_nostdinit); - outn ("#else"); - outn (yy_stdinit); - outn ("#endif"); - outn ("#else"); - outn (yy_stdinit); - outn ("#endif"); - } + if (ctrl.do_main == trit_true) + visible_define_str ( "YY_MAIN", "1"); + else if (ctrl.do_main == trit_false) + visible_define_str ( "YY_MAIN", "0"); - else { - if(!reentrant) - outn (yy_nostdinit); - } - OUT_END_CODE (); - } + if (ctrl.do_stdinit) + visible_define ( "M4_MODE_DO_STDINIT"); + else + visible_define ( "M4_MODE_NO_DO_STDINIT"); - OUT_BEGIN_CODE (); - if (fullspd) - outn ("typedef const struct yy_trans_info *yy_state_type;"); - else if (!C_plus_plus) - outn ("typedef int yy_state_type;"); - OUT_END_CODE (); + // mode switches for YY_DO_BEFORE_ACTION code generation + if (ctrl.yytext_is_array) + visible_define ( "M4_MODE_YYTEXT_IS_ARRAY"); + else + visible_define ( "M4_MODE_NO_YYTEXT_IS_ARRAY"); + if (yymore_used) + visible_define ( "M4_MODE_YYMORE_USED"); + else + visible_define ( "M4_MODE_NO_YYMORE_USED"); - if (lex_compat) - outn ("#define YY_FLEX_LEX_COMPAT"); + if (ctrl.fullspd) + visible_define ( "M4_MODE_REAL_FULLSPD"); + else + visible_define ( "M4_MODE_NO_REAL_FULLSPD"); - if (!C_plus_plus && !reentrant) { - outn ("extern int yylineno;"); - OUT_BEGIN_CODE (); - outn ("int yylineno = 1;"); - OUT_END_CODE (); + if (ctrl.fulltbl) + visible_define ( "M4_MODE_REAL_FULLTBL"); + else + visible_define ( "M4_MODE_NO_REAL_FULLTBL"); + + // niode switches for YYINPUT code generation + if (ctrl.use_read) + visible_define ( "M4_MODE_CPP_USE_READ"); + else + visible_define ( "M4_MODE_NO_CPP_USE_READ"); + + // mode switches for next-action code + if (variable_trailing_context_rules) { + visible_define ( "M4_MODE_VARIABLE_TRAILING_CONTEXT_RULES"); + } else { + visible_define ( "M4_MODE_NO_VARIABLE_TRAILING_CONTEXT_RULES"); } + if (real_reject) + visible_define ( "M4_MODE_REAL_REJECT"); + if (ctrl.reject_really_used) + visible_define ( "M4_MODE_FIND_ACTION_REJECT_REALLY_USED"); + if (reject) + visible_define ( "M4_MODE_USES_REJECT"); + else + visible_define ( "M4_MODE_NO_USES_REJECT"); + + // mode switches for computing next compressed state + if (ctrl.usemecs) + visible_define ( "M4_MODE_USEMECS"); + + // mode switches for find-action code + if (ctrl.fullspd) + visible_define ( "M4_MODE_FULLSPD"); + else if (ctrl.fulltbl) + visible_define ( "M4_MODE_FIND_ACTION_FULLTBL"); + else if (reject) + visible_define ( "M4_MODE_FIND_ACTION_REJECT"); + else + visible_define ( "M4_MODE_FIND_ACTION_COMPRESSED"); - if (C_plus_plus) { - outn ("\n#include <FlexLexer.h>"); + // mode switches for backup generation and gen_start_state + if (!ctrl.fullspd) + visible_define ( "M4_MODE_NO_FULLSPD"); + if (bol_needed) + visible_define ( "M4_MODE_BOL_NEEDED"); + else + visible_define ( "M4_MODE_NO_BOL_NEEDED"); - if (!do_yywrap) { - outn("\nint yyFlexLexer::yywrap() { return 1; }"); - } + // yylineno + if (ctrl.do_yylineno) + visible_define ( "M4_MODE_YYLINENO"); - if (yyclass) { - outn ("int yyFlexLexer::yylex()"); - outn ("\t{"); - outn ("\tLexerError( \"yyFlexLexer::yylex invoked but %option yyclass used\" );"); - outn ("\treturn 0;"); - outn ("\t}"); + // Equivalence classes + if (ctrl.useecs) + visible_define ( "M4_MODE_USEECS"); + else + visible_define ( "M4_MODE_NO_USEECS"); - out_str ("\n#define YY_DECL int %s::yylex()\n", - yyclass); - } + // mode switches for getting next action + if (gentables) + visible_define ( "M4_MODE_GENTABLES"); + else + visible_define ( "M4_MODE_NO_GENTABLES"); + if (ctrl.interactive == trit_true) + visible_define ( "M4_MODE_INTERACTIVE"); + else + visible_define ( "M4_MODE_NO_INTERACTIVE"); + if (!(ctrl.fullspd || ctrl.fulltbl)) + visible_define ( "M4_MODE_NO_FULLSPD_OR_FULLTBL"); + if (reject || ctrl.interactive == trit_true) + visible_define ( "M4_MODE_FIND_ACTION_REJECT_OR_INTERACTIVE"); + + if (ctrl.yyclass != NULL) { + visible_define ( "M4_MODE_YYCLASS"); + out_m4_define("M4_YY_CLASS_NAME", ctrl.yyclass); } - else { + if (ctrl.ddebug) + visible_define ( "M4_MODE_DEBUG"); - /* Watch out: yytext_ptr is a variable when yytext is an array, - * but it's a macro when yytext is a pointer. - */ - if (yytext_is_array) { - if (!reentrant) - outn ("extern char yytext[];\n"); - } - else { - if (reentrant) { - outn ("#define yytext_ptr yytext_r"); - } - else { - outn ("extern char *yytext;"); + if (ctrl.lex_compat) + visible_define ( "M4_MODE_OPTIONS.LEX_COMPAT"); - outn("#ifdef yytext_ptr"); - outn("#undef yytext_ptr"); - outn("#endif"); - outn ("#define yytext_ptr yytext"); - } - } + if (ctrl.do_yywrap) + visible_define ( "M4_MODE_YYWRAP"); + else + visible_define ( "M4_MODE_NO_YYWRAP"); - if (yyclass) - flexerror (_ - ("%option yyclass only meaningful for C++ scanners")); + if (ctrl.interactive == trit_true) + visible_define ( "M4_MODE_INTERACTIVE"); + + if (ctrl.noyyread) + visible_define("M4_MODE_USER_YYREAD"); + + if (is_default_backend()) { + if (ctrl.C_plus_plus) { + visible_define ( "M4_MODE_CXX_ONLY"); + } else { + visible_define ( "M4_MODE_C_ONLY"); + } } - if (useecs) - numecs = cre8ecs (nextecm, ecgroup, csize); + if (tablesext) + visible_define ( "M4_MODE_TABLESEXT"); + if (ctrl.prefix != NULL) + visible_define_str ( "M4_MODE_PREFIX", ctrl.prefix); + + if (ctrl.no_yyinput) + visible_define("M4_MODE_NO_YYINPUT"); + + if (ctrl.bufsize != 0) + visible_define_int("M4_MODE_YY_BUFSIZE", ctrl.bufsize); + + if (ctrl.yyterminate != NULL) + visible_define_str("M4_MODE_YYTERMINATE", ctrl.yyterminate); + + if (ctrl.no_yypanic) + visible_define("M4_YY_NO_YYPANIC"); + if (ctrl.no_yy_push_state) + visible_define("M4_YY_NO_PUSH_STATE"); + if (ctrl.no_yy_pop_state) + visible_define("M4_YY_NO_POP_STATE"); + if (ctrl.no_yy_top_state) + visible_define("M4_YY_NO_TOP_STATE"); + if (ctrl.no_yyunput) + visible_define("M4_YY_NO_YYUNPUT"); + if (ctrl.no_yy_scan_buffer) + visible_define("M4_YY_NO_SCAN_BUFFER"); + if (ctrl.no_yy_scan_bytes) + visible_define("M4_YY_NO_SCAN_BYTES"); + if (ctrl.no_yy_scan_string) + visible_define("M4_YY_NO_SCAN_STRING"); + if (ctrl.no_yyget_extra) + visible_define("M4_YY_NO_GET_EXTRA"); + if (ctrl.no_yyset_extra) + visible_define("M4_YY_NO_SET_EXTRA"); + if (ctrl.no_yyget_leng) + visible_define("M4_YY_NO_GET_LENG"); + if (ctrl.no_yyget_text) + visible_define("M4_YY_NO_GET_TEXT"); + if (ctrl.no_yyget_lineno) + visible_define("M4_YY_NO_GET_LINENO"); + if (ctrl.no_yyset_lineno) + visible_define("M4_YY_NO_SET_LINENO"); + if (ctrl.no_yyget_column) + visible_define("M4_YY_NO_GET_COLUMN"); + if (ctrl.no_yyset_column) + visible_define("M4_YY_NO_SET_COLUMN"); + if (ctrl.no_yyget_in) + visible_define("M4_YY_NO_GET_IN"); + if (ctrl.no_yyset_in) + visible_define("M4_YY_NO_SET_IN"); + if (ctrl.no_yyget_out) + visible_define("M4_YY_NO_GET_OUT"); + if (ctrl.no_yyset_out) + visible_define("M4_YY_NO_SET_OUT"); + if (ctrl.no_yyget_lval) + visible_define("M4_YY_NO_GET_LVAL"); + if (ctrl.no_yyset_lval) + visible_define("M4_YY_NO_SET_LVAL"); + if (ctrl.no_yyget_lloc) + visible_define("M4_YY_NO_GET_LLOC"); + if (ctrl.no_yyset_lloc) + visible_define("M4_YY_NO_SET_LLOC"); + if (ctrl.no_flex_alloc) + visible_define("M4_YY_NO_FLEX_ALLOC"); + if (ctrl.no_flex_realloc) + visible_define("M4_YY_NO_FLEX_REALLOC"); + if (ctrl.no_flex_free) + visible_define("M4_YY_NO_FLEX_FREE"); + if (ctrl.no_get_debug) + visible_define("M4_YY_NO_GET_DEBUG"); + if (ctrl.no_set_debug) + visible_define("M4_YY_NO_SET_DEBUG"); + + if (ctrl.no_unistd) + visible_define("M4_YY_NO_UNISTD_H"); + + if (ctrl.always_interactive) + visible_define("M4_YY_ALWAYS_INTERACTIVE"); + if (ctrl.never_interactive) + visible_define("M4_YY_NEVER_INTERACTIVE"); + if (ctrl.stack_used) + visible_define("M4_YY_STACK_USED"); + + if (ctrl.rewrite) + visible_define ( "M4_MODE_REWRITE"); else - numecs = csize; + visible_define ( "M4_MODE_NO_REWRITE"); - /* Now map the equivalence class for NUL to its expected place. */ - ecgroup[0] = ecgroup[csize]; - NUL_ec = ABS (ecgroup[0]); - - if (useecs) - ccl2ecl (); + comment("END of m4 controls\n"); + out ("\n"); } - /* set_up_initial_allocations - allocate memory for internal tables */ void set_up_initial_allocations (void) { - maximum_mns = (long_align ? MAXIMUM_MNS_LONG : MAXIMUM_MNS); + maximum_mns = (ctrl.long_align ? MAXIMUM_MNS_LONG : MAXIMUM_MNS); current_mns = INITIAL_MNS; firstst = allocate_integer_array (current_mns); lastst = allocate_integer_array (current_mns); @@ -1797,10 +1763,10 @@ void usage (void) { FILE *f = stdout; - if (!did_outfilename) { + if (!env.did_outfilename) { snprintf (outfile_path, sizeof(outfile_path), outfile_template, - prefix, C_plus_plus ? "cc" : "c"); - outfilename = outfile_path; + ctrl.prefix, suffix()); + env.outfilename = outfile_path; } fprintf (f, _("Usage: %s [OPTIONS] [FILE]...\n"), program_name); @@ -1822,7 +1788,7 @@ void usage (void) " -b, --backup write backing-up information to %s\n" " -p, --perf-report write performance report to stderr\n" " -s, --nodefault suppress default rule to ECHO unmatched text\n" - " -T, --trace %s should run in trace mode\n" + " -T, --env.trace %s should run in env.trace mode\n" " -w, --nowarn do not generate warnings\n" " -v, --verbose write summary of scanner statistics to stdout\n" " --hex use hexadecimal numbers instead of octal in debug outputs\n" @@ -1846,10 +1812,11 @@ void usage (void) "\n" "Generated code:\n" " -+, --c++ generate C++ scanner class\n" " -Dmacro[=defn] #define macro defn (default defn is '1')\n" + " -e, --emit=LANG Specify target language\n" " -L, --noline suppress #line directives in scanner\n" " -P, --prefix=STRING use STRING as prefix instead of \"yy\"\n" - " -R, --reentrant generate a reentrant C scanner\n" - " --bison-bridge scanner for bison pure parser.\n" + " -R, --reentrant generate a reentrant scanner\n" + " --bison-bridge scanner for Bison pure parser.\n" " --bison-locations include yylloc support.\n" " --stdinit initialize yyin/yyout to stdin/stdout\n" " --nounistd do not include <unistd.h>\n" @@ -33,83 +33,6 @@ #include "flexdef.h" #include "tables.h" -#define CMD_IF_TABLES_SER "%if-tables-serialization" -#define CMD_TABLES_YYDMAP "%tables-yydmap" -#define CMD_DEFINE_YYTABLES "%define-yytables" -#define CMD_IF_CPP_ONLY "%if-c++-only" -#define CMD_IF_C_ONLY "%if-c-only" -#define CMD_IF_C_OR_CPP "%if-c-or-c++" -#define CMD_NOT_FOR_HEADER "%not-for-header" -#define CMD_OK_FOR_HEADER "%ok-for-header" -#define CMD_PUSH "%push" -#define CMD_POP "%pop" -#define CMD_IF_REENTRANT "%if-reentrant" -#define CMD_IF_NOT_REENTRANT "%if-not-reentrant" -#define CMD_IF_BISON_BRIDGE "%if-bison-bridge" -#define CMD_IF_NOT_BISON_BRIDGE "%if-not-bison-bridge" -#define CMD_ENDIF "%endif" - -/* we allow the skeleton to push and pop. */ -struct sko_state { - bool dc; /**< do_copy */ -}; -static struct sko_state *sko_stack=0; -static int sko_len=0,sko_sz=0; -static void sko_push(bool dc) -{ - if(!sko_stack){ - sko_sz = 1; - sko_stack = malloc(sizeof(struct sko_state) * (size_t) sko_sz); - if (!sko_stack) - flexfatal(_("allocation of sko_stack failed")); - sko_len = 0; - } - if(sko_len >= sko_sz){ - sko_sz *= 2; - sko_stack = realloc(sko_stack, - sizeof(struct sko_state) * (size_t) sko_sz); - } - - /* initialize to zero and push */ - sko_stack[sko_len].dc = dc; - sko_len++; -} -static void sko_peek(bool *dc) -{ - if(sko_len <= 0) - flex_die("peek attempt when sko stack is empty"); - if(dc) - *dc = sko_stack[sko_len-1].dc; -} -static void sko_pop(bool* dc) -{ - sko_peek(dc); - sko_len--; - if(sko_len < 0) - flex_die("popped too many times in skeleton."); -} - -/* Append "#define defname value\n" to the running buffer. */ -void action_define (const char *defname, int value) -{ - char buf[MAXLINE]; - char *cpy; - - if ((int) strlen (defname) > MAXLINE / 2) { - format_pinpoint_message (_ - ("name \"%s\" ridiculously long"), - defname); - return; - } - - snprintf (buf, sizeof(buf), "#define %s %d\n", defname, value); - add_action (buf); - - /* track #defines so we can undef them when we're done. */ - cpy = xstrdup(defname); - buf_append (&defs_buf, &cpy, 1); -} - /* Append "new_text" to the running buffer. */ void add_action (const char *new_text) { @@ -208,7 +131,7 @@ void check_char (int c) lerr (_("bad character '%s' detected in check_char()"), readable_form (c)); - if (c >= csize) + if (c >= ctrl.csize) lerr (_ ("scanner requires -8 flag to use the character %s"), readable_form (c)); @@ -239,19 +162,19 @@ char *xstrdup(const char *s) int cclcmp (const void *a, const void *b) { - if (!*(const unsigned char *) a) - return 1; - else - if (!*(const unsigned char *) b) - return - 1; + if (!*(const unsigned char *) a) + return 1; else - return *(const unsigned char *) a - *(const unsigned char *) b; + if (!*(const unsigned char *) b) + return - 1; + else + return *(const unsigned char *) a - *(const unsigned char *) b; } /* dataend - finish up a block of data declarations */ -void dataend (void) +void dataend (const char *endit) { /* short circuit any output */ if (gentables) { @@ -260,7 +183,8 @@ void dataend (void) dataflush (); /* add terminator for initialization; { for vi */ - outn (" } ;\n"); + if (endit) + outn (endit); } dataline = 0; datapos = 0; @@ -275,7 +199,8 @@ void dataflush (void) if (!gentables) return; - outc ('\n'); + if (datapos > 0) + outc ('\n'); if (++dataline >= NUMDATALINES) { /* Put out a blank line so that the table is grouped into @@ -337,21 +262,20 @@ void lerr_fatal (const char *msg, ...) } -/* line_directive_out - spit out a "#line" statement */ - -void line_directive_out (FILE *output_file, int do_infile) +/* line_directive_out - spit out a "#line" statement or equivalent */ +void line_directive_out (FILE *output_file, char *path, int linenum) { + char *trace_fmt = "m4_ifdef([[M4_HOOK_TRACE_LINE_FORMAT]], [[M4_HOOK_TRACE_LINE_FORMAT([[%d]], [[%s]])]])"; char directive[MAXLINE*2], filename[MAXLINE]; char *s1, *s2, *s3; - static const char line_fmt[] = "#line %d \"%s\"\n"; - if (!gen_line_dirs) + if (!ctrl.gen_line_dirs) return; - s1 = do_infile ? infilename : "M4_YY_OUTFILE_NAME"; + s1 = (path != NULL) ? path : "M4_YY_OUTFILE_NAME"; - if (do_infile && !s1) - s1 = "<stdin>"; + if ((path != NULL) && !s1) + s1 = "<stdin>"; s2 = filename; s3 = &filename[sizeof (filename) - 2]; @@ -366,10 +290,10 @@ void line_directive_out (FILE *output_file, int do_infile) *s2 = '\0'; - if (do_infile) - snprintf (directive, sizeof(directive), line_fmt, linenum, filename); + if (path != NULL) + snprintf (directive, sizeof(directive), trace_fmt, linenum, filename); else { - snprintf (directive, sizeof(directive), line_fmt, 0, filename); + snprintf (directive, sizeof(directive), trace_fmt, 0, filename); } /* If output_file is nil then we should put the directive in @@ -453,7 +377,7 @@ void mkdata (int value) if (datapos == 0) /* Indent. */ - out (" "); + out (" "); else outc (','); @@ -579,11 +503,6 @@ void out_str (const char *fmt, const char str[]) fprintf (stdout,fmt, str); } -void out_str3 (const char *fmt, const char s1[], const char s2[], const char s3[]) -{ - fprintf (stdout,fmt, s1, s2, s3); -} - void out_str_dec (const char *fmt, const char str[], int n) { fprintf (stdout,fmt, str, n); @@ -637,7 +556,7 @@ char *readable_form (int c) case '\v': return "\\v"; default: - if(trace_hex) + if(env.trace_hex) snprintf (rform, sizeof(rform), "\\x%.2x", (unsigned int) c); else snprintf (rform, sizeof(rform), "\\%.3o", (unsigned int) c); @@ -682,138 +601,6 @@ void *reallocate_array (void *array, int size, size_t element_size) } -/* skelout - write out one section of the skeleton file - * - * Description - * Copies skelfile or skel array to stdout until a line beginning with - * "%%" or EOF is found. - */ -void skelout (void) -{ - char buf_storage[MAXLINE]; - char *buf = buf_storage; - bool do_copy = true; - - /* "reset" the state by clearing the buffer and pushing a '1' */ - if(sko_len > 0) - sko_peek(&do_copy); - sko_len = 0; - sko_push(do_copy=true); - - - /* Loop pulling lines either from the skelfile, if we're using - * one, or from the skel[] array. - */ - while (skelfile ? - (fgets (buf, MAXLINE, skelfile) != NULL) : - ((buf = (char *) skel[skel_ind++]) != 0)) { - - if (skelfile) - chomp (buf); - - /* copy from skel array */ - if (buf[0] == '%') { /* control line */ - /* print the control line as a comment. */ - if (ddebug && buf[1] != '#') { - if (buf[strlen (buf) - 1] == '\\') - out_str ("/* %s */\\\n", buf); - else - out_str ("/* %s */\n", buf); - } - - /* We've been accused of using cryptic markers in the skel. - * So we'll use emacs-style-hyphenated-commands. - * We might consider a hash if this if-else-if-else - * chain gets too large. - */ -#define cmd_match(s) (strncmp(buf,(s),strlen(s))==0) - - if (buf[1] == '#') { - /* %# indicates comment line to be ignored */ - } - else if (buf[1] == '%') { - /* %% is a break point for skelout() */ - return; - } - else if (cmd_match (CMD_PUSH)){ - sko_push(do_copy); - if(ddebug){ - out_str("/*(state = (%s) */",do_copy?"true":"false"); - } - out_str("%s\n", buf[strlen (buf) - 1] =='\\' ? "\\" : ""); - } - else if (cmd_match (CMD_POP)){ - sko_pop(&do_copy); - if(ddebug){ - out_str("/*(state = (%s) */",do_copy?"true":"false"); - } - out_str("%s\n", buf[strlen (buf) - 1] =='\\' ? "\\" : ""); - } - else if (cmd_match (CMD_IF_REENTRANT)){ - sko_push(do_copy); - do_copy = reentrant && do_copy; - } - else if (cmd_match (CMD_IF_NOT_REENTRANT)){ - sko_push(do_copy); - do_copy = !reentrant && do_copy; - } - else if (cmd_match(CMD_IF_BISON_BRIDGE)){ - sko_push(do_copy); - do_copy = bison_bridge_lval && do_copy; - } - else if (cmd_match(CMD_IF_NOT_BISON_BRIDGE)){ - sko_push(do_copy); - do_copy = !bison_bridge_lval && do_copy; - } - else if (cmd_match (CMD_ENDIF)){ - sko_pop(&do_copy); - } - else if (cmd_match (CMD_IF_TABLES_SER)) { - do_copy = do_copy && tablesext; - } - else if (cmd_match (CMD_TABLES_YYDMAP)) { - if (tablesext && yydmap_buf.elts) - outn ((char *) (yydmap_buf.elts)); - } - else if (cmd_match (CMD_DEFINE_YYTABLES)) { - if ( tablesext ) - out_str( "#define YYTABLES_NAME \"%s\"\n", - tablesname ? tablesname : "yytables" ); - } - else if (cmd_match (CMD_IF_CPP_ONLY)) { - /* only for C++ */ - sko_push(do_copy); - do_copy = C_plus_plus; - } - else if (cmd_match (CMD_IF_C_ONLY)) { - /* %- only for C */ - sko_push(do_copy); - do_copy = !C_plus_plus; - } - else if (cmd_match (CMD_IF_C_OR_CPP)) { - /* %* for C and C++ */ - sko_push(do_copy); - do_copy = true; - } - else if (cmd_match (CMD_NOT_FOR_HEADER)) { - /* %c begin linkage-only (non-header) code. */ - OUT_BEGIN_CODE (); - } - else if (cmd_match (CMD_OK_FOR_HEADER)) { - /* %e end linkage-only code. */ - OUT_END_CODE (); - } - else { - flexfatal (_("bad line in skeleton file")); - } - } - - else if (do_copy) - outn (buf); - } /* end while */ -} - - /* transition_struct_out - output a yy_trans_info structure * * outputs the yy_trans_info structure with the two elements, element_v and @@ -827,7 +614,8 @@ void transition_struct_out (int element_v, int element_n) if (!gentables) return; - out_dec2 (" {%4d,%4d },", element_v, element_n); + out_dec2 ("M4_HOOK_TABLE_OPENER[[%4d]],[[%4d]]M4_HOOK_TABLE_CONTINUE", element_v, element_n); + outc ('\n'); datapos += TRANS_STRUCT_PRINT_LENGTH; @@ -880,3 +668,20 @@ char *chomp (char *str) *p-- = 0; return str; } + +void comment(const char *txt) +{ + char buf[MAXLINE]; + bool eol; + + strncpy(buf, txt, MAXLINE-1); + eol = buf[strlen(buf)-1] == '\n'; + + if (eol) + buf[strlen(buf)-1] = '\0'; + out_str("M4_HOOK_COMMENT_OPEN [[%s]] M4_HOOK_COMMENT_CLOSE", buf); + if (eol) + outc ('\n'); +} + + diff --git a/src/mkskel.sh b/src/mkskel.sh index 2d6ae9f..8343722 100755 --- a/src/mkskel.sh +++ b/src/mkskel.sh @@ -21,26 +21,24 @@ # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR # PURPOSE. -if test ! $# = 3; then - echo 'Usage: mkskel.sh srcdir m4 version' >&2 +if test ! $# = 4; then + echo 'Usage: mkskel.sh lang srcdir m4 version' >&2 exit 1 fi -echo '/* File created from flex.skl via mkskel.sh */ - -#include "flexdef.h" - -const char *skel[] = {' -srcdir=$1 -m4=$2 -VERSION=$3 +lang=$1 +srcdir=$2 +m4=$3 +VERSION=$4 case $VERSION in *[!0-9.]*) echo 'Invalid version number' >&2; exit 1;; esac IFS=. -set $VERSION +# we do want word splitting, so we won't put double quotes around it (see IFS above) +# shellcheck disable=2086 +set -- $VERSION sed 's/4_/a4_/g s/m4preproc_/m4_/g -' "$srcdir/flex.skl" | +' "$srcdir/${lang}-flex.skl" | "$m4" -P -I "$srcdir" "-DFLEX_MAJOR_VERSION=$1" \ "-DFLEX_MINOR_VERSION=$2" \ "-DFLEX_SUBMINOR_VERSION=$3" | @@ -49,6 +47,3 @@ s/m4_/m4preproc_/g s/a4_/4_/g s/[\\"]/\\&/g s/[^\r]*/ "&",/' - -echo ' 0 -};' @@ -216,10 +216,10 @@ void finish_rule (int mach, int variable_trail_rule, int headcnt, int trailcn if (pcont_act && rule_has_nl[num_rules - 1]) rule_has_nl[num_rules] = true; - snprintf (action_text, sizeof(action_text), "case %d:\n", num_rules); + snprintf (action_text, sizeof(action_text), "M4_HOOK_NORMAL_STATE_CASE_ARM(%d)\n", num_rules); add_action (action_text); if (rule_has_nl[num_rules]) { - snprintf (action_text, sizeof(action_text), "/* rule %d can match eol */\n", + snprintf (action_text, sizeof(action_text), "M4_HOOK_COMMENT_OPEN rule %d can match eol M4_HOOK_COMMENT_CLOSE\n", num_rules); add_action (action_text); } @@ -228,7 +228,7 @@ void finish_rule (int mach, int variable_trail_rule, int headcnt, int trailcn if (variable_trail_rule) { rule_type[num_rules] = RULE_VARIABLE; - if (performance_report > 0) + if (env.performance_hint > 0) fprintf (stderr, _ ("Variable trailing context rule at line %d\n"), @@ -244,49 +244,45 @@ void finish_rule (int mach, int variable_trail_rule, int headcnt, int trailcn /* Do trailing context magic to not match the trailing * characters. */ - char *scanner_cp = "YY_G(yy_c_buf_p) = yy_cp"; - char *scanner_bp = "yy_bp"; - - add_action - ("*yy_cp = YY_G(yy_hold_char); /* undo effects of setting up yytext */\n"); + add_action ("M4_HOOK_RELEASE_YYTEXT\n"); if (headcnt > 0) { if (rule_has_nl[num_rules]) { snprintf (action_text, sizeof(action_text), - "YY_LINENO_REWIND_TO(%s + %d);\n", scanner_bp, headcnt); + "M4_HOOK_LINE_FORWARD(%d)\n", headcnt); add_action (action_text); } - snprintf (action_text, sizeof(action_text), "%s = %s + %d;\n", - scanner_cp, scanner_bp, headcnt); + snprintf (action_text, sizeof(action_text), "M4_HOOK_CHAR_FORWARD(%d)\n", + headcnt); add_action (action_text); } else { if (rule_has_nl[num_rules]) { snprintf (action_text, sizeof(action_text), - "YY_LINENO_REWIND_TO(yy_cp - %d);\n", trailcnt); + "M4_HOOK_LINE_REWIND(%d)\n", trailcnt); add_action (action_text); } - snprintf (action_text, sizeof(action_text), "%s -= %d;\n", - scanner_cp, trailcnt); + snprintf (action_text, sizeof(action_text), "M4_HOOK_CHAR_REWIND(%d)\n", + trailcnt); add_action (action_text); } add_action - ("YY_DO_BEFORE_ACTION; /* set up yytext again */\n"); + ("M4_HOOK_TAKE_YYTEXT\n"); } } /* Okay, in the action code at this point yytext and yyleng have * their proper final values for this rule, so here's the point * to do any user action. But don't do it for continued actions, - * as that'll result in multiple YY_RULE_SETUP's. + * as that'll result in multiple rule-setup calls. */ if (!continued_action) - add_action ("YY_RULE_SETUP\n"); + add_action ("M4_HOOK_SET_RULE_SETUP\n"); - line_directive_out(NULL, 1); + line_directive_out(NULL, infilename, linenum); add_action("[["); } @@ -646,9 +642,9 @@ current_mns); else { check_char (sym); - if (useecs) + if (ctrl.useecs) /* Map NUL's to csize. */ - mkechar (sym ? sym : csize, nextecm, ecgroup); + mkechar (sym ? sym : ctrl.csize, nextecm, ecgroup); } return lastnfa; diff --git a/src/options.c b/src/options.c index e98159c..56c7f42 100644 --- a/src/options.c +++ b/src/options.c @@ -33,7 +33,7 @@ #include "options.h" -/* Be sure to synchronize these options with those defined in "options.h", +/* Be sure to synchronize these options with those defined in "ctrl.h", * the giant switch() statement in "main.c", and the %option processing in * "scan.l". */ @@ -100,7 +100,11 @@ optspec_t flexopts[] = { {"-n", OPT_DONOTHING, 0} , /* For POSIX lex compatibility. */ {"--ecs", OPT_ECS, 0} - , /* Construct equivalence classes. */ + , + {"--emit=LANG", OPT_EMIT, 0} + , /* select language to emit */ + {"-e LANG", OPT_EMIT, 0} + , {"--noecs", OPT_NO_ECS, 0} , {"-F", OPT_FAST, 0} @@ -231,7 +235,9 @@ optspec_t flexopts[] = { {"--yywrap", OPT_YYWRAP, 0} , - {"--nounput", OPT_NO_UNPUT, 0} + {"--nounput", OPT_NO_YYUNPUT, 0} + , + {"--noyyunput", OPT_NO_YYUNPUT, 0} , {"--noyy_push_state", OPT_NO_YY_PUSH_STATE, 0} , diff --git a/src/options.h b/src/options.h index acee275..0b905d5 100644 --- a/src/options.h +++ b/src/options.h @@ -57,6 +57,7 @@ enum flexopt_flag_t { OPT_DEFAULT, OPT_DONOTHING, OPT_ECS, + OPT_EMIT, OPT_FAST, OPT_FULL, OPT_HEADER_FILE, @@ -78,7 +79,7 @@ enum flexopt_flag_t { OPT_NO_REENTRANT, OPT_NO_REJECT, OPT_NO_STDINIT, - OPT_NO_UNPUT, + OPT_NO_YYUNPUT, OPT_NO_WARN, OPT_NO_YYGET_EXTRA, OPT_NO_YYGET_IN, diff --git a/src/parse.y b/src/parse.y index 5a07320..3b8fb9c 100644 --- a/src/parse.y +++ b/src/parse.y @@ -2,7 +2,8 @@ %token CHAR NUMBER SECTEND SCDECL XSCDECL NAME PREVCCL EOF_OP %token TOK_OPTION TOK_OUTFILE TOK_PREFIX TOK_YYCLASS TOK_HEADER_FILE TOK_EXTRA_TYPE -%token TOK_TABLES_FILE +%token TOK_TABLES_FILE TOK_YYLMAX TOK_NUMERIC TOK_YYDECL TOK_PREACTION TOK_POSTACTION +%token TOK_USERINIT TOK_EMIT TOK_BUFSIZE TOK_YYTERMINATE %token CCE_ALNUM CCE_ALPHA CCE_BLANK CCE_CNTRL CCE_DIGIT CCE_GRAPH %token CCE_LOWER CCE_PRINT CCE_PUNCT CCE_SPACE CCE_UPPER CCE_XDIGIT @@ -87,7 +88,7 @@ int previous_continued_action; /* whether the previous rule's action was '|' */ #define CCL_EXPR(func) \ do{ \ int c; \ - for ( c = 0; c < csize; ++c ) \ + for ( c = 0; c < ctrl.csize; ++c ) \ if ( isascii(c) && func(c) ) \ ccladd( currccl, c ); \ }while(0) @@ -96,7 +97,7 @@ int previous_continued_action; /* whether the previous rule's action was '|' */ #define CCL_NEG_EXPR(func) \ do{ \ int c; \ - for ( c = 0; c < csize; ++c ) \ + for ( c = 0; c < ctrl.csize; ++c ) \ if ( !func(c) ) \ ccladd( currccl, c ); \ }while(0) @@ -134,13 +135,16 @@ goal : initlex sect1 sect1end sect2 initforrule for ( i = 1; i <= lastsc; ++i ) scset[i] = mkbranch( scset[i], def_rule ); - if ( spprdflt ) + add_action("]]"); + + if ( ctrl.spprdflt ) add_action( - "YY_FATAL_ERROR( \"flex scanner jammed\" )" ); - else - add_action( "ECHO" ); + "M4_HOOK_FATAL_ERROR(\"flex scanner jammed\")"); + else { + add_action("M4_HOOK_ECHO"); + } - add_action( ";\n\tYY_BREAK]]\n" ); + add_action( "\n\tM4_HOOK_STATE_CASE_BREAK\n" ); } ; @@ -193,21 +197,37 @@ optionlist : optionlist option option : TOK_OUTFILE '=' NAME { - outfilename = xstrdup(nmstr); - did_outfilename = 1; + env.outfilename = xstrdup(nmstr); + env.did_outfilename = 1; } | TOK_EXTRA_TYPE '=' NAME { extra_type = xstrdup(nmstr); } | TOK_PREFIX '=' NAME - { prefix = xstrdup(nmstr); - if (strchr(prefix, '[') || strchr(prefix, ']')) + { ctrl.prefix = xstrdup(nmstr); + if (strchr(ctrl.prefix, '[') || strchr(ctrl.prefix, ']')) flexerror(_("Prefix must not contain [ or ]")); } | TOK_YYCLASS '=' NAME - { yyclass = xstrdup(nmstr); } + { ctrl.yyclass = xstrdup(nmstr); } | TOK_HEADER_FILE '=' NAME - { headerfilename = xstrdup(nmstr); } - | TOK_TABLES_FILE '=' NAME - { tablesext = true; tablesfilename = xstrdup(nmstr); } + { env.headerfilename = xstrdup(nmstr); } + | TOK_YYLMAX '=' TOK_NUMERIC + { ctrl.yylmax = nmval; } + | TOK_YYDECL '=' NAME + { ctrl.yydecl = xstrdup(nmstr); } + | TOK_PREACTION '=' NAME + { ctrl.preaction = xstrdup(nmstr); } + | TOK_POSTACTION '=' NAME + { ctrl.postaction = xstrdup(nmstr); } + | TOK_BUFSIZE '=' TOK_NUMERIC + { ctrl.bufsize = nmval; } + | TOK_EMIT '=' NAME + { ctrl.emit = xstrdup(nmstr); backend_by_name(ctrl.emit); } + | TOK_USERINIT '=' NAME + { ctrl.userinit = xstrdup(nmstr); } + | TOK_YYTERMINATE '=' NAME + { ctrl.yyterminate = xstrdup(nmstr); } + | TOK_TABLES_FILE '=' NAME + { tablesext = true; tablesfilename = xstrdup(nmstr); } ; sect2 : sect2 scon initforrule flexrule '\n' @@ -260,7 +280,7 @@ flexrule : '^' rule { bol_needed = true; - if ( performance_report > 1 ) + if ( env.performance_hint > 1 ) pinpoint_message( "'^' operator results in sub-optimal performance" ); } @@ -409,7 +429,7 @@ rule : re2 re } - if ( lex_compat || (varlength && headcnt == 0) ) + if ( ctrl.lex_compat || (varlength && headcnt == 0) ) { /* variable trailing context rule */ /* Mark the first part of the rule as the * accepting "head" part of a trailing @@ -461,7 +481,7 @@ rule : re2 re varlength = true; } - if ( lex_compat || varlength ) + if ( ctrl.lex_compat || varlength ) { /* Again, see the comment in the rule for * "re2 re" above. @@ -484,7 +504,7 @@ rule : re2 re if ( trlcontxt ) { - if ( lex_compat || (varlength && headcnt == 0) ) + if ( ctrl.lex_compat || (varlength && headcnt == 0) ) /* Both head and trail are * variable-length. */ @@ -698,19 +718,19 @@ singleton : singleton '*' ccladd( ccldot, '\n' ); cclnegate( ccldot ); - if ( useecs ) + if ( ctrl.useecs ) mkeccl( ccltbl + cclmap[ccldot], ccllen[ccldot], nextecm, - ecgroup, csize, csize ); + ecgroup, ctrl.csize, ctrl.csize ); /* Create the (?s:'.') character class. */ cclany = cclinit(); cclnegate( cclany ); - if ( useecs ) + if ( ctrl.useecs ) mkeccl( ccltbl + cclmap[cclany], ccllen[cclany], nextecm, - ecgroup, csize, csize ); + ecgroup, ctrl.csize, ctrl.csize ); madeany = true; } @@ -729,9 +749,9 @@ singleton : singleton '*' */ qsort( ccltbl + cclmap[$1], (size_t) ccllen[$1], sizeof (*ccltbl), cclcmp ); - if ( useecs ) + if ( ctrl.useecs ) mkeccl( ccltbl + cclmap[$1], ccllen[$1], - nextecm, ecgroup, csize, csize ); + nextecm, ecgroup, ctrl.csize, ctrl.csize); ++rulelen; @@ -978,7 +998,7 @@ void build_eof_action(void) } } - line_directive_out(NULL, 1); + line_directive_out(NULL, infilename, linenum); add_action("[["); /* This isn't a normal rule after all - don't count it as @@ -1035,7 +1055,7 @@ void lwarn( const char *str ) void format_pinpoint_message( const char *msg, const char arg[] ) { - char errmsg[MAXLINE]; + char errmsg[MAXLINE*2]; snprintf( errmsg, sizeof(errmsg), msg, arg ); pinpoint_message( errmsg ); @@ -1054,9 +1074,9 @@ void pinpoint_message( const char *str ) void line_warning( const char *str, int line ) { - char warning[MAXLINE]; + char warning[MAXLINE*2]; - if ( ! nowarn ) + if ( ! env.nowarn ) { snprintf( warning, sizeof(warning), "warning, %s", str ); line_pinpoint( warning, line ); diff --git a/src/regex.c b/src/regex.c index f4c4163..17a7aa5 100644 --- a/src/regex.c +++ b/src/regex.c @@ -24,18 +24,17 @@ #include "flexdef.h" -static const char* REGEXP_LINEDIR = "^#line ([[:digit:]]+) \"(.*)\""; - regex_t regex_linedir; /**< matches line directives */ /** Initialize the regular expressions. * @return true upon success. */ -bool flex_init_regex(void) +bool flex_init_regex(const char *traceline_re) { - flex_regcomp(®ex_linedir, REGEXP_LINEDIR, REG_EXTENDED); - return true; + if (traceline_re != NULL) + flex_regcomp(®ex_linedir, traceline_re, REG_EXTENDED); + return true; } /** Compiles a regular expression or dies trying. @@ -45,7 +44,7 @@ bool flex_init_regex(void) */ void flex_regcomp(regex_t *preg, const char *regex, int cflags) { - int err; + int err; memset (preg, 0, sizeof (regex_t)); @@ -100,7 +99,7 @@ char *regmatch_cpy (regmatch_t * m, char *dest, const char *src) } snprintf (dest, (size_t) regmatch_len(m), "%s", src + m->rm_so); - return dest; + return dest; } /** Get the length in characters of the match. @@ -50,23 +50,9 @@ extern const char *escaped_qstart, *escaped_qend; #define ESCAPED_QEND M4QEND "]" M4QSTART M4QEND "]" M4QSTART #define ACTION_ECHO add_action( yytext ) -#define ACTION_IFDEF(def, should_define) \ - { \ - if ( should_define ) \ - action_define( def, 1 ); \ - } - #define ACTION_ECHO_QSTART add_action (ESCAPED_QSTART) #define ACTION_ECHO_QEND add_action (ESCAPED_QEND) -#define ACTION_M4_IFDEF(def, should_define) \ - do{ \ - if ( should_define ) \ - buf_m4_define( &m4defs_buf, def, NULL);\ - else \ - buf_m4_undefine( &m4defs_buf, def);\ - } while(0) - #define MARK_END_OF_PROLOG mark_prolog(); #define YY_DECL \ @@ -98,19 +84,19 @@ extern const char *escaped_qstart, *escaped_qend; if ( all_upper( str ) ) \ reject = true; -#define CHECK_YYMORE(str) \ +#define CHECK_YYREJECT(str) \ if ( all_lower( str ) ) \ - yymore_used = true; + reject = true; #define YY_USER_INIT \ if ( getenv("POSIXLY_CORRECT") ) \ - posix_compat = true; + ctrl.posix_compat = true; #define START_CODEBLOCK(x) do { \ /* Emit the needed line directive... */\ if (indented_code == false) { \ linenum++; \ - line_directive_out(NULL, 1); \ + line_directive_out(NULL, infilename, linenum); \ } \ add_action(M4QSTART); \ yy_push_state(CODEBLOCK); \ @@ -120,7 +106,7 @@ extern const char *escaped_qstart, *escaped_qend; #define END_CODEBLOCK do { \ yy_pop_state();\ add_action(M4QEND); \ - if (!indented_code) line_directive_out(NULL, 0);\ + if (!indented_code) line_directive_out(NULL, NULL, linenum); \ } while (0) %} @@ -160,6 +146,13 @@ LEXOPT [aceknopr] M4QSTART "[""[" M4QEND "]""]" +FUNARGS [^)]* + +%{ +void context_call(char *); +void context_member(char *, const char *); +#undef yyreject +%} %% static int bracelevel, didadef, indented_code; static int doing_rule_action = false; @@ -178,9 +171,13 @@ M4QEND "]""]" ^"%x"{NAME}? return XSCDECL; ^"%{".*{NL} START_CODEBLOCK(false); ^"%top"[[:blank:]]*"{"[[:blank:]]*{NL} { + char trampoline[512]; brace_start_line = linenum; ++linenum; - buf_linedir( &top_buf, infilename?infilename:"<stdin>", linenum); + snprintf(trampoline, sizeof(trampoline), + "M4_HOOK_TRACE_LINE_FORMAT(%d, [[%s]])", + linenum, infilename?infilename:"<stdin>"); + buf_strappend(&top_buf, trampoline); brace_depth = 1; yy_push_state(CODEBLOCK_MATCH_BRACE); } @@ -193,13 +190,13 @@ M4QEND "]""]" sectnum = 2; bracelevel = 0; mark_defs1(); - line_directive_out(NULL, 1); + line_directive_out(NULL, infilename, linenum); BEGIN(SECT2PROLOG); return SECTEND; } - ^"%pointer".*{NL} yytext_is_array = false; ++linenum; - ^"%array".*{NL} yytext_is_array = true; ++linenum; + ^"%pointer".*{NL} ctrl.yytext_is_array = false; ++linenum; + ^"%array".*{NL} ctrl.yytext_is_array = true; ++linenum; ^"%option" BEGIN(OPTION); return TOK_OPTION; @@ -312,7 +309,7 @@ M4QEND "]""]" <<EOF>> { linenum = brace_start_line; synerr(_("Unmatched '{'")); - yyterminate(); + return YY_NULL; } } @@ -356,105 +353,120 @@ M4QEND "]""]" {WS} option_sense = true; "=" return '='; + [[:digit:]]+ {nmval = atoi(yytext); return TOK_NUMERIC;} no option_sense = ! option_sense; - 7bit csize = option_sense ? 128 : 256; - 8bit csize = option_sense ? 256 : 128; + 7bit ctrl.csize = option_sense ? 128 : 256; + 8bit ctrl.csize = option_sense ? 256 : 128; - align long_align = option_sense; + align ctrl.long_align = option_sense; always-interactive { - ACTION_M4_IFDEF( "M4""_YY_ALWAYS_INTERACTIVE", option_sense ); - interactive = option_sense; + ctrl.always_interactive = option_sense; + ctrl.interactive = (trit)option_sense; } - array yytext_is_array = option_sense; - backup backing_up_report = option_sense; - batch interactive = ! option_sense; - bison-bridge bison_bridge_lval = option_sense; - bison-locations { if((bison_bridge_lloc = option_sense)) - bison_bridge_lval = true; - } - "c++" C_plus_plus = option_sense; + array ctrl.yytext_is_array = option_sense; + backup env.backing_up_report = option_sense; + batch ctrl.interactive = (trit)!option_sense; + bison-bridge ctrl.bison_bridge_lval = option_sense; + bison-locations { if((ctrl.bison_bridge_lloc = option_sense)) + ctrl.bison_bridge_lval = true; + } + "c++" ctrl.C_plus_plus = option_sense; caseful|case-sensitive sf_set_case_ins(!option_sense); caseless|case-insensitive sf_set_case_ins(option_sense); - debug ddebug = option_sense; - default spprdflt = ! option_sense; - ecs useecs = option_sense; + debug ctrl.ddebug = option_sense; + default ctrl.spprdflt = ! option_sense; + ecs ctrl.useecs = option_sense; fast { - useecs = usemecs = false; - use_read = fullspd = true; + ctrl.useecs = ctrl.usemecs = false; + ctrl.use_read = ctrl.fullspd = true; } full { - useecs = usemecs = false; - use_read = fulltbl = true; + ctrl.useecs = ctrl.usemecs = false; + ctrl.use_read = ctrl.fulltbl = true; } - input ACTION_IFDEF("YY_NO_INPUT", ! option_sense); - interactive interactive = option_sense; - lex-compat lex_compat = option_sense; - posix-compat posix_compat = option_sense; - line gen_line_dirs = option_sense; + input ctrl.no_yyinput = ! option_sense; + yyinput ctrl.no_yyinput = ! option_sense; + interactive ctrl.interactive = (trit)option_sense; + lex-compat ctrl.lex_compat = option_sense; + posix-compat ctrl.posix_compat = option_sense; + line ctrl.gen_line_dirs = option_sense; main { - ACTION_M4_IFDEF( "M4""_YY_MAIN", option_sense); - /* Override yywrap */ - if( option_sense == true ) - do_yywrap = false; + ctrl.do_main = option_sense; + /* Override yywrap */ + if( option_sense == true ) + ctrl.do_yywrap = false; } - meta-ecs usemecs = option_sense; + meta-ecs ctrl.usemecs = option_sense; never-interactive { - ACTION_M4_IFDEF( "M4""_YY_NEVER_INTERACTIVE", option_sense ); - interactive = !option_sense; + ctrl.never_interactive = option_sense; + ctrl.interactive = (trit)!option_sense; } - perf-report performance_report += option_sense ? 1 : -1; - pointer yytext_is_array = ! option_sense; - read use_read = option_sense; - reentrant reentrant = option_sense; - reject reject_really_used = option_sense; - stack ACTION_M4_IFDEF( "M4""_YY_STACK_USED", option_sense ); - stdinit do_stdinit = option_sense; - stdout use_stdout = option_sense; - unistd ACTION_IFDEF("YY_NO_UNISTD_H", ! option_sense); - unput ACTION_M4_IFDEF("M4""_YY_NO_UNPUT", ! option_sense); - verbose printstats = option_sense; - warn nowarn = ! option_sense; - yylineno do_yylineno = option_sense; ACTION_M4_IFDEF("M4""_YY_USE_LINENO", option_sense); - yymore yymore_really_used = option_sense; - yywrap do_yywrap = option_sense; - - yy_push_state ACTION_M4_IFDEF("M4""_YY_NO_PUSH_STATE", ! option_sense); - yy_pop_state ACTION_M4_IFDEF("M4""_YY_NO_POP_STATE", ! option_sense); - yy_top_state ACTION_M4_IFDEF("M4""_YY_NO_TOP_STATE", ! option_sense); - - yy_scan_buffer ACTION_M4_IFDEF("M4""_YY_NO_SCAN_BUFFER", ! option_sense); - yy_scan_bytes ACTION_M4_IFDEF("M4""_YY_NO_SCAN_BYTES", ! option_sense); - yy_scan_string ACTION_M4_IFDEF("M4""_YY_NO_SCAN_STRING", ! option_sense); - - yyalloc ACTION_M4_IFDEF("M4""_YY_NO_FLEX_ALLOC", ! option_sense); - yyrealloc ACTION_M4_IFDEF("M4""_YY_NO_FLEX_REALLOC", ! option_sense); - yyfree ACTION_M4_IFDEF("M4""_YY_NO_FLEX_FREE", ! option_sense); - - yyget_debug ACTION_M4_IFDEF("M4""_YY_NO_GET_DEBUG", ! option_sense); - yyset_debug ACTION_M4_IFDEF("M4""_YY_NO_SET_DEBUG", ! option_sense); - yyget_extra ACTION_M4_IFDEF("M4""_YY_NO_GET_EXTRA", ! option_sense); - yyset_extra ACTION_M4_IFDEF("M4""_YY_NO_SET_EXTRA", ! option_sense); - yyget_leng ACTION_M4_IFDEF("M4""_YY_NO_GET_LENG", ! option_sense); - yyget_text ACTION_M4_IFDEF("M4""_YY_NO_GET_TEXT", ! option_sense); - yyget_column ACTION_M4_IFDEF("M4""_YY_NO_GET_COLUMN", ! option_sense); - yyset_column ACTION_M4_IFDEF("M4""_YY_NO_SET_COLUMN", ! option_sense); - yyget_lineno ACTION_M4_IFDEF("M4""_YY_NO_GET_LINENO", ! option_sense); - yyset_lineno ACTION_M4_IFDEF("M4""_YY_NO_SET_LINENO", ! option_sense); - yyget_in ACTION_M4_IFDEF("M4""_YY_NO_GET_IN", ! option_sense); - yyset_in ACTION_M4_IFDEF("M4""_YY_NO_SET_IN", ! option_sense); - yyget_out ACTION_M4_IFDEF("M4""_YY_NO_GET_OUT", ! option_sense); - yyset_out ACTION_M4_IFDEF("M4""_YY_NO_SET_OUT", ! option_sense); - yyget_lval ACTION_M4_IFDEF("M4""_YY_NO_GET_LVAL", ! option_sense); - yyset_lval ACTION_M4_IFDEF("M4""_YY_NO_SET_LVAL", ! option_sense); - yyget_lloc ACTION_M4_IFDEF("M4""_YY_NO_GET_LLOC", ! option_sense); - yyset_lloc ACTION_M4_IFDEF("M4""_YY_NO_SET_LLOC", ! option_sense); - + perf-report env.performance_hint += option_sense ? 1 : -1; + pointer ctrl.yytext_is_array = ! option_sense; + read ctrl.use_read = option_sense; + reentrant ctrl.reentrant = option_sense; + reject ctrl.reject_really_used = option_sense; + rewrite ctrl.rewrite = option_sense; + stack ctrl.stack_used = option_sense; + stdinit ctrl.do_stdinit = option_sense; + stdout env.use_stdout = option_sense; + unistd ctrl.no_unistd = ! option_sense; + unput ctrl.no_yyunput = ! option_sense; + yyunput ctrl.no_yyunput = ! option_sense; + verbose env.printstats = option_sense; + warn env.nowarn = ! option_sense; + yylineno ctrl.do_yylineno = option_sense; + yymore ctrl.yymore_really_used = option_sense; + yywrap ctrl.do_yywrap = option_sense; + yyread ctrl.noyyread = !option_sense; + + yypanic ctrl.no_yypanic = !option_sense; + + yy_push_state ctrl.no_yy_push_state = ! option_sense; + yy_pop_state ctrl.no_yy_pop_state = ! option_sense; + yy_top_state ctrl.no_yy_top_state = ! option_sense; + + yy_scan_buffer ctrl.no_yy_scan_buffer = ! option_sense; + yy_scan_bytes ctrl.no_yy_scan_bytes = ! option_sense; + yy_scan_string ctrl.no_yy_scan_string = ! option_sense; + + yyalloc ctrl.no_flex_alloc = ! option_sense; + yyrealloc ctrl.no_flex_realloc = ! option_sense; + yyfree ctrl.no_flex_free = ! option_sense; + + yyget_debug ctrl.no_get_debug = ! option_sense; + yyset_debug ctrl.no_set_debug = ! option_sense; + yyget_extra ctrl.no_yyget_extra = ! option_sense; + yyset_extra ctrl.no_yyset_extra = ! option_sense; + yyget_leng ctrl.no_yyget_leng = ! option_sense; + yyget_text ctrl.no_yyget_text = ! option_sense; + yyget_column ctrl.no_yyget_column = ! option_sense; + yyset_column ctrl.no_yyset_column = ! option_sense; + yyget_lineno ctrl.no_yyget_lineno = ! option_sense; + yyset_lineno ctrl.no_yyset_lineno = ! option_sense; + yyget_in ctrl.no_yyget_in = ! option_sense; + yyset_in ctrl.no_yyset_in = ! option_sense; + yyget_out ctrl.no_yyget_out = ! option_sense; + yyset_out ctrl.no_yyset_out = ! option_sense; + yyget_lval ctrl.no_yyget_lval = ! option_sense; + yyset_lval ctrl.no_yyset_lval = ! option_sense; + yyget_lloc ctrl.no_yyget_lloc = ! option_sense; + yyset_lloc ctrl.no_yyset_lloc = ! option_sense; + + bufsize return TOK_BUFSIZE; + emit return TOK_EMIT; extra-type return TOK_EXTRA_TYPE; outfile return TOK_OUTFILE; prefix return TOK_PREFIX; yyclass return TOK_YYCLASS; + yylmax return TOK_YYLMAX; + yydecl return TOK_YYDECL; + yyterminate return TOK_YYTERMINATE; + pre-action return TOK_PREACTION; + post-action return TOK_POSTACTION; + user-init return TOK_USERINIT; header(-file)? return TOK_HEADER_FILE; tables-file return TOK_TABLES_FILE; tables-verify { @@ -513,7 +525,7 @@ M4QEND "]""]" <<EOF>> { mark_prolog(); sectnum = 0; - yyterminate(); /* to stop the parser */ + return YY_NULL; /* to stop the parser */ } } @@ -537,7 +549,7 @@ M4QEND "]""]" "\"" BEGIN(QUOTE); return '"'; "{"/[[:digit:]] { BEGIN(NUM); - if ( lex_compat || posix_compat ) + if ( ctrl.lex_compat || ctrl.posix_compat ) return BEGIN_REPEAT_POSIX; else return BEGIN_REPEAT_FLEX; @@ -633,8 +645,7 @@ M4QEND "]""]" ^"%%".* { sectnum = 3; - BEGIN(no_section3_escape ? SECT3_NOESCAPE : SECT3); - outn("/* Begin user sect3 */"); + BEGIN(ctrl.no_section3_escape ? SECT3_NOESCAPE : SECT3); yyterminate(); /* to stop the parser */ } @@ -662,7 +673,7 @@ M4QEND "]""]" */ && (cclval = ccllookup( nmstr )) != 0 ) { - if ( input() != ']' ) + if ( yyinput() != ']' ) synerr( _( "bad character class" ) ); yylval = cclval; @@ -722,7 +733,7 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ if (end_is_ws) unput(end_ch); - if ( lex_compat || nmdefptr[0] == '^' || + if ( ctrl.lex_compat || nmdefptr[0] == '^' || (len > 0 && nmdefptr[len - 1] == '$') || (end_is_ws && trlcontxt && !sf_skip_ws())) { /* don't use ()'s after all */ @@ -752,7 +763,7 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ } "(?#" { - if (lex_compat || posix_compat){ + if (ctrl.lex_compat || ctrl.posix_compat){ /* Push back the "?#" and treat it like a normal parens. */ yyless(1); sf_push(); @@ -763,7 +774,7 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ } "(?" { sf_push(); - if (lex_compat || posix_compat) + if (ctrl.lex_compat || ctrl.posix_compat) /* Push back the "?" and treat it like a normal parens. */ yyless(1); else @@ -885,7 +896,7 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ "," return ','; "}" { BEGIN(SECT2); - if ( lex_compat || posix_compat ) + if ( ctrl.lex_compat || ctrl.posix_compat ) return END_REPEAT_POSIX; else return END_REPEAT_FLEX; @@ -913,12 +924,19 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ <CODEBLOCK,ACTION>{ "reject" { - ACTION_ECHO; - CHECK_REJECT(yytext); + ACTION_ECHO; + CHECK_REJECT(yytext); } - "yymore" { - ACTION_ECHO; - CHECK_YYMORE(yytext); + "yyreject()" { + add_action("]""]M4_HOOK_REJECT[""["); + CHECK_YYREJECT(yytext); + } + "yymore()" { + yymore_used = true; + if (ctrl.rewrite) + context_call(yytext); + else + ACTION_ECHO; } } @@ -928,7 +946,7 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ ACTION_ECHO; if (bracelevel <= 0 || (doing_codeblock && indented_code)) { if ( doing_rule_action ) - add_action( "\tYY_BREAK]""]\n" ); + add_action( "\t]""]M4_HOOK_STATE_CASE_BREAK\n" ); doing_rule_action = doing_codeblock = false; BEGIN(SECT2); @@ -937,11 +955,34 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ } - /* Reject and YYmore() are checked for above, in PERCENT_BRACE_ACTION */ + /* yyreject and yymore() are checked for above, in PERCENT_BRACE_ACTION */ <ACTION>{ "{" ACTION_ECHO; ++bracelevel; "}" ACTION_ECHO; --bracelevel; - [^[:alpha:]_{}""''/\n\[\]]+ ACTION_ECHO; + yyecho\(\)|yyinput\(\)|yystart\({FUNARGS}\)|yybegin\({FUNARGS}\)|yyunput\({FUNARGS}\)|yypanic\({FUNARGS}\)|yyatbol\(\)|yysetbol\({FUNARGS}\) { + if (ctrl.rewrite) + context_call(yytext); + else + ACTION_ECHO; + } + yyterminate\(\)|yyless\({FUNARGS}\) { + add_action("]""]"); + add_action(yytext); + add_action("[""["); + } + (yyin|yyout|yyextra|yyleng|yytext|yyflexdebug)/[^[:alnum:]_] { + if (ctrl.rewrite) + context_member(yytext, "M4_PROPERTY_CONTEXT_FORMAT"); + else + ACTION_ECHO; + } + (yylineno|yycolumn)/[^[:alnum:]_] { + if (ctrl.rewrite) + context_member(yytext, "M4_PROPERTY_BUFFERSTACK_CONTEXT_FORMAT"); + else + ACTION_ECHO; + } + [^[:Alpha:]_{}""''/\n\[\]]+ ACTION_ECHO; {NAME} ACTION_ECHO; "'"([^''\\\n]|\\.)"'" ACTION_ECHO; /* character constant */ "'" ACTION_ECHO; BEGIN(CHARACTER_CONSTANT); @@ -951,7 +992,7 @@ nmstr[yyleng - 2 - end_is_ws] = '\0'; /* chop trailing brace */ ACTION_ECHO; if (bracelevel <= 0) { if ( doing_rule_action ) - add_action( "\tYY_BREAK]""]\n" ); + add_action( "\t]""]M4_HOOK_STATE_CASE_BREAK\n" ); doing_rule_action = false; BEGIN(SECT2); @@ -1053,3 +1094,38 @@ void set_input_file( char *file ) linenum = 1; } + +void context_call(char *txt) { + const char *context_arg = skel_property("M4_PROPERTY_CONTEXT_ARG"); + + /* if there's no such property, simply pass through */ + if (context_arg == NULL) { + add_action(txt); + } else { + char buf[BUFSIZ]; + + /* otherwise, supply the context string as a final argument */ + assert(txt[strlen(txt)-1] == ')'); + strncpy(buf, txt, sizeof(buf)); + buf[strlen(buf)-1] = '\0'; /* remove trailing ) */ + add_action(buf); + if (txt[strlen(buf)-1] != '(') { + add_action(", "); + } + add_action(context_arg); + add_action(")"); + } +} + +void context_member(char *txt, const char *prop) { + const char *context_format = skel_property(prop); + + /* if there's no such property, simply pass through */ + if (context_format == NULL) { + add_action(txt); + } else { + char buf[128]; + snprintf(buf, sizeof(buf), context_format, txt); + add_action(buf); + } +} diff --git a/src/scanopt.h b/src/scanopt.h index c6e7dec..06b7c60 100644 --- a/src/scanopt.h +++ b/src/scanopt.h @@ -76,7 +76,7 @@ extern "C" { /* Initializes scanner and checks option list for errors. * Parameters: - * options - Array of options. + * options - Array of ctrl. * argc - Same as passed to main(). * argv - Same as passed to main(). First element is skipped. * flags - Control behavior. @@ -110,7 +110,7 @@ extern "C" { * If return is zero, then optindex is the NEXT valid option index. * * Return: > 0 on success. Return value is from optspec_t->rval. - * == 0 if at end of options. + * == 0 if at end of ctrl. * < 0 on error (return value is an error code). * */ diff --git a/src/skeletons.c b/src/skeletons.c new file mode 100644 index 0000000..84e3a4f --- /dev/null +++ b/src/skeletons.c @@ -0,0 +1,255 @@ +/* flex - tool to generate fast lexical analyzers */ + +/* Copyright (c) 1990 The Regents of the University of California. */ +/* All rights reserved. */ + +/* This code is derived from software contributed to Berkeley by */ +/* Vern Paxson. */ + +/* The United States Government has rights in this work pursuant */ +/* to contract no. DE-AC03-76SF00098 between the United States */ +/* Department of Energy and the University of California. */ + +/* This file is part of flex. */ + +/* Redistribution and use in source and binary forms, with or without */ +/* modification, are permitted provided that the following conditions */ +/* are met: */ + +/* 1. Redistributions of source code must retain the above copyright */ +/* notice, this list of conditions and the following disclaimer. */ +/* 2. Redistributions in binary form must reproduce the above copyright */ +/* notice, this list of conditions and the following disclaimer in the */ +/* documentation and/or other materials provided with the distribution. */ + +/* Neither the name of the University nor the names of its contributors */ +/* may be used to endorse or promote products derived from this software */ +/* without specific prior written permission. */ + +/* THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR */ +/* IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED */ +/* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR */ +/* PURPOSE. */ + + +#include "flexdef.h" +#include "tables.h" + +/* START digested skeletons */ + +const char *cpp_skel[] = { +#include "cpp-flex.h" + 0, +}; + +const char *c99_skel[] = { +#include "c99-flex.h" + 0, +}; + +const char *go_skel[] = { +#include "go-flex.h" + 0, +}; + +/* END digested skeletons */ + +/* Method table describing a language-specific back end. + * Even if this never gets a member other than the skel + * array, it prevents us from getting lost in a maze of + * twisty array reference levels, all different. + */ +struct flex_backend_t { + const char **skel; // Digested skeleton file +}; + +static struct flex_backend_t backends[] = { + {.skel=cpp_skel}, + {.skel=c99_skel}, + {.skel=go_skel}, + {NULL} +}; + +static struct flex_backend_t *backend = &backends[0]; + +/* Functions for querying skeleton properties. */ + +bool is_default_backend(void) +{ + return backend == &backends[0]; +} + +/* Search for a string in the skeleton prolog, where macros are defined. + */ +static bool boneseeker(const char *bone) +{ + int i; + + for (i = 0; backend->skel[i] != NULL; i++) { + const char *line = backend->skel[i]; + if (strstr(line, bone) != NULL) + return true; + else if (strncmp(line, "%%", 2) == 0) + break; + } + return false; +} + +void backend_by_name(const char *name) +{ + const char *prefix_property; + if (name != NULL) { + if (strcmp(name, "nr") == 0) { + backend = &backends[0]; + ctrl.reentrant = false; + goto backend_ok; + } + if (strcmp(name, "r") == 0) { + backend = &backends[0]; + ctrl.reentrant = true; + goto backend_ok; + } + for (backend = &backends[0]; backend->skel != NULL; backend++) { + if (strcasecmp(skel_property("M4_PROPERTY_BACKEND_NAME"), name) == 0) + goto backend_ok; + } + flexerror(_("no such back end")); + } + backend_ok: + ctrl.rewrite = !is_default_backend(); + ctrl.backend_name = xstrdup(skel_property("M4_PROPERTY_BACKEND_NAME")); + ctrl.traceline_re = xstrdup(skel_property("M4_PROPERTY_TRACE_LINE_REGEXP")); + ctrl.traceline_template = xstrdup(skel_property("M4_PROPERTY_TRACE_LINE_TEMPLATE")); + ctrl.have_state_entry_format = boneseeker("m4_define([[M4_HOOK_STATE_ENTRY_FORMAT]]"); + prefix_property = skel_property("M4_PROPERTY_PREFIX"); + if (prefix_property != NULL) + ctrl.prefix = xstrdup(prefix_property); + flex_init_regex(ctrl.traceline_re); +} + +const char *suffix (void) +{ + const char *suffix; + + if (is_default_backend()) { + if (ctrl.C_plus_plus) + suffix = "cc"; + else + suffix = "c"; + } else { + suffix = skel_property("M4_PROPERTY_SOURCE_SUFFIX"); + } + + return suffix; +} + +/* Search for a m4 define of the property key, retrieve the value. The + * definition must be single-line. Don't call this a second time before + * stashing away the previous return, we cheat with static buffers. + */ +const char *skel_property(const char *propname) +{ + int i; + static char name[256], value[256], *np, *vp;; + const char *cp; + + for (i = 0; backend->skel[i] != NULL; i++) { + const char *line = backend->skel[i]; + if (line[0] == '\0') + continue; + /* only scan before first skell breakpoint */ + if (strncmp(line, "%%", 2) == 0) + break; + /* ignore anything that's not a definition */ + if (strncmp(line, "m4_define(", 10) != 0) + continue; + /* skip space and quotes before macro name */ + for (cp = line + 10; isspace(*cp) || *cp == '['; *cp++) + continue; + /* copy up to following ] into the name buffer */ + np = name; + while (*cp != ']' && *cp != '\0' && (np < name + sizeof(name)-1)) { + *np++ = *cp++; + } + *np = '\0'; + /* check for valid and matching name */ + if (*cp == ']') { + if (strcmp(name, propname) != 0) + continue; /* try next line */ + } else { + flexerror(_("unterminated or too long property name")); + continue; + } + /* skip to the property value */ + while (*cp != '\0' && (*cp == ']' || isspace(*cp) || *cp == ',')) + cp++; + while (*cp == '[') + cp++; + if (*cp == '\0') + flexerror(_("garbled property line")); + /* extract the value */ + vp = value; + while (*cp != '\0' && vp < value + sizeof(value) - 1 && (cp[0] != ']' || cp[1] != ']')) + *vp++ = *cp++; + if (*cp == ']') { + *vp = '\0'; + return value; + } else { + flexerror(_("unterminated or too long property value")); + } + } + return NULL; +} + +/* skelout - write out one section of the skeleton file + * + * Description + * Copies skelfile or skel array to stdout until a line beginning with + * "%%" or EOF is found. + */ +void skelout (bool announce) +{ + char buf_storage[MAXLINE]; + char *buf = buf_storage; + bool do_copy = true; + + /* Loop pulling lines either from the skelfile, if we're using + * one, or from the selected back end's skel[] array. + */ + while (env.skelfile != NULL ? + (fgets (buf, MAXLINE, env.skelfile) != NULL) : + ((buf = (char *) backend->skel[skel_ind++]) != 0)) { + + if (env.skelfile != NULL) + chomp (buf); + + /* copy from skel array */ + if (buf[0] == '%') { /* control line */ + /* print the control line as a comment. */ + if (ctrl.ddebug && buf[1] != '#') { + comment(buf); + outc ('\n'); + } + if (buf[1] == '#') { + /* %# indicates comment line to be ignored */ + } + else if (buf[1] == '%') { + /* %% is a break point for skelout() */ + if (announce) { + comment(buf); + outc ('\n'); + } + return; + } + else { + flexfatal (_("bad line in skeleton file")); + } + } + + else if (do_copy) + outn (buf); + } /* end while */ +} + + +/* end */ diff --git a/src/tables_shared.h b/src/tables_shared.h index feca251..e07207e 100644 --- a/src/tables_shared.h +++ b/src/tables_shared.h @@ -128,11 +128,11 @@ struct yytbl_data { #endif #ifdef FLEX_SCANNER -%not-for-header +m4_ifdef( [[M4_YY_NOT_IN_HEADER]],[[ #endif yyskel_static flex_int32_t yytbl_calc_total_len (const struct yytbl_data *tbl); #ifdef FLEX_SCANNER -%ok-for-header +]]) #endif /* vim:set noexpandtab cindent tabstop=8 softtabstop=0 shiftwidth=8 textwidth=0: */ diff --git a/src/tblcmp.c b/src/tblcmp.c index 81dfbc1..cd61c80 100644 --- a/src/tblcmp.c +++ b/src/tblcmp.c @@ -228,7 +228,7 @@ void cmptmps (void) peakpairs = numtemps * numecs + tblend; - if (usemecs) { + if (ctrl.usemecs) { /* Create equivalence classes based on data gathered on * template transitions. */ @@ -250,7 +250,7 @@ void cmptmps (void) for (j = 1; j <= numecs; ++j) { trans = tnxt[numecs * i + j]; - if (usemecs) { + if (ctrl.usemecs) { /* The absolute value of tecbck is the * meta-equivalence class of a given * equivalence class, as set up by cre8ecs(). @@ -426,7 +426,7 @@ void inittbl (void) firstfree = tblend + 1; numtemps = 0; - if (usemecs) { + if (ctrl.usemecs) { /* Set up doubly-linked meta-equivalence classes; these * are sets of equivalence classes which all have identical * transitions out of TEMPLATES. @@ -708,7 +708,7 @@ void mktemplate (int state[], int statenum, int comstate) tnxt[tmpbase + i] = comstate; } - if (usemecs) + if (ctrl.usemecs) mkeccl (transset, tsptr, tecfwd, tecbck, numecs, 0); mkprot (tnxt + tmpbase, -numtemps, comstate); diff --git a/src/yylex.c b/src/yylex.c index 521db7f..9a2e59f 100644 --- a/src/yylex.c +++ b/src/yylex.c @@ -63,7 +63,7 @@ int yylex (void) toktype = 0; } - if (trace) { + if (env.trace) { if (beglin) { fprintf (stderr, "%d\t", num_rules + 1); beglin = 0; @@ -152,7 +152,7 @@ int yylex (void) default: if (!isascii (yylval) || !isprint (yylval)) { - if(trace_hex) + if(env.trace_hex) fprintf (stderr, "\\x%02x", (unsigned int) yylval); else fprintf (stderr, "\\%.3o", (unsigned int) yylval); diff --git a/tests/.gitignore b/tests/.gitignore index e5c42b3..4aa70a6 100644 --- a/tests/.gitignore +++ b/tests/.gitignore @@ -2,16 +2,14 @@ *.trs *.o *.tables -alloc_extra -alloc_extra.c -array_nr -array_nr.c -array_r -array_r.c -basic_nr -basic_nr.c -basic_r -basic_r.c +alloc_extra_nr +alloc_extra_nr.c +alloc_extra_c99 +alloc_extra_c99.c +array* +!array.rules +basic* +!basic.rules bison_nr bison_nr_scanner.[ch] bison_nr_parser.[ch] @@ -21,12 +19,14 @@ bison_yylloc_scanner.[ch] bison_yylval bison_yylval_parser.[ch] bison_yylval_scanner.[ch] +bol* +!bol.rules c_cxx_nr c_cxx_nr.cc c_cxx_r c_cxx_r.cc -ccl -ccl.c +ccl* +!ccl.rules cxx_basic cxx_basic.cc cxx_multiple_scanners @@ -35,12 +35,14 @@ cxx_restart cxx_restart.cc cxx_yywrap.i3 cxx_yywrap.cc -debug_nr -debug_nr.c -debug_r -debug_r.c -extended -extended.c +debug* +!debug.rules +extended* +!extended.rules +fixedtrailing* +!fixedtrailing.rules +flexname* +!flexname.rules header_nr header_nr_scanner.[ch] header_r @@ -51,55 +53,70 @@ include_by_push.direct include_by_push.direct.c include_by_reentrant.direct include_by_reentrant.direct.c -lineno_nr.one -lineno_nr.c -lineno_r.one -lineno_r.c -lineno_trailing.one -lineno_trailing.c +lexcompat* +!lexcompat.rules +lineno* +!lineno.rules mem_nr mem_nr.c mem_r mem_r.c +mem_c99 +mem_c99.c multiple_scanners_nr multiple_scanners_nr_[12].[ch] multiple_scanners_r multiple_scanners_r_[12].[ch] -posix -posix.c -posixly_correct -posixly_correct.c +posix* +!posix.rules +posixlycorrect* +!posixlycorrect.rules prefix_nr prefix_nr.c prefix_r prefix_r.c +prefix_c99 +prefix_c99.c +preposix* +!preposix.rules pthread.pthread pthread.c -reject_nr.reject -reject_nr.reject.c -reject_r.reject -reject_r.reject.c -reject_ver.table -reject_ver.table.c -reject_ser.table -reject_ser.table.c +quoteincomment* +!quoteincomment.rules +quotes +quotes.c +reject* +!reject.rules rescan_nr.direct rescan_nr.direct.c rescan_r.direct rescan_r.direct.c -quote_in_comment -quote_in_comment.c -quotes -quotes.c string_nr string_nr.c string_r string_r.c +string_c99 +string_c99.c +tableopts* +!tableopts.rules top top.[ch] -yyextra -yyextra.c -tableopts_*.c +vartrailing* +!vartrailing.rules +yyextra_nr +yyextra_nr.c +yyextra_c99 +yyextra_c99.c +yyless* +!yyless.rules +yymore* +!yymore.rules +!yymorearray.rules +!yymorearraybol.rules +yyunput* +!yyunput.rules +test-yydecl-* +!test-yydecl-gen.sh *.opt *.ser *.ver diff --git a/tests/Makefile.am b/tests/Makefile.am index 65f9d65..24ab7dd 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -24,22 +24,16 @@ AM_CFLAGS = $(WARNINGFLAGS) TESTS = $(check_PROGRAMS) options.cn # The script testwrapper.sh will run most tests as is. A couple tests -# in the suite end in .reject, .table and the like so that we can pass +# in the suite end in .direct, .cn. and the like so that we can pass # different arguments to the test runner. We list those extensions so # automake knows how to distinguish between the various kinds of tests # we have. -TEST_EXTENSIONS = .reject .table .direct .cn .i3 .pthread .one .opt .ser .ver +TEST_EXTENSIONS = .direct .cn .i3 .pthread .opt .ser .ver LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh AM_LOG_FLAGS = -d $(srcdir) -r -REJECT_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh -AM_REJECT_LOG_FLAGS = -d $(srcdir) -i $(srcdir)/reject.txt -r - -TABLE_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh -AM_TABLE_LOG_FLAGS = -d $(builddir) -i $(srcdir)/reject.txt -t - DIRECT_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper-direct.sh AM_DIRECT_LOG_FLAGS = -b $(abs_builddir) -s $(srcdir) @@ -52,56 +46,49 @@ AM_I3_LOG_FLAGS = -i $(srcdir)/cxx_yywrap.txt -i $(srcdir)/cxx_yywrap.txt -i $(s PTHREAD_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh AM_PTHREAD_LOG_FLAGS = -i $(srcdir)/pthread_1.txt -i $(srcdir)/pthread_2.txt -i $(srcdir)/pthread_3.txt -i $(srcdir)/pthread_4.txt -i $(srcdir)/pthread_5.txt -ONE_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh -AM_ONE_LOG_FLAGS = -1 -d $(srcdir) +OPT_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh +AM_OPT_LOG_FLAGS = -d $(srcdir) -i $(srcdir)/tableopts.txt -r + +SER_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh +AM_SER_LOG_FLAGS = -d $(builddir) -i $(srcdir)/tableopts.txt -r + +VER_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh +AM_VER_LOG_FLAGS = -d $(builddir) -i $(srcdir)/tableopts.txt -r AM_YFLAGS = -d -p test AM_CPPFLAGS = -I$(top_builddir)/src -I$(top_srcdir)/src -check_PROGRAMS = $(simple_tests) $(reject_tests) $(TABLE_TESTS) $(DIRECT_TESTS) $(I3_TESTS) $(PTHREAD_TESTS) $(ONE_TESTS) $(TABLEOPTS_TESTS) +check_PROGRAMS = $(RULESET_TESTS) $(SPORADIC_TESTS) $(DIRECT_TESTS) $(I3_TESTS) $(PTHREAD_TESTS) -simple_tests = \ - alloc_extra \ - array_nr \ - array_r \ - basic_nr \ - basic_r \ +SPORADIC_TESTS = \ + alloc_extra_nr \ + alloc_extra_c99 \ bison_nr \ bison_yylloc \ bison_yylval \ c_cxx_nr \ c_cxx_r \ - ccl \ cxx_basic \ cxx_multiple_scanners \ cxx_restart \ - debug_nr \ - debug_r \ - extended \ header_nr \ header_r \ mem_nr \ mem_r \ + mem_c99 \ multiple_scanners_nr \ multiple_scanners_r \ - posix \ - posixly_correct \ prefix_nr \ prefix_r \ - quote_in_comment \ + prefix_c99 \ quotes \ string_nr \ string_r \ + string_c99 \ top \ - yyextra + yyextra_nr \ + yyextra_c99 -reject_tests = \ - reject_nr.reject \ - reject_r.reject - -TABLE_TESTS = \ - reject_ver.table \ - reject_ser.table DIRECT_TESTS = \ include_by_buffer.direct \ @@ -116,17 +103,8 @@ I3_TESTS = \ PTHREAD_TESTS = \ pthread.pthread -ONE_TESTS = \ - lineno_nr.one \ - lineno_r.one \ - lineno_trailing.one - -quote_in_comment_SOURCES = quote_in_comment.l -alloc_extra_SOURCES = alloc_extra.l -array_nr_SOURCES = array_nr.l -array_r_SOURCES = array_r.l -basic_nr_SOURCES = basic_nr.l -basic_r_SOURCES = basic_r.l +alloc_extra_nr_SOURCES = alloc_extra_nr.l +alloc_extra_c99_SOURCES = alloc_extra_c99.l if HAVE_BISON bison_nr_SOURCES = bison_nr_scanner.l bison_nr_parser.y bison_nr_main.c nodist_bison_nr_SOURCES = bison_nr_parser.h bison_nr_scanner.h @@ -141,14 +119,10 @@ bison_yylval_SOURCES = no_bison_stub.c endif c_cxx_nr_SOURCES = c_cxx_nr.lll c_cxx_r_SOURCES = c_cxx_r.lll -ccl_SOURCES = ccl.l cxx_basic_SOURCES = cxx_basic.ll cxx_restart_SOURCES = cxx_restart.ll cxx_multiple_scanners_SOURCES = cxx_multiple_scanners_main.cc cxx_multiple_scanners_1.ll cxx_multiple_scanners_2.ll cxx_yywrap_i3_SOURCES = cxx_yywrap.ll -debug_nr_SOURCES = debug_nr.l -debug_r_SOURCES = debug_r.l -extended_SOURCES = extended.l header_nr_SOURCES = header_nr_scanner.l header_nr_main.c nodist_header_nr_SOURCES = header_nr_scanner.h header_r_SOURCES = header_r_scanner.l header_r_main.c @@ -156,32 +130,27 @@ nodist_header_r_SOURCES = header_r_scanner.h include_by_buffer_direct_SOURCES = include_by_buffer.direct.l include_by_push_direct_SOURCES = include_by_push.direct.l include_by_reentrant_direct_SOURCES = include_by_reentrant.direct.l -lineno_nr_one_SOURCES = lineno_nr.l -lineno_r_one_SOURCES = lineno_r.l -lineno_trailing_one_SOURCES = lineno_trailing.l mem_nr_SOURCES = mem_nr.l mem_r_SOURCES = mem_r.l +mem_c99_SOURCES = mem_c99.l multiple_scanners_nr_SOURCES = multiple_scanners_nr_main.c multiple_scanners_nr_1.l multiple_scanners_nr_2.l nodist_multiple_scanners_nr_SOURCES = multiple_scanners_nr_1.h multiple_scanners_nr_2.h multiple_scanners_r_SOURCES = multiple_scanners_r_main.c multiple_scanners_r_1.l multiple_scanners_r_2.l nodist_multiple_scanners_r_SOURCES = multiple_scanners_nr_1.h multiple_scanners_nr_2.h -posix_SOURCES = posix.l -posixly_correct_SOURCES = posixly_correct.l prefix_nr_SOURCES = prefix_nr.l prefix_r_SOURCES = prefix_r.l +prefix_c99_SOURCES = prefix_c99.l pthread_pthread_SOURCES = pthread.l quotes_SOURCES = quotes.l -reject_nr_reject_SOURCES = reject.l4 -reject_r_reject_SOURCES = reject.l4 -reject_ver_table_SOURCES = reject.l4 -reject_ser_table_SOURCES = reject.l4 rescan_nr_direct_SOURCES = rescan_nr.direct.l rescan_r_direct_SOURCES = rescan_r.direct.l string_nr_SOURCES = string_nr.l string_r_SOURCES = string_r.l +string_c99_SOURCES = string_c99.l top_SOURCES = top.l top_main.c nodist_top_SOURCES = top.h -yyextra_SOURCES = yyextra.l +yyextra_nr_SOURCES = yyextra_nr.l +yyextra_c99_SOURCES = yyextra_c99.l # Normally, automake would distribute files built by flex. Since the # point of the test suite is to test the files that flex builds, and @@ -193,11 +162,8 @@ yyextra_SOURCES = yyextra.l # it. CLEANFILES = \ - alloc_extra.c \ - array_nr.c \ - array_r.c \ - basic_nr.c \ - basic_r.c \ + alloc_extra_nr.c \ + alloc_extra_c99.c \ bison_nr_parser.c \ bison_nr_parser.h \ bison_nr_scanner.c \ @@ -212,15 +178,13 @@ CLEANFILES = \ bison_yylval_scanner.h \ c_cxx_nr.cc \ c_cxx_r.cc \ - ccl.c \ + ccl_* \ + !ccl.rules \ cxx_basic.cc \ cxx_multiple_scanners_1.cc \ cxx_multiple_scanners_2.cc \ cxx_restart.cc \ cxx_yywrap.cc \ - debug_nr.c \ - debug_r.c \ - extended.c \ header_nr_scanner.c \ header_nr_scanner.h \ header_r_scanner.c \ @@ -228,11 +192,9 @@ CLEANFILES = \ include_by_buffer.direct.c \ include_by_push.direct.c \ include_by_reentrant.direct.c \ - lineno_nr.c \ - lineno_r.c \ - lineno_trailing.c \ mem_nr.c \ mem_r.c \ + mem_c99.c \ multiple_scanners_nr_1.c \ multiple_scanners_nr_1.h \ multiple_scanners_nr_2.c \ @@ -241,28 +203,25 @@ CLEANFILES = \ multiple_scanners_r_1.h \ multiple_scanners_r_2.c \ multiple_scanners_r_2.h \ - posix.c \ - posixly_correct.c \ prefix_nr.c \ prefix_r.c \ + prefix_c99.c \ pthread.c \ - quote_in_comment.c \ quotes.c \ - reject_nr.reject.c \ - reject_r.reject.c \ - reject_ser.table.c \ - reject_ser.table.tables \ - reject_ver.table.c \ - reject_ver.table.tables \ + quotes_c99.c \ rescan_nr.direct.c \ rescan_r.direct.c \ string_nr.c \ string_r.c \ + string_c99.c \ + string_c99.c \ top.c \ top.h \ - yyextra.c \ - $(tableopts_c) \ - $(tableopts_tables) + yyextra_nr.c \ + yyextra_c99.c \ + yyunput_* \ + !yyunput.rules \ + $(RULESET_REMOVABLES) dist-hook: chmod u+w $(distdir) && \ @@ -272,24 +231,19 @@ dist-hook: EXTRA_DIST = \ README \ + testmaker.m4 \ alloc_extra.txt \ array_nr.txt \ array_r.txt \ - basic_nr.txt \ - basic_r.txt \ bison_nr.txt \ bison_yylloc.txt \ bison_yylval.txt \ c_cxx_nr.txt \ c_cxx_r.txt \ - ccl.txt \ cxx_basic.txt \ cxx_multiple_scanners.txt \ cxx_restart.txt \ cxx_yywrap.txt \ - debug_nr.txt \ - debug_r.txt \ - extended.txt \ header_nr.txt \ header_r.txt \ include_by_buffer.direct.txt \ @@ -313,34 +267,35 @@ EXTRA_DIST = \ pthread_3.txt \ pthread_4.txt \ pthread_5.txt \ - reject.txt \ rescan_nr.direct.txt \ rescan_r.direct.txt \ - quote_in_comment.txt \ quotes.txt \ + tableopts.txt \ top.txt \ - yyextra.txt \ - tableopts.txt + yyextra.txt dist_noinst_SCRIPTS = \ - tableopts.sh + ruleset.sh dist_check_SCRIPTS = \ options.cn \ testwrapper-direct.sh \ - testwrapper.sh + testwrapper.sh \ + testmaker.sh pthread_pthread_LDADD = @LIBPTHREAD@ # specify how to process .l files in order to test the flex built by make all +TESTOPTS = -L + FLEX = $(top_builddir)/src/flex .l.c: $(FLEX) - $(AM_V_LEX)$(FLEX) -o $@ $< + $(AM_V_LEX)$(FLEX) $(TESTOPTS) -o $@ $< .ll.cc: $(FLEX) - $(AM_V_LEX)$(FLEX) -+ -o $@ $< + $(AM_V_LEX)$(FLEX) $(TESTOPTS) -+ -o $@ $< bison_nr_main.$(OBJEXT): bison_nr_parser.h bison_nr_scanner.h bison_nr_scanner.$(OBJEXT): bison_nr_parser.h @@ -367,7 +322,7 @@ bison_yylval_scanner.h: bison_yylval_scanner.c # so we explicitly sayhow, using the .lll suffix for the lex input file .lll.cc: $(FLEX) - $(AM_V_LEX)$(FLEX) -o $@ $< + $(AM_V_LEX)$(FLEX) $(TESTOPTS) -o $@ $< header_nr_main.$(OBJEXT): header_nr_scanner.h @@ -401,108 +356,28 @@ multiple_scanners_r_2.h: multiple_scanners_r_2.c @if test ! -f $@; then rm -f $<; else :; fi @if test ! -f $@; then $(MAKE) $(AM_MAKEFLAGS) $<; else :; fi -posixly_correct.c: posixly_correct.l $(FLEX) - $(AM_V_LEX)POSIXLY_CORRECT=1 $(FLEX) -o $@ $< - -reject_nr.reject.c: reject.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -o $@ $< - -reject_nr.reject$(EXEEXT): reject_nr.reject.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $^ - -reject_r.reject.c: reject.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape --reentrant -o $@ $< - -reject_r.reject.$(OBJEXT): reject_r.reject.c - $(AM_V_CC)$(COMPILE) -DTEST_IS_REENTRANT -c -o $@ $< - -reject_r.reject$(EXEEXT): reject_r.reject.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $^ - -reject_ver.table.c: reject.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -o $@ --tables-verify --tables-file=$(basename $@).tables $< - -reject_ver.table.$(OBJEXT): reject_ver.table.c - $(AM_V_CC)$(COMPILE) -DTEST_HAS_TABLES_EXTERNAL -c -o $@ $< - -reject_ver.table$(EXEEXT): reject_ver.table.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $^ - -reject_ser.table.c: reject.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) -o $@ --unsafe-no-m4-sect3-escape --tables-file=$(basename $@).tables $< - -reject_ser.table.$(OBJEXT): reject_ser.table.c - $(AM_V_CC)$(COMPILE) -DTEST_HAS_TABLES_EXTERNAL -c -o $@ $< - -reject_ser.table$(EXEEXT): reject_ser.table.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $^ - top_main.$(OBJEXT): top.h top.h: top.c -# We separate out the tableopts _SOURCES variables and the linking -# rules for those programs because automake has no way to specify such -# things with a loop in a variable (even though make can do such -# things) and the resultant list is both long an unenlightening. And -# it can be / is generated by a shell script, tableopts.sh. - -$(srcdir)/tableopts.am: tableopts.sh - $(SHELL) $(srcdir)/tableopts.sh > $(srcdir)/tableopts.am - -include $(srcdir)/tableopts.am - -tableopts := -Ca -Ce -Cf -CF -Cm -Cem -Cae -Caef -CaeF -Cam -Caem -tableopts_opt_tests := $(foreach opt,$(tableopts), tableopts_opt_nr$(opt) tableopts_opt_r$(opt)) -tableopts_sertests := $(foreach opt,$(tableopts), tableopts_ser_nr$(opt) tableopts_ser_r$(opt)) -tableopts_vertests := $(foreach opt,$(tableopts), tableopts_ver_nr$(opt) tableopts_ver_r$(opt)) -tableopts_tests := $(tableopts_opttests) $(tableopts_vertests) $(tableopts_sertests) - -tableopts_c := $(addsuffix .c,$(tableopts_tests)) - -OPT_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh -AM_OPT_LOG_FLAGS = -d $(srcdir) -i $(srcdir)/tableopts.txt -r - -tableopts_opt_nr%.c: tableopts.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -P $(subst -,_,$(basename $(*F))) $(*:_F=F) -o $@ $< +# Build rules for non-C back ends -tableopts_opt_nr%.$(OBJEXT): tableopts_opt_nr%.c - $(AM_V_CC)$(COMPILE) -c -o $@ $< +.l.go: + $(AM_V_LEX)$(FLEX) $(TESTOPTS) -o $@ $< -tableopts_opt_r%.c: tableopts.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -P $(subst -,_,$(basename $(*F))) --reentrant $(subst _F,F,$*) -o $@ $< - -tableopts_opt_r%.$(OBJEXT): tableopts_opt_r%.c - $(AM_V_CC)$(COMPILE) -DTEST_IS_REENTRANT -c -o $@ $< - -SER_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh -AM_SER_LOG_FLAGS = -d $(builddir) -i $(srcdir)/tableopts.txt -r -t - -tableopts_ser_nr%.c: tableopts.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -P $(subst -,_,$(basename $(*F))) --tables-file="tableopts_ser_nr$*.ser.tables" $(subst _F,F,$*) -o $@ $< - -tableopts_ser_nr%.$(OBJEXT): tableopts_ser_nr%.c - $(AM_V_CC)$(COMPILE) -DTEST_HAS_TABLES_EXTERNAL -c -o $@ $< - -tableopts_ser_r%.c: tableopts.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -P $(subst -,_,$(basename $(*F))) -R --tables-file="tableopts_ser_r$*.ser.tables" $(subst _F,F,$*) -o $@ $< - -tableopts_ser_r%.$(OBJEXT): tableopts_ser_r%.c - $(AM_V_CC)$(COMPILE) -DTEST_HAS_TABLES_EXTERNAL -DTEST_IS_REENTRANT -c -o $@ $< - -VER_LOG_COMPILER = $(SHELL) $(srcdir)/testwrapper.sh -AM_VER_LOG_FLAGS = -d $(builddir) -i $(srcdir)/tableopts.txt -r -t +# This is a temporary fake rule for use while the Go back end still +# actually generates C. +.go: + $(CC) $< -o $*_go -tableopts_ver_nr%.c: tableopts.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -P $(subst -,_,$(basename $(*F))) --tables-file="tableopts_ver_nr$*.ver.tables" --tables-verify $(subst _F,F,$*) -o $@ $< +# Most test productions can be autogenerated from ruleset files, but +# automake has no way to specify such things with a loop in a variable +# (even though make can do such things) and the resultant list is both +# long and unenlightening. And it can be / is generated by a shell +# script, ruleset.sh. -tableopts_ver_nr%.$(OBJEXT): tableopts_ver_nr%.c - $(AM_V_CC)$(COMPILE) -DTEST_HAS_TABLES_EXTERNAL -c -o $@ $< +$(srcdir)/ruleset.am: $(srcdir)/ruleset.sh $(srcdir)/*.rules + $(SHELL) $(srcdir)/ruleset.sh nr r c99 go > $(srcdir)/ruleset.am -tableopts_ver_nr%.ver$(EXEEXT): tableopts_ver_nr%.$(OBJEXT) - $(AM_V_CCLD)$(LINK) -o $@ $^ +include $(srcdir)/ruleset.am -tableopts_ver_r%.c: tableopts.l4 $(FLEX) - $(AM_V_LEX)$(FLEX) --unsafe-no-m4-sect3-escape -P $(subst -,_,$(basename $(*F))) -R --tables-file="tableopts_ver_r$*.ver.tables" --tables-verify $(subst _F,F,$*) -o $@ $< -tableopts_ver_r%.$(OBJEXT): tableopts_ver_r%.c - $(AM_V_CC)$(COMPILE) -DTEST_HAS_TABLES_EXTERNAL -DTEST_IS_REENTRANT -c -o $@ $< diff --git a/tests/README b/tests/README index f1b9ba0..f837261 100644 --- a/tests/README +++ b/tests/README @@ -8,14 +8,14 @@ this directory and its contents. * STRUCTURE OF THE TEST SUITE -The testsuite consists of several tests. Each test is centered around -a scanner known to work with the most recent version of flex. In -general, after you modify your copy of the flex distribution, you -should re-run the test suite. Some of the tests may require certain -tools to be available (e.g., bison, diff). If any test returns an -error or generates an error message, then your modifications *may* -have broken a feature of flex. At a minimum, you'll want to -investigate the failure and determine if it's truly significant. +Each test is centered around a scanner known to work with the most +recent version of flex. In general, after you modify your copy of the +flex distribution, you should re-run the test suite. Some of the tests +may require certain tools to be available (e.g., bison, diff). If any +test returns an error or generates an error message, then your +modifications *may* have broken a feature of flex. At a minimum, +you'll want to investigate the failure and determine if it's truly +significant. * HOW TO RUN THE TEST SUITE @@ -34,20 +34,27 @@ To build and execute a single test: * HOW TO ADD A NEW TEST TO THE TEST SUITE -** List your test in the TESTS variable in Makefile.am in this - directory. Note that due to the large number of tests, we use +** If possible, just write a *.rules file. Exit with status 1 to + indicate failure, e.g. with a "." production if nothing + prior properly consumed every input token. The test machinery + will expand a rules file into tests for all back ends. Note that + text following ### at the end of the ruleset will be used as input. + +** Otherwise, list your test in the TESTS variable in Makefile.am in + this directory. Note that due to the large number of tests, we use variables to group similar tests together. This also helps with handling the automake test suite requirements. Hopefully your test can be listed in SIMPLE_TESTS. You'll need to add the appropriate - automake _SOURCES variable as well. If you're unsure, then consult + automake _SOURCES variable as well, and .gitignore lines for the + binary and generated code. If you're unsure, then consult the automake manual, paying attention to the parallel test harness section. ** On success, your test should return zero. ** On error, your test should return 1 (one) and print a message to -stderr, which will have been redirected to the log file created by the -automake test suite harness. + stderr, which will have been redirected to the log file created by the + automake test suite harness. ** If your test is skipped (e.g., because bison was not found), then the test should return 77 (seventy-seven). This is the exit status that @@ -56,3 +63,43 @@ automake test suite harness. ** Once your work is done, submit a patch via the flex development mailing list, the github pull request mechanism or some other suitable means. + +* NAMING CONVENTIONS + +A test with an _nr suffix exercises a non-reentrant scanner built +with the default cpp back end. + +A test with an _r suffix exercises a reentrant scanner built +with the default cpp back end. + +A test with a c99 suffix exercises the c99 back end. All C99 +scanners are re-entrant. + +Most tests occur in groups with a common stem in the names, like +alloc_extra_ or debug_. These are exercising the same token grammar +under different back ends. As new target languages are added these +groups of parallel tests will grow. Tests that are not part of one of +these series are usually of features supported on the default cpp +back end only. + +The "generated" tests are made by wrapping boilerplate code around a +rules file. The assumption is that the test binary should consume the +data following ### in the rules file, exercising some set of Flex +features and not returning a nonzero status. + +If you can express your test as a ruleset within in this orotocol, +please do so. Those tests are easy to port to new back ends, as that +whole job can be done by enhabcing testmaker.m4. + +* WHY SOME TESTS ARE MISSING + +The "top" test is backend-independent; what it's really testing +is Flex's ability to accumulate and ship preamble code sections. + +The "quotes" test is also backend-indepedent; it's testing that +m4 expansion doesn't leak through the genertated scanner. + +The C99 backend is missing tests for the Bison bridge, header +generation, and loadable tables because it omits those features in +order to be a simpler starting point for wring new back ends. + diff --git a/tests/alloc_extra_c99.l b/tests/alloc_extra_c99.l new file mode 100644 index 0000000..d99a4ca --- /dev/null +++ b/tests/alloc_extra_c99.l @@ -0,0 +1,123 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +%{ +/* A file to build "scanner.c". */ +/* This tests that we can use "yyextra". + We buffer all input into a growable array, then print it. + We run diff on the input and output. +*/ + +#include <stdio.h> +#include <stdlib.h> +#include "config.h" + + +/* We'll store the entire input in this buffer, growing as necessary. */ +struct Check { + char foo; + char *bar; + char qux; +}; + +void *yyalloc(size_t, yyscan_t); + +/* Save char into junk array at next position. */ +static void check_extra ( yyscan_t scanner ); + +%} + +%option emit="c99" +%option 8bit +%option nounput nomain noyywrap nodefault noinput +%option warn +%option extra-type="struct Check *" +%option noyyalloc + + +%% + +.|\r|\n { check_extra (yyscanner); } + +%% + +int main(void); + +int +main (void) +{ + yyscan_t scanner; + struct Check check; + + check.foo = 'a'; + check.bar = NULL; + check.qux = 'z'; + + yylex_init_extra(&check, &scanner); + yyset_in(stdin, scanner); + yyset_out(stdout, scanner); + + /* Test to confirm that testalloc was called from + * testlex_init_extra with the testextra argument. + */ + check_extra(scanner); + + yylex(scanner); + + yylex_destroy(scanner); + return 0; +} + +/* Replaces the stock yyalloc */ +void *yyalloc(size_t size, yyscan_t scanner) +{ + struct Check *check; + check = yyget_extra(scanner); + + if (!check->bar) + check->bar = "Hello World"; + + check_extra(scanner); + + return malloc(size); +} + +/* Save char into junk array at next position. */ +static void check_extra(yyscan_t scanner) +{ + struct Check *check; + check = yyget_extra(scanner); + + if (check->foo != 'a') { + fprintf(stderr, "foo is not 'a'\n"); + exit(1); + } + if (strcmp(check->bar, "Hello World") != 0) { + fprintf(stderr, "bar is not Hello World\n"); + exit(1); + } + if (check->qux != 'z') { + fprintf(stderr, "qux is not 'z'\n"); + exit(1); + } +} diff --git a/tests/alloc_extra.l b/tests/alloc_extra_nr.l index c974777..c974777 100644 --- a/tests/alloc_extra.l +++ b/tests/alloc_extra_nr.l diff --git a/tests/array_nr.l b/tests/array.rules index f4f0f1b..0880805 100644 --- a/tests/array_nr.l +++ b/tests/array.rules @@ -1,5 +1,4 @@ -/* - * This file is part of flex. +/* This file is part of flex. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions @@ -20,36 +19,13 @@ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE. */ - -%{ -/* A template scanner file to build "scanner.c". */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" -/*#include "parser.h" */ - -%} - -%option 8bit prefix="test" +%option 8bit %option nounput nomain noyywrap noinput %option warn array - - %% .|\n { } - -%% - -int main (void); - -int -main (void) -{ - yyin = stdin; - yyout = stdout; - yylex(); - printf("TEST RETURNING OK.\n"); - return 0; -} +### +0000 foo 1111 foo 0000 bar +0000 foo 1111 foo 0000 bar diff --git a/tests/array_nr.txt b/tests/array_nr.txt deleted file mode 100644 index 7288a40..0000000 --- a/tests/array_nr.txt +++ /dev/null @@ -1,2 +0,0 @@ -0000 foo 1111 foo 0000 bar -0000 foo 1111 foo 0000 bar diff --git a/tests/array_r.txt b/tests/array_r.txt deleted file mode 100644 index 7288a40..0000000 --- a/tests/array_r.txt +++ /dev/null @@ -1,2 +0,0 @@ -0000 foo 1111 foo 0000 bar -0000 foo 1111 foo 0000 bar diff --git a/tests/basic_nr.l b/tests/basic.rules index bbe87c4..7ba254a 100644 --- a/tests/basic_nr.l +++ b/tests/basic.rules @@ -1,20 +1,20 @@ /* * This file is part of flex. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: - * + * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. - * + * * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR @@ -22,7 +22,6 @@ */ /* TEST scanner. - Basic non-reentrant scanner. Sample Input: # this is a comment @@ -30,11 +29,7 @@ bar = "string value" integer = 43 */ -%{ -#include "config.h" -%} -%option prefix="test" %option nounput noyywrap noyylineno warn nodefault noinput IDENT [[:alnum:]_-] @@ -46,19 +41,11 @@ WS [[:blank:]] ^{IDENT}+{WS}*={WS}*[[:digit:]]+{WS}*\r?\n { return 102;} ^{WS}*#.*\r?\n { } ^{WS}*\r?\n { } -.|\n { fprintf(stderr,"Invalid line.\n"); exit(-1);} +.|\n {M4_TEST_FAILMESSAGE} -%% - -int main(void); +### +# this is a comment +foo = "bar" +num = 43 +setting = false -int main (void) -{ - yyin = stdin; - yyout = stdout; - while( yylex() ) - { - } - printf("TEST RETURNING OK.\n"); - return 0; -} diff --git a/tests/basic_nr.txt b/tests/basic_nr.txt deleted file mode 100644 index 642e0fb..0000000 --- a/tests/basic_nr.txt +++ /dev/null @@ -1,5 +0,0 @@ -# this is a comment -foo = "bar" -num = 43 -setting = false - diff --git a/tests/basic_r.l b/tests/basic_r.l deleted file mode 100644 index 43d3a88..0000000 --- a/tests/basic_r.l +++ /dev/null @@ -1,68 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -/* A reentrant scanner. - This file will not compile under flex version <= 2.5.4. - Sample Input: - # this is a comment - foo = true - bar = "string value" - integer = 43 -*/ -%{ -#include "config.h" -%} - -%option prefix="test" -%option nounput noyywrap noyylineno warn nodefault noinput -%option reentrant - -IDENT [[:alnum:]_-] -WS [[:blank:]] -%% - -^{IDENT}+{WS}*={WS}*(true|false){WS}*\r?\n { return 100;} -^{IDENT}+{WS}*={WS}*\"[^\"\n\r]*\"{WS}*\r?\n { return 101;} -^{IDENT}+{WS}*={WS}*[[:digit:]]+{WS}*\r?\n { return 102;} -^{WS}*#.*\r?\n { } -^{WS}*\r?\n { } -.|\n { fprintf(stderr,"Invalid line.\n"); exit(-1);} - -%% - -int main(void); - -int main (void) -{ - yyscan_t lexer; - testlex_init( &lexer ); - testset_out ( stdout,lexer); - testset_in ( stdin, lexer); - while( testlex(lexer) ) - { - } - testlex_destroy( lexer ); - printf("TEST RETURNING OK.\n"); - return 0; -} - diff --git a/tests/basic_r.txt b/tests/basic_r.txt deleted file mode 100644 index 2160628..0000000 --- a/tests/basic_r.txt +++ /dev/null @@ -1,4 +0,0 @@ -# this is a comment -foo = "bar" -num = 43 -setting = false diff --git a/tests/bol.rules b/tests/bol.rules new file mode 100644 index 0000000..f9031ae --- /dev/null +++ b/tests/bol.rules @@ -0,0 +1,35 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +/* Test BOL matching, + * If the ^ operator does not constrain pattern patching, these rules + * will false-match on after :bar: the line of input and fail. + */ + +%option nounput noyywrap noyylineno warn nodefault noinput +%% +^foo {M4_TEST_FAILMESSAGE} +.|\n {} + +### +barfoo diff --git a/tests/ccl.l b/tests/ccl.l deleted file mode 100644 index 9511604..0000000 --- a/tests/ccl.l +++ /dev/null @@ -1,95 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -/* A template scanner file to build "scanner.c". */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" -/*#include "parser.h" */ - -#define err_abort() do{printf("ERROR: flex line %d. input line %d.\n", __LINE__, yylineno); abort();} while(0) -#define a_ok() do{printf("OK: flex line %d. input line %d.\n", __LINE__, yylineno); return 1;}while(0) -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput -%option warn - - -%% - -^"^alpha:"[[:^alpha:]]+@alpha@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^digit:"[[:^digit:]]+@digit@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^alnum:"[[:^alnum:]]+@alnum@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^upper:"[[:^upper:]]+@upper@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^lower:"[[:^lower:]]+@lower@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^space:"[[:^space:]]+@space@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^blank:"[[:^blank:]]+@blank@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^punct:"[[:^punct:]]+@punct@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^cntrl:"[[:^cntrl:]]+@cntrl@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^xdigit:"[[:^xdigit:]]+@xdigit@\n printf("OK: %s", yytext); ++yylineno; return 1; - -^"a-d:"[[:alpha:]]{-}[[:digit:]]+@a-d@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"l-xyz:"([[:lower:]]{-}[xyz])+@l-xyz@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"abcd-bc:"([abcd]{-}[bc])+@abcd-bc@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"abcde-b-c:"([abcde]{-}[b]{-}[c])+@abcde-b-c@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"^XY-^XYZ:"([^XY]{-}[^XYZ])+@^XY-^XYZ@\n printf("OK: %s", yytext); ++yylineno; return 1; - -^"a+d:"([[:alpha:]]{+}[[:digit:]])+"@a+d@"\n a_ok(); -^"a-u+Q:"([[:alpha:]]{-}[[:upper:]]{+}[Q])+"@a-u+Q@"\n a_ok(); - -^"ia:"(?i:a)+@ia@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"iabc:"(?i:abc)+@iabc@\n printf("OK: %s", yytext); ++yylineno; return 1; -^"ia-c:"(?i:[a-c]+)@ia-c@\n printf("OK: %s", yytext); ++yylineno; return 1; - - /* We don't want this one to match. */ -^"check-a:"(?i:(?-i:A))@\n err_abort(); -^"check-a:"(?i:(?-i:(?i:A)))@\n printf("OK: %s", yytext); ++yylineno; return 1; - - /* We don't want this one to match. */ -^"dot-all-1:"(?-s:XXX.*)@dot-all-1@\n err_abort(); -^"dot-all-1:"(?s:XXX.*)@dot-all-1@\n a_ok(); - -^"x1:"(?x: a | b )+@x1@\n a_ok(); -^"x2:"(?x: a | - (?# Comment ) - b - )+@x2@\n a_ok(); - - -.|\n { err_abort(); } -%% - -int main(void); - -int -main (void) -{ - yyin = stdin; - yyout = stdout; - while (yylex()) - ; - printf("TEST RETURNING OK.\n"); - return 0; -} diff --git a/tests/ccl.rules b/tests/ccl.rules new file mode 100644 index 0000000..b34d8ef --- /dev/null +++ b/tests/ccl.rules @@ -0,0 +1,99 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ +/* + * Test character-class matching + */ + +%option 8bit +%option nounput nomain noyywrap noinput +%option warn +%% + +^"^alpha:"[[:^alpha:]]+@alpha@\n {M4_TEST_DO(yyecho())} +^"^digit:"[[:^digit:]]+@digit@\n {M4_TEST_DO(yyecho())} +^"^alnum:"[[:^alnum:]]+@alnum@\n {M4_TEST_DO(yyecho())} +^"^upper:"[[:^upper:]]+@upper@\n {M4_TEST_DO(yyecho())} +^"^lower:"[[:^lower:]]+@lower@\n {M4_TEST_DO(yyecho())} +^"^space:"[[:^space:]]+@space@\n {M4_TEST_DO(yyecho())} +^"^blank:"[[:^blank:]]+@blank@\n {M4_TEST_DO(yyecho())} +^"^punct:"[[:^punct:]]+@punct@\n {M4_TEST_DO(yyecho())} +^"^cntrl:"[[:^cntrl:]]+@cntrl@\n {M4_TEST_DO(yyecho())} +^"^xdigit:"[[:^xdigit:]]+@xdigit@\n {M4_TEST_DO(yyecho())} + +^"a-d:"[[:alpha:]]{-}[[:digit:]]+@a-d@\n {M4_TEST_DO(yyecho())} +^"l-xyz:"([[:lower:]]{-}[xyz])+@l-xyz@\n {M4_TEST_DO(yyecho())} +^"abcd-bc:"([abcd]{-}[bc])+@abcd-bc@\n {M4_TEST_DO(yyecho())} +^"abcde-b-c:"([abcde]{-}[b]{-}[c])+@abcde-b-c@\n {M4_TEST_DO(yyecho())} +^"^XY-^XYZ:"([^XY]{-}[^XYZ])+@^XY-^XYZ@\n {M4_TEST_DO(yyecho())} + +^"a+d:"([[:alpha:]]{+}[[:digit:]])+"@a+d@"\n {M4_TEST_DO(yyecho())} +^"a-u+Q:"([[:alpha:]]{-}[[:upper:]]{+}[Q])+"@a-u+Q@"\n {M4_TEST_DO(yyecho())} + +^"ia:"(?i:a)+@ia@\n {M4_TEST_DO(yyecho())} +^"iabc:"(?i:abc)+@iabc@\n {M4_TEST_DO(yyecho())} +^"ia-c:"(?i:[a-c]+)@ia-c@\n {M4_TEST_DO(yyecho())} + + /* We don't want this one to match. */ +^"check-a:"(?i:(?-i:A))@\n {M4_TEST_FAILMESSAGE} +^"check-a:"(?i:(?-i:(?i:A)))@\n {M4_TEST_DO(yyecho())} + + /* We don't want this one to match. */ +^"dot-all-1:"(?-s:XXX.*)@dot-all-1@\n {M4_TEST_FAILMESSAGE} +^"dot-all-1:"(?s:XXX.*)@dot-all-1@\n {M4_TEST_DO(yyecho())} + +^"x1:"(?x: a | b )+@x1@\n {M4_TEST_DO(yyecho())} +^"x2:"(?x: a | + (?# Comment ) + b + )+@x2@\n {M4_TEST_DO(yyecho())} + + +.|\n {M4_TEST_FAILMESSAGE} +### +^alpha:0123456789 ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@alpha@ +^digit:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@digit@ +^alnum:~!@#$%^&*(){}[]':;"<>,./?\+=_-`@alnum@ +^upper:abcdefghijklmnopqrstuvwxyz0123456789 ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@upper@ +^lower:ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@lower@ +^space:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF~!@#$%^&*(){}[]':;"<>,./?\+=_-`@space@ +^blank:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF~!@#$%^&*(){}[]':;"<>,./?\+=_-`@blank@ +^punct:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF Z@punct@ +^cntrl:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF~!@#$%^&*(){}[]':;"<>,./?\+=_-`@cntrl@ +^xdigit:ghijklmnopqrstuvwxyzGHIJKLMNOPQRSTUVWXYZ ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@xdigit@ +a-d:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ@a-d@ +l-xyz:abcdefghijklmnopqrstuvw@l-xyz@ +abcd-bc:aaaaddddaaaa@abcd-bc@ +abcde-b-c:aaaaddddeeee@abcde-b-c@ +^XY-^XYZ:ZZZZZZZZZZZ@^XY-^XYZ@ +a+d:abc0123xyz789@a+d@ +a-u+Q:abcQQQQxyz@a-u+Q@ +ia:AaAa@ia@ +iabc:ABCabcAbCaBc@iabc@ +ia-c:ABCabcAbCaBc@ia-c@ +check-a:a@ +dot-all-1:XXX junk + junk + junk + @dot-all-1@ +x1:abaabb@x1@ +x2:abaabb@x2@ diff --git a/tests/ccl.txt b/tests/ccl.txt deleted file mode 100644 index b318fe6..0000000 --- a/tests/ccl.txt +++ /dev/null @@ -1,27 +0,0 @@ -^alpha:0123456789 ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@alpha@ -^digit:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@digit@ -^alnum:~!@#$%^&*(){}[]':;"<>,./?\+=_-`@alnum@ -^upper:abcdefghijklmnopqrstuvwxyz0123456789 ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@upper@ -^lower:ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@lower@ -^space:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF~!@#$%^&*(){}[]':;"<>,./?\+=_-`@space@ -^blank:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF~!@#$%^&*(){}[]':;"<>,./?\+=_-`@blank@ -^punct:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF Z@punct@ -^cntrl:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789ABCDEF~!@#$%^&*(){}[]':;"<>,./?\+=_-`@cntrl@ -^xdigit:ghijklmnopqrstuvwxyzGHIJKLMNOPQRSTUVWXYZ ~!@#$%^&*(){}[]':;"<>,./?\+=_-`@xdigit@ -a-d:abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ@a-d@ -l-xyz:abcdefghijklmnopqrstuvw@l-xyz@ -abcd-bc:aaaaddddaaaa@abcd-bc@ -abcde-b-c:aaaaddddeeee@abcde-b-c@ -^XY-^XYZ:ZZZZZZZZZZZ@^XY-^XYZ@ -a+d:abc0123xyz789@a+d@ -a-u+Q:abcQQQQxyz@a-u+Q@ -ia:AaAa@ia@ -iabc:ABCabcAbCaBc@iabc@ -ia-c:ABCabcAbCaBc@ia-c@ -check-a:a@ -dot-all-1:XXX junk - junk - junk - @dot-all-1@ -x1:abaabb@x1@ -x2:abaabb@x2@ diff --git a/tests/debug_nr.l b/tests/debug.rules index 8110ab4..5c5813d 100644 --- a/tests/debug_nr.l +++ b/tests/debug.rules @@ -20,33 +20,15 @@ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE. */ +define(`M4_TEST_ENABLEDEBUG', `') -%{ -/* A template scanner file to build "scanner.c". */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -%} - -%option 8bit prefix="test" +%option 8bit %option nounput nomain noyywrap noinput %option warn debug %% .+ { } \n { } -%% - -int main(void); - -int -main (void) -{ - yyin = stdin; - yyout = stdout; - yy_flex_debug = 1; - yylex(); - printf("TEST RETURNING OK.\n"); - return 0; -} +### +Any input will do for this test. +We are only testing if it actually runs in debug mode. diff --git a/tests/debug_nr.txt b/tests/debug_nr.txt deleted file mode 100644 index 8d6476c..0000000 --- a/tests/debug_nr.txt +++ /dev/null @@ -1,2 +0,0 @@ -Any input will do for this test. -We are only testing if it actually runs in debug mode. diff --git a/tests/debug_r.txt b/tests/debug_r.txt deleted file mode 100644 index 8d6476c..0000000 --- a/tests/debug_r.txt +++ /dev/null @@ -1,2 +0,0 @@ -Any input will do for this test. -We are only testing if it actually runs in debug mode. diff --git a/tests/extended.l b/tests/extended.rules index 9691307..6f2d03e 100644 --- a/tests/extended.l +++ b/tests/extended.rules @@ -20,54 +20,30 @@ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE. */ +/* Output should match the input. */ -%{ -/* This test is for correctness of extended (?...) patterns. */ - -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput -%option warn - - +%option 8bit nounput nomain noyywrap noinput warn %% - /* Output should match the input. */ - -abc(?# Single Line Comment )def ECHO; +abc(?# Single Line Comment )def {M4_TEST_DO(yyecho())} ghi(?# multi-line comment - )jkl ECHO; + )jkl {M4_TEST_DO(yyecho())} mno(?# multi-line // comment with ## ~~!@#$ %^&*(@-_+=\|,.<>/ ?: ; punctuation - )pqr ECHO; -(?# Start of a rule.)stu ECHO; -vwxyz(?#End of a rule.) ECHO; + )pqr {M4_TEST_DO(yyecho())} +(?# Start of a rule.)stu {M4_TEST_DO(yyecho())} +vwxyz(?#End of a rule.) {M4_TEST_DO(yyecho())} A(?x: B /* comment */ - C D) ECHO; - -\n ECHO; -%% - -int main(void); + C D) {M4_TEST_DO(yyecho())} -int -main (void) -{ - yyin = stdin; - yyout = stdout; - yylex(); - //printf("TEST RETURNING OK.\n"); - return 0; -} +\n {M4_TEST_DO(yyecho())} +### +abcdefghijklmnopqrstuvwxyz +ABCD diff --git a/tests/extended.txt b/tests/extended.txt deleted file mode 100644 index 829e23d..0000000 --- a/tests/extended.txt +++ /dev/null @@ -1,2 +0,0 @@ -abcdefghijklmnopqrstuvwxyz -ABCD diff --git a/tests/fixedtrailing.rules b/tests/fixedtrailing.rules new file mode 100644 index 0000000..9389688 --- /dev/null +++ b/tests/fixedtrailing.rules @@ -0,0 +1,42 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +/* + * Test rules with variable trailing context. + * Input should match first rule, but only + * consume the "foo". Therefore the second + * rule should not match, but the third should. + */ + +%option nounput noyywrap noyylineno warn nodefault noinput + +%% + +foo/a+ {M4_TEST_ASSERT(yyleng== 3)} +foo {M4_TEST_FAILMESSAGE} +aa { } +\n { } +. {M4_TEST_FAILMESSAGE} + +### +fooaa diff --git a/tests/flexname.rules b/tests/flexname.rules new file mode 100644 index 0000000..eb18943 --- /dev/null +++ b/tests/flexname.rules @@ -0,0 +1,38 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +/* + * Test Lex incompatibility. This is the compatibility exanmple from + * the documentation; with !lex-compat it *should* match the string "foo". + * For the complementery test, see lexcompat.rules. + */ +%option nounput nomain noyywrap noinput +%option warn nolex-compat + +NAME [A-Z][A-Z0-9]* +%% +foo{NAME}? {M4_TEST_DO(yyecho())} +\n {M4_TEST_DO(yyecho())} +. {M4_TEST_FAILMESSAGE} +### +foo diff --git a/tests/include_by_buffer.direct.l b/tests/include_by_buffer.direct.l index 79ff01c..0be055c 100644 --- a/tests/include_by_buffer.direct.l +++ b/tests/include_by_buffer.direct.l @@ -41,28 +41,28 @@ int error = 0; %{ #define MAX_INCLUDE_DEPTH 10 -YY_BUFFER_STATE include_stack[MAX_INCLUDE_DEPTH]; +yybuffer include_stack[MAX_INCLUDE_DEPTH]; int include_stack_ptr = 0; %} %% <INITIAL>{ -^"#include"[[:blank:]]+"<" { BEGIN(GET_FILENAME); } -.|\n { ECHO; } +^"#include"[[:blank:]]+"<" { yybegin(GET_FILENAME); } +.|\n { yyecho(); } } <GET_FILENAME>{ [[:alnum:]_.-]+> { /* recurse */ yytext[yyleng-1]='\0'; - include_stack[include_stack_ptr++] = YY_CURRENT_BUFFER; + include_stack[include_stack_ptr++] = yy_current_buffer(); if((yyin=fopen(yytext,"r"))==NULL) { fprintf(stderr,"*** Error: Could not open include file \"%s\".\n",yytext); error = 1; yyterminate(); } yy_switch_to_buffer( yy_create_buffer( yyin, YY_BUF_SIZE )); - BEGIN(0); + yybegin(0); } .|\n { fprintf(stderr,"Invalid input \"%s\".\n", yytext); @@ -77,7 +77,7 @@ int include_stack_ptr = 0; } else { fclose(yyin); - yy_delete_buffer( YY_CURRENT_BUFFER ); + yy_delete_buffer( yy_current_buffer() ); yy_switch_to_buffer( include_stack[include_stack_ptr] ); } } diff --git a/tests/include_by_push.direct.l b/tests/include_by_push.direct.l index 8aaea69..6f1b4ea 100644 --- a/tests/include_by_push.direct.l +++ b/tests/include_by_push.direct.l @@ -42,8 +42,8 @@ int error = 0; %% <INITIAL>{ -^"#include"[[:blank:]]+"<" { BEGIN(GET_FILENAME); } -.|\n { ECHO; } +^"#include"[[:blank:]]+"<" { yybegin(GET_FILENAME); } +.|\n { yyecho(); } } <GET_FILENAME>{ @@ -56,7 +56,7 @@ int error = 0; yyterminate(); } testpush_buffer_state( yy_create_buffer( yyin, YY_BUF_SIZE )); - BEGIN(0); + yybegin(0); } .|\n { fprintf(stderr,"Invalid input \"%s\".\n", yytext); @@ -68,7 +68,7 @@ int error = 0; <<EOF>> { fclose(yyin); testpop_buffer_state(); - if(!YY_CURRENT_BUFFER) + if(!yy_current_buffer()) yyterminate(); } diff --git a/tests/include_by_reentrant.direct.l b/tests/include_by_reentrant.direct.l index 7dbad72..544dd2f 100644 --- a/tests/include_by_reentrant.direct.l +++ b/tests/include_by_reentrant.direct.l @@ -43,8 +43,8 @@ int error = 0; %% <INITIAL>{ -^"#include"[[:blank:]]+"<" { BEGIN(GET_FILENAME); } -.|\n { ECHO; } +^"#include"[[:blank:]]+"<" { yybegin(GET_FILENAME); } +.|\n { yyecho(); } } <GET_FILENAME>{ @@ -65,7 +65,7 @@ int error = 0; testlex(scanner); testlex_destroy(scanner); - BEGIN(0); + yybegin(0); } .|\n { fprintf(stderr,"Invalid input \"%s\".\n", yytext); diff --git a/tests/lexcompat.rules b/tests/lexcompat.rules new file mode 100644 index 0000000..78ba6e1 --- /dev/null +++ b/tests/lexcompat.rules @@ -0,0 +1,38 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +/* + * Test Lex compatibility. This is the compatibility example from + * the documentation; with lex-compat it should *not* match the string + * "foo". For the complementary test, see flexname.rules. + */ +%option nounput nomain noyywrap noinput +%option warn lex-compat + +NAME [A-Z][A-Z0-9]* +%% +foo{NAME}? {M4_TEST_FAILMESSAGE} +\n {M4_TEST_DO(yyecho())} +. {M4_TEST_DO(yyecho())} +### +foo diff --git a/tests/lineno.rules b/tests/lineno.rules new file mode 100644 index 0000000..3cdfcaa --- /dev/null +++ b/tests/lineno.rules @@ -0,0 +1,71 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +/* The goal here is to test the yylineno processing by: + * - providing some rules than CAN match newlines and + * other rules that can NOT match newlines, + * - matching several newlines in one rule, + * - directly modifying yylineno. + * FIXME: Exposes a bug, probably in the Flex scanner. + * If "--yylineno" in the action is "yylineno--", the pattern + * is not recognized and the magic rewrite rule for yylineo + * does not fire. Only an issue for non-default back ends that + * do magic rewrites rather than rtelying on the C preprocessor. + */ +%option 8bit +%option nounput nomain noyywrap noinput yylineno +%option warn + +WORD [[:alpha:]]+ +DIGIT [[:digit:]] + +%% +"yylineno++" M4_TEST_INCREMENT(yylineno); +"yylineno--" M4_TEST_DECREMENT(yylineno); +[[:blank:]]+ +{WORD} +{DIGIT}+(\n{DIGIT}+)* +\n +. +<<EOF>> {M4_TEST_ASSERT(yylineno == 20) M4_TEST_DO(yyterminate())} + +### +These words +are separated +by newlines +and sometimes + spaces +too. +The next three lines are numbers with only intervening newlines +01123 +581321 +34 +And now for some blank lines.... + + +Finally, we directly modify yylineno, but then change it back afterwards +(see scanner.l): + +yylineno++ + +yylineno-- diff --git a/tests/lineno_nr.l b/tests/lineno_nr.l deleted file mode 100644 index dbe6b2d..0000000 --- a/tests/lineno_nr.l +++ /dev/null @@ -1,94 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -/* A template scanner file to build "scanner.c". - Run as: - test-lineno-nr # report flex's yylineno - test-lineno-nr 1 # report count_newlines(stdin) -*/ - -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput yylineno -%option warn - -WORD [[:alpha:]]+ -DIGIT [[:digit:]] - -%% - /* The goal here is to test the yylineno processing by: - - providing some rules than CAN match newlines and - other rules that can NOT match newlines, - - matching several newlines in one rule, - - directly modifying yylineno. - */ - -"yylineno++" yylineno++; -"yylineno--" yylineno--; -[[:blank:]]+ -{WORD} -{DIGIT}+(\n{DIGIT}+)* -\n -. -<<EOF>> { printf("%d\n", yylineno); - yyterminate(); - } - -%% - -/* returns number of '\n' characters in input, plus one. - This is what flex does, essentially. */ - -static int -count_newlines (FILE* in) -{ - int n=1,c; - while ((c=fgetc(in)) != EOF) - if( c == '\n') - n++; - return n; -} - -int main ( int, char**); - -int -main ( int argc, char **argv ) -{ - (void)argv; - - if( argc > 1 ) - printf("%d\n", count_newlines(stdin)); - - else{ - yyin = stdin; - yyout = stdout; - yylex(); - } - return 0; -} diff --git a/tests/lineno_nr.one.txt b/tests/lineno_nr.one.txt deleted file mode 100644 index c1eb961..0000000 --- a/tests/lineno_nr.one.txt +++ /dev/null @@ -1,19 +0,0 @@ -These words -are separated -by newlines -and sometimes - spaces -too. -The next three lines are numbers with only intervening newlines -01123 -581321 -34 -And now for some blank lines.... - - -Finally, we directly modify yylineno, but then change it back afterwards -(see scanner.l): - -yylineno++ - -yylineno-- diff --git a/tests/lineno_r.l b/tests/lineno_r.l deleted file mode 100644 index d7230d7..0000000 --- a/tests/lineno_r.l +++ /dev/null @@ -1,97 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -/* A template scanner file to build "scanner.c". - Run as: - test-lineno-r # report flex's yylineno - test-lineno-r 1 # report count_newlines(stdin) -*/ - -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput yylineno reentrant -%option warn - -WORD [[:alpha:]]+ -DIGIT [[:digit:]] - -%% - /* The goal here is to test the yylineno processing by: - - providing some rules than CAN match newlines and - other rules that can NOT match newlines, - - matching several newlines in one rule, - - directly modifying yylineno. - */ - -"yylineno++" yylineno++; -"yylineno--" yylineno--; -[[:blank:]]+ -{WORD} -{DIGIT}+(\n{DIGIT}+)* -\n -. -<<EOF>> { printf("%d\n", yylineno); - yyterminate(); - } - -%% - -/* returns number of '\n' characters in input, plus one. - This is what flex does, essentially. */ - -static int -count_newlines (FILE* in) -{ - int n=1,c; - while ((c=fgetc(in)) != EOF) - if( c == '\n') - n++; - return n; -} - -int main ( int argc, char** argv ); - -int -main (int argc, char **argv) -{ - (void)argv; - - if( argc > 1 ) - printf("%d\n", count_newlines(stdin)); - - else{ - yyscan_t s; - testlex_init(&s); - testset_in(stdin,s); - testset_out(stdout,s); - testlex(s); - testlex_destroy(s); - } - return 0; -} diff --git a/tests/lineno_r.one.txt b/tests/lineno_r.one.txt deleted file mode 100644 index c1eb961..0000000 --- a/tests/lineno_r.one.txt +++ /dev/null @@ -1,19 +0,0 @@ -These words -are separated -by newlines -and sometimes - spaces -too. -The next three lines are numbers with only intervening newlines -01123 -581321 -34 -And now for some blank lines.... - - -Finally, we directly modify yylineno, but then change it back afterwards -(see scanner.l): - -yylineno++ - -yylineno-- diff --git a/tests/lineno_trailing.l b/tests/lineno_trailing.l deleted file mode 100644 index cceaa23..0000000 --- a/tests/lineno_trailing.l +++ /dev/null @@ -1,87 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -/* A template scanner file to build "scanner.c". - Run as: - test-lineno-trailing # report flex's yylineno - test-lineno-trailing 1 # report count_newlines(stdin) -*/ - -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput yylineno -%option warn - -WORD [[:alpha:]]+ - -%% - /* The goal here is to test the yylineno in the context of trailing-contexts. - Using rules that have newlines in look-ahead. - */ -"Fixed_trailing:"/[\n]"test"[\n] {} -"Var_trailing:"{WORD}/[\n] {} -"Var_prefix_and_trailing:"{WORD}":"/(\n{WORD})* {} -\n {} -. {} -<<EOF>> { printf("%d\n", yylineno); - yyterminate(); - } - -%% - -/* returns number of '\n' characters in input, plus one. - This is what flex does, essentially. */ - -static int -count_newlines (FILE* in) -{ - int n=1,c; - while ((c=fgetc(in)) != EOF) - if( c == '\n') - n++; - return n; -} - -int main ( int, char**); - -int -main ( int argc, char **argv ) -{ - (void)argv; - - if( argc > 1 ) - printf("%d\n", count_newlines(stdin)); - - else{ - yyin = stdin; - yyout = stdout; - yylex(); - } - return 0; -} diff --git a/tests/lineno_trailing.one.txt b/tests/lineno_trailing.one.txt deleted file mode 100644 index 201164d..0000000 --- a/tests/lineno_trailing.one.txt +++ /dev/null @@ -1,13 +0,0 @@ -We are testing rules with trailing contexts containing newlines (see scanner.l): - -Fixed_trailing: -test - -Var_trailing:word -test - -Var_prefix_and_trailing:word: -more -text -comes -here diff --git a/tests/mem_nr.txt b/tests/mem.txt index 79aa16a..79aa16a 100644 --- a/tests/mem_nr.txt +++ b/tests/mem.txt diff --git a/tests/mem_c99.l b/tests/mem_c99.l new file mode 100644 index 0000000..d6ec025 --- /dev/null +++ b/tests/mem_c99.l @@ -0,0 +1,191 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +%{ +/* A template scanner file to build "scanner.c". + * The whole idea is to cause memory realloc by + * 1. pushing a lot on the condition stack, and + * 2. eating input greater than YY_BUF_SIZE + */ +#include <stdio.h> +#include <stdlib.h> +#include "config.h" + +/* Insanely small read buffer. This pretty much guarantees at least one realloc. */ +#ifdef YY_BUF_SIZE +#undef YY_BUF_SIZE +#endif +#define YY_BUF_SIZE 8 + +void * yyalloc(size_t, yyscan_t); +void *yyrealloc ( void *, size_t, yyscan_t yyscanner ); +void yyfree ( void *, yyscan_t yyscanner ); +%} + +%option emit="c99" bufsize=8 +%option 8bit +%option nounput nomain noyywrap noinput noyy_top_state +%option warn stack nodefault +%option noyyalloc noyyrealloc noyyfree + +%x parens + +%% + +<INITIAL>{ +"(" { printf("yy_push_state(parens)\n"); yy_push_state(parens,yyscanner); } +len=[0-9]+ { printf("About read token where %s\n",yytext); } +0+ { } +.|\n { } +} + +<parens>{ +"(" { printf("yy_push_state(parens)\n"); yy_push_state(parens,yyscanner); } +")" { printf("yy_pop_state()\n");yy_pop_state(yyscanner);} +[^()\n]+ { } +.|\n { } +} + +%% +/* total memory allocated */ +static size_t total_mem=0; + +/* track the amount of memory for ptr. */ +struct memsz { + void* p; + size_t sz; +}; + +static struct memsz * ptrs=0; /* Array of pairs. */ +static int nptrs=0; /* Number of pairs in array. */ +static int arrsz=0; /* Capacity of array. */ + +static void dump_mem(FILE* fp){ + int i; + fprintf(fp,"\tptrs[%d] = {", nptrs); + for (i=0; i < arrsz; i++) + fprintf(fp," {%#lx,%ld},", (long)ptrs[i].p, (long)ptrs[i].sz); + + fprintf(fp,"}\n"); +} + +void * yyalloc(size_t n , yyscan_t yyscanner) +{ + (void)yyscanner; + + void * p; + int i; + + total_mem += n; + p = malloc(n); + + if( nptrs >= arrsz){ + /* increase array size by 1 */ + arrsz++; + ptrs = realloc(ptrs, (size_t) arrsz * sizeof(struct memsz)); + ptrs[nptrs].p = 0; + ptrs[nptrs].sz = 0; + } + + /* find a null slot */ + for(i=0; i < arrsz ; i++) + if (ptrs[i].p == 0) { + ptrs[i].p = p; + ptrs[i].sz = n; + } + + nptrs++; + printf("yyflex_alloc(%8ld) total=%8ld return=%#10lx\n",(long)n,(long)total_mem,(long)p); + dump_mem(stdout); + return p; +} + +void * yyrealloc(void* p, size_t n , yyscan_t yyscanner) +{ + (void)yyscanner; + + int i; + for (i=0; i < arrsz; i++) + if ( ptrs[i].p == p){ + total_mem -= ptrs[i].sz; + total_mem += n; + ptrs[i].p = realloc(p, n); + ptrs[i].sz = n; + + printf("yyflex_realloc(%#10lx,%8ld) total=%8ld return=%8lx\n", + (long)p,(long)n,(long)total_mem,(long)ptrs[i].p); + dump_mem(stdout); + return ptrs[i].p; + } + + fprintf(stderr,"ERROR: yyflex_realloc could not locate pointer %#lx.\n",(long)p); + dump_mem(stdout); + exit(1); +} + +void yyfree(void* p , yyscan_t yyscanner) +{ + (void)yyscanner; + + int i; + for (i=0; i < arrsz; i++) + if ( ptrs[i].p == p){ + total_mem -= ptrs[i].sz; + free(p); + ptrs[i].p = 0; + ptrs[i].sz = 0; + nptrs--; + printf("yyflex_free(%#10lx) total=%8ld\n",(long)p,(long)total_mem); + dump_mem(stdout); + return; + } + + fprintf(stderr,"ERROR: yyflex_free could not locate pointer %#lx.\n",(long)p); + dump_mem(stdout); + exit(1); +} + +int main(void); + +int +main (void) +{ + yyscan_t scanner; + arrsz = 1; + ptrs = calloc(1, sizeof(struct memsz)); + nptrs = 0; + + yylex_init(&scanner); + yyset_in(stdin,scanner); + yyset_out(stdout,scanner); + yylex(scanner); + yylex_destroy(scanner); + free(ptrs); + + if ( nptrs > 0 || total_mem > 0){ + fprintf(stderr,"Oops. Looks like a memory leak: nptrs=%d, unfreed memory=%ld\n",nptrs,(long)total_mem); + exit(1); + } + printf("TEST RETURNING OK.\n"); + return 0; +} diff --git a/tests/mem_r.txt b/tests/mem_r.txt deleted file mode 100644 index 79aa16a..0000000 --- a/tests/mem_r.txt +++ /dev/null @@ -1,25 +0,0 @@ -First we push a lot on the stack by nesting parenthesis: - -(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((( -(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((( -((((((((((((((((((((((((((((((((((((((((((( - -(should be 200 states pushed here) - -)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) -)))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))) -))))))))))))))))))))))))))))))))))))))))))) - -Now we match progressively bigger tokens to increase the read buffer: - -len=1 0 -len=2 00 -len=4 0000 -len=8 00000000 -len=16 0000000000000000 -len=32 00000000000000000000000000000000 -len=64 0000000000000000000000000000000000000000000000000000000000000000 -len=128 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 -len=256 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 -len=512 00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 -len=1024 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 diff --git a/tests/multiple_scanners_nr_main.c b/tests/multiple_scanners_nr_main.c index 497bd9f..6bb5a89 100644 --- a/tests/multiple_scanners_nr_main.c +++ b/tests/multiple_scanners_nr_main.c @@ -31,7 +31,7 @@ main ( int argc, char** argv ) (void)argv; int S1_ok=1, S2_ok=1; - YY_BUFFER_STATE buff1, buff2; + yybuffer buff1, buff2; S1_out = S2_out = stdout; buff1 = S1__scan_string("foo on bar off"); buff2 = S2__scan_string("on blah blah off foo on bar off"); diff --git a/tests/multiple_scanners_r_main.c b/tests/multiple_scanners_r_main.c index 9c3fa8d..f35ecbb 100644 --- a/tests/multiple_scanners_r_main.c +++ b/tests/multiple_scanners_r_main.c @@ -31,7 +31,7 @@ main ( int argc, char** argv ) (void)argv; int S1_ok=1, S2_ok=1; - YY_BUFFER_STATE buff1, buff2; + yybuffer buff1, buff2; yyscan_t scan1, scan2; S1_lex_init(&scan1); diff --git a/tests/posix.l b/tests/posix.l deleted file mode 100644 index f1851d8..0000000 --- a/tests/posix.l +++ /dev/null @@ -1,79 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -/* The goal of this test is to verify that we are getting the counter-intuitive - * posix behavior of the repeat operator `{}'. - * - * ab{3} - In traditional flex, this matches "abbb". - * In posix, this matches "ababab". - */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -#define NUM_TESTS 1 -char * tests[NUM_TESTS] = { "ababab"}; -int main(void); - -int tests_ok[NUM_TESTS] = { 0 }; - -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput -%option warn posix-compat - - -%% - -ab{3} tests_ok[0] = 1; return 0; -.|\n return 0; - -%% - - -int main (void) -{ - YY_BUFFER_STATE state; - int i; - - yyin = stdin; - yyout = stdout; - - /* Run the tests */ - for (i=0; i < NUM_TESTS; i++){ - printf("Testing: test_scan_string(%s): ", tests[i]); - state = test_scan_string(tests[i]); - testlex(); - yy_delete_buffer(state); - printf("... %s\n", tests_ok[i] ? "OK" : "FAILED"); - } - - for (i=0; i < NUM_TESTS; i++) - if (!tests_ok[i]) - exit(1); - - printf("TEST RETURNING OK.\n"); - return 0; -} diff --git a/tests/array_r.l b/tests/posix.rules index 68a6299..f38ea5d 100644 --- a/tests/array_r.l +++ b/tests/posix.rules @@ -1,62 +1,41 @@ /* * This file is part of flex. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: - * + * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. - * + * * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE. */ -%{ -/* A template scanner file to build "scanner.c". */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" -/*#include "parser.h" */ - -%} - -%option 8bit prefix="test" +/* The goal of this test is to verify that we are getting the counter-intuitive + * POSIX behavior of the repeat operator `{}' when posix-compat is on. + * + * ab{3} - In traditional flex, this matches "abbb". + * In posix, this matches "ababab". + * The input file should contain just the second string. + */ %option nounput nomain noyywrap noinput -%option warn array reentrant - +%option warn posix-compat %% -.|\n { } - - -%% - -int main(void); - -int -main (void) -{ - yyscan_t lexer; - - yylex_init(&lexer); - yyset_in(stdin, lexer); - yyset_out(stdout, lexer); - - yylex( lexer ); - - yylex_destroy( lexer); - printf("TEST RETURNING OK.\n"); +ab{3} { } +\n { } +. {M4_TEST_FAILMESSAGE} - return 0; -} +### +ababab diff --git a/tests/posixly_correct.l b/tests/posixly_correct.l deleted file mode 100644 index 715bb27..0000000 --- a/tests/posixly_correct.l +++ /dev/null @@ -1,79 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -/* The goal of this test is to verify that we are getting the counter-intuitive - * posix behavior of the repeat operator `{}'. - * - * ab{3} - In traditional flex, this matches "abbb". - * In posix, this matches "ababab". - */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -#define NUM_TESTS 1 -char * tests[NUM_TESTS] = { "ababab"}; -int main(void); - -int tests_ok[NUM_TESTS] = { 0 }; - -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput -%option warn - - -%% - -ab{3} tests_ok[0] = 1; return 0; -.|\n return 0; - -%% - - -int main (void) -{ - YY_BUFFER_STATE state; - int i; - - yyin = stdin; - yyout = stdout; - - /* Run the tests */ - for (i=0; i < NUM_TESTS; i++){ - printf("Testing: test_scan_string(%s): ", tests[i]); - state = test_scan_string(tests[i]); - testlex(); - yy_delete_buffer(state); - printf("... %s\n", tests_ok[i] ? "OK" : "FAILED"); - } - - for (i=0; i < NUM_TESTS; i++) - if (!tests_ok[i]) - exit(1); - - printf("TEST RETURNING OK.\n"); - return 0; -} diff --git a/tests/debug_r.l b/tests/posixlycorrect.rules index 8c2d7a5..51972e3 100644 --- a/tests/debug_r.l +++ b/tests/posixlycorrect.rules @@ -1,59 +1,45 @@ /* * This file is part of flex. - * + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: - * + * * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. - * + * * Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. - * + * * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR * PURPOSE. */ -%{ -/* A template scanner file to build "scanner.c". */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -%} - -%option 8bit prefix="test" +/* + * The goal of this test is to verify that we are getting the + * counter-intuitive POSIX behavior of the repeat operator `{}' when + * posix_compat is off but POSIXLY_CORRECT=1 is set in the + * environment. Note: This depends on the test framework to actually + * set that variable! When it does not, Flex will match abbb + * + * ab{3} - In traditional flex, this matches "abbb". + * In posix, this matches "ababab". + * The input file should contain just the second string. + */ %option nounput nomain noyywrap noinput -%option warn debug reentrant +%option warn %% -.+ { } -\n { } -%% -int main(void); +ab{3} { } +\n { } +. {M4_TEST_FAILMESSAGE} -int main (void) -{ - yyscan_t lexer; - testlex_init( &lexer ); - testset_out ( stdout,lexer); - testset_in ( stdin, lexer); - - /* Just see if the next line compiles. */ - testset_debug (testget_debug(lexer), lexer); - - while( testlex(lexer) ) - { - } - testlex_destroy( lexer ); - printf("TEST RETURNING OK.\n"); - return 0; -} +### +ababab diff --git a/tests/prefix_nr.txt b/tests/prefix.txt index 0e6c88f..0e6c88f 100644 --- a/tests/prefix_nr.txt +++ b/tests/prefix.txt diff --git a/tests/prefix_c99.l b/tests/prefix_c99.l new file mode 100644 index 0000000..aad599f --- /dev/null +++ b/tests/prefix_c99.l @@ -0,0 +1,87 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +%{ +/* Builds "scanner.c". */ +/* The scanner itself is a no-op. A successful compilation is all we want. */ +#include <stdio.h> +#include <stdlib.h> +#include "config.h" + +%} +%option emit="c99" +%option 8bit prefix="FOO" +%option nounput nomain noyywrap noinput +%option warn + + +%% + + +.|\n|\r { + + /* Compile, but do not execute the following code. */ + if( 0 ) { + FOO_create_buffer( (FILE*)0, 0, yyscanner); + FOO_delete_buffer( (yybuffer)0, yyscanner); + FOO_flush_buffer( (yybuffer)0, yyscanner); + FOO_init_buffer( (yybuffer)0, (FILE*)0, yyscanner); + FOO_load_buffer_state( yyscanner); + FOO_scan_buffer( (char*)0, (size_t)0, yyscanner); + FOO_scan_bytes( (const char*)0, 0, yyscanner); + FOO_scan_string( (const char*)0, yyscanner); + FOO_switch_to_buffer( (yybuffer)0, yyscanner); + FOOrestart( (FILE*)0, (yyscan_t )0); + + /* Commented out because the C99 back end is more + * careful about not compiling in things it doesn't + * need. + */ + //FOOget_extra( (yyscan_t )0 ); + FOOget_in( (yyscan_t )0 ); + FOOget_leng( (yyscan_t )0 ); + FOOget_out( (yyscan_t )0 ); + FOOget_text( (yyscan_t )0 ); + FOOlex( (yyscan_t )0 ); + //FOOlex_destroy( (yyscan_t )0 ); + FOOlex_init( (yyscan_t *)0 ); + //FOOset_extra( (void *)0, (yyscan_t )0 ); + FOOset_in( (FILE*)0, (yyscan_t )0 ); + FOOset_out( (FILE*)0, (yyscan_t )0 ); + } + } +%% + +int main(void); + +int +main (void) +{ + yyscan_t scanner; + FOOlex_init( &scanner); + FOOlex( scanner); + FOOlex_destroy( scanner); + printf( "TEST RETURNING OK.\n"); + return 0; +} + diff --git a/tests/prefix_nr.l b/tests/prefix_nr.l index 2208ae0..e163789 100644 --- a/tests/prefix_nr.l +++ b/tests/prefix_nr.l @@ -42,14 +42,14 @@ /* Compile, but do not execute the following code. */ if( 0) { FOO_create_buffer((FILE*)0,0); - FOO_delete_buffer((YY_BUFFER_STATE)0); - FOO_flush_buffer((YY_BUFFER_STATE)0); - FOO_init_buffer((YY_BUFFER_STATE)0,(FILE*)0); + FOO_delete_buffer((yybuffer)0); + FOO_flush_buffer((yybuffer)0); + FOO_init_buffer((yybuffer)0,(FILE*)0); FOO_load_buffer_state(); FOO_scan_buffer((char*)0,(yy_size_t)0); - FOO_scan_bytes((yyconst char*)0, 0); - FOO_scan_string((yyconst char*)0); - FOO_switch_to_buffer((YY_BUFFER_STATE)0); + FOO_scan_bytes((const char*)0, 0); + FOO_scan_string((const char*)0); + FOO_switch_to_buffer((yybuffer)0); yyin = (FILE*)0; yyout = (FILE*)0; yyleng = 0; diff --git a/tests/prefix_r.l b/tests/prefix_r.l index 210a4de..c7f9746 100644 --- a/tests/prefix_r.l +++ b/tests/prefix_r.l @@ -43,14 +43,14 @@ /* Compile, but do not execute the following code. */ if( 0 ) { FOO_create_buffer( (FILE*)0, 0, yyscanner); - FOO_delete_buffer( (YY_BUFFER_STATE)0, yyscanner); - FOO_flush_buffer( (YY_BUFFER_STATE)0, yyscanner); - FOO_init_buffer( (YY_BUFFER_STATE)0, (FILE*)0, yyscanner); + FOO_delete_buffer( (yybuffer)0, yyscanner); + FOO_flush_buffer( (yybuffer)0, yyscanner); + FOO_init_buffer( (yybuffer)0, (FILE*)0, yyscanner); FOO_load_buffer_state( yyscanner); FOO_scan_buffer( (char*)0, (yy_size_t)0, yyscanner); - FOO_scan_bytes( (yyconst char*)0, 0, yyscanner); - FOO_scan_string( (yyconst char*)0, yyscanner); - FOO_switch_to_buffer( (YY_BUFFER_STATE)0, yyscanner); + FOO_scan_bytes( (const char*)0, 0, yyscanner); + FOO_scan_string( (const char*)0, yyscanner); + FOO_switch_to_buffer( (yybuffer)0, yyscanner); FOOrestart( (FILE*)0, (yyscan_t )0); FOOget_extra( (yyscan_t )0 ); diff --git a/tests/prefix_r.txt b/tests/prefix_r.txt deleted file mode 100644 index 0e6c88f..0000000 --- a/tests/prefix_r.txt +++ /dev/null @@ -1 +0,0 @@ -Dummy input. diff --git a/tests/preposix.rules b/tests/preposix.rules new file mode 100644 index 0000000..b43a00b --- /dev/null +++ b/tests/preposix.rules @@ -0,0 +1,42 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +/* The goal of this test is to verify that we are getting the traditional + * pre-POSIX behavior of the repeat operator `{}' when posix-compat is not on + * and POSIXLY_CORRECT is not set to 1. + * + * ab{3} - In traditional flex, this matches "abbb". + * In posix, this matches "ababab". + * The input file should contain just the first string. + */ +%option nounput nomain noyywrap noinput +%option warn + +%% + +ab{3} { } +\n { } +. {M4_TEST_FAILMESSAGE} + +### +abbb diff --git a/tests/pthread.l b/tests/pthread.l index eb99778..b7e8fa7 100644 --- a/tests/pthread.l +++ b/tests/pthread.l @@ -61,14 +61,14 @@ static int process_text(char* s, yyscan_t scanner); #define NUMBER 200 #define WORD 201 -<INITIAL>[[:digit:]]+ { BEGIN(STATE_1); process_text(yytext,yyscanner); return NUMBER; } -<INITIAL>[[:alpha:]]+ { BEGIN(STATE_2); process_text(yytext,yyscanner); return WORD; } +<INITIAL>[[:digit:]]+ { yybegin(STATE_1); process_text(yytext,yyscanner); return NUMBER; } +<INITIAL>[[:alpha:]]+ { yybegin(STATE_2); process_text(yytext,yyscanner); return WORD; } -<STATE_1>[[:alpha:]]+ { BEGIN(0); process_text(yytext,yyscanner); return WORD; } -<STATE_1>[[:digit:]]+ { BEGIN(0); process_text(yytext,yyscanner); return NUMBER; } +<STATE_1>[[:alpha:]]+ { yybegin(0); process_text(yytext,yyscanner); return WORD; } +<STATE_1>[[:digit:]]+ { yybegin(0); process_text(yytext,yyscanner); return NUMBER; } -<STATE_2>[[:alpha:]]+ { BEGIN(0); process_text(yytext,yyscanner); return WORD; } -<STATE_2>[[:digit:]]+ { BEGIN(0); process_text(yytext,yyscanner); return NUMBER; } +<STATE_2>[[:alpha:]]+ { yybegin(0); process_text(yytext,yyscanner); return WORD; } +<STATE_2>[[:digit:]]+ { yybegin(0); process_text(yytext,yyscanner); return NUMBER; } <INITIAL,STATE_1,STATE_2>" "|\t|\r|\n { process_text(yytext,yyscanner); } <INITIAL,STATE_1,STATE_2>[^[:alnum:][:space:]\t\r\n] { diff --git a/tests/quote_in_comment.l b/tests/quote_in_comment.l deleted file mode 100644 index a5743c2..0000000 --- a/tests/quote_in_comment.l +++ /dev/null @@ -1,16 +0,0 @@ -%option 8bit noyywrap -%% -.|\n { ECHO; - //' " - } -%% -int -main (void) -{ - yyin = stdin; - yyout = stdout; - while (yylex()) - ; - printf("TEST RETURNING OK.\n"); - return 0; -} diff --git a/tests/quote_in_comment.txt b/tests/quote_in_comment.txt deleted file mode 100644 index 9daeafb..0000000 --- a/tests/quote_in_comment.txt +++ /dev/null @@ -1 +0,0 @@ -test diff --git a/tests/quoteincomment.rules b/tests/quoteincomment.rules new file mode 100644 index 0000000..9aca73b --- /dev/null +++ b/tests/quoteincomment.rules @@ -0,0 +1,29 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ +%option 8bit noyywrap +%% +.|\n { M4_TEST_DO(yyecho()) + //' " + } +### +test diff --git a/tests/reject.l4 b/tests/reject.l4 deleted file mode 100644 index 9bcde22..0000000 --- a/tests/reject.l4 +++ /dev/null @@ -1,82 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" - -%} - -%option 8bit prefix="test" -%option nounput nomain noyywrap noinput -%option warn reject - - -%% - -. { REJECT; } -.|\n ; - -%% - -int main ( int argc, char** argv ) -{ - FILE* fp = NULL; - void *yyscanner=0; - M4_YY_DECL_GUTS_VAR(); - -#ifdef TEST_IS_REENTRANT - testlex_init(&yyscanner); -#else - (void)yyscanner; -#endif - -#ifdef TEST_HAS_TABLES_EXTERNAL - if((fp = fopen(argv[1],"rb"))== NULL) - YY_FATAL_ERROR("could not open tables file for reading"); - - if(yytables_fload(fp M4_YY_CALL_LAST_ARG) < 0) - YY_FATAL_ERROR("yytables_fload returned < 0"); - if(M4_YY_TABLES_VERIFY) - exit(0); -#endif - - if(argc > 2){ - if((fp = fopen(argv[2],"r"))== NULL) - YY_FATAL_ERROR("could not open input file for reading"); - yyin = fp; - } - while(testlex(M4_YY_CALL_ONLY_ARG) != 0) - ; - -#ifdef TEST_HAS_TABLES_EXTERNAL - testtables_destroy(M4_YY_CALL_ONLY_ARG); -#endif - testlex_destroy(M4_YY_CALL_ONLY_ARG); - - if(argc < 0) /* silence the compiler */ - yyscanner = (void*)fp; - - return 0; -} diff --git a/tests/reject.rules b/tests/reject.rules new file mode 100644 index 0000000..b7ed30e --- /dev/null +++ b/tests/reject.rules @@ -0,0 +1,39 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ +/* + * Test yyreject() by seeing if we jump out of the main rule below + * rather than reaching the abort. + */ + +%option 8bit +%option nounput nomain noyywrap noinput +%option warn reject +%% + +. {M4_TEST_DO(yyreject()) M4_TEST_FAILMESSAGE} +.|\n ; + +### +0000 foo 1111 foo 0000 bar +0000 foo 1111 foo 0000 bar + diff --git a/tests/reject.txt b/tests/reject.txt deleted file mode 100644 index 7288a40..0000000 --- a/tests/reject.txt +++ /dev/null @@ -1,2 +0,0 @@ -0000 foo 1111 foo 0000 bar -0000 foo 1111 foo 0000 bar diff --git a/tests/ruleset.am b/tests/ruleset.am new file mode 100644 index 0000000..b4571f1 --- /dev/null +++ b/tests/ruleset.am @@ -0,0 +1,892 @@ + +# Begin generated test rules + +array_nr_SOURCES = array_nr.l +array_nr.l: $(srcdir)/array.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +basic_nr_SOURCES = basic_nr.l +basic_nr.l: $(srcdir)/basic.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +bol_nr_SOURCES = bol_nr.l +bol_nr.l: $(srcdir)/bol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +ccl_nr_SOURCES = ccl_nr.l +ccl_nr.l: $(srcdir)/ccl.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +debug_nr_SOURCES = debug_nr.l +debug_nr.l: $(srcdir)/debug.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +extended_nr_SOURCES = extended_nr.l +extended_nr.l: $(srcdir)/extended.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +fixedtrailing_nr_SOURCES = fixedtrailing_nr.l +fixedtrailing_nr.l: $(srcdir)/fixedtrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +flexname_nr_SOURCES = flexname_nr.l +flexname_nr.l: $(srcdir)/flexname.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +lexcompat_nr_SOURCES = lexcompat_nr.l +lexcompat_nr.l: $(srcdir)/lexcompat.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +lineno_nr_SOURCES = lineno_nr.l +lineno_nr.l: $(srcdir)/lineno.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +posixlycorrect_nr_SOURCES = posixlycorrect_nr.l +posixlycorrect_nr.l: $(srcdir)/posixlycorrect.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +posix_nr_SOURCES = posix_nr.l +posix_nr.l: $(srcdir)/posix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +preposix_nr_SOURCES = preposix_nr.l +preposix_nr.l: $(srcdir)/preposix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +quoteincomment_nr_SOURCES = quoteincomment_nr.l +quoteincomment_nr.l: $(srcdir)/quoteincomment.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +reject_nr_SOURCES = reject_nr.l +reject_nr.l: $(srcdir)/reject.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_nr_SOURCES = tableopts_nr.l +tableopts_nr.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +vartrailing_nr_SOURCES = vartrailing_nr.l +vartrailing_nr.l: $(srcdir)/vartrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyless_nr_SOURCES = yyless_nr.l +yyless_nr.l: $(srcdir)/yyless.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearraybol_nr_SOURCES = yymorearraybol_nr.l +yymorearraybol_nr.l: $(srcdir)/yymorearraybol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearray_nr_SOURCES = yymorearray_nr.l +yymorearray_nr.l: $(srcdir)/yymorearray.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymore_nr_SOURCES = yymore_nr.l +yymore_nr.l: $(srcdir)/yymore.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyunput_nr_SOURCES = yyunput_nr.l +yyunput_nr.l: $(srcdir)/yyunput.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Ca_opt_SOURCES = tableopts_opt_nr-Ca.opt.l +tableopts_opt_nr-Ca.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Ce_opt_SOURCES = tableopts_opt_nr-Ce.opt.l +tableopts_opt_nr-Ce.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Cf_opt_SOURCES = tableopts_opt_nr-Cf.opt.l +tableopts_opt_nr-Cf.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_CxF_opt_SOURCES = tableopts_opt_nr-CxF.opt.l +tableopts_opt_nr-CxF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Cm_opt_SOURCES = tableopts_opt_nr-Cm.opt.l +tableopts_opt_nr-Cm.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Cem_opt_SOURCES = tableopts_opt_nr-Cem.opt.l +tableopts_opt_nr-Cem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Cae_opt_SOURCES = tableopts_opt_nr-Cae.opt.l +tableopts_opt_nr-Cae.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Caef_opt_SOURCES = tableopts_opt_nr-Caef.opt.l +tableopts_opt_nr-Caef.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_CaexF_opt_SOURCES = tableopts_opt_nr-CaexF.opt.l +tableopts_opt_nr-CaexF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Cam_opt_SOURCES = tableopts_opt_nr-Cam.opt.l +tableopts_opt_nr-Cam.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_nr_Caem_opt_SOURCES = tableopts_opt_nr-Caem.opt.l +tableopts_opt_nr-Caem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Ca_ser_SOURCES = tableopts_ser_nr-Ca.ser.l +tableopts_ser_nr-Ca.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Ce_ser_SOURCES = tableopts_ser_nr-Ce.ser.l +tableopts_ser_nr-Ce.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Cf_ser_SOURCES = tableopts_ser_nr-Cf.ser.l +tableopts_ser_nr-Cf.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_CxF_ser_SOURCES = tableopts_ser_nr-CxF.ser.l +tableopts_ser_nr-CxF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Cm_ser_SOURCES = tableopts_ser_nr-Cm.ser.l +tableopts_ser_nr-Cm.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Cem_ser_SOURCES = tableopts_ser_nr-Cem.ser.l +tableopts_ser_nr-Cem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Cae_ser_SOURCES = tableopts_ser_nr-Cae.ser.l +tableopts_ser_nr-Cae.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Caef_ser_SOURCES = tableopts_ser_nr-Caef.ser.l +tableopts_ser_nr-Caef.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_CaexF_ser_SOURCES = tableopts_ser_nr-CaexF.ser.l +tableopts_ser_nr-CaexF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Cam_ser_SOURCES = tableopts_ser_nr-Cam.ser.l +tableopts_ser_nr-Cam.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_nr_Caem_ser_SOURCES = tableopts_ser_nr-Caem.ser.l +tableopts_ser_nr-Caem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Ca_ver_SOURCES = tableopts_ver_nr-Ca.ver.l +tableopts_ver_nr-Ca.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Ce_ver_SOURCES = tableopts_ver_nr-Ce.ver.l +tableopts_ver_nr-Ce.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Cf_ver_SOURCES = tableopts_ver_nr-Cf.ver.l +tableopts_ver_nr-Cf.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_CxF_ver_SOURCES = tableopts_ver_nr-CxF.ver.l +tableopts_ver_nr-CxF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Cm_ver_SOURCES = tableopts_ver_nr-Cm.ver.l +tableopts_ver_nr-Cm.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Cem_ver_SOURCES = tableopts_ver_nr-Cem.ver.l +tableopts_ver_nr-Cem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Cae_ver_SOURCES = tableopts_ver_nr-Cae.ver.l +tableopts_ver_nr-Cae.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Caef_ver_SOURCES = tableopts_ver_nr-Caef.ver.l +tableopts_ver_nr-Caef.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_CaexF_ver_SOURCES = tableopts_ver_nr-CaexF.ver.l +tableopts_ver_nr-CaexF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Cam_ver_SOURCES = tableopts_ver_nr-Cam.ver.l +tableopts_ver_nr-Cam.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_nr_Caem_ver_SOURCES = tableopts_ver_nr-Caem.ver.l +tableopts_ver_nr-Caem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +array_r_SOURCES = array_r.l +array_r.l: $(srcdir)/array.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +basic_r_SOURCES = basic_r.l +basic_r.l: $(srcdir)/basic.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +bol_r_SOURCES = bol_r.l +bol_r.l: $(srcdir)/bol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +ccl_r_SOURCES = ccl_r.l +ccl_r.l: $(srcdir)/ccl.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +debug_r_SOURCES = debug_r.l +debug_r.l: $(srcdir)/debug.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +extended_r_SOURCES = extended_r.l +extended_r.l: $(srcdir)/extended.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +fixedtrailing_r_SOURCES = fixedtrailing_r.l +fixedtrailing_r.l: $(srcdir)/fixedtrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +flexname_r_SOURCES = flexname_r.l +flexname_r.l: $(srcdir)/flexname.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +lineno_r_SOURCES = lineno_r.l +lineno_r.l: $(srcdir)/lineno.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +posix_r_SOURCES = posix_r.l +posix_r.l: $(srcdir)/posix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +preposix_r_SOURCES = preposix_r.l +preposix_r.l: $(srcdir)/preposix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +quoteincomment_r_SOURCES = quoteincomment_r.l +quoteincomment_r.l: $(srcdir)/quoteincomment.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +reject_r_SOURCES = reject_r.l +reject_r.l: $(srcdir)/reject.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_r_SOURCES = tableopts_r.l +tableopts_r.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +vartrailing_r_SOURCES = vartrailing_r.l +vartrailing_r.l: $(srcdir)/vartrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyless_r_SOURCES = yyless_r.l +yyless_r.l: $(srcdir)/yyless.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearraybol_r_SOURCES = yymorearraybol_r.l +yymorearraybol_r.l: $(srcdir)/yymorearraybol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearray_r_SOURCES = yymorearray_r.l +yymorearray_r.l: $(srcdir)/yymorearray.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymore_r_SOURCES = yymore_r.l +yymore_r.l: $(srcdir)/yymore.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyunput_r_SOURCES = yyunput_r.l +yyunput_r.l: $(srcdir)/yyunput.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Ca_opt_SOURCES = tableopts_opt_r-Ca.opt.l +tableopts_opt_r-Ca.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Ce_opt_SOURCES = tableopts_opt_r-Ce.opt.l +tableopts_opt_r-Ce.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Cf_opt_SOURCES = tableopts_opt_r-Cf.opt.l +tableopts_opt_r-Cf.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_CxF_opt_SOURCES = tableopts_opt_r-CxF.opt.l +tableopts_opt_r-CxF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Cm_opt_SOURCES = tableopts_opt_r-Cm.opt.l +tableopts_opt_r-Cm.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Cem_opt_SOURCES = tableopts_opt_r-Cem.opt.l +tableopts_opt_r-Cem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Cae_opt_SOURCES = tableopts_opt_r-Cae.opt.l +tableopts_opt_r-Cae.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Caef_opt_SOURCES = tableopts_opt_r-Caef.opt.l +tableopts_opt_r-Caef.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_CaexF_opt_SOURCES = tableopts_opt_r-CaexF.opt.l +tableopts_opt_r-CaexF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Cam_opt_SOURCES = tableopts_opt_r-Cam.opt.l +tableopts_opt_r-Cam.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_r_Caem_opt_SOURCES = tableopts_opt_r-Caem.opt.l +tableopts_opt_r-Caem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Ca_ser_SOURCES = tableopts_ser_r-Ca.ser.l +tableopts_ser_r-Ca.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Ce_ser_SOURCES = tableopts_ser_r-Ce.ser.l +tableopts_ser_r-Ce.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Cf_ser_SOURCES = tableopts_ser_r-Cf.ser.l +tableopts_ser_r-Cf.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_CxF_ser_SOURCES = tableopts_ser_r-CxF.ser.l +tableopts_ser_r-CxF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Cm_ser_SOURCES = tableopts_ser_r-Cm.ser.l +tableopts_ser_r-Cm.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Cem_ser_SOURCES = tableopts_ser_r-Cem.ser.l +tableopts_ser_r-Cem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Cae_ser_SOURCES = tableopts_ser_r-Cae.ser.l +tableopts_ser_r-Cae.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Caef_ser_SOURCES = tableopts_ser_r-Caef.ser.l +tableopts_ser_r-Caef.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_CaexF_ser_SOURCES = tableopts_ser_r-CaexF.ser.l +tableopts_ser_r-CaexF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Cam_ser_SOURCES = tableopts_ser_r-Cam.ser.l +tableopts_ser_r-Cam.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_r_Caem_ser_SOURCES = tableopts_ser_r-Caem.ser.l +tableopts_ser_r-Caem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Ca_ver_SOURCES = tableopts_ver_r-Ca.ver.l +tableopts_ver_r-Ca.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Ce_ver_SOURCES = tableopts_ver_r-Ce.ver.l +tableopts_ver_r-Ce.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Cf_ver_SOURCES = tableopts_ver_r-Cf.ver.l +tableopts_ver_r-Cf.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_CxF_ver_SOURCES = tableopts_ver_r-CxF.ver.l +tableopts_ver_r-CxF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Cm_ver_SOURCES = tableopts_ver_r-Cm.ver.l +tableopts_ver_r-Cm.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Cem_ver_SOURCES = tableopts_ver_r-Cem.ver.l +tableopts_ver_r-Cem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Cae_ver_SOURCES = tableopts_ver_r-Cae.ver.l +tableopts_ver_r-Cae.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Caef_ver_SOURCES = tableopts_ver_r-Caef.ver.l +tableopts_ver_r-Caef.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_CaexF_ver_SOURCES = tableopts_ver_r-CaexF.ver.l +tableopts_ver_r-CaexF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Cam_ver_SOURCES = tableopts_ver_r-Cam.ver.l +tableopts_ver_r-Cam.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_r_Caem_ver_SOURCES = tableopts_ver_r-Caem.ver.l +tableopts_ver_r-Caem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +array_c99_SOURCES = array_c99.l +array_c99.l: $(srcdir)/array.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +basic_c99_SOURCES = basic_c99.l +basic_c99.l: $(srcdir)/basic.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +bol_c99_SOURCES = bol_c99.l +bol_c99.l: $(srcdir)/bol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +ccl_c99_SOURCES = ccl_c99.l +ccl_c99.l: $(srcdir)/ccl.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +debug_c99_SOURCES = debug_c99.l +debug_c99.l: $(srcdir)/debug.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +extended_c99_SOURCES = extended_c99.l +extended_c99.l: $(srcdir)/extended.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +fixedtrailing_c99_SOURCES = fixedtrailing_c99.l +fixedtrailing_c99.l: $(srcdir)/fixedtrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +flexname_c99_SOURCES = flexname_c99.l +flexname_c99.l: $(srcdir)/flexname.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +lineno_c99_SOURCES = lineno_c99.l +lineno_c99.l: $(srcdir)/lineno.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +posix_c99_SOURCES = posix_c99.l +posix_c99.l: $(srcdir)/posix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +preposix_c99_SOURCES = preposix_c99.l +preposix_c99.l: $(srcdir)/preposix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +quoteincomment_c99_SOURCES = quoteincomment_c99.l +quoteincomment_c99.l: $(srcdir)/quoteincomment.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +reject_c99_SOURCES = reject_c99.l +reject_c99.l: $(srcdir)/reject.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_c99_SOURCES = tableopts_c99.l +tableopts_c99.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +vartrailing_c99_SOURCES = vartrailing_c99.l +vartrailing_c99.l: $(srcdir)/vartrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyless_c99_SOURCES = yyless_c99.l +yyless_c99.l: $(srcdir)/yyless.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearraybol_c99_SOURCES = yymorearraybol_c99.l +yymorearraybol_c99.l: $(srcdir)/yymorearraybol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearray_c99_SOURCES = yymorearray_c99.l +yymorearray_c99.l: $(srcdir)/yymorearray.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymore_c99_SOURCES = yymore_c99.l +yymore_c99.l: $(srcdir)/yymore.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyunput_c99_SOURCES = yyunput_c99.l +yyunput_c99.l: $(srcdir)/yyunput.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Ca_opt_SOURCES = tableopts_opt_c99-Ca.opt.l +tableopts_opt_c99-Ca.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Ce_opt_SOURCES = tableopts_opt_c99-Ce.opt.l +tableopts_opt_c99-Ce.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Cf_opt_SOURCES = tableopts_opt_c99-Cf.opt.l +tableopts_opt_c99-Cf.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_CxF_opt_SOURCES = tableopts_opt_c99-CxF.opt.l +tableopts_opt_c99-CxF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Cm_opt_SOURCES = tableopts_opt_c99-Cm.opt.l +tableopts_opt_c99-Cm.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Cem_opt_SOURCES = tableopts_opt_c99-Cem.opt.l +tableopts_opt_c99-Cem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Cae_opt_SOURCES = tableopts_opt_c99-Cae.opt.l +tableopts_opt_c99-Cae.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Caef_opt_SOURCES = tableopts_opt_c99-Caef.opt.l +tableopts_opt_c99-Caef.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_CaexF_opt_SOURCES = tableopts_opt_c99-CaexF.opt.l +tableopts_opt_c99-CaexF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Cam_opt_SOURCES = tableopts_opt_c99-Cam.opt.l +tableopts_opt_c99-Cam.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_c99_Caem_opt_SOURCES = tableopts_opt_c99-Caem.opt.l +tableopts_opt_c99-Caem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Ca_ser_SOURCES = tableopts_ser_c99-Ca.ser.l +tableopts_ser_c99-Ca.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Ce_ser_SOURCES = tableopts_ser_c99-Ce.ser.l +tableopts_ser_c99-Ce.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Cf_ser_SOURCES = tableopts_ser_c99-Cf.ser.l +tableopts_ser_c99-Cf.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_CxF_ser_SOURCES = tableopts_ser_c99-CxF.ser.l +tableopts_ser_c99-CxF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Cm_ser_SOURCES = tableopts_ser_c99-Cm.ser.l +tableopts_ser_c99-Cm.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Cem_ser_SOURCES = tableopts_ser_c99-Cem.ser.l +tableopts_ser_c99-Cem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Cae_ser_SOURCES = tableopts_ser_c99-Cae.ser.l +tableopts_ser_c99-Cae.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Caef_ser_SOURCES = tableopts_ser_c99-Caef.ser.l +tableopts_ser_c99-Caef.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_CaexF_ser_SOURCES = tableopts_ser_c99-CaexF.ser.l +tableopts_ser_c99-CaexF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Cam_ser_SOURCES = tableopts_ser_c99-Cam.ser.l +tableopts_ser_c99-Cam.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_c99_Caem_ser_SOURCES = tableopts_ser_c99-Caem.ser.l +tableopts_ser_c99-Caem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Ca_ver_SOURCES = tableopts_ver_c99-Ca.ver.l +tableopts_ver_c99-Ca.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Ce_ver_SOURCES = tableopts_ver_c99-Ce.ver.l +tableopts_ver_c99-Ce.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Cf_ver_SOURCES = tableopts_ver_c99-Cf.ver.l +tableopts_ver_c99-Cf.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_CxF_ver_SOURCES = tableopts_ver_c99-CxF.ver.l +tableopts_ver_c99-CxF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Cm_ver_SOURCES = tableopts_ver_c99-Cm.ver.l +tableopts_ver_c99-Cm.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Cem_ver_SOURCES = tableopts_ver_c99-Cem.ver.l +tableopts_ver_c99-Cem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Cae_ver_SOURCES = tableopts_ver_c99-Cae.ver.l +tableopts_ver_c99-Cae.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Caef_ver_SOURCES = tableopts_ver_c99-Caef.ver.l +tableopts_ver_c99-Caef.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_CaexF_ver_SOURCES = tableopts_ver_c99-CaexF.ver.l +tableopts_ver_c99-CaexF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Cam_ver_SOURCES = tableopts_ver_c99-Cam.ver.l +tableopts_ver_c99-Cam.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_c99_Caem_ver_SOURCES = tableopts_ver_c99-Caem.ver.l +tableopts_ver_c99-Caem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +array_go_SOURCES = array_go.l +array_go.l: $(srcdir)/array.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +basic_go_SOURCES = basic_go.l +basic_go.l: $(srcdir)/basic.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +bol_go_SOURCES = bol_go.l +bol_go.l: $(srcdir)/bol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +ccl_go_SOURCES = ccl_go.l +ccl_go.l: $(srcdir)/ccl.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +debug_go_SOURCES = debug_go.l +debug_go.l: $(srcdir)/debug.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +extended_go_SOURCES = extended_go.l +extended_go.l: $(srcdir)/extended.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +fixedtrailing_go_SOURCES = fixedtrailing_go.l +fixedtrailing_go.l: $(srcdir)/fixedtrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +flexname_go_SOURCES = flexname_go.l +flexname_go.l: $(srcdir)/flexname.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +lineno_go_SOURCES = lineno_go.l +lineno_go.l: $(srcdir)/lineno.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +posix_go_SOURCES = posix_go.l +posix_go.l: $(srcdir)/posix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +preposix_go_SOURCES = preposix_go.l +preposix_go.l: $(srcdir)/preposix.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +quoteincomment_go_SOURCES = quoteincomment_go.l +quoteincomment_go.l: $(srcdir)/quoteincomment.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +reject_go_SOURCES = reject_go.l +reject_go.l: $(srcdir)/reject.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_go_SOURCES = tableopts_go.l +tableopts_go.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +vartrailing_go_SOURCES = vartrailing_go.l +vartrailing_go.l: $(srcdir)/vartrailing.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyless_go_SOURCES = yyless_go.l +yyless_go.l: $(srcdir)/yyless.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearraybol_go_SOURCES = yymorearraybol_go.l +yymorearraybol_go.l: $(srcdir)/yymorearraybol.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymorearray_go_SOURCES = yymorearray_go.l +yymorearray_go.l: $(srcdir)/yymorearray.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yymore_go_SOURCES = yymore_go.l +yymore_go.l: $(srcdir)/yymore.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +yyunput_go_SOURCES = yyunput_go.l +yyunput_go.l: $(srcdir)/yyunput.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Ca_opt_SOURCES = tableopts_opt_go-Ca.opt.l +tableopts_opt_go-Ca.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Ce_opt_SOURCES = tableopts_opt_go-Ce.opt.l +tableopts_opt_go-Ce.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Cf_opt_SOURCES = tableopts_opt_go-Cf.opt.l +tableopts_opt_go-Cf.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_CxF_opt_SOURCES = tableopts_opt_go-CxF.opt.l +tableopts_opt_go-CxF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Cm_opt_SOURCES = tableopts_opt_go-Cm.opt.l +tableopts_opt_go-Cm.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Cem_opt_SOURCES = tableopts_opt_go-Cem.opt.l +tableopts_opt_go-Cem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Cae_opt_SOURCES = tableopts_opt_go-Cae.opt.l +tableopts_opt_go-Cae.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Caef_opt_SOURCES = tableopts_opt_go-Caef.opt.l +tableopts_opt_go-Caef.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_CaexF_opt_SOURCES = tableopts_opt_go-CaexF.opt.l +tableopts_opt_go-CaexF.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Cam_opt_SOURCES = tableopts_opt_go-Cam.opt.l +tableopts_opt_go-Cam.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_opt_go_Caem_opt_SOURCES = tableopts_opt_go-Caem.opt.l +tableopts_opt_go-Caem.opt.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Ca_ser_SOURCES = tableopts_ser_go-Ca.ser.l +tableopts_ser_go-Ca.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Ce_ser_SOURCES = tableopts_ser_go-Ce.ser.l +tableopts_ser_go-Ce.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Cf_ser_SOURCES = tableopts_ser_go-Cf.ser.l +tableopts_ser_go-Cf.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_CxF_ser_SOURCES = tableopts_ser_go-CxF.ser.l +tableopts_ser_go-CxF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Cm_ser_SOURCES = tableopts_ser_go-Cm.ser.l +tableopts_ser_go-Cm.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Cem_ser_SOURCES = tableopts_ser_go-Cem.ser.l +tableopts_ser_go-Cem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Cae_ser_SOURCES = tableopts_ser_go-Cae.ser.l +tableopts_ser_go-Cae.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Caef_ser_SOURCES = tableopts_ser_go-Caef.ser.l +tableopts_ser_go-Caef.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_CaexF_ser_SOURCES = tableopts_ser_go-CaexF.ser.l +tableopts_ser_go-CaexF.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Cam_ser_SOURCES = tableopts_ser_go-Cam.ser.l +tableopts_ser_go-Cam.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ser_go_Caem_ser_SOURCES = tableopts_ser_go-Caem.ser.l +tableopts_ser_go-Caem.ser.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Ca_ver_SOURCES = tableopts_ver_go-Ca.ver.l +tableopts_ver_go-Ca.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Ce_ver_SOURCES = tableopts_ver_go-Ce.ver.l +tableopts_ver_go-Ce.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Cf_ver_SOURCES = tableopts_ver_go-Cf.ver.l +tableopts_ver_go-Cf.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_CxF_ver_SOURCES = tableopts_ver_go-CxF.ver.l +tableopts_ver_go-CxF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Cm_ver_SOURCES = tableopts_ver_go-Cm.ver.l +tableopts_ver_go-Cm.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Cem_ver_SOURCES = tableopts_ver_go-Cem.ver.l +tableopts_ver_go-Cem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Cae_ver_SOURCES = tableopts_ver_go-Cae.ver.l +tableopts_ver_go-Cae.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Caef_ver_SOURCES = tableopts_ver_go-Caef.ver.l +tableopts_ver_go-Caef.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_CaexF_ver_SOURCES = tableopts_ver_go-CaexF.ver.l +tableopts_ver_go-CaexF.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Cam_ver_SOURCES = tableopts_ver_go-Cam.ver.l +tableopts_ver_go-Cam.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +tableopts_ver_go_Caem_ver_SOURCES = tableopts_ver_go-Caem.ver.l +tableopts_ver_go-Caem.ver.l: $(srcdir)/tableopts.rules $(srcdir)/testmaker.sh $(srcdir)/testmaker.m4 + $(SHELL) $(srcdir)/testmaker.sh $@ + +posixlycorrect_nr.c: posixlycorrect_nr.l $(FLEX) + $(AM_V_LEX)POSIXLY_CORRECT=1 $(FLEX) $(TESTOPTS) -o $@ $< + +test-yydecl-nr.sh$(EXEEXT): test-yydecl-gen.sh + $(SHELL) test-yydecl-gen.sh nr >test-yydecl-nr.sh$(EXEEXT) + chmod a+x test-yydecl-nr.sh$(EXEEXT) + +posixlycorrect_r.c: posixlycorrect_r.l $(FLEX) + $(AM_V_LEX)POSIXLY_CORRECT=1 $(FLEX) $(TESTOPTS) -o $@ $< + +test-yydecl-r.sh$(EXEEXT): test-yydecl-gen.sh + $(SHELL) test-yydecl-gen.sh r >test-yydecl-r.sh$(EXEEXT) + chmod a+x test-yydecl-r.sh$(EXEEXT) + +posixlycorrect_c99.c: posixlycorrect_c99.l $(FLEX) + $(AM_V_LEX)POSIXLY_CORRECT=1 $(FLEX) $(TESTOPTS) -o $@ $< + +test-yydecl-c99.sh$(EXEEXT): test-yydecl-gen.sh + $(SHELL) test-yydecl-gen.sh c99 >test-yydecl-c99.sh$(EXEEXT) + chmod a+x test-yydecl-c99.sh$(EXEEXT) + +posixlycorrect_go.go: posixlycorrect_go.l $(FLEX) + $(AM_V_LEX)POSIXLY_CORRECT=1 $(FLEX) $(TESTOPTS) -o $@ $< + +test-yydecl-go.sh$(EXEEXT): test-yydecl-gen.sh + $(SHELL) test-yydecl-gen.sh go >test-yydecl-go.sh$(EXEEXT) + chmod a+x test-yydecl-go.sh$(EXEEXT) + + +# End generated test rules +RULESET_TESTS = array_nr basic_nr bol_nr ccl_nr debug_nr extended_nr fixedtrailing_nr flexname_nr lexcompat_nr lineno_nr posixlycorrect_nr posix_nr preposix_nr quoteincomment_nr reject_nr tableopts_nr vartrailing_nr yyless_nr yymorearraybol_nr yymorearray_nr yymore_nr yyunput_nr tableopts_opt_nr-Ca.opt tableopts_opt_nr-Ce.opt tableopts_opt_nr-Cf.opt tableopts_opt_nr-CxF.opt tableopts_opt_nr-Cm.opt tableopts_opt_nr-Cem.opt tableopts_opt_nr-Cae.opt tableopts_opt_nr-Caef.opt tableopts_opt_nr-CaexF.opt tableopts_opt_nr-Cam.opt tableopts_opt_nr-Caem.opt tableopts_ser_nr-Ca.ser tableopts_ser_nr-Ce.ser tableopts_ser_nr-Cf.ser tableopts_ser_nr-CxF.ser tableopts_ser_nr-Cm.ser tableopts_ser_nr-Cem.ser tableopts_ser_nr-Cae.ser tableopts_ser_nr-Caef.ser tableopts_ser_nr-CaexF.ser tableopts_ser_nr-Cam.ser tableopts_ser_nr-Caem.ser tableopts_ver_nr-Ca.ver tableopts_ver_nr-Ce.ver tableopts_ver_nr-Cf.ver tableopts_ver_nr-CxF.ver tableopts_ver_nr-Cm.ver tableopts_ver_nr-Cem.ver tableopts_ver_nr-Cae.ver tableopts_ver_nr-Caef.ver tableopts_ver_nr-CaexF.ver tableopts_ver_nr-Cam.ver tableopts_ver_nr-Caem.ver array_r basic_r bol_r ccl_r debug_r extended_r fixedtrailing_r flexname_r lineno_r posix_r preposix_r quoteincomment_r reject_r tableopts_r vartrailing_r yyless_r yymorearraybol_r yymorearray_r yymore_r yyunput_r tableopts_opt_r-Ca.opt tableopts_opt_r-Ce.opt tableopts_opt_r-Cf.opt tableopts_opt_r-CxF.opt tableopts_opt_r-Cm.opt tableopts_opt_r-Cem.opt tableopts_opt_r-Cae.opt tableopts_opt_r-Caef.opt tableopts_opt_r-CaexF.opt tableopts_opt_r-Cam.opt tableopts_opt_r-Caem.opt tableopts_ser_r-Ca.ser tableopts_ser_r-Ce.ser tableopts_ser_r-Cf.ser tableopts_ser_r-CxF.ser tableopts_ser_r-Cm.ser tableopts_ser_r-Cem.ser tableopts_ser_r-Cae.ser tableopts_ser_r-Caef.ser tableopts_ser_r-CaexF.ser tableopts_ser_r-Cam.ser tableopts_ser_r-Caem.ser tableopts_ver_r-Ca.ver tableopts_ver_r-Ce.ver tableopts_ver_r-Cf.ver tableopts_ver_r-CxF.ver tableopts_ver_r-Cm.ver tableopts_ver_r-Cem.ver tableopts_ver_r-Cae.ver tableopts_ver_r-Caef.ver tableopts_ver_r-CaexF.ver tableopts_ver_r-Cam.ver tableopts_ver_r-Caem.ver array_c99 basic_c99 bol_c99 ccl_c99 debug_c99 extended_c99 fixedtrailing_c99 flexname_c99 lineno_c99 posix_c99 preposix_c99 quoteincomment_c99 reject_c99 tableopts_c99 vartrailing_c99 yyless_c99 yymorearraybol_c99 yymorearray_c99 yymore_c99 yyunput_c99 tableopts_opt_c99-Ca.opt tableopts_opt_c99-Ce.opt tableopts_opt_c99-Cf.opt tableopts_opt_c99-CxF.opt tableopts_opt_c99-Cm.opt tableopts_opt_c99-Cem.opt tableopts_opt_c99-Cae.opt tableopts_opt_c99-Caef.opt tableopts_opt_c99-CaexF.opt tableopts_opt_c99-Cam.opt tableopts_opt_c99-Caem.opt tableopts_ser_c99-Ca.ser tableopts_ser_c99-Ce.ser tableopts_ser_c99-Cf.ser tableopts_ser_c99-CxF.ser tableopts_ser_c99-Cm.ser tableopts_ser_c99-Cem.ser tableopts_ser_c99-Cae.ser tableopts_ser_c99-Caef.ser tableopts_ser_c99-CaexF.ser tableopts_ser_c99-Cam.ser tableopts_ser_c99-Caem.ser tableopts_ver_c99-Ca.ver tableopts_ver_c99-Ce.ver tableopts_ver_c99-Cf.ver tableopts_ver_c99-CxF.ver tableopts_ver_c99-Cm.ver tableopts_ver_c99-Cem.ver tableopts_ver_c99-Cae.ver tableopts_ver_c99-Caef.ver tableopts_ver_c99-CaexF.ver tableopts_ver_c99-Cam.ver tableopts_ver_c99-Caem.ver array_go basic_go bol_go ccl_go debug_go extended_go fixedtrailing_go flexname_go lineno_go posix_go preposix_go quoteincomment_go reject_go tableopts_go vartrailing_go yyless_go yymorearraybol_go yymorearray_go yymore_go yyunput_go tableopts_opt_go-Ca.opt tableopts_opt_go-Ce.opt tableopts_opt_go-Cf.opt tableopts_opt_go-CxF.opt tableopts_opt_go-Cm.opt tableopts_opt_go-Cem.opt tableopts_opt_go-Cae.opt tableopts_opt_go-Caef.opt tableopts_opt_go-CaexF.opt tableopts_opt_go-Cam.opt tableopts_opt_go-Caem.opt tableopts_ser_go-Ca.ser tableopts_ser_go-Ce.ser tableopts_ser_go-Cf.ser tableopts_ser_go-CxF.ser tableopts_ser_go-Cm.ser tableopts_ser_go-Cem.ser tableopts_ser_go-Cae.ser tableopts_ser_go-Caef.ser tableopts_ser_go-CaexF.ser tableopts_ser_go-Cam.ser tableopts_ser_go-Caem.ser tableopts_ver_go-Ca.ver tableopts_ver_go-Ce.ver tableopts_ver_go-Cf.ver tableopts_ver_go-CxF.ver tableopts_ver_go-Cm.ver tableopts_ver_go-Cem.ver tableopts_ver_go-Cae.ver tableopts_ver_go-Caef.ver tableopts_ver_go-CaexF.ver tableopts_ver_go-Cam.ver tableopts_ver_go-Caem.ver test-yydecl-nr.sh test-yydecl-r.sh test-yydecl-c99.sh test-yydecl-go.sh +RULESET_REMOVABLES = array_nr array_nr.c array_nr.l array.txt basic_nr basic_nr.c basic_nr.l basic.txt bol_nr bol_nr.c bol_nr.l bol.txt ccl_nr ccl_nr.c ccl_nr.l ccl.txt debug_nr debug_nr.c debug_nr.l debug.txt extended_nr extended_nr.c extended_nr.l extended.txt fixedtrailing_nr fixedtrailing_nr.c fixedtrailing_nr.l fixedtrailing.txt flexname_nr flexname_nr.c flexname_nr.l flexname.txt lexcompat_nr lexcompat_nr.c lexcompat_nr.l lexcompat.txt lineno_nr lineno_nr.c lineno_nr.l lineno.txt posixlycorrect_nr posixlycorrect_nr.c posixlycorrect_nr.l posixlycorrect.txt posix_nr posix_nr.c posix_nr.l posix.txt preposix_nr preposix_nr.c preposix_nr.l preposix.txt quoteincomment_nr quoteincomment_nr.c quoteincomment_nr.l quoteincomment.txt reject_nr reject_nr.c reject_nr.l reject.txt tableopts_nr tableopts_nr.c tableopts_nr.l tableopts.txt vartrailing_nr vartrailing_nr.c vartrailing_nr.l vartrailing.txt yyless_nr yyless_nr.c yyless_nr.l yyless.txt yymorearraybol_nr yymorearraybol_nr.c yymorearraybol_nr.l yymorearraybol.txt yymorearray_nr yymorearray_nr.c yymorearray_nr.l yymorearray.txt yymore_nr yymore_nr.c yymore_nr.l yymore.txt yyunput_nr yyunput_nr.c yyunput_nr.l yyunput.txt tableopts_opt_nr-Ca.opt tableopts_opt_nr-Ca.opt.c tableopts_opt_nr-Ca.opt.l tableopts_opt_nr-Ca.opt.tables tableopts_opt_nr-Ce.opt tableopts_opt_nr-Ce.opt.c tableopts_opt_nr-Ce.opt.l tableopts_opt_nr-Ce.opt.tables tableopts_opt_nr-Cf.opt tableopts_opt_nr-Cf.opt.c tableopts_opt_nr-Cf.opt.l tableopts_opt_nr-Cf.opt.tables tableopts_opt_nr-CxF.opt tableopts_opt_nr-CxF.opt.c tableopts_opt_nr-CxF.opt.l tableopts_opt_nr-CxF.opt.tables tableopts_opt_nr-Cm.opt tableopts_opt_nr-Cm.opt.c tableopts_opt_nr-Cm.opt.l tableopts_opt_nr-Cm.opt.tables tableopts_opt_nr-Cem.opt tableopts_opt_nr-Cem.opt.c tableopts_opt_nr-Cem.opt.l tableopts_opt_nr-Cem.opt.tables tableopts_opt_nr-Cae.opt tableopts_opt_nr-Cae.opt.c tableopts_opt_nr-Cae.opt.l tableopts_opt_nr-Cae.opt.tables tableopts_opt_nr-Caef.opt tableopts_opt_nr-Caef.opt.c tableopts_opt_nr-Caef.opt.l tableopts_opt_nr-Caef.opt.tables tableopts_opt_nr-CaexF.opt tableopts_opt_nr-CaexF.opt.c tableopts_opt_nr-CaexF.opt.l tableopts_opt_nr-CaexF.opt.tables tableopts_opt_nr-Cam.opt tableopts_opt_nr-Cam.opt.c tableopts_opt_nr-Cam.opt.l tableopts_opt_nr-Cam.opt.tables tableopts_opt_nr-Caem.opt tableopts_opt_nr-Caem.opt.c tableopts_opt_nr-Caem.opt.l tableopts_opt_nr-Caem.opt.tables tableopts_ser_nr-Ca.ser tableopts_ser_nr-Ca.ser.c tableopts_ser_nr-Ca.ser.l tableopts_ser_nr-Ca.ser.tables tableopts_ser_nr-Ce.ser tableopts_ser_nr-Ce.ser.c tableopts_ser_nr-Ce.ser.l tableopts_ser_nr-Ce.ser.tables tableopts_ser_nr-Cf.ser tableopts_ser_nr-Cf.ser.c tableopts_ser_nr-Cf.ser.l tableopts_ser_nr-Cf.ser.tables tableopts_ser_nr-CxF.ser tableopts_ser_nr-CxF.ser.c tableopts_ser_nr-CxF.ser.l tableopts_ser_nr-CxF.ser.tables tableopts_ser_nr-Cm.ser tableopts_ser_nr-Cm.ser.c tableopts_ser_nr-Cm.ser.l tableopts_ser_nr-Cm.ser.tables tableopts_ser_nr-Cem.ser tableopts_ser_nr-Cem.ser.c tableopts_ser_nr-Cem.ser.l tableopts_ser_nr-Cem.ser.tables tableopts_ser_nr-Cae.ser tableopts_ser_nr-Cae.ser.c tableopts_ser_nr-Cae.ser.l tableopts_ser_nr-Cae.ser.tables tableopts_ser_nr-Caef.ser tableopts_ser_nr-Caef.ser.c tableopts_ser_nr-Caef.ser.l tableopts_ser_nr-Caef.ser.tables tableopts_ser_nr-CaexF.ser tableopts_ser_nr-CaexF.ser.c tableopts_ser_nr-CaexF.ser.l tableopts_ser_nr-CaexF.ser.tables tableopts_ser_nr-Cam.ser tableopts_ser_nr-Cam.ser.c tableopts_ser_nr-Cam.ser.l tableopts_ser_nr-Cam.ser.tables tableopts_ser_nr-Caem.ser tableopts_ser_nr-Caem.ser.c tableopts_ser_nr-Caem.ser.l tableopts_ser_nr-Caem.ser.tables tableopts_ver_nr-Ca.ver tableopts_ver_nr-Ca.ver.c tableopts_ver_nr-Ca.ver.l tableopts_ver_nr-Ca.ver.tables tableopts_ver_nr-Ce.ver tableopts_ver_nr-Ce.ver.c tableopts_ver_nr-Ce.ver.l tableopts_ver_nr-Ce.ver.tables tableopts_ver_nr-Cf.ver tableopts_ver_nr-Cf.ver.c tableopts_ver_nr-Cf.ver.l tableopts_ver_nr-Cf.ver.tables tableopts_ver_nr-CxF.ver tableopts_ver_nr-CxF.ver.c tableopts_ver_nr-CxF.ver.l tableopts_ver_nr-CxF.ver.tables tableopts_ver_nr-Cm.ver tableopts_ver_nr-Cm.ver.c tableopts_ver_nr-Cm.ver.l tableopts_ver_nr-Cm.ver.tables tableopts_ver_nr-Cem.ver tableopts_ver_nr-Cem.ver.c tableopts_ver_nr-Cem.ver.l tableopts_ver_nr-Cem.ver.tables tableopts_ver_nr-Cae.ver tableopts_ver_nr-Cae.ver.c tableopts_ver_nr-Cae.ver.l tableopts_ver_nr-Cae.ver.tables tableopts_ver_nr-Caef.ver tableopts_ver_nr-Caef.ver.c tableopts_ver_nr-Caef.ver.l tableopts_ver_nr-Caef.ver.tables tableopts_ver_nr-CaexF.ver tableopts_ver_nr-CaexF.ver.c tableopts_ver_nr-CaexF.ver.l tableopts_ver_nr-CaexF.ver.tables tableopts_ver_nr-Cam.ver tableopts_ver_nr-Cam.ver.c tableopts_ver_nr-Cam.ver.l tableopts_ver_nr-Cam.ver.tables tableopts_ver_nr-Caem.ver tableopts_ver_nr-Caem.ver.c tableopts_ver_nr-Caem.ver.l tableopts_ver_nr-Caem.ver.tables array_r array_r.c array_r.l array.txt basic_r basic_r.c basic_r.l basic.txt bol_r bol_r.c bol_r.l bol.txt ccl_r ccl_r.c ccl_r.l ccl.txt debug_r debug_r.c debug_r.l debug.txt extended_r extended_r.c extended_r.l extended.txt fixedtrailing_r fixedtrailing_r.c fixedtrailing_r.l fixedtrailing.txt flexname_r flexname_r.c flexname_r.l flexname.txt lineno_r lineno_r.c lineno_r.l lineno.txt posix_r posix_r.c posix_r.l posix.txt preposix_r preposix_r.c preposix_r.l preposix.txt quoteincomment_r quoteincomment_r.c quoteincomment_r.l quoteincomment.txt reject_r reject_r.c reject_r.l reject.txt tableopts_r tableopts_r.c tableopts_r.l tableopts.txt vartrailing_r vartrailing_r.c vartrailing_r.l vartrailing.txt yyless_r yyless_r.c yyless_r.l yyless.txt yymorearraybol_r yymorearraybol_r.c yymorearraybol_r.l yymorearraybol.txt yymorearray_r yymorearray_r.c yymorearray_r.l yymorearray.txt yymore_r yymore_r.c yymore_r.l yymore.txt yyunput_r yyunput_r.c yyunput_r.l yyunput.txt tableopts_opt_r-Ca.opt tableopts_opt_r-Ca.opt.c tableopts_opt_r-Ca.opt.l tableopts_opt_r-Ca.opt.tables tableopts_opt_r-Ce.opt tableopts_opt_r-Ce.opt.c tableopts_opt_r-Ce.opt.l tableopts_opt_r-Ce.opt.tables tableopts_opt_r-Cf.opt tableopts_opt_r-Cf.opt.c tableopts_opt_r-Cf.opt.l tableopts_opt_r-Cf.opt.tables tableopts_opt_r-CxF.opt tableopts_opt_r-CxF.opt.c tableopts_opt_r-CxF.opt.l tableopts_opt_r-CxF.opt.tables tableopts_opt_r-Cm.opt tableopts_opt_r-Cm.opt.c tableopts_opt_r-Cm.opt.l tableopts_opt_r-Cm.opt.tables tableopts_opt_r-Cem.opt tableopts_opt_r-Cem.opt.c tableopts_opt_r-Cem.opt.l tableopts_opt_r-Cem.opt.tables tableopts_opt_r-Cae.opt tableopts_opt_r-Cae.opt.c tableopts_opt_r-Cae.opt.l tableopts_opt_r-Cae.opt.tables tableopts_opt_r-Caef.opt tableopts_opt_r-Caef.opt.c tableopts_opt_r-Caef.opt.l tableopts_opt_r-Caef.opt.tables tableopts_opt_r-CaexF.opt tableopts_opt_r-CaexF.opt.c tableopts_opt_r-CaexF.opt.l tableopts_opt_r-CaexF.opt.tables tableopts_opt_r-Cam.opt tableopts_opt_r-Cam.opt.c tableopts_opt_r-Cam.opt.l tableopts_opt_r-Cam.opt.tables tableopts_opt_r-Caem.opt tableopts_opt_r-Caem.opt.c tableopts_opt_r-Caem.opt.l tableopts_opt_r-Caem.opt.tables tableopts_ser_r-Ca.ser tableopts_ser_r-Ca.ser.c tableopts_ser_r-Ca.ser.l tableopts_ser_r-Ca.ser.tables tableopts_ser_r-Ce.ser tableopts_ser_r-Ce.ser.c tableopts_ser_r-Ce.ser.l tableopts_ser_r-Ce.ser.tables tableopts_ser_r-Cf.ser tableopts_ser_r-Cf.ser.c tableopts_ser_r-Cf.ser.l tableopts_ser_r-Cf.ser.tables tableopts_ser_r-CxF.ser tableopts_ser_r-CxF.ser.c tableopts_ser_r-CxF.ser.l tableopts_ser_r-CxF.ser.tables tableopts_ser_r-Cm.ser tableopts_ser_r-Cm.ser.c tableopts_ser_r-Cm.ser.l tableopts_ser_r-Cm.ser.tables tableopts_ser_r-Cem.ser tableopts_ser_r-Cem.ser.c tableopts_ser_r-Cem.ser.l tableopts_ser_r-Cem.ser.tables tableopts_ser_r-Cae.ser tableopts_ser_r-Cae.ser.c tableopts_ser_r-Cae.ser.l tableopts_ser_r-Cae.ser.tables tableopts_ser_r-Caef.ser tableopts_ser_r-Caef.ser.c tableopts_ser_r-Caef.ser.l tableopts_ser_r-Caef.ser.tables tableopts_ser_r-CaexF.ser tableopts_ser_r-CaexF.ser.c tableopts_ser_r-CaexF.ser.l tableopts_ser_r-CaexF.ser.tables tableopts_ser_r-Cam.ser tableopts_ser_r-Cam.ser.c tableopts_ser_r-Cam.ser.l tableopts_ser_r-Cam.ser.tables tableopts_ser_r-Caem.ser tableopts_ser_r-Caem.ser.c tableopts_ser_r-Caem.ser.l tableopts_ser_r-Caem.ser.tables tableopts_ver_r-Ca.ver tableopts_ver_r-Ca.ver.c tableopts_ver_r-Ca.ver.l tableopts_ver_r-Ca.ver.tables tableopts_ver_r-Ce.ver tableopts_ver_r-Ce.ver.c tableopts_ver_r-Ce.ver.l tableopts_ver_r-Ce.ver.tables tableopts_ver_r-Cf.ver tableopts_ver_r-Cf.ver.c tableopts_ver_r-Cf.ver.l tableopts_ver_r-Cf.ver.tables tableopts_ver_r-CxF.ver tableopts_ver_r-CxF.ver.c tableopts_ver_r-CxF.ver.l tableopts_ver_r-CxF.ver.tables tableopts_ver_r-Cm.ver tableopts_ver_r-Cm.ver.c tableopts_ver_r-Cm.ver.l tableopts_ver_r-Cm.ver.tables tableopts_ver_r-Cem.ver tableopts_ver_r-Cem.ver.c tableopts_ver_r-Cem.ver.l tableopts_ver_r-Cem.ver.tables tableopts_ver_r-Cae.ver tableopts_ver_r-Cae.ver.c tableopts_ver_r-Cae.ver.l tableopts_ver_r-Cae.ver.tables tableopts_ver_r-Caef.ver tableopts_ver_r-Caef.ver.c tableopts_ver_r-Caef.ver.l tableopts_ver_r-Caef.ver.tables tableopts_ver_r-CaexF.ver tableopts_ver_r-CaexF.ver.c tableopts_ver_r-CaexF.ver.l tableopts_ver_r-CaexF.ver.tables tableopts_ver_r-Cam.ver tableopts_ver_r-Cam.ver.c tableopts_ver_r-Cam.ver.l tableopts_ver_r-Cam.ver.tables tableopts_ver_r-Caem.ver tableopts_ver_r-Caem.ver.c tableopts_ver_r-Caem.ver.l tableopts_ver_r-Caem.ver.tables array_c99 array_c99.c array_c99.l array.txt basic_c99 basic_c99.c basic_c99.l basic.txt bol_c99 bol_c99.c bol_c99.l bol.txt ccl_c99 ccl_c99.c ccl_c99.l ccl.txt debug_c99 debug_c99.c debug_c99.l debug.txt extended_c99 extended_c99.c extended_c99.l extended.txt fixedtrailing_c99 fixedtrailing_c99.c fixedtrailing_c99.l fixedtrailing.txt flexname_c99 flexname_c99.c flexname_c99.l flexname.txt lineno_c99 lineno_c99.c lineno_c99.l lineno.txt posix_c99 posix_c99.c posix_c99.l posix.txt preposix_c99 preposix_c99.c preposix_c99.l preposix.txt quoteincomment_c99 quoteincomment_c99.c quoteincomment_c99.l quoteincomment.txt reject_c99 reject_c99.c reject_c99.l reject.txt tableopts_c99 tableopts_c99.c tableopts_c99.l tableopts.txt vartrailing_c99 vartrailing_c99.c vartrailing_c99.l vartrailing.txt yyless_c99 yyless_c99.c yyless_c99.l yyless.txt yymorearraybol_c99 yymorearraybol_c99.c yymorearraybol_c99.l yymorearraybol.txt yymorearray_c99 yymorearray_c99.c yymorearray_c99.l yymorearray.txt yymore_c99 yymore_c99.c yymore_c99.l yymore.txt yyunput_c99 yyunput_c99.c yyunput_c99.l yyunput.txt tableopts_opt_c99-Ca.opt tableopts_opt_c99-Ca.opt.c tableopts_opt_c99-Ca.opt.l tableopts_opt_c99-Ca.opt.tables tableopts_opt_c99-Ce.opt tableopts_opt_c99-Ce.opt.c tableopts_opt_c99-Ce.opt.l tableopts_opt_c99-Ce.opt.tables tableopts_opt_c99-Cf.opt tableopts_opt_c99-Cf.opt.c tableopts_opt_c99-Cf.opt.l tableopts_opt_c99-Cf.opt.tables tableopts_opt_c99-CxF.opt tableopts_opt_c99-CxF.opt.c tableopts_opt_c99-CxF.opt.l tableopts_opt_c99-CxF.opt.tables tableopts_opt_c99-Cm.opt tableopts_opt_c99-Cm.opt.c tableopts_opt_c99-Cm.opt.l tableopts_opt_c99-Cm.opt.tables tableopts_opt_c99-Cem.opt tableopts_opt_c99-Cem.opt.c tableopts_opt_c99-Cem.opt.l tableopts_opt_c99-Cem.opt.tables tableopts_opt_c99-Cae.opt tableopts_opt_c99-Cae.opt.c tableopts_opt_c99-Cae.opt.l tableopts_opt_c99-Cae.opt.tables tableopts_opt_c99-Caef.opt tableopts_opt_c99-Caef.opt.c tableopts_opt_c99-Caef.opt.l tableopts_opt_c99-Caef.opt.tables tableopts_opt_c99-CaexF.opt tableopts_opt_c99-CaexF.opt.c tableopts_opt_c99-CaexF.opt.l tableopts_opt_c99-CaexF.opt.tables tableopts_opt_c99-Cam.opt tableopts_opt_c99-Cam.opt.c tableopts_opt_c99-Cam.opt.l tableopts_opt_c99-Cam.opt.tables tableopts_opt_c99-Caem.opt tableopts_opt_c99-Caem.opt.c tableopts_opt_c99-Caem.opt.l tableopts_opt_c99-Caem.opt.tables tableopts_ser_c99-Ca.ser tableopts_ser_c99-Ca.ser.c tableopts_ser_c99-Ca.ser.l tableopts_ser_c99-Ca.ser.tables tableopts_ser_c99-Ce.ser tableopts_ser_c99-Ce.ser.c tableopts_ser_c99-Ce.ser.l tableopts_ser_c99-Ce.ser.tables tableopts_ser_c99-Cf.ser tableopts_ser_c99-Cf.ser.c tableopts_ser_c99-Cf.ser.l tableopts_ser_c99-Cf.ser.tables tableopts_ser_c99-CxF.ser tableopts_ser_c99-CxF.ser.c tableopts_ser_c99-CxF.ser.l tableopts_ser_c99-CxF.ser.tables tableopts_ser_c99-Cm.ser tableopts_ser_c99-Cm.ser.c tableopts_ser_c99-Cm.ser.l tableopts_ser_c99-Cm.ser.tables tableopts_ser_c99-Cem.ser tableopts_ser_c99-Cem.ser.c tableopts_ser_c99-Cem.ser.l tableopts_ser_c99-Cem.ser.tables tableopts_ser_c99-Cae.ser tableopts_ser_c99-Cae.ser.c tableopts_ser_c99-Cae.ser.l tableopts_ser_c99-Cae.ser.tables tableopts_ser_c99-Caef.ser tableopts_ser_c99-Caef.ser.c tableopts_ser_c99-Caef.ser.l tableopts_ser_c99-Caef.ser.tables tableopts_ser_c99-CaexF.ser tableopts_ser_c99-CaexF.ser.c tableopts_ser_c99-CaexF.ser.l tableopts_ser_c99-CaexF.ser.tables tableopts_ser_c99-Cam.ser tableopts_ser_c99-Cam.ser.c tableopts_ser_c99-Cam.ser.l tableopts_ser_c99-Cam.ser.tables tableopts_ser_c99-Caem.ser tableopts_ser_c99-Caem.ser.c tableopts_ser_c99-Caem.ser.l tableopts_ser_c99-Caem.ser.tables tableopts_ver_c99-Ca.ver tableopts_ver_c99-Ca.ver.c tableopts_ver_c99-Ca.ver.l tableopts_ver_c99-Ca.ver.tables tableopts_ver_c99-Ce.ver tableopts_ver_c99-Ce.ver.c tableopts_ver_c99-Ce.ver.l tableopts_ver_c99-Ce.ver.tables tableopts_ver_c99-Cf.ver tableopts_ver_c99-Cf.ver.c tableopts_ver_c99-Cf.ver.l tableopts_ver_c99-Cf.ver.tables tableopts_ver_c99-CxF.ver tableopts_ver_c99-CxF.ver.c tableopts_ver_c99-CxF.ver.l tableopts_ver_c99-CxF.ver.tables tableopts_ver_c99-Cm.ver tableopts_ver_c99-Cm.ver.c tableopts_ver_c99-Cm.ver.l tableopts_ver_c99-Cm.ver.tables tableopts_ver_c99-Cem.ver tableopts_ver_c99-Cem.ver.c tableopts_ver_c99-Cem.ver.l tableopts_ver_c99-Cem.ver.tables tableopts_ver_c99-Cae.ver tableopts_ver_c99-Cae.ver.c tableopts_ver_c99-Cae.ver.l tableopts_ver_c99-Cae.ver.tables tableopts_ver_c99-Caef.ver tableopts_ver_c99-Caef.ver.c tableopts_ver_c99-Caef.ver.l tableopts_ver_c99-Caef.ver.tables tableopts_ver_c99-CaexF.ver tableopts_ver_c99-CaexF.ver.c tableopts_ver_c99-CaexF.ver.l tableopts_ver_c99-CaexF.ver.tables tableopts_ver_c99-Cam.ver tableopts_ver_c99-Cam.ver.c tableopts_ver_c99-Cam.ver.l tableopts_ver_c99-Cam.ver.tables tableopts_ver_c99-Caem.ver tableopts_ver_c99-Caem.ver.c tableopts_ver_c99-Caem.ver.l tableopts_ver_c99-Caem.ver.tables array_go array_go.c array_go.l array.txt basic_go basic_go.c basic_go.l basic.txt bol_go bol_go.c bol_go.l bol.txt ccl_go ccl_go.c ccl_go.l ccl.txt debug_go debug_go.c debug_go.l debug.txt extended_go extended_go.c extended_go.l extended.txt fixedtrailing_go fixedtrailing_go.c fixedtrailing_go.l fixedtrailing.txt flexname_go flexname_go.c flexname_go.l flexname.txt lineno_go lineno_go.c lineno_go.l lineno.txt posix_go posix_go.c posix_go.l posix.txt preposix_go preposix_go.c preposix_go.l preposix.txt quoteincomment_go quoteincomment_go.c quoteincomment_go.l quoteincomment.txt reject_go reject_go.c reject_go.l reject.txt tableopts_go tableopts_go.c tableopts_go.l tableopts.txt vartrailing_go vartrailing_go.c vartrailing_go.l vartrailing.txt yyless_go yyless_go.c yyless_go.l yyless.txt yymorearraybol_go yymorearraybol_go.c yymorearraybol_go.l yymorearraybol.txt yymorearray_go yymorearray_go.c yymorearray_go.l yymorearray.txt yymore_go yymore_go.c yymore_go.l yymore.txt yyunput_go yyunput_go.c yyunput_go.l yyunput.txt tableopts_opt_go-Ca.opt tableopts_opt_go-Ca.opt.c tableopts_opt_go-Ca.opt.l tableopts_opt_go-Ca.opt.tables tableopts_opt_go-Ce.opt tableopts_opt_go-Ce.opt.c tableopts_opt_go-Ce.opt.l tableopts_opt_go-Ce.opt.tables tableopts_opt_go-Cf.opt tableopts_opt_go-Cf.opt.c tableopts_opt_go-Cf.opt.l tableopts_opt_go-Cf.opt.tables tableopts_opt_go-CxF.opt tableopts_opt_go-CxF.opt.c tableopts_opt_go-CxF.opt.l tableopts_opt_go-CxF.opt.tables tableopts_opt_go-Cm.opt tableopts_opt_go-Cm.opt.c tableopts_opt_go-Cm.opt.l tableopts_opt_go-Cm.opt.tables tableopts_opt_go-Cem.opt tableopts_opt_go-Cem.opt.c tableopts_opt_go-Cem.opt.l tableopts_opt_go-Cem.opt.tables tableopts_opt_go-Cae.opt tableopts_opt_go-Cae.opt.c tableopts_opt_go-Cae.opt.l tableopts_opt_go-Cae.opt.tables tableopts_opt_go-Caef.opt tableopts_opt_go-Caef.opt.c tableopts_opt_go-Caef.opt.l tableopts_opt_go-Caef.opt.tables tableopts_opt_go-CaexF.opt tableopts_opt_go-CaexF.opt.c tableopts_opt_go-CaexF.opt.l tableopts_opt_go-CaexF.opt.tables tableopts_opt_go-Cam.opt tableopts_opt_go-Cam.opt.c tableopts_opt_go-Cam.opt.l tableopts_opt_go-Cam.opt.tables tableopts_opt_go-Caem.opt tableopts_opt_go-Caem.opt.c tableopts_opt_go-Caem.opt.l tableopts_opt_go-Caem.opt.tables tableopts_ser_go-Ca.ser tableopts_ser_go-Ca.ser.c tableopts_ser_go-Ca.ser.l tableopts_ser_go-Ca.ser.tables tableopts_ser_go-Ce.ser tableopts_ser_go-Ce.ser.c tableopts_ser_go-Ce.ser.l tableopts_ser_go-Ce.ser.tables tableopts_ser_go-Cf.ser tableopts_ser_go-Cf.ser.c tableopts_ser_go-Cf.ser.l tableopts_ser_go-Cf.ser.tables tableopts_ser_go-CxF.ser tableopts_ser_go-CxF.ser.c tableopts_ser_go-CxF.ser.l tableopts_ser_go-CxF.ser.tables tableopts_ser_go-Cm.ser tableopts_ser_go-Cm.ser.c tableopts_ser_go-Cm.ser.l tableopts_ser_go-Cm.ser.tables tableopts_ser_go-Cem.ser tableopts_ser_go-Cem.ser.c tableopts_ser_go-Cem.ser.l tableopts_ser_go-Cem.ser.tables tableopts_ser_go-Cae.ser tableopts_ser_go-Cae.ser.c tableopts_ser_go-Cae.ser.l tableopts_ser_go-Cae.ser.tables tableopts_ser_go-Caef.ser tableopts_ser_go-Caef.ser.c tableopts_ser_go-Caef.ser.l tableopts_ser_go-Caef.ser.tables tableopts_ser_go-CaexF.ser tableopts_ser_go-CaexF.ser.c tableopts_ser_go-CaexF.ser.l tableopts_ser_go-CaexF.ser.tables tableopts_ser_go-Cam.ser tableopts_ser_go-Cam.ser.c tableopts_ser_go-Cam.ser.l tableopts_ser_go-Cam.ser.tables tableopts_ser_go-Caem.ser tableopts_ser_go-Caem.ser.c tableopts_ser_go-Caem.ser.l tableopts_ser_go-Caem.ser.tables tableopts_ver_go-Ca.ver tableopts_ver_go-Ca.ver.c tableopts_ver_go-Ca.ver.l tableopts_ver_go-Ca.ver.tables tableopts_ver_go-Ce.ver tableopts_ver_go-Ce.ver.c tableopts_ver_go-Ce.ver.l tableopts_ver_go-Ce.ver.tables tableopts_ver_go-Cf.ver tableopts_ver_go-Cf.ver.c tableopts_ver_go-Cf.ver.l tableopts_ver_go-Cf.ver.tables tableopts_ver_go-CxF.ver tableopts_ver_go-CxF.ver.c tableopts_ver_go-CxF.ver.l tableopts_ver_go-CxF.ver.tables tableopts_ver_go-Cm.ver tableopts_ver_go-Cm.ver.c tableopts_ver_go-Cm.ver.l tableopts_ver_go-Cm.ver.tables tableopts_ver_go-Cem.ver tableopts_ver_go-Cem.ver.c tableopts_ver_go-Cem.ver.l tableopts_ver_go-Cem.ver.tables tableopts_ver_go-Cae.ver tableopts_ver_go-Cae.ver.c tableopts_ver_go-Cae.ver.l tableopts_ver_go-Cae.ver.tables tableopts_ver_go-Caef.ver tableopts_ver_go-Caef.ver.c tableopts_ver_go-Caef.ver.l tableopts_ver_go-Caef.ver.tables tableopts_ver_go-CaexF.ver tableopts_ver_go-CaexF.ver.c tableopts_ver_go-CaexF.ver.l tableopts_ver_go-CaexF.ver.tables tableopts_ver_go-Cam.ver tableopts_ver_go-Cam.ver.c tableopts_ver_go-Cam.ver.l tableopts_ver_go-Cam.ver.tables tableopts_ver_go-Caem.ver tableopts_ver_go-Caem.ver.c tableopts_ver_go-Caem.ver.l tableopts_ver_go-Caem.ver.tables test-yydecl-nr.sh test-yydecl-r.sh test-yydecl-c99.sh test-yydecl-go.sh + diff --git a/tests/ruleset.sh b/tests/ruleset.sh new file mode 100644 index 0000000..8832a57 --- /dev/null +++ b/tests/ruleset.sh @@ -0,0 +1,87 @@ +#!/bin/sh +# Generate make productions for testing files from rulesets. Also set up SOURCES +# variables for link time. Pass it a list of back-end suffixes. +# +# This script exists because automake isn't able to handle the pattern rules that +# would be natural to use. Output is written to standard output for +# inclusion in a Makefile.am, typically by redirecting the output and +# then an automake include directive. + +set -eu + +RULESET_TESTS="" +RULESET_REMOVABLES="" + +printf "\n# Begin generated test rules\n\n" + +compatible() { + mybackend=$1 + myruleset=$2 + # Some options are both a pain to test outside the default back end and really don't need to be + # tested in more than one back end anyway. An option is in this category if it doesn't affect + # any conditionals in the code generation, just the way the Flex scanner generates its NDFSA tables. + [ "${mybackend}" = "nr" ] || [ "${myruleset}" != "lexcompat.rules" -a "${myruleset}" != "posixlycorrect.rules" ] +} + +for backend in "$@" ; do + for ruleset in *.rules; do + if compatible "${backend}" "${ruleset}" ; then + testname="${ruleset%.*}_${backend}" + echo "${testname}_SOURCES = ${testname}.l" + echo "${testname}.l: \$(srcdir)/${ruleset} \$(srcdir)/testmaker.sh \$(srcdir)/testmaker.m4" + # we're deliberately single-quoting this because we _don't_ want those variables to be expanded yet + # shellcheck disable=2016 + printf '\t$(SHELL) $(srcdir)/testmaker.sh $@\n\n' + RULESET_TESTS="${RULESET_TESTS} ${testname}" + RULESET_REMOVABLES="${RULESET_REMOVABLES} ${testname} ${testname}.c ${testname}.l ${ruleset%.*}.txt" + fi + done + for kind in opt ser ver ; do + for opt in -Ca -Ce -Cf -CF -Cm -Cem -Cae -Caef -CaeF -Cam -Caem ; do + bare_opt=${opt#-} + # The filenames must work on case-insensitive filesystems. + bare_opt=$(echo ${bare_opt}| sed 's/F$/xF/') + testname=tableopts_${kind}_${backend}-${bare_opt}.${kind} + RULESET_TESTS="${RULESET_TESTS} ${testname}" + RULESET_REMOVABLES="${RULESET_REMOVABLES} ${testname} ${testname}.c ${testname}.l ${testname}.tables" + cat << EOF +tableopts_${kind}_${backend}_${bare_opt}_${kind}_SOURCES = ${testname}.l +${testname}.l: \$(srcdir)/tableopts.rules \$(srcdir)/testmaker.sh \$(srcdir)/testmaker.m4 + \$(SHELL) \$(srcdir)/testmaker.sh \$@ + +EOF + done + done +done + +# posixlycorrect is a special case becaae we need to set POSIXLY_CORRECT +# in Flex's environment while these .l files are being processed. +for backend in "$@" ; do + case $backend in + nr|r|c99) ext="c" ;; + *) ext=${backend} ;; + esac + # shellcheck disable=SC2059 + printf "posixlycorrect_${backend}.${ext}: posixlycorrect_${backend}.l \$(FLEX)\n" + printf "\t\$(AM_V_LEX)POSIXLY_CORRECT=1 \$(FLEX) \$(TESTOPTS) -o \$@ \$<\n" + echo "" + + echo "test-yydecl-${backend}.sh\$(EXEEXT): test-yydecl-gen.sh" + # shellcheck disable=SC2059 + printf "\t\$(SHELL) test-yydecl-gen.sh ${backend} >test-yydecl-${backend}.sh\$(EXEEXT)\n" + # shellcheck disable=SC2059 + printf "\tchmod a+x test-yydecl-${backend}.sh\$(EXEEXT)\n" + echo "" + + RULESET_TESTS="${RULESET_TESTS} test-yydecl-${backend}.sh" + RULESET_REMOVABLES="${RULESET_REMOVABLES} test-yydecl-${backend}.sh" +done + +echo "" +printf "# End generated test rules\n" + +echo RULESET_TESTS = "${RULESET_TESTS}" +echo RULESET_REMOVABLES = "${RULESET_REMOVABLES}" +echo + + diff --git a/tests/string_c99.l b/tests/string_c99.l new file mode 100644 index 0000000..d8eabfc --- /dev/null +++ b/tests/string_c99.l @@ -0,0 +1,104 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +%{ +/* A template scanner file to build "scanner.c". */ +#include <stdio.h> +#include <stdlib.h> +#include "config.h" + +#define NUMBER 200 +#define WORD 201 + +%} + +%option emit="c99" +%option 8bit prefix="test" +%option nounput nomain nodefault noyywrap noinput +%option warn + + +%% + +[[:space:]]+ { } +[[:digit:]]+ { printf("NUMBER "); fflush(stdout);} +[[:alpha:]]+ { printf("WORD "); fflush(stdout);} +. { + fprintf(stderr,"*** Error: Unrecognized character '%c' while scanning.\n", + yytext[0]); + yyterminate(); + } + +<<EOF>> { printf("<<EOF>>\n"); yyterminate();} + +%% + + +#define INPUT_STRING_1 "1234 foo bar" +#define INPUT_STRING_2 "1234 foo bar *@&@&###@^$#&#*" + +int main(void); + +int +main (void) +{ + char * buf; + size_t len; + yybuffer state; + yyscan_t scanner=NULL; + + + /* Scan a good string. */ + printf("Testing: test_scan_string(%s): ",INPUT_STRING_1); fflush(stdout); + testlex_init(&scanner); + state = test_scan_string ( INPUT_STRING_1 ,scanner); + testlex(scanner); + test_delete_buffer(state, scanner); + testlex_destroy(scanner); + + /* Scan only the first 12 chars of a string. */ + printf("Testing: test_scan_bytes(%s): ",INPUT_STRING_2); fflush(stdout); + testlex_init(&scanner); + state = test_scan_bytes ( INPUT_STRING_2, 12 ,scanner); + testlex(scanner); + test_delete_buffer(state,scanner); + testlex_destroy(scanner); + + /* Scan directly from a buffer. + We make a copy, since the buffer will be modified by flex.*/ + printf("Testing: test_scan_buffer(%s): ",INPUT_STRING_1); fflush(stdout); + len = strlen(INPUT_STRING_1) + 2; + buf = malloc(len); + strcpy( buf, INPUT_STRING_1); + buf[ len -2 ] = 0; /* Flex requires two NUL bytes at end of buffer. */ + buf[ len -1 ] =0; + + testlex_init(&scanner); + state = test_scan_buffer( buf, len ,scanner); + testlex(scanner); + test_delete_buffer(state,scanner); + testlex_destroy(scanner); + + printf("TEST RETURNING OK.\n"); + return 0; +} diff --git a/tests/string_nr.l b/tests/string_nr.l index 8352b09..fdeffbf 100644 --- a/tests/string_nr.l +++ b/tests/string_nr.l @@ -63,8 +63,7 @@ main (void) { char * buf; size_t len; - YY_BUFFER_STATE state; - + yybuffer state; /* Scan a good string. */ printf("Testing: test_scan_string(%s): ",INPUT_STRING_1); fflush(stdout); diff --git a/tests/string_r.l b/tests/string_r.l index 174eb6c..6770386 100644 --- a/tests/string_r.l +++ b/tests/string_r.l @@ -63,7 +63,7 @@ main (void) { char * buf; size_t len; - YY_BUFFER_STATE state; + yybuffer state; yyscan_t scanner=NULL; diff --git a/tests/tableopts.am b/tests/tableopts.am deleted file mode 100644 index 8460c04..0000000 --- a/tests/tableopts.am +++ /dev/null @@ -1,333 +0,0 @@ -tableopts_opt_nr_Ca_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Ca.opt$(EXEEXT): tableopts_opt_nr-Ca.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Ce_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Ce.opt$(EXEEXT): tableopts_opt_nr-Ce.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Cf_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Cf.opt$(EXEEXT): tableopts_opt_nr-Cf.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_C_F_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-C_F.opt$(EXEEXT): tableopts_opt_nr-C_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Cm_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Cm.opt$(EXEEXT): tableopts_opt_nr-Cm.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Cem_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Cem.opt$(EXEEXT): tableopts_opt_nr-Cem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Cae_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Cae.opt$(EXEEXT): tableopts_opt_nr-Cae.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Caef_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Caef.opt$(EXEEXT): tableopts_opt_nr-Caef.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Cae_F_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Cae_F.opt$(EXEEXT): tableopts_opt_nr-Cae_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Cam_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Cam.opt$(EXEEXT): tableopts_opt_nr-Cam.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_nr_Caem_opt_SOURCES = tableopts.l4 - -tableopts_opt_nr-Caem.opt$(EXEEXT): tableopts_opt_nr-Caem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Ca_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Ca.opt$(EXEEXT): tableopts_opt_r-Ca.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Ce_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Ce.opt$(EXEEXT): tableopts_opt_r-Ce.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Cf_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Cf.opt$(EXEEXT): tableopts_opt_r-Cf.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_C_F_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-C_F.opt$(EXEEXT): tableopts_opt_r-C_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Cm_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Cm.opt$(EXEEXT): tableopts_opt_r-Cm.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Cem_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Cem.opt$(EXEEXT): tableopts_opt_r-Cem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Cae_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Cae.opt$(EXEEXT): tableopts_opt_r-Cae.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Caef_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Caef.opt$(EXEEXT): tableopts_opt_r-Caef.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Cae_F_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Cae_F.opt$(EXEEXT): tableopts_opt_r-Cae_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Cam_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Cam.opt$(EXEEXT): tableopts_opt_r-Cam.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_opt_r_Caem_opt_SOURCES = tableopts.l4 - -tableopts_opt_r-Caem.opt$(EXEEXT): tableopts_opt_r-Caem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Ca_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Ca.ser$(EXEEXT): tableopts_ser_nr-Ca.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Ce_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Ce.ser$(EXEEXT): tableopts_ser_nr-Ce.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Cf_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Cf.ser$(EXEEXT): tableopts_ser_nr-Cf.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_C_F_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-C_F.ser$(EXEEXT): tableopts_ser_nr-C_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Cm_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Cm.ser$(EXEEXT): tableopts_ser_nr-Cm.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Cem_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Cem.ser$(EXEEXT): tableopts_ser_nr-Cem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Cae_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Cae.ser$(EXEEXT): tableopts_ser_nr-Cae.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Caef_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Caef.ser$(EXEEXT): tableopts_ser_nr-Caef.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Cae_F_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Cae_F.ser$(EXEEXT): tableopts_ser_nr-Cae_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Cam_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Cam.ser$(EXEEXT): tableopts_ser_nr-Cam.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_nr_Caem_ser_SOURCES = tableopts.l4 - -tableopts_ser_nr-Caem.ser$(EXEEXT): tableopts_ser_nr-Caem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Ca_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Ca.ser$(EXEEXT): tableopts_ser_r-Ca.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Ce_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Ce.ser$(EXEEXT): tableopts_ser_r-Ce.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Cf_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Cf.ser$(EXEEXT): tableopts_ser_r-Cf.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_C_F_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-C_F.ser$(EXEEXT): tableopts_ser_r-C_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Cm_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Cm.ser$(EXEEXT): tableopts_ser_r-Cm.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Cem_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Cem.ser$(EXEEXT): tableopts_ser_r-Cem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Cae_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Cae.ser$(EXEEXT): tableopts_ser_r-Cae.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Caef_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Caef.ser$(EXEEXT): tableopts_ser_r-Caef.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Cae_F_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Cae_F.ser$(EXEEXT): tableopts_ser_r-Cae_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Cam_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Cam.ser$(EXEEXT): tableopts_ser_r-Cam.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ser_r_Caem_ser_SOURCES = tableopts.l4 - -tableopts_ser_r-Caem.ser$(EXEEXT): tableopts_ser_r-Caem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Ca_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Ca.ver$(EXEEXT): tableopts_ver_nr-Ca.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Ce_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Ce.ver$(EXEEXT): tableopts_ver_nr-Ce.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Cf_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Cf.ver$(EXEEXT): tableopts_ver_nr-Cf.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_C_F_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-C_F.ver$(EXEEXT): tableopts_ver_nr-C_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Cm_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Cm.ver$(EXEEXT): tableopts_ver_nr-Cm.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Cem_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Cem.ver$(EXEEXT): tableopts_ver_nr-Cem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Cae_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Cae.ver$(EXEEXT): tableopts_ver_nr-Cae.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Caef_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Caef.ver$(EXEEXT): tableopts_ver_nr-Caef.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Cae_F_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Cae_F.ver$(EXEEXT): tableopts_ver_nr-Cae_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Cam_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Cam.ver$(EXEEXT): tableopts_ver_nr-Cam.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_nr_Caem_ver_SOURCES = tableopts.l4 - -tableopts_ver_nr-Caem.ver$(EXEEXT): tableopts_ver_nr-Caem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Ca_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Ca.ver$(EXEEXT): tableopts_ver_r-Ca.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Ce_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Ce.ver$(EXEEXT): tableopts_ver_r-Ce.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Cf_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Cf.ver$(EXEEXT): tableopts_ver_r-Cf.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_C_F_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-C_F.ver$(EXEEXT): tableopts_ver_r-C_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Cm_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Cm.ver$(EXEEXT): tableopts_ver_r-Cm.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Cem_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Cem.ver$(EXEEXT): tableopts_ver_r-Cem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Cae_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Cae.ver$(EXEEXT): tableopts_ver_r-Cae.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Caef_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Caef.ver$(EXEEXT): tableopts_ver_r-Caef.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Cae_F_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Cae_F.ver$(EXEEXT): tableopts_ver_r-Cae_F.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Cam_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Cam.ver$(EXEEXT): tableopts_ver_r-Cam.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -tableopts_ver_r_Caem_ver_SOURCES = tableopts.l4 - -tableopts_ver_r-Caem.ver$(EXEEXT): tableopts_ver_r-Caem.$(OBJEXT) - $(AM_V_CCLD)$(LINK) $< - -TABLEOPTS_TESTS = tableopts_opt_nr-Ca.opt tableopts_opt_nr-Ce.opt tableopts_opt_nr-Cf.opt tableopts_opt_nr-C_F.opt tableopts_opt_nr-Cm.opt tableopts_opt_nr-Cem.opt tableopts_opt_nr-Cae.opt tableopts_opt_nr-Caef.opt tableopts_opt_nr-Cae_F.opt tableopts_opt_nr-Cam.opt tableopts_opt_nr-Caem.opt tableopts_opt_r-Ca.opt tableopts_opt_r-Ce.opt tableopts_opt_r-Cf.opt tableopts_opt_r-C_F.opt tableopts_opt_r-Cm.opt tableopts_opt_r-Cem.opt tableopts_opt_r-Cae.opt tableopts_opt_r-Caef.opt tableopts_opt_r-Cae_F.opt tableopts_opt_r-Cam.opt tableopts_opt_r-Caem.opt tableopts_ser_nr-Ca.ser tableopts_ser_nr-Ce.ser tableopts_ser_nr-Cf.ser tableopts_ser_nr-C_F.ser tableopts_ser_nr-Cm.ser tableopts_ser_nr-Cem.ser tableopts_ser_nr-Cae.ser tableopts_ser_nr-Caef.ser tableopts_ser_nr-Cae_F.ser tableopts_ser_nr-Cam.ser tableopts_ser_nr-Caem.ser tableopts_ser_r-Ca.ser tableopts_ser_r-Ce.ser tableopts_ser_r-Cf.ser tableopts_ser_r-C_F.ser tableopts_ser_r-Cm.ser tableopts_ser_r-Cem.ser tableopts_ser_r-Cae.ser tableopts_ser_r-Caef.ser tableopts_ser_r-Cae_F.ser tableopts_ser_r-Cam.ser tableopts_ser_r-Caem.ser tableopts_ver_nr-Ca.ver tableopts_ver_nr-Ce.ver tableopts_ver_nr-Cf.ver tableopts_ver_nr-C_F.ver tableopts_ver_nr-Cm.ver tableopts_ver_nr-Cem.ver tableopts_ver_nr-Cae.ver tableopts_ver_nr-Caef.ver tableopts_ver_nr-Cae_F.ver tableopts_ver_nr-Cam.ver tableopts_ver_nr-Caem.ver tableopts_ver_r-Ca.ver tableopts_ver_r-Ce.ver tableopts_ver_r-Cf.ver tableopts_ver_r-C_F.ver tableopts_ver_r-Cm.ver tableopts_ver_r-Cem.ver tableopts_ver_r-Cae.ver tableopts_ver_r-Caef.ver tableopts_ver_r-Cae_F.ver tableopts_ver_r-Cam.ver tableopts_ver_r-Caem.ver - -tableopts_tables = tableopts_ser_nr-Ca.ser.tables tableopts_ser_nr-Ce.ser.tables tableopts_ser_nr-Cf.ser.tables tableopts_ser_nr-C_F.ser.tables tableopts_ser_nr-Cm.ser.tables tableopts_ser_nr-Cem.ser.tables tableopts_ser_nr-Cae.ser.tables tableopts_ser_nr-Caef.ser.tables tableopts_ser_nr-Cae_F.ser.tables tableopts_ser_nr-Cam.ser.tables tableopts_ser_nr-Caem.ser.tables tableopts_ser_r-Ca.ser.tables tableopts_ser_r-Ce.ser.tables tableopts_ser_r-Cf.ser.tables tableopts_ser_r-C_F.ser.tables tableopts_ser_r-Cm.ser.tables tableopts_ser_r-Cem.ser.tables tableopts_ser_r-Cae.ser.tables tableopts_ser_r-Caef.ser.tables tableopts_ser_r-Cae_F.ser.tables tableopts_ser_r-Cam.ser.tables tableopts_ser_r-Caem.ser.tables tableopts_ver_nr-Ca.ver.tables tableopts_ver_nr-Ce.ver.tables tableopts_ver_nr-Cf.ver.tables tableopts_ver_nr-C_F.ver.tables tableopts_ver_nr-Cm.ver.tables tableopts_ver_nr-Cem.ver.tables tableopts_ver_nr-Cae.ver.tables tableopts_ver_nr-Caef.ver.tables tableopts_ver_nr-Cae_F.ver.tables tableopts_ver_nr-Cam.ver.tables tableopts_ver_nr-Caem.ver.tables tableopts_ver_r-Ca.ver.tables tableopts_ver_r-Ce.ver.tables tableopts_ver_r-Cf.ver.tables tableopts_ver_r-C_F.ver.tables tableopts_ver_r-Cm.ver.tables tableopts_ver_r-Cem.ver.tables tableopts_ver_r-Cae.ver.tables tableopts_ver_r-Caef.ver.tables tableopts_ver_r-Cae_F.ver.tables tableopts_ver_r-Cam.ver.tables tableopts_ver_r-Caem.ver.tables diff --git a/tests/tableopts.l4 b/tests/tableopts.l4 deleted file mode 100644 index ac85c29..0000000 --- a/tests/tableopts.l4 +++ /dev/null @@ -1,84 +0,0 @@ -/* - * This file is part of flex. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * 1. Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * Neither the name of the University nor the names of its contributors - * may be used to endorse or promote products derived from this software - * without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE. - */ - -%{ -/* A template scanner file to build "scanner.c". */ -#include <stdio.h> -#include <stdlib.h> -#include "config.h" -%} - -%option 8bit -%option nounput nomain noyywrap noinput -%option warn yylineno - - - -%% - -foo|bar ; -[[:digit:]]+ ; -[[:blank:]]+ ; -.|\n ; -%% - -int main ( int argc, char** argv ) -{ - FILE* fp = NULL; - void *yyscanner=0; - M4_YY_DECL_GUTS_VAR(); - -#ifdef TEST_IS_REENTRANT - yylex_init(&yyscanner); -#else - (void)yyscanner; -#endif - -#ifdef TEST_HAS_TABLES_EXTERNAL - if((fp = fopen(argv[1],"rb"))== NULL) - YY_FATAL_ERROR("could not open tables file for reading"); - - if(yytables_fload(fp M4_YY_CALL_LAST_ARG) < 0) - YY_FATAL_ERROR("yytables_fload returned < 0"); - if(M4_YY_TABLES_VERIFY) - exit(0); -#endif - - if(argc > 2){ - if((fp = fopen(argv[2],"r"))== NULL) - YY_FATAL_ERROR("could not open input file for reading"); - yyin = fp; - } - while(yylex(M4_YY_CALL_ONLY_ARG) != 0) - ; - -#ifdef TEST_HAS_TABLES_EXTERNAL - yytables_destroy(M4_YY_CALL_ONLY_ARG); -#endif - yylex_destroy(M4_YY_CALL_ONLY_ARG); - - if(argc < 0) /* silence the compiler */ - yyscanner = (void*)fp; - - return 0; -} diff --git a/tests/tableopts.rules b/tests/tableopts.rules new file mode 100644 index 0000000..8787f8c --- /dev/null +++ b/tests/tableopts.rules @@ -0,0 +1,36 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + /* The simple grammar used as a test of table-compression options */ + +%option 8bit +%option nounput nomain noyywrap noinput +%option warn yylineno +%% + +foo|bar ; +[[:digit:]]+ ; +[[:blank:]]+ ; +.|\n ; +### +0000 foo 1111 foo 0000 bar foobar +0000 foo 1111 foo 0000 bar foobar diff --git a/tests/tableopts.sh b/tests/tableopts.sh deleted file mode 100755 index 60fcee8..0000000 --- a/tests/tableopts.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/bin/sh -set -eu -# IFS=$'\n\t' -IFS=' -'' ' - -# This script is present to generate the automake _SOURCES variables -# for the tableopts_* tests. It also generates the linking rules for -# each test since automake isn't able to handle the pattern rules that -# would be natural to use. Output is written to standard output for -# inclusion in a Makefile.am, typically by redirecting the output and -# then an automake include directive. - -TABLEOPTS_TESTS="" -tableopts_tables="" - -for kind in opt ser ver ; do - for threading in nr r ; do - for opt in -Ca -Ce -Cf -CF -Cm -Cem -Cae -Caef -CaeF -Cam -Caem ; do - bare_opt=${opt#-} - # The filenames must work on case-insensitive filesystems. - bare_opt=`echo ${bare_opt}| sed 's/F$/_F/'` - - testname=tableopts_${kind}_${threading}-${bare_opt}.${kind} - if [ "${TABLEOPTS_TESTS}" = "" ] ;then - TABLEOPTS_TESTS=${testname} - if [ "$kind" = "ser" ] || [ "$kind" = "ver" ] ; then - tableopts_tables=${testname}.tables - fi - else - TABLEOPTS_TESTS="${TABLEOPTS_TESTS} ${testname}" - if [ "$kind" = "ser" ] || [ "$kind" = "ver" ] ; then - tableopts_tables="${tableopts_tables} ${testname}.tables" - fi - fi - - cat << EOF -tableopts_${kind}_${threading}_${bare_opt}_${kind}_SOURCES = tableopts.l4 - -${testname}\$(EXEEXT): tableopts_${kind}_${threading}-${bare_opt}.\$(OBJEXT) - \$(AM_V_CCLD)\$(LINK) \$< - -EOF - done - done -done - -echo TABLEOPTS_TESTS = ${TABLEOPTS_TESTS} -echo -echo tableopts_tables = ${tableopts_tables} diff --git a/tests/tableopts.txt b/tests/tableopts.txt deleted file mode 100644 index aec9276..0000000 --- a/tests/tableopts.txt +++ /dev/null @@ -1,2 +0,0 @@ -0000 foo 1111 foo 0000 bar foobar -0000 foo 1111 foo 0000 bar foobar diff --git a/tests/test-yydecl-gen.sh b/tests/test-yydecl-gen.sh new file mode 100755 index 0000000..0a70915 --- /dev/null +++ b/tests/test-yydecl-gen.sh @@ -0,0 +1,30 @@ +#! /bin/sh +# Generate test for the %yydecl option of flex on a specified back end +backend="$1" + +cat <<EOF_OUTER +#! /bin/sh +# Test %yydecl option of flex on ${backend} backend. +trap 'rm /tmp/td\$\$' EXIT HUP INT QUIT TERM + +teeout=/dev/null +if [ "\$1" = "-d" ] +then + shift + teeout=/dev/stderr +fi + +cat >/tmp/td\$\$ <<EOF +%option yydecl="int foobar(void)" emit="${backend}" +%% +%% +EOF +if ../src/flex -t /tmp/td\$\$ | tee \${teeout} | grep "int foobar(void)" >/dev/null +then + echo "%yydecl test on ${backend} SUCCEEDED" + exit 0 +else + echo "%yydecl test on ${backend} FAILED." >&2; + exit 1 +fi +EOF_OUTER diff --git a/tests/test-yylmax b/tests/test-yylmax new file mode 100755 index 0000000..a89fea9 --- /dev/null +++ b/tests/test-yylmax @@ -0,0 +1,14 @@ +#! /bin/sh +# Test %yylmax option of flex in any cpp-based back end + +trap 'rm /tmp/td$$' EXIT HUP INT QUIT TERM +cat >/tmp/td$$ <<EOF +%option yylmax=333 +%% +%% +EOF + +( ../src/flex -t /tmp/td$$ | grep "#define YYLMAX 333" >/dev/null ) || (echo "%yylmax test failed." >&2; exit 1) + + + diff --git a/tests/testmaker.m4 b/tests/testmaker.m4 new file mode 100644 index 0000000..bddbd44 --- /dev/null +++ b/tests/testmaker.m4 @@ -0,0 +1,202 @@ +dnl Multi-backend test template +dnl +dnl Supply per-backend boilerplate for Flex tests. +dnl See testmaker.sh for how it's used. +dnl +dnl M4_TEST_BACKEND must be set to a recognized back end name. +dnl For purposes of this file, "nr" and "r" are treated as +dnl separate back ends - non-reentrant and reentrant C through +dnl the default cpp skeleton. The c99 backend is treated here +dnl as re-entrant C, but may have a different skeleton file. +dnl +dnl The purpose of this file is to properly set these macros: +dnl +dnl M4_TEST_PREAMBLE = the contyents of s top secton, if required, and +dnl the emit option required to srt the back end. +dnl +dnl M4_TEST_COMPRESSION = compression option. +dnl +dnl M4_TEST_DO = Add a statement terminator if target language +dnl requires it. +dnl +dnl M4_TEST_FAILMESSAGE = a line of code required to issue dnl a +dnl failure notification to standard error and exit with a failure status. +dnl +dnl M4_TEST_INCREMENT = increment the argument variable. +dnl +dnl M4_TEST_DECREMENT = decrement the argument variable. +dnl +dnl M4_TEST_POSTAMBLE = the test main. +dnl +dnl M4_TEST_TABLE_SERIALIZATION = define this to exercise table +dnl serialization. +dnl +dnl M4_TEST_TABLE_VERIFICATION = define this to exercise table +dnl verification and exit. +dnl +dnl M4_TEST_INITHOOK = define this in a rules file to splice code into +dnl the test main as a compilation test. +dnl +dnl These macros are to be expanded by files with a .rules extension +dnl that contain pure flex rulesets and no backend-specific code. +dnl +changecom +define(`M4_TEST_FAILMESSAGE', `INVALID BACK END')dnl +define(`M4_TEST_INITHOOK', `')dnl +dnl +ifelse(M4_TEST_BACKEND, `nr', `dnl +define(`M4_TEST_PREAMBLE', `dnl +%{ +#include "config.h" +#include <stdio.h> +%} +')dnl close preamble +define(`M4_TEST_DO', `$1;') +define(`M4_TEST_FAILMESSAGE', `fprintf(stderr,"TEST FAILED: %d:\"%s\".\n", yylineno, yytext); exit(1);') +define(`M4_TEST_ASSERT', `if (!($1)) {fprintf(stderr,"ASSERT FAILED: %d:\"%s\"\n", yylineno, yytext); exit(1);}') +m4_ifdef(`M4_TEST_ENABLEDEBUG', `define(`M4_TEST_INITHOOK', `flex_debug = 1;')') +define(`M4_TEST_INCREMENT', `++$1') +define(`M4_TEST_DECREMENT', `--$1') +define(`M4_TEST_POSTAMBLE', `dnl +int main (int argc, char **argv) +{ +ifdef(`M4_TEST_TABLE_SERIALIZATION', `dnl + FILE *fp; +')dnl table_serialization + yyin = stdin; + yyout = stdout; +ifdef(`M4_TEST_TABLE_SERIALIZATION', `dnl + if((fp = fopen(argv[1],"rb"))== NULL) + yypanic("could not open tables file for reading"); + + if(yytables_fload(fp) < 0) + yypanic("yytables_fload returned < 0"); + ifdef(`M4_TEST_TABLE_VERIFICATION', `exit(0);') + M4_TEST_INITHOOK +')dnl table_serialization + while( yylex() != YY_NULL ) + { + } +ifdef(`M4_TEST_TABLE_SERIALIZATION', `dnl + yytables_destroy(); +')dnl table_serialization + printf("TEST RETURNING OK.\n"); + return 0; +} +')dnl close postamble +')dnl close nr +dnl +ifelse(M4_TEST_BACKEND, `r', `dnl +define(`M4_TEST_PREAMBLE', `dnl +%{ +#include "config.h" +#include <stdio.h> +%} +')dnl close preamble +define(`M4_TEST_DO', `$1;') +define(`M4_TEST_FAILMESSAGE', `fprintf(stderr,"TEST FAILED: %d:\"%s\".\n", yylineno, yytext); exit(1);') +define(`M4_TEST_ASSERT', `if (!$1) {fprintf(stderr,"ASSERT FAILED: %d:\"%s\"\n", yylineno, yytext); exit(1);}') +m4_ifdef(`M4_TEST_ENABLEDEBUG', `define(`M4_TEST_INITHOOK', `yyset_debug (yyget_debug(lexer), lexer);')') +define(`M4_TEST_INCREMENT', `++$1') +define(`M4_TEST_DECREMENT', `--$1') +define(`M4_TEST_POSTAMBLE', `dnl +int main (int argc, char **argv) +{ +ifdef(`M4_TEST_TABLE_SERIALIZATION', `dnl + FILE *fp; +')dnl table_serialization + yyscan_t lexer; + yylex_init( &lexer ); + yyset_out ( stdout,lexer); + yyset_in ( stdin, lexer); +ifdef(`M4_TEST_TABLE_SERIALIZATION', `dnl + if((fp = fopen(argv[1],"rb"))== NULL) + yypanic("could not open tables file for reading", lexer); + + if(yytables_fload(fp, yyscanner) < 0) + yypanic("yytables_fload returned < 0", lexer); + ifdef(`M4_TEST_TABLE_VERIFICATION', `exit(0);') +')dnl table_serialization + M4_TEST_INITHOOK + while( yylex(lexer) != YY_NULL ) + { + } +ifdef(`M4_TEST_TABLE_SERIALIZATION', `dnl + yytables_destroy(yyscanner); +')dnl table_serialization + yylex_destroy( lexer ); + printf("TEST RETURNING OK.\n"); + return 0; +} +')dnl close postamble +')dnl close r +dnl +dnl This is a fake Go wrapper that will only work as long as the "go" +dnl back end is actually generating C. +ifelse(M4_TEST_BACKEND, `go', `dnl +define(`M4_TEST_PREAMBLE', `dnl +%{ +#include "config.h" +#include <stdio.h> +%} +%option emit="go" +')dnl close preamble +define(`M4_TEST_DO', `$1;') +define(`M4_TEST_FAILMESSAGE', `fprintf(stderr,"TEST FAILED: %d:\"%s\".\n", yylineno, yytext); exit(1);') +define(`M4_TEST_ASSERT', `if (!$1) {fprintf(stderr,"ASSERT FAILED: %d:\"%s\"\n", yylineno, yytext); exit(1);}') +m4_ifdef(`M4_TEST_ENABLEDEBUG', `define(`M4_TEST_INITHOOK', `yyset_debug (yyget_debug(lexer), lexer);')') +define(`M4_TEST_INCREMENT', `++$1') +define(`M4_TEST_DECREMENT', `--$1') +define(`M4_TEST_POSTAMBLE', `dnl +int main (int argc, char **argv) +{ + FlexLexer *lexer; + yylex_init( &lexer ); + yyset_out ( stdout,lexer); + yyset_in ( stdin, lexer); + M4_TEST_INITHOOK + while( yylex(lexer) != flexEOF ) + { + } + yylex_destroy( lexer ); + printf("TEST RETURNING OK.\n"); + return 0; +} +')dnl close postamble +')dnl close fake-go +dnl +dnl A hypothetical example +ifelse(M4_TEST_BACKEND, `hypothetical-go', `dnl +define(`M4_TEST_PREAMBLE', `dnl +%top{ +package main + +import ( + "fmt" + "log" + "os" +) +%} +%option emit="hypothetical-go" +')dnl close preamble +define(`M4_TEST_DO', `$1') +define(`M4_TEST_FAILMESSAGE', `fmt.Fprintf(os.Stderr, "TEST FAILMESSAGE: %d:\"%s\"\n", yylineno, yytext); os.Exit(1);') +define(`M4_TEST_ASSERT', `if !$1 {fmt.Fprintf(os.Stderr,"ASSERT FAILED: %d:\"%s\"\n", yylineno, yytext); os.Exit(1);}') +m4_ifdef(`M4_TEST_ENABLEDEBUG', `define(`M4_TEST_INITHOOK', `lexer.yysetDebug(lexer.yygetDebug())')') +define(`M4_TEST_INCREMENT', `++$1') +define(`M4_TEST_DECREMENT', `--$1') +define(`M4_TEST_POSTAMBLE', `dnl +func main(void) { + lexer := new(FlexLexer) + lexer.yysetOut(os.Stdout) + lexer.yysetIn(os.Stdin) + M4_TEST_INITHOOK + for lexer.yylex() != flexEOF { + } + fmt.Printf("TEST RETURNING OK.\n") + os.Exit(0) +} +')dnl close postamble +')dnl close go +dnl +dnl Additional backends go here diff --git a/tests/testmaker.sh b/tests/testmaker.sh new file mode 100644 index 0000000..2fe5478 --- /dev/null +++ b/tests/testmaker.sh @@ -0,0 +1,99 @@ +#!/bin/sh +# +# testmaker.sh - asssemble tests from backend-independent rulesets and +# backend-dependent boilerplate. Generates both a Flex source file +# and an input text for it. +# +# The single argument is a testfile name to be generated. +# With the -d option, dump to stdourather than crating the file. +# +# To add a new back end named "foo", append "|foo" to the +# third case arm marked "# Add new back ends on this line". + +if [ "$1" = -d ] ; then + shift + outdev=/dev/stdout + filter=cat +else + outdev="$1" + filter=m4 +fi + +testfile=$1 + +trap 'rm -f /tmp/testmaker$$' EXIT INT QUIT + +# we do want word splitting, so we won't put double quotes around it +# shellcheck disable=2046 +set $(echo "${testfile}" | tr '.' ' ') +for last; do :; done +if [ "${last}" != "l" ] +then + echo "$0: Don't know how to make anything but a .l file: ${last}" >&2 + exit 1 +fi + +# ditto +# shellcheck disable=2046 +set -- $(echo "${1}" | tr '_' ' ') +stem=$1 +options="" +backend=nr +for part in "$@"; do + case ${part} in + nr) backend=nr; ;; + r) backend=r; options="${options} reentrant";; + c99|go) backend=${part}; options="${options} emit=\"${part}\"" ;; # Add new back ends on this line + ser) serialization=yes ;; + ver) serialization=yes; verification=yes; options="${options} tables-verify" ;; + Ca) options="${options} align" ;; + Ce) options="${options} ecs" ;; + Cf) options="${options} full" ;; + CxF|Cxf) options="${options} fast" ;; + Cm) options="${options} meta-ecs";; + Cem) options="${options} ecs meta-ecs" ;; + Cae) options="${options} align ecs" ;; + Caef) options="${options} align ecs full" ;; + CaexF|Caexf) options="${options} algin ecs fast" ;; + Cam) options="${options} align meta-ecs" ;; + Caem) options="${options} align ecs meta-ecs" ;; + esac +done +# Special case: C99 back end uses same boilerplate as reentrant C. +case ${backend} in c99) backend=r ;; esac + +m4def() { + define="${1}" + value="${2}" + # we'll be careful, I promise + printf "define(\`%s', \`${value}')dnl\n" "${define}" +} + +( + m4def M4_TEST_BACKEND "${backend}" + if [ -n "${verification}" ] ; then + m4def M4_TEST_TABLE_VERIFICATION + fi + if [ -n "${serialization}" ] ; then + options="${options} tables-file=\"${testfile%.l}.tables\"" + m4def M4_TEST_TABLE_SERIALIZATION + fi + if [ -z "${options}" ] ; then + m4def M4_TEST_OPTIONS + else + m4def M4_TEST_OPTIONS "%%option${options}\n" + fi + cat testmaker.m4 + echo "M4_TEST_PREAMBLE\`'dnl" + echo "M4_TEST_OPTIONS\`'dnl" + sed <"${stem}.rules" -e "/###/Q0" + echo "%%" + echo "M4_TEST_POSTAMBLE\`'dnl" +) | "${filter}" >"${outdev}" + +if [ "${outdev}" != /dev/stdout ] && [ ! -f "${stem}.txt" ] +then + sed <"${stem}.rules" -e "1,/###/d" >"${stem}.txt" +fi + +# end diff --git a/tests/testwrapper-direct.sh b/tests/testwrapper-direct.sh index 9160501..53d59a0 100755 --- a/tests/testwrapper-direct.sh +++ b/tests/testwrapper-direct.sh @@ -9,12 +9,15 @@ while getopts :b:s: OPTION ; do case $OPTION in b) BINARY_DIR=$OPTARG ;; s) SOURCE_DIR=$OPTARG ;; + *) echo "Usage: ${0} [-b BINARY_DIR] [-s SOURCE_DIR] TESTNAME" + exit 1 + ;; esac done -shift $(($OPTIND-1)) +shift $((OPTIND-1)) TESTNAME=$1 -INPUT_NAME=`basename "${TESTNAME%.exe}"`.txt +INPUT_NAME=$(basename "${TESTNAME%.exe}").txt -cd ${SOURCE_DIR} && ${BINARY_DIR}/${TESTNAME} ${INPUT_NAME} +cd "${SOURCE_DIR}" && "${BINARY_DIR}/${TESTNAME}" "${INPUT_NAME}" diff --git a/tests/testwrapper.sh b/tests/testwrapper.sh index 293f8cc..f07f798 100755 --- a/tests/testwrapper.sh +++ b/tests/testwrapper.sh @@ -13,7 +13,7 @@ INPUT_COUNT=0 USE_REDIRECT=0 DO_COMPARISON=0 -while getopts :d:i:rt1 OPTION ; do +while getopts d:i:r OPTION ; do case $OPTION in d) INPUT_DIRECTORY=$OPTARG ;; i) @@ -22,36 +22,58 @@ while getopts :d:i:rt1 OPTION ; do else INPUT_NAME="$INPUT_NAME $OPTARG" fi - INPUT_COUNT=$(($INPUT_COUNT+1)) + INPUT_COUNT=$((INPUT_COUNT+1)) ;; r) USE_REDIRECT=1 ;; - t) USE_TABLES=1 ;; - 1) DO_COMPARISON=1 ;; + *) echo "Usage: ${0} [-d INPUT_DIRECTORY] [-i INPUT_NAME] [-r] TESTNAME" + exit 1 + ;; esac done -shift $(($OPTIND-1)) +shift $((OPTIND-1)) TESTNAME=$1 -INPUT_NAME=${INPUT_NAME:-$INPUT_DIRECTORY/`basename "${TESTNAME%.exe}"`.txt} +# There may be a specific input file for this test +INPUT_NAME=${INPUT_NAME:-$INPUT_DIRECTORY/$(basename "${TESTNAME%.exe}").txt} -if [ $DO_COMPARISON = 1 ] ; then - TEST_OUTPUT=`$TESTNAME < $INPUT_NAME` - REF_OUTPUT=`$TESTNAME 1 < $INPUT_NAME` - test "$TEST_OUTPUT" -eq "$REF_OUTPUT" - exit $? -fi +# If it doesn't exist, try stripping out a backend suffix. +# There might be a generic input for all tests with this stem. +# For this purpose we consider _r and _nr to be back ends. +inputs=$INPUT_NAME +INPUT_NAME= +for input in $inputs; do + if [ ! -f "${input}" ] ; then + input=$(echo "${input}" | sed -e 's/_[a-z0-9]*.txt$/.txt/') + fi + if [ "${INPUT_NAME}" = "" ] ; then + INPUT_NAME="${input}" + else + INPUT_NAME="${INPUT_NAME} ${input}" + fi +done + +# Detect if the test wants a table argument. If it does, the test maker will have generated +# a tables=-file option do that the a table file named afte the test was created when the +# scanner was built. Thus we can assume that it will be looking for the table data at +# the matching path we're about to generate. +case ${TESTNAME} in + *ver|*ser|*_ver_*|*_ser_*) USE_TABLES=1 ;; +esac if [ $INPUT_COUNT -gt 1 ] ; then - $TESTNAME ${USE_TABLES:+${INPUT_DIRECTORY}/${TESTNAME%.exe}.tables} ${INPUT_NAME} + # INPUT_NAME has multiple filenames, so we do want word expansion + # shellcheck disable=SC2086 + $TESTNAME ${USE_TABLES:+"${INPUT_DIRECTORY}/${TESTNAME%.exe}.tables"} ${INPUT_NAME} exit $? fi -if [ -f ${INPUT_NAME} ] ; then +# Set up input redirection as required +if [ -f "${INPUT_NAME}" ] ; then if [ $USE_REDIRECT = 1 ] ; then - $TESTNAME ${USE_TABLES:+${INPUT_DIRECTORY}/${TESTNAME%.exe}.tables} < $INPUT_NAME + $TESTNAME ${USE_TABLES:+${INPUT_DIRECTORY}/${TESTNAME%.exe}.tables} < "$INPUT_NAME" else - $TESTNAME ${USE_TABLES:+${INPUT_DIRECTORY}/${TESTNAME%.exe}.tables} $INPUT_NAME + $TESTNAME ${USE_TABLES:+${INPUT_DIRECTORY}/${TESTNAME%.exe}.tables} "$INPUT_NAME" fi else $TESTNAME diff --git a/tests/top.l b/tests/top.l index 25f4773..cefb2e7 100644 --- a/tests/top.l +++ b/tests/top.l @@ -24,7 +24,8 @@ %{ /* Build "scanner.c". The scanner is not important. - This test is really about compilation. See "main.c". + This test is backend-independent; what it's really testing + is Flex's ability to accumulate and ship preamble code sections. */ #include <stdio.h> #include <stdlib.h> diff --git a/tests/vartrailing.rules b/tests/vartrailing.rules new file mode 100644 index 0000000..4d4a37f --- /dev/null +++ b/tests/vartrailing.rules @@ -0,0 +1,57 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +/* The goal here is to test the yylineno in the context of trailing-contexts. + * Using rules that have newlines in look-ahead. + */ + +%option 8bit +%option nounput nomain noyywrap noinput yylineno +%option warn + +WORD [[:alpha:]]+ + +%% +"Fixed_trailing:"/[\n]"test"[\n] {} +"Var_trailing:"{WORD}/[\n] {} +"Var_prefix_and_trailing:"{WORD}":"/(\n{WORD})* {} +\n {} +. {} +<<EOF>> {M4_TEST_ASSERT(yylineno==16) M4_TEST_DO(yyterminate())} + +### +We are testing rules with trailing contexts containing newlines (see scanner.l): + +Fixed_trailing: +test + +Var_trailing:word +test + +Var_prefix_and_trailing:word: +more +text +comes +here + + diff --git a/tests/yyextra_c99.l b/tests/yyextra_c99.l new file mode 100644 index 0000000..4486882 --- /dev/null +++ b/tests/yyextra_c99.l @@ -0,0 +1,120 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + +%{ +/* A file to build "scanner.c". */ +/* This tests that we can use "yyextra". + We buffer all input into a growable array, then print it. + We run diff on the input and output. +*/ + +#include <stdio.h> +#include <stdlib.h> +#include "config.h" + + +/* We'll store the entire input in this buffer, growing as necessary. */ +struct Buffer { + int curr_len; + int max_len; + int grow_len; + char * data; +}; + +/* Save char into junk array at next position. */ +static void append_char (char c, yyscan_t scanner ); + +%} + +%option emit="c99" +%option 8bit prefix="test" +%option nounput nomain noyywrap nodefault noyyinput +%option warn +%option extra-type="struct Buffer *" + +%% + +.|\r|\n { append_char (yytext[0],yyscanner); } + +%% + +int main(void); + +int +main (void) +{ + yyscan_t scanner; + struct Buffer * buf; + int i; + + buf = malloc(sizeof(struct Buffer)); + buf->curr_len =0; + buf->max_len = 4; + buf->grow_len = 100; + buf->data = malloc((size_t) buf->max_len); + + testlex_init(&scanner); + testset_in( stdin, scanner); + testset_out( stdout, scanner); + testset_extra( buf, scanner ); + testlex(scanner); + + buf = testget_extra(scanner); + for(i=0; i < buf->curr_len; i++) + fputc( buf->data[i], stdout ); + free( buf->data); + free( buf); + + testlex_destroy(scanner); + return 0; +} + +/* Save char into junk array at next position. */ +static void append_char (char c, yyscan_t scanner ) +{ + struct Buffer *buf, *new_buf; + buf = testget_extra(scanner); + + /* Grow buffer if necessary. */ + + if( buf->curr_len >= buf->max_len ) + { + new_buf = malloc(sizeof(struct Buffer)); + new_buf->max_len = buf->max_len + buf->grow_len; + new_buf->grow_len = buf->grow_len; + new_buf->data = malloc((size_t) new_buf->max_len); + for( new_buf->curr_len = 0; + new_buf->curr_len < buf->curr_len; + new_buf->curr_len++ ) + { + new_buf->data[ new_buf->curr_len] = buf->data [ new_buf->curr_len]; + } + free( buf->data ); + free( buf ); + buf = new_buf; + testset_extra( buf, scanner ); + } + + + buf->data[ buf->curr_len++ ] = c; +} diff --git a/tests/yyextra.l b/tests/yyextra_nr.l index bf2a6eb..5a5a014 100644 --- a/tests/yyextra.l +++ b/tests/yyextra_nr.l @@ -40,7 +40,6 @@ struct Buffer { int grow_len; char * data; }; -#define YY_EXTRA_TYPE struct Buffer * /* Save char into junk array at next position. */ static void append_char (char c, yyscan_t scanner ); @@ -48,10 +47,10 @@ static void append_char (char c, yyscan_t scanner ); %} %option 8bit prefix="test" -%option nounput nomain noyywrap nodefault noinput +%option nounput nomain noyywrap nodefault noyyinput %option warn %option reentrant - +%option extra-type="struct Buffer *" %% diff --git a/tests/yyless.rules b/tests/yyless.rules new file mode 100644 index 0000000..8d02d16 --- /dev/null +++ b/tests/yyless.rules @@ -0,0 +1,40 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + /* + * Test yyless. What should haooen is: + * 1. "foobar" is matched by the first rule. + * 2. "bar" is pushed back by yyless(3) + * 3. "bar" is matched by the second rule + * 4. Trailing \n is matched and discareded + * 5. Final rule is never reached. + */ + +%option nounput noyywrap noyylineno warn nodefault noinput +%% +foobar {M4_TEST_DO(yyless(3))} +bar ; +\n ; +. {M4_TEST_FAILMESSAGE} + +### +foobar diff --git a/tests/yymore.rules b/tests/yymore.rules new file mode 100644 index 0000000..50c0b87 --- /dev/null +++ b/tests/yymore.rules @@ -0,0 +1,35 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + /* + * Test yymore with default pointer buffer. What should happen is: + * 1. "foo" is matched by the first rule but not consumed. + * 2. "bar" is matched, but the token length is 6 rather than 3. + * We use a ^ to force the extra code for BOL checking. + */ + +%option nounput noyywrap noyylineno warn noinput +%% +foo {M4_TEST_DO(yymore())} +bar {M4_TEST_ASSERT(yyleng == 6)} +### +^foobar diff --git a/tests/yymorearray.rules b/tests/yymorearray.rules new file mode 100644 index 0000000..270ff33 --- /dev/null +++ b/tests/yymorearray.rules @@ -0,0 +1,36 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + /* + * Test yymore with array buffer representation. What should happen is: + * 1. "foo" is matched by the first rule but not consumed. + * 2. "bar" is matched, but the token length is 6 rather than 3. + * Also see the yymorearraybol ruleset. + */ + +%array +%option nounput noyywrap noyylineno warn noinput +%% +foo {M4_TEST_DO(yymore())} +bar {M4_TEST_ASSERT(yyleng == 6)} +### +foobar diff --git a/tests/yymorearraybol.rules b/tests/yymorearraybol.rules new file mode 100644 index 0000000..8683ac2 --- /dev/null +++ b/tests/yymorearraybol.rules @@ -0,0 +1,39 @@ +%{ +#include <stdio.h> +%} +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED AS IS AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + /* + * Test yymore with array buffer representation. What should happen is: + * 1. "foo" is matched by the first rule but not consumed. + * 2. "bar" is matched, but the token length is 6 rather than 3. + * We use a ^ to force the extra code for BOL checking. + */ + +%array +%option nounput noyywrap yylineno warn noinput +%% +^foo {yymore();} +bar {if (!(yyleng == 6)) {fprintf(stderr,"ASSERT FAILED.\n"); exit(1);}} +### +foobar diff --git a/tests/yyunput.rules b/tests/yyunput.rules new file mode 100644 index 0000000..1979e51 --- /dev/null +++ b/tests/yyunput.rules @@ -0,0 +1,34 @@ +/* + * This file is part of flex. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR + * IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED + * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR + * PURPOSE. + */ + /* A test of the yyunput function */ + +%option 8bit +%option nomain noyywrap noinput +%option warn yylineno array +%% + +foo {yyunput('k');} +kick ; +\n ; +. {M4_TEST_FAILMESSAGE} +### |