From 76e0622b687d795bb1379cf183c6ce8613e14658 Mon Sep 17 00:00:00 2001 From: David Gibson Date: Thu, 26 Jun 2008 17:08:57 +1000 Subject: dtc: Clean up lexing of include files Currently we scan the /include/ directive as two tokens, the "/include/" keyword itself, then the string giving the file name to include. We use a special scanner state to keep the two linked together, and use the scanner state stack to keep track of the original state while we're parsing the two /include/ tokens. This does mean that we need to enable the 'stack' option in flex, which results in a not-easily-suppressed warning from the flex boilerplate code. This is mildly irritating. However, this two-token scanning of the /include/ directive also has some extremely strange edge cases, because there are a variety of tokens recognized in all scanner states, including INCLUDE. For example the following strange dts file: /include/ /dts-v1/; / { /* ... */ }; Will be processed successfully with the /include/ being effectively ignored: the '/dts-v1/' and ';' are recognized even in INCLUDE state, then the ';' transitions us to PROPNODENAME state, throwing away INCLUDE, and the previous state is never popped off the stack. Or for another example this construct: foo /include/ = "somefile.dts" will be parsed as though it were: foo = /include/ "somefile.dts" Again, the '=' is scanned without leaving INCLUDE state, then the next string triggers the include logic. And finally, we use a different regexp for the string with the included filename than the normal string regexpt, which is also potentially weird. This patch, therefore, cleans up the lexical handling of the /include/ directive. Instead of the INCLUDE state, we instead scan the whole include directive, both keyword and filename as a single token. This does mean a bit more complexity in extracting the filename out of yytext, but I think it's worth it to avoid the strageness described above. It also means it's no longer possible to put a comment between the /include/ and the filename, but I'm really not very worried about breaking files using such a strange construct. --- convert-dtsv0-lexer.l | 24 ++++++++---------------- 1 file changed, 8 insertions(+), 16 deletions(-) (limited to 'convert-dtsv0-lexer.l') diff --git a/convert-dtsv0-lexer.l b/convert-dtsv0-lexer.l index 64e2916..12b45ea 100644 --- a/convert-dtsv0-lexer.l +++ b/convert-dtsv0-lexer.l @@ -17,7 +17,7 @@ * USA */ -%option noyywrap nounput stack +%option noyywrap nounput %x INCLUDE %x BYTESTRING @@ -26,6 +26,11 @@ PROPNODECHAR [a-zA-Z0-9,._+*#?@-] PATHCHAR ({PROPNODECHAR}|[/]) LABEL [a-zA-Z_][a-zA-Z0-9_]* +STRING \"([^\\"]|\\.)*\" +WS [[:space:]] +COMMENT "/*"([^*]|\*+[^*/])*\*+"/" +LINECOMMENT "//".*\n +GAP ({WS}|{COMMENT}|{LINECOMMENT})* %{ #include @@ -91,16 +96,7 @@ const struct { %} %% -<*>"/include/" { - ECHO; - yy_push_state(INCLUDE); - } - -\"[^"\n]*\" { - ECHO; - yy_pop_state(); - } - +<*>"/include/"{GAP}{STRING} ECHO; <*>\"([^\\"]|\\.)*\" ECHO; @@ -193,11 +189,7 @@ const struct { BEGIN(INITIAL); } -<*>[[:space:]]+ ECHO; - -<*>"/*"([^*]|\*+[^*/])*\*+"/" ECHO; - -<*>"//".*\n ECHO; +<*>{GAP} ECHO; <*>- { /* Hack to convert old style memreserves */ saw_hyphen = 1; -- cgit v1.2.1