diff options
| -rw-r--r-- | ChangeLog | 13 | ||||
| -rw-r--r-- | NEWS | 11 | ||||
| -rw-r--r-- | doc/m4.texinfo | 242 | ||||
| -rw-r--r-- | src/freeze.c | 85 | ||||
| -rw-r--r-- | tests/freeze.at | 42 |
5 files changed, 302 insertions, 91 deletions
@@ -1,3 +1,16 @@ +2008-05-18 Eric Blake <ebb9@byu.net> + + Allow freezing the trace status of macros. + * src/freeze.c (produce_symbol_dump): Let undefined traced macros + through. + (dump_symbol_CB): Also freeze trace state. + * tests/freeze.at (reloading traced macros): New test. + * doc/m4.texinfo (Using frozen files): Update documentation to + mention new state. + (Frozen file format 1): Improve synchronization with branch. + (Frozen file format 2): Reorder directives, and add `d', `t'. + * NEWS: Document this change. + 2008-05-15 Eric Blake <ebb9@byu.net> Fix frozen file regression in pushdef stacks from 2001-09-01. @@ -195,11 +195,12 @@ promoted to 2.0. *** The syntax of frozen files format V2 has been improved to save additional state. This includes the `R' directive for default regular - expression syntax. Also, a V2 file can now be represented completely - in ASCII, thanks to escape sequences. Unfortunately, files frozen by - M4 1.4q that contain \ in macros cannot be read by 1.9b, but since 1.4q - was not widely distributed, this is not expected to be much of an - issue, and comes with the territory of using a beta release. + expression syntax, the `t' directive for traced macros, and the `d' + directive for debug mode. Also, a V2 file can now be represented + completely in ASCII, thanks to escape sequences. Unfortunately, files + frozen by M4 1.4q cannot be read by 1.9b, but since 1.4q was not widely + distributed, this is not expected to be much of an issue, and comes + with the territory of using a beta release. - FIXME: format 2 still needs to catch more missing state; once 2.0 is released, any further changes would introduce format 3. diff --git a/doc/m4.texinfo b/doc/m4.texinfo index 8c4c2370..6978714e 100644 --- a/doc/m4.texinfo +++ b/doc/m4.texinfo @@ -8265,15 +8265,14 @@ $ @kbd{m4 -R file2.m4f -F file3.m4f file3.m4} $ @kbd{m4 -R file3.m4f file4.m4} @end example -@comment FIXME - merge the rest of this section. -Some care is necessary because not every effort has been made for -this to work in all cases. In particular, the trace attribute of -macros is not handled. -Also, interactions for some options of @code{m4} being used in one call -and not for the next, have not been fully analyzed yet. On the other -end, you may be confident that stacks of @code{pushdef}'ed definitions -are handled correctly, so are @code{undefine}'d or renamed builtins, -changed strings for quotes or comments. +Some care is necessary because the frozen file does not save all state +information. Stacks of macro definitions via @code{pushdef} are +accurately stored, along with all renamed or undefined builtins, as are +the current syntax rules such as from @code{changequote}. However, the +value of @code{sysval} and text saved in @code{m4wrap} are not currently +preserved. Also, changing command line options between runs may cause +unexpected behavior. A future release of @acronym{GNU} M4 may improve +on the quality of frozen files. When an @code{m4} run is to be frozen, the automatic undiversion which takes place at end of execution is inhibited. Instead, all @@ -8284,14 +8283,83 @@ A frozen file to be reloaded need not reside in the current directory. It is looked up the same way as an @code{include} file (@pxref{Search Path}). +If the frozen file was generated with a newer version of @code{m4}, and +contains directives that an older @code{m4} cannot parse, attempting to +load the frozen file with option @option{-R} will cause @code{m4} to +exit with status 63 to indicate version mismatch. + @node Frozen file format 1 @section Frozen file format 1 @cindex frozen file format 1 @cindex file format, frozen file version 1 -Wow - thanks for really reading the manual. Report this as a bug if -this text is not removed before a release. -FIXME - split out the two formats into separate nodes. +Frozen files are sharable across architectures. It is safe to write +a frozen file on one machine and read it on another, given that the +second machine uses the same or newer version of @acronym{GNU} @code{m4}. +It is conventional, but not required, to give a frozen file the suffix +of @code{.m4f}. + +Older versions of @acronym{GNU} @code{m4} create frozen files with +syntax version 1. These files can be read by the current version, but +are no longer produced. Version 1 files are mostly text files, although +any macros or diversions that contained nonprintable characters or long +lines cause the resulting frozen file to do likewise, since there are no +escape sequences. The file can be edited to change the state that +@code{m4} will start with. It is composed of several directives, each +starting with a single letter and ending with a newline (@key{NL}). +Wherever a directive is expected, the character @samp{#} can be used +instead to introduce a comment line; empty lines are also ignored if +they are not part of an embedded string. + +In the following descriptions, each @var{len} refers to the length of a +corresponding subsequent @var{str}. Numbers are always expressed in +decimal, and an omitted number defaults to 0. The valid directives in +version 1 are: + +@table @code +@item V @var{number} @key{NL} +Confirms the format of the file. Version 1 is recognized when +@var{number} is 1. This directive must be the first non-comment in the +file, and may not appear more than once. + +@item C @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +Uses @var{str1} and @var{str2} as the begin-comment and +end-comment strings. If omitted, then @samp{#} and @key{NL} are the +comment delimiters. + +@item D @var{number}, @var{len} @key{NL} @var{str} @key{NL} +Selects diversion @var{number}, making it current, then copy @var{str} +in the current diversion. @var{number} may be a negative number for a +diversion that discards text. To merely specify an active selection, +use this command with an empty @var{str}. With 0 as the diversion +@var{number}, @var{str} will be issued on standard output at reload +time. @acronym{GNU} @code{m4} will not produce the @samp{D} directive +with non-zero length for diversion 0, but this can be done with manual +edits. This directive may appear more than once for the same diversion, +in which case the diversion is the concatenation of the various uses. +If omitted, then diversion 0 is current. + +@item F @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +Defines, through @code{pushdef}, a definition for @var{str1} expanding +to the function whose builtin name is @var{str2}. If the builtin does +not exist (for example, if the frozen file was produced by a copy of +@code{m4} compiled with the now-abandoned @code{changeword} support), +the reload is silent, but any subsequent use of the definition of +@var{str1} will result in a warning. This directive may appear more +than once for the same name, and its order, along with @samp{T}, is +important. If omitted, you will have no access to any builtins. + +@item Q @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +Uses @var{str1} and @var{str2} as the begin-quote and end-quote +strings. If omitted, then @samp{`} and @samp{'} are the quote +delimiters. + +@item T @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +Defines, though @code{pushdef}, a definition for @var{str1} +expanding to the text given by @var{str2}. This directive may appear +more than once for the same name, and its order, along with @samp{F}, is +important. +@end table When loading format 1, the syntax categories @samp{@{} and @samp{@}} are disabled (reverting braces to be treated like plain characters). This @@ -8305,75 +8373,105 @@ because a newer version of M4 reloaded the file. @cindex frozen file format 2 @cindex file format, frozen file version 2 -Frozen files are sharable across architectures. It is safe to write -a frozen file on one machine and read it on another, given that the -second machine uses the same, or a newer version of GNU @code{m4}. -These are simple (editable) text files, made up of directives, -each starting with a capital letter and ending with a newline -(@key{NL}). Wherever a directive is expected, the character -@samp{#} introduces a comment line, empty lines are also ignored. -In the following descriptions, @var{length}s always refer to -corresponding @var{string}s. Numbers are always expressed in decimal. -The directives are: +The syntax of version 1 has some drawbacks; if any macro or diversion +contained non-printable characters or long lines, the resulting frozen +file would not qualify as a text file, making it harder to edit with +some vendor tools. The concatenation of multiple strings on a single +line, such as for the @samp{T} directive, makes distinguishing the two +strings a bit more difficult. Finally, the format lacks support for +several items of @code{m4} state, such that a reloaded file did not +always behave the same as the original file. + +These shortcomings have been addressed in version 2 of the frozen file +syntax. New directives have been added, and existing directives have +additional, and sometimes optional, parameters. +@c FIXME - change implementation to match this: +@c All @var{str} instances +@c in the grammar are now followed by @key{NL}, which makes the split +@c between consecutive strings easier to recognize. +Strings may now +contain escape sequences modeled after C, such as @samp{\n} for newline +or @samp{\0} for @sc{nul}, so that the frozen file can be pure +@sc{ascii} (although when hand-editing a frozen file, it is still +acceptable to use the original byte rather than an escape sequence for +all bytes except @samp{\}). Also in the context of a @var{str}, the +escape sequence @samp{\@key{NL}} is discarded, allowing a user to split +lines that are too long for some platform tools. @table @code @item V @var{number} @key{NL} -Confirms the format of the file. For the version documented here, -@var{number} should be 2. It is backwards compatible with the previous -version though, so version 1 frozen files can be loaded too if necessary. - -@item C @var{length1} , @var{length2} @key{NL} @var{string1} @var{string2} @key{NL} -Uses @var{string1} and @var{string2} as the beginning comment and -end comment strings. - -@item Q @var{length1} , @var{length2} @key{NL} @var{string1} @var{string2} @key{NL} -Uses @var{string1} and @var{string2} as the beginning quote and end quote -strings. - -@item R @var{length} @key{NL} @var{string} @key{NL} -Sets the default regexp syntax, where @var{string} encodes one of the +Confirms the format of the file. @code{m4} @value{VERSION} only creates +frozen files where @var{number} is 2. This directive must be the first +non-comment in the file, and may not appear more than once. + +@c C @var{len1} , @var{len2} @key{NL} @var{str1} @key{NL} @var{str2} @key{NL} +@item C @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +Uses @var{str1} and @var{str2} as the begin-comment and +end-comment strings. If omitted, then @samp{#} and @key{NL} are the +comment delimiters. + +@item d @var{len} @key{NL} @var{str} @key{NL} +Sets the debug flags, using @var{str} as the argument to +@code{debugmode}. If omitted, then the debug flags start in their +default state. + +@item D @var{number} , @var{len} @key{NL} @var{str} @key{NL} +Selects diversion @var{number}, making it current, then copy @var{str} +in the current diversion. @var{number} may be a negative number for a +diversion that discards text. To merely specify an active selection, +use this command with an empty @var{string}. With 0 as the diversion +@var{number}, @var{str} will be issued on standard output at reload +time. @acronym{GNU} @code{m4} will not produce the @samp{D} directive +with non-zero length for diversion 0, but this can be done with manual +edits. This directive may appear more than once for the same diversion, +in which case the diversion is the concatenation of the various uses. +If omitted, then diversion 0 is current. + +@comment FIXME - the first usage, with only one string, is not supported +@comment in the current code +@item F @var{len1} @key{NL} @var{str1} @key{NL} +@itemx F @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +@itemx F @var{len1} , @var{len2} , @var{len3} @key{NL} @var{str1} @var{str2} @var{str3} @key{NL} +Defines, through @code{pushdef}, a definition for @var{str1} expanding +to the function whose builtin name is given by @var{str2} (defaulting to +@var{str1} if not present). With two arguments, the builtin name is +searched for among the intrinsic builtin functions only; with three +arguments, the builtin name is searched for amongst the builtin +functions defined by the module named by @var{str3}. + +@item M @var{len} @key{NL} @var{str} @key{NL} +Names a module which will be searched for according to the module search +path and loaded. Modules loaded from a frozen file don't add their +builtin entries to the symbol table. Modules must be loaded prior to +specifying module-specific builtins via the three-argument @code{F} or +@code{T}. + +@item Q @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +Uses @var{str1} and @var{str2} as the begin-quote and end-quote strings. +If omitted, then @samp{`} and @samp{'} are the quote delimiters. + +@item R @var{len} @key{NL} @var{str} @key{NL} +Sets the default regexp syntax, where @var{str} encodes one of the regular expression syntaxes supported by @acronym{GNU} M4. @xref{Changeresyntax}, for more details. -@item M @var{length} @key{NL} @var{string} @key{NL} -Names a module which will be searched for according to the module search path -and loaded. Modules loaded from a frozen file don't add their builtin entries -to the symbol table. - -@item F @var{length} @key{NL} @var{string} @key{NL} -Defines, through @code{pushdef}, a definition for @var{string} -expanding to the function whose builtin name is also @var{string}. The builtin -name is searched for among the intrinsic builtin functions only. - -@item F @var{length1} , @var{length2} @key{NL} @var{string1} @var{string2} @key{NL} -Defines, through @code{pushdef}, a definition for @var{string1} -expanding to the function whose builtin name is @var{string2}. With two -arguments, the builtin name is searched for among the intrinsic builtin -functions only. - -@item F @var{length1} , @var{length2} , @var{length3} @key{NL} @var{string1} @var{string2} @var{string3} @key{NL} -Defines, through @code{pushdef}, a definition for @var{string1} -expanding to the function whose builtin name is @var{string2}. With three -arguments, the builtin name is searched for amongst the builtin functions -defined by the module named by @var{string3}. - -@item S @var{syntax-code} @var{length} @key{NL} @var{string} @key{NL} +@item S @var{syntax-code} @var{len} @key{NL} @var{str} @key{NL} Defines, through @code{changesyntax}, a syntax category for each of the -characters in @var{string}. The @var{syntax-code} must be one of the +characters in @var{str}. The @var{syntax-code} must be one of the characters described in @ref{Changesyntax}. -@item T @var{length1} , @var{length2} @key{NL} @var{string1} @var{string2} @key{NL} -Defines, though @code{pushdef}, a definition for @var{string1} -expanding to the text given by @var{string2}. - -@item D @var{number}, @var{length} @key{NL} @var{string} @key{NL} -Selects diversion @var{number}, making it current, then copy -@var{string} in the current diversion. @var{number} may be a negative -number for a non-existing diversion. To merely specify an active -selection, use this command with an empty @var{string}. With 0 as the -diversion @var{number}, @var{string} will be issued on standard output -at reload time, however this may not be produced from within @code{m4}. - +@item t @var{len} @key{NL} @var{str} @key{NL} +Enables tracing for any macro named @var{str}, similar to using the +@code{traceon} builtin. This option may occur more than once for +multiple macros; if omitted, no macro starts out as traced. + +@comment FIXME - what about a module's text macros, like __gnu__? +@item T @var{len1} , @var{len2} @key{NL} @var{str1} @var{str2} @key{NL} +Defines, though @code{pushdef}, a definition for @var{str1} expanding to +the text given by @var{str2}. With two arguments, the builtin name is +searched for among the intrinsic builtin functions only; with three +arguments, the builtin name is searched for amongst the builtin +functions defined by the module named by @var{str3}. @end table @node Compatibility diff --git a/src/freeze.c b/src/freeze.c index ac67d565..db5a1cf7 100644 --- a/src/freeze.c +++ b/src/freeze.c @@ -103,6 +103,43 @@ produce_syntax_dump (FILE *file, m4_syntax_table *syntax, char ch) } } +/* Store the debug mode in textual format. */ +static void +produce_debugmode_state (FILE *file, int flags) +{ + /* This code tracks the number of bits in M4_DEBUG_TRACE_VERBOSE. */ + char str[13]; + int offset = 0; + assert ((1 << (sizeof str - 1)) - 1 == M4_DEBUG_TRACE_VERBOSE); + if (flags & M4_DEBUG_TRACE_ARGS) + str[offset++] = 'a'; + if (flags & M4_DEBUG_TRACE_EXPANSION) + str[offset++] = 'e'; + if (flags & M4_DEBUG_TRACE_QUOTE) + str[offset++] = 'q'; + if (flags & M4_DEBUG_TRACE_ALL) + str[offset++] = 't'; + if (flags & M4_DEBUG_TRACE_LINE) + str[offset++] = 'l'; + if (flags & M4_DEBUG_TRACE_FILE) + str[offset++] = 'f'; + if (flags & M4_DEBUG_TRACE_PATH) + str[offset++] = 'p'; + if (flags & M4_DEBUG_TRACE_CALL) + str[offset++] = 'c'; + if (flags & M4_DEBUG_TRACE_INPUT) + str[offset++] = 'i'; + if (flags & M4_DEBUG_TRACE_CALLID) + str[offset++] = 'x'; + if (flags & M4_DEBUG_TRACE_MODULE) + str[offset++] = 'm'; + if (flags & M4_DEBUG_TRACE_STACK) + str[offset++] = 's'; + str[offset] = '\0'; + if (offset) + xfprintf (file, "d%d\n%s\n", offset, str); +} + /* The modules must be dumped in the order in which they will be reloaded from the frozen file. libltdl stores handles in a push down stack, so we need to dump them in the reverse order to that. */ @@ -127,7 +164,7 @@ produce_module_dump (FILE *file, m4_module *module) static void produce_symbol_dump (m4 *context, FILE *file, m4_symbol_table *symtab) { - if (m4_symtab_apply (symtab, false, dump_symbol_CB, file)) + if (m4_symtab_apply (symtab, true, dump_symbol_CB, file)) assert (false); } @@ -210,6 +247,8 @@ dump_symbol_CB (m4_symbol_table *symtab, const char *symbol_name, value = VALUE_NEXT (value); } reverse_symbol_value_stack (last); + if (m4_get_symbol_traced (symbol)) + xfprintf (file, "t%zu\n%s\n", symbol_len, symbol_name); return NULL; } @@ -254,15 +293,16 @@ produce_frozen_state (m4 *context, const char *name) } /* Dump regular expression syntax. */ - produce_resyntax_dump (context, file); /* Dump syntax table. */ - str = "I@WLBOD${}SA(),RE"; while (*str) produce_syntax_dump (file, M4SYNTAX, *str++); + /* Dump debugmode state. */ + produce_debugmode_state (file, m4_get_debug_level_opt (context)); + /* Dump all loaded modules. */ produce_module_dump (file, m4__module_next (NULL)); @@ -548,6 +588,26 @@ reload_frozen_state (m4 *context, const char *name) _("ill-formed frozen file, unknown directive %c"), character); + case 'd': + /* Set debugmode flags. */ + if (version < 2) + { + /* 'd' operator is not supported in format version 1. */ + m4_error (context, EXIT_FAILURE, 0, NULL, _("\ +ill-formed frozen file, version 2 directive `%c' encountered"), 'd'); + } + + GET_CHARACTER; + GET_NUMBER (number[0], false); + VALIDATE ('\n'); + GET_STRING (file, string[0], allocated[0], number[0]); + VALIDATE ('\n'); + + m4_set_debug_level_opt (context, m4_debug_decode (context, 0, + string[0])); + + break; + case 'F': GET_CHARACTER; @@ -686,6 +746,25 @@ ill-formed frozen file, version 2 directive `%c' encountered"), 'S'); } break; + case 't': + /* Trace a macro name. */ + if (version < 2) + { + /* 't' operator is not supported in format version 1. */ + m4_error (context, EXIT_FAILURE, 0, NULL, _("\ +ill-formed frozen file, version 2 directive `%c' encountered"), 't'); + } + + GET_CHARACTER; + GET_NUMBER (number[0], false); + VALIDATE ('\n'); + GET_STRING (file, string[0], allocated[0], number[0]); + VALIDATE ('\n'); + + m4_set_symbol_name_traced (M4SYMTAB, string[0], true); + + break; + case 'C': case 'D': case 'Q': diff --git a/tests/freeze.at b/tests/freeze.at index e8ada67e..209c6115 100644 --- a/tests/freeze.at +++ b/tests/freeze.at @@ -301,6 +301,22 @@ define{`a.b', `hello $1'}dnl [[a.b{world} ]]) +## ------- ## +## pushdef ## +## ------- ## + +# Check for pushdef stacks; broken 2001-09-01, fixed 2008-05-15. +AT_TEST_FREEZE([reloading pushdef stack], +[[pushdef(`foo', `1') +pushdef(`foo', defn(`len')) +pushdef(`foo', `3') +]], +[[foo(`abc')popdef(`foo') +foo(`ab')popdef(`foo') +foo(`a')popdef(`foo') +foo +]]) + ## ------------- ## ## regexp syntax ## ## ------------- ## @@ -313,20 +329,24 @@ AT_TEST_FREEZE([reloading regexp syntax], regexp(`GNUs not Unix', `\w\(\w*\)$', `GNU_M4') ]]) -## ------- ## -## pushdef ## -## ------- ## +## ----- ## +## trace ## +## ----- ## -# Check for pushdef stacks; broken 2001-09-01, fixed 2008-05-15. -AT_TEST_FREEZE([reloading pushdef stack], -[[pushdef(`foo', `1') -pushdef(`foo', defn(`len')) -pushdef(`foo', `3') +# Check for macro tracing, both single and global. +AT_TEST_FREEZE([reloading traced macros], +[[define(`text', `hello world')dnl +define(`foo', `bar')dnl +traceon(`blah', `divnum', `text')dnl +traceon ]], -[[foo(`abc')popdef(`foo') -foo(`ab')popdef(`foo') -foo(`a')popdef(`foo') +[[foo +traceoff foo +text +divnum +ifdef(`blah', `', `define(`blah', `finally')')dnl +blah ]]) ## ---------------- ## |