Document join, in order to fix bug in m4wrap example.

* examples/join.m4: New file.
* examples/wraplifo2.m4: Likewise.
* Makefile.am (EXTRA_DIST): Add new files.
* doc/m4.texinfo (Improved m4wrap): New node.
(Defn, Location): Enhance tests.
(Shift): Document the composit macro join.
(Incompatibilities): Move documentation of LIFO vs. FIFO...
(M4wrap): ...here, to match improved example.

Signed-off-by: Eric Blake <ebb9@byu.net>

5 files changed, 321 insertions, 1 deletions

diff --git a/ChangeLog b/ChangeLog
index 8ac919ec..6e602e1e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2008-03-15  Eric Blake  <ebb9@byu.net>
+
+	Document join, in order to fix bug in m4wrap example.
+	* examples/join.m4: New file.
+	* examples/wraplifo2.m4: Likewise.
+	* Makefile.am (EXTRA_DIST): Add new files.
+	* doc/m4.texinfo (Improved m4wrap): New node.
+	(Defn, Location): Enhance tests.
+	(Shift): Document the composit macro join.
+	(Incompatibilities): Move documentation of LIFO vs. FIFO...
+	(M4wrap): ...here, to match improved example.
+
 2008-03-13  Eric Blake  <ebb9@byu.net>
 
 	Stage 19c: allow builtin tokens in more macros.
diff --git a/Makefile.am b/Makefile.am
index 2ef72617..1f53c7c5 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -329,6 +329,7 @@ dist_pkgdata_DATA = \
 		  examples/incl.m4 \
 		  examples/include.m4 \
 		  examples/indir.m4 \
+		  examples/join.m4 \
 		  examples/loop.m4 \
 		  examples/misc.m4 \
 		  examples/multiquotes.m4 \
@@ -342,7 +343,10 @@ dist_pkgdata_DATA = \
 		  examples/translit.m4 \
 		  examples/undivert.incl \
 		  examples/undivert.m4 \
-		  examples/wrap.m4
+		  examples/wrap.m4 \
+		  examples/wrapfifo.m4 \
+		  examples/wraplifo.m4 \
+		  examples/wraplifo2.m4
 
 ## ----------- ##
 ## Test suite. ##
diff --git a/doc/m4.texinfo b/doc/m4.texinfo
index 6d024b3f..175d923b 100644
--- a/doc/m4.texinfo
+++ b/doc/m4.texinfo
@@ -288,6 +288,7 @@ Correct version of some examples
 * Improved exch::               Solution for @code{exch}
 * Improved forloop::            Solution for @code{forloop}
 * Improved foreach::            Solution for @code{foreach}
+* Improved m4wrap::             Solution for @code{m4wrap}
 * Improved cleardivert::        Solution for @code{cleardivert}
 * Improved capitalize::         Solution for @code{capitalize}
 * Improved fatal_error::        Solution for @code{fatal_error}
@@ -2337,6 +2338,11 @@ dumpdef(`')
 m4symbols(defn(`divnum'))
 @error{}m4:stdin:19: Warning: m4symbols: invalid macro name ignored
 @result{}
+define(`foo', `define(`$1', $2)')dnl
+foo(`bar', defn(`divnum'))
+@result{}
+bar
+@result{}0
 @end example
 
 A warning is issued if @var{name} is undefined.  Also, at present,
@@ -3065,6 +3071,8 @@ shift(`foo', `bar', `baz')
 
 An example of the use of @code{shift} is this macro:
 
+@cindex reversing arguments
+@cindex arguments, reversing
 @deffn Composite reverse (@dots{})
 Takes any number of arguments, and reverses their order.
 @end deffn
@@ -3142,6 +3150,113 @@ example2(`feeling rather indecisive today')
 @result{}default answer: 4
 @end example
 
+@cindex joining arguments
+@cindex arguments, joining
+@cindex concatenating arguments
+Another common task that requires iteration is joining a list of
+arguments into a single string.
+
+@deffn Composite join (@ovar{separator}, @ovar{args@dots{}})
+@deffnx Composite joinall (@ovar{separator}, @ovar{args@dots{}})
+Generate a single-quoted string, consisting of each @var{arg} separated
+by @var{separator}.  While @code{joinall} always outputs a
+@var{separator} between arguments, @code{join} avoids the
+@var{separator} for an empty @var{arg}.
+@end deffn
+
+Here are some examples of its usage, based on the implementation
+@file{m4-@value{VERSION}/@/examples/@/join.m4} distributed in this
+package:
+
+@comment examples
+@example
+$ @kbd{m4 -I examples}
+include(`join.m4')
+@result{}
+join,join(`-'),join(`-', `'),join(`-', `', `')
+@result{},,,
+joinall,joinall(`-'),joinall(`-', `'),joinall(`-', `', `')
+@result{},,,-
+join(`-', `1')
+@result{}1
+join(`-', `1', `2', `3')
+@result{}1-2-3
+join(`', `1', `2', `3')
+@result{}123
+join(`-', `', `1', `', `', `2', `')
+@result{}1-2
+joinall(`-', `', `1', `', `', `2', `')
+@result{}-1---2-
+join(`,', `1', `2', `3')
+@result{}1,2,3
+define(`nargs', `$#')dnl
+nargs(join(`,', `1', `2', `3'))
+@result{}1
+@end example
+
+Examining the implementation shows some interesting points about several
+m4 programming idioms.
+
+@comment examples
+@example
+$ @kbd{m4 -I examples}
+undivert(`join.m4')dnl
+@result{}divert(`-1')
+@result{}# join(sep, args) - join each non-empty ARG into a single
+@result{}# string, with each element separated by SEP
+@result{}define(`join',
+@result{}`ifelse(`$#', `2', ``$2'',
+@result{}  `ifelse(`$2', `', `', ``$2'_')$0(`$1', shift(shift($@@)))')')
+@result{}define(`_join',
+@result{}`ifelse(`$#$2', `2', `',
+@result{}  `ifelse(`$2', `', `', ``$1$2'')$0(`$1', shift(shift($@@)))')')
+@result{}# joinall(sep, args) - join each ARG, including empty ones,
+@result{}# into a single string, with each element separated by SEP
+@result{}define(`joinall', ``$2'_$0(`$1', shift($@@))')
+@result{}define(`_joinall',
+@result{}`ifelse(`$#', `2', `', ``$1$3'$0(`$1', shift(shift($@@)))')')
+@result{}divert`'dnl
+@end example
+
+First, notice that this implementation creates helper macros
+@code{_join} and @code{_joinall}.  This division of labor makes it
+easier to output the correct number of @var{separator} instances:
+@code{join} and @code{joinall} are responsible for the first argument,
+without a separator, while @code{_join} and @code{_joinall} are
+responsible for all remaining arguments, always outputting a separator
+when outputting an argument.
+
+Next, observe how @code{join} decides to iterate to itself, because the
+first @var{arg} was empty, or to output the argument and swap over to
+@code{_join}.  If the argument is non-empty, then the nested
+@code{ifelse} results in an unquoted @samp{_}, which is concatenated
+with the @samp{$0} to form the next macro name to invoke.  The
+@code{joinall} implementation is simpler since it does not have to
+suppress empty @var{arg}; it always executes once then defers to
+@code{_joinall}.
+
+Another important idiom is the idea that @var{separator} is reused for
+each iteration.  Each iteration has one less argument, but rather than
+discarding @samp{$1} by iterating with @code{$0(shift($@@))}, the macro
+discards @samp{$2} by using @code{$0(`$1', shift(shift($@@)))}.
+
+Next, notice that it is possible to compare more than one condition in a
+single @code{ifelse} test.  The test of @samp{$#$2} against @samp{2}
+allows @code{_join} to iterate for two separate reasons---either there
+are still more than two arguments, or there are exactly two arguments
+but the last argument is not empty.
+
+Finally, notice that these macros require exactly two arguments to
+terminate recursion, but that they still correctly result in empty
+output when given no @var{args} (i.e., zero or one macro argument).  On
+the first pass when there are too few arguments, the @code{shift}
+results in no output, but leaves an empty string to serve as the
+required second argument for the second pass.  Put another way,
+@samp{`$1', shift($@@)} is not the same as @samp{$@@}, since only the
+former guarantees at least two arguments.
+
+@cindex quote manipulation
+@cindex manipulating quotes
 Sometimes, a recursive algorithm requires adding quotes to each element,
 or treating multiple arguments as a single element:
 
@@ -3208,6 +3323,37 @@ undivert(`quote.m4')dnl
 @result{}divert`'dnl
 @end example
 
+It is worth pointing out that @samp{quote(@var{args})} is more efficient
+than @samp{joinall(`,', @var{args})} for producing the same output.
+
+@cindex nine arguments, more than
+@cindex more than nine arguments
+@cindex arguments, more than nine
+One more useful macro based on @code{shift} allows portably selecting
+an arbitrary argument (usually greater than the ninth argument), without
+relying on the @acronym{GNU} extension of multi-digit arguments
+(@pxref{Arguments}).
+
+@deffn Composite argn (@var{n}, @dots{})
+Expands to argument @var{n} out of the remaining arguments.  @var{n}
+must be a positive number.  Usually invoked as
+@samp{argn(`@var{n}',$@@)}.
+@end deffn
+
+It is implemented as:
+
+@example
+define(`argn', `ifelse(`$1', 1, ``$2'',
+  `argn(decr(`$1'), shift(shift($@@)))')')
+@result{}
+argn(`1', `a')
+@result{}a
+define(`foo', `argn(`11', $@@)')
+@result{}
+foo(`a', `b', `c', `d', `e', `f', `g', `h', `i', `j', `k', `l')
+@result{}k
+@end example
+
 @node Forloop
 @section Iteration by counting
 
@@ -5088,6 +5234,64 @@ which the saved text is reread is undefined.  If @code{m4wrap} is not used
 recursively, the saved pieces of text are reread in the opposite order
 in which they were saved (LIFO---last in, first out).
 
+It is possible to emulate @acronym{POSIX} behavior even
+with older versions of @acronym{GNU} M4 by including the file
+@file{m4-@value{VERSION}/@/examples/@/wrapfifo.m4} from the
+distribution:
+
+@comment examples
+@example
+$ @kbd{m4 -I examples}
+undivert(`wrapfifo.m4')dnl
+@result{}dnl Redefine m4wrap to have FIFO semantics.
+@result{}define(`_m4wrap_level', `0')dnl
+@result{}define(`m4wrap',
+@result{}`ifdef(`m4wrap'_m4wrap_level,
+@result{}       `define(`m4wrap'_m4wrap_level,
+@result{}               defn(`m4wrap'_m4wrap_level)`$1')',
+@result{}       `builtin(`m4wrap', `define(`_m4wrap_level',
+@result{}                                  incr(_m4wrap_level))dnl
+@result{}m4wrap'_m4wrap_level)dnl
+@result{}define(`m4wrap'_m4wrap_level, `$1')')')dnl
+include(`wrapfifo.m4')
+@result{}
+m4wrap(`a`'m4wrap(`c
+', `d')')m4wrap(`b')
+@result{}
+^D
+@result{}abc
+@end example
+
+It is likewise possible to emulate LIFO behavior without resorting to
+the @acronym{GNU} M4 extension of @code{builtin}, by including the file
+@file{m4-@value{VERSION}/@/examples/@/wraplifo.m4} from the
+distribution.  (Unfortunately, both examples shown here share some
+subtle bugs.  See if you can find and correct them; or @pxref{Improved
+m4wrap, , Answers}).
+
+@comment examples
+@example
+$ @kbd{m4 -I examples}
+undivert(`wraplifo.m4')dnl
+@result{}dnl Redefine m4wrap to have LIFO semantics.
+@result{}define(`_m4wrap_level', `0')dnl
+@result{}define(`_m4wrap', defn(`m4wrap'))dnl
+@result{}define(`m4wrap',
+@result{}`ifdef(`m4wrap'_m4wrap_level,
+@result{}       `define(`m4wrap'_m4wrap_level,
+@result{}               `$1'defn(`m4wrap'_m4wrap_level))',
+@result{}       `_m4wrap(`define(`_m4wrap_level', incr(_m4wrap_level))dnl
+@result{}m4wrap'_m4wrap_level)dnl
+@result{}define(`m4wrap'_m4wrap_level, `$1')')')dnl
+include(`wraplifo.m4')
+@result{}
+m4wrap(`a`'m4wrap(`c
+', `d')')m4wrap(`b')
+@result{}
+^D
+@result{}bac
+@end example
+
 Here is an example of implementing a factorial function using
 @code{m4wrap}:
 
@@ -7570,7 +7774,11 @@ __line__
 @result{}8
 __line__
 @result{}11
+m4wrap(`__line__
+')
+@result{}
 ^D
+@result{}12
 @result{}6
 @result{}6
 @end example
@@ -8287,6 +8495,7 @@ presented here.
 * Improved exch::               Solution for @code{exch}
 * Improved forloop::            Solution for @code{forloop}
 * Improved foreach::            Solution for @code{foreach}
+* Improved m4wrap::             Solution for @code{m4wrap}
 * Improved cleardivert::        Solution for @code{cleardivert}
 * Improved capitalize::         Solution for @code{capitalize}
 * Improved fatal_error::        Solution for @code{fatal_error}
@@ -8712,6 +8921,77 @@ foreachq(`x', ```active'', ``active''', `<x>
 @result{}<active>
 @end example
 
+@node Improved m4wrap
+@section Solution for @code{m4wrap}
+
+The replacement @code{m4wrap} versions presented above, designed to
+guarantee FIFO or LIFO order regardless of the underlying M4
+implementation, share a bug when dealing with wrapped text that looks
+like parameter expansion.  Note how the invocation of
+@code{m4wrap@var{n}} interprets these parameters, while using the
+builtin preserves them for their intended use.
+
+@comment examples
+@example
+$ @kbd{m4 -I examples}
+include(`wraplifo.m4')
+@result{}
+m4wrap(`define(`foo', ``$0:'-$1-$*-$#-')foo(`a', `b')
+')
+@result{}
+builtin(`m4wrap', ``'define(`bar', ``$0:'-$1-$*-$#-')bar(`a', `b')
+')
+@result{}
+^D
+@result{}bar:-a-a,b-2-
+@result{}m4wrap0:---0-
+@end example
+
+Additionally, the computation of @code{_m4wrap_level} and creation of
+multiple @code{m4wrap@var{n}} placeholders in the original examples is
+more expensive in time and memory than strictly necessary.  Notice how
+the improved version grabs the wrapped text via @code{defn} to avoid
+parameter expansion, then undefines @code{_m4wrap_text}, before
+stripping a level of quotes with @code{_arg1} to expand the text.  That
+way, each level of wrapping reuses the single placeholder, which starts
+each nesting level in an undefined state.
+
+Finally, it is worth emulating the @acronym{GNU} M4 extension of saving
+all arguments to @code{m4wrap}, separated by a space, rather than saving
+just the first argument.  This is done with the @code{join} macro
+documented previously (@pxref{Shift}).  The improved LIFO example is
+shipped as @file{m4-@value{VERSION}/@/examples/@/wraplifo2.m4}, and can
+easily be converted to a FIFO solution by swapping the adjacent
+invocations of @code{joinall} and @code{defn}.
+
+@comment examples
+@example
+$ @kbd{m4 -I examples}
+include(`wraplifo2.m4')
+@result{}
+undivert(`wraplifo2.m4')dnl
+@result{}dnl Redefine m4wrap to have LIFO semantics, improved example.
+@result{}include(`join.m4')dnl
+@result{}define(`_m4wrap', defn(`m4wrap'))dnl
+@result{}define(`_arg1', `$1')dnl
+@result{}define(`m4wrap',
+@result{}`ifdef(`_$0_text',
+@result{}       `define(`_$0_text', joinall(` ', $@@)defn(`_$0_text'))',
+@result{}       `_$0(`_arg1(defn(`_$0_text')undefine(`_$0_text'))')dnl
+@result{}define(`_$0_text', joinall(` ', $@@))')')dnl
+m4wrap(`define(`foo', ``$0:'-$1-$*-$#-')foo(`a', `b')
+')
+@result{}
+m4wrap(`lifo text
+m4wrap(`nested', `', `$@@
+')')
+@result{}
+^D
+@result{}lifo text
+@result{}foo:-a-a,b-2-
+@result{}nested  $@@
+@end example
+
 @node Improved cleardivert
 @section Solution for @code{cleardivert}
 
diff --git a/examples/join.m4 b/examples/join.m4
new file mode 100644
index 00000000..8687ac70
--- /dev/null
+++ b/examples/join.m4
@@ -0,0 +1,15 @@
+divert(`-1')
+# join(sep, args) - join each non-empty ARG into a single
+# string, with each element separated by SEP
+define(`join',
+`ifelse(`$#', `2', ``$2'',
+  `ifelse(`$2', `', `', ``$2'_')$0(`$1', shift(shift($@)))')')
+define(`_join',
+`ifelse(`$#$2', `2', `',
+  `ifelse(`$2', `', `', ``$1$2'')$0(`$1', shift(shift($@)))')')
+# joinall(sep, args) - join each ARG, including empty ones,
+# into a single string, with each element separated by SEP
+define(`joinall', ``$2'_$0(`$1', shift($@))')
+define(`_joinall',
+`ifelse(`$#', `2', `', ``$1$3'$0(`$1', shift(shift($@)))')')
+divert`'dnl
diff --git a/examples/wraplifo2.m4 b/examples/wraplifo2.m4
new file mode 100644
index 00000000..5b450a76
--- /dev/null
+++ b/examples/wraplifo2.m4
@@ -0,0 +1,9 @@
+dnl Redefine m4wrap to have LIFO semantics, improved example.
+include(`join.m4')dnl
+define(`_m4wrap', defn(`m4wrap'))dnl
+define(`_arg1', `$1')dnl
+define(`m4wrap',
+`ifdef(`_$0_text',
+       `define(`_$0_text', joinall(` ', $@)defn(`_$0_text'))',
+       `_$0(`_arg1(defn(`_$0_text')undefine(`_$0_text'))')dnl
+define(`_$0_text', joinall(` ', $@))')')dnl