troff's `box' and `boxa' requests didn't preserve temporary

indentation.

* src/roff/troff/div.h (diversion): Add `saved_saved_indent' and
`saved_target_text_length' members.
* src/roff/troff/div.cc (do_divert): Use them.

Tabs (but not yet leaders and fields) are now handled correctly by
`asciify' request.

* node.h (hmotion_node): Add field `was_tab'.
* env.cc (environment::make_tab_node): Set it.
* node.cc (hmotion_node::copy, hmotion_node::asciify): Use it.

Added new request `unformat' which will, contrary to `asciify',
preserve font information after unformatting (i.e., only nodes
dealing with horizontal space are converted back to input
characters).

* input.cc (asciify_macro): Renamed to ...
(do_asciify_macro): this, having a new parameter `unformat_only'.
(asciify_macro, unformat_macro): New; using do_asciify_macro.
(init_input_requests): Added `unformat'.
* node.cc (*_node::asciify), asciify_reverse_node_list: Added
parameter to control asciification process.
* node.h (*_node::asciify): Ditto.
* NEWS, src/roff/troff/troff.man, man/groff.man: Document it.


* doc/groff.texinfo: Minor improvement of `.in' documentation.


* test-groff: Updated.

13 files changed, 165 insertions, 99 deletions

diff --git a/ChangeLog b/ChangeLog
index aa3bd35d8..dedeb2647 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,37 @@
+2001-01-23  Werner LEMBERG  <wl@gnu.org>
+
+	troff's `box' and `boxa' requests didn't preserve temporary
+	indentation.
+
+	* src/roff/troff/div.h (diversion): Add `saved_saved_indent' and
+	`saved_target_text_length' members.
+	* src/roff/troff/div.cc (do_divert): Use them.
+
+	Tabs (but not yet leaders and fields) are now handled correctly by
+	`asciify' request.
+
+	* node.h (hmotion_node): Add field `was_tab'.
+	* env.cc (environment::make_tab_node): Set it.
+	* node.cc (hmotion_node::copy, hmotion_node::asciify): Use it.
+
+	Added new request `unformat' which will, contrary to `asciify',
+	preserve font information after unformatting (i.e., only nodes
+	dealing with horizontal space are converted back to input
+	characters).
+
+	* input.cc (asciify_macro): Renamed to ...
+	(do_asciify_macro): this, having a new parameter `unformat_only'.
+	(asciify_macro, unformat_macro): New; using do_asciify_macro.
+	(init_input_requests): Added `unformat'.
+	* node.cc (*_node::asciify), asciify_reverse_node_list: Added
+	parameter to control asciification process.
+	* node.h (*_node::asciify): Ditto.
+	* NEWS, src/roff/troff/troff.man, man/groff.man: Document it.
+
+2001-01-20  Werner LEMBERG  <wl@gnu.org>
+
+	* doc/groff.texinfo: Minor improvement of `.in' documentation.
+
 2001-01-18  Werner LEMBERG  <wl@gnu.org>
 
 	* tmac/ps.tmac: Remove call of psold.tmac.
@@ -8,6 +42,8 @@
 	* font/devutf8/R.proto: Adding `shc' glyph.
 	* font/devutf8/NOTES: Updated.
 
+	* test-groff: Updated.
+
 2001-01-15  Gaius Mulley  <gaius@glam.ac.uk>
 
 	First cut of the new html device driver.  Changes to pre-html and
diff --git a/NEWS b/NEWS
index 4c33460c0..2ae42654a 100644
--- a/NEWS
+++ b/NEWS
@@ -43,8 +43,15 @@ o `box' and `boxa' are two new requests which behave similarly to `di' and
   ending the diversion).
 
 o The `asciify' request has been extended to `unformat' space characters
-  also.  This makes it possible to reformat diversions; for example the
-  following
+  also.
+
+  `\ ' will no longer be unformatted as a space but remains an unpaddable,
+  unbreakable space character.
+
+o The new `unformat' request is similar to `asciify' but only converts
+  (formatted) horizontal space back to its input form, retaining font
+  information in.  This makes it possible to reformat diversions; for
+  example the following
 
     .ll 3i
     .
@@ -53,14 +60,14 @@ o The `asciify' request has been extended to `unformat' space characters
     .box box1
     .ev 1
     .nf
-    b01 b02 b03 b04 b05 b06 b07 b08 b09 b10.
+    \f[B]b01 b02 b03 b04 b05 b06 b07 b08 b09 b10.\f[P]
     .br
     .ev
     .box
     .
     c01 c02 c03 c04 c05 c06 c07 c08 c09 c10.
     .
-    .asciify box1
+    .unformat box1
     .box1
 
   gives
@@ -71,7 +78,7 @@ o The `asciify' request has been extended to `unformat' space characters
     b02 b03 b04 b05  b06  b07  b08
     b09 b10.
 
-  Without the `asciify' request, space characters are converted to word
+  Without the `unformat' request, space characters are converted to word
   space nodes which are no longer stretchable, and the result would be
 
     a01  a02  a03  a04 a05 a06 a07
@@ -80,9 +87,6 @@ o The `asciify' request has been extended to `unformat' space characters
     b02 b03 b04 b05 b06 b07 b08
     b09 b10.
 
-  Note that `\ ' will no longer be unformatted as a space but remains an
-  unpaddable, unbreakable space character.
-
 o Two new requests `tm1' and `tmc' have been added to improve writing
   messages to the terminal.  `tm1' is similar to `tm' but allows leading
   whitespace.  `tmc' is similar to `tm1' but doesn't emit a final newline.
diff --git a/doc/groff.texinfo b/doc/groff.texinfo
index 9c55cb20f..ced6b183f 100644
--- a/doc/groff.texinfo
+++ b/doc/groff.texinfo
@@ -4996,7 +4996,7 @@ If a negative indentation value is specified (which is not allowed),
 indentation to zero.
 
 The effect of @code{in} is delayed until a partially collected line (if
-it exists) is output.
+it exists) is output.  A temporary indent value is reset to zero also.
 
 @vindex .i
 The current indentation (as set by @code{in}) can be found in the
diff --git a/man/groff.man b/man/groff.man
index b9e3e2188..a8c443194 100644
--- a/man/groff.man
+++ b/man/groff.man
@@ -1704,6 +1704,9 @@ Underline (italicize in troff)
 .argument N
 input lines.
 .
+.REQ .unformat diversion
+Unformat horizontal space (no glyphs) in
+.argument diversion .
 .REQ .vpt n
 Enable vertical position traps if
 .argument n
diff --git a/src/roff/troff/div.cc b/src/roff/troff/div.cc
index 46661941e..4b0b861b3 100644
--- a/src/roff/troff/div.cc
+++ b/src/roff/troff/div.cc
@@ -99,6 +99,8 @@ void do_divert(int append, int boxing)
 	curenv->line = curdiv->saved_line;
 	curenv->width_total = curdiv->saved_width_total;
 	curenv->space_total = curdiv->saved_space_total;
+	curenv->saved_indent = curdiv->saved_saved_indent;
+	curenv->target_text_length = curdiv->saved_target_text_length;
       }
       diversion *temp = curdiv;
       curdiv = curdiv->prev;
@@ -115,6 +117,8 @@ void do_divert(int append, int boxing)
       curdiv->saved_line = curenv->line;
       curdiv->saved_width_total = curenv->width_total;
       curdiv->saved_space_total = curenv->space_total;
+      curdiv->saved_saved_indent = curenv->saved_indent;
+      curdiv->saved_target_text_length = curenv->target_text_length;
       curenv->line = 0;
       curenv->start_line();
     }
diff --git a/src/roff/troff/div.h b/src/roff/troff/div.h
index e73fa44d4..096b74799 100644
--- a/src/roff/troff/div.h
+++ b/src/roff/troff/div.h
@@ -25,6 +25,8 @@ class diversion {
   node *saved_line;
   hunits saved_width_total;
   int saved_space_total;
+  hunits saved_saved_indent;
+  hunits saved_target_text_length;
 protected:
   symbol nm;
   vunits vertical_position;
diff --git a/src/roff/troff/env.cc b/src/roff/troff/env.cc
index eee3f1083..fb87c68a8 100644
--- a/src/roff/troff/env.cc
+++ b/src/roff/troff/env.cc
@@ -2598,7 +2598,7 @@ node *environment::make_tab_node(hunits d, node *next)
     leader_node = 0;
   }
   if (!leader_node)
-    return new hmotion_node(d, next);
+    return new hmotion_node(d, 1, next);
   node *n = new hline_node(d, leader_node, next);
   leader_node = 0;
   return n;
diff --git a/src/roff/troff/input.cc b/src/roff/troff/input.cc
index aeca49c12..d89db35a3 100644
--- a/src/roff/troff/input.cc
+++ b/src/roff/troff/input.cc
@@ -3671,7 +3671,7 @@ void length_macro()
     set_number_reg(ret, len);
 }
 
-void asciify_macro()
+void do_asciify_macro(int unformat_only)
 {
   symbol s = get_name(1);
   if (!s.is_null()) {
@@ -3690,7 +3690,7 @@ void asciify_macro()
 	if (c != 0)
 	  am.append(c);
 	else
-	  nd->asciify(&am);
+	  nd->asciify(&am, unformat_only);
       }
       *m = am;
     }
@@ -3698,6 +3698,16 @@ void asciify_macro()
   skip_line();
 }
 
+void asciify_macro()
+{
+  do_asciify_macro(0);
+}
+
+void unformat_macro()
+{
+  do_asciify_macro(1);
+}
+
 static void interpolate_environment_variable(symbol nm)
 {
   const char *s = getenv(nm.contents());
@@ -6251,6 +6261,7 @@ void init_input_requests()
   init_request("substring", substring_macro);
   init_request("length", length_macro);
   init_request("asciify", asciify_macro);
+  init_request("unformat", unformat_macro);
   init_request("warn", warn_request);
   init_request("open", open_request);
   init_request("opena", opena_request);
diff --git a/src/roff/troff/node.cc b/src/roff/troff/node.cc
index 1c46c2024..c8a59fa03 100644
--- a/src/roff/troff/node.cc
+++ b/src/roff/troff/node.cc
@@ -1551,7 +1551,7 @@ public:
   hyphen_list *get_hyphen_list(hyphen_list *ss = 0);
   node *add_self(node *, hyphen_list **);
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   int character_type();
   int same(node *);
   const char *type();
@@ -1575,7 +1575,7 @@ public:
   node *add_self(node *, hyphen_list **);
   hyphen_list *get_hyphen_list(hyphen_list *ss = 0);
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   int same(node *);
   const char *type();
   int force_tprint();
@@ -1601,7 +1601,7 @@ public:
   hyphenation_type get_hyphenation_type();
   int ends_sentence();
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   int same(node *);
   const char *type();
   int force_tprint();
@@ -1630,7 +1630,7 @@ public:
   void split(int, node **, node **);
   hyphenation_type get_hyphenation_type();
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   int same(node *);
   const char *type();
   int force_tprint();
@@ -2198,7 +2198,7 @@ public:
   ~italic_corrected_node();
   node *copy();
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   hunits width();
   node *last_char_node();
   void vertical_extent(vunits *, vunits *);
@@ -2345,7 +2345,7 @@ public:
   void tprint(troff_output_file *);
   void zero_width_tprint(troff_output_file *);
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   hyphenation_type get_hyphenation_type();
   int overlaps_vertically();
   int overlaps_horizontally();
@@ -2462,7 +2462,7 @@ node *vertical_size_node::copy()
 
 node *hmotion_node::copy()
 { 
-  return new hmotion_node(n);
+  return new hmotion_node(n, was_tab);
 }
 
 node *space_char_hmotion_node::copy()
@@ -2980,106 +2980,120 @@ void space_char_hmotion_node::ascii_print(ascii_output_file *ascii)
 
 /* asciify methods */
 
-void node::asciify(macro *m)
+void node::asciify(macro *m, int)
 {
   m->append(this);
 }
       
-void glyph_node::asciify(macro *m)
+void glyph_node::asciify(macro *m, int unformat_only)
 {
-  unsigned char c = ci->get_ascii_code();
-  if (c != 0) {
-    m->append(c);
-    delete this;
-  }
-  else
+  if (unformat_only)
     m->append(this);
+  else {
+    unsigned char c = ci->get_ascii_code();
+    if (c != 0) {
+      m->append(c);
+      delete this;
+    }
+    else
+      m->append(this);
+  }
 }
 
-void kern_pair_node::asciify(macro *m)
+void kern_pair_node::asciify(macro *m, int unformat_only)
 {
-  n1->asciify(m);
-  n2->asciify(m);
+  n1->asciify(m, unformat_only);
+  n2->asciify(m, unformat_only);
   n1 = n2 = 0;
   delete this;
 }
 
-static void asciify_reverse_node_list(macro *m, node *n)
+static void asciify_reverse_node_list(macro *m, node *n, int unformat_only)
 {
   if (n == 0)
     return;
-  asciify_reverse_node_list(m, n->next);
-  n->asciify(m);
+  asciify_reverse_node_list(m, n->next, unformat_only);
+  n->asciify(m, unformat_only);
 }
 
-void dbreak_node::asciify(macro *m)
+void dbreak_node::asciify(macro *m, int unformat_only)
 {
-  asciify_reverse_node_list(m, none);
+  asciify_reverse_node_list(m, none, unformat_only);
   none = 0;
   delete this;
 }
 
-void ligature_node::asciify(macro *m)
+void ligature_node::asciify(macro *m, int unformat_only)
 {
-  n1->asciify(m);
-  n2->asciify(m);
+  n1->asciify(m, unformat_only);
+  n2->asciify(m, unformat_only);
   n1 = n2 = 0;
   delete this;
 }
 
-void break_char_node::asciify(macro *m)
+void break_char_node::asciify(macro *m, int unformat_only)
 {
-  ch->asciify(m);
+  ch->asciify(m, unformat_only);
   ch = 0;
   delete this;
 }
 
-void italic_corrected_node::asciify(macro *m)
+void italic_corrected_node::asciify(macro *m, int unformat_only)
 {
-  n->asciify(m);
+  n->asciify(m, unformat_only);
   n = 0;
   delete this;
 }
 
-void left_italic_corrected_node::asciify(macro *m)
+void left_italic_corrected_node::asciify(macro *m, int unformat_only)
 {
   if (n) {
-    n->asciify(m);
+    n->asciify(m, unformat_only);
     n = 0;
   }
   delete this;
 }
 
+void hmotion_node::asciify(macro *m, int)
+{
+  if (was_tab) {
+    m->append('\t');
+    delete this;
+  }
+  else
+    m->append(this);
+}
+
 space_char_hmotion_node::space_char_hmotion_node(hunits i, node *next)
 : hmotion_node(i, next)
 {
 }
 
-void space_char_hmotion_node::asciify(macro *m)
+void space_char_hmotion_node::asciify(macro *m, int)
 {
   m->append(ESCAPE_SPACE);
   delete this;
 }
 
-void word_space_node::asciify(macro *m)
+void word_space_node::asciify(macro *m, int)
 {
   for (int i = 0; i < num_spaces; i++)
     m->append(' ');
   delete this;
 }
 
-void unbreakable_space_node::asciify(macro *m)
+void unbreakable_space_node::asciify(macro *m, int)
 {
   m->append(ESCAPE_TILDE);
   delete this;
 }
 
-void line_start_node::asciify(macro *)
+void line_start_node::asciify(macro *, int)
 {
   delete this;
 }
 
-void vertical_size_node::asciify(macro *)
+void vertical_size_node::asciify(macro *, int)
 {
   delete this;
 }
@@ -3526,7 +3540,7 @@ public:
   void tprint(troff_output_file *);
   hyphenation_type get_hyphenation_type();
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   hyphen_list *get_hyphen_list(hyphen_list *tail);
   node *add_self(node *, hyphen_list **);
   tfont *get_tfont();
@@ -3590,15 +3604,19 @@ hyphenation_type composite_node::get_hyphenation_type()
   return HYPHEN_MIDDLE;
 }
 
-void composite_node::asciify(macro *m)
+void composite_node::asciify(macro *m, int unformat_only)
 {
-  unsigned char c = ci->get_ascii_code();
-  if (c != 0) {
-    m->append(c);
-    delete this;
-  }
-  else
+  if (unformat_only)
     m->append(this);
+  else {
+    unsigned char c = ci->get_ascii_code();
+    if (c != 0) {
+      m->append(c);
+      delete this;
+    }
+    else
+      m->append(this);
+  }
 }
 
 void composite_node::ascii_print(ascii_output_file *ascii)
diff --git a/src/roff/troff/node.h b/src/roff/troff/node.h
index c861262e2..80095c207 100644
--- a/src/roff/troff/node.h
+++ b/src/roff/troff/node.h
@@ -76,7 +76,7 @@ struct node {
   virtual node *add_self(node *, hyphen_list **);
   virtual hyphen_list *get_hyphen_list(hyphen_list *s = 0);
   virtual void ascii_print(ascii_output_file *);
-  virtual void asciify(macro *);
+  virtual void asciify(macro *, int);
   virtual int discardable();
   virtual void spread_space(int *, hunits *);
   virtual void freeze_space();
@@ -136,7 +136,7 @@ public:
   int same(node *);
   int force_tprint();
   const char *type();
-  void asciify(macro *);
+  void asciify(macro *, int);
 };
 
 class space_node : public node {
@@ -183,7 +183,7 @@ public:
   node *copy();
   void tprint(troff_output_file *);
   int same(node *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   const char *type();
   int merge_space(hunits);
   int force_tprint();
@@ -195,7 +195,7 @@ public:
   unbreakable_space_node(hunits, node * = 0);
   node *copy();
   int same(node *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   const char *type();
   int force_tprint();
   breakpoint *get_breakpoints(hunits width, int nspaces, breakpoint *rest = 0,
@@ -244,7 +244,7 @@ class vertical_size_node : public node {
 public:
   vertical_size_node(vunits i) : n(i) {}
   void set_vertical_size(vertical_size *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   node *copy();
   int same(node *);
   const char *type();
@@ -254,9 +254,13 @@ public:
 class hmotion_node : public node {
 protected:
   hunits n;
+  int was_tab;
 public:
-  hmotion_node(hunits i, node *next = 0) : node(next), n(i) {}
+  hmotion_node(hunits i, node *next = 0) : node(next), n(i), was_tab(0) {}
+  hmotion_node(hunits i, int flag, node *next = 0)
+    : node(next), n(i), was_tab(flag) {}
   node *copy();
+  void asciify(macro *, int);
   void tprint(troff_output_file *);
   hunits width();
   void ascii_print(ascii_output_file *);
@@ -270,7 +274,7 @@ public:
   space_char_hmotion_node(hunits i, node *next = 0);
   node *copy();
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   int same(node *);
   const char *type();
   int force_tprint();
@@ -363,7 +367,7 @@ public:
   ~left_italic_corrected_node();
   void tprint(troff_output_file *);
   void ascii_print(ascii_output_file *);
-  void asciify(macro *);
+  void asciify(macro *, int);
   node *copy();
   int same(node *);
   const char *type();
diff --git a/src/roff/troff/troff.man b/src/roff/troff/troff.man
index c0c0c9285..f183b09a3 100644
--- a/src/roff/troff/troff.man
+++ b/src/roff/troff/troff.man
@@ -743,11 +743,9 @@ and space characters that were formatted and diverted into
 will be treated like ordinary input characters when
 .I xx
 is reread.
-Useful in conjunction with the
-.B .box
-and
+Useful for diversions in conjunction with the
 .B .writem
-requests.
+request.
 It can be also used for gross hacks; for example, this
 .RS
 .IP
@@ -768,6 +766,9 @@ It can be also used for gross hacks; for example, this
 will set register
 .B n
 to 1.
+Note that glyph information (font, font size, etc.) is not preserved; use
+.B .unformat
+instead.
 .TP
 .B .backtrace
 Print a backtrace of the input stack on stderr.
@@ -1503,6 +1504,17 @@ This undoes the effect of the
 .B nroff
 request.
 .TP
+.BI .unformat\  xx
+This request `unformats' the diversion
+.IR xx ,
+converting all horizontal space back into input characters.
+Glyph information (font, font size, etc.) is retained.
+Useful in conjunction with the
+.B .box
+and
+.B .boxa
+requests.
+.TP
 .BI .vpt\  n
 Enable vertical position traps if
 .I n
diff --git a/test-groff b/test-groff
index 88ee77bc1..5d8da65de 100644
--- a/test-groff
+++ b/test-groff
@@ -19,14 +19,14 @@ $builddir/preproc/tbl:\
 $builddir/preproc/grn:\
 $builddir/preproc/refer:\
 $builddir/preproc/soelim:\
-$builddir/preproc/html2:\
+$builddir/preproc/html:\
 $builddir/devices/grops:\
 $builddir/devices/grodvi:\
 $builddir/devices/grotty:\
 $builddir/devices/grolj4:\
 $builddir/devices/grolbp:\
 $builddir/devices/grohtml:\
-$builddir/devices/grohtml2:\
+$builddir/devices/grohtml-old:\
 $srcdir/$builddir/xditview:\
 $PATH
 
diff --git a/tmac/psnew.tmac b/tmac/psnew.tmac
deleted file mode 100755
index 9e18aa029..000000000
--- a/tmac/psnew.tmac
+++ /dev/null
@@ -1,28 +0,0 @@
-.\" psnew.tmac
-.\"
-.\" Undo the effect of psold.tmac.  This gives access to the additional
-.\" characters that are present in the text fonts of newer PostScript
-.\" printers.  It is a bad idea to use this if you are going to
-.\" distribute the resulting PostScript output to others.
-.nr _C \n(.C
-.cp 0
-.rchar \('y\('Y\(12\(14\(34\(S1\(S2\(S3\(bb\(de\(Tp\(TP\(-D\(Sd
-.tr \[char166]\[char166]
-.tr \[char176]\[char176]
-.tr \[char177]\[char177]
-.tr \[char178]\[char178]
-.tr \[char179]\[char179]
-.tr \[char181]\[char181]
-.tr \[char185]\[char185]
-.tr \[char188]\[char188]
-.tr \[char189]\[char189]
-.tr \[char190]\[char190]
-.tr \[char208]\[char208]
-.tr \[char215]\[char215]
-.tr \[char221]\[char221]
-.tr \[char222]\[char222]
-.tr \[char240]\[char240]
-.tr \[char247]\[char247]
-.tr \[char253]\[char253]
-.tr \[char254]\[char254]
-.cp \n(_C