documentation update from tchrist

	Message-Id: <199806140419.WAA20549@chthon.perl.com>
	Subject: doc patches

p4raw-id: //depot/perl@1132

1 files changed, 290 insertions, 71 deletions

diff --git a/pod/perlfaq4.pod b/pod/perlfaq4.pod
index 4c38d906ba..8c57db3df0 100644
--- a/pod/perlfaq4.pod
+++ b/pod/perlfaq4.pod
@@ -12,6 +12,10 @@ data issues.
 
 =head2 Why am I getting long decimals (eg, 19.9499999999999) instead of the numbers I should be getting (eg, 19.95)?
 
+The infinite set that a mathematician thinks of as the real numbers can
+only be approximate on a computer, since the computer only has a finite
+number of bits to store an infinite number of, um, numbers.
+
 Internally, your computer represents floating-point numbers in binary.
 Floating-point numbers read in from a file, or appearing as literals
 in your program, are converted from their decimal floating-point
@@ -37,6 +41,7 @@ are consequently slower.
 
 To get rid of the superfluous digits, just use a format (eg,
 C<printf("%.2f", 19.95)>) to get the required precision.
+See L<perlop/"Floating-point Arithmetic">.
 
 =head2 Why isn't my octal data interpreted correctly?
 
@@ -54,11 +59,10 @@ umask(), or sysopen(), which all want permissions in octal.
     chmod(644,  $file);	# WRONG -- perl -w catches this
     chmod(0644, $file);	# right
 
-=head2 Does perl have a round function?  What about ceil() and floor()?
-Trig functions?
+=head2 Does perl have a round function?  What about ceil() and floor()?  Trig functions?
 
-For rounding to a certain number of digits, sprintf() or printf() is
-usually the easiest route.
+Remember that int() merely truncates toward 0.  For rounding to a certain
+number of digits, sprintf() or printf() is usually the easiest route.
 
 The POSIX module (part of the standard perl distribution) implements
 ceil(), floor(), and a number of other mathematical and trigonometric
@@ -131,11 +135,13 @@ Get the http://www.perl.com/CPAN/modules/by-module/Roman module.
 
 =head2 Why aren't my random numbers random?
 
+John von Neumann said, ``Anyone who attempts to generate random numbers by
+deterministic means is, of course, living in a state of sin.''  
+
 The short explanation is that you're getting pseudorandom numbers, not
-random ones, because that's how these things work.  A longer
-explanation is available on
-http://www.perl.com/CPAN/doc/FMTEYEWTK/random, courtesy of Tom
-Phoenix.
+random ones, because that's how these things work.  A longer explanation
+is available on http://www.perl.com/CPAN/doc/FMTEYEWTK/random, courtesy
+of Tom Phoenix.
 
 You should also check out the Math::TrulyRandom module from CPAN.
 
@@ -177,27 +183,33 @@ Instead, there is an example of Julian date calculation in
 http://www.perl.com/CPAN/authors/David_Muir_Sharnoff/modules/Time/JulianDay.pm.gz,
 which should help.
 
-=head2 Does Perl have a year 2000 problem?
+=head2 Does Perl have a year 2000 problem?  Is Perl Y2K compliant?
 
-Not unless you use Perl to create one. The date and time functions
-supplied with perl (gmtime and localtime) supply adequate information
-to determine the year well beyond 2000 (2038 is when trouble strikes).
-The year returned by these functions when used in an array context is
-the year minus 1900. For years between 1910 and 1999 this I<happens>
-to be a 2-digit decimal number. To avoid the year 2000 problem simply
-do not treat the year as a 2-digit number.  It isn't.
+Perl is just as Y2K compliant as your pencil--no more, and no less.
+The date and time functions supplied with perl (gmtime and localtime)
+supply adequate information to determine the year well beyond 2000 (2038
+is when trouble strikes).  The year returned by these functions when used
+in an array context is the year minus 1900.  For years between 1910 and
+1999 this I<happens> to be a 2-digit decimal number. To avoid the year
+2000 problem simply do not treat the year as a 2-digit number.  It isn't.
 
-When gmtime() and localtime() are used in a scalar context they return
+When gmtime() and localtime() are used in scalar context they return
 a timestamp string that contains a fully-expanded year.  For example,
 C<$timestamp = gmtime(1005613200)> sets $timestamp to "Tue Nov 13 01:00:00
 2001".  There's no year 2000 problem here.
 
+That doesn't mean that Perl can't be used to create non-Y2K compliant
+programs.  It can.  But so can your pencil.  It's the fault of the user,
+not the language.  At the risk of inflaming the NRA: ``Perl doesn't
+break Y2K, people do.''  See http://language.perl.com/news/y2k.html for
+a longer exposition.
+
 =head1 Data: Strings
 
 =head2 How do I validate input?
 
 The answer to this question is usually a regular expression, perhaps
-with auxiliary logic.  See the more specific questions (numbers, email
+with auxiliary logic.  See the more specific questions (numbers, mail
 addresses, etc.) for details.
 
 =head2 How do I unescape a string?
@@ -220,7 +232,7 @@ To turn "abbcccd" into "abccd":
 
 This is documented in L<perlref>.  In general, this is fraught with
 quoting and readability problems, but it is possible.  To interpolate
-a subroutine call (in a list context) into a string:
+a subroutine call (in list context) into a string:
 
     print "My sub returned @{[mysub(1,2,3)]} that time.\n";
 
@@ -241,16 +253,23 @@ multiple ones, then something more like C</alpha(.*?)omega/> would
 be needed.  But none of these deals with nested patterns, nor can they.
 For that you'll have to write a parser.
 
+One destructive, inside-out approach that you might try is to pull
+out the smallest nesting parts one at a time:
+
+    while (s/BEGIN(.*?)END//gs) {
+	# do something with $1
+    } 
+
 =head2 How do I reverse a string?
 
-Use reverse() in a scalar context, as documented in
+Use reverse() in scalar context, as documented in
 L<perlfunc/reverse>.
 
     $reversed = reverse $string;
 
 =head2 How do I expand tabs in a string?
 
-You can do it the old-fashioned way:
+You can do it yourself:
 
     1 while $string =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e;
 
@@ -299,6 +318,23 @@ into "whosoever" or "whomsoever", case insensitively.
 	    : $1		# renege and leave it there
     }igex;
 
+In the more general case, you can use the C</g> modifier in a C<while>
+loop, keeping count of matches.
+
+    $WANT = 3;
+    $count = 0;
+    while (/(\w+)\s+fish\b/gi) {
+        if (++$count == $WANT) {
+            print "The third fish is a $1 one.\n";
+            # Warning: don't `last' out of this loop
+        }
+    }
+
+That prints out: "The third fish is a red one."  You can also use a
+repetition count and repeated pattern like this:
+
+    /(?:\w+\s+fish\s+){2}(\w+)\s+fish/i;
+
 =head2 How can I count the number of occurrences of a substring within a string?
 
 There are a number of ways, with varying efficiency: If you want a
@@ -345,6 +381,10 @@ To force each word to be lower case, with the first letter upper case:
 
         $line =~ s/(\w+)/\u\L$1/g;
 
+You can (and probably should) enable locale awareness of those
+characters by placing a C<use locale> pragma in your program.
+See L<perllocale> for endless details.
+
 =head2 How can I split a [character] delimited string except when inside
 [character]? (Comma-separated files)
 
@@ -382,11 +422,12 @@ distribution) lets you say:
 
 =head2 How do I strip blank space from the beginning/end of a string?
 
-The simplest approach, albeit not the fastest, is probably like this:
+Although the simplest approach would seem to be:
 
     $string =~ s/^\s*(.*?)\s*$/$1/;
 
-It would be faster to do this in two steps:
+This is unneccesarily slow, destructive, and fails with embedded newlines.
+It is much better faster to do this in two steps:
 
     $string =~ s/^\s+//;
     $string =~ s/\s+$//;
@@ -398,9 +439,39 @@ Or more nicely written as:
 	s/\s+$//;
     }
 
+This idiom takes advantage of the for(each) loop's aliasing
+behavior to factor out common code.  You can do this
+on several strings at once, or arrays, or even the 
+values of a hash if you use a slide:
+
+    # trim whitespace in the scalar, the array, 
+    # and all the values in the hash
+    foreach ($scalar, @array, @hash{keys %hash}) {
+        s/^\s+//;
+        s/\s+$//;
+    }
+
 =head2 How do I extract selected columns from a string?
 
 Use substr() or unpack(), both documented in L<perlfunc>.
+If you prefer thinking in terms of columns instead of widths, 
+you can use this kind of thing:
+
+    # determine the unpack format needed to split Linux ps output
+    # arguments are cut columns
+    my $fmt = cut2fmt(8, 14, 20, 26, 30, 34, 41, 47, 59, 63, 67, 72);
+
+    sub cut2fmt { 
+	my(@positions) = @_;
+	my $template  = '';
+	my $lastpos   = 1;
+	for my $place (@positions) {
+	    $template .= "A" . ($place - $lastpos) . " "; 
+	    $lastpos   = $place;
+	}
+	$template .= "A*";
+	return $template;
+    }
 
 =head2 How do I find the soundex value of a string?
 
@@ -411,15 +482,26 @@ Use the standard Text::Soundex module distributed with perl.
 Let's assume that you have a string like:
 
     $text = 'this has a $foo in it and a $bar';
+
+If those were both global variables, then this would
+suffice:
+
     $text =~ s/\$(\w+)/${$1}/g;
 
-Before version 5 of perl, this had to be done with a double-eval
-substitution:
+But since they are probably lexicals, or at least, they could
+be, you'd have to do this:
 
     $text =~ s/(\$\w+)/$1/eeg;
+    die if $@;			# needed on /ee, not /e
 
-Which is bizarre enough that you'll probably actually need an EEG
-afterwards. :-)
+It's probably better in the general case to treat those
+variables as entries in some special hash.  For example:
+
+    %user_defs = ( 
+	foo  => 23,
+	bar  => 19,
+    );
+    $text =~ s/\$(\w+)/$user_defs{$1}/g;
 
 See also "How do I expand function calls in a string?" in this section
 of the FAQ.
@@ -458,6 +540,12 @@ that actually do care about the difference between a string and a
 number, such as the magical C<++> autoincrement operator or the
 syscall() function.
 
+Stringification also destroys arrays.  
+
+    @lines = `command`;
+    print "@lines";		# WRONG - extra blanks
+    print @lines;		# right
+
 =head2 Why don't my <<HERE documents work?
 
 Check for these three things:
@@ -472,6 +560,72 @@ Check for these three things:
 
 =back
 
+If you want to indent the text in the here document, you 
+can do this:
+
+    # all in one
+    ($VAR = <<HERE_TARGET) =~ s/^\s+//gm;
+        your text
+        goes here
+    HERE_TARGET
+
+But the HERE_TARGET must still be flush against the margin.
+If you want that indented also, you'll have to quote 
+in the indentation.
+
+    ($quote = <<'    FINIS') =~ s/^\s+//gm;
+            ...we will have peace, when you and all your works have
+            perished--and the works of your dark master to whom you
+            would deliver us. You are a liar, Saruman, and a corrupter
+            of men's hearts.  --Theoden in /usr/src/perl/taint.c
+        FINIS
+    $quote =~ s/\s*--/\n--/;
+
+A nice general-purpose fixer-upper function for indented here documents
+follows.  It expects to be called with a here document as its argument.
+It looks to see whether each line begins with a common substring, and
+if so, strips that off.  Otherwise, it takes the amount of leading
+white space found on the first line and removes that much off each
+subsequent line.
+
+    sub fix {
+        local $_ = shift;
+        my ($white, $leader);  # common white space and common leading string
+        if (/^\s*(?:([^\w\s]+)(\s*).*\n)(?:\s*\1\2?.*\n)+$/) {
+            ($white, $leader) = ($2, quotemeta($1));
+        } else {
+            ($white, $leader) = (/^(\s+)/, '');
+        }
+        s/^\s*?$leader(?:$white)?//gm;
+        return $_;
+    }
+
+This owrks with leading special strings, dynamically determined:
+
+    $remember_the_main = fix<<'    MAIN_INTERPRETER_LOOP';
+	@@@ int
+	@@@ runops() {
+	@@@     SAVEI32(runlevel);
+	@@@     runlevel++;
+	@@@     while ( op = (*op->op_ppaddr)() ) ;
+	@@@     TAINT_NOT;
+	@@@     return 0;
+	@@@ }
+    MAIN_INTERPRETER_LOOP
+
+Or with a fixed amount of leading white space, with remaining
+indentation correctly preserved:
+
+    $poem = fix<<EVER_ON_AND_ON;
+       Now far ahead the Road has gone,
+	  And I must follow, if I can,
+       Pursuing it with eager feet,
+	  Until it joins some larger way
+       Where many paths and errands meet.
+	  And whither then? I cannot say.
+		--Bilbo in /usr/src/perl/pp_ctl.c
+    EVER_ON_AND_ON
+
 =head1 Data: Arrays
 
 =head2 What is the difference between $array[1] and @array[1]?
@@ -500,6 +654,7 @@ ordered and whether you wish to preserve the ordering.
 =over 4
 
 =item a) If @in is sorted, and you want @out to be sorted:
+(this assumes all true values in the array)
 
     $prev = 'nonesuch';
     @out = grep($_ ne $prev && ($prev = $_), @in);
@@ -531,11 +686,15 @@ duplicates.
 
 =back
 
-=head2 How can I tell whether an array contains a certain element?
+=head2 How can I tell whether a list or array contains a certain element?
+
+Hearing the word "in" is an I<in>dication that you probably should have
+used a hash, not a list or array, to store your data.  Hashes are
+designed to answer this question quickly and efficiently.  Arrays aren't.
 
-There are several ways to approach this.  If you are going to make
-this query many times and the values are arbitrary strings, the
-fastest way is probably to invert the original array and keep an
+That being said, there are several ways to approach this.  If you
+are going to make this query many times over arbitrary string values,
+the fastest way is probably to invert the original array and keep an
 associative array lying about whose keys are the first array's values.
 
     @blues = qw/azure cerulean teal turquoise lapis-lazuli/;
@@ -605,8 +764,11 @@ Now C<$found_index> has what you want.
 
 In general, you usually don't need a linked list in Perl, since with
 regular arrays, you can push and pop or shift and unshift at either end,
-or you can use splice to add and/or remove arbitrary number of elements
-at arbitrary points.
+or you can use splice to add and/or remove arbitrary number of elements at
+arbitrary points.  Both pop and shift are both O(1) operations on perl's
+dynamic arrays.  In the absence of shifts and pops, push in general
+needs to reallocate on the order every log(N) times, and unshift will
+need to copy pointers each time.
 
 If you really, really wanted, you could use structures as described in
 L<perldsc> or L<perltoot> and do just what the algorithm book tells you
@@ -622,7 +784,23 @@ lists, or you could just do something like this with an array:
 
 =head2 How do I shuffle an array randomly?
 
-Here's a shuffling algorithm which works its way through the list,
+Use this:
+
+    # fisher_yates_shuffle( \@array ) : 
+    # generate a random permutation of @array in place
+    sub fisher_yates_shuffle {
+        my $array = shift;
+        my $i;
+        for ($i = @$array; --$i; ) {
+            my $j = int rand ($i+1);
+            next if $i == $j;
+            @$array[$i,$j] = @$array[$j,$i];
+        }
+    }
+
+    fisher_yates_shuffle( \@array );    # permutes @array in place
+
+You've probably seen shuffling algorithms that works using splice,
 randomly picking another element to swap the current element with:
 
     srand;
@@ -632,65 +810,70 @@ randomly picking another element to swap the current element with:
 	push(@new, splice(@old, rand @old, 1));
     }
 
-For large arrays, this avoids a lot of the reshuffling:
-
-    srand;
-    @new = ();
-    @old = 1 .. 10000;  # just a demo
-    for( @old ){
-        my $r = rand @new+1;
-        push(@new,$new[$r]);
-        $new[$r] = $_;
-    }
+This is bad because splice is already O(N), and since you do it N times,
+you just invented a quadratic algorithm; that is, O(N**2).  This does
+not scale, although Perl is so efficient that you probably won't notice
+this until you have rather largish arrays.
 
 =head2 How do I process/modify each element of an array?
 
 Use C<for>/C<foreach>:
 
     for (@lines) {
-	s/foo/bar/;
-	tr[a-z][A-Z];
+	s/foo/bar/;	# change that word
+	y/XZ/ZX/;	# swap those letters
     }
 
 Here's another; let's compute spherical volumes:
 
-    for (@radii) {
+    for (@volumes = @radii) {   # @volumes has changed parts
 	$_ **= 3;
 	$_ *= (4/3) * 3.14159;  # this will be constant folded
     }
 
+If you want to do the same thing to modify the values of the hash,
+you may not use the C<values> function, oddly enough.  You need a slice:
+
+    for $orbit ( @orbits{keys %orbits} ) {
+	($orbit **= 3) *= (4/3) * 3.14159; 
+    }
+
 =head2 How do I select a random element from an array?
 
 Use the rand() function (see L<perlfunc/rand>):
 
+    # at the top of the program:
     srand;			# not needed for 5.004 and later
+
+    # then later on
     $index   = rand @array;
     $element = $array[$index];
 
+Make sure you I<only call srand once per program, if then>.
+If you are calling it more than once (such as before each 
+call to rand), you're almost certainly doing something wrong.
+
 =head2 How do I permute N elements of a list?
 
 Here's a little program that generates all permutations
 of all the words on each line of input.  The algorithm embodied
-in the permut() function should work on any list:
+in the permute() function should work on any list:
 
     #!/usr/bin/perl -n
-    # permute - tchrist@perl.com
-    permut([split], []);
-    sub permut {
-	my @head = @{ $_[0] };
-	my @tail = @{ $_[1] };
-	unless (@head) {
-	    # stop recursing when there are no elements in the head
-	    print "@tail\n";
+    # tsc-permute: permute each word of input
+    permute([split], []);
+    sub permute {
+        my @items = @{ $_[0] };
+        my @perms = @{ $_[1] };
+        unless (@items) {
+            print "@perms\n";
 	} else {
-	    # for all elements in @head, move one from @head to @tail
-	    # and call permut() on the new @head and @tail
-	    my(@newhead,@newtail,$i);
-	    foreach $i (0 .. $#head) {
-		@newhead = @head;
-		@newtail = @tail;
-		unshift(@newtail, splice(@newhead, $i, 1));
-		permut([@newhead], [@newtail]);
+            my(@newitems,@newperms,$i);
+            foreach $i (0 .. $#items) {
+                @newitems = @items;
+                @newperms = @perms;
+                unshift(@newperms, splice(@newitems, $i, 1));
+                permute([@newitems], [@newperms]);
 	    }
 	}
     }
@@ -796,7 +979,7 @@ See L<perlfunc/defined> in the 5.004 release or later of Perl.
 Use the each() function (see L<perlfunc/each>) if you don't care
 whether it's sorted:
 
-    while (($key,$value) = each %hash) {
+    while ( ($key, $value) = each %hash) {
 	print "$key = $value\n";
     }
 
@@ -862,6 +1045,7 @@ L<perllocale>).
 
 You can look into using the DB_File module and tie() using the
 $DB_BTREE hash bindings as documented in L<DB_File/"In Memory Databases">.
+The Tie::IxHash module from CPAN might also be instructive.
 
 =head2 What's the difference between "delete" and "undef" with hashes?
 
@@ -953,7 +1137,7 @@ they end up doing is not what they do with ordinary hashes.
 
 =head2 How do I reset an each() operation part-way through?
 
-Using C<keys %hash> in a scalar context returns the number of keys in
+Using C<keys %hash> in scalar context returns the number of keys in
 the hash I<and> resets the iterator associated with the hash.  You may
 need to do this if you use C<last> to exit a loop early so that when you
 re-enter it, the hash iterator has been reset.
@@ -1055,14 +1239,37 @@ Assuming that you don't care about IEEE notations like "NaN" or
 "Infinity", you probably just want to use a regular expression.
 
    warn "has nondigits"        if     /\D/;
-   warn "not a whole number"   unless /^\d+$/;
-   warn "not an integer"       unless /^-?\d+$/;  # reject +3
+    warn "not a natural number" unless /^\d+$/;             # rejects -3
+    warn "not an integer"       unless /^-?\d+$/;           # rejects +3
    warn "not an integer"       unless /^[+-]?\d+$/;
    warn "not a decimal number" unless /^-?\d+\.?\d*$/;  # rejects .2
    warn "not a decimal number" unless /^-?(?:\d+(?:\.\d*)?|\.\d+)$/;
    warn "not a C float"
        unless /^([+-]?)(?=\d|\.\d)\d*(\.\d*)?([Ee]([+-]?\d+))?$/;
 
+If you're on a POSIX system, Perl's supports the C<POSIX::strtod>
+function.  Its semantics are somewhat cumbersome, so here's a C<getnum>
+wrapper function for more convenient access.  This function takes
+a string and returns the number it found, or C<undef> for input that
+isn't a C float.  The C<is_numeric> function is a front end to C<getnum>
+if you just want to say, ``Is this a float?''
+
+    sub getnum {
+        use POSIX qw(strtod);
+        my $str = shift;
+        $str =~ s/^\s+//;
+        $str =~ s/\s+$//;
+        $! = 0;
+        my($num, $unparsed) = strtod($str);
+        if (($str eq '') || ($unparsed != 0) || $!) {
+            return undef;
+        } else {
+            return $num;
+        } 
+    } 
+
+    sub is_numeric { defined &getnum } 
+
 Or you could check out
 http://www.perl.com/CPAN/modules/by-module/String/String-Scanf-1.1.tar.gz
 instead.  The POSIX module (part of the standard Perl distribution)
@@ -1096,6 +1303,18 @@ Get the Business::CreditCard module from CPAN.
 
 =head1 AUTHOR AND COPYRIGHT
 
-Copyright (c) 1997 Tom Christiansen and Nathan Torkington.
-All rights reserved.  See L<perlfaq> for distribution information.
-
+Copyright (c) 1997, 1998 Tom Christiansen and Nathan Torkington.
+All rights reserved.
+
+When included as part of the Standard Version of Perl, or as part of
+its complete documentation whether printed or otherwise, this work
+may be distributed only under the terms of Perl's Artistic License.
+Any distribution of this file or derivatives thereof I<outside>
+of that package require that special arrangements be made with
+copyright holder.
+
+Irrespective of its distribution, all code examples in this file
+are hereby placed into the public domain.  You are permitted and
+encouraged to use this code in your own programs for fun
+or for profit as you see fit.  A simple comment in the code giving
+credit would be courteous but is not required.