More Buffer descriptions and examples.

Updated Script examples for New Script methods.

1 files changed, 310 insertions, 137 deletions

diff --git a/doc/api.markdown b/doc/api.markdown
index 7144ac4b72..51680b70bc 100644
--- a/doc/api.markdown
+++ b/doc/api.markdown
@@ -25,123 +25,185 @@ it with the node program
 All of the examples in the documentation can be run similarly.
 
 
-## Modules
 
-Node uses the CommonJS module system.
+## Standard Modules
 
-Node has a simple module loading system.  In Node, files and modules are in
-one-to-one correspondence.  As an example, `foo.js` loads the module
-`circle.js` in the same directory.
+Node comes with a number of modules that are compiled in to the process,
+most of which are documented below.  The most common way to use these modules
+is with `require('name')` and then assigning the return value to a local
+variable with the same name as the module.
 
-The contents of `foo.js`:
+Example:
 
-    var circle = require('./circle');
     var sys = require('sys');
-    sys.puts( 'The area of a circle of radius 4 is '
-            + circle.area(4));
+    
+It is possible to extend node with other modules.  See `'Modules'`
 
-The contents of `circle.js`:
 
-    var PI = 3.14;
+## Buffers
 
-    exports.area = function (r) {
-      return PI * r * r;
-    };
+Pure Javascript is Unicode friendly but not nice to binary data.  When
+dealing with TCP streams or the file system, it's necessary to handle octet
+streams. Node has several strategies for manipulating, creating, and
+consuming octet streams.
 
-    exports.circumference = function (r) {
-      return 2 * PI * r;
-    };
+Raw data is stored in instances of the `Buffer` class. A `Buffer` is similar
+to an array of integers but corresponds to a raw memory allocation outside
+the V8 heap. A `Buffer` cannot be resized.
+Access the class with `require('buffer').Buffer`.
 
-The module `circle.js` has exported the functions `area()` and
-`circumference()`.  To export an object, add to the special `exports`
-object.  (Alternatively, one can use `this` instead of `exports`.) Variables
-local to the module will be private. In this example the variable `PI` is
-private to `circle.js`. The function `puts()` comes from the module `'sys'`,
-which is a built-in module. Modules which are not prefixed by `'./'` are
-built-in module--more about this later.
+Converting between Buffers and JavaScript string objects requires an explicit encoding
+method.  Node supports 3 string encodings: UTF-8 (`'utf8'`), ASCII (`'ascii'`), and
+Binary (`'binary'`).
 
-A module prefixed with `'./'` is relative to the file calling `require()`.
-That is, `circle.js` must be in the same directory as `foo.js` for
-`require('./circle')` to find it.
+* `'ascii'` - for 7 bit ASCII data only.  This encoding method is very fast, and will
+strip the high bit if set.
+    
+* `'binary'` - for 8 bit binary data such as images.
 
-Without the leading `'./'`, like `require('assert')` the module is searched
-for in the `require.paths` array. `require.paths` on my system looks like
-this: 
+* `'utf8'` - Unicode characters.  Many web pages and other document formats use UTF-8.
 
-`[ '/home/ryan/.node_libraries' ]`
 
-That is, when `require('assert')` is called Node looks for: 
+### new Buffer(size)
 
-  * 1: `/home/ryan/.node_libraries/assert.js`
-  * 2: `/home/ryan/.node_libraries/assert.node`
-  * 3: `/home/ryan/.node_libraries/assert/index.js`
-  * 4: `/home/ryan/.node_libraries/assert/index.node`
+Allocates a new buffer of `size` octets.
 
-interrupting once a file is found. Files ending in `'.node'` are binary Addon
-Modules; see the section below about addons. `'index.js'` allows one to
-package a module as a directory.
+### buffer.write(string, encoding, offset)
 
-`require.paths` can be modified at runtime by simply unshifting new
-paths onto it, or at startup with the `NODE_PATH` environmental
-variable (which should be a list of paths, colon separated).
+Writes `string` to the buffer at `offset` using the given encoding. Returns
+number of octets written.  If `buffer` did not contain enough space to fit
+the entire string it will write a partial amount of the string. In the case
+of `'utf8'` encoding, the method will not write partial characters.
 
+Example: write a utf8 string into a buffer, then print it
 
-## Buffers
+    var sys = require('sys'),
+      Buffer = require('buffer').Buffer,
+      buf = new Buffer(256),
+      len;
 
-Pure Javascript is Unicode friendly but not nice to pure binary data.  When
-dealing with TCP streams or the file system, it's necessary to handle octet
-streams. Node has several strategies for manipulating, creating, and
-consuming octet streams.
+    len = buf.write('\u00bd + \u00bc = \u00be', 'utf8', 0);
+    sys.puts(len + " bytes: " + buf.toString('utf8', 0, len));
 
-Raw data is stored in instances of the `Buffer` class. A `Buffer` is similar
-to an array of integers but correspond to a raw memory allocation outside
-the V8 heap. A `Buffer` cannot be resized.
-Access the class at `require('buffer').Buffer`.
+    // 12 bytes: ½ + ¼ = ¾
+    
+
+### buffer.toString(encoding, start, end)
+
+Decodes and returns a string from buffer data encoded with `encoding`
+beginning at `start` and ending at `end`.
+
+See `buffer.write()` example, above.
+
+
+### buffer[index]
+
+Get and set the octet at `index`. The values refer to individual bytes,
+so the legal range is between `0x00` and `0xFF` hex or `0` and `255`.
+
+Example: copy an ASCII string into a buffer, one byte at a time:
+
+    var sys = require('sys'),
+      Buffer = require('buffer').Buffer,
+      str = "node.js",
+      buf = new Buffer(str.length),
+      i;
+
+    for (i = 0; i < str.length ; i += 1) {
+      buf[i] = str.charCodeAt(i);
+    }
+
+    sys.puts(buf);
+
+    // node.js
 
-Node supports 3 string encodings. UTF-8 (`'utf8'`), ASCII (`'ascii'`), and
-Binary (`'binary'`). `'ascii'` and `'binary'` only look at the first 8 bits
-of the 16bit JavaScript string characters.
 
 ### Buffer.byteLength(string, encoding)
-Gives the actual byte length of a string.  This is not the same as
+
+Gives the actual byte length of a string.  This is not the same as 
 `String.prototype.length` since that returns the number of *characters* in a
 string.
 
-    // Takes in a UTF8 string, gives back a buffer
-    function stringToBuffer(string) {
-      var buffer = new Buffer(Buffer.byteLength(string));
-      buffer.utf8Write(string);
-      return buffer;
-    };
+Example:
 
-### new Buffer(size)
-Allocates a new buffer of `size` octets.
+    var sys = require('sys'),
+      Buffer = require('buffer').Buffer,
+      str = '\u00bd + \u00bc = \u00be';
+
+    sys.puts(str + ": " + str.length + " characters, " + 
+      Buffer.byteLength(str, 'utf8') + " bytes");
+
+    // ½ + ¼ = ¾: 9 characters, 12 bytes
 
-### buffer[index]
-Get and set the octet at `index`. The value can be between `0x00` and `0xFF`.
 
 ### buffer.length
-length in octets.
 
-### buffer.copy(targetBuffer, targetStart, start, end)
+The size of the buffer in bytes.  Note that this is not necessarily the size
+of the contents. `length` refers to the amount of memory allocated for the 
+buffer object.  It does not change when the contents of the buffer are changed.
+
+    var sys = require('sys'),
+      Buffer = require('buffer').Buffer,
+      buf = new Buffer(1234);
+
+    sys.puts(buf.length);
+    buf.write("some string", "ascii", 0);
+    sys.puts(buf.length);
+
+    // 1234
+    // 1234
+
+### buffer.copy(targetBuffer, targetStart, sourceStart, sourceEnd)
+
 Does a memcpy() between buffers.
 
+Example: build two Buffers, then copy `buf1` from byte 16 through byte 20
+into `buf2`, starting at the 8th byte in `buf2`.
+
+    var sys = require('sys'),
+      Buffer = require('buffer').Buffer,
+      buf1 = new Buffer(26),
+      buf2 = new Buffer(26),
+      i;
+  
+    for (i = 0 ; i < 26 ; i += 1) {
+      buf1[i] = i + 97; // 97 is ASCII a
+      buf2[i] = 33; // ASCII !
+    }
+
+    buf1.copy(buf2, 8, 16, 20);
+    sys.puts(buf2.toString('ascii', 0, 25));
+
+    // !!!!!!!!qrst!!!!!!!!!!!!!
+    
+
 ### buffer.slice(start, end)
+
 Returns a new buffer which references the
 same memory as the old, but offset and cropped by the `start` and `end`
-indexes. **Modifying the new buffer slice will modify memory in the original
-buffer!**
+indexes.
 
-### buffer.write(string, encoding, offset)
-Writes `string` to the buffer at `offset` using the given encoding. Returns
-number of octets written.  If `buffer` did not contain enough space to fit
-the entire string it will write a partial amount of the string. In the case
-of `encoding=='utf8'`, the method will not write partial characters.
+**Modifying the new buffer slice will modify memory in the original buffer!**
 
-### buffer.toString(encoding, start, end)
-Decodes and returns a string assuming in the given encoding beginning at
-`start` and ending at `end`.
+Example: build a Buffer with the ASCII alphabet, take a slice, then modify one byte
+from the original Buffer.
+
+    var sys = require('sys'),
+      Buffer = require('buffer').Buffer,
+      buf1 = new Buffer(26), buf2,
+      i;
+  
+    for (i = 0 ; i < 26 ; i += 1) {
+      buf1[i] = i + 97; // 97 is ASCII a
+    }
 
+    buf2 = buf1.slice(0, 3);
+    sys.puts(buf2.toString('ascii', 0, buf2.length));
+    buf1[0] = 33;
+    sys.puts(buf2.toString('ascii', 0, buf2.length));
+
+    // abc
+    // !bc
 
 
 ## EventEmitter
@@ -244,12 +306,15 @@ Makes the data event emit a string instead of a `Buffer`. `encoding` can be
 `'utf8'`, `'ascii'`, or `'binary'`.
 
 ### stream.pause()
+
 Pauses the incoming `'data'` events.
 
 ### stream.resume()
+
 Resumes the incoming `'data'` events after a `pause()`.
 
 ### stream.destroy()
+
 Closes the underlying file descriptor. Stream will not emit any more events.
 
 
@@ -278,6 +343,7 @@ Emitted on error with the exception `e`.
 Emitted when the underlying file descriptor has been closed.
 
 ### stream.write(string, encoding)
+
 Writes `string` with the given `encoding` to the stream.  Returns `true` if
 the string has been flushed to the kernel buffer.  Returns `false` to
 indicate that the kernel buffer is full, and the data will be sent out in
@@ -286,19 +352,24 @@ empty again. The `encoding` defaults to `'utf8'`.
 
 
 ### stream.write(buffer)
+
 Same as the above except with a raw buffer.
 
 ### stream.end()
+
 Terminates the stream with EOF or FIN.
 
 ### stream.end(string, encoding)
+
 Sends `string` with the given `encoding` and terminates the stream with EOF
 or FIN. This is useful to reduce the number of packets sent.
 
 ### stream.end(buffer)
+
 Same as above but with a `buffer`.
 
 ### stream.destroy()
+
 Closes the underlying file descriptor. Stream will not emit any more events.
 
 
@@ -307,25 +378,51 @@ Closes the underlying file descriptor. Stream will not emit any more events.
 These object are available in the global scope and can be accessed from anywhere.
 
 ### global
+
 The global namespace object.
 
 ### process
-The process object. Most stuff lives in here. See the 'process object'
+
+The process object. Most stuff lives in here. See the `'process object'`
 section.
 
 ### require()
-To require modules. See the modules section.
+
+To require modules. See the `'Modules'` section.
 
 ### require.paths
-The search path for absolute path arguments to `require()`.
+
+An array of search paths for `require()`.  This array can be modified to add custom paths.
+
+Example: add a new path to the beginning of the search list
+
+    var sys = require('sys');
+
+    require.paths.unshift('/usr/local/node');
+    sys.puts(require.paths);
+    // /usr/local/node,/Users/mjr/.node_libraries
+
 
 ### __filename
-The filename of the script being executed.
+
+The filename of the script being executed.  This is the absolute path, and not necessarily
+the same filename passed in as a command line argument.
 
 ### __dirname
+
 The dirname of the script being executed.
 
+Example: running `node example.js` from `/Users/mjr`
+
+    var sys = require('sys');
+    sys.puts(__filename);
+    sys.puts(__dirname);
+    // /Users/mjr/example.js
+    // /Users/mjr
+
+
 ### module
+
 A reference to the current module (of type `process.Module`). In particular
 `module.exports` is the same as the `exports` object. See `src/process.js`
 for more information.
@@ -361,7 +458,6 @@ Example of listening for `exit`:
 
 `function (err) { } `
 
-
 Emitted when an exception bubbles all the way back to the event loop. If a
 listener is added for this exception, the default action (which is to print
 a stack trace and exit) will not occur.
@@ -419,7 +515,6 @@ Example: the definition of `sys.puts`
     };
 
 
-
 ### process.openStdin()
 
 Opens the standard input stream, returns a readable stream.
@@ -462,7 +557,6 @@ This will generate:
     4: four
 
 
-
 ### process.chdir(directory)
 
 Changes the current working directory of the process or throws an exception if that fails.
@@ -519,7 +613,6 @@ Returns the current working directory of the process.
 An object containing the user environment. See environ(7).
 
 
-
 ### process.exit(code)
 
 Ends the process with the specified `code`.  If omitted, exit uses the 
@@ -568,7 +661,7 @@ Gets/sets the user identity of the process. (See setuid(2).)  This is the numeri
 
 A compiled-in property that exposes `NODE_PREFIX`.
 
-    require('sys').puts('Install prefix: ' + process.installPrefix);
+    require('sys').puts('Prefix: ' + process.installPrefix);
 
 
 ### process.kill(pid, signal)
@@ -909,13 +1002,14 @@ output, and return it all in a callback.
         exec  = require('child_process').exec,
         child;
 
-    child = exec('cat *.js bad_file | wc -l', function (error, stdout, stderr) {
-      sys.print('stdout: ' + stdout);
-      sys.print('stderr: ' + stderr);
-      if (error !== null) {
-        sys.puts('exec error: ' + error);
-      }
-    });
+    child = exec('cat *.js bad_file | wc -l', 
+      function (error, stdout, stderr) {
+        sys.print('stdout: ' + stdout);
+        sys.print('stderr: ' + stderr);
+        if (error !== null) {
+          sys.puts('exec error: ' + error);
+        }
+      });
 
 The callback gets the arguments `(error, stdout, stderr)`. On success, `error`
 will be `null`.  On error, `error` will be an instance of `Error` and `err.code`
@@ -939,13 +1033,14 @@ the child process is killed.
 
 ## Script
 
-`Script` class provides with compiling, remembering and running pieces of Javascript code. You can get `Script` class by issuing
+`Script` class compiles and runs JavaScript code. You can access this class with:
 
     var Script = process.binding('evals').Script;
 
+New JavaScript code can be compiled and run immediately or compiled, saved, and run later.
 
 
-### Script.runInThisContext(code, filename='evalmachine.< anonymous >')
+### Script.runInThisContext(code, filename)
 
 Similar to `process.compile`.  `Script.runInThisContext` compiles `code` as if it were loaded from `filename`,
 runs it and returns the result. Running code does not have access to local scope. `filename` is optional.
@@ -954,12 +1049,16 @@ Example of using `Script.runInThisContext` and `eval` to run the same code:
 
     var sys = require('sys'),
         localVar = 123,
-        usingscript, evaled;
+        usingscript, evaled,
+        Script = process.binding('evals').Script;
 
-    usingscript = Script.runInThisContext('localVar = 1;', 'myfile.js');
-    sys.puts('localVar: ' + localVar + ', usingscript: ' + usingscript);
+    usingscript = Script.runInThisContext('localVar = 1;',
+      'myfile.js');
+    sys.puts('localVar: ' + localVar + ', usingscript: ' +
+      usingscript);
     evaled = eval('localVar = 1;');
-    sys.puts('localVar: ' + localVar + ', evaled: ' + evaled);
+    sys.puts('localVar: ' + localVar + ', evaled: ' +
+      evaled);
 
     // localVar: 123, usingscript: 1
     // localVar: 1, evaled: 1
@@ -971,92 +1070,106 @@ In case of syntax error in `code`, `Script.runInThisContext` emits the syntax er
 and throws.an exception.
 
 
-
-### Script.runInNewContext(code, sandbox={}, filename='evalmachine.< anonymous >')
+### Script.runInNewContext(code, sandbox, filename)
 
 `Script.runInNewContext` compiles `code` to run in `sandbox` as if it were loaded from `filename`,
 then runs it and returns the result. Running code does not have access to local scope and
 the object `sandbox` will be used as the global object for `code`.
 `sandbox` and `filename` are optional.
 
+Example: compile and execute code that increments a global variable and sets a new one.
+These globals are contained in the sandbox.
+
     var sys = require('sys'),
+        Script = process.binding('evals').Script,
         sandbox = {
           animal: 'cat',
           count: 2
         };
 
-    Script.runInNewContext('count += 1; name = 'kitty'', sandbox, 'myfile.js');
+    Script.runInNewContext(
+      'count += 1; name = "kitty"', sandbox, 'myfile.js');
     sys.puts(sys.inspect(sandbox));
 
+    // { animal: 'cat', count: 3, name: 'kitty' }
+
 Note that running untrusted code is a tricky business requiring great care.  To prevent accidental
-global variable leakage, `Script.runInNewContext` is quite useful, but to safely run untrusted code, many more steps
-must be taken.
+global variable leakage, `Script.runInNewContext` is quite useful, but safely running untrusted code
+requires a separate process.
 
 In case of syntax error in `code`, `Script.runInThisContext` emits the syntax error to stderr
-and throws.an exception.
-
+and throws an exception.
 
 
-### new Script(code, filename='evalmachine.< anonymous >')
+### new Script(code, filename)
 
 `new Script` compiles `code` as if it were loaded from `filename`,
-but does not run it. Instead, it returns Script object representing this compiled code.
+but does not run it. Instead, it returns a `Script` object representing this compiled code.
 This script can be run later many times using methods below.
-The returned script is not bound to any global object,
-it is bound before each run, just for that run. `filename` is optional.
+The returned script is not bound to any global object.
+It is bound before each run, just for that run. `filename` is optional.
 
 In case of syntax error in `code`, `new Script` emits the syntax error to stderr
-and throws.an exception.
-
+and throws an exception.
 
 
 ### script.runInThisContext()
 
-Similar to "static" version in `Script.runInThisContext`, but now being a method of precompiled Script object.
-`script.runInThisContext` runs the code of `script` and returns the result. Running code does not have access to local scope
-and is run for actual `global` object (v8: in actual context).
+Similar to `Script.runInThisContext` (note capital 'S'), but now being a method of a precompiled Script object.
+`script.runInThisContext` runs the code of `script` and returns the result.
+Running code does not have access to local scope, but does have access to the `global` object
+(v8: in actual context).
 
-Example of using `script.runInThisContext` and `eval` to run the same code:
+Example of using `script.runInThisContext` to compile code once and run it multiple times:
 
     var sys = require('sys'),
-        localVar = 123,
-        script, usingscript, evaled;
+        Script = process.binding('evals').Script,
+        scriptObj, i;
+    
+    globalVar = 0;
 
-    script = new Script('localVar = 1', 'myfile.js');
-    usingscript = script.runInThisContext();
-    sys.puts('localVar: ' + localVar + ', usingscript: ' + usingscript);
-    evaled = eval('localVar = 1;');
-    sys.puts('localVar: ' + localVar + ', evaled: ' + evaled);
+    scriptObj = new Script('globalVar += 1', 'myfile.js');
 
-    // localVar: 123, usingscript: 1
-    // localVar: 1, evaled: 1
+    for (i = 0; i < 1000 ; i += 1) {
+      scriptObj.runInThisContext();
+    }
 
-`script.runInThisContext` does not have access to the local scope, so `localVar` is unchanged.
-`eval` does have access to the local scope, so `localVar` is changed.
+    sys.puts(globalVar);
 
+    // 1000
 
 
-### script.runInNewContext(sandbox={})
+### script.runInNewContext(sandbox)
 
-Similar to "static" version in `Script.runInNewContext`, but now being a method of precompiled Script object.
-`script.runInNewContext` runs the code of `script` in a `sandbox` and returns the result.
-Running code does not have access to local scope and the object `sandbox` will be used as the global object for the code.
-`sandbox`is optional.
+Similar to `Script.runInNewContext` (note capital 'S'), but now being a method of a precompiled Script object.
+`script.runInNewContext` runs the code of `script` with `sandbox` as the global object and returns the result.
+Running code does not have access to local scope. `sandbox` is optional.
+
+Example: compile code that increments a global variable and sets one, then execute this code multiple times.
+These globals are contained in the sandbox.
 
     var sys = require('sys'),
-        script = new Script('count += 1; name = 'kitty'', 'myfile.js'),
+        Script = process.binding('evals').Script,
+        scriptObj, i,
         sandbox = {
           animal: 'cat',
           count: 2
         };
 
-    script.runInNewContext(sandbox);
+    scriptObj = new Script(
+        'count += 1; name = "kitty"', 'myfile.js');
+
+    for (i = 0; i < 10 ; i += 1) {
+      scriptObj.runInNewContext(sandbox);
+    }
+
     sys.puts(sys.inspect(sandbox));
 
-Note that running untrusted code is a tricky business requiring great care.  To prevent accidental
-global variable leakage, `Script.runInNewContext` is quite useful, but to safely run untrusted code, many more steps
-must be taken.
+    // { animal: 'cat', count: 12, name: 'kitty' }
 
+Note that running untrusted code is a tricky business requiring great care.  To prevent accidental
+global variable leakage, `script.runInNewContext` is quite useful, but safely running untrusted code
+requires a separate process.
 
 
 ## File System
@@ -2489,6 +2602,66 @@ There are a few special REPL commands:
   - `.help` - Show this list of special commands.
   
 
+## Modules
+
+Node uses the CommonJS module system.
+
+Node has a simple module loading system.  In Node, files and modules are in
+one-to-one correspondence.  As an example, `foo.js` loads the module
+`circle.js` in the same directory.
+
+The contents of `foo.js`:
+
+  var circle = require('./circle'),
+      sys = require('sys');
+  sys.puts( 'The area of a circle of radius 4 is '
+          + circle.area(4));
+
+The contents of `circle.js`:
+
+  var PI = 3.14;
+
+  exports.area = function (r) {
+    return PI * r * r;
+  };
+
+  exports.circumference = function (r) {
+    return 2 * PI * r;
+  };
+
+The module `circle.js` has exported the functions `area()` and
+`circumference()`.  To export an object, add to the special `exports`
+object.  (Alternatively, one can use `this` instead of `exports`.) Variables
+local to the module will be private. In this example the variable `PI` is
+private to `circle.js`. The function `puts()` comes from the module `'sys'`,
+which is a built-in module. Modules which are not prefixed by `'./'` are
+built-in module--more about this later.
+
+A module prefixed with `'./'` is relative to the file calling `require()`.
+That is, `circle.js` must be in the same directory as `foo.js` for
+`require('./circle')` to find it.
+
+Without the leading `'./'`, like `require('assert')` the module is searched
+for in the `require.paths` array. `require.paths` on my system looks like
+this: 
+
+`[ '/home/ryan/.node_libraries' ]`
+
+That is, when `require('assert')` is called Node looks for: 
+
+* 1: `/home/ryan/.node_libraries/assert.js`
+* 2: `/home/ryan/.node_libraries/assert.node`
+* 3: `/home/ryan/.node_libraries/assert/index.js`
+* 4: `/home/ryan/.node_libraries/assert/index.node`
+
+interrupting once a file is found. Files ending in `'.node'` are binary Addon
+Modules; see 'Addons' below. `'index.js'` allows one to package a module as
+a directory.
+
+`require.paths` can be modified at runtime by simply unshifting new
+paths onto it, or at startup with the `NODE_PATH` environmental
+variable (which should be a list of paths, colon separated).
+
 
 ## Addons