More documentation

git-svn-id: http://pyfilesystem.googlecode.com/svn/trunk@645 67cdc799-7952-0410-af00-57a81ceafa0f

7 files changed, 163 insertions, 86 deletions

diff --git a/docs/concepts.rst b/docs/concepts.rst
index 0a911be..297ffa7 100644
--- a/docs/concepts.rst
+++ b/docs/concepts.rst
@@ -6,9 +6,9 @@ It is generally quite easy to get in to the mind-set of using PyFilesystem inter
 Sandboxing
 ----------
 
-FS objects are not permitted to work with any files / directories outside of the Filesystem they represent. If you attempt to open a file or directory outside the root of the FS (e.g. by using "../" in the path) you will get a ValueError.
+FS objects are not permitted to work with any files / directories outside of the Filesystem they represent. If you attempt to open a file or directory outside the root of the FS (e.g. by using "../" in the path) you will get a ``ValueError``.
 
-There is no concept of a current working directory in PyFilesystem, since it is a common source of bugs and not all filesytems even have such a notion. If you want to work with a sub-directory of a FS object, you can use the `opendir` method which returns another FS object representing the sub-directory.
+There is no concept of a current working directory in PyFilesystem, since it is a common source of bugs and not all filesytems even have such a notion. If you want to work with a sub-directory of a FS object, you can use the :meth:`~fs.base.FS.opendir` method which returns another FS object representing the sub-directory.
 
 For example, consider the following directory structure. The directory `foo` contains two sub-directories; `bar` and `baz`::
 
@@ -25,7 +25,7 @@ We can open the `foo` directory with the following code::
 	from fs.osfs import OSFS
 	foo_fs = OSFS('foo')
 
-The `foo_fs` object can work with any of the contents of `bar` and `baz`, which may not be desirable, especially if we are passing `foo_fs` to an untrusted function or one that could potentially delete files. Fortunately we can isolate a single sub-directory with then `opendir` method::
+The `foo_fs` object can work with any of the contents of `bar` and `baz`, which may not be desirable, especially if we are passing `foo_fs` to an untrusted function or one that could potentially delete files. Fortunately we can isolate a single sub-directory with then :meth:`~fs.base.FS.opendir` method::
 
 	bar_fs = foo_fs.opendir('bar')
 
@@ -53,14 +53,15 @@ When working with paths in FS objects, keep in mind the following:
  * A single dot means 'current directory'
  * A double dot means 'previous directory'
  
-Note that paths used by the FS interface will use this format, but the constructor or additional methods may not. Notably the ``osfs.OSFS`` constructor which requires an OS path -- the format of which can be platform-dependant.
+Note that paths used by the FS interface will use this format, but the constructor or additional methods may not.
+Notably the :mod:`~fs.osfs.OSFS` constructor which requires an OS path -- the format of which can be platform-dependant.
 
 There are many helpful functions for working with paths in the :mod:`fs.path` module.
 
 System Paths
 ++++++++++++
 
-Not all Python modules can use file-like objects, especially those which interface with C libraries. For these situations you will need to retrieve the `system path` from an FS object you are working with. You can do this with the `getsyspath` method which converts a valid path in the context of the FS object to an absolute path on the system, if one exists.
+Not all Python modules can use file-like objects, especially those which interface with C libraries. For these situations you will need to retrieve the `system path` from an FS object you are working with. You can do this with the :meth:`~fs.base.FS.getsyspath` method which converts a valid path in the context of the FS object to an absolute path on the system, should one exist.
 
 For example::
 
@@ -69,12 +70,13 @@ For example::
 	>>> home_fs.getsyspath('test.txt')
 	u'/home/will/test.txt'
 
-Not all FS implementation will map to a valid system path (e.g. the FTP FS object). If you call `getsyspath` on such FS objects you will either get a `NoSysPathError` exception or a return value of None, if you call `getsyspath` with `allow_none=True`.
+Not all FS implementation will map to a valid system path (e.g. the FTP FS object).
+If you call :meth:`~fs.base.FS.getsyspath` on such FS objects you will either get a :class:`~fs.errors.NoSysPathError` exception or a return value of ``None``, if you call ``getsyspath`` with `allow_none=True`.
 
 Errors
 ------
 
-PyFilesystem converts all exceptions to a common type, so that you need only write your exception handling code once. For example, if you try to open a file that doesn't exist, PyFilesystem will throw a ``fs.errors.ResourceNotFoundError`` regardless of whether the filesystem is local, on a ftp server or in a zip file::
+PyFilesystem converts all exceptions to a common type, so that you need only write your exception handling code once. For example, if you try to open a file that doesn't exist, PyFilesystem will throw a :class:`fs.errors.ResourceNotFoundError` regardless of whether the filesystem is local, on a ftp server or in a zip file::
 
 	>>> from fs.osfs import OSFS
 	>>> root_fs = OSFS('/')
diff --git a/docs/implementersguide.rst b/docs/implementersguide.rst
index aa20f9d..2c59955 100644
--- a/docs/implementersguide.rst
+++ b/docs/implementersguide.rst
@@ -1,11 +1,38 @@
+.. _implementers:
+
 A Guide For Filesystem Implementers 
 ===================================
 
-PyFilesystems objects are designed to be as generic as possible and still expose as much functionality as possible.
+PyFilesystems objects are designed to be as generic as possible and still expose the full filesystem functionality.
 With a little care, you can write a wrapper for your filesystem that allows it to work interchangeably with any of the built-in FS classes and tools. 
 
 To create a working PyFilesystem interface, derive a class from :py:class:`fs.base.FS` and implement the 9 :ref:`essential-methods`.
+The base class uses these essential methods as a starting point for providing a lot of extra functionality,
+but in some cases the default implementation may not be the most efficient.
+For example, most filesystems have some atomic way of moving a file without having to copy data,
+whereas the default implementation of :meth:`~fs.base.FS.move` method must copy all the bytes in the file.
+Although any of the :ref:`non-essential-methods` may be overriden, efficient versions of the following methods will have the greatest impact on performance:     
+
+	* :meth:`~fs.base.FS.copy` copy a file
+	* :meth:`~fs.base.FS.copydir` copy a directory
+	* :meth:`~fs.base.FS.exists` check if a file / directory exists
+	* :meth:`~fs.base.FS.getsyspath` get a system path for a given resource, if it exists 	
+	* :meth:`~fs.base.FS.move` move a file
+	* :meth:`~fs.base.FS.movedir` move a directory
+
+For network based filesystems (i.e. where the physical data is pulled from a network),
+there are a few methods which can reduce the number of round trips to the server,
+if an efficient implementation is provided:
+	
+	* :meth:`~fs.base.FS.listdirinfo` returns the directory contents and info dictionary in one call
+	* :meth:`~fs.base.FS.ilistdir` a generator version of :meth:`~fs.base.FS.listdir` 
+	* :meth:`~fs.base.FS.ilistdirinfo` a generator version of :meth:`~fs.base.FS.listdirinfo`
 
+The generator methods (beginning with ``i``) are intended for use with filesystems that contain a lot of files,
+where reading the directory in one go may be expensive.
+
+Other methods in the :doc:`interface` are unlikely to require a non-default implementation,
+but there is nothing preventing you from implementing them -- just be careful to use the same signature and replicate expected functionality. 
 
 Filesystem Errors
 -----------------
@@ -27,4 +54,69 @@ and passes the original exception as an argument.::
     except someapi.UnableToOpen, e:
         raise errors.ResourceNotFoundError(path=path, details=e)
 		
-Any code written to catch the generic error, can also retrieve the original exception if it contains additional information. 
\ No newline at end of file
+Any code written to catch the generic error, can also retrieve the original exception if it contains additional information.
+
+Thread Safety
+-------------
+
+All PyFilesystems methods, other than the constructor, should be thread-safe where-ever possible.
+One way to do this is to pass ``threads_synchronize=True`` to the base constructor and use the :func:`~fs.base.synchronize` decorator to lock the FS object when a method is called.
+
+If the implementation can not be made thread-safe for technical reasons, ensure that ``getmeta("thread_safe")`` returns ``False``.
+
+
+Meta Values
+-----------
+
+The :meth:`~fs.base.FS.getmeta` method is designed to return implementation specific information.
+PyFilesystem implementations should return as much of the standard set of meta values as possible.
+
+Implementations are also free to reserve a dotted namespace notation for themselves, to provide an interface to highly specific information.
+If you do this, please avoid generic terms as they may conflict with existing or future implementations.
+For example ``"bobs_ftpfs.author"``, rather than ``"ftpfs.author"``.
+
+If your meta values are static, i.e. they never change, then create a dictionary class attribute called ``_meta`` in your implementation that contains all the meta keys, values. 
+The default ``getmeta`` implementation will pull the meta values from this dictionary.
+
+.. _essential-methods:
+
+Essential Methods
+-----------------
+
+The following methods are required for a minimal Filesystem interface:
+
+    * :meth:`~fs.base.FS.open` Opens a file for read/writing
+    * :meth:`~fs.base.FS.isfile` Check wether the path exists and is a file
+    * :meth:`~fs.base.FS.isdir` Check wether a path exists and is a directory
+    * :meth:`~fs.base.FS.listdir` List the contents of a directory
+    * :meth:`~fs.base.FS.makedir` Create a new directory
+    * :meth:`~fs.base.FS.remove` Remove an existing file
+    * :meth:`~fs.base.FS.removedir` Remove an existing directory
+    * :meth:`~fs.base.FS.rename` Atomically rename a file or directory
+    * :meth:`~fs.base.FS.getinfo` Return information about the path e.g. size, mtime
+    
+
+.. _non-essential-methods:
+
+Non - Essential Methods
+-----------------------
+
+The following methods have default implementations in :py:class:`fs.base.FS` and aren't required for a functional FS interface. They may be overriden if an alternative implementation can be supplied:
+
+    * :meth:`~fs.base.FS.copy` Copy a file to a new location
+    * :meth:`~fs.base.FS.copydir` Recursively copy a directory to a new location
+    * :meth:`~fs.base.FS.desc` Return a short destriptive text regarding a path
+    * :meth:`~fs.base.FS.exists` Check whether a path exists as file or directory    
+    * :meth:`~fs.base.FS.listdirinfo` Get a directory listing along with the info dict for each entry
+    * :meth:`~fs.base.FS.ilistdir` Generator version of the listdir method
+    * :meth:`~fs.base.FS.ilistdirinfo` Generator version of the listdirinfo method
+    * :meth:`~fs.base.FS.getpathurl` Get an external URL at which the given file can be accessed, if possible
+    * :meth:`~fs.base.FS.getsyspath` Get a file's name in the local filesystem, if possible
+    * :meth:`~fs.base.FS.getmeta` Get the value of a filesystem meta value, if it exists
+    * :meth:`~fs.base.FS.getmmap` Gets an mmap object for the given resource, if supported
+    * :meth:`~fs.base.FS.hassyspath` Check if a path maps to a system path (recognised by the OS)    
+    * :meth:`~fs.base.FS.haspathurl` Check if a path maps to an external URL    
+    * :meth:`~fs.base.FS.hasmeta` Check if a filesystem meta value exists
+    * :meth:`~fs.base.FS.move` Move a file to a new location        
+    * :meth:`~fs.base.FS.movedir` Recursively move a directory to a new location    
+    * :meth:`~fs.base.FS.settimes` Sets the accessed and modified times of a path
diff --git a/docs/index.rst b/docs/index.rst
index 241c58a..0f3b610 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -50,6 +50,7 @@ Code Documentation
    sftpfs.rst
    tempfs.rst
    utils.rst
+   watch.rst
    wrapfs/index.rst
    zipfs.rst
    contrib/index.rst
diff --git a/docs/interface.rst b/docs/interface.rst
index f39ed65..aeb56a1 100644
--- a/docs/interface.rst
+++ b/docs/interface.rst
@@ -1,73 +1,52 @@
+.. _filesystem-interface:
+
 Filesystem Interface
 ====================
 
-It requires a relatively small number of methods to implement a working FS object.
-
-If you are looking to implement a working FS object, derive a class from :py:class:`fs.base.FS` and implement the essential methods (below). Be sure to convert all exceptions to instances of :class:`fs.errors.FSError`.
-
-It may also be worthwhile implementing some of the non-essential methods, as the default implementations may not be optimal. For example, the method :meth:`fs.base.FS.move` is implemented as a file copy followed by a delete, but many filesystems can move a file without copying data.
-
-If the filesystem you are implementing maps paths to the native filesystem, be sure to implement :py:meth:`~fs.base.FS.getsyspath`. Doing so will improve performance, especially when copying / moving files between FS objects.
-
-.. _essential-methods:
-
-Essential Methods
------------------
-
-The following methods are required for a minimal Filesystem interface:
-
-    * :meth:`~fs.base.FS.open` Opens a file for read/writing
-    * :meth:`~fs.base.FS.isfile` Check wether the path exists and is a file
-    * :meth:`~fs.base.FS.isdir` Check wether a path exists and is a directory
-    * :meth:`~fs.base.FS.listdir` List the contents of a directory
-    * :meth:`~fs.base.FS.makedir` Create a new directory
-    * :meth:`~fs.base.FS.remove` Remove an existing file
-    * :meth:`~fs.base.FS.removedir` Remove an existing directory
-    * :meth:`~fs.base.FS.rename` Atomically rename a file or directory
-    * :meth:`~fs.base.FS.getinfo` Return information about the path e.g. size, mtime
-    
-
-.. _non-essential-methods:
-
-Non - Essential Methods
------------------------
-
-The following methods have default implementations in :py:class:`fs.base.FS` and aren't required for a functional FS interface. They may be overriden if an alternative implementation can be supplied:
-
-    * :meth:`~fs.base.FS.copy` Copy a file to a new location
-    * :meth:`~fs.base.FS.copydir` Recursively copy a directory to a new location
-    * :meth:`~fs.base.FS.desc` Return a short destriptive text regarding a path
-    * :meth:`~fs.base.FS.exists` Check whether a path exists as file or directory    
-    * :meth:`~fs.base.FS.listdirinfo` Get a directory listing along with the info dict for each entry
-    * :meth:`~fs.base.FS.ilistdir` Generator version of the listdir method
-    * :meth:`~fs.base.FS.ilistdirinfo` Generator version of the listdirinfo method
-    * :meth:`~fs.base.FS.getsyspath` Get a file's name in the local filesystem, if possible
-    * :meth:`~fs.base.FS.hassyspath` Check if a path maps to a system path (recognised by the OS)
-    * :meth:`~fs.base.FS.getpathurl` Get an external URL at which the given file can be accessed, if possible
-    * :meth:`~fs.base.FS.haspathurl` Check if a path maps to an external URL
-    * :meth:`~fs.base.FS.getmeta` Get the value of a filesystem meta value, if it exists
-    * :meth:`~fs.base.FS.hasmeta` Check if a filesystem meta value exists
-    * :meth:`~fs.base.FS.move` Move a file to a new location        
-    * :meth:`~fs.base.FS.movedir` Recursively move a directory to a new location
-    * :meth:`~fs.base.FS.opendir` Opens a directory and returns an FS object that represents it
-    * :meth:`~fs.base.FS.safeopen` Like :meth:`~fs.base.FS.open` but returns a :class:`~fs.base.NullFile` if the file could not be opened
-    * :meth:`~fs.base.FS.settimes` Sets the accessed and modified times of a path
-
-
-.. _utility-methods:
-
-Utility Methods
----------------
-
-The following members have implementations in :py:class:`fs.base.FS` and will probably never require a non-default implementation, although there is nothing to prevent a derived class from implementing these:
-
-    * :meth:`~fs.base.FS.createfile` Create a file with data
-    * :meth:`~fs.base.FS.getcontents` Returns the contents of a file as a string
-    * :meth:`~fs.base.FS.setcontents` Sets the contents of a file as a string or file-like object
-    * :meth:`~fs.base.FS.getsize` Returns the number of bytes used for a given file or directory
-    * :meth:`~fs.base.FS.isdirempty` Checks if a directory contains no files
-    * :meth:`~fs.base.FS.makeopendir` Creates a directroy (if it exists) and returns an FS object for that directory
-    * :meth:`~fs.base.FS.walk` Like `listdir` but descends in to sub-directories
-    * :meth:`~fs.base.FS.walkfiles` Returns an iterable of file paths in a directory, and its sub-directories
-    * :meth:`~fs.base.FS.walkdirs` Returns an iterable of paths to sub-directories
-    
+The following methods are available in all PyFilesystem implementation:
+	
+	* :meth:`~fs.base.FS.close` Close the filesystem and free any resources
+	* :meth:`~fs.base.FS.copy` Copy a file to a new location
+	* :meth:`~fs.base.FS.copydir` Recursively copy a directory to a new location
+	* :meth:`~fs.base.FS.cachehint` Permit implementation to use aggressive caching for performance reasons
+	* :meth:`~fs.base.FS.createfile` Create a file with data
+	* :meth:`~fs.base.FS.desc` Return a short descriptive text regarding a path
+	* :meth:`~fs.base.FS.exists` Check whether a path exists as file or directory
+	* :meth:`~fs.base.FS.getcontents` Returns the contents of a file as a string
+	* :meth:`~fs.base.FS.getinfo` Return information about the path e.g. size, mtime
+	* :meth:`~fs.base.FS.getmeta` Get the value of a filesystem meta value, if it exists
+	* :meth:`~fs.base.FS.getmmap` Gets an mmap object for the given resource, if supported
+	* :meth:`~fs.base.FS.getpathurl` Get an external URL at which the given file can be accessed, if possible
+	* :meth:`~fs.base.FS.getsize` Returns the number of bytes used for a given file or directory
+	* :meth:`~fs.base.FS.getsyspath` Get a file's name in the local filesystem, if possible
+	* :meth:`~fs.base.FS.hasmeta` Check if a filesystem meta value exists
+	* :meth:`~fs.base.FS.haspathurl` Check if a path maps to an external URL
+	* :meth:`~fs.base.FS.hassyspath` Check if a path maps to a system path (recognised by the OS)
+	* :meth:`~fs.base.FS.ilistdir` Generator version of the :meth:`~fs.base.FS.listdir` method
+	* :meth:`~fs.base.FS.ilistdirinfo` Generator version of the :meth:`~fs.base.FS.listdirinfo` method
+	* :meth:`~fs.base.FS.isdir` Check whether a path exists and is a directory
+	* :meth:`~fs.base.FS.isdirempty` Checks if a directory contains no files
+	* :meth:`~fs.base.FS.isfile` Check whether the path exists and is a file
+	* :meth:`~fs.base.FS.listdir` List the contents of a directory
+	* :meth:`~fs.base.FS.listdirinfo` Get a directory listing along with the info dict for each entry
+	* :meth:`~fs.base.FS.makedir` Create a new directory
+	* :meth:`~fs.base.FS.makeopendir` Make a directory and returns the FS object that represents it
+	* :meth:`~fs.base.FS.move` Move a file to a new location
+	* :meth:`~fs.base.FS.movedir` Recursively move a directory to a new location
+	* :meth:`~fs.base.FS.open` Opens a file for read/writing
+	* :meth:`~fs.base.FS.opendir` Opens a directory and returns a FS object that represents it
+	* :meth:`~fs.base.FS.remove` Remove an existing file
+	* :meth:`~fs.base.FS.removedir` Remove an existing directory
+	* :meth:`~fs.base.FS.rename` Atomically rename a file or directory
+	* :meth:`~fs.base.FS.safeopen` Like :meth:`~fs.base.FS.open` but returns a :class:`~fs.base.NullFile` if the file could not be opened
+	* :meth:`~fs.base.FS.setcontents` Sets the contents of a file as a string or file-like object
+	* :meth:`~fs.base.FS.setcontents_async` Sets the contents of a file asynchronously
+	* :meth:`~fs.base.FS.settimes` Sets the accessed and modified times of a path
+	* :meth:`~fs.base.FS.tree` Display an ascii rendering of the directory structure
+	* :meth:`~fs.base.FS.walk` Like :meth:`~fs.base.FS.listdir` but descends in to sub-directories
+	* :meth:`~fs.base.FS.walkdirs` Returns an iterable of paths to sub-directories
+	* :meth:`~fs.base.FS.walkfiles` Returns an iterable of file paths in a directory, and its sub-directories
+
+See :py:class:`fs.base.FS` for the method signature and full details.
+
+If you intend to implement an FS object, see :ref:`implementers`.
\ No newline at end of file
diff --git a/docs/introduction.rst b/docs/introduction.rst
index ae32dce..dc6f58c 100644
--- a/docs/introduction.rst
+++ b/docs/introduction.rst
@@ -3,12 +3,12 @@ Introduction
 
 PyFilesystem is a Python module that provides a common interface to any filesystem.
 
-Think of PyFilesystem (FS) objects as the next logical step to Python's `file` objects. Just as file-like objects abstract a single file, FS objects abstract the whole filesystem by providing a common interface to operations such as reading directories, getting file information, opening/copying/deleting files etc.
+Think of PyFilesystem (FS) objects as the next logical step to Python's ``file`` class. Just as *file-like* objects abstract a single file, FS objects abstract the whole filesystem by providing a common interface to operations such as reading directories, getting file information, opening/copying/deleting files etc.
 
 Even if you only want to work with the local filesystem, PyFilesystem simplifies a number of common operations and reduces the chance of error.
 
-About PyFilestem
-----------------
+About PyFilesystem
+------------------
 
 PyFilesystem was initially created by Will McGugan (http://www.willmcgugan.com) and is now a joint effort with Ryan Kelly (http://www.rfk.id.au/).
 
@@ -16,7 +16,7 @@ PyFilesystem was initially created by Will McGugan (http://www.willmcgugan.com)
 Need Help?
 ----------
 
-PyFilesystem is in development, but should be pretty stable. If you have any problems or questions, please contact the developers through one of the following channels:
+If you have any problems or questions, please contact the developers through one of the following channels:
 
 Bugs
 ####
diff --git a/docs/opening.rst b/docs/opening.rst
index 5bfe120..d54e36f 100644
--- a/docs/opening.rst
+++ b/docs/opening.rst
@@ -10,13 +10,14 @@ you create an instance of the appropriate class. For example, the following open
 This is fine if you know beforehand where the directory you want to work with is located, and on what medium.
 However, there are occasions where the location of the files may change at runtime or should be specified in a config file or from the command line.
 
-In these situations you can use an _opener_ which is a generic way of specifying a filesystem. For example, the following is equivalent to the code above::
+In these situations you can use an *opener*, which is a generic way of specifying a filesystem. For example, the following is equivalent to the code above::
 	
 	from fs.opener import fsopen
 	my_fs = fsopen('/foo/bar')
 
-The `fsopen` method takes a string that identifies the filesystem, but if called with a regular path, it will return an OSFS instance.
-To open a different kind of filesystem, you specify it with a URI like syntax. The following code opens an ftp filesystem rather than a directory on your harddrive::
+The `fsopen` method takes a string that identifies the filesystem with a URI syntax, but if called with a regular path will return an OSFS instance.
+To open a different kind of filesystem, precede the path with the required protocol.
+For example, the following code opens an FTP filesystem rather than a directory on your harddrive::
 
 	from fs.opener import fsopen
 	my_fs = fsopen('ftp://example.org/foo/bar')
diff --git a/docs/watch.rst b/docs/watch.rst
new file mode 100644
index 0000000..2ff1c8e
--- /dev/null
+++ b/docs/watch.rst
@@ -0,0 +1,2 @@
+.. automodule:: fs.watch
+    :members:
\ No newline at end of file