updated changelog to include python 3 compat; various other documentation updates

13 files changed, 151 insertions, 145 deletions

diff --git a/CHANGES b/CHANGES
index fb22537..988f564 100644
--- a/CHANGES
+++ b/CHANGES
@@ -6,30 +6,62 @@ Release History
 
 **1.5** (NOT YET RELEASED)
 
+    *"20% more unicode than the other leading breakfast cereal"*
+
+    The main new feature in this release is that
+    Passlib now supports Python 3 (via the 2to3 tool).
+    Everything has been recoded to have better separation
+    between unicode and bytes, and to use unicode internally
+    where possible.
+    When run under Python 2, Passlib 1.5 attempts
+    to provide the same behavior as Passlib 1.4;
+    but when run under Python 3, most functions
+    will return unicode instead of ascii bytes.
+    Besides this major change, there have
+    been some other additions:
+
     Hashes
-        * added support for FSHP family of hashes
+
         * added support for Cryptacular's PBKDF2 format
+        * added support for the FSHP family of hashes
         * added support for using BCryptor as BCrypt backend
-    
+
     CryptContext
+
         * interpolation deprecation:
-        
-          CryptPolicy.from_file() / .from_string() now
-          use SafeConfigParser instead of ConfigParser.
+
+          :meth:`CryptPolicy.from_file` and :meth:`CryptPolicy.from_string`
+          now use SafeConfigParser instead of ConfigParser.
           This may cause some existing config files containing unescaped ``%``
-          to result in errors; passlib 1.5 will demote these to warnings,
+          to result in errors; Passlib 1.5 will demote these to warnings,
           but any extant config files should be updated,
-          as the errors will be fatal in passlib 1.6.
-    
-        * added encoding keyword to CryptContext's from_path, from_string,
-          and to_string methods.          
-    
+          as the errors will be fatal in Passlib 1.6.
+
+        * added encoding keyword to :class:`!CryptPolicy`'s
+          :meth:`!.from_path()`, :meth:`!.from_string`,
+          and :meth:`!.to_string` methods.
+
+        * both classes in :mod:`passlib.apache`
+          now support specifying an encoding for the username/realm.
+
     Documentation
-        * added quickstart guide to documentation
-        
+
+        * Password Hash API expanded to include explicit
+          :ref:`unicode vs bytes policy <hash-unicode-behavior>`.
+        * Added quickstart guide to documentation.
+        * Various minor improvements.
+
+    Utilities
+
+        * Added more handler utility functions to reduce code duplication.
+        * Expanded kdf helpers in :mod:`passlib.utils.pbkdf2`.
+        * Removed deprecated parts of :mod:`passlib.utils.handlers`.
+
     Other
-        * removed deprecated parts of :mod:`passlib.utils.handlers`.
-    
+
+        * Builtin tests now use :mod:`!unittest2` if available.
+        * Setup script no longer requires distribute or setuptools.
+
 **1.4** (2011-05-04)
 
     This release contains a large number of changes, both large and small.
diff --git a/README b/README
index 9849ce8..576828a 100644
--- a/README
+++ b/README
@@ -6,17 +6,18 @@ The PassLib Python Library
 
 Welcome
 =======
-PassLib is a password hash library, which provides cross-platform
-implementations of over 20 password hashing algorithms; as well as a framework for managing
-and migrating existing password hashes. It's designed to be useful
-for any task from quickly verifying a hash found in /etc/shadow,
+Passlib is a password hashing library for Python 2 & 3,
+which provides cross-platform implementations of over 20
+password hashing algorithms, as well as a framework for
+managing existing password hashes. It's designed to be useful
+for a wide range of tasks, from verifying a hash found in /etc/shadow,
 to providing full-strength password hashing for multi-user application.
 
 The latest documentation can be found online at `<http://packages.python.org/passlib>`_.
 
 Requirements
 ============
-* Python 2.5 - 2.7
+* Python 2.5 - 2.7 or Python 3
 * PyBCrypt or BCryptor (optional; required only if bcrypt support is needed)
 * M2Crypto (optional; accelerates PBKDF2-based hashes)
 
@@ -31,7 +32,7 @@ For more detailed installation & testing instructions, see "docs/install.rst"
 
 Online Resources
 ================
-* Homepage -   http://code.google.com/p/passlib
+* Homepage -   http://passlib.googlecode.com
 * Docs -       http://packages.python.org/passlib
 * Discussion - http://groups.google.com/group/passlib-users
 
diff --git a/docs/index.rst b/docs/index.rst
index 97ae621..bdb2ca5 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -4,28 +4,29 @@ PassLib |release| documentation
 
 Welcome
 =======
-Passlib is a password hashing library for Python, which provides cross-platform
-implementations of over 20 password hashing algorithms;
-as well as a framework for managing and migrating existing password hashes.
-It's designed to be useful for a large range of tasks:
+Passlib is a password hashing library for Python 2 & 3,
+which provides cross-platform implementations of over 20 password hashing algorithms,
+as well as a framework for managing existing password hashes.
+It's designed to be useful for a large range of tasks, including:
 
-* as a cross-platform replacement for stdlib's crypt ~
-  :data:`passlib.hosts.host_context`
+* quick-start password hashing for new python applications ~
+  :doc:`quickstart guide <new_app_quickstart>`
 
-* encrypting & verifying most hash formats used by:
+* constructing a configurable hashing policy
+  to match the needs of any python application ~
+  :data:`passlib.context`
 
-    - Linux & BSD shadow files ~ :mod:`passlib.hosts`
-    - Apache htpasswd & htdigest files ~ :mod:`passlib.apache`
-    - MySQL, PostgreSQL, and Oracle, OpenLDAP, and other applications ~
-      :mod:`passlib.apps`
+* reading & writing Apache htpasswd / htdigest files ~
+  :mod:`passlib.apache`
 
-* quick-start password hashing for new python applications ~
-  :doc:`new application quickstart <new_app_quickstart>`
+* creating & verifying hashes used by MySQL, PostgreSQL, OpenLDAP,
+  and other applications ~
+  :mod:`passlib.apps`
 
-* construct a configurable hashing policy
-  to match the needs of any python application ~ :data:`passlib.context`
+* creating & verifying hashes found in Unix "shadow" files ~
+  :data:`passlib.hosts`
 
-See the :doc:`library overview <overview>` for more details and usage examples.
+See the library overview for more details and usage examples.
 
 Quick Links
 ===========
@@ -41,34 +42,36 @@ Quick Links
             </p>
 
             <p class="biglink">
-                <a class="biglink" href="lib/passlib.hash.html"><i>passlib.hash</i> module</a><br>
-                <span class="linkdescr">complete list of supported password hash algorithms</span>
+                <a class="biglink" href="install.html">Installation</a><br>
+                <span class="linkdescr">requirements and installation instructions</span>
             </p>
         </td>
         <td width="50%" valign="top">
             <p class="biglink">
-                <a class="biglink" href="install.html">Installation</a><br>
-                <span class="linkdescr">requirements and installation instructions</span>
+                <a class="biglink" href="lib/passlib.hash.html">Supported Hashes</a><br>
+                <span class="linkdescr">complete list of supported password hash algorithms</span>
             </p>
 
             <p class="biglink">
                 <a class="biglink" href="history.html">Changelog</a><br>
                 <span class="linkdescr">history of current and past releases</span>
             </p>
+        </td>
     </tr>
     </table>
 
 Online Resources
 ================
 
-.. rst-class:: html-plain-table
-
-=============== ===================================================
-**Homepage**:   `<http://code.google.com/p/passlib>`_
-**Docs**:       `<http://packages.python.org/passlib>`_
-**Discussion**: `<http://groups.google.com/group/passlib-users>`_
-
-**PyPI**:       `<http://pypi.python.org/pypi/passlib>`_
-**Downloads**:  `<http://code.google.com/p/passlib/downloads>`_
-**Source**:     `<http://code.google.com/p/passlib/source>`_
-=============== ===================================================
+    .. rst-class:: html-plain-table
+
+    ================ ===================================================
+    **Homepage**:    `<http://passlib.googlecode.com>`_
+    **Online Docs**: `<http://packages.python.org/passlib>`_
+    **Discussion**:  `<http://groups.google.com/group/passlib-users>`_
+    ---------------- ---------------------------------------------------
+    ---------------- ---------------------------------------------------
+    **PyPI**:        `<http://pypi.python.org/pypi/passlib>`_
+    **Downloads**:   `<http://code.google.com/p/passlib/downloads>`_
+    **Source**:      `<http://code.google.com/p/passlib/source>`_
+    ================ ===================================================
diff --git a/docs/install.rst b/docs/install.rst
index bdd14e2..fe2c048 100644
--- a/docs/install.rst
+++ b/docs/install.rst
@@ -4,16 +4,16 @@ Installation
 
 Supported Platforms
 ===================
-Passlib requires Python 2.5 or newer.
+Passlib requires Python 2 (>= 2.5) or Python 3.
 It should work with the following Python implementations:
     
-* CPython 2 (2.5 or newer)
-* CPython 3 (all versions)
-* `PyPy <www.pypy.org>`_ 1.5 or newer.
-* `Jython <www.jython.org>`_ 2.5 or newer.
+* CPython 2 -- 2.5 or newer.
+* CPython 3 -- all versions.
+* PyPy -- 1.5 or newer.
+* Jython -- 2.5 or newer.
 
-Passlib should work with all operating systems.
-Passlib contains builtin fallbacks
+Passlib should work with all operating systems,
+as it contains builtin fallbacks
 for almost all OS-dependant features. 
 
 Optional Libraries
diff --git a/docs/lib/passlib.apps.rst b/docs/lib/passlib.apps.rst
index 29be43c..1273094 100644
--- a/docs/lib/passlib.apps.rst
+++ b/docs/lib/passlib.apps.rst
@@ -41,7 +41,7 @@ Custom Applications
 
 .. seealso::
 
-    The :doc:`/new_app_quickstart` guide.
+    The :doc:`/new_app_quickstart`.
 
 .. _ldap-contexts:
 
diff --git a/docs/lib/passlib.hash.oracle10.rst b/docs/lib/passlib.hash.oracle10.rst
index a276a0f..832082f 100644
--- a/docs/lib/passlib.hash.oracle10.rst
+++ b/docs/lib/passlib.hash.oracle10.rst
@@ -76,14 +76,14 @@ Oracle10 hashes can be formed by the following procedure:
 Security Issues
 ===============
 This algorithm it not suitable for *any* use besides manipulating existing
-Oracle10 account passwords, due to the following flaws [#flaws]:
+Oracle10 account passwords, due to the following flaws [#flaws]_:
 
 * It's use of the username as a salt value means that common usernames
   (eg ``system``) will occur more frequently as salts,
   weakening the effectiveness of the salt in foiling pre-computed tables.
 
 * The fact that is it case insensitive, and simply concatenates the username
-  and password, greatly reduces the requirements for brute-force
+  and password, greatly reduces the keyspace for brute-force
   or pre-computed attacks.
 
 * It's simplicity makes high-speed brute force attacks much more feasible.
diff --git a/docs/modular_crypt_format.rst b/docs/modular_crypt_format.rst
index 3db7c6a..b91ab51 100644
--- a/docs/modular_crypt_format.rst
+++ b/docs/modular_crypt_format.rst
@@ -125,19 +125,19 @@ Identifiers & Platform Support
 The following table lists of all the major MCF hashes supported by passlib,
 and indicates which operating systems offer native support. 
 
-==================================== ================== =========== =========== =========== =========== =======
-Scheme                               Prefix             Linux       FreeBSD     NetBSD      OpenBSD     Solaris
-==================================== ================== =========== =========== =========== =========== =======
-:class:`~passlib.hash.des_crypt`     n/a                y           y           y           y           y
-:class:`~passlib.hash.bsdi_crypt`    ``_``                          y           y
-:class:`~passlib.hash.md5_crypt`     ``$1$``            y           y           y           y           y
-:class:`~passlib.hash.sun_md5_crypt` ``$md5$``                                                          y
-:class:`~passlib.hash.bcrypt`        ``$2$``, ``$2a$``              y           y           y           y
-:class:`~passlib.hash.nthash`        ``$3$``                        y
-:class:`~passlib.hash.sha256_crypt`  ``$5$``            y                                               y
-:class:`~passlib.hash.sha512_crypt`  ``$6$``            y                                               y
-:class:`~passlib.hash.sha1_crypt`    ``$sha1$``                                 y 
-==================================== ================== =========== =========== =========== =========== =======
+==================================== ==================== =========== =========== =========== =========== =======
+Scheme                               Prefix               Linux       FreeBSD     NetBSD      OpenBSD     Solaris
+==================================== ==================== =========== =========== =========== =========== =======
+:class:`~passlib.hash.des_crypt`     n/a                  y           y           y           y           y
+:class:`~passlib.hash.bsdi_crypt`    ``_``                            y           y
+:class:`~passlib.hash.md5_crypt`     ``$1$``              y           y           y           y           y
+:class:`~passlib.hash.sun_md5_crypt` ``$md5$``, ``$md5,``                                                            y
+:class:`~passlib.hash.bcrypt`        ``$2$``, ``$2a$``                y           y           y           y
+:class:`~passlib.hash.nthash`        ``$3$``                          y
+:class:`~passlib.hash.sha256_crypt`  ``$5$``              y                                               y
+:class:`~passlib.hash.sha512_crypt`  ``$6$``              y                                               y
+:class:`~passlib.hash.sha1_crypt`    ``$sha1$``                                   y 
+==================================== ==================== =========== =========== =========== =========== =======
 
 The following table lists the other MCF hashes supported by passlib,
 most of which are only used by applications:
diff --git a/docs/new_app_quickstart.rst b/docs/new_app_quickstart.rst
index c7309da..b86042f 100644
--- a/docs/new_app_quickstart.rst
+++ b/docs/new_app_quickstart.rst
@@ -49,16 +49,19 @@ For new applications, there are really only three choices:
 :class:`~passlib.hash.sha512_crypt`, and
 :class:`~passlib.hash.pbkdf2_sha512` [#choices]_.
 
-These hashes all have the following properties:
+All of these hashes share the following properties:
 
     * no known vulnerabilties.
     * based on documented & widely reviewed algorithms.
-    * basic architecture has seen scrutiny and use for at least 10 years. 
+    * basic architecture has been under heavy scrutiny and use for at least 10 years. 
     * public-domain or BSD-licensed reference implementations available.
     * in use across a number of OSes and/or a wide variety of applications.
     * variable rounds for configuring flexible cpu cost on a per-hash basis.
     * at least 96 bits of salt.
 
+The following comparison should help you choose which hash is
+most appropriate for your application. 
+
 BCrypt
 ------
 :class:`~passlib.hash.bcrypt`
@@ -82,6 +85,7 @@ of simultaneous logon attempts (eg web apps).
     For BCrypt support on non-BSD systems,
     Passlib requires a C-extension module
     provided by the external pybcrypt or bcryptor packages.
+    Neither of these currently supports Python 3.
 
 SHA512-Crypt
 ------------
@@ -157,8 +161,8 @@ Insert the following code into your application::
     pwd_context = CryptContext(
         #replace this list with the hash(es) you wish to support.
         #this example sets pbkdf2_sha256 as the default,
-        #with support for sha256_crypt.
-        schemes=["pbkdf2_sha256", "sha256_crypt"],
+        #with support for legacy des_crypt hashes.
+        schemes=["pbkdf2_sha256", "des_crypt" ],
         default="pbkdf2_sha256",
 
         #vary rounds parameter randomly when creating new hashes...
@@ -168,7 +172,6 @@ Insert the following code into your application::
         #(appropriate values may vary for different schemes,
         # and the amount of time you wish it to take)
         pbkdf2_sha256__default_rounds = 8000,
-        sha256_crypt__default_rounds = 40000,
         )
 
 Using a CryptContext
diff --git a/docs/notes.txt b/docs/notes.txt
index 035e5e9..078843f 100644
--- a/docs/notes.txt
+++ b/docs/notes.txt
@@ -9,12 +9,6 @@ Internal Changes
 
 Other Hash Formats
 ------------------
-* FSHP (a pbkdf1 variant)
-  https://github.com/bdd/fshp
-
-* rfc2307 format hashes
-  http://tools.ietf.org/html/rfc2307
-
 * Mac OSX hash formats
 
 * SCrypt
@@ -24,17 +18,6 @@ Other Hash Formats
 Notes on various hash formats
 =============================
 
-LDAP
-----
-still trying to verify if {CRYPT} can contain other formats besides des-crypt
-(eg, md5-crypt etc). see some references to {CRYPT}$1$
-
-this page http://www.ldapexplorer.com/en/manual/107070207-editor-password.htm
-has a longer list than some, but no sample hashes, so can't verify formats.
-
-that page also implies some systems support variable length salts for SSHA etc.
-currently implementation is locked at 4 bytes.
-
 Cisco PIX
 ---------
 sample hashes found - http://www.freerainbowtables.com/phpBB3/viewtopic.php?f=2&t=1441
diff --git a/docs/overview.rst b/docs/overview.rst
index 4d83ad5..c9aad2c 100644
--- a/docs/overview.rst
+++ b/docs/overview.rst
@@ -2,15 +2,20 @@
 Library Overview
 ================
 
-PassLib is a collection of routines for managing password hashes
-as found in unix /etc/shadow files, as returned by stdlib `crypt()`,
+Passlib is a collection of routines for managing password hashes
+such as found in unix "shadow" files, as returned by stdlib's :func:`!crypt`,
 as stored in mysql and postgres, and various other places.
-PassLib's contents can be roughly grouped into three categories:
+Passlib's contents can be roughly grouped into three categories:
 password hashes, password contexts, and utility functions.
 
+.. note::
+
+    New applications which just need drop-in password hashing support
+    should see the :doc:`new_app_quickstart`.
+
 Password Hashes
 ===============
-All of the hash schemes supported by passlib are implemented
+All of the hash schemes supported by Passlib are implemented
 as classes importable from the :mod:`passlib.hash` module.
 All of these classes support a single uniform interface of standard class methods.
 These methods are documented in detail by the :ref:`password hash api <password-hash-api>`.
@@ -74,16 +79,9 @@ A quick example of how a password context can be used::
     >>> lc.verify('password', 'q1Oyx5r9mdGZ2')
     True
 
-.. note::
-
-    For new applications which just need drop-in support for some manner
-    of password encryption, so they can secure store passwords
-    and then forget about it, they should use
-    the :data:`passlib.apps.custom_app_context` object.
-
 Predefined Password Contexts
 ============================
-In addition to the :mod:`passlib.context` module,
+In addition to the :mod:`!passlib.context` module,
 PassLib provides a number of pre-configured :class:`!CryptContext` instances
 in order to get users started quickly:
 
diff --git a/docs/password_hash_api.rst b/docs/password_hash_api.rst
index 951d69d..fe3b499 100644
--- a/docs/password_hash_api.rst
+++ b/docs/password_hash_api.rst
@@ -235,7 +235,7 @@ which scheme a hash belongs to when multiple schemes are in use.
         In order to accomplish this, many implementations perform only minimal
         validation of the candidate hashes. Thus, they may return ``True``
         for hashes which are identifiable, but malformed enough that
-        a :exc:`ValueError` when the string  is passed to
+        a :exc:`ValueError` is raised when the string is passed to
         :func:`~PasswordHash.verify` or :func:`~PasswordHash.genhash`.
         Because of this, applications should rely on this method only for identification,
         not confirmation that a hash is correctly formed.
@@ -482,13 +482,15 @@ the following attributes are usually exposed.
     xxx: what about a bits_per_salt_char or some such, so effective salt strength
     can be compared?
 
+.. _hash-unicode-behavior:
+
 Unicode Behavior
 ================
 
-.. versionadded:: Passlib 1.5
+.. versionadded:: 1.5
 
 Quick summary
-------------
+-------------
 For the application developer in a hurry:
 
 * Passwords should be provided as :class:`unicode` if possible.
diff --git a/passlib/apache.py b/passlib/apache.py
index 411e346..55786e1 100644
--- a/passlib/apache.py
+++ b/passlib/apache.py
@@ -1,26 +1,4 @@
-"""passlib.apache - apache password support
-
-.. todo::
-
-    support htpasswd context
-
-    needs ldap_sha1 support
-    detect when crypt should be used, and what ones.
-
-.. todo::
-    support htdigest context
-
-.. todo::
-
-    support reading / writing htpasswd & htdigest files using this module.
-
-    references -
-        http://httpd.apache.org/docs/2.2/misc/password_encryptions.html
-        http://httpd.apache.org/docs/2.0/programs/htpasswd.html
-
-    NOTE: htdigest format is md5(user ":" realm ":" passwd).hexdigest()
-        file format is "user:realm:hash"
-"""
+"""passlib.apache - apache password support"""
 #=========================================================
 #imports
 #=========================================================
diff --git a/setup.py b/setup.py
index 4f39e81..b5b09c0 100644
--- a/setup.py
+++ b/setup.py
@@ -50,22 +50,28 @@ vh.close()
 SUMMARY = "comprehensive password hashing framework supporting over 20 schemes"
 
 DESCRIPTION = """\
-PassLib is a password hash library, which provides cross-platform
-implementations of over 20 password hashing algorithms; as well as a framework for managing
-and migrating existing password hashes. It's designed to be useful
-for any task from quickly verifying a hash found in /etc/shadow,
-to providing full-strength password hashing for multi-user applications.
+Passlib is a password hashing library for Python 2 & 3,
+which provides cross-platform implementations of over 20
+password hashing algorithms, as well as a framework for
+managing existing password hashes. It's designed to be useful
+for a wide range of tasks, from verifying a hash found in /etc/shadow,
+to providing full-strength password hashing for multi-user application.
 
-* See the `online documentation <http://packages.python.org/passlib>`_ for details and examples.
+* See the `online documentation <http://packages.python.org/passlib>`_
+  for details, installation instructions, and examples.
 
-* See the `passlib homepage <http://passlib.googlecode.com>`_ for the latest news, more information, and additional downloads. 
+* See the `passlib homepage <http://passlib.googlecode.com>`_
+  for the latest news, more information, and additional downloads. 
 
-* See the `changelog <http://packages.python.org/passlib/history.html>`_ for list of what's new in passlib.
+* See the `changelog <http://packages.python.org/passlib/history.html>`_
+  for description of what's new in Passlib.
 
-All releases are signed with the gpg key `4CE1ED31 <http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4D8592DF4CE1ED31>`_.
+All releases are signed with the gpg key
+`4CE1ED31 <http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4D8592DF4CE1ED31>`_.
 """
 
-KEYWORDS = "password secret hash security crypt md5-crypt sha256-crypt sha512-crypt bcrypt apache htpasswd htdigest pbkdf2 ntlm"
+KEYWORDS = "password secret hash security crypt md5-crypt \
+sha256-crypt sha512-crypt bcrypt apache htpasswd htdigest pbkdf2 ntlm"
 
 #=========================================================
 #run setup