replaced utils.h64 module with utils.Base64Engine instance

* new utils.Base64Engine() provides flexible class for encoding arbitrary base64 charmaps.
  it should also be a bit faster than the old h64 module.
* predefined 'h64' instance has mostly the same methods as the old h64 module
  which it takes the place off - so imports should be unaffected.
  (the only the exception of the xxx_dc_xxx methods, which now use the 'h64big' instance)
* replaced utils._blowfish base64 encoding with custom Base64Engine instance
  to reduce code duplication.
* more through unittests for Base64Engine.

10 files changed, 102 insertions, 36 deletions

diff --git a/docs/lib/passlib.hash.bigcrypt.rst b/docs/lib/passlib.hash.bigcrypt.rst
index 4106a71..e16c261 100644
--- a/docs/lib/passlib.hash.bigcrypt.rst
+++ b/docs/lib/passlib.hash.bigcrypt.rst
@@ -27,11 +27,11 @@ Format
 An example hash (of the string ``passphrase``) is ``S/8NbAAlzbYO66hAa9XZyWy2``.
 A bigcrypt hash string has the format :samp:`{salt}{checksum_1}{checksum_2...}{checksum_n}` for some integer :samp:`{n}>0`, where:
 
-* :samp:`{salt}` is the salt, stored as a 2 character :func:`hash64 <passlib.utils.h64.encode_int12>`-encoded
+* :samp:`{salt}` is the salt, stored as a 2 character :data:`hash64 <passlib.utils.h64>`-encoded
   12-bit integer (``S/`` in the example).
 
 * each :samp:`{checksum_i}` is a separate checksum, stored as an 11 character
-  :func:`hash64 <passlib.utils.h64.encode_dc_int64>`-encoded 64-bit integer (``8NbAAlzbYO6`` and ``6hAa9XZyWy2``
+  :data:`hash64-big <passlib.utils.h64big>`-encoded 64-bit integer (``8NbAAlzbYO6`` and ``6hAa9XZyWy2``
   in the example).
 
 * the integer :samp:`n` (the number of checksums) is determined by the formula
@@ -54,7 +54,7 @@ The bigcrypt algorithm is designed to re-use the original des-crypt algorithm:
 
 4. The 2 character salt string is decoded to a 12-bit integer salt value;
    The salt string uses little-endian
-   :func:`hash64 <passlib.utils.h64.decode_int12>` encoding.
+   :data:`hash64 <passlib.utils.h64>` encoding.
 
 5. 25 repeated rounds of modified DES encryption are performed;
    starting with a null input block,
@@ -68,7 +68,7 @@ The bigcrypt algorithm is designed to re-use the original des-crypt algorithm:
    lsb-padded with 2 zero bits.
 
 7. The resulting 66-bit integer is encoded in big-endian order
-   using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+   using the :data:`hash64-big <passlib.utils.h64big>` format.
    This forms the first checksum segment.
 
 8. For each additional block of 8 bytes in the padded password (from step 2),
diff --git a/docs/lib/passlib.hash.bsdi_crypt.rst b/docs/lib/passlib.hash.bsdi_crypt.rst
index 464cd67..6a60f79 100644
--- a/docs/lib/passlib.hash.bsdi_crypt.rst
+++ b/docs/lib/passlib.hash.bsdi_crypt.rst
@@ -44,7 +44,7 @@ A bsdi_crypt hash string consists of a 21 character string of the form :samp:`_{
 All characters except the underscore prefix are drawn from ``[./0-9A-Za-z]``.
 
 * ``_`` - the underscore is used to distinguish this scheme from others, such as des-crypt.
-* :samp:`{rounds}` is the number of rounds, stored as a 4 character :mod:`hash64 <passlib.utils.h64>`-encoded 24-bit integer (``EQ0.`` in the example).
+* :samp:`{rounds}` is the number of rounds, stored as a 4 character :data:`hash64 <passlib.utils.h64>`-encoded 24-bit integer (``EQ0.`` in the example).
 * :samp:`{salt}` is the salt, stored as as a 4 character hash64-encoded 24-bit integer (``jzhS`` in the example).
 * :samp:`{checksum}` is the checksum, stored as an 11 character hash64-encoded 64-bit integer (``VeUyoSqLupI`` in the example).
 
@@ -60,12 +60,12 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
 1. Given a password string, a salt string, and rounds string.
 
 2. The 4 character rounds string is decoded to a 24-bit integer rounds value;
-   The rounds string uses little-endian
-   :func:`hash64 <passlib.utils.h64.decode_int24>` encoding.
+   The rounds string uses little-endian :data:`hash64 <passlib.utils.h64>`
+   encoding.
 
 3. The 4 character salt string is decoded to a 24-bit integer salt value;
-   The salt string uses little-endian
-   :func:`hash64 <passlib.utils.h64.decode_int24>` encoding.
+   The salt string uses little-endian :data:`hash64 <passlib.utils.h64>`
+   encoding.
 
 4. The password is NULL-padded on the end to the smallest non-zero multiple of 8 bytes.
 
@@ -95,7 +95,7 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
    lsb-padded with 2 zero bits.
 
 9. The resulting 66-bit integer is encoded in big-endian order
-   using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+   using the :data:`hash64-big <passlib.utils.h64big>` format.
 
 Security Issues
 ===============
diff --git a/docs/lib/passlib.hash.crypt16.rst b/docs/lib/passlib.hash.crypt16.rst
index 9fd2a21..2cdd5eb 100644
--- a/docs/lib/passlib.hash.crypt16.rst
+++ b/docs/lib/passlib.hash.crypt16.rst
@@ -32,12 +32,13 @@ Format
 An example hash (of the string ``passphrase``) is ``aaX/UmCcBrceQ0kQGGWKTbuE``.
 A crypt16 hash string has the format :samp:`{salt}{checksum_1}{checksum_2}`, where:
 
-* :samp:`{salt}` is the salt, stored as a 2 character :func:`hash64 <passlib.utils.h64.encode_int12>`-encoded
-  12-bit integer (``aa`` in the example).
+* :samp:`{salt}` is the salt, stored as a 2 character
+  :data:`hash64 <passlib.utils.h64>`-encoded 12-bit integer (``aa`` in the
+  example).
 
 * each :samp:`{checksum_i}` is a separate checksum, stored as an 11 character
-  :func:`hash64 <passlib.utils.h64.encode_dc_int64>`-encoded 64-bit integer (``X/UmCcBrceQ`` and ``0kQGGWKTbuE``
-  in the example).
+  :data:`hash64-big <passlib.utils.h64big>`-encoded 64-bit integer
+  (``X/UmCcBrceQ`` and ``0kQGGWKTbuE`` in the example).
 
 .. rst-class:: html-toggle
 
@@ -48,8 +49,8 @@ The crypt16 algorithm uses a weakened version of the des-crypt algorithm:
 1. Given a password string and a salt string.
 
 2. The 2 character salt string is decoded to a 12-bit integer salt value;
-   The salt string uses little-endian
-   :func:`hash64 <passlib.utils.h64.decode_int12>` encoding.
+   The salt string uses little-endian :data:`hash64 <passlib.utils.h64>`
+   encoding.
 
 3. If the password is larger than 16 bytes, the end is truncated to 16 bytes.
    If the password is smaller than 16 bytes, the end is NULL padded to 16 bytes.
@@ -63,16 +64,16 @@ The crypt16 algorithm uses a weakened version of the des-crypt algorithm:
    starting with a null input block,
    and using the 56-bit integer from step 4 as the DES key.
 
-   The salt value from step 2 is used to to mutate the normal 
-   DES encrypt operation by swapping bits :samp:`{i}` and :samp:`{i}+24` 
-   in the DES E-Box output if and only if bit :samp:`{i}` is set in 
+   The salt value from step 2 is used to to mutate the normal
+   DES encrypt operation by swapping bits :samp:`{i}` and :samp:`{i}+24`
+   in the DES E-Box output if and only if bit :samp:`{i}` is set in
    the salt value.
 
 6. The 64-bit result of the last round of step 5 is then
    lsb-padded with 2 zero bits.
 
 7. The resulting 66-bit integer is encoded in big-endian order
-   using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+   using the :data:`hash64-big <passlib.utils.h64big>` format.
    This is the first checksum segment.
 
 8. The second checksum segment is created by repeating
diff --git a/docs/lib/passlib.hash.des_crypt.rst b/docs/lib/passlib.hash.des_crypt.rst
index 06d4f85..ba5202e 100644
--- a/docs/lib/passlib.hash.des_crypt.rst
+++ b/docs/lib/passlib.hash.des_crypt.rst
@@ -39,7 +39,7 @@ Interface
 Format
 ======
 A des-crypt hash string consists of 13 characters, drawn from ``[./0-9A-Za-z]``.
-The first 2 characters form a :mod:`hash64 <passlib.utils.h64>`-encoded
+The first 2 characters form a :data:`hash64 <passlib.utils.h64>`-encoded
 12 bit integer used as the salt, with the remaining characters
 forming a hash64-encoded 64-bit integer checksum.
 
@@ -57,8 +57,8 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
 1. Given a password string and a salt string.
 
 2. The 2 character salt string is decoded to a 12-bit integer salt value;
-   The salt string uses little-endian
-   :func:`hash64 <passlib.utils.h64.decode_int12>` encoding.
+   The salt string uses little-endian :data:`hash64 <passlib.utils.h64>`
+   encoding.
 
 3. If the password is less than 8 bytes, it's NULL padded at the end to 8 bytes.
 
@@ -83,8 +83,8 @@ The checksum is formed by a modified version of the DES cipher in encrypt mode:
 6. The 64-bit result of the last round of step 5 is then
    lsb-padded with 2 zero bits.
 
-7. The resulting 66-bit integer is encoded in big-endian order
-   using the :func:`hash 64 <passlib.utils.h64.encode_int>` format.
+7. The resulting 66-bit integer is encoded in big-endian order using the
+   :data:`hash64-big <passlib.utils.h64big>` format.
 
 Security Issues
 ===============
@@ -115,7 +115,7 @@ This implementation of des-crypt differs from others in a few ways:
 * Restricted salt string character set:
 
   The underlying algorithm expects salt strings to use the
-  :mod:`hash64 <passlib.utils.h64>` character set to encode
+  :data:`hash64 <passlib.utils.HASH64_CHARS>` character set to encode
   a 12-bit integer. Many implementations of des-crypt will
   accept a salt containing other characters, but
   vary wildly in how they are handled, including errors and implementation-specific value mappings.
diff --git a/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst b/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst
index 794e506..1dfd832 100644
--- a/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst
+++ b/docs/lib/passlib.hash.dlitz_pbkdf2_sha1.rst
@@ -51,7 +51,7 @@ where:
   (``.pPqsEwHD7MiECU0`` in the example).
 
 * :samp:`{checksum}` is 32 characters, which encode
-  the resulting 24-byte PBKDF2 derived key using :func:`~passlib.utils.adapted_b64_encode`
+  the resulting 24-byte PBKDF2 derived key using :func:`~passlib.utils.ab64_encode`
   (``b8TQ5AMQemtlaSgegw5Je.JBE3QQhLbO`` in the example).
 
 In order to generate the checksum, the password is first encoded into UTF-8 if it's unicode.
diff --git a/docs/lib/passlib.hash.md5_crypt.rst b/docs/lib/passlib.hash.md5_crypt.rst
index d033723..a75187f 100644
--- a/docs/lib/passlib.hash.md5_crypt.rst
+++ b/docs/lib/passlib.hash.md5_crypt.rst
@@ -81,7 +81,7 @@ The MD5-Crypt algorithm [#f1]_ calculates a checksum as follows:
 
 9. Add the password to digest A.
 
-10. Add the constant string ``$1$`` to digest A. 
+10. Add the constant string ``$1$`` to digest A.
     (The Apache variant of MD5-Crypt uses ``$apr1$`` instead,
     this is the only change made by this variant).
 
@@ -122,7 +122,7 @@ The MD5-Crypt algorithm [#f1]_ calculates a checksum as follows:
     following order: ``12,6,0,13,7,1,14,8,2,15,9,3,5,10,4,11``.
 
 18. Encode the resulting 16 byte string into a 22 character
-    :mod:`hash 64 <passlib.utils.h64.encode_bytes>`-encoded string
+    :data:`hash64 <passlib.utils.h64>`-encoded string
     (the 2 msb bits encoded by the last hash64 character are used as 0 padding).
     This results in the portion of the md5 crypt hash string referred to as :samp:`{checksum}` in the format section.
 
@@ -151,7 +151,7 @@ PassLib's implementation of md5-crypt differs from the reference implementation
   The underlying algorithm can unambigously handle salt strings
   which contain any possible byte value besides ``\x00`` and ``$``.
   However, PassLib strictly limits salts to the
-  :mod:`hash 64 <passlib.utils.h64>` character set,
+  :data:`hash64 <passlib.utils.HASH64_CHARS>` character set,
   as nearly all implementations of md5-crypt generate
   and expect salts containing those characters,
   but may have unexpected behaviors for other character values.
diff --git a/docs/lib/passlib.hash.pbkdf2_digest.rst b/docs/lib/passlib.hash.pbkdf2_digest.rst
index 6cd3ea6..e732c5d 100644
--- a/docs/lib/passlib.hash.pbkdf2_digest.rst
+++ b/docs/lib/passlib.hash.pbkdf2_digest.rst
@@ -89,10 +89,10 @@ follow the same format, :samp:`$pbkdf2-{digest}${rounds}${salt}${checksum}`.
   this is encoded as a positive decimal number with no zero-padding
   (``6400`` in the example).
 
-* :samp:`{salt}` - this is the :func:`adapted base64 encoding <passlib.utils.adapted_b64_encode>`
+* :samp:`{salt}` - this is the :func:`adapted base64 encoding <passlib.utils.ab64_encode>`
   of the raw salt bytes passed into the PBKDF2 function.
 
-* :samp:`{checksum}` - this is the :func:`adapted base64 encoding <passlib.utils.adapted_b64_encode>`
+* :samp:`{checksum}` - this is the :func:`adapted base64 encoding <passlib.utils.ab64_encode>`
   of the raw derived key bytes returned from the PBKDF2 function.
   Each scheme uses output size of it's specific :samp:`{digest}`
   as the size of the raw derived key. This is enlarged
@@ -104,7 +104,7 @@ The password is encoded into UTF-8 if not already encoded,
 and passed through :func:`~passlib.utils.pbkdf2.pbkdf2`
 along with the decoded salt, the number of rounds,
 and a prf built from HMAC + the respective message digest.
-The result is then encoded using :func:`~passlib.utils.adapted_b64_encode`.
+The result is then encoded using :func:`~passlib.utils.ab64_encode`.
 
 .. rubric:: Footnotes
 
diff --git a/docs/lib/passlib.hash.sha1_crypt.rst b/docs/lib/passlib.hash.sha1_crypt.rst
index cae9c42..fa80a6b 100644
--- a/docs/lib/passlib.hash.sha1_crypt.rst
+++ b/docs/lib/passlib.hash.sha1_crypt.rst
@@ -80,7 +80,7 @@ in a few ways:
   The underlying algorithm can unambigously handle salt strings
   which contain any possible byte value besides ``\x00`` and ``$``.
   However, PassLib strictly limits salts to the
-  :mod:`hash 64 <passlib.utils.h64>` character set,
+  :data:`hash64 <passlib.utils.HASH64_CHARS>` character set,
   as nearly all implementations of sha1-crypt generate
   and expect salts containing those characters.
 
diff --git a/docs/lib/passlib.hash.sha512_crypt.rst b/docs/lib/passlib.hash.sha512_crypt.rst
index d9aa1ea..f9e438c 100644
--- a/docs/lib/passlib.hash.sha512_crypt.rst
+++ b/docs/lib/passlib.hash.sha512_crypt.rst
@@ -91,7 +91,7 @@ and other implementations, in a few ways:
   The underlying algorithm can unambigously handle salt strings
   which contain any possible byte value besides ``\x00`` and ``$``.
   However, PassLib strictly limits salts to the
-  :mod:`hash 64 <passlib.utils.h64>` character set,
+  :data:`hash64 <passlib.utils.HASH64_CHARS>` character set,
   as nearly all implementations of sha512-crypt generate
   and expect salts containing those characters,
   but may have unexpected behaviors for other character values.
diff --git a/docs/lib/passlib.utils.rst b/docs/lib/passlib.utils.rst
index 580c830..b052f5a 100644
--- a/docs/lib/passlib.utils.rst
+++ b/docs/lib/passlib.utils.rst
@@ -50,6 +50,72 @@ Bytes Manipulation
 .. autofunction:: xor_bytes
 .. autofunction:: consteq
 
+Base64 Encoding
+===============
+
+Base64Engine Class
+------------------
+Passlib has to deal with a number of different Base64 encodings,
+with varying endianness, as well as wildly different value <-> character
+mappings. This is all encapsulated in the :class:`Base64Engine` class,
+which provides common encoding actions for an arbitrary base64-style encoding
+scheme. There are also a couple of predefined instances which are commonly
+used by the hashes in Passlib.
+
+.. autoclass:: Base64Engine
+
+Common Character Maps
+---------------------
+.. data:: BASE64_CHARS
+
+    Character map used by standard MIME-compatible Base64 encoding scheme.
+
+.. data:: HASH64_CHARS
+
+    Base64 character map used by a number of hash formats;
+    the ordering is wildly different from the standard base64 character map.
+    (see :data:`h64` for details).
+
+.. data:: BCRYPT_CHARS
+
+    Base64 character map used by :class:`~passlib.hash.bcrypt`.
+    The ordering is wildly different from both the standard base64 character map,
+    and the common hash64 character map.
+
+Predefined Instances
+--------------------
+.. data:: h64
+    Predefined instance of :class:`Base64Engine` which uses
+    the :data:`HASH64_CHARS` character map and little-endian encoding.
+
+    This encoding system appears to have originated with
+    :class:`~passlib.hash.des_crypt`, but is used by
+    :class:`~passlib.hash.md5-crypt`, `~passlib.hash.sha256_crypt`,
+    and others. Within Passlib, this encoding is referred as ``hash64`` encoding
+    to distinguish it from normal base64 and other encodings.
+
+.. data:: h64big
+    Predefined variant of :data:`h64` which uses big-endian encoding.
+    This is mainly used by :class:`~passlib.hash.des_crypt`.
+
+.. note::
+
+   *changed in Passlib 1.6:* the :mod:`passlib.utils.h64` module used by
+   Passlib <= 1.5 has been replaced by the the ``h64`` and ``h64big``
+   instances; but the interface remains mostly unchanged.
+
+..
+
+    .. data:: AB64_CHARS
+
+        Variant of standard Base64 character map used by some
+        custom Passlib hashes (see :func:`ab64_encode`).
+
+    Other
+    -----
+    .. autofunction:: ab64_encode
+    .. autofunction:: ab64_decode
+
 Randomness
 ==========
 .. data:: rng
@@ -83,7 +149,6 @@ There are also a few sub modules which provide additional utility functions:
     :maxdepth: 1
 
     passlib.utils.des
-    passlib.utils.h64
     passlib.utils.md4
     passlib.utils.pbkdf2
     passlib.utils.handlers