Much more documentation

8 files changed, 115 insertions, 19 deletions

diff --git a/doc/cli.rst b/doc/cli.rst
index 911c446..5bd123d 100644
--- a/doc/cli.rst
+++ b/doc/cli.rst
@@ -1,5 +1,43 @@
 Commandline interface
 ==================================================
 
-.. todo::
-    write me!
+A lot of the Python-RSA functionality is also available as commandline
+scripts. On Linux and other unix-like systems they are executable
+Python scripts, on Windows they are .exe files.
+
+All scripts accept a ``--help`` parameter that give you instructions
+on how to use them. Here is a short overview:
+
+.. index:: CLI interface
+.. index:: pyrsa-keygen, pyrsa-encrypt, pyrsa-decrypt, pyrsa-sign
+.. index:: pyrsa-verify, pyrsa-priv2pub, pyrsa-encrypt-bigfile
+.. index:: pyrsa-decrypt-bigfile, pyrsa-decrypt-bigfile
+
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| Command               | Usage                                            | Core function                           |
++=======================+==================================================+=========================================+
+| pyrsa-keygen          | Generates a new RSA keypair in PEM or DER format | :py:func:`rsa.newkeys`                  |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| pyrsa-encrypt         | Encrypts a file. The file must be shorter than   | :py:func:`rsa.encrypt`                  |
+|                       | the key length in order to be encrypted.         |                                         |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| pyrsa-decrypt         | Decrypts a file.                                 | :py:func:`rsa.decrypt`                  |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| pyrsa-sign            | Signs a file, outputs the signature.             | :py:func:`rsa.sign`                     |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| pyrsa-verify          | Verifies a signature. The result is written to   | :py:func:`rsa.verify`                   |
+|                       | the console as well as returned in the exit      |                                         |
+|                       | status code.                                     |                                         |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| pyrsa-priv2pub        | Reads a private key and outputs the              |                                         |
+|                       | corresponding public key.                        |                                         |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| pyrsa-encrypt-bigfile | Encrypts a file to an encrypted VARBLOCK file.   | :py:func:`rsa.bigfile.encrypt_bigfile`  |
+|                       | The file can be larger than the key length, but  |                                         |
+|                       | the output file is only compatible with          |                                         |
+|                       | Python-RSA.                                      |                                         |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+| pyrsa-decrypt-bigfile | Decrypts an encrypted VARBLOCK file.             | :py:func:`rsa.bigfile.encrypt_bigfile`  |
++-----------------------+--------------------------------------------------+-----------------------------------------+
+
+
diff --git a/doc/compatibility.rst b/doc/compatibility.rst
index 0cb2ba6..c9dac39 100644
--- a/doc/compatibility.rst
+++ b/doc/compatibility.rst
@@ -1,6 +1,9 @@
 Compatibility with standards and other software
 ==================================================
 
+.. index:: OpenSSL
+.. index:: compatibility
+
 Python-RSA implements encryption and signatures according to PKCS#1
 version 1.5. This makes it compatible with the OpenSSL RSA module.
 
@@ -21,6 +24,8 @@ public keys, which are not supported.
 :Public keys:
     PKCS#1 v1.5 in PEM and DER format, ASN.1 type RSAPublicKey
 
+:VARBLOCK encryption:
+    Python-RSA only, not compatible with any other known application.
 
 
 Public keys from OpenSSL
diff --git a/doc/conf.py b/doc/conf.py
index ad49886..6493ad7 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -217,3 +217,6 @@ man_pages = [
     ('index', 'python-rsa', u'Python-RSA Documentation',
      [u'Sybren A. Stüvel'], 1)
 ]
+
+todo_include_todos = True
+
diff --git a/doc/index.rst b/doc/index.rst
index 5486224..9c84331 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -18,10 +18,13 @@ This RSA implementation has seen the eyes of a security expert, and it
 uses an industry standard random padding method. However, there are
 still possible vectors of attack. Just to name one example, it doesn't
 compress the input stream to remove repetitions, and if you display
-the stack trace of a ``Decryptionerror`` exception you'll leak
-information about the reason why decryption failed. And I'm sure that
-those aren't the only insecurities. Use your own judgement to decide
-whether this module is secure enough for your application.
+the stack trace of a :py:class:`rsa.pkcs1.CryptoError` exception
+you'll leak information about the reason why decryption or
+verification failed.
+
+I'm sure that those aren't the only insecurities. Use your own
+judgement to decide whether this module is secure enough for your
+application.
 
 If you have the time and skill to improve the implementation, by all
 means be my guest. The best way is to clone the Mercurial repository
@@ -43,10 +46,4 @@ Contents
     reference
 
 
-Indices and tables
-==================
-
 * :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
-
diff --git a/doc/installation.rst b/doc/installation.rst
index 47f137f..89d0979 100644
--- a/doc/installation.rst
+++ b/doc/installation.rst
@@ -20,9 +20,12 @@ or if that doesn't work::
     sudo python setup.py install
 
 
-.. todo::
+The sources are tracked in our `Mercurial repository`_ at
+bitbucket.org. It also hosts the `issue tracker`_.
 
-    Add a source link here
+.. _`Mercurial repository`: https://bitbucket.org/sybren/python-rsa
+.. _`issue tracker`:
+    https://bitbucket.org/sybren/python-rsa/issues?status=new&status=open
 
 
 Dependencies
@@ -33,6 +36,7 @@ you only need Python itself. Loading and saving keys does require an
 extra module, though: pyasn1. If you used pip or easy_install like
 described above, you should be ready to go.
 
+
 Development dependencies
 --------------------------------------------------
 
@@ -46,7 +50,7 @@ pip or easy_install to install the following packages:
 
 Once these are installed, use Mercurial_ to get a copy of the source::
 
-    hg clone http://hg.stuvel.eu/python-rsa
+    hg clone https://sybren@bitbucket.org/sybren/python-rsa
     sudo python setup.py develop
 
 
diff --git a/doc/intro.rst b/doc/intro.rst
index cc89a19..14d0177 100644
--- a/doc/intro.rst
+++ b/doc/intro.rst
@@ -1,4 +1,4 @@
-Introduction
+Introduction & history
 ==================================================
 
 Python-RSA's history starts in 2006. As a student assignment for the
@@ -6,7 +6,8 @@ University of Amsterdam we wrote a RSA implementation. We chose Python
 for various reasons; one of the most important reasons was the
 `unlimited precision integer`_ support.
 
-.. _`unlimited precision integer`: http://docs.python.org/library/stdtypes.html#numeric-types-int-float-long-complex
+.. _`unlimited precision integer`:
+    http://docs.python.org/library/stdtypes.html#numeric-types-int-float-long-complex
 
 It started out as just a module for calculating large primes, and RSA
 encryption, decryption, signing and verification using those large
@@ -14,6 +15,24 @@ numbers. It also included generating public and private keys. There
 was no functionality for working with byte sequences (such as files)
 yet.
 
-.. todo:: write more history
+Version 1.0 did include support for byte sequences, but quite clunky,
+mostly because it didn't support 0-bytes and thus was unsuitable for
+binary messages.
+
+Version 2.0 introduced a lot of improvements by Barry Mead, but still
+wasn't compatible with other RSA implementations and used no random
+padding.
+
+Version 3.0 introduced PKCS#1 v1.5 functionality, which resulted in
+compatibility with OpenSSL and many others implementing the same
+standard. Random padding was introduced that considerably increased
+security, which also resulted in the ability to encrypt and decrypt
+binary messages.
+
+Key generation was also improved in version 3.0, ensuring that you
+really get the number of bits you asked for. At the same time key
+generation speed was greatly improved. The ability to save and load
+public and private keys in PEM and DER format as also added.
+
 
 
diff --git a/doc/reference.rst b/doc/reference.rst
index c667840..a1a7d35 100644
--- a/doc/reference.rst
+++ b/doc/reference.rst
@@ -14,6 +14,11 @@ Functions
 
 .. autofunction:: rsa.newkeys(keysize)
 
+.. autofunction:: rsa.bigfile.encrypt_bigfile
+
+.. autofunction:: rsa.bigfile.decrypt_bigfile
+
+
 Classes
 --------------------------------------------------
 
@@ -35,4 +40,28 @@ Exceptions
 .. autoclass:: rsa.pkcs1.VerificationError(CryptoError)
 
 
+.. index:: VARBLOCK (file format)
+
+The VARBLOCK file format
+--------------------------------------------------
+
+The VARBLOCK file format allows us to encrypt files that are larger
+than the RSA key. The format is as follows; || denotes byte string
+concatenation::
+
+ VARBLOCK := VERSION || BLOCK || BLOCK || ...
+
+ VERSION := 1
+
+ BLOCK := LENGTH || DATA
+
+ LENGTH := varint-encoded length of the followng data, in bytes
+
+ DATA := the data to store in the block
+
+The varint-format was taken from Google's Protobuf_, and allows us to
+efficiently encode an arbitrarily long integer.
+
+.. _Protobuf:
+    http://code.google.com/apis/protocolbuffers/docs/encoding.html#varints
 
diff --git a/doc/usage.rst b/doc/usage.rst
index 68bffca..09954e2 100644
--- a/doc/usage.rst
+++ b/doc/usage.rst
@@ -216,7 +216,7 @@ though.
 Encrypting works as follows: the input file is split into blocks that
 are just large enough to encrypt with your RSA key. Every block is
 then encrypted using RSA, and the encrypted blocks are assembled into
-the output file.
+the output file. This file format is called the VARBLOCK format.
 
 Decrypting works in reverse. The encrypted file is separated into
 encrypted blocks. Those are decrypted, and assembled into the original
@@ -253,3 +253,4 @@ As does decryption:
     :py:func:`rsa.sign` and :py:func:`rsa.verify` work on arbitrarily
     long files, so they do not have a "bigfile" equivalent.
 
+