Fixing HTTP range headers for negative length larger than content size.

According to the RFC, clients are allows to request
the "last N bytes", even if N is actually larger
than the content size.  Here's the relevant section:

    RFC 2616 Section 14.35.1:
    If the entity is shorter than the specified suffix-length,
    the entire entity-body is used.

httpd follows this just fine.  test.txt is only 11 bytes long,
but if we request 500 bytes, we do fine:

    $ curl http://www.sf.cloudera.com/~philip/test.txt  -r -500 -v
    > GET /~philip/test.txt HTTP/1.1
    > Range: bytes=-500
    > User-Agent: curl/7.30.0
    > Host: www.sf.cloudera.com
    > Accept: */*
    >
    < HTTP/1.1 206 Partial Content
    < Date: Fri, 25 Jul 2014 23:31:39 GMT
    < Server: Apache/2.2.15 (CentOS)
    < Last-Modified: Fri, 25 Jul 2014 23:14:01 GMT
    < ETag: "221772d-b-4ff0cba900ab6"
    < Accept-Ranges: bytes
    < Content-Length: 11
    < Content-Range: bytes 0-10/11
    < Connection: close
    < Content-Type: text/plain; charset=UTF-8
    <
    1234567890

However, cherrypy would fail in this case.

    $curl -v http://localhost:8080/test.txt -r -100
    > GET /test.txt HTTP/1.1
    > Range: bytes=-100
    > User-Agent: curl/7.30.0
    > Host: localhost:8080
    > Accept: */*
    >
    < HTTP/1.1 500 Internal Server Error
    < Content-Length: 1104
    < Server: CherryPy/3.5.1
    < Date: Mon, 28 Jul 2014 16:22:11 GMT
    < Content-Type: text/html;charset=utf-8
    <
    [...]
        <title>500 Internal Server Error</title>
    [...]
            <pre id="traceback">Traceback (most recent call last):
      File "/Users/philip/src/cherrypy/cherrypy/_cprequest.py", line 667, in respond
        self.hooks.run('before_handler')
      File "/Users/philip/src/cherrypy/cherrypy/_cprequest.py", line 114, in run
        raise exc
    IOError: [Errno 22] Invalid argument
    [...]

With this commit, cherrypy now works fine:

    $curl -v http://localhost:8080/test.txt -r -100
    > GET /test.txt HTTP/1.1
    > Range: bytes=-100
    > User-Agent: curl/7.30.0
    > Host: localhost:8080
    > Accept: */*
    >
    < HTTP/1.1 206 Partial Content
    < Content-Length: 11
    < Accept-Ranges: bytes
    < Server: CherryPy/3.5.1
    < Last-Modified: Mon, 28 Jul 2014 16:20:47 GMT
    < Content-Range: bytes 0-10/11
    < Date: Mon, 28 Jul 2014 16:25:46 GMT
    < Content-Type: text/plain
    <
    0123456789


The above is run with the below minimalist script.

    import cherrypy
    import os

    class X(object):
      pass

    if __name__ == '__main__':
        conf = {
            '/': {
                'tools.staticdir.on': True,
                'tools.staticdir.root': os.path.abspath(os.getcwd()),
                'tools.staticdir.dir': './'
            }
        }
        cherrypy.quickstart(X(), '/', conf)

I've added a simple test to check this case and have run it
with "nosetests -s test/test_core.py".

2 files changed, 14 insertions, 1 deletions

diff --git a/cherrypy/lib/httputil.py b/cherrypy/lib/httputil.py
index 3de801d0..69a18d45 100644
--- a/cherrypy/lib/httputil.py
+++ b/cherrypy/lib/httputil.py
@@ -103,7 +103,14 @@ def get_ranges(headervalue, content_length):
                 # See rfc quote above.
                 return None
             # Negative subscript (last N bytes)
-            result.append((content_length - int(stop), content_length))
+            #
+            # RFC 2616 Section 14.35.1:
+            #   If the entity is shorter than the specified suffix-length,
+            #   the entire entity-body is used.
+            if int(stop) > content_length:
+              result.append((0, content_length))
+            else:
+              result.append((content_length - int(stop), content_length))
 
     return result
 
diff --git a/cherrypy/test/test_core.py b/cherrypy/test/test_core.py
index 6966d325..ae4728de 100644
--- a/cherrypy/test/test_core.py
+++ b/cherrypy/test/test_core.py
@@ -496,6 +496,12 @@ class CoreRequestHandlingTest(helper.CPWebCase):
         self.getPage("/ranges/get_ranges?bytes=2-4,-1")
         self.assertBody("[(2, 5), (7, 8)]")
 
+        # Test a suffix-byte-range longer than the content
+        # length. Note that in this test, the content length
+        # is 8 bytes.
+        self.getPage("/ranges/get_ranges?bytes=-100")
+        self.assertBody("[(0, 8)]")
+
         # Get a partial file.
         if cherrypy.server.protocol_version == "HTTP/1.1":
             self.getPage("/ranges/slice_file", [('Range', 'bytes=2-5')])