summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--README.rst4
-rw-r--r--doc/manpages/swift.133
-rw-r--r--doc/source/cli.rst37
-rw-r--r--doc/source/client-api.rst24
-rw-r--r--swiftclient/service.py119
-rwxr-xr-xswiftclient/shell.py70
-rw-r--r--swiftclient/utils.py73
-rw-r--r--tests/unit/test_service.py153
-rw-r--r--tests/unit/test_shell.py192
-rw-r--r--tests/unit/test_utils.py87
10 files changed, 590 insertions, 202 deletions
diff --git a/README.rst b/README.rst
index 688acff..42ec277 100644
--- a/README.rst
+++ b/README.rst
@@ -2,8 +2,8 @@
Team and repository tags
========================
-.. image:: http://governance.openstack.org/badges/python-swiftclient.svg
- :target: http://governance.openstack.org/reference/tags/index.html
+.. image:: https://governance.openstack.org/badges/python-swiftclient.svg
+ :target: https://governance.openstack.org/reference/tags/index.html
.. Change things from this point on
diff --git a/doc/manpages/swift.1 b/doc/manpages/swift.1
index b65170c..1f288d6 100644
--- a/doc/manpages/swift.1
+++ b/doc/manpages/swift.1
@@ -130,14 +130,33 @@ programs, such as jq.
capabilities \-\-json
.RE
-\fBtempurl\fR [\fIcommand-option\fR] \fImethod\fR \fIseconds\fR \fIpath\fR \fIkey\fR
+\fBtempurl\fR [\fIcommand-option\fR] \fImethod\fR \fItime\fR \fIpath\fR \fIkey\fR
.RS 4
Generates a temporary URL allowing unauthenticated access to the Swift object
-at the given path, using the given HTTP method, for the given number of
-seconds, using the given TempURL key. With the optional \-\-prefix\-based option a
-prefix-based URL is generated. If optional \-\-absolute argument is
-provided, seconds is instead interpreted as a Unix timestamp at which the URL
-should expire. \fBExample\fR: tempurl GET $(date \-d "Jan 1 2016" +%s)
+at the given path, using the given HTTP method, for the given time,
+using the given TempURL key.
+
+The time can be specified either as an integer
+denoting the amount of seconds the temporary URL is valid, or as an ISO 8601
+timestamp in one of following formats: Complete date: YYYY\-MM\-DD (eg 1997\-07\-16),
+complete date plus hours, minutes and seconds: YYYY\-MM\-DDThh:mm:ss
+(eg 1997\-07\-16T19:20:30) or complete date plus hours, minutes and seconds with
+UTC designator: YYYY\-MM\-DDThh:mm:ssZ (eg 1997\-07\-16T19:20:30Z). Be aware that
+if you do not use the latter format, the timestamp is generated using your locale
+timezone. If the first format is used, the time part used will equal to 00:00:00.
+
+With the \-\-prefix\-based option a
+prefix-based URL is generated.
+
+The option \-\-iso8601 provides ISO 8601 UTC timestamps
+instead of Unix timestamps inside the generated URL.
+
+If optional \-\-absolute argument is
+provided and the time argument is specified in seconds, the seconds are
+interpreted as a Unix timestamp at which the URL
+should expire.
+
+\fBExample\fR: tempurl GET $(date \-d "Jan 1 2016" +%s)
/v1/AUTH_foo/bar_container/quux.md my_secret_tempurl_key \-\-absolute
.RE
@@ -186,4 +205,4 @@ swift \-A https://127.0.0.1:443/auth/v1.0 \-U swiftops:swiftops \-K swiftops sta
.SH DOCUMENTATION
.LP
More in depth documentation about OpenStack Swift as a whole can be found at
-.BI http://swift.openstack.org
+.BI https://docs.openstack.org/developer/swift
diff --git a/doc/source/cli.rst b/doc/source/cli.rst
index 1f76b05..8d80d1b 100644
--- a/doc/source/cli.rst
+++ b/doc/source/cli.rst
@@ -85,7 +85,7 @@ below:
.. code-block:: bash
- swift -A https://auth.api.rackspacecloud.com/v1.0 -U user -K api_key list
+ swift -A https://api.example.com/v1.0 -U user -K api_key list
Specifying the options above manually on the command line can be avoided by
setting the following environment variables:
@@ -93,7 +93,7 @@ setting the following environment variables:
.. code-block:: bash
ST_AUTH_VERSION=1.0
- ST_AUTH=https://auth.api.rackspacecloud.com/v1.0
+ ST_AUTH=https://api.example.com/v1.0
ST_USER=user
ST_KEY=key
@@ -228,18 +228,39 @@ Capabilities
Tempurl
-------
- ``tempurl [command-options] [method] [seconds] [path] [key]``
+ ``tempurl [command-options] [method] [time] [path] [key]``
Generates a temporary URL for a Swift object. ``method`` option sets an HTTP method to
- allow for this temporary URL that is usually 'GET' or 'PUT'. ``seconds`` option sets
- the amount of time in seconds the temporary URL will be valid for; or, if ``--absolute``
- is passed, the Unix timestamp when the temporary URL will expire. ``path`` option sets
- the full path to the Swift object. Example: ``/v1/AUTH_account/c/o``. ``key`` option is
+ allow for this temporary URL that is usually ``GET` or ``PUT``. ``time`` option sets
+ the amount of time the temporary URL will be valid for.
+ ``time`` can be specified as an integer, denoting the number of seconds
+ from now on until the URL shall be valid; or, if ``--absolute``
+ is passed, the Unix timestamp when the temporary URL will expire.
+ But beyond that, ``time`` can also be specified as an ISO 8601 timestamp
+ in one of following formats:
+
+ i) Complete date: YYYY-MM-DD (eg 1997-07-16)
+
+ ii) Complete date plus hours, minutes and seconds:
+ YYYY-MM-DDThh:mm:ss
+ (eg 1997-07-16T19:20:30)
+
+ iii) Complete date plus hours, minutes and seconds with UTC designator:
+ YYYY-MM-DDThh:mm:ssZ
+ (eg 1997-07-16T19:20:30Z)
+
+ Please be aware that if you don't provide the UTC designator (i.e., Z)
+ the timestamp is generated using your local timezone. If only a date is
+ specified, the time part used will equal to ``00:00:00``.
+
+ ``path`` option sets the full path to the Swift object.
+ Example: ``/v1/AUTH_account/c/o``. ``key`` option is
the secret temporary URL key set on the Swift cluster. To set a key, run
``swift post -m "Temp-URL-Key: <your secret key>"``. To generate a prefix-based temporary
URL use the ``--prefix-based`` option. This URL will contain the path to the prefix. Do not
forget to append the desired objectname at the end of the path portion (and before the
- query portion) before sharing the URL.
+ query portion) before sharing the URL. It is possible to use ISO 8601 UTC timestamps within the
+ URL by using the ``--iso8601`` option.
Auth
----
diff --git a/doc/source/client-api.rst b/doc/source/client-api.rst
index 57f49cb..b993642 100644
--- a/doc/source/client-api.rst
+++ b/doc/source/client-api.rst
@@ -35,10 +35,10 @@ Keystone Session
project_domain_name='Default')
# Create session
- session = session.Session(auth=auth)
+ keystone_session = session.Session(auth=auth)
# Create swiftclient Connection
- swift_conn = Connection(session=session)
+ swift_conn = Connection(session=keystone_session)
Keystone v3
~~~~~~~~~~~
@@ -63,26 +63,6 @@ Keystone v3
auth_version=_auth_version
)
-.. code-block:: python
-
- _authurl = 'http://127.0.0.1:5000/v3/'
- _auth_version = '3'
- _user = 'tester'
- _key = 'testing'
- _os_options = {
- 'user_domain_id': 'Default',
- 'project_domain_id': 'Default',
- 'project_id': 'Default'
- }
-
- conn = Connection(
- authurl=_authurl,
- user=_user,
- key=_key,
- os_options=_os_options,
- auth_version=_auth_version
- )
-
Keystone v2
~~~~~~~~~~~
diff --git a/swiftclient/service.py b/swiftclient/service.py
index 8c6880a..223641b 100644
--- a/swiftclient/service.py
+++ b/swiftclient/service.py
@@ -43,7 +43,8 @@ from swiftclient.command_helpers import (
)
from swiftclient.utils import (
config_true_value, ReadableToIterable, LengthWrapper, EMPTY_ETAG,
- parse_api_response, report_traceback, n_groups, split_request_headers
+ parse_api_response, report_traceback, n_groups, split_request_headers,
+ n_at_a_time
)
from swiftclient.exceptions import ClientException
from swiftclient.multithreading import MultiThreadingManager
@@ -473,7 +474,7 @@ class SwiftService(object):
or container, or an iterator for returning the results of the
stat operations on a list of objects.
- :raises: SwiftError
+ :raises SwiftError:
"""
if options is not None:
options = dict(self._options, **options)
@@ -637,7 +638,7 @@ class SwiftService(object):
container/account, or an iterator for returning the results
of posts to a list of objects.
- :raises: SwiftError
+ :raises SwiftError:
"""
if options is not None:
options = dict(self._options, **options)
@@ -1031,8 +1032,8 @@ class SwiftService(object):
'download_object' dictionary containing the results of an
individual file download.
- :raises: ClientException
- :raises: SwiftError
+ :raises ClientException:
+ :raises SwiftError:
"""
if options is not None:
options = dict(self._options, **options)
@@ -1396,8 +1397,8 @@ class SwiftService(object):
:returns: A generator for returning the results of the uploads.
- :raises: SwiftError
- :raises: ClientException
+ :raises SwiftError:
+ :raises ClientException:
"""
if options is not None:
options = dict(self._options, **options)
@@ -1714,6 +1715,7 @@ class SwiftService(object):
segment_name),
'log_line': '%s segment %s' % (obj_name, segment_index),
}
+ fp = None
try:
fp = open(path, 'rb')
fp.seek(segment_start)
@@ -1761,6 +1763,9 @@ class SwiftService(object):
if results_queue is not None:
results_queue.put(res)
return res
+ finally:
+ if fp is not None:
+ fp.close()
def _get_chunk_data(self, conn, container, obj, headers, manifest=None):
chunks = []
@@ -2008,29 +2013,36 @@ class SwiftService(object):
else:
res['large_object'] = False
obr = {}
- if path is not None:
- content_length = getsize(path)
- contents = LengthWrapper(open(path, 'rb'),
- content_length,
- md5=options['checksum'])
- else:
- content_length = None
- contents = ReadableToIterable(stream,
- md5=options['checksum'])
-
- etag = conn.put_object(
- container, obj, contents,
- content_length=content_length, headers=put_headers,
- response_dict=obr
- )
- res['response_dict'] = obr
-
- if (options['checksum'] and
- etag and etag != contents.get_md5sum()):
- raise SwiftError('Object upload verification failed: '
- 'md5 mismatch, local {0} != remote {1} '
- '(remote object has not been removed)'
- .format(contents.get_md5sum(), etag))
+ fp = None
+ try:
+ if path is not None:
+ content_length = getsize(path)
+ fp = open(path, 'rb')
+ contents = LengthWrapper(fp,
+ content_length,
+ md5=options['checksum'])
+ else:
+ content_length = None
+ contents = ReadableToIterable(stream,
+ md5=options['checksum'])
+
+ etag = conn.put_object(
+ container, obj, contents,
+ content_length=content_length, headers=put_headers,
+ response_dict=obr
+ )
+ res['response_dict'] = obr
+
+ if (options['checksum'] and
+ etag and etag != contents.get_md5sum()):
+ raise SwiftError(
+ 'Object upload verification failed: '
+ 'md5 mismatch, local {0} != remote {1} '
+ '(remote object has not been removed)'
+ .format(contents.get_md5sum(), etag))
+ finally:
+ if fp is not None:
+ fp.close()
if old_manifest or old_slo_manifest_paths:
drs = []
@@ -2124,8 +2136,8 @@ class SwiftService(object):
'bulk_delete' dictionary containing the results of an
individual delete operation.
- :raises: ClientException
- :raises: SwiftError
+ :raises ClientException:
+ :raises SwiftError:
"""
if options is not None:
options = dict(self._options, **options)
@@ -2140,11 +2152,15 @@ class SwiftService(object):
rq = Queue()
obj_dels = {}
- if self._should_bulk_delete(objects):
- for obj_slice in n_groups(
- objects, self._options['object_dd_threads']):
- self._bulk_delete(container, obj_slice, options,
- obj_dels)
+ bulk_page_size = self._bulk_delete_page_size(objects)
+ if bulk_page_size > 1:
+ page_at_a_time = n_at_a_time(objects, bulk_page_size)
+ for page_slice in page_at_a_time:
+ for obj_slice in n_groups(
+ page_slice,
+ self._options['object_dd_threads']):
+ self._bulk_delete(container, obj_slice, options,
+ obj_dels)
else:
self._per_item_delete(container, objects, options,
obj_dels, rq)
@@ -2197,23 +2213,36 @@ class SwiftService(object):
and not res['success']):
cancelled = True
- def _should_bulk_delete(self, objects):
- if len(objects) < 2 * self._options['object_dd_threads']:
+ def _bulk_delete_page_size(self, objects):
+ '''
+ Given the iterable 'objects', will return how many items should be
+ deleted at a time.
+
+ :param objects: An iterable that supports 'len()'
+ :returns: The bulk delete page size (i.e. the max number of
+ objects that can be bulk deleted at once, as reported by
+ the cluster). If bulk delete is disabled, return 1
+ '''
+ if len(objects) <= 2 * self._options['object_dd_threads']:
# Not many objects; may as well delete one-by-one
- return False
+ return 1
try:
cap_result = self.capabilities()
if not cap_result['success']:
# This shouldn't actually happen, but just in case we start
# being more nuanced about our capabilities result...
- return False
+ return 1
except ClientException:
# Old swift, presumably; assume no bulk middleware
- return False
+ return 1
swift_info = cap_result['capabilities']
- return 'bulk_delete' in swift_info
+ if 'bulk_delete' in swift_info:
+ return swift_info['bulk_delete'].get(
+ 'max_deletes_per_request', 10000)
+ else:
+ return 1
def _per_item_delete(self, container, objects, options, rdict, rq):
for obj in objects:
@@ -2493,7 +2522,7 @@ class SwiftService(object):
:returns: A generator returning the results of copying the given list
of objects.
- :raises: SwiftError
+ :raises SwiftError:
"""
if options is not None:
options = dict(self._options, **options)
@@ -2635,7 +2664,7 @@ class SwiftService(object):
:returns: A dictionary containing the capabilities of the cluster.
- :raises: ClientException
+ :raises ClientException:
"""
if not refresh_cache and url in self.capabilities_cache:
return self.capabilities_cache[url]
diff --git a/swiftclient/shell.py b/swiftclient/shell.py
index ceca592..841ed6e 100755
--- a/swiftclient/shell.py
+++ b/swiftclient/shell.py
@@ -1224,8 +1224,8 @@ def st_auth(parser, args, thread_manager):
print('export OS_AUTH_TOKEN=%s' % sh_quote(token))
-st_tempurl_options = '''[--absolute] [--prefix-based]
- <method> <seconds> <path> <key>'''
+st_tempurl_options = '''[--absolute] [--prefix-based] [--iso8601]
+ <method> <time> <path> <key>'''
st_tempurl_help = '''
@@ -1234,9 +1234,35 @@ Generates a temporary URL for a Swift object.
Positional arguments:
<method> An HTTP method to allow for this temporary URL.
Usually 'GET' or 'PUT'.
- <seconds> The amount of time in seconds the temporary URL will be
- valid for; or, if --absolute is passed, the Unix
- timestamp when the temporary URL will expire.
+ <time> The amount of time the temporary URL will be
+ valid. The time can be specified in two ways:
+ an integer representing the time in seconds or an
+ ISO 8601 timestamp in a specific format.
+ If --absolute is passed and time
+ is an integer, the seconds are intepreted as the Unix
+ timestamp when the temporary URL will expire. The ISO
+ 8601 timestamp can be specified in one of following
+ formats:
+
+ i) Complete date: YYYY-MM-DD (eg 1997-07-16)
+
+ ii) Complete date plus hours, minutes and seconds:
+
+ YYYY-MM-DDThh:mm:ss
+
+ (eg 1997-07-16T19:20:30)
+
+ iii) Complete date plus hours, minutes and seconds with
+ UTC designator:
+
+ YYYY-MM-DDThh:mm:ssZ
+
+ (eg 1997-07-16T19:20:30Z)
+
+ Please be aware that if you don't provide the UTC
+ designator (i.e., Z) the timestamp is generated using
+ your local timezone. If only a date is specified,
+ the time part used will equal to 00:00:00.
<path> The full path or storage URL to the Swift object.
Example: /v1/AUTH_account/c/o
or: http://saio:8080/v1/AUTH_account/c/o
@@ -1245,10 +1271,14 @@ Positional arguments:
"Temp-URL-Key:b3968d0207b54ece87cccc06515a89d4"\'
Optional arguments:
- --absolute Interpret the <seconds> positional argument as a Unix
+ --absolute Interpret the <time> positional argument as a Unix
timestamp rather than a number of seconds in the
- future.
- --prefix-based If present, a prefix-based tempURL will be generated.
+ future. If an ISO 8601 timestamp is passed for <time>,
+ this argument is ignored.
+ --prefix-based If present, a prefix-based temporary URL will be
+ generated.
+ --iso8601 If present, the generated temporary URL will contain an
+ ISO 8601 UTC timestamp instead of a Unix timestamp.
'''.strip('\n')
@@ -1256,14 +1286,21 @@ def st_tempurl(parser, args, thread_manager):
parser.add_argument(
'--absolute', action='store_true',
dest='absolute_expiry', default=False,
- help=("If present, seconds argument will be interpreted as a Unix "
- "timestamp representing when the tempURL should expire, rather "
- "than an offset from the current time"),
+ help=("If present, and time argument is an integer, "
+ "time argument will be interpreted as a Unix "
+ "timestamp representing when the temporary URL should expire, "
+ "rather than an offset from the current time."),
)
parser.add_argument(
'--prefix-based', action='store_true',
default=False,
- help=("If present, a prefix-based tempURL will be generated."),
+ help=("If present, a prefix-based temporary URL will be generated."),
+ )
+ parser.add_argument(
+ '--iso8601', action='store_true',
+ default=False,
+ help=("If present, the temporary URL will contain an ISO 8601 UTC "
+ "timestamp instead of a Unix timestamp."),
)
(options, args) = parse_args(parser, args)
@@ -1272,7 +1309,7 @@ def st_tempurl(parser, args, thread_manager):
thread_manager.error('Usage: %s tempurl %s\n%s', BASENAME,
st_tempurl_options, st_tempurl_help)
return
- method, seconds, path, key = args[:4]
+ method, timestamp, path, key = args[:4]
parsed = urlparse(path)
@@ -1281,9 +1318,10 @@ def st_tempurl(parser, args, thread_manager):
'tempurl specified, possibly an error' %
method.upper())
try:
- path = generate_temp_url(parsed.path, seconds, key, method,
+ path = generate_temp_url(parsed.path, timestamp, key, method,
absolute=options['absolute_expiry'],
- prefix=options['prefix_based'],)
+ iso8601=options['iso8601'],
+ prefix=options['prefix_based'])
except ValueError as err:
thread_manager.error(err)
return
@@ -1442,7 +1480,7 @@ Positional arguments:
Examples:
%(prog)s download --help
- %(prog)s -A https://auth.api.rackspacecloud.com/v1.0 \\
+ %(prog)s -A https://api.example.com/v1.0 \\
-U user -K api_key stat -v
%(prog)s --os-auth-url https://api.example.com/v2.0 \\
diff --git a/swiftclient/utils.py b/swiftclient/utils.py
index 47856c2..8afcde9 100644
--- a/swiftclient/utils.py
+++ b/swiftclient/utils.py
@@ -13,6 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
"""Miscellaneous utility functions for use with Swift."""
+from calendar import timegm
import collections
import gzip
import hashlib
@@ -25,6 +26,10 @@ import traceback
TRUE_VALUES = set(('true', '1', 'yes', 'on', 't', 'y'))
EMPTY_ETAG = 'd41d8cd98f00b204e9800998ecf8427e'
+EXPIRES_ISO8601_FORMAT = '%Y-%m-%dT%H:%M:%SZ'
+SHORT_EXPIRES_ISO8601_FORMAT = '%Y-%m-%d'
+TIME_ERRMSG = ('time must either be a whole number or in specific '
+ 'ISO 8601 format.')
def config_true_value(value):
@@ -64,39 +69,65 @@ def prt_bytes(num_bytes, human_flag):
def generate_temp_url(path, seconds, key, method, absolute=False,
- prefix=False):
+ prefix=False, iso8601=False):
"""Generates a temporary URL that gives unauthenticated access to the
Swift object.
:param path: The full path to the Swift object or prefix if
a prefix-based temporary URL should be generated. Example:
/v1/AUTH_account/c/o or /v1/AUTH_account/c/prefix.
- :param seconds: If absolute is False then this specifies the amount of time
- in seconds for which the temporary URL will be valid. If absolute is
- True then this specifies an absolute time at which the temporary URL
- will expire.
+ :param seconds: time in seconds or ISO 8601 timestamp.
+ If absolute is False and this is the string representation of an
+ integer, then this specifies the amount of time in seconds for which
+ the temporary URL will be valid.
+ If absolute is True then this specifies an absolute time at which the
+ temporary URL will expire.
:param key: The secret temporary URL key set on the Swift
cluster. To set a key, run 'swift post -m
"Temp-URL-Key: <substitute tempurl key here>"'
:param method: A HTTP method, typically either GET or PUT, to allow
for this temporary URL.
- :param absolute: if True then the seconds parameter is interpreted as an
- absolute Unix time, otherwise seconds is interpreted as a relative time
- offset from current time.
+ :param absolute: if True then the seconds parameter is interpreted as a
+ Unix timestamp, if seconds represents an integer.
:param prefix: if True then a prefix-based temporary URL will be generated.
- :raises: ValueError if seconds is not a whole number or path is not to
- an object.
+ :param iso8601: if True, a URL containing an ISO 8601 UTC timestamp
+ instead of a UNIX timestamp will be created.
+ :raises ValueError: if timestamp or path is not in valid format.
:return: the path portion of a temporary URL
"""
try:
- seconds = float(seconds)
- if not seconds.is_integer():
- raise ValueError()
- seconds = int(seconds)
- if seconds < 0:
- raise ValueError()
+ try:
+ timestamp = float(seconds)
+ except ValueError:
+ formats = (
+ EXPIRES_ISO8601_FORMAT,
+ EXPIRES_ISO8601_FORMAT[:-1],
+ SHORT_EXPIRES_ISO8601_FORMAT)
+ for f in formats:
+ try:
+ t = time.strptime(seconds, f)
+ except ValueError:
+ t = None
+ else:
+ if f == EXPIRES_ISO8601_FORMAT:
+ timestamp = timegm(t)
+ else:
+ # Use local time if UTC designator is missing.
+ timestamp = int(time.mktime(t))
+
+ absolute = True
+ break
+
+ if t is None:
+ raise ValueError()
+ else:
+ if not timestamp.is_integer():
+ raise ValueError()
+ timestamp = int(timestamp)
+ if timestamp < 0:
+ raise ValueError()
except ValueError:
- raise ValueError('seconds must be a whole number')
+ raise ValueError(TIME_ERRMSG)
if isinstance(path, six.binary_type):
try:
@@ -121,9 +152,9 @@ def generate_temp_url(path, seconds, key, method, absolute=False,
'possibly an error', method.upper())
if not absolute:
- expiration = int(time.time() + seconds)
+ expiration = int(time.time() + timestamp)
else:
- expiration = seconds
+ expiration = timestamp
hmac_body = u'\n'.join([method.upper(), str(expiration),
('prefix:' if prefix else '') + path_for_body])
@@ -132,6 +163,10 @@ def generate_temp_url(path, seconds, key, method, absolute=False,
key = key.encode('utf-8')
sig = hmac.new(key, hmac_body.encode('utf-8'), hashlib.sha1).hexdigest()
+ if iso8601:
+ expiration = time.strftime(
+ EXPIRES_ISO8601_FORMAT, time.gmtime(expiration))
+
temp_url = u'{path}?temp_url_sig={sig}&temp_url_expires={exp}'.format(
path=path_for_body, sig=sig, exp=expiration)
if prefix:
diff --git a/tests/unit/test_service.py b/tests/unit/test_service.py
index 2fc827c..260f1cb 100644
--- a/tests/unit/test_service.py
+++ b/tests/unit/test_service.py
@@ -14,6 +14,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import unicode_literals
+import contextlib
import mock
import os
import six
@@ -473,6 +474,40 @@ class TestServiceDelete(_TestServiceBase):
self.assertLessEqual(r['error_timestamp'], after)
self.assertIn('Traceback', r['traceback'])
+ @mock.patch.object(swiftclient.service.SwiftService, 'capabilities',
+ lambda *a: {'action': 'capabilities',
+ 'timestamp': time.time(),
+ 'success': True,
+ 'capabilities': {
+ 'bulk_delete':
+ {'max_deletes_per_request': 10}}
+ })
+ def test_bulk_delete_page_size(self):
+ # make a list of 100 objects
+ obj_list = ['x%02d' % i for i in range(100)]
+ errors = []
+
+ # _bulk_delete_page_size uses 2x the number of threads to determine
+ # if if there are "many" object to delete or not
+
+ # format is: [(thread_count, expected result), ...]
+ obj_threads_exp = [
+ (10, 10), # something small
+ (49, 10), # just under the bounds
+ (50, 1), # cutover point
+ (51, 1), # just over bounds
+ (100, 1), # something big
+ ]
+ for thread_count, exp in obj_threads_exp:
+ s = SwiftService(options={'object_dd_threads': thread_count})
+ res = s._bulk_delete_page_size(obj_list)
+ if res != exp:
+ msg = 'failed for thread_count %d: got %r expected %r' % \
+ (thread_count, res, exp)
+ errors.append(msg)
+ if errors:
+ self.fail('_bulk_delete_page_size() failed\n' + '\n'.join(errors))
+
class TestSwiftError(unittest.TestCase):
@@ -1002,7 +1037,6 @@ class TestService(unittest.TestCase):
@mock.patch('swiftclient.service.stat')
@mock.patch('swiftclient.service.getmtime', return_value=1.0)
@mock.patch('swiftclient.service.getsize', return_value=4)
- @mock.patch.object(builtins, 'open', return_value=six.StringIO('asdf'))
def test_upload_with_relative_path(self, *args, **kwargs):
service = SwiftService({})
objects = [{'path': "./test",
@@ -1012,7 +1046,9 @@ class TestService(unittest.TestCase):
{'path': ".\\test",
'strt_indx': 2}]
for obj in objects:
- with mock.patch('swiftclient.service.Connection') as mock_conn:
+ with mock.patch('swiftclient.service.Connection') as mock_conn, \
+ mock.patch.object(builtins, 'open') as mock_open:
+ mock_open.return_value = six.StringIO('asdf')
mock_conn.return_value.head_object.side_effect = \
ClientException('Not Found', http_status=404)
mock_conn.return_value.put_object.return_value =\
@@ -1032,10 +1068,29 @@ class TestService(unittest.TestCase):
self.assertEqual(upload_obj_resp['object'],
obj['path'][obj['strt_indx']:])
self.assertEqual(upload_obj_resp['path'], obj['path'])
+ self.assertTrue(mock_open.return_value.closed)
class TestServiceUpload(_TestServiceBase):
+ @contextlib.contextmanager
+ def assert_open_results_are_closed(self):
+ opened_files = []
+ builtin_open = builtins.open
+
+ def open_wrapper(*a, **kw):
+ opened_files.append((builtin_open(*a, **kw), a, kw))
+ return opened_files[-1][0]
+
+ with mock.patch.object(builtins, 'open', open_wrapper):
+ yield
+ for fp, args, kwargs in opened_files:
+ formatted_args = [repr(a) for a in args]
+ formatted_args.extend('%s=%r' % kv for kv in kwargs.items())
+ formatted_args = ', '.join(formatted_args)
+ self.assertTrue(fp.closed,
+ 'Failed to close open(%s)' % formatted_args)
+
def test_upload_object_job_file_with_unicode_path(self):
# Uploading a file results in the file object being wrapped in a
# LengthWrapper. This test sets the options in such a way that much
@@ -1109,11 +1164,14 @@ class TestServiceUpload(_TestServiceBase):
f.write(b'c' * 10)
f.flush()
- # Mock the connection to return an empty etag. This
- # skips etag validation which would fail as the LengthWrapper
- # isn't read from.
+ # run read() when put_object is called to calculate md5sum
+ def _consuming_conn(*a, **kw):
+ contents = a[2]
+ contents.read() # Force md5 calculation
+ return contents.get_md5sum()
+
mock_conn = mock.Mock()
- mock_conn.put_object.return_value = ''
+ mock_conn.put_object.side_effect = _consuming_conn
type(mock_conn).attempts = mock.PropertyMock(return_value=2)
expected_r = {
'action': 'upload_segment',
@@ -1125,21 +1183,22 @@ class TestServiceUpload(_TestServiceBase):
'log_line': 'test_o segment 2',
'success': True,
'response_dict': {},
- 'segment_etag': '',
+ 'segment_etag': md5(b'b' * 10).hexdigest(),
'attempts': 2,
}
s = SwiftService()
- r = s._upload_segment_job(conn=mock_conn,
- path=f.name,
- container='test_c',
- segment_name='test_s_1',
- segment_start=10,
- segment_size=10,
- segment_index=2,
- obj_name='test_o',
- options={'segment_container': None,
- 'checksum': True})
+ with self.assert_open_results_are_closed():
+ r = s._upload_segment_job(conn=mock_conn,
+ path=f.name,
+ container='test_c',
+ segment_name='test_s_1',
+ segment_start=10,
+ segment_size=10,
+ segment_index=2,
+ obj_name='test_o',
+ options={'segment_container': None,
+ 'checksum': True})
self.assertEqual(r, expected_r)
@@ -1153,10 +1212,6 @@ class TestServiceUpload(_TestServiceBase):
contents = mock_conn.put_object.call_args[0][2]
self.assertIsInstance(contents, utils.LengthWrapper)
self.assertEqual(len(contents), 10)
- # This read forces the LengthWrapper to calculate the md5
- # for the read content.
- self.assertEqual(contents.read(), b'b' * 10)
- self.assertEqual(contents.get_md5sum(), md5(b'b' * 10).hexdigest())
def test_etag_mismatch_with_ignore_checksum(self):
def _consuming_conn(*a, **kw):
@@ -1215,16 +1270,17 @@ class TestServiceUpload(_TestServiceBase):
type(mock_conn).attempts = mock.PropertyMock(return_value=2)
s = SwiftService()
- r = s._upload_segment_job(conn=mock_conn,
- path=f.name,
- container='test_c',
- segment_name='test_s_1',
- segment_start=10,
- segment_size=10,
- segment_index=2,
- obj_name='test_o',
- options={'segment_container': None,
- 'checksum': True})
+ with self.assert_open_results_are_closed():
+ r = s._upload_segment_job(conn=mock_conn,
+ path=f.name,
+ container='test_c',
+ segment_name='test_s_1',
+ segment_start=10,
+ segment_size=10,
+ segment_index=2,
+ obj_name='test_o',
+ options={'segment_container': None,
+ 'checksum': True})
self.assertIn('md5 mismatch', str(r.get('error')))
@@ -1259,21 +1315,29 @@ class TestServiceUpload(_TestServiceBase):
}
expected_mtime = '%f' % os.path.getmtime(f.name)
+ # run read() when put_object is called to calculate md5sum
+ # md5sum is verified in _upload_object_job.
+ def _consuming_conn(*a, **kw):
+ contents = a[2]
+ contents.read() # Force md5 calculation
+ return contents.get_md5sum()
+
mock_conn = mock.Mock()
- mock_conn.put_object.return_value = ''
+ mock_conn.put_object.side_effect = _consuming_conn
type(mock_conn).attempts = mock.PropertyMock(return_value=2)
s = SwiftService()
- r = s._upload_object_job(conn=mock_conn,
- container='test_c',
- source=f.name,
- obj='test_o',
- options={'changed': False,
- 'skip_identical': False,
- 'leave_segments': True,
- 'header': '',
- 'segment_size': 0,
- 'checksum': True})
+ with self.assert_open_results_are_closed():
+ r = s._upload_object_job(conn=mock_conn,
+ container='test_c',
+ source=f.name,
+ obj='test_o',
+ options={'changed': False,
+ 'skip_identical': False,
+ 'leave_segments': True,
+ 'header': '',
+ 'segment_size': 0,
+ 'checksum': True})
mtime = r['headers']['x-object-meta-mtime']
self.assertEqual(expected_mtime, mtime)
@@ -1292,11 +1356,6 @@ class TestServiceUpload(_TestServiceBase):
contents = mock_conn.put_object.call_args[0][2]
self.assertIsInstance(contents, utils.LengthWrapper)
self.assertEqual(len(contents), 30)
- # This read forces the LengthWrapper to calculate the md5
- # for the read content. This also checks that LengthWrapper was
- # initialized with md5=True
- self.assertEqual(contents.read(), b'a' * 30)
- self.assertEqual(contents.get_md5sum(), md5(b'a' * 30).hexdigest())
@mock.patch('swiftclient.service.time', return_value=1400000000)
def test_upload_object_job_stream(self, time_mock):
diff --git a/tests/unit/test_shell.py b/tests/unit/test_shell.py
index f7e9c1a..fba9be7 100644
--- a/tests/unit/test_shell.py
+++ b/tests/unit/test_shell.py
@@ -23,6 +23,7 @@ import os
import tempfile
import unittest
import textwrap
+from time import localtime, mktime, strftime, strptime
from requests.packages.urllib3.exceptions import InsecureRequestWarning
import six
@@ -36,7 +37,9 @@ from os.path import basename, dirname
from .utils import (
CaptureOutput, fake_get_auth_keystone, _make_fake_import_keystone_client,
FakeKeystone, StubResponse, MockHttpTest)
-from swiftclient.utils import EMPTY_ETAG
+from swiftclient.utils import (
+ EMPTY_ETAG, EXPIRES_ISO8601_FORMAT,
+ SHORT_EXPIRES_ISO8601_FORMAT, TIME_ERRMSG)
if six.PY2:
@@ -903,8 +906,8 @@ class TestShell(unittest.TestCase):
'x-object-meta-mtime': mock.ANY},
response_dict={})
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: False)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 0)
@mock.patch('swiftclient.service.Connection')
def test_delete_bad_threads(self, mock_connection):
mock_connection.return_value.get_container.return_value = (None, [])
@@ -934,8 +937,8 @@ class TestShell(unittest.TestCase):
check_good(["--object-threads", "1"])
check_good(["--container-threads", "1"])
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: False)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 1)
@mock.patch('swiftclient.service.Connection')
def test_delete_account(self, connection):
connection.return_value.get_account.side_effect = [
@@ -971,8 +974,8 @@ class TestShell(unittest.TestCase):
mock.call('container2', response_dict={}, headers={}),
mock.call('empty_container', response_dict={}, headers={})])
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: True)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 10)
@mock.patch('swiftclient.service.Connection')
def test_delete_bulk_account(self, connection):
connection.return_value.get_account.side_effect = [
@@ -1085,8 +1088,80 @@ class TestShell(unittest.TestCase):
self.assertEqual(connection.return_value.get_capabilities.mock_calls,
[mock.call(None)]) # only one /info request
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: False)
+ @mock.patch('swiftclient.service.Connection')
+ def test_delete_bulk_account_with_capabilities_and_pages(self, connection):
+ connection.return_value.get_capabilities.return_value = {
+ 'bulk_delete': {
+ 'max_deletes_per_request': 2,
+ 'max_failed_deletes': 1000,
+ },
+ }
+ connection.return_value.get_account.side_effect = [
+ [None, [{'name': 'container'}]],
+ [None, [{'name': 'container2'}]],
+ [None, [{'name': 'empty_container'}]],
+ [None, []],
+ ]
+ connection.return_value.get_container.side_effect = [
+ [None, [{'name': 'object'}, {'name': 'obj\xe9ct2'},
+ {'name': 'z_object'}, {'name': 'z_obj\xe9ct2'}]],
+ [None, []],
+ [None, [{'name': 'object'}, {'name': 'obj\xe9ct2'},
+ {'name': 'z_object'}, {'name': 'z_obj\xe9ct2'}]],
+ [None, []],
+ [None, []],
+ ]
+ connection.return_value.attempts = 0
+ argv = ["", "delete", "--all", "--object-threads", "1"]
+ connection.return_value.post_account.return_value = {}, (
+ b'{"Number Not Found": 0, "Response Status": "200 OK", '
+ b'"Errors": [], "Number Deleted": 1, "Response Body": ""}')
+ swiftclient.shell.main(argv)
+ # check that each bulk call was only called with 2 objects
+ self.assertEqual(
+ connection.return_value.post_account.mock_calls, [
+ mock.call(query_string='bulk-delete',
+ data=b''.join([
+ b'/container/object\n',
+ b'/container/obj%C3%A9ct2\n',
+ ]),
+ headers={'Content-Type': 'text/plain',
+ 'Accept': 'application/json'},
+ response_dict={}),
+ mock.call(query_string='bulk-delete',
+ data=b''.join([
+ b'/container/z_object\n',
+ b'/container/z_obj%C3%A9ct2\n'
+ ]),
+ headers={'Content-Type': 'text/plain',
+ 'Accept': 'application/json'},
+ response_dict={}),
+ mock.call(query_string='bulk-delete',
+ data=b''.join([
+ b'/container2/object\n',
+ b'/container2/obj%C3%A9ct2\n',
+ ]),
+ headers={'Content-Type': 'text/plain',
+ 'Accept': 'application/json'},
+ response_dict={}),
+ mock.call(query_string='bulk-delete',
+ data=b''.join([
+ b'/container2/z_object\n',
+ b'/container2/z_obj%C3%A9ct2\n'
+ ]),
+ headers={'Content-Type': 'text/plain',
+ 'Accept': 'application/json'},
+ response_dict={})])
+ self.assertEqual(
+ connection.return_value.delete_container.mock_calls, [
+ mock.call('container', response_dict={}, headers={}),
+ mock.call('container2', response_dict={}, headers={}),
+ mock.call('empty_container', response_dict={}, headers={})])
+ self.assertEqual(connection.return_value.get_capabilities.mock_calls,
+ [mock.call(None)]) # only one /info request
+
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 1)
@mock.patch('swiftclient.service.Connection')
def test_delete_container(self, connection):
connection.return_value.get_container.side_effect = [
@@ -1103,8 +1178,8 @@ class TestShell(unittest.TestCase):
'container', 'object', query_string=None, response_dict={},
headers={})
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: False)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 1)
@mock.patch('swiftclient.service.Connection')
def test_delete_container_headers(self, connection):
connection.return_value.get_container.side_effect = [
@@ -1122,8 +1197,8 @@ class TestShell(unittest.TestCase):
'container', 'object', query_string=None, response_dict={},
headers={'Skip-Middleware': 'Test'})
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: True)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 10)
@mock.patch('swiftclient.service.Connection')
def test_delete_bulk_container(self, connection):
connection.return_value.get_container.side_effect = [
@@ -1176,8 +1251,8 @@ class TestShell(unittest.TestCase):
self.assertTrue(out.out.find(
't\u00e9st_c [after 2 attempts]') >= 0, out)
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: False)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 1)
@mock.patch('swiftclient.service.Connection')
def test_delete_per_object(self, connection):
argv = ["", "delete", "container", "object"]
@@ -1188,8 +1263,8 @@ class TestShell(unittest.TestCase):
'container', 'object', query_string=None, response_dict={},
headers={})
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: True)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 10)
@mock.patch('swiftclient.service.Connection')
def test_delete_bulk_object(self, connection):
argv = ["", "delete", "container", "object"]
@@ -1550,7 +1625,7 @@ class TestShell(unittest.TestCase):
swiftclient.shell.main(argv)
temp_url.assert_called_with(
'/v1/AUTH_account/c/o', "60", 'secret_key', 'GET', absolute=False,
- prefix=False)
+ iso8601=False, prefix=False)
@mock.patch('swiftclient.shell.generate_temp_url', return_value='')
def test_temp_url_prefix_based(self, temp_url):
@@ -1559,7 +1634,28 @@ class TestShell(unittest.TestCase):
swiftclient.shell.main(argv)
temp_url.assert_called_with(
'/v1/AUTH_account/c/', "60", 'secret_key', 'GET', absolute=False,
- prefix=True)
+ iso8601=False, prefix=True)
+
+ @mock.patch('swiftclient.shell.generate_temp_url', return_value='')
+ def test_temp_url_iso8601_in(self, temp_url):
+ dates = ('1970-01-01T00:01:00Z', '1970-01-01T00:01:00',
+ '1970-01-01')
+ for d in dates:
+ argv = ["", "tempurl", "GET", d, "/v1/AUTH_account/c/",
+ "secret_key"]
+ swiftclient.shell.main(argv)
+ temp_url.assert_called_with(
+ '/v1/AUTH_account/c/', d, 'secret_key', 'GET', absolute=False,
+ iso8601=False, prefix=False)
+
+ @mock.patch('swiftclient.shell.generate_temp_url', return_value='')
+ def test_temp_url_iso8601_out(self, temp_url):
+ argv = ["", "tempurl", "GET", "60", "/v1/AUTH_account/c/",
+ "secret_key", "--iso8601"]
+ swiftclient.shell.main(argv)
+ temp_url.assert_called_with(
+ '/v1/AUTH_account/c/', "60", 'secret_key', 'GET', absolute=False,
+ iso8601=True, prefix=False)
@mock.patch('swiftclient.shell.generate_temp_url', return_value='')
def test_absolute_expiry_temp_url(self, temp_url):
@@ -1568,7 +1664,7 @@ class TestShell(unittest.TestCase):
swiftclient.shell.main(argv)
temp_url.assert_called_with(
'/v1/AUTH_account/c/o', "60", 'secret_key', 'GET', absolute=True,
- prefix=False)
+ iso8601=False, prefix=False)
def test_temp_url_output(self):
argv = ["", "tempurl", "GET", "60", "/v1/a/c/o",
@@ -1595,6 +1691,42 @@ class TestShell(unittest.TestCase):
"&temp_url_prefix=\n" % sig)
self.assertEqual(expected, output.out)
+ argv = ["", "tempurl", "GET", "60", "/v1/a/c/",
+ "secret_key", "--absolute", "--prefix", '--iso8601']
+ with CaptureOutput(suppress_systemexit=True) as output:
+ swiftclient.shell.main(argv)
+ sig = '00008c4be1573ba74fc2ab9bce02e3a93d04b349'
+ expires = '1970-01-01T00:01:00Z'
+ expected = ("/v1/a/c/?temp_url_sig=%s&temp_url_expires=%s"
+ "&temp_url_prefix=\n" % (sig, expires))
+ self.assertEqual(expected, output.out)
+
+ dates = ("1970-01-01T00:01:00Z",
+ strftime(EXPIRES_ISO8601_FORMAT[:-1], localtime(60)))
+ for d in dates:
+ argv = ["", "tempurl", "GET", d, "/v1/a/c/o",
+ "secret_key"]
+ with CaptureOutput(suppress_systemexit=True) as output:
+ swiftclient.shell.main(argv)
+ sig = "63bc77a473a1c2ce956548cacf916f292eb9eac3"
+ expected = "/v1/a/c/o?temp_url_sig=%s&temp_url_expires=60\n" % sig
+ self.assertEqual(expected, output.out)
+
+ ts = str(int(
+ mktime(strptime('2005-05-01', SHORT_EXPIRES_ISO8601_FORMAT))))
+
+ argv = ["", "tempurl", "GET", ts, "/v1/a/c/",
+ "secret_key", "--absolute"]
+ with CaptureOutput(suppress_systemexit=True) as output:
+ swiftclient.shell.main(argv)
+ expected = output.out
+
+ argv = ["", "tempurl", "GET", '2005-05-01', "/v1/a/c/",
+ "secret_key", "--absolute"]
+ with CaptureOutput(suppress_systemexit=True) as output:
+ swiftclient.shell.main(argv)
+ self.assertEqual(expected, output.out)
+
def test_temp_url_error_output(self):
expected = 'path must be full path to an object e.g. /v1/a/c/o\n'
for bad_path in ('/v1/a/c', 'v1/a/c/o', '/v1/a/c/', '/v1/a//o',
@@ -1614,7 +1746,17 @@ class TestShell(unittest.TestCase):
swiftclient.shell.main(argv)
self.assertEqual(expected, output.err,
'Expected %r but got %r for path %r' %
- (expected, output.err, bad_path))
+ (expected, output.err, '/v1/a/c'))
+
+ expected = TIME_ERRMSG + '\n'
+ for bad_time in ('not_an_int', '-1', '2015-05', '2015-05-01T01:00'):
+ argv = ["", "tempurl", "GET", bad_time, '/v1/a/c/o',
+ "secret_key", "--absolute"]
+ with CaptureOutput(suppress_systemexit=True) as output:
+ swiftclient.shell.main(argv)
+ self.assertEqual(expected, output.err,
+ 'Expected %r but got %r for time %r' %
+ (expected, output.err, bad_time))
@mock.patch('swiftclient.service.Connection')
def test_capabilities(self, connection):
@@ -2714,8 +2856,8 @@ class TestCrossAccountObjectAccess(TestBase, MockHttpTest):
return status
return on_request
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: False)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 1)
@mock.patch('swiftclient.service.Connection')
def test_upload_bad_threads(self, mock_connection):
mock_connection.return_value.put_object.return_value = EMPTY_ETAG
@@ -2897,8 +3039,8 @@ class TestCrossAccountObjectAccess(TestBase, MockHttpTest):
self.assertIn(expected_err, out.err)
self.assertEqual('', out)
- @mock.patch.object(swiftclient.service.SwiftService, '_should_bulk_delete',
- lambda *a: False)
+ @mock.patch.object(swiftclient.service.SwiftService,
+ '_bulk_delete_page_size', lambda *a: 1)
@mock.patch('swiftclient.service.Connection')
def test_download_bad_threads(self, mock_connection):
mock_connection.return_value.get_object.return_value = [{}, '']
diff --git a/tests/unit/test_utils.py b/tests/unit/test_utils.py
index c5961e8..adead00 100644
--- a/tests/unit/test_utils.py
+++ b/tests/unit/test_utils.py
@@ -18,6 +18,7 @@ import unittest
import mock
import six
import tempfile
+from time import gmtime, localtime, mktime, strftime, strptime
from hashlib import md5, sha1
from swiftclient import utils as u
@@ -151,6 +152,67 @@ class TestTempURL(unittest.TestCase):
self.assertIsInstance(url, type(self.url))
@mock.patch('hmac.HMAC')
+ def test_generate_temp_url_iso8601_argument(self, hmac_mock):
+ hmac_mock().hexdigest.return_value = 'temp_url_signature'
+ url = u.generate_temp_url(self.url, '2014-05-13T17:53:20Z',
+ self.key, self.method)
+ self.assertEqual(url, self.expected_url)
+
+ # Don't care about absolute arg.
+ url = u.generate_temp_url(self.url, '2014-05-13T17:53:20Z',
+ self.key, self.method, absolute=True)
+ self.assertEqual(url, self.expected_url)
+
+ lt = localtime()
+ expires = strftime(u.EXPIRES_ISO8601_FORMAT[:-1], lt)
+
+ if not isinstance(self.expected_url, six.string_types):
+ expected_url = self.expected_url.replace(
+ b'1400003600', bytes(str(int(mktime(lt))), encoding='ascii'))
+ else:
+ expected_url = self.expected_url.replace(
+ '1400003600', str(int(mktime(lt))))
+ url = u.generate_temp_url(self.url, expires,
+ self.key, self.method)
+ self.assertEqual(url, expected_url)
+
+ expires = strftime(u.SHORT_EXPIRES_ISO8601_FORMAT, lt)
+ lt = strptime(expires, u.SHORT_EXPIRES_ISO8601_FORMAT)
+
+ if not isinstance(self.expected_url, six.string_types):
+ expected_url = self.expected_url.replace(
+ b'1400003600', bytes(str(int(mktime(lt))), encoding='ascii'))
+ else:
+ expected_url = self.expected_url.replace(
+ '1400003600', str(int(mktime(lt))))
+ url = u.generate_temp_url(self.url, expires,
+ self.key, self.method)
+ self.assertEqual(url, expected_url)
+
+ @mock.patch('hmac.HMAC')
+ @mock.patch('time.time', return_value=1400000000)
+ def test_generate_temp_url_iso8601_output(self, time_mock, hmac_mock):
+ hmac_mock().hexdigest.return_value = 'temp_url_signature'
+ url = u.generate_temp_url(self.url, self.seconds,
+ self.key, self.method,
+ iso8601=True)
+ key = self.key
+ if not isinstance(key, six.binary_type):
+ key = key.encode('utf-8')
+
+ expires = strftime(u.EXPIRES_ISO8601_FORMAT, gmtime(1400003600))
+ if not isinstance(self.url, six.string_types):
+ self.assertTrue(url.endswith(bytes(expires, 'utf-8')))
+ else:
+ self.assertTrue(url.endswith(expires))
+ self.assertEqual(hmac_mock.mock_calls, [
+ mock.call(),
+ mock.call(key, self.expected_body, sha1),
+ mock.call().hexdigest(),
+ ])
+ self.assertIsInstance(url, type(self.url))
+
+ @mock.patch('hmac.HMAC')
@mock.patch('time.time', return_value=1400000000)
def test_generate_temp_url_prefix(self, time_mock, hmac_mock):
hmac_mock().hexdigest.return_value = 'temp_url_signature'
@@ -198,31 +260,34 @@ class TestTempURL(unittest.TestCase):
absolute=True)
self.assertEqual(url, expected_url)
- def test_generate_temp_url_bad_seconds(self):
+ def test_generate_temp_url_bad_time(self):
with self.assertRaises(ValueError) as exc_manager:
u.generate_temp_url(self.url, 'not_an_int', self.key, self.method)
- self.assertEqual(exc_manager.exception.args[0],
- 'seconds must be a whole number')
+ self.assertEqual(exc_manager.exception.args[0], u.TIME_ERRMSG)
with self.assertRaises(ValueError) as exc_manager:
u.generate_temp_url(self.url, -1, self.key, self.method)
- self.assertEqual(exc_manager.exception.args[0],
- 'seconds must be a whole number')
+ self.assertEqual(exc_manager.exception.args[0], u.TIME_ERRMSG)
with self.assertRaises(ValueError) as exc_manager:
u.generate_temp_url(self.url, 1.1, self.key, self.method)
- self.assertEqual(exc_manager.exception.args[0],
- 'seconds must be a whole number')
+ self.assertEqual(exc_manager.exception.args[0], u.TIME_ERRMSG)
with self.assertRaises(ValueError) as exc_manager:
u.generate_temp_url(self.url, '-1', self.key, self.method)
- self.assertEqual(exc_manager.exception.args[0],
- 'seconds must be a whole number')
+ self.assertEqual(exc_manager.exception.args[0], u.TIME_ERRMSG)
with self.assertRaises(ValueError) as exc_manager:
u.generate_temp_url(self.url, '1.1', self.key, self.method)
- self.assertEqual(exc_manager.exception.args[0],
- 'seconds must be a whole number')
+ self.assertEqual(exc_manager.exception.args[0], u.TIME_ERRMSG)
+ with self.assertRaises(ValueError) as exc_manager:
+ u.generate_temp_url(self.url, '2015-05', self.key, self.method)
+ self.assertEqual(exc_manager.exception.args[0], u.TIME_ERRMSG)
+
+ with self.assertRaises(ValueError) as exc_manager:
+ u.generate_temp_url(
+ self.url, '2015-05-01T01:00', self.key, self.method)
+ self.assertEqual(exc_manager.exception.args[0], u.TIME_ERRMSG)
def test_generate_temp_url_bad_path(self):
with self.assertRaises(ValueError) as exc_manager:
View Lorry Controller Status