| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Previously, we only accepted iterables of strings like 'Header: Value'.
Now, we'll also accept lists of tuples like ('Header', 'Value') as well
as dictionaries like {'Header': 'Value'}.
This should be more intuitive for application developers, who are
already used to being able to pass dicts or lists of tuples to libraries
like requests.
Change-Id: I93ed2f1e8305f0168b7a4bd90c205b04730da836
|
| |/
|
|
|
|
|
|
|
| |
Implements client-side functionality for
prefix-based tempurls.
Please see: https://review.openstack.org/#/c/274048/
Change-Id: I8d7701daee888ed1120271a96c0660b01543ca2d
|
| |
|
|
|
| |
Change-Id: I69276ba711057c122f97deac412e492e313c34dd
Closes-Bug: 1615830
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously the tempurl subcommand would dump a traceback
due to a TypeError if the seconds option was not an int
value. With this patch it will now return the same
error message as if the seconds option were negative or not
a number.
Also changes the error message to state that the seconds
option should be a "whole number" rather than a "positive
integer", since 0 is a valid value.
Change-Id: Ie940d470f2be8006aa8eb7fe242f092457aeae21
Closes-Bug: #1621817
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the supplied path is not of the form /v1/a/c/o then
swift tempurl subcommand will now return an error message.
Also removes redundant check for seconds parameter being an int
from shell.py because the same check is made when calling
utils.generate_temp_url.
Drive-by fix for missing param definition for generate_temp_url.
Change-Id: I41f4389948b01fadaa5fc4939ea12e0ed2167345
Related-Change: I0fb2ce125fe12d660e4deb778265016bdd5ff31b
|
| |
|
|
|
|
|
|
|
|
|
| |
Previously, we would accept gzip-encoded responses, but only because we
were letting requests decode *all* responses (even object data). This
restores the previous capability, but with tighter controls about which
requests will accept gzipped responses and where the decoding happens.
Change-Id: I4fd8b97207b9ab01b1bcf825cc16efd8ad46344a
Related-Bug: 1282861
Related-Bug: 1338464
|
| |
|
|
|
|
|
| |
Previously, we weren't encoding paths and keys as UTF-8, which would
trigger a UnicodeEncodeError on py27.
Change-Id: I2fad428369406c2ae32343a5e943ffb2cd1ca6ef
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When issuing `delete` commands that would require three or more
individual deletes, check whether the cluster supports bulk deletes
and use that if it's available.
Additionally, a new option is added to the `delete` command:
* --prefix <prefix>
Delete all objects that start with <prefix>. This is similar to the
--prefix option for the `list` command.
Example:
$ swift delete c --prefix obj_prefix/
...will delete from container "c" all objects whose name begins with
"obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar".
Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
|
| |
|
|
|
|
|
|
|
|
|
|
| |
When we introduced LengthWrapper, we neglected to make it resettable. As
a result, upload failures result in errors like:
put_object(...) failure and no ability to reset contents for reupload.
Now, LengthWrappers will be resettable if their _readable has seek/tell.
Related-Change: I6c8bc1366dfb591a26d934a30cd21c9e6b9a04ce
Change-Id: I21f43f06e8c78b24d1fc081efedf2687942e042f
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, if you attempt to stream an upload from an iterator, as in
def data():
yield 'foo'
yield ''
yield 'bar'
conn.put_object('c', 'o', data())
... requests will faithfully emit a zero-length chunk, ending the
transfer. Swift will then close the connection, possibly (if Connection:
keep-alive was set) after attempting to parse the next chunk as a new
request.
Now, Swift will receive all of the bytes from the iterable, and any
zero-byte chunks will be ignored.
This will be fixed in requests [1], but not until an eventual 3.0.0
release.
[1] https://github.com/kennethreitz/requests/pull/2631
Change-Id: I19579ed7a0181ac3f488433e7c1839f7f7a040b8
|
| |\ |
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This commit adds the utils module to those for which Sphinx
automatically generates documentation from docstrings. (Many of the
functions here may be of little interest to users, but
`generate_temp_url`, at least, definitely deserves to be in the
documentation; in this way, this commit can be seen as a spiritual
companion to ca70dd9e.)
Also, a few markup errors and perceived infelicities in existing
docstrings are amended.
Change-Id: I8c66a23cb359d7dd9302a16459fad9825fedb690
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Always use testtools.TestCase, since we're relying on testtools
* Always use mock (as opposed to unittest.mock) since we're relying on
mock
* Add note about when a missing logging handler was added
* Stop %-formatting the giant usage string that doesn't actually need
any formatting
* Prefer assertIs, assertIn, assertIsInstance over assertTrue
* Use else-self.fail instead of sentinel values
* Check resp.get('error') is None before checking resp['success'] is
True, so test failures actually tell you something useful
* Tighten some isinstance assertions
* Import MockHttpTest from correct location
* Only populate clean_os_environ once
* Use setUp for setup, not __init__
* Replace assertIn(key, dict) and assertEqual(foo, dict[key]) with
assertEqual(foo, dict.get(key)) when key is a literal and foo is not
None
* Use mock.patch.object instead of manually patching for tests
* Use six.binary_type instead of type(''.encode('utf-8'))
* Stop shadowing builtin bytes
* Reclaim some margin
* Stop checking the return-type of encode_utf8; we already know it's
bytes
Change-Id: I2138ea553378ce88810b7353147c8645a8f8c90e
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The `tempurl` subcommand's second positional argument is called
`seconds` and has heretofore interpreted as the number of seconds for
which the tempURL should be valid, counting from the moment of running
the command. This is indeed a common, if not the most common,
use-case. But some users, occasionally, might want to generate a tempURL
that expires at some particular ("absolute") time, rather than a
particular amount of time relative to the moment of happening to run the
command. (One might make an analogy to the way in which Swift's expiring
object support supports an `X-Delete-At` header in addition to
`X-Delete-After`—and it's the former that must be regarded as
ontologically prior.) Thus, this commit adds an `--absolute` optional
argument to the `tempurl` subcommand; if present, the `seconds` argument
will be interpreted as a Unix timestamp of when the tempURL should be
expire, rather than a duration for which the tempURL should be valid
starting from "now".
Change-Id: If9ded96f2799800958d5063127f3de812f50ef06
|
| |
|
|
|
|
|
|
|
| |
This patch adds exception logging to the swift service API. Each
operation that results in failure of any operation will now log
the exception as well as report a timestamp and full stack trace
in the results returned by the service API calls.
Change-Id: I7336b28354e7740ea7d048bdf355e3c1a1b4436c
|
| |
|
|
|
|
|
| |
In addition to removing an unnecessary dependency, this closes another
hole that was allowing raw bytes to appear in user-facing messages.
Change-Id: Ia0b76426a38e5a5c368c4c7e7ba2aef286758aca
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With MD5Sum checking being added, a concern was brought up that It was
a change with no possibility of reverting to the old behaviour.
This change adds the flag '--ignore-checksum' to the upload subcommand
allowing the checks to be turned off.
Changed occurrences of the magic string for a null md5 to use a descriptive
constant instead.
Updated Error messages generated when validation fails. They should now be more descriptive
and not output a literal newline sequence.
Change-Id: Id1756cbb6700bb7e38f0ee0e75bc535e37f777ed
|
| |
|
|
|
|
|
|
|
|
|
| |
Changed existing code to calculate the MD5 of the object
during the upload stream. Checks this MD5 against the etag
returned in the response.
An exception is raised if they do not match.
Closes-Bug: 1379263
Change-Id: I6c8bc1366dfb591a26d934a30cd21c9e6b9a04ce
|
| |
|
|
|
|
|
|
|
|
| |
Temporary URLs allow a user to sign an object URL with a shared
secret to so that the object can be downloaded without auth for
a specified amount of time.
http://docs.openstack.org/trunk/config-reference/content/object-storage-tempurl.html
Change-Id: Ife0b6c98c975e074d4dad0a31145573b784747c5
|
| |
|
|
|
|
|
|
|
|
|
| |
* Replace unicode with six.text_type
* Replace basestring with six.string_types
* The long type doesn't exist in Python 3 anymore: replace 1L with long(1) and
only test this type with Python 2
* Fix quote(): quote the URL if the string is a byte string. Use "bytes" type
instead of "str" to be Python 3 compatible.
Change-Id: I1df5aa85e4e7d07191fb5c654d52fc4bd8b9f440
|
| |
|
|
|
|
| |
Closes-Bug:#1284360
Change-Id: Iec63a3fde77bb8195bfe46c764403b367999ff43
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Currently, httplib implementation does not support SSL certificate
verification. This patch fixes this. Note that ssl compression parameter
and 100-continue thing is still missing from requests, though those are
lower priority.
Requests now takes care of:
* proxy configuration (get_environ_proxies),
* chunked encoding (with data generator),
* bulk uploading (with files dictionary),
* SSL certificate verification (with 'insecure' and 'cacert' parameter).
This patch have been tested with requests 1.1.0 (CentOS 6) and requests
2.2.1 (current version).
Change-Id: Ib5de962f4102d57c71ad85fd81a615362ef175dc
Closes-Bug: #1199783
DocImpact
SecurityImpact
|
| |
|
|
|
|
|
|
|
| |
As far as proxies usage is concerned, keystone-client API and swift-client API behave differently because the former uses python Request library while the latter uses raw httplib. As a result, Keystone authentication honors environment variables http_proxy, https_proxy and no_proxy while Swift doesn't.
This patch, which code is mainly borrowed from Python Requests, makes Swift
data connections and Swift authentication connections behaving homogeneously.
Change-Id: Ic8a0089c35c458d7ed96e572e22429014298fe4c
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
When you stat a container or object with the verbose flag the full path of the
reousrce will be displayed with the token similarlly to how an account stat
displays the auth url and token.
* move some logic out of bin/swift.st_stat to test it
* new module swiftclient.commnad_helpers for code you want to test
* moved prt_bytes into swiftclient.utils to test it
* fixed IndexError with prt_bytes on sizes >= 1024Y
Change-Id: Iaaa96e0308b08c554205b0055b8a04de581fefa4
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch extracts the multi-threading code from bin/swift into
swiftclient/multithreading and adds tests. In particular, this new way
of doing it (with context managers) will prevent non-daemonic threads
from wedging the process when unexpected exceptions happen.
I enabled reporting of which lines, specifically, are not covered by
unit tests (added -m option to "coverage report" in .unittests).
This patch includes a drive-by fix for uploading a segmented file with
--use-slo when that object already exists. A key of "name" was used
instead of "path", raising KeyError.
There's also another drive-by fix for uploading segmented objects with
--use-slo. Commit 874e0e4427b80e1b15b74a1557b73ba9d61443ca regressed
this by removing the capturing of thread-worker results in
QueueFunctionThread.run(). This patch restores that functionality and
the feature (uploading SLO objects).
Change-Id: I0b4f677e4a734e83d1a25088d9a74f7d46384e53
|
|
|
Add env[SWIFTCLIENT_INSECURE] as default of --insecure option.
If set to 'true', allow to access insecure keystone server.
The name follows 'NOVACLIENT_INSECURE' in novaclient.
Change-Id: I322674eba9c07e6def97bce339815fa15191a92d
|
