summaryrefslogtreecommitdiff
path: root/swiftclient
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Unindent a chunk of st_list"Jenkins2015-03-051-57/+52
|\
| * Unindent a chunk of st_listPete Zaitcev2015-02-261-57/+52
| | | | | | | | | | | | | | | | The chunk looks like nicely containable, but it pushes against the right edge of the screen annoyingly. This patch _only_ prettifies, without any observable changes. Change-Id: I96fcb5848e3e5e0e8a8ebbe398b1d4a0bea5fa24
* | Add improvements to MD5 validation.Daniel Wakefield2015-03-043-23/+34
|/ | | | | | | | | | | | | | | With MD5Sum checking being added, a concern was brought up that It was a change with no possibility of reverting to the old behaviour. This change adds the flag '--ignore-checksum' to the upload subcommand allowing the checks to be turned off. Changed occurrences of the magic string for a null md5 to use a descriptive constant instead. Updated Error messages generated when validation fails. They should now be more descriptive and not output a literal newline sequence. Change-Id: Id1756cbb6700bb7e38f0ee0e75bc535e37f777ed
* Merge "Release connection after consuming the content"Jenkins2015-02-251-3/+14
|\
| * Release connection after consuming the contentIan Cordasco2015-02-251-3/+14
| | | | | | | | | | | | | | | | | | | | | | | | When using stream=True, we need to make sure we release a connection back to the connection pool to avoid warnings clogging up the logs. We can do this by releasing the connection once we've read all the content from the response. In situations where stream=False, requests already does this for us. Related-bug: 1424732 Closes-bug: 1341777 Change-Id: Id1c72ad86135a49d4b985992d788736b65f7dbda
* | Merge "Fix crash when stat'ing objects with non-ascii names"Jenkins2015-02-251-3/+1
|\ \ | |/ |/|
| * Fix crash when stat'ing objects with non-ascii namesJoel Wright2015-02-171-3/+1
| | | | | | | | | | | | | | | | This patch fixes a crash and stack trace in stat when an object name contains non-ascii characters. Change-Id: Ib6dc686771e593243de56cafc100b17e51d9d9d5 Closes-Bug: 1411665
* | Merge "Fix crash with -l, -d /, and pseudo folders"Jenkins2015-02-251-3/+6
|\ \
| * | Fix crash with -l, -d /, and pseudo foldersPete Zaitcev2015-02-181-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using -l, -d /, or a pseduo folder by itself works fine, but if you combine all of them, this happens: [zaitcev@guren s]$ swift list -d / testcont amazon_lili.jpg pseudodir/ rin_shibuya_pixiv.jpg swift-c134855-5a.diff [zaitcev@guren s]$ swift list -d / -l testcont 151954 2014-06-20 17:13:52 amazon_lili.jpg Traceback (most recent call last): File "/q/zaitcev/hail/python-swiftclient-tip/bin/swift", line 24, in <module> sys.exit(main()) File "/q/zaitcev/hail/python-swiftclient-tip/swiftclient/shell.py", line 1301, in main globals()['st_%s' % args[0]](parser, argv[1:], output) File "/q/zaitcev/hail/python-swiftclient-tip/swiftclient/shell.py", line 414, in st_list total_bytes += item_bytes TypeError: unsupported operand type(s) for +=: 'int' and 'NoneType' The patch is a straightforward fix that assigns expected variables. Change-Id: If24dcc2491e8b8982455107d2aca30c0491a398a Closes-Bug: 1399678
* | | Verify MD5 of uploaded objects.Daniel Wakefield2015-02-193-42/+140
|/ / | | | | | | | | | | | | | | | | | | | | Changed existing code to calculate the MD5 of the object during the upload stream. Checks this MD5 against the etag returned in the response. An exception is raised if they do not match. Closes-Bug: 1379263 Change-Id: I6c8bc1366dfb591a26d934a30cd21c9e6b9a04ce
* | Fix missing ca-certificate parameter to get_auth.Charles Hsu2015-02-071-2/+4
|/ | | | | | | | | Missing cacert parameter in get_auth_1_0() and it makes requests_args['verify'] is True in default(likes "swift --os-cacert ssl.crti list"). The requests_args['verify'] should be a ca-certificate file name("ssl.crt"). Change-Id: Id3110f3818f55922ad19eeb1ead80fface824009 Closes-Bug: 1418473
* Fix deleting SLO segments on overwriteJoel Wright2015-02-041-10/+9
| | | | | | | | This patch fixes a bug that results in SLO segments being left behind regardless of the setting of --leave-segments. Change-Id: I2b3c8ab4817df6facb4b5961dbe570645441a552 Closes-Bug: 1418007
* Merge "This patch fixes downloading files to stdout."Jenkins2015-02-033-60/+73
|\
| * This patch fixes downloading files to stdout.Joel Wright2015-01-233-60/+73
| | | | | | | | | | | | | | | | | | | | | | | | This patch fixes downloading files to stdout and modifies _SwiftReader to operate as an iterator that performs file checks at the end of iteration as well as a context manager. File verification checks have been removed from __exit__ and added to __iter__. Change-Id: I3250bdeeef8484a9122c4b5b854756a7c8f8731e Closes-Bug: 1395922 Closes-Bug: 1387376
* | Merge "Fix cross account upload using --os-storage-url"Jenkins2015-02-023-50/+45
|\ \
| * | Fix cross account upload using --os-storage-urlAlistair Coles2015-01-063-50/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removes an account stat from the object upload path. This stat fails when user is not account admin even though the user may have container ACL permission to write objects. Reduces the severity of the CLI output message when upload fails to create the given container (this is not an error since the container may exist - the user just does not have permission to PUT or POST the container). Changes the 'swift upload' exit return code from 1 to 0 if container PUT fails but object PUT succeeds. For segment uploads, makes the attempt to create the segment container conditional on it not being the same as the manifest container. This avoids an unnecessary container PUT. Fixes another bug that became apparent: with segmented upload a container HEAD may be attempted to determine the policy to be used for the segment container. When this failed the result dict has headers=None which was causing an exception in the shell result handler. Add unit tests for object upload/download and container list with --os-storage-url option. Closes-Bug: #1371650 Change-Id: If1f8a02ee7459ea2158ffa6e958f67d299ec529e
* | | Fix environment sanitization for TestServiceUtilsClay Gerrard2015-01-211-35/+39
| |/ |/| | | | | | | | | | | | | | | | | | | | | I'm not sure how test_process_options_defaults ever passed for anyone that had all the ST_(AUTH|USER|KEY) set in their environment when they ran it. Despite our efforts to patch os.environ swiftclient.servce was acctually importing environ from os and then sticking the results into some global state. The tests seemed to work fine on infra, but for me locally until I `unset ST_AUTH` the test would always fail. Seems to work now and looks fairly reasonable on the surface. Change-Id: Id68590d6af16040a2877e719bc38d1e126603e42
* | Merge "Check that content_type header exists before using"Jenkins2015-01-101-1/+2
|\ \ | |/ |/|
| * Check that content_type header exists before usingChris Buccella2014-11-071-1/+2
| | | | | | | | | | | | | | | | | | | | When downloading an object, the content_type header is inspected to determine if a directory needs to be created. This header is assumed to always be in the response; if it isn't an Exception is raised. swiftclient should not assume content_type will always be set. Change-Id: I156195c02b6e0bc398fa962eb1f78c4dbddd1596
* | Print info message about incorrect --totals usageJiří Suchomel2015-01-051-0/+5
| | | | | | | | | | | | | | | | when neither -l nor --lh is provided. Added test coverage for --totals. Change-Id: I3245e715c26ec28457a21dec07311a58c475c066 Closes-Bug: 1258392
* | Merge "Fix misplaced check for None in SwiftUploadObject."Jenkins2015-01-041-21/+13
|\ \
| * | Fix misplaced check for None in SwiftUploadObject.Daniel Wakefield2014-12-161-21/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Check for None was being done after a method call which caused an attribute error if the value was None. The Method call was also a mistake and has been corrected to the hasattr function like intended. Added Tests. Closes-Bug: 1392651 Change-Id: Ifb1c84e26533bccbaddcce5738f434f36feca74e
* | | Merge "Suppress 'No handlers...' message from keystoneclient logger"Jenkins2015-01-021-0/+2
|\ \ \ | |_|/ |/| |
| * | Suppress 'No handlers...' message from keystoneclient loggerAlistair Coles2014-11-131-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | keystoneclient isn't adding a NullHandler to its logger, so we see 'No handlers could be found for logger...' messages when using the swift cli with keystone auth. Since keystoneclient follows the usual pattern of naming loggers from module name, we can add a NullHandler to the 'keystoneclient' logger to handle any logging from the library. Note that this makes no change to keystoneclient logging being emitted when using 'swift --debug' or 'swift --info'. No test because I am not sure how to usefully test this without ending up just testing a mock of keystoneclient. Change-Id: Ic81cf8e5a590ae9224d35e3a5b1c5f2a8c33835a
* | | Merge "Make preauth params work"Jenkins2014-12-311-9/+16
|\ \ \
| * | | Make preauth params workClay Gerrard2014-12-171-9/+16
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If you specify a token and storage url when creating a Connection, regardless of the auth api version the first request will be made directly to swift. You can either provide a preauthurl and preauthtoken or fall back to os_options' object_storage_url and auth_token keys (exposed as --os-storage-url and --os-auth-token on the command line or OS_STORAGE_URL and OS_AUTH_TOKEN in the environment). If a _retry wrapped request on a Connection fails because of invalid authentication (401) the Connection's cached token and url will be invalidated. If the Connection's retries attribute is > 0 the subsequent attempt will call get_auth to refresh the token, but the pre-configured storage_url will always be re-used. This is consistent with current auth v2 behavior and less surprising for auth v1. The pre-existing, but previously undocumented behavior/interface of get_auth would override the storage_url returned by the auth service if the 'os_storage_url' option was provided in the os_options dict. To ensure that this behavior is consistent across auth v1 and v2 from the command line and when using the Connection class as a library - the preauthurl is stashed in the os_options dict when provided. Improved Connection.get_capabilities storage_url handling to better support the consistent behavior of a preauthurl/object_storage_url on the connection regardless of auth version. Fixed up some test infrastructure to enable setting up and testing multiple requests/responses. Change-Id: I6950fb73f3e28fdddb62760cae9320e2f4336776
* | | Merge "Change tests to use new CaptureOutput class."Jenkins2014-12-302-1/+4
|\ \ \ | |/ / |/| |
| * | Change tests to use new CaptureOutput class.Daniel Wakefield2014-12-052-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added the ability to clear the buffers in the CaptureOutput class so it can be easily used multiple times in the same context manager. Also added a option to suppress the SystemExit associated with printing an error. Change-Id: Ib59bbbe88256f215eed0a8ebc8282e02181d4377
* | | Merge "Show warning when auth_version >= 2 and keystoneclient is missing"Jenkins2014-12-151-2/+1
|\ \ \
| * | | Show warning when auth_version >= 2 and keystoneclient is missingChristian Schwede2014-11-251-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Swiftclient hangs forever without any reasonable hint if someone uses a recent version of swiftclient together with auth version 2 or later. The reason for this is the sys.exit() call in a thread - the main thread never exists, and swift just waits forever. Raising a ClientException instead of calling sys.exit() fixes this and returns the desired information back to the user. Change-Id: I1fdff85da2dd4b883d337d6098b1695e448d6f2b
* | | | Fix misnamed dictionary key.Daniel Wakefield2014-12-082-69/+48
| |/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The response dictionary in _delete_segment attach's any error it encounters to the dict key 'exception', all other response dict's use 'error' to store the exception. Changed to make it consistent and added tests Also added a third branch in st_delete as messages created in _delete_container where being silently dropped. Change-Id: Ifbc3b1fae78910fbc6acf4a86cfb0f60bb1aa336
* | | Merge "Fix misnamed variable in SwiftReader."Jenkins2014-11-171-17/+18
|\ \ \
| * | | Fix misnamed variable in SwiftReader.Daniel Wakefield2014-11-071-17/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | actual_md5 was being used instead of _actual_md5 only when assigning the md5 object. This resulted in all checks and updates of the md5 to be skipped. When fixed it revealed another bug that the md5 of the read content is always checked even if there is no _expected_etag. Added tests Change-Id: Iaf1e21324ce592049d02cd5be123604b99833e86
* | | | Add unit tests for _encode_meta_headersDaniel Wakefield2014-11-131-30/+26
| |_|/ |/| | | | | | | | | | | Also move it to a module level function. Change-Id: I2da64876815619a6db7ff208061df2191767806c
* | | Merge "Make swift post output an error message when failing"Jenkins2014-11-132-73/+72
|\ \ \
| * | | Make swift post output an error message when failingAlistair Coles2014-10-242-73/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | swiftclient.shell.st_post was ignoring the result dict returned from SwiftService.post and therefore giving no indication when posts failed e.g. due to invalid auth credentials. This patch ensures that SwiftService always returns a result_dict from post() and then checks the result dict. On failure the shell now outputs the ClientException message and exits with error code 1. Also adds unit tests and cleans up some unnecessary mocked return values in existing tests. Closes-Bug: 1383243 Change-Id: I111150eb3d026c8d01c2cac13d3613ca7304e5b9
* | | | Merge "Fix KeyError raised from client Connection"Jenkins2014-11-131-1/+1
|\ \ \ \ | |/ / / | | / / | |/ / |/| |
| * | Fix KeyError raised from client ConnectionAlistair Coles2014-10-201-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some client.Connection methods will raise a KeyError if a response_dict argument is passed and an error occurs during authentication or making the request. The fix is straightforward: add a test for existence of a response_dict before attempting to get it from kwargs. The bulk of this patch is adding unit tests for the response_dict feature. Closes-Bug: 1381304 Change-Id: Ic7e1b3dfae33909533931c52ac97355867a08a07
* | | Merge "Adds user friendly message when --segment-size is a non-integer"Jenkins2014-10-281-1/+7
|\ \ \ | |_|/ |/| |
| * | Adds user friendly message when --segment-size is a non-integerMahati Chamarthy2014-10-281-1/+7
| | | | | | | | | | | | Change-Id: Ic2c062110b2fe2e725b0f4a4517a300cfbf663a8
* | | Replaces Stacktraces with useful error messages.Daniel Wakefield2014-10-213-13/+19
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changed the message shown when a user doesn't enter project/tenant id/name to be more informative. When attempting to stat a container without supplying project/tenant name or id, an empty response was being returned instead of an error being raised. Changed the error raised in swiftclient.client when no tenant or project is specified to be more specific. Add tests for basic regression checking. Closes-Bug: #1372589 Change-Id: I4eb6964d9f1702db119cc0294edc02841b1ecd5f
* | Fix race in shell when testing for errors to raise SysExitAlistair Coles2014-10-201-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SysExit is raised if the OutputManager has non-zero error_count. However, error_count is incremented in a print thread, but tested on the main thread. Currently error_count is tested before the OutputManager print threads have exited, which results in a race. This race means that SysExit is not always raised even when an error has occured. This change moves the error_count test after the OutputManager context manager has exited, which ensures that the OutputManager threads have exited. Change-Id: I5ef7d9f10057fe2b41f48ab95066a7265b86a3ac Closes-Bug: #1379229
* | Fix race between container create jobs during uploadAlistair Coles2014-10-201-9/+16
|/ | | | | | | | | | | | | | | | | | | During a segmented object upload, attempts are made to create the container for the manifest and the segment container. These jobs are currently placed on separate threads and can occur out of order which causes the unit test failure seen in the bug report. The container jobs should be ordered because the segment container job attempts to HEAD the manifest container to determine correct storage policy. This patch enforces ordering of the two jobs and modifies the unit test to assert that both container PUTs occur in the expected order. Change-Id: If90bec126867a4be2df34067ccefee660202f59f Closes-Bug: #1376878
* Merge "Fix the info command with --insecure"Jenkins2014-10-191-3/+3
|\
| * Fix the info command with --insecurePete Zaitcev2014-10-171-3/+3
| | | | | | | | | | | | | | | | | | We forgot to propagate options into http_connection() for the info aka capabilities command. Fix is to use the proper helper. Fixes bug: 1381866 Change-Id: I9515203c969a1bba38dd909412355080383905f9
* | Merge "Allow segment size to be specified in a human readable way."Jenkins2014-10-171-1/+17
|\ \ | |/ |/|
| * Allow segment size to be specified in a human readable way.Daniel Wakefield2014-10-101-1/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Instead of always specifying segment size in bytes the user can now use B,K,M or G as suffixes for the corresponding size. Conversion is done with Binary units (1024) rather than SI units (1000). e.g swift upload test_container -S 1073741824 large_file can now be written swift upload test_container -S 1G large_file The change is backwards compatible as it ignores arguments to -S that don't have a valid suffix. Updated unit tests and help message. Change-Id: I6314b4e45cf2fbffde2fe57a02df77a25e911e84
* | Merge "Use skipTest from testtools instead of inherited Exception"Jenkins2014-10-091-4/+0
|\ \
| * | Use skipTest from testtools instead of inherited ExceptionChristian Schwede2014-10-081-4/+0
| |/ | | | | | | | | | | | | | | | | | | SkipTest is raised if there is no func_test section in the Swift test config file. However, the currently raised Exception will result in a failed test, not in a test marked as skipped. Since we already use testtools we can use the included skipTest easily. Change-Id: I1bbb1f9dbe31fb0698d774550708d1196b266625
* | Do not crash with "swift list --lh" for Ceph RadosGW.Jiří Suchomel2014-10-011-1/+1
|/ | | | | | | | | | If 'x-timestamp' header is not present in 'swift list' result, date string cannot be correctly generated. Check for correct exception to catch this case and return a special string value. This is just a workaround for faulty RadosGW behavior. Change-Id: I172843a2668e33efac00924906ff0dca3e790050 Closes-Bug: 1349528
View Lorry Controller Status