summaryrefslogtreecommitdiff
path: root/src/obsync
Commit message (Collapse)AuthorAgeFilesLines
* obsync: if OrdinaryCallingFormat fails, try SubdomainCallingFormatSage Weil2012-09-111-2/+10
| | | | | | | | | | | This blindly tries the Subdomain calling format if the ordinary method fails. In particular, this works around buckets that present a PermanentRedirect message. See bug #3128. Signed-off-by: Sage Weil <sage@inktank.com> Reviewed-by: Matthew Wodrich <matthew.wodrich@dreamhost.com>
* obsync: add missing package specifier to format_excMatthew Wodrich2012-08-161-1/+1
| | | | | | Fixes: #2873 Signed-off-by: Matthew Wodrich <matthew.wodrich@dreamhost.com> Reviewed-by: Dan Mick <dan.mick@inktank.com>
* obsync: make obsync run without cloudfiles installedKyle Marsh2012-01-041-1/+12
| | | | | Cloudfiles probably shouldn't be a requirement for running obsync, so this commit makes it optional.
* obsync: pull object metadata from swift storeKyle Marsh2011-12-201-0/+1
| | | | | Obsync wasn't pulling object metadata from swift stores and thus wasn't syncing metadata when reading from a swift store. This commit fixes that.
* obsync: add authurl to CLIKyle Marsh2011-12-161-13/+37
| | | | | | s3 connections require the hostname and swift connections require the authurl. obsync treats these as equivalent internally, but breaks them apart on the command line interface for clarity for the users.
* obsync: add vvprint back inKyle Marsh2011-12-141-0/+4
| | | | | Commit ebe5fc60d20f92a0037c53c1e7bd7ae512be3da4 removed the definition of vvprint without removint all the places that used it.
* obsync: add swift support to obsyncKyle Marsh2011-12-091-72/+217
| | | | | | | | | A single "url" doesn't make sense for a swift object store the way it does for an S3 store or local file, so this commit explodes the interface from a single url argument to three arguments: host, bucket, and object_prefix. It also adds a SwiftStore and related classes. Syncing ACLs with swift is not yet implemented.
* obsync: tear out rgwYehuda Sadeh2011-11-221-261/+1
|
* obsync: add SRC_SECURE, DST_SECUREColin Patrick McCabe2011-08-121-3/+9
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* Add ObsyncExceptionColin Patrick McCabe2011-07-261-230/+298
| | | | | | | | | | | This class has subclasses representing temporary and permanent exceptions, as well as argument parsing errors. An instance of this class can be created from a message or from another exception. We always print out the type of exception on the last line of stderr, right after the exception information. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* botol_tool: add --xattr argument for PUTColin Patrick McCabe2011-07-151-3/+21
| | | | | | | boto_tool can now set extended attributes when performing a PUT operation. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* boto_tool: add --get-obj-xattrColin Patrick McCabe2011-07-151-0/+11
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obsync: add man page, documentation lineColin Patrick McCabe2011-07-011-0/+1
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* boto_tool: add put_bucket_aclColin Patrick McCabe2011-06-101-0/+13
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* boto_tool: add get_bucket_aclMatthew Wodrich2011-06-011-1/+17
| | | | Signed-off-by: "Colin McCabe" <colin.mcccabe@dreamhost.com>
* obysnc: rgw target: validate all usersColin Patrick McCabe2011-05-262-15/+35
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* test-obsync: test sync directly from s3->rgwColin Patrick McCabe2011-05-261-0/+9
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obsync: fix bucket creation through rgw targetColin Patrick McCabe2011-05-262-31/+40
| | | | | | The rgw: target can now create buckets. Add a test. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* test-obsync: test big objects, user-defined xattrColin Patrick McCabe2011-05-261-0/+7
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obysnc: fix content-type on RGWStoreColin Patrick McCabe2011-05-252-7/+15
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* test-obsync: compare_directory now compares xattrsColin Patrick McCabe2011-05-251-10/+50
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obsync: more fixes for RgwStoreColin Patrick McCabe2011-05-252-18/+38
| | | | | | | | | | | | * Fix content-type handling * add vvprint and use it in Object::equals. * support RgwStore::prefix * more tests Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obysnc: RgwStore: make sure destination users existColin Patrick McCabe2011-05-251-0/+17
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obsync: fix DST_OWNERColin Patrick McCabe2011-05-251-10/+23
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obsync: Add boto_retries, remove rgw_store.prefixColin Patrick McCabe2011-05-252-12/+17
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* RgwStore: fix some ACL issuesColin Patrick McCabe2011-05-251-13/+21
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* test-obysnc.py: support librgw testingColin Patrick McCabe2011-05-252-70/+124
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* Rename RadosStore to RgwStoreColin Patrick McCabe2011-05-251-21/+21
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* test-obsync: refactor a little bitColin Patrick McCabe2011-05-251-14/+18
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* Proper ACL support for rados targetsColin Patrick McCabe2011-05-251-36/+73
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* Merge remote branch 'origin/next'Josh Durgin2011-05-173-1/+1
|\
| * debian: obsyncSage Weil2011-05-172-0/+1300
| | | | | | | | Signed-off-by: Sage Weil <sage@newdream.net>
| * obsync: no .pySage Weil2011-05-173-1301/+1
| | | | | | | | Signed-off-by: Sage Weil <sage@newdream.net>
* | obsync: preserve user-defined metadataColin Patrick McCabe2011-05-162-4/+23
| | | | | | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obsync: filestore: separate xattr metadata nspaceColin Patrick McCabe2011-05-162-30/+36
| | | | | | | | | | | | | | User-defined s3 metadata lives in a separate namespace from regular S3 metadata like Content-Type, etc. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obysnc: preserve Content-TypeColin Patrick McCabe2011-05-162-20/+139
| | | | | | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obsync: FileStore: test storing ACLs in xattrsColin Patrick McCabe2011-05-162-45/+49
| | | | | | | | | | | | Update unit tests now that we're storing ACLs in xattrs. Fix a bug. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obsync: FileStore: store ACLs in xattrsColin Patrick McCabe2011-05-161-34/+13
| | | | | | | | | | | | Store the ACL XML in extended attributes rather than in side files. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obsync: FileStore: test for xattr supportColin Patrick McCabe2011-05-161-0/+19
| | | | | | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obsync: fix eventual consistency handlerColin Patrick McCabe2011-05-131-33/+28
| | | | | | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obsync: add DST_CONSISTENCYColin Patrick McCabe2011-05-132-4/+21
| | | | | | | | | | | | | | | | The DST_CONSISTENCY variable allows us to specify that the destination is expected to use read-after-write consistency. If that is the case, we don't have to do slow retries if certain operations fail. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | boto_tool.py: add --rmobjects, --rm_rfColin Patrick McCabe2011-05-131-0/+22
| | | | | | | | | | | | Add some options to help destroy buckets. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | boto_tool.py: fix old-style argument-passingColin Patrick McCabe2011-05-131-1/+1
| | | | | | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | boto_tool.py: use s3-tests config fileColin Patrick McCabe2011-05-131-211/+149
| | | | | | | | | | | | | | | | | | boto_tool now grabs the configuration variables it needs from the s3-tests config file, similar to s3-tests and test-obsync. Also, changed argument parsing to be more straightforward. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | rgw: put XML-to-bin translation into a librgwColin Patrick McCabe2011-05-131-1/+1
| | | | | | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | obsync: handle eventual consistency issuesColin Patrick McCabe2011-05-111-10/+51
| | | | | | | | | | | | | | Handle eventual consistency issues so that obsync will be usable on more S3 stores. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* | test-obsync.sh: fix obsync unit testsColin Patrick McCabe2011-05-111-5/+12
|/ | | | | | | | Fix the obsync unit tests to take into account the new ACL changes. ACLs must be either translated or ignored when copying between the main and alt test buckets. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* test-obsync.py: allow better nonce,better printoutColin Patrick McCabe2011-05-052-13/+28
| | | | | | Support deterministic nonces. Better debug printouts. Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* obsync: remove misguided strip_ownerColin Patrick McCabe2011-05-051-27/+21
| | | | Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
* test-obsync: use s3-tests configuration fileColin Patrick McCabe2011-05-051-140/+173
| | | | | | | Use the same configuration file format as s3-tests. This should make it easier for people to run obsync tests! Signed-off-by: Colin McCabe <colin.mccabe@dreamhost.com>
View Lorry Controller Status