summaryrefslogtreecommitdiff
path: root/trove/backup
Commit message (Collapse)AuthorAgeFilesLines
* Fix illegal shell characters18.0.0.0rc118.0.0wu.chunyang2022-09-141-0/+3
| | | | | | | | | | This change checks if the user input is legal. if illegal, we raise an InvalidValue excepiton. Story: 2010004 Task: 45128 Change-Id: Ib81646b8f8a01fcbc31d033ec205491b76a7b755
* Support to restore backup from remote locationLingxian Kong2021-02-183-50/+106
| | | | | | | | In multi-region deployment with geo-replicated Swift, the user can restore a backup in one region by manually specifying the original backup data location created in another region. Change-Id: Iefef3bf969163af707935445bc23299400dc88c3
* Add project_id to backup responseLingxian Kong2020-09-151-0/+1
| | | | Change-Id: I99fb3d0c7ea7036bbb622d86eef9f31e53065a0a
* Support getting backups of a specific projectLingxian Kong2020-09-132-3/+8
| | | | Change-Id: I6a6778ddbb1ee92da28a709be91b04873fe749fe
* Support backup strategy APILingxian Kong2020-07-173-6/+165
| | | | Change-Id: I0ddd7214dae6e29ddfaf045fdb282f4980a8afff
* Datastore containerizationLingxian Kong2020-05-271-1/+1
| | | | | | | | | | | | | Significant changes: * Using docker image to install datastore. * Datastore image is common to different datastores. * Using backup docker image to do backup and restore. * Support MariaDB replication * Set most of the functional jobs as non-voting as nested virtualization is not supported in CI. Change-Id: Ia9c97a63a961eebc336b70d28dc77638144c1834
* Use dedicated service credential configLingxian Kong2019-10-101-3/+3
| | | | | | The current admin credential config options are confusing Change-Id: I92c15c065edf364b955f2243b3599e0b8cae8b4f
* Support to create public trove instanceLingxian Kong2019-09-111-8/+8
| | | | | | | | | | | | | | | | | | | | - The users need to specify the network to create Trove instance, but trove-taskmanager will create port in that network for Nova instance creation. Using port gives Trove more capabilities to define how the database service is exposed. - Deprecate ICMP protocol for the instance. - Restrict 'nics' parameter for creating instance. - Add 'access' parameter for creating instance. - Add 'public_network_id' option in order to create floating IP for the instance. - Do not create records for security groups, but Trove can still delete existing instances for backward compatibility. - Delete unreasonable Host, Account, Storage API. Story: 2006500 Task: 36468 Task: 36466 Change-Id: I80827e1ad5e6b130cbf94c2bb7a909c44d5cf1e5
* Fix all_projects filtering for backupsLingxian Kong2019-08-251-1/+2
| | | | Change-Id: Ie0f7bb9fad73cb29084702d1044ae3ed7eb3b8ce
* Support backup filteringLingxian Kong2019-08-232-12/+24
| | | | | | | | | | Support to filter backups by instance_id and all_projects(admin only by default). Story: #2006433 Task: #36343 Change-Id: Ia483bbafb8d106a9d46ab908cf5659f06fb3b7ed
* Fix replication failure when Swift isn't availablerocky-em10.0.0.0rc110.0.0Dariusz Krol2018-08-021-1/+3
| | | | | | | | | | | | | | | * add Swift token verification in create instance method in Trove API when slave_id is given * catch ConnectionError exception in verify_swift_auth_token * add Swift token verification to guestagent before backup and restore * add new exception representing Swift connection error * set missing fault information when replication snapshot fails * mock verify auth token method in replication and restore unit tests Closes-Bug: #1395523 Change-Id: I6a21ba2ba890a82875f9b6dae3c6b93bc9fdb4b0 Signed-off-by: Dariusz Krol <d.krol@samsung.com>
* Sync the data fields of DB* class and table fields.zhanggang2018-07-171-2/+2
| | | | | | | | | | | | This update try to fix some problems: 1. define the 'id' both in _data_fields and _auto_generated_attrs. 2. some of tables don't have the 'id' field. 3. _data_fields of DBInstance miss some table fields. 4. Change the key of persisted_models as the same of table name to make it more readable. Change-Id: I0ad96824e8de978ff5b6766085549b1e1a2509cf Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com>
* Remove log translationsZhao Chao2018-01-242-7/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to discussions on the ML, log messages should not be translated any more. This patch also: * removes all usage of _LI, _LW, _LE, _LC; * updates log translation hacking rule. ML discussions: http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html The original commit messages are kept as following to make a historic note: remove unwanted log translations recent discussions on the ML have led to the decision to eliminate all translations for messages that are destined for log files but retain them for messages that are going to be returned as exceptions and potentially shown to end users via an API. see [1], [2]. This change does that as follows. 1. If a string is being used to generate an exception, it will still be translated. Still Trove used both _LE and _ for these translations, there are some cases where _LE has been simply changed to _, and not removed. 2. If a string is used for a logging message, remove the use of _, _LE, _LW, _LI. Also, I have long felt that we have had a creep of checks in the pep8 tests that apparently make the code easier to read. I strongly believe that these kinds of "easier to read" things make sense if they are followed across all projects and not just gratuitously added one project at a time. I've taken this opportunity to reduce the merge mess caused by this change, to sync up our ignore flags with a long list of ignores from Nova. When they made the change for removing log translation, they could do it in an automated-way like I have because they didn't have to deal with under and overindented lines for visual edit (E127). Same for Cinder. Changes 448443 [3] and 447839 [4] were inadequate because they only addressed a little part of the problem, namely removing the use of _LE, _LI, and _LW, and I think this is a change we don't need to dribble in a few files at a time. The changes are straightforward and should be taken in a single lump to make it easy to deal with the merges coming. [1] http://lists.openstack.org/pipermail/openstack-operators/2017-March/012887.html [2] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html [3] https://review.openstack.org/448443 [4] https://review.openstack.org/447839 Co-Authored-By: Amrith Kumar <amrith@amrith.org> Co-Authored-By: Valencia Serrao <vserrao@us.ibm.com> Change-Id: I5f86c982469e625997fc8bd15c6fae0fc77a8c64
* Avoid deleting parent backup failed caused by 404Fan Zhang2017-10-201-1/+4
| | | | | | | | Catch NotFound exception in for loop, to avoid dropping out while deleting parent backup that has multiple children backups. Change-Id: Idab9fc8d6c09b0b58015b0b9fdf99811d075191f Closes-bug: #1718572
* Handle log message interpolation by the logger in backup/Gábor Antal2017-02-142-6/+6
| | | | | | | | | | According to OpenStack Guideline[1], logged string message should be interpolated by the logger. [1]: http://docs.openstack.org/developer/oslo.i18n/guidelines.html#adding-variables-to-log-messages Change-Id: Ia79addd292000dfd1e3ecb15884b8dccac4d9284 Related-Bug: #1642552
* Add support for Oslo Policies to TrovePetr Malik2016-12-061-0/+8
| | | | | | | | | | | | | | | | | | The Oslo Policy library provides support for RBAC policy enforcement across all OpenStack services. Update the devstack plugin to copy the default policy file over to /etc/trove in the gate environments. Note: Not adding a rule for 'reset-password' instance action as that API was discontinued years ago and is now just waiting for removal (Bug: 1645866). DocImpact Co-Authored-By: Ali Adil <aadil@tesora.com> Change-Id: Ic443a4c663301840406cad537159eab7b0b5ed1c Implements: blueprint trove-policy
* Remove cfg and logging import unusedCao Xuan Hoang2016-09-231-2/+0
| | | | | | | | This patch removes cfg and logging import unused in trove Change-Id: Ic50a58123481faa37b8898565a597dcdb1f1fa46 Closes-Bug: #1618281 Closes-Bug: #1618347
* Add --incremental flag to backup-createAli Adil2016-09-062-4/+18
| | | | | | | | | | | Add --incremental flag to backup-create and set the default value to False. When the flag is passed it will create a backup based on the last good known full/incremental backup. If there is no previous backup a full backup will be created. Change-Id: I6fcd60f6068dfb0145fa64248a2d8b3fdfc1004d Depends-On: I96498180890b8b1692b1bac3b19988532f6eb077 Partial-Bug: #1607429
* Implement DBaaS Ceilometer NotificationsMorgan Jones2016-02-271-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | Defines and implements create|end|error notifications for all state-changing Trove API calls. Adds a notification to the TroveContext to transfer the notification to the guest and conductor so that errors on asynchronous commands can be forwarded to the Conductor to be transferred to the control plane bus. Also did some cleanup on the existing notifications to bring them all under a common framework in trove/common/notifications.py. The trove.instance.exists notification was not integrated into the new framework due to its close-coupling with the Nova notification code. Reworked the cluster action mechanism to move routing functionality from the strategy to the Cluster base class. This was done to support tying notifications to cluster specific actions. Implements Blueprint: ceilometer-integration Change-Id: I9c57d24f80d8d3116fc0cc8948094087a0495135
* Switch to the oslo_log librarySergey Vilgelm2015-07-232-2/+3
| | | | | | | | Remove the log module of oslo-incubator. Move the WritableLogger wrapper to the base_wsgi module. Add oslo.log to the requirements. Change-Id: I724fa6090cebf40e7d7c78cc6b8458dfba9508a8
* Fixes hacking rulesSushil Kumar2015-05-292-17/+17
| | | | | | | | | | | | | | | | | This patchset helps reducing the ignored pep8 errors. Fixed some ignored hacking rules, as listed follows: - E111 indentation is not a multiple of four - E122 continuation line missing indentation or outdented - E123 closing bracket does not match indentation of opening bracket's line - E128 continuation line under-indented for visual indent - E251 unexpected spaces around keyword / parameter equals - E265 block comment should start with '# ' - E713 test for membership should be 'not in' - H105 Don't use author tags - H306 imports not in alphabetical order Change-Id: Iadf2af4f6ec90420153ad63b5a41650392ef2cbd
* Replication V2Morgan Jones2015-03-181-0/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The V2 implentation of replication, including: - Ubuntu 14.04 - Mysql 5.6 - GTID Replication Strategy - promote-to-replica-source - eject-replica-source - --replica_count option to create - replica creation from incremental backups By agreement with Slicknik (Trove PTL), the following exclusions from the spec have been made: - slave promoted to master by eject not replaced - slave promoted by eject chosen by txn count rather than most recent txn Authored-By: Morgan Jones <morgan@parelastic.com> Co-Authored-By: Peter Stachowski <peter@tesora.com> Co-Authored-By: Doug Shelley <doug@parelastic.com> Co-Authored-By: Duk Loi <duk@tesora.com> Co-Authored-By: Petr Malik <pmalik@tesora.com> Change-Id: I7f921b75e66081cac9ec5a60fb204d61162e9a12 Depends-On: I8eec708f41e791e3db04a2c7b7c12855118b64ac Depends-On: I9075365a8fae754e29d27d6e371a1d8c8980e26b Implements: blueprint replication-v2
* Provide more readable error message when swift is not installedMayuri Ganguly2015-02-131-0/+2
| | | | | | | | | | When creating a backup with no swift, the error message is very general. Added an exception handler to provide a more meaningful error message. Closes-bug 1395107 Change-Id: I7cdc9ddd29d094493ce652425addfa461138675f
* Fix backup state check while restoring an instanceDenis Makogon2015-01-291-0/+4
| | | | | | | | | | | | | | | | | | | Reasons: - Trove API service checks backup only for NEW/BUILDING state, FAILED state also significant since users are not able to restore an instance from FAILED backup. Changes: - adding 'is_done_successfuly' property to DBBackup object; - adding status validation at restoring; - fake int-tests proposed. Behaviour: - server side raises HTTP Conflict(409) when attempting to restore an instance from backup that is still running or failed. Change-Id: Ide9d0e6f67adaea0c6d99679eba73cebb01c807f Closes-Bug: #1338996
* Obsolete oslo-incubator modules - gettextutils (now oslo.i18n)Amrith Kumar2014-12-252-2/+2
| | | | | | | | | | | | | | | | | | This change is part of a multi-part change set to handle obsolete and graduated oslo modules. This commit handles the gettextutils module which is now oslo.i18n. The change here is to remove it from openstack-common.conf and point all people who used to use that at oslo.i18n. requirements.txt has been updated to point to this dependency. We aren't deleting gettextutils.py from openstack/common because it is used by other oslo-incubator modules. This change has been rebased on https://review.openstack.org/#/c/129668/ Change-Id: I39fd288fb29a8317f896812a2f64bcd69a6b8e21 Partial-Bug: #1380789 blueprint: retire-unused-oslo-incubator-modules
* Removing dependency on trove models in the guest agentRobert Myers2014-10-062-11/+27
| | | | | | | | | | | | | Reasons: - The guest agent is importing backup models and agent heartbeat, this triggers the all of the trove database setup logic which bloats the guest process on the host. - Moving the state to its own module and removing an unused function decreases the memory usage by about 15 - 20 megs. Closes-Bug: #1375311 Change-Id: I126c0b89c170b325d85b3f09afca399b4f5de9e8
* Clusters API Implementationamcrn2014-08-291-0/+2
| | | | | | | | | | | | | adds clusters api, with mongodb sharding as the first implemenation. Co-Authored-By: Michael Yu <michayu@ebaysf.com> Co-Authored-By: Mat Lowery <mlowery@ebaysf.com> Co-Authored-By: rumale <rumale@ebaysf.com> Co-Authored-By: Timothy He <the@ebaysf.com> Partially implements: blueprint clustering Change-Id: Icab6fc3baab72e97f3231eaa4476b56b8dafb2a3
* Restrict backup-list on instance to tenantNikhil Manchanda2014-07-171-2/+7
| | | | | | | | | Fixed backup-list on an instance, so that we show all available backups for the instance iff the user making the request is an admin. For a non-admin user, we return only the backups in his particular tenant. Change-Id: I2aff6dca053d8261bb70083bf52dac46806faabe Closes-bug: 1295325
* Logging audit for trove/backup moduleDoug Shelley2014-06-262-12/+18
| | | | | | | Adjust log messages to conform to logging standards. Change-Id: Id11f7fec952a7f9d4af3c4b40d3283d8871026d4 Partial-Bug: #1324206
* Add datastore version to backupsSteve Leon2014-06-123-17/+52
| | | | | | | | | | | | | | | This fix adds datastore information to the backup APIs. These API includes backup list, backup list by instance and backup show. Also this fix enhances backup-list to optionally receive a datastore to filter the backups by. The integration tests will fail until the update for the client gets merged: https://review.openstack.org/#/c/90462/ Change-Id: Icfec975b92cd9523e639ad6a2d6787ee4d4cb39d Implements: blueprint backup-metadata
* Enabled H402 flake8 ruleSushilKM2014-06-021-1/+1
| | | | | | | | | | | | | | | Added support for flake8 H402: one line docstring needs punctuation. Reasons: - H402 is disabled in tox.ini, it needs to be enabled to make code more flake8 compliant. Changes: - Updated tox.ini to enable H402 rule. - Updated code for H402 violation. - Updated one word docstrings to be more meaningful. Change-Id: I5e67b641717c9704ef4b9f6267390561e4840fff
* Merge "Remove unused variables"Jenkins2014-04-041-1/+0
|\
| * Remove unused variablesKIYOHIRO ADACHI2014-03-031-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | o trove/backup/models.py:SWIFT_CONTAINER = CONF.backup_swift_container This variable was added in the following commit. commit ce208d6cabba227576f492e7647d9da5a89e7c25 Author: ruiyuan-shen <ruiyuan.shen@hp.com> Date: Mon Mar 18 14:07:37 2013 -0700 Add snapshot ORM to reddwarf But there is no evidence that was used. o trove/dns/rsdns/driver.py:DNS_HOSTNAME = CONF.dns_hostname This variable was added in the following commit. commit 0c34c75dc44876245709987a18ea004607fca815 Author: Sudarshan Acharya <sudarshan.acharya@rackspace.com> Date: Fri May 18 15:49:12 2012 -0500 DNS Support for Instance IP. But there is no evidence that was used. o trove/instance/models.py:SERVER_INVALID_ACTION_STATUSES = ["BUILD", "REBOOT", "REBUILD", "RESIZE"] This variable was removed in the following commit. commit 54a4ada785fd90af5134e7f38360938e0b435696 Author: Ed Cranford <ed.cranford@rackspace.com> Date: Thu Jul 12 15:01:01 2012 -0500 Delete can now delete errored instances. o trove/instance/models.py:VALID_ACTION_STATUSES = ["ACTIVE"] This variable was removed in the following commit. commit 9ccef6fea42454bb22c0d67bd4d2a0765f13d370 Author: Tim Simpson <tim.simpson@rackspace.com> Date: Wed Jun 27 11:12:28 2012 -0500 Allowing resizes to be performed when MySQL is down. Closes-Bug: #1286990 Change-Id: I79f5bb124f2702317d8d337c969c9ff8ea69a769
* | Added Backup/Restore validationsRamashri Umale2014-03-171-1/+13
|/ | | | | | | | | | | | Added checks for backup-create API call to ensure datastore supports backup strategy Also for trove-create API call (when restoring from backup) added checks to ensure that datastore version of backup being restored matches with datastore version of instance being created Change-Id: I9f9cf76ce3d2533612515e6382a184e955e09a32 Closes-Bug: #1285876
* Remove extraneous vim configuration commentsHe Yongli2014-02-272-4/+0
| | | | | | | | | | | Remove vim setting, Remove line containing: comment - # vim: tabstop=4 shiftwidth=4 softtabstop=4 at the top of source code files, except for files in openstack/common. Change-Id: I5e2aad6dca8407890fc3c2afbda541bd4f124ea8 Closes-Bug: #1229324
* Adding Incremental BackupsRobert Myers2014-02-073-4/+23
| | | | | | | | | | | | * Adding a config flag to register incremental runners * Adding InnoBackupExIncremental runner and restore strategy. * Adding save_metadata and load_metadata methods for swift storage strategy. * Adding incremental backup/restore integration test. Implements: blueprint incremental-backups Change-Id: I1f0d4e5967097498f86a5052c33c55471e24a137
* Remove copyright from empty filesAlexander Ignatov2014-01-161-13/+0
| | | | | | | | | According to policy change in HACKING: http://docs.openstack.org/developer/hacking/#openstack-licensing empty files should no longer contain copyright notices. Change-Id: I3992a72c3614e2d0c65aee91e69d89ff806824fc Closes-Bug: #1262424
* Paginate backup list apiRobert Myers2013-12-122-12/+40
| | | | | | | | | | | * Order list by updated at to show latest first * Add pagination support for backup list and instance backup list * Add tests for pagination and ordering Implements: blueprint paginate-backup-list Change-Id: Ie98bd2bc0500dfe4bbe0bb8b420ad80ad7f5a5df
* Removes unused import statements2014.1.b1Sushil Kumar2013-12-031-1/+0
| | | | | | | | | Updates tox.ini to remove suppressed F401 standard. Updates the code files containing unused import statements Change-Id: Ib44bd1509fdb17f457a205f9e62caf85cebee830 Closes-Bug: #1257267
* Conductor proxies host db access for guestsEd Cranford2013-11-261-2/+9
| | | | | | | | | | | | | | | | | | | | | Previously, instances updated their status by updating the database on the host directly. Necessarily, each instance would need access to the database to stay updated. Trove's new conductor service eliminates that need by working as a proxy for those instances. By sending a heartbeat to conductor via RPC, conductor updates the database on the host on behalf of the instance. As backups also made use of the host database, the backup code has been refactored to take richer inputs to remove the need to query the host database, and now conductor is also used to submit updates to backup states. Implements: blueprint trove-conductor Change-Id: I4cb34baedd0e3a50051f9e66de95c9028c66e4b5
* Fix checksum verification exception.Tim Simpson2013-10-301-1/+1
| | | | | | | | | | | | | Previously if an invalid checksum was found, the exception would not be raised. This commit also makes the tests for backup run in fake mode run again (turns out there was no reason to disable them) since they helped find this, and cleans up an issue with the restore backup test (it used the current size of an instance created elsewhere for no reason). Fixes bug 1246368 Change-Id: I96a570a93695b017e5c6966fa1482d6ec5aef426
* Add Backup/Restore Checksum ValidationJoe Cruz2013-10-251-15/+23
| | | | | | | | | | | | | | | | | | | | Store the swift checksum for swift storage backups. The swift checksum is the checksum of the concatenated segment checksums. This is the checksum that is stored with the DB backup record so we can verify that the checksum on restore through a swift object HEAD call. Adds restore from backup checksum validation at API and guestagent level. * Integrate checksum verification with check object exists at API level. * Verify checksum right before downloading swift file. * Verify the current swift backup file checksum matches the original backup checksum to prevent loading and restoring a tampered backup file. Add swift checksum storage unit tests Also move backup verification logic outside quotas. Implements: blueprint verify-backup-checksum-on-restore Change-Id: I2844cfb8a8b31f6ea7622f99eff3bea9455161e1
* Replace OpenStack LLC with OpenStack FoundationZhiQiang Fan2013-09-202-2/+2
| | | | | | | | | | | Some files still use trademark OpenStack LLC in header, which should be changed to OpenStack Foundation. NOTE: tools/install_venv.py is not touched, should sync with oslo, so does trove/openstack/common/* Change-Id: I7218ae896eb7db681bb224a209df47f203952606 Fixes-Bug: #1214176
* Merge "Adding volume size to the backup views/models."Jenkins2013-08-061-0/+1
|\
| * Adding volume size to the backup views/models.Robert Myers2013-08-011-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | * Save the size of the guest before running a backup * Display the size in the backup GET/LIST views * Simplify the dbaas module removing redundant class. * Switch get_filesystem_volume_stats to use os.statvfs() * Change output of volume stats to return used in GB to simplify various view that display it. Implements: Blueprint backup-volume-size Change-Id: I860427d3491acb34e43ee23f91e4ffbb39e0042c
* | Clear new or building backups on reset-task-statusEd Cranford2013-07-261-0/+10
|/ | | | | | | | | | The reset-task-status management call now also clears off any new or building backups for the instance by marking those backups as FAILED. Fixes Bug # 1205471 Change-Id: I478782b5e90c9965b91169a784b2ea65bda8a8b4
* Change the swift file deletion to use the manifest.2013.2.b2Robert Myers2013-07-161-1/+2
| | | | | | | | | | * Adding a _parse_manifest helper to parse the x-object-manifest header * Delete file using the prefix in the manifest. * Fix error handling in the delete_backup method. Fixes Bug: #1194653 Change-Id: I34268f768a5cdb8085607269e2c2cb95974a539d
* API Validation for Trove APIjustin-hopper2013-07-011-12/+2
| | | | | | | | | | | Replaced validation with new and improved json schema validation Fixed malformed json schema bug #1177969 Fixed integration test for create_user; correct databases Fixed integration for create_users; invalid character_set param Implements: json-schema-support Change-Id: I6ca09803654d9e78362fde69185b5b9e05a5eb6b
* Fixed backups GET and DELETE to restrict access to the owner of backupNikhil Manchanda2013-06-242-4/+7
| | | | | | | | | | | Fixed the db model find_by so that when a model instance is looked up and a context is passed in, the find_by operation fails if the tenant_id in the instance is not the same as the tenant_id in the context (except if the tenant is an admin). Fixed bug 1188822 Change-Id: I0374f999601fa3f2bc817901bbc428b8be219683
* Rename from reddwarf to trove.Michael Basnight2013-06-244-0/+353
Implements Blueprint reddwarf-trove-rename Change-Id: Ia9ee609bbc06a1d8b9d6917642529f30347541fd
View Lorry Controller Status