summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Prepare for using standard python testsnewton-eolstable/newtonAndreas Jaeger2017-02-022-0/+61
| | | | | | | | | | | | | | | | | | Add simple script to setup mysql and postgresql databases, this script can be run by users during testing and will be run by CI systems for specific setup before running unit tests. This allows to change in project-config the python-db jobs to python-jobs since python-jobs will call this script initially. Update CONTRIBUTING for this. See also http://lists.openstack.org/pipermail/openstack-dev/2016-November/107784.html Change-Id: I3463e75057d0d4544f6a0212da888759ab5e171b Needed-By: I92e6e6502c2c516babf2bf66f3514875f77c460e (cherry picked from commit 6bd5974610cd3b77d966dd8ce68117ddbf3a371b)
* Allow splitting test groups into sub groupsPetr Malik2016-11-251-65/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When registering a new test group we give it a group name and a list of tests to run. register(['foo'], [t1, t2, t3]) creates group 'foo' running tests t1, t2 and t3. This method has been extended to allow passing a dict of tests with keys used as group name suffixes. register(['foo'], sub1=[t1, t2], sub2=[t3]) creates group 'foo' running all tests t1, t2 and t3 and groups 'foo_sub1' running t1, t2 and group 'foo_sub2' running t3 The plan is to split long-running datastore tests into multiple sub-groups that can run as separate jobs. Generally, we will want to run replication and clustering in one job separate from everything else. I have registered a 'single' (tests on a single instance) and 'multi' (tests woring on a set of instances) sub-groups for all datastores (for consistency). This does not change how the tests run. It just creates '*_single' and '*_multi' sub-groups in addition to the existing '<datastore>_supported' groups. We will need to update the project config to run sub-jobs for datastores that require it. Change-Id: Iea2a996043b4c2d1889995e160891af33ffab5b6 (cherry picked from commit 93f1ae8f09cf68cec62459822dfe39baf43a37a8)
* Skip 'eject valid master' replication test6.0.0.0rc26.0.0Peter Stachowski2016-09-221-3/+7
| | | | | | | | | | | | | | | | This test was skipped in the scenario runs however it was not disabled in the original api tests. Although it doesn't fail as often there it still happens, so the same change has been applied to it. See: https://review.openstack.org/#/c/368230 The test will also print out the bug number during each run as a reminder (using the new SkipKnownBug method). Change-Id: I931d6d72a70cc93dcd8248d9840eadf1160b9bab Related-Bug: #1622014 (cherry picked from commit e89dfbd51d4cad01a2ad701e9300ccfe6e7eac5a)
* Fix Postgresql configuration test and guest-logPetr Malik2016-09-218-32/+174
| | | | | | | | | | | | | | | | | | | | | | | - Move 'log_min_duration_statement' into non-dynamic values. Although the property itself is dynamic, it still requires restart because we don't provide default value for it in our template. - Add the 'ms' suffix to the 'log_min_duration_statement' time. Postgres by default returns the default suffix. Specifing the suffix explicitly fixes the comparison. - Change the validation rules for PostgreSQL long_min_duration_statement to string from int. - Enhance the ConfigurationManager to be able to apply system overrides *before* user overrides, as well as after. This allows the logging to be overridden by the user config values. - Added to the guest-log scenario test to make sure that the user log doesn't change after being disabled, and that it can be enabled again. Co-Authored-By: Petr Malik <pmalik@tesora.com> Co-Authored-By: Peter Stachowski <peter@tesora.com> Closes-Bug: #1624492 Change-Id: I059357d52fa24c609e45c2dc108d2b4ba77b3458
* Merge "Updated from global requirements" into stable/newtonJenkins2016-09-212-3/+3
|\
| * Updated from global requirementsOpenStack Proposal Bot2016-09-202-3/+3
| | | | | | | | Change-Id: I74d697b2bc3c8e37f8b4a48490a3e639ac729dec
* | improve pylint; generate errors and config in sorted orderAmrith Kumar2016-09-205-1691/+1030
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since the config is not in a deterministic order makes it hard to compare two config's and see what changed. Personally, I'm not positive I understand this use-case; i.e. you have an existing config file, you save it, and then rebuild and then diff the two files. I'd have thought you'd just run check and the output of the tool was the diff. I however do see the value in sorting the file so that when someone submits a change that includes a change to the config, reviewers can see more easily what the change is doing. Similarly, the output from pylint (errors) are generated one file at a time and os.walk makes no guarantee of deterministic order. So we should collect all errors (across all files) and then print an ordered list for human consumption. The intent is also to make pylint voting soon (in master). the changes to contributing.rst and tox.ini are to make that easier. The config file has also been sorted in place. This change was motivated by an email exchange with Peter so I am marking him as a co-conspirator. The line numbers were removed from the tools/trove-pylint.config file as these would change whenever the line numbers in the file changed (since they are currently not being used in the comparison; they can be re-added if deemed necessary at the cost of having every 'rebuild' run create a different file). The tools/trove-pylint.config was regenerated as well, since the remaining two errors seem to be innocuous: ERROR: trove/taskmanager/manager.py 392: E1101 no-member, Manager.upgrade: Instance of 'BuiltInstance' has no 'upgrade' member (new method introduced by instance upgrade; other BuiltInstance member errors are already ignored.) and ERROR: trove/guestagent/datastore/experimental/postgresql/service/ access.py 80: E1101 no-member, PgSqlAccess.list_access: Instance ofi 'PgSqlAccess' has no '_find_user' member (this is due to the fact that PostgreSQL is spread over multiple files and pylint should cease to complain once https://review.openstack.org/#/c/346082/ lands.) Change-Id: I910c738d3845b7749e57910f76523150ec5a5bff Closes-Bug: #1625158 Closes-Bug: #1625245 Co-Authored-By: Peter Stachowski <peter@tesora.com> (cherry picked from commit a2d336de2a2275b6abd6e7933347af72363bcc03)
* Merge "Fix pylint Error:PGSQL_RECOVERY_CONFIG" into stable/newtonJenkins2016-09-201-1/+1
|\
| * Fix pylint Error:PGSQL_RECOVERY_CONFIGjiansong2016-09-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | we should use pgsql_recovery_config, the method which has now been renamed to lowercase. The earlier code used a constant (upper-case) but in the change this one was likely missed. pgsql_recovery_config is an @property and should be referenced as such. This has nothing to do with case per-se, just a missing function/variable. Closes-Bug: #1624996 Change-Id: Icc9d794b45503340d988101fec710fae76f0b915 (cherry picked from commit a01beb4173b96accf92abbf018733a0ee52a7e8c)
* | Fix call to to_mbPeter Stachowski2016-09-191-2/+1
|/ | | | | | | | | | In a recent change (https://review.openstack.org/#/c/364431 ) the code for to_mb was moved, however one of the references to it was not changed. Found with pylint and fixed. Change-Id: I5a2178abaab4aea56e69165428ad27c5196d26f7 Closes-Bug: #1624908 (cherry picked from commit 3bf1944203c70abfbe8de9733f0033a17585aafc)
* Merge "Update UPPER_CONSTRAINTS_FILE for stable/newton" into stable/newtonJenkins2016-09-191-1/+1
|\
| * Update UPPER_CONSTRAINTS_FILE for stable/newtonDoug Hellmann2016-09-181-1/+1
| | | | | | | | | | Depends-On: I63cea3d9902735079c3e8783adb28ba2c336d802 Change-Id: I96af2446263acc9e0f4955dd0e4b21d13a0d2fed
* | Missing logging importAmrith Kumar2016-09-181-0/+3
| | | | | | | | | | | | | | | | A logging import was missing in a recent change. Found in pylint. Change-Id: Ide89ae809217d2ae3799de2b5dd20b7bcdbc9232 Closes-Bug: #1624702 (cherry picked from commit 095c7a1becd51591dbc28330a9615f6f4822456e)
* | Fix error: option --no-parallel not recognizedTrevor McCasland2016-09-161-1/+1
|/ | | | | | | | | | Passing --no-parallel through to testr causes an error in the coverage jobs. http://logs.openstack.org/de/dee522d8b14b1fdf73668e678374a91784b86ba3 Change-Id: I57e6f416105efc0595f507e7e4a671761f11d0fe (cherry picked from commit 092966a2d831bf08218d660779ab169957917722)
* Update .gitreview for stable/newtonDoug Hellmann2016-09-151-0/+1
| | | | Change-Id: I2916e36da45936d3cfc81c16cd1cfb14ac1073e9
* Merge "Handle the exceptions and errors correctly for DB2"6.0.0.0rc1Jenkins2016-09-151-12/+31
|\
| * Handle the exceptions and errors correctly for DB2mariamj2016-09-141-12/+31
| | | | | | | | | | | | | | Some of the exceptions were not handled correctly and this was exposed when run against pylint. Change-Id: I04206b661c4daa3381678d1b2d2a3fff0429021b
* | Merge "initial chkin of pylint"Jenkins2016-09-1415-3/+3036
|\ \
| * | initial chkin of pylintAmrith Kumar2016-09-1415-3/+3036
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | this is a pylint wrapper for Trove's tox tests. This commit includes a basic infrastructure for running pylint through tox. It also fixes some very obvious import errors that are flagged by the tool. One is to handle missing imports for _ and _LE. There is one instance where an exception is being raised but trove.common.exception isn't imported, and another where an exception is being incorrectly thrown. A short readme is also provided. Change-Id: I0a38f5efde3cb491f1f6c27f6c6500ab29987968 Partial-Bug: #1621636
* | | Merge "Call GuestError with proper options"Jenkins2016-09-144-17/+18
|\ \ \
| * | | Call GuestError with proper optionsPeter Stachowski2016-08-314-17/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Many calls to raise GuestError are not passing in the message argument 'original_message.' This causes the text 'An error occurred communicating with the guest: ' to be omitted from the message. These have been fixed. Change-Id: I25477513204ef2124af7b73683f1b63ba1051f3d Closes-Bug: #1618922 Depends-On: I924cfd1ca93bef7f3dc79149519df10305390779
* | | | Merge "Deprecate 'guest_log_long_query_time'"Jenkins2016-09-1412-13/+87
|\ \ \ \
| * | | | Deprecate 'guest_log_long_query_time'Petr Malik2016-08-1712-13/+87
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Slow query time should be configurable on per-instance basis via the existing configuration group mechanism (see the bug). We need to deprecate the existing global conf property first. The property has been deprecated on all datastores. Validation rules have been updated on those that support log retrieval as well: - MySQL: long_query_time - Percona: long_query_time - Percona XtraDB Cluster: long_query_time - PostgreSQL: log_min_duration_statement - MariaDB: long_query_time The logging settings will be saved with lower 'priority' than any user settings. This will ensure the user settings (if present) always override the global value. Once the deprecated options go away we can again apply the logging settings at the system level. Change-Id: I9c28ceb933b6819b8e96556584b2e26cab2155fb DocImpact: Document deprecated properties Partial-Bug: 1542485
* | | | | Merge "Stop adding ServiceAvailable group option"Jenkins2016-09-142-18/+9
|\ \ \ \ \ | |_|_|_|/ |/| | | |
| * | | | Stop adding ServiceAvailable group optionDaniel Mellado2016-09-072-18/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit modifies the tempest plugin to avoid keep adding already existing option groups. Change-Id: Idc9e571b4b4f5fe15c4342547cecd69de52ab552 Closes-Bug: #1621036
* | | | | Merge "Separate database and user create in prepare"Jenkins2016-09-141-1/+5
|\ \ \ \ \
| * | | | | Separate database and user create in prepareMatt Van Dijk2016-09-091-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During guest prepare, if the creation of database(s) throws an exception then if will skip the creation of user(s). To fix do a try/except on each call individually. Change-Id: Ibdd2081e5f4f5b31deb3fb70883e5c0140b4bdbd Closes-Bug: 1621976
* | | | | | Merge "Insulate TroveContext from oslo.context changes"Jenkins2016-09-141-0/+11
|\ \ \ \ \ \ | |_|_|_|_|/ |/| | | | |
| * | | | | Insulate TroveContext from oslo.context changesDoug Shelley2016-09-081-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Most of the recent issues with compatibility between running guests and the control plane have been related to changes in oslo.context. By putting some compatibility code into TroveContext, we should be able to insulate the environment during upgrades. Goal here is to reduce the chance of issues when the controller is upgraded before the guests (which is what every operator wants to do). The basic idea here is to introspect the current TroveContext including the RequestContext base class and remove any unknown arguments coming off the wire before the constructor is called. Change-Id: Ia4752c7dfe7bcfedebfaca9f13fb8bc6323d6001 Closes-bug: 1551468
* | | | | | Merge "Cluster code looks for wrong field in request for volume type"Jenkins2016-09-142-1/+4
|\ \ \ \ \ \
| * | | | | | Cluster code looks for wrong field in request for volume typechenshujuan2016-09-142-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cluster code on the server side is looking for a field called 'volume_type' in the request. The correct field should be 'type'. Closes-Bug: #1623005 Change-Id: I5f46b887e028819dcd2d18da6390d38401528194 Author: chenshujuan <chenshujuan@chinac.com> Owner: chenshujuan <chenshujuan@chinac.com>
* | | | | | | Merge "Update #noqa for out 79 character"Jenkins2016-09-142-2/+4
|\ \ \ \ \ \ \
| * | | | | | | Update #noqa for out 79 characterjiansong2016-09-112-2/+4
| | |_|_|_|_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently has some use # noqa because out 79 - character limit, in order to conform the pep8 specification, I think we can make some changes Change-Id: Ia91ec8175521854e0b413441a2e05a23330aa8c3
* | | | | | | Merge "Cassandra fix lost keyspace(s) after cluster grow"Jenkins2016-09-141-43/+38
|\ \ \ \ \ \ \
| * | | | | | | Cassandra fix lost keyspace(s) after cluster growPetr Malik2016-08-181-43/+38
| | |_|_|_|_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A seed node does not bootstrap, so a new node can't be configured as one immediately. It may be promoted only after bootstrapped. Also fix comprehension conditions. Nodes must be compared on 'id'. They cannot be compared as objects because each contains a reference to guest instance. Change-Id: Iaf8115548e54457a672048ed0e9d296ae893013e Closes-Bug: 1614577
* | | | | | | Merge "Add support for modules in cluster create/grow"Jenkins2016-09-1413-16/+35
|\ \ \ \ \ \ \ | |_|_|/ / / / |/| | | | | |
| * | | | | | Add support for modules in cluster create/growPeter Stachowski2016-09-0813-16/+35
| | |_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the module support was added, passing the modules to the actual create was not done on the server side. This has been added. Partial Bug: #1578917 Change-Id: I80c8e230f8bff550851f5cad146aa5976f45e5c4
* | | | | | Merge "Allow for invalid packet sequence in keepalive"Jenkins2016-09-1418-152/+240
|\ \ \ \ \ \
| * | | | | | Allow for invalid packet sequence in keepalivePeter Stachowski2016-09-1318-152/+240
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the SQLAlchemy keep_alive class, MariaDB is failing as pymysql reports an invalid packet sequence. MariaDB seems to timeout the client in a different way than MySQL and PXC, which manifests itself as the aforementioned invalid sequence. It is now handled as a special-case exception. With this fix, the MariaDB scenario tests now pass. The scenario tests were also tweaked a bit, which aided in the testing of the fix. 'group=instance' was created, plus instance_error properly interleaved with instance_create. _has_status now calls get_instance with the admin client so that any faults are accompanied by a relevant stack trace. Cases where the result code was being checked out-of-sequence were removed, and explicit calls to check the http code for the right client were added. The replication error messages for promote and eject were enhanced as well to attempt to debug spurious failures. One of those failures was 'Replication is not on after 60 seconds.' This was fixed by setting 'MASTER_CONNECT_RETRY' in the mariadb gtid replication strategy as was done in: https://review.openstack.org/#/c/188933 Finally, backup_incremental was added to MariaDB supported groups and cleaned up elsewhere. Closes-Bug: #1621702 Change-Id: Id6bde5a34e1d79eece3084f761dcd153c38ccbad
* | | | | | | Merge "Implement full online backups for DB2"Jenkins2016-09-1414-121/+333
|\ \ \ \ \ \ \
| * | | | | | | Implement full online backups for DB2Mariam John2016-09-1314-121/+333
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently Trove supports full offline backups for DB2. In this implementation, we have added support for full online backups for DB2 using archival logging. Change-Id: I30b5b0b85120fd105cc3db57983b062fad5fab5a Implements: blueprint db2-online-backup
* | | | | | | | Merge "Module re-apply does not reflect updated values"Jenkins2016-09-137-36/+178
|\ \ \ \ \ \ \ \ | |_|/ / / / / / |/| | | | | | |
| * | | | | | | Module re-apply does not reflect updated valuesPeter Stachowski2016-08-157-36/+178
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If you apply a module, remove it, update the module with a new name, and apply it again, module-query will still show the old name. This also applies to datastore, datastore_version and type. All four attributes are now updated correctly and scenario tests added. When writing the scenario tests it became evident that resetting a module from a datastore to all_datastores was broken too, so this was fixed (along with all_datastore_versions). This fix required a change to the Trove client as well. Change-Id: Ic2e9226bcd31c4a79d84ecb7941a47875eabd73d Depends-On: I0f226f09db08f635c542b27d8d2f898050d327fa Closes-Bug: #1611525 Closes-Bug: #1612430
* | | | | | | | Merge "Quota Management"Jenkins2016-09-135-3/+33
|\ \ \ \ \ \ \ \
| * | | | | | | | Quota ManagementMorgan Jones2016-09-015-3/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Modifies the management extension for Quotas to include usage information and update return values to current paradigms. Change-Id: I39122673082544e33bec0d3d5070df2c17aacb55 Depends-On: I6ca1a87fbc46781f83bc9b7bd33745783b8aaab1 Implements blueprint: quota-management
* | | | | | | | | Merge "Add command to delete BUILD instances and clusters"Jenkins2016-09-1313-12/+229
|\ \ \ \ \ \ \ \ \
| * | | | | | | | | Add command to delete BUILD instances and clustersAli Adil2016-09-1313-12/+229
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes an instance/cluster can be stuck in BUILD state forever. Attempting to delete the instance in this state is currently not allowed. Add force-delete and reset-status command. Reset-status will reset the status of an instance to ERROR and cluster to NONE. The reset-status command can only be used if the instance/cluster is in BUILD or ERROR state. Resetting the status of an instance in ERROR state can be useful as an instance might go ACTIVE after the specified timeout. Once the status has been reset it is possible for an instance to go ACTIVE if it receives a hearbeat from the guestagent. Force-delete will combine functionality of reset-status and delete. Change-Id: I83f6cdcdd884e51d002295b0d1f07341990e512c Depends-On: I957b4be5030e493e0eb8c6b6855d41b942b2823c Partial-Bug: #1579141
* | | | | | | | | | Merge "Tests verify cluster data via every node"Jenkins2016-09-131-7/+11
|\ \ \ \ \ \ \ \ \ \ | |/ / / / / / / / / |/| | | | | | | | |
| * | | | | | | | | Tests verify cluster data via every nodePetr Malik2016-08-261-7/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cluster tests currently verify data only via the first node. This may easily miss bugs when data is not properly replicated within the cluster due to some configuration issue. Change-Id: I9e280aeaa1e83eb1902b83f64a5e1d5bf2ba7a4f
* | | | | | | | | | Merge "Implement Instance Upgrade"Jenkins2016-09-1328-56/+504
|\ \ \ \ \ \ \ \ \ \ | |_|_|_|/ / / / / / |/| | | | | | | | |