summaryrefslogtreecommitdiff
path: root/test-requirements.txt
Commit message (Collapse)AuthorAgeFilesLines
...
| * Run Python 3 unit tests in parallelVictor Stinner2016-06-221-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace testtools.run test runner with os-testr for Python 3 to run tests in parallel. Replace also the whitelist with a blacklist to quickly identify tests which still must be ported to Python 3. Blacklist backup.test_backup_models: even if it worked when run with testtools.run, it fails when run using os-testr. We can investigate this issue later, it's more important to reduce the duration of the py34 job to reduce the risk of random timeout. Add a dependency to os-testr. Partially implements: blueprint trove-python3 Change-Id: I5ea7ebf7eec15b75a5fed07aab7bd15e5e6cd6d6
* | Updated from global requirementsOpenStack Proposal Bot2016-06-211-1/+1
|/ | | | Change-Id: I8f5137d0db3c4044267d2dcf2d45413c4143889c
* Add Install GuideAndreas Jaeger2016-06-161-0/+1
| | | | | | | | | | | | | | | | | | | | | | | This adds trove-specific contents of the OpenStack Installation Guide in the trove repo per [1]. A separate change will remove the heat contents from the OpenStack Installation Guide for Newton per [2]. This is a verbatim import from the content in openstack-manuals with minimal changes to get content building. Also adds tox.ini environment for install-guide and adds openstackdocs-theme to test-requirements.txt. From lmichaels -- added a statement to the next-steps doc. [1] http://specs.openstack.org/openstack/docs-specs/specs/newton/project-specific-installguides.html [2] http://specs.openstack.org/openstack/docs-specs/specs/newton/installguide.html [3] https://review.openstack.org/#/c/314229/ Partially-Implements: blueprint projectspecificinstallguides Change-Id: Idf8cb90536fd4b4d9a2d0695ef30f498b1283410
* Updated from global requirementsOpenStack Proposal Bot2016-05-261-1/+1
| | | | Change-Id: I5f28b20ad2888172e5ffaa873b155c193d29b5aa
* Add bandit security configPeter MacKinnon2016-05-131-0/+1
| | | | | | | | | | | Bandit is an OpenStack security linter that uses python AST modules to perform static code analysis of a project. It can help identify low, medium, and high security "hotspots" in a project codebase. This change provides an initial Bandit config based on the existing one used by Sahara. Change-Id: Idcfb5641007bb27de8df262b236b9a46d7e11d5d Closes-Bug: 1547730
* Updated from global requirementsOpenStack Proposal Bot2016-04-101-1/+1
| | | | Change-Id: Idfa054cee896891efd639fcdc85e575384d24ece
* Updated from global requirementsOpenStack Proposal Bot2016-04-071-1/+1
| | | | Change-Id: Id2e0ecae3694a3b6d8e920406cda89c8abad1ba6
* Implement Backup and Restore for CouchDBMariam John2016-03-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The recommended method for doing full backups in CouchDB has been a simple filesystem copy of the data files. This is because CouchDB stores data in wholly contained append only files. For example, when a user creates a database, a corresponding <database-name>.couch file is created in the database directory. The backup functionality has been implemented by compressing the database directory and then encrypting it and sending it over to store in Swift. Similarly, the restore functionality has been implemented by fetching the files from Swift and uncompressing them into the database directory. After this, the ownership of the directory needs to be updated. To test the changes, follow the steps: - Create a CouchDB instance - Access the admin console called Futon using the following url: http://10.0.0.5:5984/_utils/ - Create a database from there and create one or more documents - Create a backup of this CouchDB instance - Create another CouchDB instance from the backup created above - Access the admin console for this new instance and verify that the database created above is there couchdb client library for the integration tests has been added to global-requirements: https://review.openstack.org/#/c/285191/ Change-Id: Iad6d69bf60ace73825819081964a43ad53d6c6fc Implements: blueprint couchdb-backup-restore
* Updated from global requirementsOpenStack Proposal Bot2016-03-121-2/+2
| | | | Change-Id: I25f4eb7fa72ce198dfd6bcd40abd0c867745c5e3
* Block pymongo version 3.1Amrith Kumar2016-03-081-1/+1
| | | | | | | | | | | | | pymongo version 3.1 produces segmentation faults when running unit tests and has broken the trove gate, see bug 1512870. This is one part of the solution, the other part of the solution is to push the change to global-requirements.txt. That's in the review https://review.openstack.org/#/c/290233/2 Change-Id: I0ac00931cc696e53f1845b60c23f84c9846a6a5e Depends-On: I94dd0d031bb2c6ecae45ec9482cc2f481e77ce03 Closes-Bug: 1554838 Related-Bug: 1512870
* Updated from global requirementsOpenStack Proposal Bot2016-03-031-1/+1
| | | | Change-Id: Icc859403a5ca9ae4256062ac8967a75936499b45
* Server side of module maintenance commandsPeter Stachowski2016-02-251-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changeset handles the details of creating, updating, listing and deleting Trove 'modules.' Two new tables have been added to the Trove database: modules instance_modules although the instance_modules table is at present unused. Scenario tests have been written as well, to exercise the new functionality. These tests can be run by: ./redstack int-tests --group=module_create In the follow-up changeset, all module tests can be run by: ./redstack int-tests --group=module Since module support is available for all datastores (controlled by a CONF option) the module test has been added to the common modules group. Note: Trying to do admin tasks with none admin credentials results in an Unauthorized exception being thrown, instead of Forbidden. This is due to the fact that Forbidden is in the HTTPUnauthorized section of wsgi.py instead of the HTTPForbidden section. Moving the exception caused too many failures, so I created a 'Module' Forbidden exception and put it in the right section. Change-Id: I755b0431b33b870ae02d903527f071fd8e23130d Depends-On: I54d37025275dee4731ad49ebbd21612c4464e4c4 Depends-On: I779c24472d3d96a7b2fe4ed0284fd5869cdef93b Partially-Implements: blueprint module-maintenance
* Implement user functions for Cassandra datastorePetr Malik2016-02-101-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch set implements the following functionality for Cassandra datastore. create/delete/get user list users change password grant/revoke/list access update attributes create/delete database list databases Notes on Cassandra users: In Cassandra only SUPERUSERS can create other users and grant permissions to database resources. Trove uses the 'os_admin' superuser to perform its administrative tasks. It proactively removes the built-in 'cassandra' superuser on prepare. The users it creates are all 'normal' (NOSUPERUSER) accounts. The permissions it can grant are also limited to non-superuser operations. This is to prevent anybody from creating a new superuser via the Trove API. Updatable attributes include username and password. The configuration template had to be updated to enable authentication and authorization support (original configuration allowed anonymous connections). Default implementations used are: authenticator: org.apache.cassandra.auth.PasswordAuthenticator authorizer: org.apache.cassandra.auth.CassandraAuthorizer The superuser password is set to a random Trove password which is then stored in a Trove-read-only file in '~/.cassandra/cqlshrc' which is also the default location for client settings. Notes on Cassandra keyspaces: Cassandra stores replicas on multiple nodes to ensure reliability and fault tolerance. All replicas are equally important; there is no primary or master. A replication strategy determines the nodes where replicas are placed. The total number of replicas across the cluster is referred to as the replication factor. The above 'create database' implementation uses 'SimpleStrategy' with just a single replica on the guest machine. This is a very simplistic configuration only good for the most basic applications and demonstration purposes. SimpleStrategy is for a single data center only. The following system keyspaces have been included in the default 'ignore_dbs' configuration list and therefore excluded from all database operations: 'system', 'system_auth', 'system_traces' Notes on user rename: Cassandra does not have a native way for renaming users. The reason why Cassandra itself does not implement rename is apparently just lack of demand for that feature. We implement it by creating a new user, transferring permissions and dropping the old one (which also removes its existing permissions). I asked about the sanity of this rename approach on the Cassandra mailing list and IRC channel and there should not be anything inherently wrong with the proposed procedure. This method, however, requires the user to always provide a password. Additional notes: Trove uses the official open-source Python driver for Cassandra to connect to the database and execute queries. The connection is implemented in CassandraConnection. It is now also used to obtain the current database status as opposed to the original method of parsing output of the client tool. The 'common/operating_system' module was extended with two new functions for reading/writing ini-style and YAML configuration files to/from Python dicts. Unit tests were added to 'guestagent/test_operating_system'. The existing Manager unit tests were extended to include the added functionality. Also includes some minor improvements to comments and log messages. Used the existing operating_system interface to update file ownership. The system module was removed and its contents moved to the Application class. This is to reduce the number of files and help facilitate overriding. Implements: blueprint cassandra-database-user-functions Depends-On: I0faa3a4b9c7302064fb2413b572e2fc515efff0d Change-Id: I7021f6a0e9a3a933f00cfb7a5d987dc6fe2f95a6
* Updated from global requirementsOpenStack Proposal Bot2016-02-031-1/+1
| | | | Change-Id: I51c2d365a87ce802d81e7abf21fb02433abd3ccc
* Fix issue of mismatched test-requirements.txt fileAmrith Kumar2016-01-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | When you run devstack, the trove plugin installs python-troveclient by forcing upper-constraints.txt in requirements to be changed as below. -python-troveclient===2.0.0 +-e file:///opt/stack/python-troveclient#egg=python-troveclient As a result, pinning test-requirements.txt to use a file for python-troveclient to come from tarballs.openstack.org causes a failure. The error you will see is "Could not satisfy constraints for 'python-troveclient': installation from path or url cannot be constrained to a version". This change modifies test-requirements.txt and matches global-requirements.txt, and devstack makes the change (above) to upper-constraints.txt which accomplishes the same thing. Change-Id: I3bb748a575d67f9f419f51837555d18ed5d7ca32 Closes-Bug: 1539818
* Implement Guest Log File RetrievalMorgan Jones2016-01-261-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Implements log file retrieval from the guest agent. The contents of the log file are pushed up to a swift container as a series of objects that represent a subset of the lines in the log. The following trove CLI commands are now supported: trove log-list <instance> : lists log files available on guest trove log-enable <instance> <log> : enables writing to log file trove log-disable <instance> <log>: disables writing to log file trove log-publish <instance> <log>: publishes updates to swift container trove log-discard <instance> <log>: discards published logs trove log-tail <instance> <log> : displays last lines of log trove log-save <instance> <log> : saves the entire log to a file Log declarations and scenario tests were added for MySQL and PostgreSQL. Co-Authored-By: Morgan Jones <morgan@tesora.com> Co-Authored-By: Alex Tomic <atomic@tesora.com> Co-Authored-By: Peter Stachowski <peter@tesora.com> Implements: blueprint datastore-log-operations Change-Id: I16c3bba4a3183d05af2971be6ba56110105797a6
* Updated from global requirementsOpenStack Proposal Bot2016-01-161-15/+15
| | | | Change-Id: If1f1c389137eb0214a584ddf5ca01e2af1da0ad1
* Merge "Fix PostgreSQL root functions"Jenkins2015-12-041-0/+1
|\
| * Fix PostgreSQL root functionsPetr Malik2015-11-191-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The default PostgreSQL administration account is 'postgres'. This account always exists and cannot be removed. Its attributes and access can however be altered. Clients can connect from the localhost or remotely via TCP/IP: Local clients (e.g. psql) can connect from a preset *system* account called 'postgres'. This system account has no password and is *locked* by default, so that it can be used by *local* users only. It should *never* be enabled (or it's password set)!!! That would just open up a new attack vector on the system account. Remote clients should use a build-in *database* account of the same name. It's password can be changed using the "ALTER USER" statement. Access to this account is disabled by Trove exposed only once the superuser access is requested. In the current implementation Trove uses the 'postgres' account and return a new superuser called 'root' when the root access is requested. The user 'root' has however no special meaning in PostgreSQL and the existing applications may rely on the default superuser name 'postgres'. This patch includes the following fixes: - Make Trove create and use its own administrative account (os_admin). - Disable (disable logins) the built-in 'postgres' account by default. - Enable and return the 'postgres' user when the root access is requested. - Fix failure on repeated 'root-enable' calls. The new implementation just regenerates the user's password. - Prevent users from hijacking the 'os_admin' account by allowing only local access. - Use the existing framework to generate secure passwords. - Stop using the system 'postgres' user to execute command-line client calls. Communicate with database via a native Python interface (like other datastores) instead. - Remove unnecessary complex result-set caching on the guest. - Simplify the status-checking code using the native exceptions. Avoid performing complex checks with uncertain benefits (i.e. attempt to guess the state of the process). Notes: The current implementation is broken for variaous reasons: - It uses UUIDs in place of 'secure' password. - It creates a 'root' user, but no database for it. The clients won't be able to authenticate without explicitly providing an existing database name. - The created 'root' user has no 'SUPERUSER' attribute and hence is not a real superuser (cannot perform certain tasks)... - The implementation suffers a defect that allows a non-root user gain root access to an instance without marking is as 'root-enabled' A similar defect exists in other datastores (MySQL) too: 1. Create an instance. 2. Enable root. 3. Use your root access to change the password of the built-in 'postgres' account (Trove will still work because it uses the 'peer' authentication method - the UNIX account). 4. Login as 'postgres' using the changed password and drop the created 'root' account. 5. Backup & restore the instance. 6. Trove reports the root has never been enabled (it checks for existence of superuser accounts other than the built-in 'postgres'). 7. You enjoy the root access of the 'postgres' user (the password is not reset on restore). Depends-On: I9d6b3df5bebc3c499cea8306f5a1a6bab18adef6 Change-Id: Ie56ac0850bf4de742ce7841195ce29ca3b30c9ef Closes-Bug: 1514569
* | Updated from global requirementsOpenStack Proposal Bot2015-11-291-2/+2
|/ | | | Change-Id: Ieb287a9cab0b17dad9762f018001c064804fc675
* Add reno for release notes managementCraig Vyvial2015-11-051-2/+1
| | | | | Depends-On: I2a445cdea20e70399e331ea4e97042ecd0137c2e Change-Id: Ia0520aff6d96ce3fcf20e94e2a100dae0f90aa76
* Removing unused dependency: discoverBertrand Lallau2015-08-281-1/+0
| | | | Change-Id: I605f6d2221081af53a3bf28e575edba90af07261
* Updated from global requirementsOpenStack Proposal Bot2015-07-231-2/+1
| | | | Change-Id: I98baed8882ae7c0c5adca20adcbc2c74b657d8fe
* Configuration Groups for RedisPetr Malik2015-07-201-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Also includes: Provide a reusable interface for the OS service 'start', 'stop', 'enable_on_boot' and 'disable_on_boot' commands using the existing service discovery functionality. Implement a Trove Redis client based on the native Python 'redis-py' https://pypi.python.org/pypi/redis API reference: https://kushal.fedorapeople.org/redis-py/html/api.html The client can connect via TCP/IP or a Unix socket (currently used for Linux OSs) and accepts an optional password retrieved from the configuration file. The driver already exists in OpenStack global requirements. It does not have to be included in the 'requirements.txt' file, but it will need to be added to 'test-requirements.txt' file to enable unit tests. The image provider will be required to install it ('redis') in the Redis images. Change-Id: Iad8f16acaf18588733162b235925d711b97a366f Implements blueprint: redis-configuration-groups
* Merge "Support authentication in the MongoDB guest agent"Jenkins2015-07-171-1/+1
|\
| * Support authentication in the MongoDB guest agentMatt Van Dijk2015-07-151-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | NOTE: The code to handle MongoDB authentication is being added, but the auth switch will NOT be turned on in this commit. Implement client authentication in the MongoDB guest agent. This involves creating an os_admin user, authenticating the client before all requests, and passing a keyfile to new cluster instances. A MongoDB client driver is being added to improve request handling. The os_admin password and the keyfile key will be randomly generated and stored on the guests. This work should be transparent to the user. The result of Trove instance and cluster creation should not be effected. New classes: * MongoDBSchema - represents Mongo databases in Trove * MongoDBUser - represents Mongo users in Trove * MongoDBAdmin - wrapper for running administrative commands * MongoDBClient - wrapper for the PyMongo client driver * MongoDBCredentials - utility class for handling username/password Test: * Added 'pymongo' to test-requirements.txt * Refactored the test_mongodb_manager unit test to make it easier to add new tests. * Cleaned up dangling mocks. * Added test for the 'secure' procedure during 'prepare'. RELATED CHANGES: Upcoming patch sets that enable Trove management of MongoDB databases and users require these changes. See the dependancy tree below. --\ authentication --\ users create/list/show/delete --- users root enable/show --- users access grant/show/revoke --- database Closes-Bug: #1467513 Partially Implements: blueprint mongodb-database Change-Id: I90793f95984b360a2c3668421cb5351aac89e46f
* | Updated from global requirementsOpenStack Proposal Bot2015-07-151-1/+2
|/ | | | Change-Id: I99aa804116a86365a06c5bf92c107ff28cc3b305
* Updated from global requirementsOpenStack Proposal Bot2015-07-081-0/+1
| | | | Change-Id: If62ea1e4ee079288374d0bdc60a32fc777c100de
* Updated from global requirementsOpenStack Proposal Bot2015-07-061-0/+1
| | | | Change-Id: Ic42cbd2b3a66c1ca8f18b12b215238b9abb92909
* Updated from global requirementsOpenStack Proposal Bot2015-06-301-6/+7
| | | | Change-Id: I297a333457514bd67e2964e955edd3cf8b9cb812
* Fix gate failure on gate-trove-pep8Peter Stachowski2015-05-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | The gate-trove-pep8 job fails on Jenkins with the following error: pkg_resources.VersionConflict: (pbr 1.0.1 (/home/jenkins/workspace/gate-trove-pep8/ .tox/pep8/lib/python2.7/site-packages), Requirement.parse('pbr<1.0,>=0.5.21')) This is caused by the fact that pbr has now released 1.0.1 and although most of the dependencies in Trove have switched to: pbr>=0.11,<2.0 we are still using the hacking module from last year (hacking>=0.8.0,<0.9), which has pbr pegged to <1.0 The hacking module needs to be updated and the new rules ignored. The following rules are now ignored: E111,E122,E123,E128,E251,E265,E713,F821,H105,H237,H238,H301,H305,H306, H307,H402,H404,H405,H407,H501,H904 Change-Id: Ief129369bbffbaec7694e3f0d36668dde1772f05 Closes-Bug: #1458985
* Updated from global requirementsOpenStack Proposal Bot2015-05-071-1/+1
| | | | Change-Id: I8ae26ba4a64d478718817aad005f690461da0f91
* Updated from global requirementsOpenStack Proposal Bot2015-04-231-1/+1
| | | | Change-Id: Id2e71464f74bf8436a905ba6e651d425ed2db577
* Updated from global requirementsOpenStack Proposal Bot2015-03-211-1/+1
| | | | Change-Id: I60b82965bd2c0d2e592833d66179f384dac44560
* Updated from global requirementsOpenStack Proposal Bot2015-03-061-1/+1
| | | | Change-Id: Ifb7328d69e187c42f249d7414713fdfb1afb3786
* Updated from global requirementsOpenStack Proposal Bot2015-01-291-1/+1
| | | | Change-Id: Ic7ea82114a28916de74adf13fe83dbbb9d2f9168
* Updated from global requirementsOpenStack Proposal Bot2014-11-201-1/+1
| | | | Change-Id: I30b63fc22d4c97af3be4ce64f24d9418a38d26ab
* Updated from global requirementsOpenStack Proposal Bot2014-11-181-1/+1
| | | | Change-Id: I800b86b2ce700752422a80ccf53d2bc6c63095ea
* Updated from global requirementsOpenStack Proposal Bot2014-11-101-1/+1
| | | | Change-Id: I00ff8b1b482abd8f94aba4a579eda532a471bba8
* Updated from global requirementsOpenStack Proposal Bot2014-10-151-2/+2
| | | | Change-Id: I7b685b79cfe0f7dcc864ec31121a696ec536edfc
* Updated from global requirementsOpenStack Proposal Bot2014-09-191-1/+4
| | | | Change-Id: I14ec93eaa97fc52b39e00175adcd718dfc4579b4
* Updated from global requirementsOpenStack Proposal Bot2014-08-301-2/+2
| | | | Change-Id: Ibac0c48326fbd51c5d6a8f409443700e3ef95a6b
* Load trove API extensions using stevedoreboden2014-08-131-1/+0
| | | | | | | | | | | | | | | | | | The current trove impl loads trove API extensions by searching on a single file path which is exposed as a conf property. This results in a less than optimal approach from a consumer extension POV. This change replaces the single extension path approach with dyanmic loading using stevedore. Consumers can now bind into the API extensions using the 'trove.api.extensions' entry point; a standard means to define extension points. Moreover this change refactors to the trove.openstack.common.extensions logic into trove.common.extensions. In addition this change includes base unit tests to ensure the existing trove proper extension points are loaded and that some basic checks are in place W/R/T validating trove extension points. Change-Id: Id3e712b536aef3bc9c9d1603367cdc0d4816b970 Implements: blueprint dynamic-extension-loading
* Updated from global requirementsOpenStack Proposal Bot2014-07-301-1/+1
| | | | Change-Id: Id3c6f344cb11cbec8860d62e69a799b008b60614
* Updated from global requirementsOpenStack Proposal Bot2014-07-031-1/+1
| | | | Change-Id: I250ee1327ba856687bef7898cd03d67ddab9a993
* Updated from global requirementsOpenStack Proposal Bot2014-06-271-3/+3
| | | | Change-Id: I9656ca53d9f638f3d3091ba60144a450279d7b81
* switch from mox to mox3Christian Berendt2014-05-301-1/+1
| | | | | | Mox3 is an unofficial port of the Google mox framework to Python 3. Closes-bug: 1283482 Change-Id: I3f558420bc720880eb91fca7b936bdcd6949a513
* Updated from global requirementsOpenStack Proposal Bot2014-05-211-1/+1
| | | | Change-Id: Iff9ed526e36185fd524683a50eb4264c362bda23
* Use six.StringIO/BytesIO instead of StringIO.StringIOHe Yongli2014-05-061-0/+2
| | | | | | | | | | | to keep Python 3.x compatibility, use six.StringIO/BytesIO to replace StringIO.StringIO StringIO works for unicode BytesIO works for bytes Change-Id: Icffbbc8b1fcbf2003f7206530c4cd13fd376a59a Closes-Bug: #1280100
* Moved the apidocs from openstack/database-apiTim Simpson2014-04-241-0/+1
| | | | | | | | | By hosting the docs here we'll be able to better maintain that developers are being mindful of changes or even occassionally writing documentation for new API features. This should also allow us to autogenerate the snippets. Change-Id: I0111bcd38341b1736a47b0a3d3e6ff96d2ca1c58
View Lorry Controller Status