summaryrefslogtreecommitdiff
path: root/oslo_db
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Add JSON-encoded types for sqlalchemy"Jenkins2015-08-032-0/+148
|\
| * Add JSON-encoded types for sqlalchemyDmitry Tantsur2015-08-032-0/+148
| | | | | | | | | | | | | | | | | | | | | | This code is imported from Ironic with new tests, so that it can be reused in other projects (I need it for ironic-inspector). Some small enhancements were made to the imported code: * Non-capitalized JSON word in names * Base type can be used on its own Change-Id: Ic991f34c5b5f091d8627643367cdaa73ad2b1236
* | Merge "Python 3: Use use_unicode=1 under Python 3"Jenkins2015-07-312-7/+16
|\ \ | |/ |/|
| * Python 3: Use use_unicode=1 under Python 3fumihiko kakuma2015-07-302-7/+16
| | | | | | | | | | | | | | | | | | | | Setting use_unicode=1 allows DBAPI to handle the encoding and decoding of unicode. We can see the details from the following url. http://docs.sqlalchemy.org/en/rel_0_9/dialects/mysql.html#unicode-encoding-decoding Change-Id: I4bb67d9c8a439ee18acdcf227140dfff1a70cd93 Closes-Bug: #1479313
* | Merge "Fix test_migrations on Python 3"Jenkins2015-07-302-10/+16
|\ \ | |/ |/|
| * Fix test_migrations on Python 3Victor Stinner2015-07-282-10/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | migrate.versioning.api.VerNum() conversion to integer doesn't work in all cases. Cast explicitly to int in WalkVersionsMixin.walk_versions() to fix this issue. Modify unit tests to use VerNum objects instead of using directly integers. The bug was not seen in tests because tests used int numbers. Example of error: Traceback (most recent call last): ... File "oslo_db/sqlalchemy/test_migrations.py", line 189, in walk_versions versions = range(self.INIT_VERSION + 1, self.REPOSITORY.latest + 1) TypeError: 'VerNum' object cannot be interpreted as an integer This bug impact for example glance.tests.unit.test_migrations of the Glance project. Change-Id: I043a66268b957b97f2e8f652c0ee8aec3a00e8dd
* | Improve failure mode handling in enginefacadeMike Bayer2015-07-273-7/+90
|/ | | | | | | | | | | | | | | | | | | | Check explicitly for the cases where no "sql_connection" attribute was set when running _start(), so that the lack of this parameter is documented by the exception rather than failing into create_engine() with an unclear failure mode. If _start() fails as it will here, make sure _started stays False so that repeated calls to _start() continue to raise the same exception, rather than raising attribute errors. When accessing the "session" or "connection" attributes of the context when these attributes were not requested by the decorator or context manager, raise explicit exceptions for each, rather than returning None which leads to hard-to-debug NoneType errors. Change-Id: Iadfbf4707daed4140285a3a472009f6863b18275 Closes-bug: 1477080
* Test that concurrent sqlalchemy transactions don't blockAngus Lees2015-07-162-0/+152
| | | | | | | | | | | | | | | Neutron (and presumably other projects) expect to be able to have overlapping transactions open from two eventlets at once, without deadlock. Note that the default mysql sqlalchemy engine (mysqldb) fails this test. This change modifies py27 tox env to run the full test suite separately with and without TEST_EVENTLET environment variable set. When set, tests are run with eventlet imported and monkey_patched. Change-Id: Ib35c95defea8ace5b456af28801659f2ba67eb96 Related-Bug: #1350149
* Merge "Allow projects that use test_models_sync to filter some changes"2.1.0Jenkins2015-07-012-2/+69
|\
| * Allow projects that use test_models_sync to filter some changesDavanum Srinivas2015-07-012-2/+69
| | | | | | | | | | | | | | | | | | Adding a filter method so subclasses can choose to implement their own whitelist/blacklist of differences between their schema and models. Closes-Bug: #1468463 Change-Id: I6a5940abd100553da7c7e37efbbcf1d5eac996e5
* | Add legacy get_sessionmaker() methodMike Bayer2015-06-292-0/+19
| | | | | | | | | | | | | | | | Some projects are calling EngineFacade._session_maker for some use cases. To suit these cases more cleanly on the legacy API, add a public get_sessionmaker() method. Change-Id: Ica9574d70a6b8be80bd1a5e7cf250828455c5ade
* | Fix sqlalchemy.ModelBase.__contains__() behaviourVictor Stinner2015-06-262-1/+21
|/ | | | | | | | | | | | | | | | | | | | | | | | | | Currently, sqlalchemy.ModelBase.__contains__() catches any exception, and by doing that, it hides real bugs. For example, a Nova unit test raises the following error, but __contains__() simply returns False: sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <InstanceExtra at 0x7fea635bc5f8> is not bound to a Session; deferred load operation of attribute 'pci_requests' cannot proceed On Python 3, hasattr() calls getattr(): it returns True if getattr() succeeds, False if getattr() raises an AttributeError, or passes through the exception, if getattr() failed for a different reason. On Python 2, hasattr() also calls getattr(), but it catches *any* exception. This change replaces hasattr() with getattr(), and it only catches AttributeError as Python 3, so passes through sqlalchemy exceptions. Add an unit test to test the new behaviour. Closes-Bug: #1469225 Change-Id: If9c3ccc03f1dc9746936b0b83ea132508491e577
* Merge "Allow additional exceptions in wrap_db_retry"Jenkins2015-06-262-18/+46
|\
| * Allow additional exceptions in wrap_db_retryKevin Benton2015-06-252-18/+46
| | | | | | | | | | | | | | | | | | | | | | | | There are cases where we want to catch and retry some additional exceptions and treat them as RetryRequests depending on method that is being excecuted (e.g. StaleDataError or ObjectDeletedError). This patch changes the wrap_db_retry decorator to take an extra parameter that is a list of additional exception classes that will be caught and treated as a RetryRequest. Change-Id: I6c61b1d2f0331cac31dfde0afa523e44d470b83f
* | Remove implicit RequestContext decorationCedric Brandily2015-06-252-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change removes RequestContext implicit decoration as some applications have their own session management for the moment or don't need this feature. Moreover this implicit decoration doesn't help debugging. This change doesn't disallow RequestContext decoration but applications should require it explicitly: from oslo_db.sqlalchemy import enginefacade enginefacade.transaction_context_provider(oslo_context.RequestContext) Closes-Bug: #1468707 Change-Id: I143f30c41e788c7aa9887c0e994f49ee55c94651
* | Add a new ModelBase.items() methodVictor Stinner2015-06-242-3/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change is required to port applications using oslo.db to Python 3. The iteritems() method of Python 2 dictionaries was removed in Python 3 (renamed to items()). Currently, it's not possible to replace obj.iteritems() with six.iteritems(obj) or obj.items() for oslo.db objects. Nova has a function calling obj.iteritems(), the function gets oslo.db objects and dictionaries. On Python 3, dictionaries have no iteritems() method, whereas oslo.db objects have no items() method. This change allows to write obj.items() which works on Python 2 and Python 3, on dictionaries and oslo.db objects. Change-Id: Ibd4f454183c2bd3465539c3e572fa23208975bdb
* | Remove oslo namespace packageDoug Hellmann2015-06-2119-4395/+6
|/ | | | | | | | | | | | Remove the oslo namespace package shims, and the tests for importing using those names. Blueprint remove-namespace-packages Depends-on: Id25875f3ef51c2b84cf2b9362039a5196feaa531 for openstack/tuskar Change-Id: If3036e03a68c2e865b5f0495c74fe8303de93d21
* Drop use of 'oslo' namespace packageDoug Hellmann2015-06-186-8/+8
| | | | | | | | | | | | | | | | | | | | | The Oslo libraries have moved all of their code out of the 'oslo' namespace package into per-library packages. The namespace package was retained during kilo for backwards compatibility, but will be removed by the liberty-2 milestone. This change removes the use of the namespace package, replacing it with the new package names. The patches in the libraries will be put on hold until application patches have landed, or L2, whichever comes first. At that point, new versions of the libraries without namespace packages will be released as a major version update. Please merge this patch, or an equivalent, before L2 to avoid problems with those library releases. Blueprint: remove-namespace-packages https://blueprints.launchpad.net/oslo-incubator/+spec/remove-namespace-packages Change-Id: Iaf9247f90ba26615d02e5431d6bc98f22351caa7
* Switch from MySQL-python to PyMySQLJeremy Stanley2015-06-186-65/+148
| | | | | | | | | | | | | As discussed in the Liberty Design Summit "Moving apps to Python 3" cross-project workshop, the way forward in the near future is to switch to the pure-python PyMySQL library as a default. Added a special test environment to keep MySQL-python support. Documentation modified. https://etherpad.openstack.org/p/liberty-cross-project-python3 Change-Id: I12b32dc097a121bd43991bc38dd4d289b65e86c1
* Merge "Implement new oslo.db.sqlalchemy.enginefacade module"Jenkins2015-06-1411-823/+3318
|\
| * Implement new oslo.db.sqlalchemy.enginefacade moduleMike Bayer2015-06-0411-823/+3318
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This module presents a replacement for the EngineFacade system. At the center is the oslo.db.sqlalchemy.enginefacade module, which when imported, provides decorators and context managers which perform all database and ORM connectivity functions transparently. The docstrings as well as the blueprint provide an introduction. The patch includes a refactoring of sqlalchemy/session.py into three dependent modules engines.py, orm.py and enginefacade.py. This is to maintain a non-cyclical import structure as well as to maintain the import behavior of oslo.db overall, as some projects such as glance currently have dependencies on this structure. There is also a slimming down and attempt at modernizing some very old documentation in session.py. The enginefacade system should be preferred moving forward. Implements: blueprint make-enginefacade-a-facade Change-Id: I9a3d0c26bb727eb2c0bd823b9a12fde57cc7c9c3
* | Replace utils method with oslo.utils reflection provided one1.11.0Joshua Harlow2015-06-062-27/+3
| | | | | | | | Change-Id: Ife14a6acd01a16c4b1092b43fb94942db0e03036
* | Allow to fail instead of skip in DbFixtureCedric Brandily2015-06-041-3/+11
|/ | | | | | | | | | | | | | | | | Tests using DbFixture[1] are currently skipped if the database is unavailable (no db service, no permissions, etc.) which is fine when tests are runned by developers but not when runned by the gate jobs. This change defines the attribute DbFixture.SKIP_ON_UNAVAILABLE_DB: when the database is unvailable, tests are: * skipped when SKIP_ON_UNAVAILABLE_DB=True (default, legacy behaviour), * failed when SKIP_ON_UNAVAILABLE_DB=False. [1] oslo.db.sqlalchemy.test_base Change-Id: I067f46417fefe252c650e1e5e590e83547b11b6a Closes-Bug: #1404093
* Merge "Ensure that create_engine() close test connection"Jenkins2015-05-221-1/+5
|\
| * Ensure that create_engine() close test connectionVictor Sergeyev2015-01-281-1/+5
| | | | | | | | | | | | | | | | | | | | handle_connect_error() open and return a test connection, but nobody uses it in create_engine(), so (depending on Python interpreter) this connection can stay opened in the pool for some time. Co-Authored-By: Roman Podoliaka <rpodolyaka@mirantis.com> Change-Id: Ibef07f0fa1fec61aa10963c07c410a731d7758ef
* | Merge "Remove pre-SQLAlchemy-0.9.7 compat utilities"Jenkins2015-05-208-322/+21
|\ \
| * | Remove pre-SQLAlchemy-0.9.7 compat utilitiesMike Bayer2015-04-288-322/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now that requirements.txt is at SQLAlchemy 0.9.7 at the minimum, we can remove the engine_connect compatibility module as well as part of the handle_error compatibility module. In particular, this gives us the ability to add and remove engine_connect events using the event API directly which can be handy in test fixtures. Change-Id: I48450ad9d472d4377913ad391a0f5e3ba0f1471f
* | | Add a keys() method to SQLAlchemy ModelBaseVictor Stinner2015-05-192-3/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With this additional method, it now possible to write directly dict(obj), instead of dict(obj.iteritems()), to cast an object to a dictionary. Modify also ModelIterator: it doesn't inherit from ModelBase anymore. Change-Id: I702be362a58155a28482e733e60539d36c039509
* | | Sort model fields using getattr(), not inspect()Mike Bayer2015-05-052-1/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While inspect(mapping).all_orm_descriptors is a good way to get at attributes that are known to be mapped or otherwise described by SQLAlchemy, it does not produce a SQL expression in the case of a hybrid descriptor, where it produces the descriptor, not the result of calling it. This fix ensures that the descriptor is evaulated, while still retaining the fact that we check the attribute as coming from an ORM-level descriptor. Change-Id: I04a03be713348158b8cdfda683624fea01fe34ea Closes-Bug: #1451880
* | | Merge "Update to latest hacking"Jenkins2015-04-302-10/+10
|\ \ \ | |/ / |/| |
| * | Update to latest hackingDoug Hellmann2015-04-032-10/+10
| | | | | | | | | | | | | | | | | | | | | Update to the version of hacking used in kilo and fix class declarations that cause the new version to report errors. Change-Id: I69c4976a2d611a19675cd5919d498abda7856a88
* | | Merge "Sanity check after migration"Jenkins2015-04-103-5/+11
|\ \ \
| * | | Sanity check after migrationMarco Fargetta2015-04-103-5/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added a sanity_check call after the migration, so if a table in the migration scripts is created not correctly the exception arise before to go in production. Closes-Bug: #1428065 Change-Id: Ib91f2471fa389a6072e8c2aa302504a6c068aa0a
* | | | Merge "exc_filters: support for ForeignKey error on delete"Jenkins2015-04-102-5/+80
|\ \ \ \ | |/ / / |/| | |
| * | | exc_filters: support for ForeignKey error on deleteJulien Danjou2015-04-072-5/+80
| |/ / | | | | | | | | | | | | | | | | | | | | | The current code only supports this type of constraint on INSERT, this adds support for DELETE. Change-Id: I981eb5f7b71bd37a35a93bbd3fada01d74aba12d Closes-Bug: #1439358
* | | Merge "Handle CHECK constraint integrity in PostgreSQL"Jenkins2015-04-103-0/+50
|\ \ \
| * | | Handle CHECK constraint integrity in PostgreSQLJulien Danjou2015-04-033-0/+50
| |/ / | | | | | | | | | | | | | | | | | | | | | PostgreSQL offers CHECK constraints integrity which are currently raised as generic error. This patch add a filter so they can have their own exception type. Change-Id: I7e36ef4385b227d1e62e18277d470047a369a238
* | | Merge "Add filters for DBDataError exception"Jenkins2015-04-103-0/+69
|\ \ \
| * | | Add filters for DBDataError exceptionRoman Podoliaka2015-04-093-0/+69
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We currently don't distinguish DB errors, that are due to problems with the processed data (like division by zero, numeric value out of range, incorrect data type passed, etc) This actually led to a problem when MySQL threw two different errors depending on the randomly generated UUID value and we wrapped only one of those cases properly, which caused the test case to fail time to time in the gate. Closes-Bug: #1442178 Change-Id: I02891f3b1ede6f33834e5637be43496499cc88e7
* | | Merge "Catch DBDuplicateError in MySQL if primary key is binary"Jenkins2015-04-092-1/+23
|\ \ \ | |/ / |/| |
| * | Catch DBDuplicateError in MySQL if primary key is binaryJulien Danjou2015-03-262-1/+23
| | | | | | | | | | | | | | | | | | | | | | | | In Gnocchi we use binary primary keys, and one a duplicate entry is inserted the error is not catched. This patches fixes that. Change-Id: I488628df1a3b91d842eb9fa76bc1878f12d37acd Closes-Bug: #1368297
* | | Merge "Fixed bug in InsertFromSelect columns order"Jenkins2015-04-012-13/+105
|\ \ \
| * | | Fixed bug in InsertFromSelect columns orderTimofey Durakov2015-03-132-13/+105
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previous realization generates query string like: `INSERT INTO tbl1 SELECT tbl2.ID, tbl2.name, tbl2.currdate FROM tbl2;`. It doesn't specify columns in INSERT clause. If tbl1 columns differs from tbl2, execution of insert fails, or silently insert data in incorrect columns. To fix this implementation of class changed to default SQLAlchemy insert from select. added extra parameter 'columns', which allows to determine column order in INSERT clause. Closes-Bug:#1430884 Change-Id: Ica5cdea10d9aa253f395ec553559dba54e062028
* | | | Merge "Avoid excessing logging of RetryRequest exception"Jenkins2015-03-301-1/+5
|\ \ \ \ | |_|/ / |/| | |
| * | | Avoid excessing logging of RetryRequest exceptionEugene Nikanorov2015-03-171-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Such exception indicates of error condition only if retries are exceeded. In other cases error-level logs pollute the log file and complicate log analysis. Change-Id: I318867043a68a6499851631593859a0fffac6038 Closes-Bug: #1433001
* | | | Merge "Implement generic update-on-match feature"Jenkins2015-03-234-0/+987
|\ \ \ \
| * | | | Implement generic update-on-match featureMike Bayer2015-03-054-0/+987
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This feature provides the query.update_on_match() and query.update_returning_pk() methods, as well as the manufacture_persistent_object(), manufacture_entity_criteria(), and manufacture_criteria() utility functions. query.update_on_match() is used to UPDATE a row based on a variety of criteria, and to then return a fully persistent object state representing the row that was matched. It essentially intends to provide an UPDATE that is guaranteed to have matched a specific row in the presence of potential race conditions without using any locking, and to then return a record of that row as if it had been SELECTed. query.update_returning_pk() is a public method that also serves as part of the implementation of query.update_on_match(); this method delivers an UPDATE statement such that the primary key of the single row matched is returned; if zero or multiple rows are matched, and error is raised. To handle this, several backend-specific strategies are provided, which are automatically selected based on the best available. The lowest strategy performs a re-SELECT, but still assumes there's a simple unique column to be queried on, as is currently the use case in Nova (uuid is present). On Postgresql, MySQL and other databases besides SQLite and possibly DB2, more atomic strategies are used. Change-Id: I059f4ae6e72cfa6681a179314144214639f283ef
* | | | | Provide working SQLA_VERSION attributeMike Bayer2015-03-183-8/+10
| |/ / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repairs the parsing within test_utils tests to use the alphabetic-supporting version parser inside of compat/utils. Necessary in order to work with beta / dev versions of SQLAlchemy such as the current 1.0.0b1 release. Change-Id: Ic1ae9fa18171b687b1bb9cfac7411fac14e29a0f
* | | | Merge "Add process guards + invalidate to the connection pool"Jenkins2015-03-152-0/+61
|\ \ \ \
| * | | | Add process guards + invalidate to the connection poolMike Bayer2015-03-122-0/+61
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It's not safe for a database TCP connection to be shared to a child process, as this is a file descriptor which will maintain its state on both sides. Applications such as Cinder which spin up multiprocessing subprocesses at startup time are subject to race conditions as a result. Instead of requiring that engines be explicitly prepared within a child process, we can detect and accommodate this situation in the connection pool itself, by tracking the originating pid of a connection, and if it changes on checkout, by invalidating. Change-Id: If116f7b7140b3eba064d8147e5f637a05beb1cd8
View Lorry Controller Status