summaryrefslogtreecommitdiff
path: root/taskflow/persistence/backends/sqlalchemy
Commit message (Collapse)AuthorAgeFilesLines
* Fix atomdetails failure column sizePavlo Shchelokovskyy2022-01-271-0/+36
| | | | | | | | | failure and revert_failure fields in atomdetails is defined as a JSON type, but its data type is 'text' in mysql, which is limited to 64kbytes. JSON data type should have the same size as a LONGTEXT. Closes-Bug: #1959243 Change-Id: I65b6a6d896d3e8aad871dc19b0f8d0eddf48bdd6
* Merge "Use custom JSONType columns"4.6.3Zuul2021-10-261-8/+27
|\
| * Use custom JSONType columnsPavlo Shchelokovskyy2021-07-131-8/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the JSONType from sqlalchemy_utils is quite brittle as it only does primitive json.dumps on values. This leads to various sorts of StorageFailure exceptions in taskflow when, for example, an unserializable exception bubbles up to the 'failure' field of AtomDetails. This patch sublclasses the JSONType from sqlalchemy_utils and overrides two of its methods that do (de)serialization to work via oslo.serialization functions. They deal with such occurencies much better, for example, by providing 'str' as a fallback default. Change-Id: I3b9e9498b155199a4e707006a0cf22cda0567c06 Related-Bug: #1935957
* | Fix flowdetails meta sizeGregory Thiemonge2021-05-031-0/+34
|/ | | | | | | | | | meta field in flowdetails is defined as a JSON data type, but its data type is 'text' in mysql, which is limited to 64kbytes. JSON data type should have the same size as a LONGTEXT. Closes-Bug: #1926304 Change-Id: I9f89badfc697f0f26245ca7f4c22d62e220be5f9
* Fix deprecated Alembic function args4.6.0Adam Harwell2021-01-281-9/+9
| | | | | | | | create_index and create_foreign_key were using long-deprecated argument names which were finally removed as of 1.5.0: https://alembic.sqlalchemy.org/en/latest/changelog.html#change-da0bea3199b905783ddbd0cde968beec Change-Id: I03405e3d0daf085d0849a6e3effc65ec05ceb7a1
* Stop to use the __future__ module.Hervé Beraud2020-06-021-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | The __future__ module [1] was used in this context to ensure compatibility between python 2 and python 3. We previously dropped the support of python 2.7 [2] and now we only support python 3 so we don't need to continue to use this module and the imports listed below. Imports commonly used and their related PEPs: - `division` is related to PEP 238 [3] - `print_function` is related to PEP 3105 [4] - `unicode_literals` is related to PEP 3112 [5] - `with_statement` is related to PEP 343 [6] - `absolute_import` is related to PEP 328 [7] [1] https://docs.python.org/3/library/__future__.html [2] https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html [3] https://www.python.org/dev/peps/pep-0238 [4] https://www.python.org/dev/peps/pep-3105 [5] https://www.python.org/dev/peps/pep-3112 [6] https://www.python.org/dev/peps/pep-0343 [7] https://www.python.org/dev/peps/pep-0328 Change-Id: Ie7c18f2f1b6294e3d19c36112d35f4472ce77e7b
* Use mysql LONGTEXT for atomdetails resultstrain-em3.7.1Ann Taraday2019-08-091-0/+43
| | | | | | | | | | | | | | | With a lot of possible retries using mysql sa.Text for result field atomdetails table is not enough. Updated 'results' column type to LONGTEXT in mysql backend case. PosgreSQL seems not to be affected as postgres text type is not having length limitation. Co-Authored-By: Michael Johnson <johnsomor@gmail.com> Change-Id: I4e44e3288ae49d85c5b5ce8ff26ba83f3b9d2c7b Closes-bug: #1838015
* python3.0 has deprecated LOG.warnxhzhf2017-03-071-2/+2
| | | | | | | | python3.0 has deprecated LOG.warn https://docs.python.org/3/library/logging.html#logging.warning Closes-Bug: #1650843 Change-Id: Ib8473cbdb84547f385e619fe08ed723b1ee7bcdb
* Merge "Use alembic upgrade function/command directly"Jenkins2016-01-062-26/+15
|\
| * Use alembic upgrade function/command directlyJoshua Harlow2015-06-152-26/+15
| | | | | | | | | | | | | | Instead of implementing a similar version to what the upgrade function does, just use it directly instead. Change-Id: I61a3c9f09c6e0724f2b55951989171ef4aaafe0c
* | Fix currently broken and inactive mysql testsJoshua Harlow2015-12-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit 868dd8485456ce78a396b687ec7744dc365f06be added a migration script that caused there to be two heads (starting points) of migrations which would cause alembic to be unable to do the initial schema creation (and upgrade). So this fixes that by correctly making one head only and not skipping tests for a database if it can not be upgraded for whatever reason (which was silencing the test failures). Closes-bug: #1528683 Change-Id: Id571072eec1dc3b6cbb9e868854f6db0b271e5f8
* | Use the sqlalchemy-utils json type instead of our ownJoshua Harlow2015-09-092-23/+66
| | | | | | | | Change-Id: Ie01ea85e74f1daed6bfa5158c0faa476d06873ba
* | Retain atom 'revert' result (or failure)Joshua Harlow2015-07-102-0/+44
| | | | | | | | | | | | | | | | | | When a atom is reverted it can be useful to retain the result of that 'revert' method being called, so that it can be later analyzed (or used for various purposes) so adjust the storage, and actions to enable it to be stored. Change-Id: I38a9a5f3bf7550e924468bb4a86652cb8beb306c
* | Rename logbook module -> models moduleJoshua Harlow2015-07-082-4/+4
|/ | | | | | | | | | | Since this module contains more than the logbook class and really is a our generic models that are used to hold the runtime structure it is more appropriate to place it under a models module and deprecate the usage of the old module by placing a warning there (so that when it is imported that warning is triggered). Change-Id: I79def5ee08f560d38f2c9dcefd0b33becc2a4d36
* Fix updated_at column of sqlalchemy tablesFredrik Bergroth2015-05-211-3/+3
| | | | | | | | The column updated_at does not update its value after the refactoring made in 687ec913790653f79badc8f5d656c86792e94271. Closes-Bug: #1457309 Change-Id: Iff45f386b5dc8efc3fe82ca3b1e961a0c23d7ac7
* Switch to non-namespaced module importsDavanum Srinivas2015-03-121-3/+3
| | | | Change-Id: I25686df6a05bf69a89b9b6be169db72eab89d32a
* Use the new table length constantsJoshua Harlow2015-02-121-14/+24
| | | | | | | | | | Instead of having constants for lengths that aren't shared among the table master schema and the upgrade scripts move the constants that exist in the schema upgrade scripts (which are used to arrive at the master schema) to use the same constants the master table schema module uses. Change-Id: I312760ee329ebd3161c09b316d55afe129ff2d09
* Rework the sqlalchemy backendJoshua Harlow2015-02-102-97/+99
| | | | | | | | | | | | | | | We can just simplify the usage of an sqlalchemy if we just use (we already have our own ORM like objects anyway) sqlalchemy core in the first place and have a very tiny layer that converts back and forth from our very limited object model that we use in our persistence layer. This change makes that adjustment, which makes it easier to read and understand the actions the sqlalchemy backend is doing when saving, reading and updating data, and avoids yet another layer that isn't useful for our purposes anyway. Change-Id: I911c509f65e7845aee86fed1622eaa56970741f2
* Switch to using 'oslo_serialization' vs 'oslo.serialization'Joshua Harlow2015-01-151-1/+1
| | | | | | | | Prefer the non-deprecated 'oslo_serialization' instead of the namespaced 'oslo.serialization' wherever it was previously used. Change-Id: I652cf0b56e28d727c59fe0c060949bb2bd920d11
* Switch to using 'oslo_utils' vs 'oslo.utils'Joshua Harlow2015-01-141-2/+2
| | | | | | | | Prefer the non-deprecated 'oslo_utils' instead of the namespaced 'oslo.utils' wherever it was previously used. Change-Id: I9a78150ef5266e1ff22147278162fe3cfe1b2e3f
* Move over to using oslo.utils [reflection, uuidutils]Joshua Harlow2014-12-181-1/+1
| | | | | | | | | | | | | | The reflection module is now part of oslo.utils so we should remove our local version and use that version instead; this also goes for the uuidutils module which is now part of oslo.utils as well so we no longer need our local version copied from the incubator... Note that one reflection method `find_subclasses` which was to specific to taskflow is now moved to the misc utility module instead of its prior home in the reflection module. Change-Id: I069881c80b0b2916cc0c414992b80171f7eeb79f
* Switch to using oslo.utils and oslo.serializationJoshua Harlow2014-09-201-2/+2
| | | | | | | | | | | | Instead of copying modules from the incubator into taskflow we can now directly use these same modules from supported libraries instead so this moves the usage of everything except uuidutils which wasn't moved over to using those newly published libraries. Part of blueprint integrate-and-use-oslo-utils-serialization Change-Id: I1183bda96e1ddb062d9cab91990186f0f56f0a0e
* Enabled hacking checks H305 and H307Christian Berendt2014-07-173-1/+2
| | | | | | | * H305 imports not grouped correctly * H307 like imports should be grouped together Change-Id: If1dd9c89f65ede6959865a885777cb08c263eca0
* Persistence cleanup part oneJoshua Harlow2014-03-263-30/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | - Convert the various functions that take a task detail into ones that take atom details (since this is now the generic type they should take). - Don't expose the detail type strings as part of the atom detail api, leave those as private hidden strings and provide conversion functions from string<->class instead. - Have the logbook objects contain the following new methods to reduce the dependence on persistence_utils to do the same. - to_dict() which converts the current object into a dict - from_dict() which converts the provided dict into a object - merge() which merges a incoming objects data with the current objects - Have the persistence backends + storage + action engine use these new methods instead of there current usage. - Don't compare to logbook.RETRY_DETAIL or logbook.TASK_DETAIL since python has the isinstance function just use it (ideally we should fix the code so that this isn't even needed, usage of isinstance means something is not designed/structured right). - In storage tests we can't assume that failures will be non-lossy since under certain backends when a failure is stored information about the internally held exc_info is lost, so take this into account when testing by using matches() where applicable. Change-Id: Ie8a274cfd4cb4e64e87c355dc99d466d74a4e82c
* Merge "Removed copyright from empty files"Jenkins2014-03-231-16/+0
|\
| * Removed copyright from empty filesyangxurong2014-02-221-16/+0
| | | | | | | | | | | | | | | | | | According to policy change in HACKING: http://docs.openstack.org/developer/hacking/#openstack-licensing empty files should no longer contain copyright notices. Change-Id: I1f825add9036623ef626b48c66206ac2b452c592 Partial-Bug: #1262424
* | Remove extraneous vim configuration commentsIvan A. Melnikov2014-03-192-4/+0
| | | | | | | | Change-Id: Ib0f065b02e053ca6dd176eb8d6a75cf71e0982d0
* | Add atom intentions for tasks and retriesAnastasia Karpinska2014-03-182-0/+41
| | | | | | | | | | | | | | | | | | | | | | | | Add atom intentions: REVERT, EXECUTE, RETRY and IGNORE. Intentions will be used by action engine to schedule tasks correctly. Add intention to task detail and extend storage to work with atom intentions. Add alembic migration to add intentions column to database. Change-Id: I79c9bb5f11861658dbfedfd64ef93eb92b29cb2d
* | Extend logbook and storage to work with retryAnastasia Karpinska2014-03-142-1/+50
|/ | | | | | | | - Store AtomType in TaskDetail (TASK_DETAIL or RETRY_DETAIL) - Extend storage to work with retries - Add alembic migration for database Change-Id: I07896d4b79ebe5de33f65e00a3bd32af543a09b4
* Remove extraneous vim configuration commentsyangxurong2014-02-146-12/+0
| | | | | | | | | Remove line containing comment - # vim: tabstop=4 shiftwidth=4 softtabstop=4 Change-Id: I7581cc88b8de433d5609ed06c6570b0b45c13573 Closes-Bug:#1229324
* Doc strings and comments clean-upskudriashev2014-01-261-2/+2
| | | | | | | | * Added missing period for doc strings * Correct syntax errors * Remove H402 from flake8 ignore list Change-Id: Ia8592bf99378e3658d6cca2ceb148bf9eb0b5de8
* Fix sqlalchemy 0.8 issuesOlga Kopylova2013-12-171-2/+2
| | | | | | | | | | | Use import of exc instead exception. Mutability tracking is changed in SQLAlchemy 0.8, but it seems that sqlalchmy.types.MutableType is not needed in classes Json and Failure from models. Closes-bug: 1206669 Change-Id: Ib13c2575ead495b22ed5624e68e044e2f4927201
* Bump requirements to the latestJoshua Harlow2013-10-053-0/+54
| | | | | | | | | | | Match the requirements for taskflow with the openstack requirements (note the upstream requirements are missing 'futures' and 'networkx') and update the new hacking violations that were detected due to the hacking requirement version bump. Change-Id: I8d1326cf2a8b1ea062f5e9aacd0c4f8261c6531a
* Wrapping and serializing failuresIvan A. Melnikov2013-10-042-2/+53
| | | | | | | | | | | | * save task failures to sqlalchemy backend and restore them from there; * for any wrapped exception use common WrappedFailre exception type, which makes it easier to handle particular exception types; * several minor improvements. Affects-db: this change adds db migration that looses exception information for saved task details. Change-Id: I575282002e6999646bbf51f492b82a7e3525787a
* Remove usage of oslo.db and oslo.configJoshua Harlow2013-09-121-4/+9
| | | | | | | | | | | Remove further reference to oslo-incubator database layer which itself uses oslo.config and remove its dependencies and resync with oslo-incubator after applying these changes. Implements blueprint eliminate-oslo-cfg Change-Id: Ie1ee7affae75c60833c012c921e50d7b14026101
* Move toward using a backend+connection modelJoshua Harlow2013-09-123-263/+17
| | | | | | | | | | | | | | | | | | Instead of having a pretty restrictive module based api for saving logbook objects it is much more friendly and extensible to move toward a more ceilometer-influenced engine and connection based storage backend using stevedore to do the backend loading instead of a custom registration/fetching mechanism. This allows us to provide a base object oriented backend api that can be easily inherited from to allow for customized & pluggable backend storage modules. Implements blueprint stevedore-based-backends Implements blueprint ceilometer-influenced-backends Change-Id: Ib5868d3d9018b7aa1a3354858dcb90ca1a04055d
* Rework the persistence layerJoshua Harlow2013-08-2411-710/+513
| | | | | | | | | | | | | | | | 1. Simplify the exposed api to reduce race conditions which could occur if we allowed flow details, task details and log books to be deleted at different times; reduce this down to just being able to save and delete from logbooks (and only save/update for flow and task details to) to reduce the problem cases. 2. Introduce a alembic migration with a proper schema so that the initial database can be created in the first place, adjust its exposed fields and relations to be defined by said schema. 3. Use oslo db models instead of our own. Change-Id: I78bbedf87d506d3f39157198638937c933235b7b
* File movementsJoshua Harlow2013-08-175-0/+826
In order to rework some of the persistence layer I would like to move around some of the files first, keeping job specifics in a jobs folder. Having some of the items which are root level taskflow items (flow, task) be at the root of the hiearchy. Also for now until the celery work is commited move that, since it doesn't make sense in backends anyway. Change-Id: If6c149710b40f70d4ec69ee8e523defe8f5e766d
View Lorry Controller Status