summaryrefslogtreecommitdiff
path: root/heat/objects
Commit message (Collapse)AuthorAgeFilesLines
* db: Remove layer of indirectionStephen Finucane2023-03-2515-19/+15
| | | | | | | | We don't have another ORM to content with here. Simplify 'heat.db.sqlalchemy' to 'heat.db'. Signed-off-by: Stephen Finucane <stephenfin@redhat.com> Change-Id: Id1db6c0ff126859f436c6c9b1187c250f38ebb62
* Don't update status for replaced resourceramishra2021-04-011-2/+1
| | | | | | | | | | | | | | With I04e7ad90944c2d03ce0e59ba16af9d60d6e01222 we allowed update of previously-existing resource if replacement creation has failed during last update. However, we force update the status of the existing resource to 'COMPLETE'. Therefore, in the next update if properties/type has not changed for the resource it won't try to update/replace the previously existing resource resulting in false positive of stack updated successfully. Task: 42194 Change-Id: Icc90a921ec67e49aec2c4acfad72235c57c78421
* Merge "Logging Consistency"Zuul2020-05-042-2/+2
|\
| * Logging ConsistencyMarc Methot2020-04-272-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | A simple matter of having log lines fit with all the others. It will now match the ambiguous standard. Also change comments inline to alias with log fixes. Change-Id: I4a2ed6134f70c2965811f75ccb6ab8221fa0e960 Story: #2007319 Task: 38830
* | Merge "Remove six and python 2.7 full support"Zuul2020-05-023-11/+8
|\ \ | |/ |/|
| * Remove six and python 2.7 full supportHervé Beraud2020-04-233-11/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Six is in use to help us to keep support for python 2.7. Since the ussuri cycle we decide to remove the python 2.7 support so we can go ahead and also remove six usage from the python code. Review process and help ----------------------- Removing six introduce a lot of changes and an huge amount of modified files To simplify reviews we decided to split changes into several patches to avoid painful reviews and avoid mistakes. To review this patch you can use the six documentation [1] to obtain help and understand choices. Additional informations ----------------------- Changes related to 'six.b(data)' [2] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ six.b [2] encode the given datas in latin-1 in python3 so I did the same things in this patch. Latin-1 is equal to iso-8859-1 [3]. This encoding is the default encoding [4] of certain descriptive HTTP headers. I suggest to keep latin-1 for the moment and to move to another encoding in a follow-up patch if needed to move to most powerful encoding (utf8). HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5]. Note that this commit message is autogenerated and not necesserly contains changes related to 'six.b' [1] https://six.readthedocs.io/ [2] https://six.readthedocs.io/#six.b [3] https://docs.python.org/3/library/codecs.html#standard-encodings [4] https://www.w3schools.com/charsets/ref_html_8859.asp [5] https://www.w3schools.com/html/html_charset.asp Patch 14 of a serie of 28 patches Change-Id: Iccb743056c631f2e0728013942d764cd8b1ecfe0
* | Use random exponential delayPavlo Shchelokovskyy2020-01-141-1/+1
|/ | | | | | | this is much more suited for resolving contention when accessing a (lockable) shared resource. Change-Id: I93abdbf8a9532af1aec7ad15a6f384beb9fc3ff9
* Streamline conversion of resources to convergenceZane Bitter2018-12-171-0/+6
| | | | | | | | | | Use a single write to the database to convert each resource. Add a method to the versioned object class that encapsulates the DB-specific information, and get rid of the Resource.set_requires() classmethod that just calls a method on the versioned object instance that's passed to it. Change-Id: Ieca7e0f0642c38c44fb8d7729333a0ccd93c9cb4
* Set a concurrency limit on ResourceChainThomas Herve2018-10-051-0/+8
| | | | | | | | | | | | This adds a limit to the number of resources that ResourceChain creates in parallel. It reduces memory usage drastically when ResourceChain contains lots of resources. We use the number of active services, which seems to give a reasonable balance between performance and memory consumption. Story: #2003975 Task: #26917 Change-Id: Id20fb89813b32927298b3a7c0abe23da710b04d1
* Merge "Allow lazy load of raw_template but log warning"Zuul2018-07-041-3/+23
|\
| * Allow lazy load of raw_template but log warningCrag Wolfe2017-07-311-3/+23
| | | | | | | | | | | | | | | | | | To avoid hard-to-debug bugs, rather than represent a db_stack's raw_template as None if it hasn't been eagerly loaded, lazy load it. In the lazy loading case, log a warning so developers are aware there is a non-eager loading query that needs to be fixed. Change-Id: Ic06a4bd7afbd4838ca97be302d75dede5402ae6e
* | Remove stack watch servicerabi2018-01-282-147/+0
| | | | | | | | | | | | | | This removes the rpc api and related code. Change-Id: Ib89bcc3ff6a542f49467e2ad6c7e2a716a0dc2b4 Partial-Bug: #1743707
* | Merge "Remove unused variable"Zuul2017-12-131-1/+1
|\ \
| * | Remove unused variableBéla Vancsics2017-03-011-1/+1
| | | | | | | | | | | | | | | | | | TrivialFix Change-Id: I81e3a62c890f0ad043dd604443f56766ba6c11b4
* | | Speed up event listing with nested stacksZane Bitter2017-11-201-4/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When handling the command "openstack stack event list --nested-depth=n", we obtain the list of nested stacks by querying the database for all resources that share the root stack ID. However, since all we're getting is the stack IDs, there's no need to query all of the fields and construct a versioned object for each resource. Just get the set of stack IDs. Change-Id: I12155433e2ac1af919aa4b5e780fb965cd5885d8 Related-Bug: #1588561
* | | Eager-load properties for nested resource listZane Bitter2017-11-202-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When doing "openstack stack resource list --nested-depth=n", we were lazy-loading the resources' properties data. This is expensive, especially when there are a large number of resources. Eager-load the data, as we always use it to show the resources. For consistency, always eager-load the resource 'data' (even in resource_get), because the Resource versioned object accesses it unconditionally. Change-Id: Idb871fddf77bf24828878c315e19e200c28841be Related-Bug: #1665503
* | | Create replacement resource atomicallyZane Bitter2017-11-031-0/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use a single transaction to create the replacement resource and set it as the replaced_by link in the old resource. Also, ensure that no other traversal has taken a lock on the old resource before we modify it. If we end up bailing out and not creating a replacement or sending an RPC message to check it, make sure we retrigger any new traversal. Change-Id: I23db4f06a4060f3d26a78f7b26700de426f355e3 Closes-Bug: #1727128
* | | Merge "Eager load resource_properties_data in resource"Jenkins2017-10-131-8/+21
|\ \ \
| * | | Eager load resource_properties_data in resourceCrag Wolfe2017-07-311-8/+21
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Eager load resource_properties_data in resources in the typical resource-loading scenarios where properties data will be accessed. Thus, we can save an extra db query per resource when loading all the resources in a stack, for instance. Fall back to lazy loading properties data in other scenarios. Also, the resource object doesn't need to store a copy of its ResourcePropertiesData object in self.rsrc_prop_data, so don't. Change-Id: Ib7684af3fe06f818628fd21f1216de5047872948 Closes-Bug: #1665503
* | | Set resource._properties_data=None when loading from dbrabi2017-08-221-1/+1
|/ / | | | | | | | | | | | | | | | | | | | | | | In I462ce7161497306483286b78416f9037ac80d6fa we changed to use the frozen_defintion properties for delete. However, When deleting a resource from backup stack, where the resource is in INIT_COMPLETE, setting the _stored_properties_data(_properties_data) to {} when loading the resource from the db, results in error, when resources access properties in handle_delete. Change-Id: If76372c7ef9aee258efb1bfbc724d8637bc6a32c Closes-Bug: #1709682
* | Store resource attributes in the DBCrag Wolfe2017-06-212-7/+31
| | | | | | | | | | | | | | | | | | Store resource attributes that may be cached in the DB, saving the cost of re-resolving them later. This works for most resources, specifically those that do not override the get_attribute() method. Change-Id: I71f8aa431a60457326167b8c82adc03ca750eda6 Partial-Bug: #1660831
* | Merge "Load "lighter" db resources when appropriate"Jenkins2017-06-051-4/+13
|\ \
| * | Load "lighter" db resources when appropriateCrag Wolfe2017-04-191-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes we know we will only access particular fields of a resource object, rather than *all* of them. This commit allows the caller to specify (optionally) the fields that should be populated when the resource object is instantiated. This saves memory, trips to the db, and in some cases avoids extra join queries (e.g. for resource.data or resource.rsrc_prop_data). Change-Id: I405888f46451d2657aa28f610f8ca555215ff5cf Partial-Bug: #1680658
* | | Corrected max secs for concurrent trans retriesZane Bitter2017-05-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This was most likely meant as a max 2s delay here, not a max 2ms delay. Also includes a related change: when retries for metadata updates are attempted, make sure we do not have a stale value of the atomic_key (otherwise we'll just inevitably hit the ConcurrentTransaction issue). Co-Authored-By: Crag Wolfe <cwolfe@redhat.com> Partial-Bug: #1651768 Change-Id: Ie56e0e4ff93633db1f4752859d2b2a9506922911
* | | Consolidate resource locking with state changesCrag Wolfe2017-04-251-0/+11
|/ / | | | | | | | | Change-Id: I261b2f0968e16d35b7d5d791a3edb4b265a4f1d1 Closes-Bug: #1662585
* | Remove log translationsliyi2017-03-253-13/+10
| | | | | | | | | | | | | | | | | | | | | | | | Log messages are no longer being translated. This removes all use of the _LE, _LI, and _LW translation markers to simplify logging and to avoid confusion with new contributions. See: http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html Change-Id: Ieec8028305099422e1b0f8fc84bc90c9ca6c694f
* | Merge "Eager load event's resource prop. data when appropiate"Jenkins2017-03-151-3/+22
|\ \
| * | Eager load event's resource prop. data when appropiateCrag Wolfe2017-03-011-3/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Eager load an event's resource properties data in the list_events "show event" case, i.e. when an event uuid is specified. When showing multiple events, the events' resource properties data should not be eager loaded. Change-Id: I12e371dcbdae398360e28200e68f4a75831f8f17 Closes-bug: #1665506
* | | Merge "Remove unused event_get methods"Jenkins2017-03-021-10/+0
|\ \ \ | |/ /
| * | Remove unused event_get methodsCrag Wolfe2017-03-011-10/+0
| |/ | | | | | | | | | | | | Remove unused event get methods from the sqlalchemy db api and event object. Change-Id: Ide5780d456755e180572194ad4d06277749c8c79
* | Merge "Data migration enabling storage of resource attributes"Jenkins2017-03-011-0/+9
|\ \ | |/ |/|
| * Data migration enabling storage of resource attributesCrag Wolfe2017-02-161-0/+9
| | | | | | | | | | | | | | | | | | | | Working towards the goal of storing resource attributes in the db so as to avoid re-resolving them when appropriate. Adds an 'attr_data' object to the resource object, defined as a relationship on the already existing resource_properties_data table. Change-Id: I2104078d850da08b22547d7feab2bde00c543478 Partial-Bug: #1660831
* | Lazy-load resource_properties_data for eventsCrag Wolfe2017-02-062-10/+28
|/ | | | | | | Typically when retrieving a large number of events, we don't need to load resource_properties_data for each event. Change-Id: I6564fea91d19c24409075152a614cc036dc1ee34
* Add resource_properties_data assoc. to resource, event objsCrag Wolfe2016-12-222-8/+52
| | | | | | | | Add the resource_properties_data association to resource and event objects. The resource and event engine objects do not use yet it but will soon. Change-Id: Idecaafffbc5e9bfcd2355e2a165836a5ed89b16f
* Add the ResourcePropertiesData objectCrag Wolfe2016-12-221-0/+70
| | | | | | | Prepare for the future when both resources and events refer to ResourcePropertiesData rather than store the data themselves. Change-Id: I0c5aefa9d73e1281e6477d46fe181d8948aee71b
* Remove db.api wrapperThomas Herve2016-12-1316-16/+16
| | | | | | | | The db.api module provides a useless indirection to the only implementation we ever had, sqlalchemy. Let's use that directly instead of the wrapper. Change-Id: I80353cfed801b95571523515fd3228eae45c96ae
* Handle ambiguous physical resource IDsZane Bitter2016-12-051-3/+4
| | | | | | | | | It's possible that we could end up with multiple resources with the same physical resource ID, but that would be undetectable since we return only one from the database layer. This change allows us to detect the problem an return an error where the result is rendered ambiguous. Change-Id: I2c5ddbe6731c33a09ec7c4a7b91dcfe414da4385
* Refactor, add encrypt/decrypt data dict functions to cryptCrag Wolfe2016-11-281-13/+3
| | | | | | | | | | | | Just a refactor, no change in functionality. The functions added to crypt are used to encrypt / decrypt resource properties data dicts. Note that they should not be used for encrypting / decrypting other things such as params or user creds (which are just strings). An intermediate json conversion of each value in a dict takes place before it is encrypted/decrypted. Change-Id: Id6bcc90cbf430095719315ac7e9d3e8c9e745012
* Merge "Don't query raw_template in stacks uselessly"Jenkins2016-11-171-5/+9
|\
| * Don't query raw_template in stacks uselesslyThomas Herve2016-10-181-5/+9
| | | | | | | | | | | | | | | | | | | | | | When partially removing eager loading of raw_template in stacks (260b79ed28b5dc48f70fe77dfdfc074991ad1e99), we didn't get into account that accessing the sqlalchemy field would create an additional query whereas it was previously eager loaded. This removes it by only filling the field when it's already fetched. Change-Id: Ifa17c74e3559adaef56593a205101d92e9a37da5 Closes-Bug: #1634127
* | Replace retrying with tenacityBoden R2016-11-151-6/+7
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We are replacing all usages of the 'retrying' package with 'tenacity' as the author of retrying is not actively maintaining the project. Tenacity is a fork of retrying, but has improved the interface and extensibility (see [1] for more details). Our end goal here is removing the retrying package from our requirements. Tenacity provides the same functionality as retrying, but has the following major differences to account for: - Tenacity uses seconds rather than ms as retrying did. - Tenacity has different kwargs for the decorator and Retrying class itself. - Tenacity has a different approach for retrying args by using classes for its stop/wait/retry kwargs. - By default tenacity raises a RetryError if a retried callable times out; retrying raises the last exception from the callable. Tenacity provides backwards compatibility here by offering the 'reraise' kwarg. - Tenacity defines 'time.sleep' as a default value for a kwarg. That said consumers who need to mock patch time.sleep need to account for this via mocking of time.sleep before tenacity is imported. - For retries that check a result, tenacity will raise if the retried function raises, whereas retrying retried on all exceptions. This patch updates all usages of retrying with tenacity. Unit tests will be added/removed where applicable. [1] https://github.com/jd/tenacity Closes-Bug: #1635388 Change-Id: Iec0822cc0d5589b04c1764db518478d286455031
* Don't always eagerly load the raw_template for a stackZane Bitter2016-10-041-2/+3
| | | | | | | | | | | | | | | | | | | | | | | Always loading the raw template in situations where we didn't need it - e.g. in identify_stack, where we just want the name + id (given one of them), or when getting the summary stack list - uses up DB bandwidth and memory unnecessarily. This partially reverts commit 3ab0ede98c6dc0c0327977be994c139113fc0489. * The eager_load option to get_stack() is reinstated, but with the default flipped to True. In places where we explicitly do not want to load the template, we pass False. * stack_get_by_name() no longer eagerly loads the template. There were no instances of this where we subsequently use the template. * stack_get_all() acquires an eager_load option, with the default set to False. Direct users of objects.stack.Stack.get_all() will not eagerly load by default, but users of engine.stack.Stack.load_all() will get the template eagerly loaded. This practically always corresponds to what you want. Change-Id: I1f156c25ea26322be5b606a61dd77d80cadd65fc Related-Bug: #1626675
* Merge "A context cache for Resource objects"Jenkins2016-08-241-2/+24
|\
| * A context cache for Resource objectsSteve Baker2016-08-161-2/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A context cache which memoizes the resources fetched by calls to Resource.get_all_by_root_stack(..., cache=True) which are recalled by subsequent calls to Resource.get_all_by_stack. Because get_all_by_stack returns a collection instead of a single resource, there is no way of taking advantage of the SQLAlchemy identity map [1]. [1] http://docs.sqlalchemy.org/en/latest/orm/session_basics.html#is-the-session-a-cache Change-Id: Ia5aae0c86a586041020e9798566c9e0af48c180d Partial-Bug: #1578854
* | Migrate stacks from legacy to convergence engineOleksii Chuprykov2016-08-221-2/+11
| | | | | | | | | | | | | | | | | | | | | | Run `heat-manage migrate-convergence-1 <stack_id>` to migrate legacy stack to convergence engine. Heat engine is used for doing migration i.e. migration can't be done offline. Change-Id: Ie7c2498b37937438f16d154b154b3a6ecbf9ff74 Implements-bp: convergence-migrate-stack
* | Remove tenant_safe flag from rpc and db apirabi2016-08-161-2/+1
|/ | | | | | | | | | | | | | We can use admin_context to have access to stacks and software configs across projects. This removes the tenant_safe flag from rpc and db api. This is backward compatible with older rpc clients. We still support use of global_tenant flag for listing stacks and software configs. However, by default an admin(user with admin role in admin_project) would not need that. Change-Id: I12303dbf30fb80290f95baba0c67cdf684f5f409
* Always eager load the raw_template for a stackSteve Baker2016-08-152-7/+9
| | | | | | | | | | | | | The vast majority of stack fetches are immediately followed by a raw_template fetch, so this change always eagerly fetches the raw_template for every stack fetch. During stack versioned object creation the stack's raw_template object is used to construct the versioned raw template object. Change-Id: I1a6fb8fb7d069b50dd5d623c989acd5582818ae1 Related-Bug: #1479723 Related-Bug: #1523748
* Merge "Remove remaining methods from db objects"Jenkins2016-07-191-4/+2
|\
| * Remove remaining methods from db objectsSteve Baker2016-07-131-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | - HeatBase.expire, not used anywhere - HeatBase.save_and_update, moved to sqlalchemy.api - SoftDelete.soft_delete moved to sqlalchemy.api update_and_save creates a transaction, so it needs to be in sqlalchemy.api so that a context manager can eventually manage the transaction. Change-Id: I84749f4fd0781ed9a2d62327b39ce6eee0f07b35
* | Merge "Remove the db object delete method"Jenkins2016-07-191-2/+1
|\ \ | |/
View Lorry Controller Status