summaryrefslogtreecommitdiff
path: root/heat/engine/resources/stack_resource.py
Commit message (Collapse)AuthorAgeFilesLines
* Explicitly pass error kwarg in nested StackValidationFailedTobias Urdin2022-12-191-1/+1
| | | | | | | | | | This fixes the issue when a nested stack is serialized in oslo.messaging and deserialization fails because we pass a kwarg as an arg when we generate the exception remotely. Story: #2010115 Task: #45695 Change-Id: Id75398d2ed2d4fab467df51057c4167cd77bb32f
* Merge "Workaround client race in legacy nested stack delete"Zuul2021-03-111-0/+30
|\
| * Workaround client race in legacy nested stack deleteZane Bitter2019-12-231-0/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we delete a legacy stack, the delete call may return when a previous delete is still IN_PROGRESS and we are waiting for it to be cancelled - for up to 250s by default plus the time it takes to kill any remaining threads. This means that if the client subsequently sees a DELETE_FAILED status, it can be ambiguous as to whether it refers to the previous delete or the one it just commanded. Work around this when Heat itself is the client (i.e. we are deleting a nested stack). First, put the time the delete started into the status_reason message when the stack goes into the DELETE_IN_PROGRESS state, so that we can distinguish. Then, require the stack to report its status as DELETE_FAILED twice in a row, 10s apart, before we mark it as failed - unless we have definitively seen a different delete commence in the meantime. Change-Id: Ie23ba5f4538a8f5c2e7d64caa6cbe530c013b520 Story: #1669608 Task: 17214
* | Merge "Remove handling for client status races"Zuul2020-11-161-24/+0
|\ \
| * | Remove handling for client status racesZane Bitter2019-12-231-24/+0
| |/ | | | | | | | | | | | | | | | | | | | | Now that we guarantee that resources are UPDATE_IN_PROGRESS before the stack update call returns for legacy stacks (this was already true for convergence stacks), there is no need to have special handling to check the updated_time either in StackResources or in functional tests. Change-Id: I5bf7ed6cb9ba6c77a77dd36eb173e1927065c53e Story: #1669608 Task: 23176
* | Remove six and python 2.7 full supportHervé Beraud2020-04-231-2/+1
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Six is in use to help us to keep support for python 2.7. Since the ussuri cycle we decide to remove the python 2.7 support so we can go ahead and also remove six usage from the python code. Review process and help ----------------------- Removing six introduce a lot of changes and an huge amount of modified files To simplify reviews we decided to split changes into several patches to avoid painful reviews and avoid mistakes. To review this patch you can use the six documentation [1] to obtain help and understand choices. Additional informations ----------------------- Changes related to 'six.b(data)' [2] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ six.b [2] encode the given datas in latin-1 in python3 so I did the same things in this patch. Latin-1 is equal to iso-8859-1 [3]. This encoding is the default encoding [4] of certain descriptive HTTP headers. I suggest to keep latin-1 for the moment and to move to another encoding in a follow-up patch if needed to move to most powerful encoding (utf8). HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5]. Note that this commit message is autogenerated and not necesserly contains changes related to 'six.b' [1] https://six.readthedocs.io/ [2] https://six.readthedocs.io/#six.b [3] https://docs.python.org/3/library/codecs.html#standard-encodings [4] https://www.w3schools.com/charsets/ref_html_8859.asp [5] https://www.w3schools.com/html/html_charset.asp Patch 12 of a serie of 28 patches Change-Id: I2a60193d27a3496e96d383c1897b8665ad481efb
* Pre-empt in-progress nested stack updates on new updateZane Bitter2019-10-291-1/+8
| | | | | | | | | | | | | | If the parent resource of a nested stack is locked due to an IN_PROGRESS update, cancel the nested stack update (which will result in the parent resource being marked FAILED and releasing the lock so that the new traversal can begin acting on it). This also cancels all descendants of the nested stack. This means that a concurrent update no longer gets blocked at a nested stack boundary until the previous update has finished. Change-Id: I5f14453ebab75d89672c6eea12de46d48a5147f3 Task: 17760
* Revert "Use OutputDefinition to generate attributes schema"Zane Bitter2018-11-281-2/+2
| | | | | | | | | This reverts commit 85eff5fc3d03dce7d0d17d965f27246bd3306c53. Change-Id: I325593df0cd7a48cbc85edf7838cddbc0911b4b6 Closes-Bug: #1805589 Depends-On: https://review.openstack.org/620457 Depends-On: https://review.openstack.org/619786
* Use OutputDefinition to generate attributes schemaZane Bitter2018-10-181-2/+2
| | | | | | | We have the OutputDefinition API now, and no longer need to treat templates like CFN JSON blobs to obtain information about outputs. Change-Id: I8dfea1e9855a56fb85d2e3af40996d5f337d7859
* Ignore spurious nested stack locks in convergenceZane Bitter2018-08-301-4/+8
| | | | | | | | | | | | | | | | | | | | | | | | Several operations (e.g. stack check) are yet to be converted to convergence-style workflows, and still create locks. While we try to always remove dead locks, it's possible that if one of these gets left behind we won't notice, since convergence doesn't actually use stack locks for most regular operations. When doing _check_status_complete() on a nested stack resource, we wait for the operation to release the nested stack's lock before reporting the resource completed, to ensure that for legacy operations the nested stack is ready to perform another action on as soon as the resource is complete. For convergence stacks this is an unnecessary DB call, and it can lead to resources never completing if a stray lock happens to be left in the database. Only check the nested stack's stack lock for operations where we are not taking a resource lock. This corresponds exactly to legacy-style operations. Change-Id: I4eb20ad82cc3e9434da34500fafa3880567d0959 Story: #1727142 Task: 24939
* Keep old files in file map for rolling updaterabi2018-05-111-1/+6
| | | | | | | | | | | We would need old resource definitions when doing rolling update of certain group resources like RG/ASG. Therefore update the file map with the files from old template. Change-Id: I8f880e5b23c25159ecab1c63b594329d8df33973 Closes-Bug: #1765454 Task: #17360 Story: #1765454
* Return nested parameters for resource group.Thomas Herve2018-02-261-0/+18
| | | | | | | | | | | This refactors the building of schema from parameter validation to use a new method (which doesn't keep stacks in memory), and use that new method for providing proper schema for resource group when the size is 0. Change-Id: Id3020e8f3fd94e2cef413d5eb9de9d1cd16ddeaa Closes-Bug: #1751074 Closes-Bug: #1626025
* Avoid always loading nested stack on updateZane Bitter2018-01-081-4/+8
| | | | | | | | | | | | | | | | | | Previously, when calling StackResource._validate_nested_resources() (which we do whenever we create or update a nested stack), we would load the nested stack into memory to validate the number of resources in the nested stack, unless the max_resources_per_stack config option was set to -1. This meant we would load the nested stack into memory in the same engine as the parent on every update. To reduce the memory high-water mark, fetch the information we need over RPC from another engine instead. To ensure this is only called once, move the call into the validate code. (Previously it was called again in the create/update itself.) Change-Id: I78d12ecc8240c697e26893ae2d7172b60883fb93 Partial-Bug: #1731349
* Use appropriate exception in StackResource.get_output()Zane Bitter2017-12-191-7/+8
| | | | | | | | | | | Don't raise InvalidTemplateAttribute in StackResource.get_output() when an output does not exist - it's not the case that get_output() is only used for fetching attributes. Instead, raise NotFound from get_output(), and translate that to InvalidTemplateAttribute in the caller when we are actually fetching an attribute. Change-Id: I4f883e4b583a965785d0a595c8c33b47dc94118c Related-Bug: #1719333
* Avoid RPC call in TemplateResource.get_reference_id()Zane Bitter2017-11-271-3/+15
| | | | | | | | | | | | | Most TemplateResources probably don't have an OS::stack_id output defined in the template, so it's unfortunate that we have to make an RPC call to check that every time we retrieve the resource's reference_id, which we have to do for most stack operations. To cut down on unnecessary RPC calls, check the template first locally to see if there is an OS::stack_id output present. Change-Id: Ia32ed6bca453b391371f544ad0a07d49dc0616e3 Related-Bug: #1719333
* Add converge flag in stack update for observing on realityricolin2017-08-071-1/+2
| | | | | | | | | | | Add converge parameter for stack update API and RPC call, that allow triggering observe on reality. This will be triggered by API call with converge argument (with True or False value) within. This flag also works for resources within nested stack. Implements bp get-reality-for-resources Change-Id: I151b575b714dcc9a5971a1573c126152ecd7ea93
* Improve StackValidationFailed exceptionrabi2017-06-291-3/+13
| | | | | | | | | We use StackValidationFailed in many different scenarios and the the message is at times extremely unhelpful, specifically when the validation error is deep in a nested stack. Change-Id: I0183bdf81442e62325a427b4eec5c4cd9b7cb91f Closes-Bug: #1686360
* Remove deprecated functionshuangtianhua2017-05-271-7/+0
| | | | Change-Id: Ic41eb9782a3bb4bd716f6f06dccc3428c4a94306
* Merge "Validate property values in nested stacks again"Jenkins2017-04-051-1/+5
|\
| * Validate property values in nested stacks againZane Bitter2017-03-311-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In ced6f78aa065c1a7e6400c3be9ec3322e1e87416 we stopped doing validations of nested stacks at stack creation time, on the assumption that they had been validated when the parent stack was created. This assumption was incorrect; for children of the stack being created, the strict_validate global is always set during validation, so property values of resources will not be validated until it comes time to create the resource. Instead, prevent only redundant non-strict validations of stacks at nested depth 2 and greater. This means that every stack other than the root will be validated exactly twice - once without validating property values when the root is created, and again including property validation when the nested stack itself is created. Most of the performance benefits should remain; in the case of a large ResourceGroup using index substitution, we will now have to validate a lot of nearly-identical resource properties, however we still will not load into memory and validate a nested stack for each one as we originally did. Since that happens synchronously, it was likely the main contributor to RPC timeouts when dealing with large scaling groups. (During the validation at the creation of the root stack, only a single member of a ResourceGroup is validated even when index substitution is used. For scaling groups with identical members, only one member is validated since 3aebdabf2e78ac9e920b9dd8c748c4fad0d723c3.) This change reverts commit ced6f78aa065c1a7e6400c3be9ec3322e1e87416. Change-Id: I97cf789cee75931edef58b78c88f02da204d2a08 Closes-Bug: #1675589 Related-Bug: #1645336
* | Merge "Only recreate CHECK FAILED resources in ResourceGroup"Jenkins2017-04-041-5/+15
|\ \ | |/ |/|
| * Only recreate CHECK FAILED resources in ResourceGroupJason Dunsmore2017-04-041-5/+15
| | | | | | | | | | | | | | | | | | | | Previously, ResourceGroup._needs_update() would raise UpdateReplace when the ResourceGroup was in CHECK_FAILED status, resulting in delete/replacement of every resource in the group. Now, it will return True so that only resources in the CHECK_FAILED status. Change-Id: I800c4f58feec7c1aaa4897c2ba056e5a74200e5d Closes-Bug: #1671592
* | Use exception_filter in RPC clientZane Bitter2017-03-271-3/+1
| | | | | | | | | | | | | | This provides both better stack traces for exceptions that are not ignored, and nicer syntax. Change-Id: I3467c17272e4a6f81ae8293f723f95dfad3180ab
* | Remove log translationsliyi2017-03-251-4/+2
|/ | | | | | | | | | | | Log messages are no longer being translated. This removes all use of the _LE, _LI, and _LW translation markers to simplify logging and to avoid confusion with new contributions. See: http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html Change-Id: Ieec8028305099422e1b0f8fc84bc90c9ca6c694f
* Pass on outputs errors to parent stacksZane Bitter2017-02-201-7/+15
| | | | | | | | | If getting an output from a child stack fails with an error, we didn't pass on the error message to the parent stack that was requesting it but instead reported essentially that the given output did not exist. Change-Id: I5653baf310a29dc4829ad570c769cf67ce12695e Partial-Bug: #1599114
* Just to migrate existing resource to backup stackhuangtianhua2017-02-101-1/+1
| | | | | | | | | | | | | | Don't create a resource record for backup stack if the resource record exists, just to migrate it to backup stack, to avoid redundant data remaining for existing stack. This patch also adds resource.store() which covers _store() and _store_or_update() implemention. And then we can delete the two methods. Change-Id: I0b4b983306ea84fab0e2c81876ef407a80d25989 Closes-Bug: #1662095
* Merge "Don't delete RawTemplate if it referenced by a stack"Jenkins2016-11-151-10/+8
|\
| * Don't delete RawTemplate if it referenced by a stackZane Bitter2016-09-211-10/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When creating a nested stack and pre-storing the raw template, we previously used a try/finally block to ensure that if the stack creation was never started (e.g. because the template failed validation at the other end) that we would delete the raw template again rather than leave it lying around in the DB. However, this catches too broad a range of exceptions. If an exception such as GreenletExit occurs inside this block, then we will attempt to delete the raw template, resulting in a DB IntegrityError either here (because the template id is already referenced by a stack) or when trying to store the stack (because the template is gone). Resolve this by only doing the cleanup on HeatExceptions, which indicates that the other engine has bailed out without storing the stack. Change-Id: Ie24a82b7ce16281c72d4eb66cca8683248628b46 Closes-Bug: #1626256
* | Do not throw an exception if stack outputs is not setAbhishek Chanda2016-11-111-1/+1
| | | | | | | | | | | | | | | | | | In some rare cases, stack output is not set and this line throws an exception. This causes stack deletion to fail. Change-Id: I7710f160ee881e355e3a3541fe8729b06ff29b38 Closes-Bug: #1638741 Co-Authored-By: Crag Wolfe <cwolfe@redhat.com>
* | Use RPC to retrieve nested stack outputThomas Herve2016-10-111-5/+15
| | | | | | | | | | | | | | | | Instead of loading the stack in memory, use RPC to get the stack and its output. It releases memory pressure from the main engine in legacy mode. Change-Id: Id3da88e8c5d9b6d564b1b71960d9937867543d79 Related-Bug: #1626675
* | Avoid loading nested stacks in memory where possibleZane Bitter2016-10-111-32/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Prior to changing StackResource to do stack operations over RPC, we made liberal use of the StackResource.nested() method to access the nested stack that was likely always loaded in memory. Now that that is no longer required, it adds additional memory overhead that we need not have. We can now obtain the stack identifier without loading the stack, and that is sufficient for performing operations over RPC. The exceptions are prepare_abandon(), which cannot be done over RPC at present, and get_output(), which may be addressed in a separate patch. The gratuitous loading of the nested stack in TemplateResource.get_attribute() is eliminated, so although it still ends up loading the nested stack in many cases, it will no longer do so once get_output() stops doing it. Change-Id: I669d2a077381d7e4e913f6ad1a86fb3f094da6c5 Co-Authored-By: Thomas Herve <therve@redhat.com> Related-Bug: #1626675
* | Break cycle between Resource and AttributesThomas Herve2016-09-271-1/+2
| | | | | | | | | | | | | | Attributes hold a reference to Resource via the attribute resolution method. Let's add a weakref here to break the cycle. Change-Id: I1cd22476c7d2d9ca0a090947b8f28d8afc413fc7
* | Don't use cast() to do StackResource deleteZane Bitter2016-09-211-2/+3
|/ | | | | | | | | | | | | | If an exception was raised in delete_stack when deleting a nested stack, the parent stack would never hear about it because we were accidentally using cast() instead of call() to do the stack delete. This meant the parent resource would remain DELETE_IN_PROGRESS until timeout when the nested stack had already failed and raised an exception. In the case of bug 1499669, the exception being missed was StopActionFailed. Change-Id: I039eb8f6c6a262653c1e9edc8173e5680d81e31b Partial-Bug: #1499669
* Handle outputs with an OutputDefinition classZane Bitter2016-09-091-6/+4
| | | | | | | | | | | | This is the last remaining area of the code where we were slinging around CloudFormation-formatted snippets of template. Replace it with a proper API along the lines of ResourceDefinition. The default implementation of the new template.Template.outputs() method is equivalent to what we have previously done spread throughout the code, so that third-party Template plugins will not be affected until they have had time to write their own custom implementations. Change-Id: Ib65dad6db55ae5dafab473bebba67e841ca9a984
* Begin move of UpdateReplace back to its rightful locationZane Bitter2016-08-181-3/+3
| | | | | | | | | | | | | | | | | | | A deeply misguided effort to move all exceptions out of the heat.engine.resource module, where they belong, and into the heat.common.exception module, where they largely do not, broke the API for third-party resource plugins. Unfortunately this happened a couple of releases back already, so we can't simply put UpdateReplace back where it belongs as that would also break the de-facto third-party API. This change adds an alias in the correct location and a comment indicating that it will move back at some time in the future. It also switches all of the in-tree uses back to heat.engine.resource.UpdateReplace, in the hope that third-party developers will be more inclined to copy from that. This reverts commit 4e2cfb991ada853023cd0994db2bf9a39126c538. Change-Id: Iedd5d07d6c0c07e39e51a0fb810665b3e9c61f87 Closes-Bug: #1611104
* Catch NotSupported when cancelling a nested stackZane Bitter2016-08-091-8/+15
| | | | | | | | | | | If a nested stack is in a state where we can't cancel it - generally because it has actually already finished before we attempt to stop it - then a NotSupported exception is raised. Catch and ignore this exception so that we can continue (by issuing an update to the previous template, in the case of a rollback). Change-Id: Ied4356193e383881d9da33429809339a4f41ac0b Related-Bug: #1591337
* Cancel nested stack creation when sibling failsZane Bitter2016-07-281-0/+3
| | | | | | | | | | | | | When a resource fails and a sibling resource in the same stack is a nested stack and is being created, we should (gracefully) cancel the creation of the nested stack. This allows the user to correct the issue and try to recover the stack with a stack update within a reasonable period of time. Otherwise, they have to wait for the entire nested stack to complete, fail or time out before they will be able to successfully do an update to correct the problem. Change-Id: I02c3894718d595282f57ca63dc2c2ba9b7baa440 Closes-Bug: #1591341
* Use handle_update_cancel() to cancel nested stack updatesZane Bitter2016-07-281-14/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | The code designed to deal with the case where an in-progress StackResource update is cancelled by stopping the update of the nested stack has never worked because it was erroneously trying to catch StopIteration exceptions instead of the actual exception raised when a generator is closed, GeneratorExit. This change moves the handling code to the new handle_update_cancel() mechanism instead. Using this mechanism also means that StackResources will be cancelled immediately, rather than waiting for up to error_wait_time before cancelling the nested stack update. The previous patch adds a grace period for IN_PROGRESS resources within the nested stack to complete, so we want to notify it straight away to begin that period (and to not start any new work). Since the grace period on the child stack means that the child is likely to still be IN_PROGRESS at the time we want to roll back, handle rollback by sending an update-cancel with rollback message to the child. Only if the child is no longer IN_PROGRESS will we start an update with the previous template. If it is still IN_PROGRESS then just wait for it to reach ROLLBACK_COMPLETE. Change-Id: Ide210f695446fe8de2057b3b74c276165a6f9f3f Closes-Bug: #1591337 Closes-Bug: #1446252
* Use exception_filter from oslo_utilsZane Bitter2016-07-191-3/+1
| | | | | | | | | The ExceptionFilter class was moved to oslo_utils as oslo_utils.excutils.exception_filter by Ib986ccaf95a19ef14fef1cebe39fc87c17c2769f and is available in oslo_utils 3.9.0. We already require oslo_utils>=3.15.0 in requirements.txt. Change-Id: I53c21ff077531881a3971e7b0d3a792b8a9c760f
* Merge "Add context to stack lock function arguments"Jenkins2016-06-301-1/+2
|\
| * Add context to stack lock function argumentsSteve Baker2016-06-221-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | Context is added as the first argument to stack_lock_* db functions but is currently ignored for getting the session used for stack lock operations. This is required later for bug 1479723 but is added in its own change here to ease reviewing load. Change-Id: Ieb3e4e2ee67150777cbe1e961d0d1806cf1f7e46 Related-Bug: #1479723
* | Fix the definition of has_nested()Zane Bitter2016-06-021-4/+4
|/ | | | | | | | | | | | | | | | | Since Iab7a997c0b4d9dcb8ceb3d2f49692216fe611df1 StackResource.has_nested() returns True unconditionally. However, some parts of the code expect that if has_nested() returns True, the caller will be able to call nested() and not get None returned (this was the original definition of has_nested()). This change restores the definition without the need to load the nested stack (which is expensive, and the reason for changing this in the first place). This also allows the change from the original patch to be a bit tidier, without a recurrence of the issue it initially caused (bug 1586906) that was later fixed by I2ededcc57057f43206922b99bb72c43332336814. Change-Id: Id26dc62b7513853b3f02a656518c59b3ff8d509a Related-Bug: #1578854
* Fix nested identifier when not createdThomas Herve2016-05-301-0/+2
| | | | | | | | With Iab7a997c0b4d9dcb8ceb3d2f49692216fe611df1 we started returning invalid identifier on nested stack when the creation is in progress, as resource_id is not set. In this case we should not return an identifier. Change-Id: I2ededcc57057f43206922b99bb72c43332336814
* Don't load nested stack to build the identifierSteve Baker2016-05-251-20/+12
| | | | | | | | | StackResource is already building HeatIdentifier for the nested stack, so this should be used by the resource formatter to avoid the overhead of loading the nested stack. Change-Id: Iab7a997c0b4d9dcb8ceb3d2f49692216fe611df1 Partial-Bug: #1578854
* Avoid passing templates/files over RPCZane Bitter2016-05-161-9/+33
| | | | | | | | | | | Pre-store the template for nested stacks in the database and use the new template_id parameter in the the RPC API to pass the ID over RPC, instead of passing a copy of the whole template and all the files. This will help us to safely de-duplicate templates in the database, reduce the size of messages, and prevent unnecessary copies of the files (in particular) being loaded in memory. Change-Id: I1305572cd70845a28df60756440dd417733b0da1
* Don't use two different names for the same flagZane Bitter2016-05-011-3/+2
| | | | | | | | | The nested_abandon_in_progress flag in the StackResource class is used in exactly the same way and does exactly the same thing as the abandon_in_progress flag does in every other Resource class, except that it does the Wrong Thing when adopting a stack fails. Change-Id: I47e412169f58734b8684a645470180d95e13def6
* Make RPC call to abandon nested stackDrago Rosson2016-04-211-1/+6
| | | | | | | | | | | | Due to stacks being processable by multiple engines, resources in a nested stack cannot be marked for retainment from the parent resource of the nested stack. This patch makes the parent resource of the nested stack call stack abandon instead of stack delete through the RPC client so that nested stacks are deleted while nested stack resources are retained. Change-Id: I7103f46d86db35e08b014d9790215740fcae441d Closes-bug: #1509912
* Revert "Enable abandon option to nested resource"Drago Rosson2016-04-211-1/+0
| | | | | | | | | This reverts commit 103a09bc1c3e9f2e4894bba19963ca91b6652155. Reverting this patch because it caused the nested stacks to hang around, when they should really be deleted too during an abandon. Change-Id: I7865dd97b43aec7f17734b8311b78facb9d75ccc
* Use a context manager to re-raise local exceptions in StackResourceZane Bitter2016-03-291-10/+14
| | | | | | | Avoid changing the traceback for local exceptions when translating remote exceptions in StackResources. Change-Id: I1ce76dc1b82c37e2cc95518e964e9f0c181f9375
* Merge "Use oslo.utils.reflection to extract class name"Jenkins2016-03-061-3/+5
|\
View Lorry Controller Status